Week 5 Assignment

Week5ArticlebyPPN.pdf

Home >Psychology homework help >Week 5 Assignment

For More Information Visit RAND at www.rand.org

Explore the RAND Corporation

View document details

Support RAND Browse Reports & Bookstore

Make a charitable contribution

Limited Electronic Distribution Rights This document and trademark(s) contained herein are protected by law as indicated in a notice appearing later in this work. This electronic representation of RAND intellectual property is provided for non- commercial use only. Unauthorized posting of RAND electronic documents to a non-RAND website is prohibited. RAND electronic documents are protected under copyright law. Permission is required from RAND to reproduce, or reuse in another form, any of our research documents for commercial use. For information on reprint and linking permissions, please see RAND Permissions.

Skip all front matter: Jump to Page 16

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis.

This electronic document was made available from www.rand.org as a public service of the RAND Corporation.

CHILDREN AND FAMILIES

EDUCATION AND THE ARTS

ENERGY AND ENVIRONMENT

HEALTH AND HEALTH CARE

INFRASTRUCTURE AND TRANSPORTATION

INTERNATIONAL AFFAIRS

LAW AND BUSINESS

NATIONAL SECURITY

POPULATION AND AGING

PUBLIC SAFETY

SCIENCE AND TECHNOLOGY

TERRORISM AND HOMELAND SECURITY

http://www.rand.org/pdfrd/about.html

http://www.rand.org/pdfrd/

http://www.rand.org/pdfrd/about.html

http://www.rand.org/pdfrd/pubs/tools/TL145.html

http://www.rand.org/pdfrd/pubs/online.html

http://www.rand.org/pdfrd/giving/contribute.html

http://www.rand.org/pdfrd/publications/permissions.html

http://www.rand.org/pdfrd/

http://www.rand.org/pdfrd/topics/children-and-families.html

http://www.rand.org/pdfrd/topics/education-and-the-arts.html

http://www.rand.org/pdfrd/topics/energy-and-environment.html

http://www.rand.org/pdfrd/topics/health-and-health-care.html

http://www.rand.org/pdfrd/topics/infrastructure-and-transportation.html

http://www.rand.org/pdfrd/topics/international-affairs.html

http://www.rand.org/pdfrd/topics/law-and-business.html

http://www.rand.org/pdfrd/topics/national-security.html

http://www.rand.org/pdfrd/topics/population-and-aging.html

http://www.rand.org/pdfrd/topics/public-safety.html

http://www.rand.org/pdfrd/topics/science-and-technology.html

http://www.rand.org/pdfrd/topics/terrorism-and-homeland-security.html

This report is part of the RAND Corporation tool series. RAND tools may include models, databases, calculators, computer code, GIS mapping tools, practitioner guide- lines, web applications, and various toolkits. All RAND tools undergo rigorous peer review to ensure both high data standards and appropriate methodology in keeping with RAND’s commitment to quality and objectivity.

C O R P O R A T I O N

RAND Tool

Programs That Work, from the Promising Practices Network on Children, Families and Communities

M. Rebecca Kilburn, editor

Jill S. Cannon, Teryn Mattox, Rebecca Shaw, co-editors

Limited Print and Electronic Distribution Rights

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions.html.

The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest.

RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors.

Support RAND Make a tax-deductible charitable contribution at

www.rand.org/giving/contribute

www.rand.org

For more information on this publication, visit www.rand.org/t/tl145

Published by the RAND Corporation, Santa Monica, Calif.

R® is a registered trademark.

http://www.rand.org/t/tl145

http://www.rand.org/pubs/permissions.html

http://www.rand.org/giving/contribute

http://www.rand.org

Preface

- iii -

http://www.promisingpractices.net/

mailto:Darleen_Opfer@rand.org

Table of Contents

Preface ....................................................................................................... iii

List of Tables .............................................................................................. vi

Acknowledgements ................................................................................... vii

Introduction ................................................................................................ 1

Types of Programs ...................................................................................... 3

Programs That Work ................................................................................. 26

The Abecedarian Project ...................................................................................... 26 Accelerated Academics Academy (AAA) ................................................................. 36 Accelerated Reader............................................................................................. 40 Adolescent Coping with Depression Course ............................................................ 43 Athletes Training and Learning to Avoid Steroids (ATLAS) ....................................... 52 Attachment-Based Family Therapy ....................................................................... 56 Be Proud! Be Responsible! ................................................................................... 61 Big Brothers Big Sisters of America ...................................................................... 68 Child Development Project .................................................................................. 75 Child Sexual Abuse Prevention: Teacher Training Workshop .................................... 81 Child-Parent Centers ........................................................................................... 85 Class Wide Peer Tutoring Program ........................................................................ 93 Cognitive Behavioral Intervention for Trauma in Schools (CBITS) ............................. 97 Cognitive Relaxation Coping Skills ...................................................................... 102 Cognitively Guided Instruction (CGI) .................................................................. 106 Communities In Schools .................................................................................... 110 Cooperative Integrated Reading and Composition................................................. 115 Coping Cat ...................................................................................................... 121 Coping with Stress Course ................................................................................. 128 Core Knowledge ............................................................................................... 132 Counselors Care (CARE) .................................................................................... 137 Creating Lasting Family Connections ................................................................... 142 DARE to be You ................................................................................................ 148 Direct Instruction ............................................................................................. 152 Early Childhood Education and Assistance Program (ECEAP) .................................. 161 Early Head Start .............................................................................................. 165 Early Intervention in Reading ............................................................................. 172 The Effective Learning Program .......................................................................... 175 Family Foundations .......................................................................................... 179 Family Support and Parenting Education in the Home ........................................... 186 Family Thriving Program ................................................................................... 189 Father/Male Involvement Preschool Teacher Education Program ............................. 193 FluText ........................................................................................................... 196 Gang Resistance Education and Training (G.R.E.A.T.) ........................................... 200 Get Real About AIDS ........................................................................................ 205 Go Grrrls ......................................................................................................... 208 Guiding Good Choices ....................................................................................... 212

- iv -

Head Start ...................................................................................................... 217 Healthy Families New York (HFNY) ..................................................................... 224 Healthy Start ................................................................................................... 232 Healthy Steps for Young Children ....................................................................... 236 HighScope Perry Preschool Program ................................................................... 242 Hip-Hop to Health Jr. ........................................................................................ 250 Incredible Years ............................................................................................... 252 Infant Health and Development Program ............................................................. 277 Know Your Body ............................................................................................... 286 LifeSkills Training ............................................................................................. 290 Making Proud Choices! ...................................................................................... 303 Midwestern Prevention Project/Project STAR ........................................................ 307 Multisystemic Therapy (MST) ............................................................................. 314 National Guard Youth ChalleNGe Program ........................................................... 321 New Hope Project ............................................................................................. 327 New York City's Small Schools of Choice ............................................................. 336 Newborn Individualized Developmental Care and Assessment Program (NIDCAP) ..... 340 Nurse Family Partnership .................................................................................. 346 Parent-Child Home Program .............................................................................. 354 Parents as Teachers ......................................................................................... 358 Parents' Fair Share ........................................................................................... 368 Partners in Reading .......................................................................................... 373 Peer-Assisted Learning Strategies (PALS) ............................................................ 377 Postponing Sexual Involvement/Human Sexuality Educational Series ...................... 382 Primary and Secondary Control Enhancement Training.......................................... 387 Project ALERT .................................................................................................. 391 Project Northland ............................................................................................. 398 Project TRUST .................................................................................................. 402 Quantum Opportunity Program (QOP) ................................................................. 405 Reaching Educators, Children, and Parents (RECAP) ............................................. 410 Reading Recovery ............................................................................................ 414 Reciprocal Teaching .......................................................................................... 420 Reducing the Risk ............................................................................................ 427 Resolving Conflict Creatively Program (RCCP) ...................................................... 432 SafeCare ......................................................................................................... 439 Safe Child Program ........................................................................................... 442 Seattle Social Development Project .................................................................... 446 Second Step Violence Prevention ........................................................................ 457 Self Center (School-Linked Reproductive Health Services) ..................................... 464 Smart Start ..................................................................................................... 469 Social Decision Making/Problem Solving .............................................................. 473 SPORT ............................................................................................................ 478 Student Achievement Guarantee in Education (SAGE) ........................................... 483 Syracuse Family Development Research Program ................................................. 489 Talent Development Secondary .......................................................................... 495 Targeted Reading Intervention (TRI) .................................................................. 501 Team Accelerated Instruction: Math ................................................................... 504 Teen Talk ........................................................................................................ 509 Text4Health-Adolescents ................................................................................... 513 Triple P Positive Parenting Program .................................................................... 516 Triple Play ....................................................................................................... 521 Who Do You Tell? ............................................................................................. 525 Wyman's Teen Outreach Program....................................................................... 529

- v -

List of Tables

Table 1. Primary PPN Evidence Criteria .................................................... 2

Table 2. PPN Programs, by Age of Child ................................................... 6

Table 3. PPN Programs, by Type of Setting ............................................ 10

Table 4. PPN Programs, by Type of Service ............................................ 15

Table 5. PPN Programs, by Type of Outcome Addressed .......................... 20

- vi -

Acknowledgements

Dozens of RAND researchers and other scholars contributed to the Promising Practices Network (PPN) by either producing or reviewing content. Rebecca Kilburn served as editor from 2000 to 2014, and three individuals served as co-editors at various points: Jill S. Cannon, Teryn Mattox, and Rebecca Shaw. Sue Phillips served as the Web Manager from 2000 to 2014, and Patrice Lester, Carolyn Rogers, Ruth Eagle-Winsick, and Lance Tan were project administrators. Editors for the project have included James Torr, Christina Pitcher, and Nancy DelFavero. Individuals who identified, screened, assessed, and summarized program evaluations for PPN include: Jill Cannon, Amy Coombe, Laura Hickman, Rebecca Kilburn, Shannon Maloney, Teryn Mattox, Rebecca Shaw, Lilah Shapiro, Amber Smith, and Jennifer Wong. The following individuals have served as peer reviewers for PPN programs: Megan Beckett, Mark Berends, James Bigelow, Barbara Burns, Anita Chandra, Peggy Chen, Rebecca Collins, Janet Currie, Ashlesha Datar, Lois Davis, Patricia Ebener, Daniela Golinelli, Laura Hamilton, Gina Ikemoto, Lisa Jaycox, Lynn Karoly, Kelly Kelleher, Vi-Nhuan Le, John Love, Jens Ludwig, Steve Martino, Dan McCaffrey, Jennifer McCombs, Christopher Nelson, Michael Pollard, Rajeev Ramchand, Terry Schell, Dana Schultz, Heather Schwartz, Claude Setodji, Bradley Stein, and Mary Kay Stein. We thank Christopher Dirks for helping us produce this document. RAND and the Network member organizations are grateful to the many organizations that contributed funding to PPN over its history, including The Annie E. Casey Foundation; The California Wellness Foundation; The Colorado Trust; the Community Foundation of North Louisiana; The David and Lucile Packard Foundation; Doris Duke Charitable Foundation; Family Communications, Inc.; Family and Community Trust (Missouri); Foundation Consortium for California’s Children & Youth; Georgia Family Connection Partnership; Grantmakers for Chidlren, Youth and Families; Hands On Mississippi; Kansas Action for Children; KidsOhio.org; New York State Office of Children and Family Services; Northwest Early Childhood Institute; Oregon Commission on Children and Families; Parents Action for Children; RAND Corporation; and The Spencer Foundation. We also express our gratitude to these individuals who served on the PPN Board of Advisors and helped to support the project financially: Doug Brengel, Lloyd Morrisett, and Paul O’Neill.

- vii -

Introduction

Between 1998 and 2014, the Promising Practices Network (PPN) on Children, Families and Communities (www.promisingpractices.net) provided information on programs and practices that credible research indicated are effective in improving outcomes for children, youth, and families. The information on this website pertained to children from the prenatal period to age 18, as well as the families and communities in which they live. This site provided useful information to decisionmakers, practitioners, and program funders who had to choose among many possibilities for improving results for children, youth, and families. The website content included summaries of evidence-based programs, issue briefs, and other products that helped decisionmakers access high-quality research relevant to child and family policy. PPN was founded in 1998 by four state-level intermediary organizations: the Colorado Foundation for Families and Children, Georgia Family Connection Partnership (formerly Georgia Academy), Missouri’s Family and Community Trust, and the Foundation Consortium for California’s Children & Youth. These organizations started PPN because they recognized the value of providing better access to evidence-based information on improving outcomes for children and families, and they saw an opportunity to realize efficiencies by pooling resources to undertake this common activity. The RAND Corporation began managing PPN in 2000 and operated the project until 2014, when the project concluded due to a lack of funding. At the time the project concluded, the site was archived at www.promisingpractices.net. This document contains the summaries of the Programs That Work section of the PPN website, as of June 2014. PPN staff reviewed hundreds of programs’ evaluations and assessed whether the evidence of effectiveness met the preestablished criteria. Programs with evidence meeting the criteria were summarized in a brief description in this section of the PPN website. We reproduce the summaries here to serve as a permanent archive for policymakers, researchers, and other stakeholders. Note that for quality assurance, all summaries were peer reviewed by other researchers and reviewed by program contacts to ensure accuracy. In the next section of this document, we list all programs by categories, such as age of the child when the intervention takes place, delivery setting, and outcomes improved. Programs listed on the PPN site were generally assigned either a Proven or a Promising rating, depending on whether they met the project evidence criteria. In some cases, a program received a Proven rating for one indicator and a Promising rating for a different indicator. In these cases, the evidence

- 1 -

http://www.promisingpractices.net

level assigned will be Proven/Promising, and the program summary will specify how the evidence levels were assigned by indicator. Reviewers were trained to review programs using the PPN program review Reference Guide, which provided detailed instructions for reviewers on the three-phase program review process using 19 quality criteria for evaluation. The criteria include the primary criteria listed in Table 1, as well as secondary criteria such as attrition, quality of outcome measures, and independence of the evaluator.

Table 1. Primary PPN Evidence Criteria

Type of Information

Proven Program Promising Program Not Listed on Site

Program must meet all of these criteria to be listed as Proven.

Program must meet all of these criteria to be listed as Promising.

If a program meets any of these conditions, it will not be listed on the site.

Type of Outcomes Affected

Program must directly impact one of the indicators used on the site.

Program may impact an intermediary outcome for which there is evidence that it is associated with one of the PPN indicators.

Program impacts an outcome that is not related to children or their families, or for which there is little or no evidence that it is related to a PPN indicator (such as the number of applications for teaching positions).

Substantial Effect Size

At least one outcome is changed by 20%, 0.25 standard deviations, or more.

Change in outcome is more than 1%.

No outcome is changed more than 1%.

Statistical Significance

At least one outcome with a substantial effect size is statistically significant at the 5% level.

Outcome change is significant at the 10% level (marginally significant).

No outcome change is significant at less than the 10% level.

Comparison Groups

Study design uses a convincing comparison group to identify program impacts, including randomized- control trial (experimental design) or some quasi- experimental designs.

Study has a comparison group, but it may exhibit some weaknesses (for example, the groups lack comparability on preexisting variables, or the analysis does not employ appropriate statistical controls).

Study does not use a convincing comparison group (for example, the use of before and after comparisons for the treatment group only).

Sample Size Sample size of evaluation exceeds 30 in both the treatment and comparison groups.

Sample size of evaluation exceeds 10 in both the treatment and comparison groups.

Sample size of evaluation includes fewer than 10 in the treatment or comparison group.

Availability of Program Evaluation Documentation

Publicly available. Publicly available. Distribution is restricted (for example, only to the sponsor of the evaluation).

NOTE: Additional considerations play a role on a case-by-case basis. These may include attrition, quality of outcome measures, and others.

- 2 -

Types of Programs PPN programs are organized by the following topics and may be related to more than one topic. Tables 2 through 5 list the programs by topic. Clicking on the links in the tables will take you to the program summary.

Topic Definitions

Below are definitions of the topics that were used to organize the PPN Programs That Work.

Age of Child

Early Childhood (0–8) The program targets children below age 9 (before birth through grade 3).

Middle Childhood (9–12) The program targets children ages 9 through 12 (grades 4 through 6).

Adolescence (13–18) The program targets children ages 13 through 18 (grades 7 through 12).

Type of Setting

Child Care/Preschool The program, or some portion of the program, is delivered in a preschool setting or provides care for children before they enter kindergarten.

Elementary School The program, or some portion of the program, is delivered in an elementary school (grades K–5 or K–6) setting.

Middle School The program, or some portion of the program, is delivered in a middle school (grades 6–8 or 7–9, also called junior high) setting.

High School The program, or some portion of the program, is delivered in a high school (grades 9–12 or 10–12) setting.

Out-of-School Time The program includes some component explicitly delivered after school hours, on evenings, on weekends, or during the summer during the traditional adult workday. (This does not include childcare programs.) Also known as after-school care.

Community-Based Service Provider The program includes some component that can be delivered by a community- based service provider, including but not limited to faith-based organizations and nonprofit service providers.

- 3 -

Health Care Provider The program includes some component in which services are or have the potential to be delivered by a health care provider or in a health care setting.

Home Visiting The program includes a home visitation component, through which services are delivered within the family's home.

Type of Service

Case Management The program includes some component of case management, in which children or youth are assigned to a case manager who assesses their service needs and those of their families and coordinates any care or services required.

Family Support The program includes some component related to family support, with the goal of encouraging safe, stable, and supportive families. This includes programs aimed at reducing or preventing risk factors within the home, such as economic insecurity and weak parenting skills.

Health Care Services The program includes some component of health care services, broadly defined. Such services include physical health screenings; general medical care, including preventive care (e.g., vaccinations); reproductive health; and mental health care screenings and intervention. This topic does not include programs whose only health-related services are referrals to health care services outside of the context of the program.

Health Education The program includes education about health, broadly defined, including substance use and abuse, sex education, sexual abuse prevention, and parent education about nutrition, injury prevention, well-child care, and preventive care.

Instructional Support The program includes some component of instructional support provided to students or teachers, with the aim of improving the academic performance of students or increasing students' knowledge in a particular area.

Mentoring The program includes some component in which a role model is matched with a child or young person, with the purpose of offering the young person guidance, support, and friendship.

Parent Education The program aims to improve parents' knowledge and skills related to parenting, including knowledge of appropriate child development and appropriate parenting strategies.

- 4 -

http://www.promisingpractices.net/programs_topic_list.asp?topicid=4

http://www.promisingpractices.net/programs_topic_list.asp?topicid=39

http://www.promisingpractices.net/programs_topic_list.asp?topicid=10

Youth Development Using developmental tasks and emotional benchmarks, the program aims to support youths' transition from childhood to adolescence and into adulthood.

Type of Outcome Addressed

Behavior Problems The program aims to address or prevent behavior-related outcomes, broadly defined, such as acting out in the classroom, antisocial behaviors, and other internalizing or externalizing behavior problems at school or in the home.

Child Abuse and Neglect The program aims to address or prevent child emotional, physical, and sexual abuse; child neglect; and child endangerment, broadly defined. This includes outcomes associated with parenting practices related to child abuse and neglect.

Cognitive Development/School Performance The program aims to improve young children's school readiness, older children and adolescents' school performance, and children and adolescents' cognitive development beyond the classroom.

Juvenile Justice The program aims to address outcomes related to juvenile justice, broadly defined. This includes programs aimed at preventing juvenile delinquency and improving outcomes for youth already involved in the juvenile justice system, such as programs aimed at preventing reentry or improving behavioral or social skills.

Mental Health The program aims to address mental health–related outcomes that impact a child, broadly defined. This includes preventing or intervening in depression, anxiety, anger, hyperactivity, and suicidal ideation, and improving any problems with social adjustment, self-esteem, self-efficacy, coping, and parents' mental health.

Physical Health The program aims to address health-related outcomes, broadly defined. This includes reducing risky health behavior (e.g., substance abuse, risky sexual activities), access to and utilization of appropriate health services, prevention of unintended injury, etc. This topic does not include programs that exclusively address mental health outcomes with no associated physical health component (see Mental Health topic).

Poverty/Welfare The program aims to improve families' economic security by reducing poverty or welfare utilization or increasing family income. This topic does not include programs whose sole poverty/welfare component is referring families to welfare services.

Substance Use and Dependence The program aims to prevent or reduce youths' initial or ongoing use or abuse of

- 5 -

http://www.promisingpractices.net/programs_topic_list.asp?topicid=16

http://www.promisingpractices.net/programs_topic_list.asp?topicid=15

http://www.promisingpractices.net/programs_topic_list.asp?topicid=14

http://www.promisingpractices.net/programs_topic_list.asp?topicid=20

http://www.promisingpractices.net/programs_topic_list.asp?topicid=18

any chemical substance. This topic does not include programs that address parental substance use (see Family Support topic).

Teen Sex/Pregnancy The program aims to improve adolescents' understanding of risky sexual behaviors or to reduce teen pregnancy and the rates of sexually transmitted diseases among teens.

Violent Behavior The program aims to reduce or prevent children and youths’ violent behaviors directed at themselves or others, including peers. This topic does not include programs that address violence by adults directed toward youth (see Child Abuse topic).

Table 2. PPN Programs, by Age of Child

Proven Programs

Promising Programs

Proven/Promising Programs

Early Childhood (0–8)

The Abecedarian Project

Accelerated Reader

Big Brothers Big Sisters of America

Child-Parent Centers

DARE to be You

Early Head Start

Family Thriving Program

FluText

Healthy Families New York (HFNY)

HighScope Perry Preschool Program

Incredible Years

New Hope Project

Newborn Individualized Developmental Care and Assessment Program (NIDCAP)

Nurse Family Partnership

Reading Recovery

Child Development Project

Child Sexual Abuse Prevention: Teacher Training Workshop

Cognitively Guided Instruction (CGI)

Communities In Schools

Cooperative Integrated Reading and Composition

Coping Cat

Core Knowledge

Direct Instruction

Early Childhood Education and Assistance Program (ECEAP)

Early Intervention in Reading

Father/Male Involvement Preschool Teacher Education Program

Gang Resistance Education and Training (G.R.E.A.T.)

Head Start

Healthy Start

Healthy Steps for Young Children

Family Foundations

Family Support and Parenting Education in the Home

Infant Health and Development Program

- 6 -

Proven Programs

Promising Programs

Proven/Promising Programs

Hip-Hop to Health Jr.

Parent-Child Home Program

Parents as Teachers

Parents' Fair Share

Partners in Reading

Peer-Assisted Learning Strategies (PALS)

Primary and Secondary Control Enhancement Training

Project TRUST

Reaching Educators, Children, and Parents (RECAP)

Resolving Conflict Creatively Program (RCCP)

Safe Child Program

SafeCare

Seattle Social Development Project

Second Step Violence Prevention

Smart Start

Social Decision Making/Problem Solving

Student Achievement Guarantee in Education (SAGE)

Syracuse Family Development Research Program

Targeted Reading Intervention (TRI)

Triple P Positive Parenting Program

Who Do You Tell?

Middle Childhood (9–12)

Accelerated Reader

Adolescent Coping with Depression Course

Big Brothers Big Sisters of America

Class Wide Peer Tutoring Program

Accelerated Academics Academy (AAA)

Be Proud! Be Responsible!

Child Development Project

Child Sexual Abuse Prevention: Teacher Training Workshop

Cognitive Relaxation Coping Skills

- 7 -

Proven Programs

Promising Programs

Proven/Promising Programs

Cognitive Behavioral Intervention for Trauma in Schools (CBITS)

Guiding Good Choices

Incredible Years

LifeSkills Training

Midwestern Prevention Project/Project STAR

New Hope Project

Project ALERT

Project Northland

Cognitively Guided Instruction (CGI)

Communities In Schools

Cooperative Integrated Reading and Composition

Coping Cat

Core Knowledge

Creating Lasting Family Connections

Direct Instruction

Early Intervention in Reading

Gang Resistance Education and Training (G.R.E.A.T.)

Get Real About AIDS

Go Grrrls

Know Your Body

Making Proud Choices!

Parents' Fair Share

Peer-Assisted Learning Strategies (PALS)

Primary and Secondary Control Enhancement Training

Project TRUST

Reaching Educators, Children, and Parents (RECAP)

Reciprocal Teaching

Resolving Conflict Creatively Program (RCCP)

Safe Child Program

Seattle Social Development Project

Second Step Violence Prevention

Social Decision Making/Problem Solving

SPORT

Team Accelerated Instruction: Math

Text4Health-Adolescents

- 8 -

Proven Programs

Promising Programs

Proven/Promising Programs

Triple P Positive Parenting Program

Triple Play

Who Do You Tell?

Adolescence (13–18)

Accelerated Reader

Adolescent Coping with Depression Course

Attachment-Based Family Therapy

Big Brothers Big Sisters of America

Cognitive Behavioral Intervention for Trauma in Schools (CBITS)

Coping with Stress Course

The Effective Learning Program

Guiding Good Choices

LifeSkills Training

Midwestern Prevention Project/Project STAR

Multisystemic Therapy (MST)

National Guard Youth ChalleNGe Program

New Hope Project

Project ALERT

Project Northland

Accelerated Academics Academy (AAA)

Athletes Training and Learning to Avoid Steroids (ATLAS)

Be Proud! Be Responsible!

Child Sexual Abuse Prevention: Teacher Training Workshop

Communities In Schools

Coping Cat

Core Knowledge

Counselors Care (CARE)

Creating Lasting Family Connections

Direct Instruction

Get Real About AIDS

Go Grrrls

Know Your Body

Making Proud Choices!

New York City's Small Schools of Choice

Parents' Fair Share

Peer-Assisted Learning Strategies (PALS)

Postponing Sexual Involvement/Human Sexuality Educational Series

Reciprocal Teaching

Reducing the Risk

Self Center (School-Linked Reproductive Health Services)

Social Decision Making/Problem Solving

SPORT

Quantum Opportunity Program (QOP)

- 9 -

Proven Programs

Promising Programs

Proven/Promising Programs

Talent Development Secondary

Teen Talk

Text4Health-Adolescents

Triple P Positive Parenting Program

Triple Play

Wyman's Teen Outreach Program

Table 3. PPN Programs, by Type of Setting

Proven Programs

Promising Programs

Proven/Promising Programs

Child Care/Preschool

The Abecedarian Project

Child-Parent Centers

DARE to be You

Early Head Start

HighScope Perry Preschool Program

Incredible Years

New Hope Project

Early Childhood Education and Assistance Program (ECEAP)

Father/Male Involvement Preschool Teacher Education Program

Head Start

Hip-Hop to Health Jr.

Reaching Educators, Children, and Parents (RECAP)

Safe Child Program

Second Step Violence Prevention

Smart Start

Syracuse Family Development Research Program

Infant Health and Development Program

Elementary School

Accelerated Reader

Class Wide Peer Tutoring Program

Cognitive Behavioral Intervention for Trauma in Schools (CBITS)

Guiding Good Choices

Incredible Years

LifeSkills Training

Child Development Project

Child Sexual Abuse Prevention: Teacher Training Workshop

Cognitive Relaxation Coping Skills

Cognitively Guided Instruction (CGI)

Communities In Schools

- 10 -

Proven Programs

Promising Programs

Proven/Promising Programs

Reading Recovery

Cooperative Integrated Reading and Composition

Coping Cat

Core Knowledge

Creating Lasting Family Connections

Direct Instruction

Early Intervention in Reading

Gang Resistance Education and Training (G.R.E.A.T.)

Get Real About AIDS

Know Your Body

Partners in Reading

Peer-Assisted Learning Strategies (PALS)

Primary and Secondary Control Enhancement Training

Project TRUST

Reaching Educators, Children, and Parents (RECAP)

Reciprocal Teaching

Resolving Conflict Creatively Program (RCCP)

Safe Child Program

Seattle Social Development Project

Second Step Violence Prevention

Social Decision Making/Problem Solving

Student Achievement Guarantee in Education (SAGE)

Targeted Reading Intervention (TRI)

Team Accelerated Instruction: Math

Who Do You Tell?

- 11 -

Proven Programs

Promising Programs

Proven/Promising Programs

Middle School

Accelerated Reader

Adolescent Coping with Depression Course

Class Wide Peer Tutoring Program

Cognitive Behavioral Intervention for Trauma in Schools (CBITS)

Coping with Stress Course

Guiding Good Choices

LifeSkills Training

Midwestern Prevention Project/Project STAR

Project ALERT

Project Northland

Accelerated Academics Academy (AAA)

Be Proud! Be Responsible!

Child Sexual Abuse Prevention: Teacher Training Workshop

Cognitive Relaxation Coping Skills

Communities In Schools

Cooperative Integrated Reading and Composition

Coping Cat

Core Knowledge

Creating Lasting Family Connections

Direct Instruction

Gang Resistance Education and Training (G.R.E.A.T.)

Get Real About AIDS

Know Your Body

Making Proud Choices!

Peer-Assisted Learning Strategies (PALS)

Postponing Sexual Involvement/Human Sexuality Educational Series

Primary and Secondary Control Enhancement Training

Reciprocal Teaching

Reducing the Risk

Resolving Conflict Creatively Program (RCCP)

Seattle Social Development Project

Second Step Violence Prevention

Self Center (School-Linked Reproductive Health Services)

Social Decision Making/Problem Solving

SPORT

- 12 -

Proven Programs

Promising Programs

Proven/Promising Programs

Talent Development Secondary

Team Accelerated Instruction: Math

Teen Talk

Wyman's Teen Outreach Program

High School

Accelerated Reader

Adolescent Coping with Depression Course

Coping with Stress Course

The Effective Learning Program

Guiding Good Choices

LifeSkills Training

Multisystemic Therapy (MST)

National Guard Youth ChalleNGe Program

Athletes Training and Learning to Avoid Steroids (ATLAS)

Be Proud! Be Responsible!

Child Sexual Abuse Prevention: Teacher Training Workshop

Communities In Schools

Coping Cat

Counselors Care (CARE)

Creating Lasting Family Connections

Direct Instruction

Get Real About AIDS

Making Proud Choices!

New York City's Small Schools of Choice

Peer-Assisted Learning Strategies (PALS)

Reciprocal Teaching

Reducing the Risk

Self Center (School-Linked Reproductive Health Services)

SPORT

Talent Development Secondary

Teen Talk

Wyman's Teen Outreach Program

Quantum Opportunity Program (QOP)

Out-of-School Time

Incredible Years

National Guard Youth ChalleNGe Program

Gang Resistance Education and Training (G.R.E.A.T.)

Go Grrrls

Making Proud Choices!

- 13 -

Proven Programs

Promising Programs

Proven/Promising Programs

SPORT

Student Achievement Guarantee in Education (SAGE)

Teen Talk

Triple Play

Wyman's Teen Outreach Program

Community-Based Service Provider

Adolescent Coping with Depression Course

Big Brothers Big Sisters of America

Coping with Stress Course

DARE to be You

Guiding Good Choices

Incredible Years

Multisystemic Therapy (MST)

New Hope Project

Be Proud! Be Responsible!

Communities In Schools

Coping Cat

Creating Lasting Family Connections

Early Childhood Education and Assistance Program (ECEAP)

Get Real About AIDS

Healthy Start

Making Proud Choices!

Parents as Teachers

Parents' Fair Share

Reducing the Risk

Smart Start

Social Decision Making/Problem Solving

Syracuse Family Development Research Program

Teen Talk

Triple P Positive Parenting Program

Triple Play

Wyman's Teen Outreach Program

Family Foundations

Family Support and Parenting Education in the Home

Quantum Opportunity Program (QOP)

Health Care Provider

Adolescent Coping with Depression Course

Attachment-Based Family Therapy

Coping with Stress Course

Be Proud! Be Responsible!

Communities In Schools

Coping Cat

Creating Lasting Family Connections

Family Foundations

Family Support and Parenting Education in the Home

Infant Health and Development Program

- 14 -

http://www.promisingpractices.net/program.asp?programid=43

Proven Programs

Promising Programs

Proven/Promising Programs

FluText

Guiding Good Choices

Multisystemic Therapy (MST)

New Hope Project

Newborn Individualized Developmental Care and Assessment Program (NIDCAP)

Nurse Family Partnership

Early Childhood Education and Assistance Program (ECEAP)

Healthy Start

Healthy Steps for Young Children

Making Proud Choices!

Postponing Sexual Involvement/Human Sexuality Educational Series

Reducing the Risk

Self Center (School-Linked Reproductive Health Services)

Smart Start

Teen Talk

Text4Health-Adolescents

Wyman's Teen Outreach Program

Home Visiting

The Abecedarian Project

Early Head Start

Family Thriving Program

Healthy Families New York (HFNY)

HighScope Perry Preschool Program

Nurse Family Partnership

Early Childhood Education and Assistance Program (ECEAP)

Healthy Steps for Young Children

Parent-Child Home Program

Parents as Teachers

SafeCare

Syracuse Family Development Research Program

Triple P Positive Parenting Program

Family Support and Parenting Education in the Home

Infant Health and Development Program

Table 4. PPN Programs, by Type of Service

Proven Programs

Promising Programs

Proven/Promising Programs

Case Management

Newborn Individualized Developmental Care and Assessment Program (NIDCAP)

Communities In Schools

Syracuse Family Development Research Program

Quantum Opportunity Program (QOP)

- 15 -

http://www.promisingpractices.net/program.asp?programid=270

Proven Programs

Promising Programs

Proven/Promising Programs

Family Support

The Abecedarian Project

Attachment-Based Family Therapy

Child-Parent Centers

DARE to be You

Early Head Start

Family Thriving Program

Guiding Good Choices

Healthy Families New York (HFNY)

Incredible Years

Multisystemic Therapy (MST)

New Hope Project

Newborn Individualized Developmental Care and Assessment Program (NIDCAP)

Nurse Family Partnership

Communities In Schools

Creating Lasting Family Connections

Early Childhood Education and Assistance Program (ECEAP)

Gang Resistance Education and Training (G.R.E.A.T.)

Head Start

Healthy Start

Parents as Teachers

Parents' Fair Share

SafeCare

Smart Start

Syracuse Family Development Research Program

Triple P Positive Parenting Program

Family Foundations

Family Support and Parenting Education in the Home

Infant Health and Development Program

Health Care Services

The Abecedarian Project

Adolescent Coping with Depression Course

Attachment-Based Family Therapy

Child-Parent Centers

Cognitive Behavioral Intervention for Trauma in Schools (CBITS)

Early Head Start

FluText

Healthy Families New York (HFNY)

Newborn Individualized Developmental Care and Assessment Program (NIDCAP)

Nurse Family Partnership

Cognitive Relaxation Coping Skills

Coping Cat

Counselors Care (CARE)

Early Childhood Education and Assistance Program (ECEAP)

Healthy Start

Healthy Steps for Young Children

Parents as Teachers

Primary and Secondary Control Enhancement Training

Self Center (School-Linked Reproductive Health Services)

Smart Start

Text4Health-Adolescents

Infant Health and Development Program

- 16 -

Proven Programs

Promising Programs

Proven/Promising Programs

Health Education

Adolescent Coping with Depression Course

Coping with Stress Course

DARE to be You

FluText

Guiding Good Choices

Healthy Families New York (HFNY)

LifeSkills Training

Newborn Individualized Developmental Care and Assessment Program (NIDCAP)

Nurse Family Partnership

Project ALERT

Project Northland

Athletes Training and Learning to Avoid Steroids (ATLAS)

Be Proud! Be Responsible!

Child Sexual Abuse Prevention: Teacher Training Workshop

Creating Lasting Family Connections

Get Real About AIDS

Healthy Start

Healthy Steps for Young Children

Hip-Hop to Health Jr.

Know Your Body

Making Proud Choices!

Postponing Sexual Involvement/Human Sexuality Educational Series

Reducing the Risk

Seattle Social Development Project

Self Center (School-Linked Reproductive Health Services)

Smart Start

SPORT

Syracuse Family Development Research Program

Teen Talk

Triple Play

Wyman's Teen Outreach Program

Family Support and Parenting Education in the Home

Infant Health and Development Program

Instructional Support

The Abecedarian Project

Accelerated Reader

Child-Parent Centers

Class Wide Peer Tutoring Program

Early Head Start

Accelerated Academics Academy (AAA)

Child Development Project

Cognitively Guided Instruction (CGI)

Cooperative Integrated Reading and Composition

Quantum Opportunity Program (QOP)

- 17 -

Proven Programs

Promising Programs

Proven/Promising Programs

The Effective Learning Program

HighScope Perry Preschool Program

Incredible Years

National Guard Youth ChalleNGe Program

Reading Recovery

Core Knowledge

Direct Instruction

Early Childhood Education and Assistance Program (ECEAP)

Early Intervention in Reading

Head Start

New York City's Small Schools of Choice

Partners in Reading

Peer-Assisted Learning Strategies (PALS)

Reaching Educators, Children, and Parents (RECAP)

Reciprocal Teaching

Seattle Social Development Project

Student Achievement Guarantee in Education (SAGE)

Syracuse Family Development Research Program

Talent Development Secondary

Targeted Reading Intervention (TRI)

Team Accelerated Instruction: Math

Mentoring

Big Brothers Big Sisters of America

National Guard Youth ChalleNGe Program

Gang Resistance Education and Training (G.R.E.A.T.)

Quantum Opportunity Program (QOP)

Parent Education

Child-Parent Centers

Cognitive Behavioral Intervention for Trauma in Schools (CBITS)

DARE to be You

Early Head Start

Family Thriving Program

Creating Lasting Family Connections

Early Childhood Education and Assistance Program (ECEAP)

Father/Male Involvement Preschool Teacher Education Program

Family Foundations

Family Support and Parenting Education in the Home

Infant Health and Development Program

- 18 -

Proven Programs

Promising Programs

Proven/Promising Programs

Guiding Good Choices

Healthy Families New York (HFNY)

HighScope Perry Preschool Program

Incredible Years

Midwestern Prevention Project/Project STAR

Multisystemic Therapy (MST)

Nurse Family Partnership

Project ALERT

Project Northland

Gang Resistance Education and Training (G.R.E.A.T.)

Head Start

Healthy Start

Healthy Steps for Young Children

Parent-Child Home Program

Parents as Teachers

Parents' Fair Share

Reaching Educators, Children, and Parents (RECAP)

SafeCare

Seattle Social Development Project

Second Step Violence Prevention

Smart Start

Syracuse Family Development Research Program

Triple P Positive Parenting Program

Who Do You Tell?

Youth Development

Adolescent Coping with Depression Course

Attachment-Based Family Therapy

Big Brothers Big Sisters of America

Cognitive Behavioral Intervention for Trauma in Schools (CBITS)

Coping with Stress Course

The Effective Learning Program

Guiding Good Choices

Incredible Years

LifeSkills Training

Midwestern Prevention Project/Project STAR

Multisystemic Therapy (MST)

Accelerated Academics Academy (AAA)

Athletes Training and Learning to Avoid Steroids (ATLAS)

Be Proud! Be Responsible!

Cognitive Relaxation Coping Skills

Communities In Schools

Coping Cat

Counselors Care (CARE)

Creating Lasting Family Connections

Gang Resistance Education and Training (G.R.E.A.T.)

Get Real About AIDS

Go Grrrls

Know Your Body

Quantum Opportunity Program (QOP)

- 19 -

Proven Programs

Promising Programs

Proven/Promising Programs

National Guard Youth ChalleNGe Program

Project ALERT

Project Northland

Making Proud Choices!

Postponing Sexual Involvement/Human Sexuality Educational Series

Primary and Secondary Control Enhancement Training

Project TRUST

Reaching Educators, Children, and Parents (RECAP)

Reducing the Risk

Resolving Conflict Creatively Program (RCCP)

Safe Child Program

Seattle Social Development Project

Second Step Violence Prevention

Self Center (School-Linked Reproductive Health Services)

Social Decision Making/Problem Solving

Talent Development Secondary

Teen Talk

Triple Play

Who Do You Tell?

Wyman's Teen Outreach Program

Table 5. PPN Programs, by Type of Outcome Addressed

Proven Programs

Promising Programs

Proven/Promising Programs

Behavior Problems

Big Brothers Big Sisters of America

Child-Parent Centers

Cognitive Behavioral Intervention for Trauma in Schools (CBITS)

Child Development Project

Cognitive Relaxation Coping Skills

Creating Lasting Family Connections

Family Foundations

Infant Health and Development Program

Quantum Opportunity Program (QOP)

- 20 -

Proven Programs

Promising Programs

Proven/Promising Programs

DARE to be You

Early Head Start

Incredible Years

Multisystemic Therapy (MST)

New Hope Project

Nurse Family Partnership

Gang Resistance Education and Training (G.R.E.A.T.)

Head Start

Parents as Teachers

Reaching Educators, Children, and Parents (RECAP)

Resolving Conflict Creatively Program (RCCP)

Seattle Social Development Project

Second Step Violence Prevention

Social Decision Making/Problem Solving

Syracuse Family Development Research Program

Child Abuse and Neglect

Child-Parent Centers

Family Thriving Program

Healthy Families New York (HFNY)

Nurse Family Partnership

Child Sexual Abuse Prevention: Teacher Training Workshop

Parents as Teachers

Project TRUST

Safe Child Program

SafeCare

Triple P Positive Parenting Program

Who Do You Tell?

Family Support and Parenting Education in the Home

Cognitive Development/School Performance

The Abecedarian Project

Accelerated Reader

Big Brothers Big Sisters of America

Child-Parent Centers

Class Wide Peer Tutoring Program

Cognitive Behavioral Intervention for Trauma in Schools (CBITS)

Early Head Start

The Effective Learning Program

Accelerated Academics Academy (AAA)

Child Development Project

Cognitively Guided Instruction (CGI)

Communities In Schools

Cooperative Integrated Reading and Composition

Core Knowledge

Direct Instruction

Infant Health and Development Program

Quantum Opportunity Program (QOP)

- 21 -

Proven Programs

Promising Programs

Proven/Promising Programs

HighScope Perry Preschool Program

National Guard Youth ChalleNGe Program

New Hope Project

Newborn Individualized Developmental Care and Assessment Program (NIDCAP)

Nurse Family Partnership

Reading Recovery

Early Childhood Education and Assistance Program (ECEAP)

Early Intervention in Reading

Head Start

New York City's Small Schools of Choice

Parent-Child Home Program

Parents as Teachers

Partners in Reading

Peer-Assisted Learning Strategies (PALS)

Reciprocal Teaching

Resolving Conflict Creatively Program (RCCP)

Seattle Social Development Project

Smart Start

Student Achievement Guarantee in Education (SAGE)

Syracuse Family Development Research Program

Talent Development Secondary

Targeted Reading Intervention (TRI)

Team Accelerated Instruction: Math

Wyman's Teen Outreach Program

Juvenile Justice

Child-Parent Centers

HighScope Perry Preschool Program

Multisystemic Therapy (MST)

National Guard Youth ChalleNGe Program

Nurse Family Partnership

Gang Resistance Education and Training (G.R.E.A.T.)

Seattle Social Development Project

Syracuse Family Development Research Program

- 22 -

Proven Programs

Promising Programs

Proven/Promising Programs

Mental Health

Adolescent Coping with Depression Course

Attachment-Based Family Therapy

Cognitive Behavioral Intervention for Trauma in Schools (CBITS)

Coping with Stress Course

Family Thriving Program

Incredible Years

Newborn Individualized Developmental Care and Assessment Program (NIDCAP)

Athletes Training and Learning to Avoid Steroids (ATLAS)

Cognitive Relaxation Coping Skills

Coping Cat

Counselors Care (CARE)

Go Grrrls

Primary and Secondary Control Enhancement Training

Reaching Educators, Children, and Parents (RECAP)

Seattle Social Development Project

Social Decision Making/Problem Solving

Syracuse Family Development Research Program

Family Foundations

Physical Health

The Abecedarian Project

Big Brothers Big Sisters of America

Child-Parent Centers

Early Head Start

FluText

Guiding Good Choices

Healthy Families New York (HFNY)

LifeSkills Training

Midwestern Prevention Project/Project STAR

Multisystemic Therapy (MST)

National Guard Youth ChalleNGe Program

Newborn Individualized Developmental Care and Assessment Program (NIDCAP)

Nurse Family Partnership

Athletes Training and Learning to Avoid Steroids (ATLAS)

Be Proud! Be Responsible!

Child Development Project

Counselors Care (CARE)

Creating Lasting Family Connections

Early Childhood Education and Assistance Program (ECEAP)

Gang Resistance Education and Training (G.R.E.A.T.)

Get Real About AIDS

Head Start

Healthy Start

Healthy Steps for Young Children

Hip-Hop to Health Jr.

Know Your Body

Making Proud Choices!

Family Support and Parenting Education in the Home

Infant Health and Development Program

- 23 -

Proven Programs

Promising Programs

Proven/Promising Programs

Project ALERT

Project Northland Postponing Sexual Involvement/Human Sexuality Educational Series

Reducing the Risk

Seattle Social Development Project

Self Center (School-Linked Reproductive Health Services)

Social Decision Making/Problem Solving

SPORT

Teen Talk

Text4Health-Adolescents

Triple P Positive Parenting Program

Triple Play

Wyman's Teen Outreach Program

Poverty/Wealth

HighScope Perry Preschool Program

National Guard Youth ChalleNGe Program

New Hope Project

Nurse Family Partnership

Early Childhood Education and Assistance Program (ECEAP)

Parents' Fair Share

Substance Use and Dependence

The Abecedarian Project

Big Brothers Big Sisters of America

DARE to be You

Guiding Good Choices

Healthy Families New York (HFNY)

LifeSkills Training

Midwestern Prevention Project/Project STAR

Athletes Training and Learning to Avoid Steroids (ATLAS)

Child Development Project

Counselors Care (CARE)

Creating Lasting Family Connections

Gang Resistance Education and Training (G.R.E.A.T.)

Get Real About AIDS

Healthy Start

Know Your Body

Infant Health and Development Program

- 24 -

Proven Programs

Promising Programs

Proven/Promising Programs

Multisystemic Therapy (MST)

National Guard Youth ChalleNGe Program

Nurse Family Partnership

Project ALERT

Project Northland

Seattle Social Development Project

Social Decision Making/Problem Solving

SPORT

Teen Sex/Pregnancy

LifeSkills Training

National Guard Youth ChalleNGe Program

Nurse Family Partnership

Be Proud! Be Responsible!

Get Real About AIDS

Making Proud Choices!

Postponing Sexual Involvement/Human Sexuality Educational Series

Reducing the Risk

Seattle Social Development Project

Self Center (School-Linked Reproductive Health Services)

Teen Talk

Wyman's Teen Outreach Program

Quantum Opportunity Program (QOP)

Violent Behavior

Big Brothers Big Sisters of America

HighScope Perry Preschool Program

Multisystemic Therapy (MST)

New Hope Project

Nurse Family Partnership

Child Development Project

Cognitive Relaxation Coping Skills

Gang Resistance Education and Training (G.R.E.A.T.)

Resolving Conflict Creatively Program (RCCP)

Seattle Social Development Project

Second Step Violence Prevention

Syracuse Family Development Research Program

- 25 -

Programs That Work The Abecedarian Project

Program Info

Outcome Areas Healthy and Safe Children Children Ready for School Children Succeeding in School

Indicators Youths not using alcohol, tobacco, or illegal drugs Students performing at grade level or meeting state curriculum standards Children ages 0 to 5 exhibiting age-appropriate mental and physical development Children experiencing good physical health

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Home Visiting Type of Service Family Support Health Care Services Instructional Support Type of Outcome Addressed Cognitive Development/School Performance Physical Health Substance Use and Dependence

Evidence Level Proven

Program Overview

The Abecedarian Project was a comprehensive early education program for young children at risk for developmental delays and school failure. The program operated at a single site between 1972 and 1985, in North Carolina, and underwent numerous assessments of its long-term effects on participants. The Abecedarian Project involved two components: a preschool intervention and a school-age intervention. The preschool program was offered in a day care center setting, the main goal of which was to create an educational, stimulating, and structured environment to promote growth and learning and to enhance school readiness. The curriculum was designed to enhance cognitive and linguistic development and to provide an enriched language environment that was responsive to children's needs and interests. In addition, children received nutritional supplements and disposable diapers, along with pediatric care and supportive social work services. Infants began attending the preschool program between six weeks and three months of age, and continued until entry into kindergarten. Children attended the day care center six to eight hours per day, five days per week, 50 weeks per year. After the children turned three years old, they received a more structured set of educational curricula, which became increasingly similar to programs in the local public kindergartens as the children grew older. The school-age intervention began at kindergarten entry and continued through the first three years of elementary school. A resource teacher was assigned to each child and family for the length of the

- 26 -

program. The resource teacher prepared an individualized set of home activities to supplement the school's basic curriculum in reading and math, taught parents how to use these activities with their children, tutored children directly, met regularly with classroom teachers to ensure that home activities aligned with the skills being taught in the classroom, served as a consultant for the classroom teacher when problems arose, and advocated for the child and family within the school and community. Resource teachers made approximately 17 school visits and approximately 15 home visits per year for each child. In addition, they offered children a variety of summertime supports, including summer activity packets, help in arranging summer camp experiences, trips to the public library, and tutoring in reading skills.

Program Participants

The Abecedarian Project focused its enrollment efforts on at-risk families with infants up to six months of age. At-risk families were referred to the project through local hospitals, clinics, the Department of Social Services, and other referral sources. These families were visited at home by a staff member, who explained the program to them and determined whether or not they met certain selection criteria. If so, mothers were invited to the Frank Porter Graham Center at the University of North Carolina at Chapel Hill for an interview and psychological assessment. They were screened using the High Risk Index, which included 13 socio-demographic factors associated with poor intellectual and scholastic progress. The scores on these factors were weighted and combined to arrive at a total for each family. If a family scored 11 or more points, that family was judged to be at elevated risk and eligible for inclusion in the study. The families taking part in the evaluated program were low-income and predominantly African- American (98 percent). The majority of the families were headed by a single mother (83 percent) with an average age of 20 years, an average IQ of 85 (15th percentile), and a low level of education. Target children were predominantly firstborns and included slightly more females (53 percent) than males.

Evaluation Methods

The Abecedarian project was a two-part intervention, consisting of a preschool intervention and a primary school-aged intervention. Admission to the original preschool intervention took place over a five-year period, from 1972 to 1977, with four groups of approximately 28 children each being admitted during that time span. Children and families were matched on the basis of High Risk Index scores and were then randomized to preschool intervention or control groups from birth to five years. The final sample included 111 infants from 109 families at baseline, 57 of whom were randomly assigned to the intervention group and 54 to the control group. Four children withdrew early in the study, leaving 107 children in the initial analysis sample (Ramey and Campbell, 1984). In the intervention for primary school-age children, children in the preschool intervention and control groups were further randomized into primary school intervention and control groups. Prior to kindergarten entry, pairs of children within the preschool intervention and control groups were equated on the basis of their 48-month IQ scores, with one of each pair randomly assigned to the school-age intervention group for the first three years of school and the other to the control group. This research design resulted in four separate groups for comparison. On admission to kindergarten, 96 children remained in the study, and, by the end of the follow-up study period, data on 92 children were available (Horacek et al., 1987). Therefore, the participants consisted of children who had undergone one of the following:

• eight years of intervention—five in preschool and three in primary school (the experimental/experimental [or EE] group with 25 children)

• five years of intervention in preschool only (the experimental/control [or EC] group with 24 children)

- 27 -

• three years of primary school intervention only (the control/experimental [or CE] group with 21 children)

• no educational intervention at all (the control/control [or CC] group with 22 children).

Studies assessing outcomes for children after primary school were analyzed in one of two (or both) ways. Outcomes were analyzed either among all four groups of children, or by collapsing the preschool experimental groups (EE and EC) together to form a single preschool intervention group and collapsing the preschool control groups (CE and CC) together to form a single preschool control group. Subsequent to both the original set of studies, which collected data in preschool and at the end of the kindergarten year, data have been obtained and analyzed for the participants at ages 8, 12, 15, 18, and 21 years. Outcomes assessed included the following:

• Early childhood outcomes were assessed at six-month intervals from six months of age

through 54 months (4.5 years old) (Ramey et al., 1984; Burchinal, Lee, and Ramey, 1989; Martin, Ramey, and Ramey, 1990):

o Bayley Scales of Infant Development

o Stanford-Binet Intelligence Scale (IQ)

o McCarthy Scales of Children's Abilities (to test the development of specific skills at specific ages)

o Wechsler Preschool and Primary Scale of Intelligence (WPPSI).

• Primary school-age outcomes were assessed at ages 5, 6, 7, and 8 (Horacek et al., 1987;

Ramey and Campbell, 1994):

o grade retention

o Woodcock-Johnson achievement test scores in reading, math, written language, and knowledge

o Wechsler Preschool and Primary Intelligence Scale for Children (WPPSI)

o IQ.

• Middle school- and high school-age outcomes were assessed at ages 12, 15, and 18

(Campbell and Ramey, 1994; Campbell and Ramey, 1995; Clarke and Campbell, 1998):

o grade retention

o Wechsler Intelligence Scale for Children (WISC-R)

o Delinquency (including number of convictions for misdemeanors or felonies and amount of time incarcerated or on probation)

• Adult outcomes were assessed at age 21 (Campbell et al., 2002; Muennig et al., 2011):

o Wechsler Adult Intelligence Scale (WAIS-R)

o use of alcohol, tobacco, marijuana, cocaine, or other illegal drugs

o education levels, including attendance at a four-year college

o employment

o teen pregnancy

o conviction and incarceration rates

- 28 -

o health composite scale, including

• Brief Symptom Inventory depression scale

• number of hospitalizations in previous year

• self-reported health problems since age 15

o behavioral risk factor composite scale, including 11 measures concerning traffic safety, drug use, and access to primary care.

Key Evaluation Findings

Early Childhood Outcomes

The initial study of the Abecedarian Project (Ramey and Campbell, 1984) found that the preschool intervention group scored significantly higher than the control group on the following measures:

• the Bayley Mental Development Index at 18 months (but not at 6 or 12 months)

• IQ at 24, 36, and 48 months

• the McCarthy General Cognitive Index at 42 and 54 months

• the McCarthy Verbal Scale Index at 30, 42, and 54 months

• the McCarthy Perceptual-Performance Scale Index at 42 and 54 months

• the McCarthy Quantitative Scale Index at 42 and 54 months

• the McCarthy Memory Scale Index at 54 months (but not at 42 months).

No significant differences were found for motor skill development at any of the measurement intervals. A second study analyzing data from the first 54 months of the study (Martin, Ramey, and Ramey, 1990), found that

• The intervention groups had higher IQ scores than the control groups even when the effects of maternal IQ and home environment were controlled for.

• The positive impact of educational day care was especially pronounced for the children whose mothers scored lowest on maternal IQ.

The third preschool study (Burchinal, Lee, and Ramey, 1989) used three groups for analysis purposes, comparing the Abecedarian intervention group with both a no-day-care control group and a community day care control group. The researchers found that the intervention group children scored significantly higher than both the community group and the no-day-care group children over time on the following measures:

• the Bayley Mental Development Index at 6, 12, and 18 months

• IQ tests at 24, 36, and 48 months

• the McCarthy General Cognitive Index at 42 and 54 months.

In addition, the differences between the intervention group and no-day-care group were larger than the differences between the intervention group and the community day care group. The community day care group outscored the no-day-care group on all measures at all time intervals.

- 29 -

Primary School Findings

The first study following the children to age 8 (Horacek et al., 1987) found that by the end of their third year in school,

• Both the preschool intervention and the primary school intervention were associated with lower grade retention. Specifically,

o 50 percent of the children who had received no intervention at either level (the CC children) and 38 percent of the children who had received no preschool intervention but had received the primary school intervention (the CE children) had failed at least one grade in school.

o In comparison, 29 percent of the children who had received the preschool intervention but not the primary school intervention (the EC children) and 16 percent of the children who had received intervention at both levels (the EE children) had failed a grade.

• When achievement test results in reading were compared, scores were shown to have improved with increasing length of time spent in the Abecedarian program intervention. That is, the group with no intervention (CC) exhibited the poorest scores (75 percent of the group was in the bottom quartile of reading scores), while the group with eight years of intervention (EE) demonstrated the best performance (44 percent of the group was in the bottom quartile of reading scores).

• Similarly for math scores, 50 percent of the CC group children scored in the lowest quartile, compared with 28 percent of the children in the EE group, while the EC and CE groups fell somewhere in between.

Another study assessing outcomes up to age 8 (Ramey and Campbell, 1994) found that

• The preschool intervention groups (EE and EC) scored significantly higher, on average, on the Wechsler Intelligence Scale, relative to with the preschool control groups (CE and CC), across time, i.e., at measurements taken at 60, 78, and 96 months.

• Educational intervention that began during the primary school grades (i.e., without the preschool intervention) was not associated with positive effects on Wechsler Intelligence Scale performance.

Middle School- and High School-Age Outcomes

A follow-up study to age 12 (Campbell and Ramey, 1994) found a significant effect for

• The preschool intervention, but not the school-age intervention, on children's overall Wechsler Intelligence score (verbal and performance intelligence combined).

• The preschool intervention on Wechsler verbal intelligence (e.g., vocabulary, comprehension, arithmetic) but not on Wechsler performance intelligence (e.g., picture completion, object assembly, mazes)

• The school-age intervention on performance intelligence but not on verbal intelligence.

• The preschool intervention on reading, math, knowledge, and written language scores. There were no significant effects found for the school-age intervention on these measures.

• Grade retention, with children from the preschool intervention group retained in grade less often. During the first seven years of school, 32 percent of EE students and 38 percent of EC students were retained in grade, compared with 52 percent of CE students and 57 percent of CC students.

- 30 -

A follow-up at age 15 (Campbell and Ramey, 1995) found the following:

• Higher Wechsler Intelligence Scale scores over time were positively related to the preschool intervention. EE and EC students had overall average intelligence scores of 99.4 and 100.2, respectively, while CE and CC students had average scores of 88.9 and 93.1, respectively.

• The preschool intervention group earned significantly higher scores on Woodcock-Johnson measures of reading and mathematics than did the control group.

• Significantly fewer preschool intervention group students were retained in grade during the first ten years of school, compared with the control group students (31.2 percent compared with 54.5 percent).

A follow-up at age 18 focusing on criminal activity (Clarke and Campbell, 1998) found no significant differences between the preschool group and no-preschool group on any measures of crime or arrests, including age at first criminal charge. Adult Outcomes

A follow-up at age 21 (Campbell et al., 2002) found significant differences favoring the preschool intervention group over the control group for

• the full-scale Wechsler intelligence test and verbal intelligence test

• Woodcock-Johnson scales for broad reading, letter-word identification, broad mathematics, calculation, reading-grade equivalent, and math-grade equivalent

• average years of education by age 21 (12.2 versus 11.6 years)

• percentage attending or having attended a four-year college (35.9 percent versus 13.7%)

• percentage having become teenaged parents (26 percent versus 45%)

• percentage self-reporting marijuana use within the past 30 days (18 percent versus 39%)

• percentage indicating they were regular cigarette smokers (39 percent versus 55%).

No significant differences were found between the preschool intervention and control groups on

• the Wechsler performance intelligence test

• percentage graduating high school

• percentage currently employed

• conviction and incarceration rates

• use of illegal drugs (other than marijuana)

• binge drinking or use of alcohol in the past 30 days.

An analysis of the age-21 follow-up data examining the effects of the preschool intervention (Muennig et al., 2011) found the following: • Health scale

o Scores on the overall health composite scale score (a composite index including depression symptoms, prior year hospitalizations, and self-reported health) were significantly improved in the treatment group relative to the control group.

- 31 -

o None of the three individual elements that comprised the health scale were significantly different when comparing the treatment group with the control group.

• Behavioral risk factor scale

o Scores on the overall behavioral risk composite scale (composed of 11 measures regarding traffic safety, drug use, and access to primary care) were significantly improved in the treatment group compared with the control group.

o When considered individually, three of the 11 measures on the behavioral risk factor scale were higher for the treatment group. All three significantly different measures indicated improvement in the treatment group relative to the control group. These measures included age at which participant first began smoking, age at which participant first tried marijuana, and frequency of marijuana use in the past month.

Finally, an additional analysis of the age-21 data examined the relationship between project participation and mental health outcomes (McLaughlin et al., 2007). This analysis found a significant program effect on depression as measured by the Brief Symptom Inventory (BSI), finding that participation in the Abecedarian project was associated with an average reduction of over 4 points on the BSI as compared to the control group. This finding appears to be contrary to the findings for the same measure reported in Muennig et al. (2011), but it should be noted that the McLaughlin et al. paper considered the interaction between early childhood intervention and the quality of the early home environment as joint predictors of self-reported depressive symptoms in young adults. McLaughlin et al. showed that the early intervention reduced the impact of a high risk home environment on the later depressive feelings of persons from high-risk backgrounds.

Probable Implementers

Public and private child care programs and elementary schools

Funding

Early phases of the research were primarily funded by a series of grants from

• the Mental Retardation and Developmental Disabilities Branch of the National Institutes of Child Health and Human Development

• the Department of Human Resources of the State of North Carolina.

The age-21 follow-up study of the Abecedarian program was funded jointly by

• the Maternal and Child Health Bureau of the Department of Health and Human Services

• the National Institute on Early Childhood Development and Education of the Office of Educational Research and Improvement, Department of Education

• the David and Lucile Packard Foundation

• the Frank Porter Graham Child Development Center.

A cost-benefit analysis of the Abecedarian program (Masse and Barnett, 2002) determined that the average annual cost of the program is about $17,099 per child (in 2011 dollars, adjusted for inflation). The researchers concluded that the long-term benefits of the program, including future earnings and maternal earnings as well as health benefits from a potential decrease in smoking, outweighed costs by a factor of four dollars for every one dollar spent.

- 32 -

Implementation Detail

Program Design

• The day care center operated from 7:45 a.m. to 5:30 p.m., five days per week, 50 weeks per year. Free transportation was provided to families if needed.

• Infants up to walking age were cared for in a nursery area, while toddlers and preschoolers were grouped in other areas according to age and developmental levels.

• Families were encouraged to participate in parent group sessions on topics related to parenting and family development.

• Social workers were available to provide assistance to parents with such issues as housing and social services as well as personal counseling.

• Pediatric care was provided by a team of on-site research nurses and pediatricians.

Curriculum: Preschool

• The curriculum for the first three years of life was developed by Sparling and Lewis (1981). The curriculum consisted of language, motor, social, and cognitive skill development, and these items were individually presented to each child. With increasing age, the curriculum was sometimes presented in small groups.

• After the age of 3, other standard preschool curricula were introduced in the day care center, such as the Peabody Early Experiences Kit (Dunn et al., 1976), Bridge-to-Reading (Greenberg and Epstein, 1973), the GOAL math program (Karnes, 1973), the My Friends and Me social skills program (Davis, 1977), and the Wallach and Wallach (1976) pre-phonics reading program.

• The preschool program focused on the development of communication skills, and each child was conversed with and read to daily. Derived from programs developed by Tough (1976) and Blank (1973), the program emphasized teachers' informative and nondirective verbal interchanges with the children.

Curriculum: Primary School

• No specific curriculum was used for the school-age intervention, as the content depended on the curricula used by the local public schools.

• The resource teachers met with classroom teachers to learn which concepts (in mathematics and reading) were currently being taught in school, after which they designed enjoyable and compatible home activities. The resource teacher discussed these activities with the children's families and encouraged parents to use them with their children on a regular basis.

• In a typical year, approximately 60 different learning activities were designed for each child. The average amount of time parents reported working with their children on the activities was about 15 minutes per day.

Staffing

• In the Abecedarian Project, the preschool staff included a director, 12 teachers and aides, and an administrative staff. Teacher-child ratios began at one-to-three in the nursery and gradually increased to one-to-six in the last preschool year.

• The teaching staff varied in their professional background from those with graduate degrees in early childhood education to paraprofessionals, but all had had extensive experience in working with young children.

• In-service training and technical assistance were provided to the educational staff through periodic on-site programs by educational, psychological, and pediatric consultants and by teachers' participation in local and national educational conferences and workshops.

- 33 -

• For the school-age intervention, the resource teachers were graduate-level teachers with backgrounds in primary education who worked with approximately 12 children each per year.

The Abecedarian Project is not currently in operation, and there are no available materials to support replication.

Issues to Consider

This program received a "proven" rating. The evaluation of the program used rigorous standards, including a randomized experimental design and longitudinal follow-up, and the participants experienced significant and sizeable gains across most of the cognitive and academic performance outcomes. Although the Abecedarian Project did not set out to select at-risk families based on race, 98 percent of the resulting group of children was African-American. In addition, 83 percent of the families included in the intervention group were female-headed and were otherwise characterized by extreme poverty, unstable employment, and low educational attainment for mothers and fathers. It is unclear whether similar results would be obtained with other types of families. In addition, the experiment took place in a small, relatively affluent university town with a wide range of human services agencies, including public hospitals and programs providing housing, fuel subsidies, food supplements, and job training. Thus, a relatively high degree of social, fiscal, and material supports were already available to participants (both intervention and control), and results may not be the same in cities with markedly different features. Besides the main finding of significant effects for the Abecedarian program as a whole at enhancing long-term IQ and academic achievement, another important conclusion to be drawn from the evaluations is that five years of the Abecedarian preschool intervention is almost equally as effective as five years of the preschool program with a three-year primary school follow-up, and definitely more effective than the three-year Abecedarian primary school intervention alone. An age-21 follow-up analysis was conducted examining the relationship between depressive symptoms in young adults, the quality of the early home environment, and participation in the intervention. Authors found a significant relationship between the quality of the home environment and depression in the control group, but not in the treatment group, suggesting that the program mitigated any negative effects of a low-quality home environment (McLaughlin et al., 2007).

Example Sites

The Abecedarian Project was implemented at a single site in North Carolina.

Contact Information

Frances Campbell, Ph.D. Senior Scientist Frank Porter Graham Child Development Center University of North Carolina Campus Box 8180 105 Smith Level Road Chapel Hill, NC 27599-8180 (919) 966-4529 Fax: (919) 966-7532 Campbell@mail.fpg.unc.edu

- 34 -

mailto:Campbell@mail.fpg.unc.edu

Available Resources

The Abecedarian Project Program Package (available for purchase on this page: http://www.socio.com/eipardd01.php) includes a range of resources that can help providers to replicate the Abecedarian curriculum. General information about the Abecedarian Project is available on the Abecedarian Project homepage, from the Frank Porter Graham Child Development Institute at the University of North Carolina at Chapel Hill: http://www.fpg.unc.edu/~abc/

Bibliography

Blank, Marion, Teaching Learning in the Preschool: A Dialogue Approach, Columbus, Ohio: Merrill, 1973. Burchinal, Margaret, Marvin Lee, and Craig T. Ramey, "Type of Day-Care and Preschool Intellectual Development in Disadvantaged Children," Child Development, Vol. 60, No. 1, 1989, pp. 128-137. Campbell, Frances A., and Craig T. Ramey, "Cognitive and School Outcomes for High-Risk African- American Students at Middle Adolescence: Positive Effects of Early Intervention," American Educational Research Journal, Vol. 32, No. 4, 1995, pp. 743-772. Campbell, Frances A., and Craig T. Ramey, "Effects of Early Intervention on Intellectual and Academic Achievement: A Follow-Up Study of Children from Low-Income Families," Child Development, Vol. 65, 1994, pp. 684-698. Campbell, Frances A., Craig T. Ramey, Elizabeth Pungello, Joseph Sparling, and Shari Miller-Johnson, "Early Childhood Education: Young Adult Outcomes from the Abecedarian Project," Applied Developmental Science, Vol. 6, No. 1, 2002, pp. 42-57. Clarke, Stevens H., and Frances A. Campbell, "Can Intervention Early Prevent Crime Later? The Abecedarian Project Compared with Other Programs," Early Childhood Research Quarterly, Vol. 13, No. 2, 1998, pp. 31-343. Davis, Duane Q., My Friends and Me, Circle Pines, Minn.: American Guidance Service, 1977. Dunn, Lloyd M., Lillie T. Chun, Doris C. Crowell, Leota M. Dunn, Lynne Grossman Halevi, and Eleanor R. Yackel, Peabody Early Experiences Kit, Circle Pines, Minn.: American Guidance Service, 1976. Greenberg, Polly, and Bea Epstein, Bridge-to-Reading: Curriculum for Pre-School, Kindergarten, and First Grade, Morristown, N.J.: General Learning Corp, 1973. Horacek, H. Joseph, Craig T. Ramey, Frances A. Campbell, Kathleen P. Hoffmann, and Robert H. Fletcher, "Predicting School Failure and Assessing Early Intervention with High-Risk Children," American Academy of Child and Adolescent Psychiatry, Vol. 26, No. 5, 1987, pp. 758-763. Karnes Merle B., GOAL Program: Mathematical Concepts, Springfield, Mass.: Milton-Bradley, 1973. Martin, Sandra L., Craig T. Ramey, and Sharon Ramey, "The Prevention of Intellectual Impairment in Children of Impoverished Families: Findings of a Randomized Trial of Educational Day Care," American Journal of Public Health, Vol. 80, No. 7, 1990, pp. 844-847. Masse, Leonard N., and W. Steven Barnett, A Benefit-Cost Analysis of the Abecedarian Early Childhood Intervention, New Brunswick, N.J.: National Institute for Early Education Research, 2002. McLaughlin, Andrea, Frances A. Campbell, Elizabeth P. Pungello, and Martie Skinner, "Depressive Symptoms in Young Adults: The Influences of the Early Home Environment and Early Educational Child Care," Child Development, Vol. 78, No. 3, May/June 2007, pp. 746-756.

- 35 -

http://www.socio.com/eipardd01.php

http://www.fpg.unc.edu/%7Eabc/

Muennig, Peter, Dylan Robertson, Gretchen Johnson, Frances Campbell, Elizabeth P. Pungello, and Matthew Neidell, "The Effect of an Early Education Program on Adult Health: The Carolina Abecedarian Project Randomized Controlled Trial," American Journal of Public Health, Vol. 101, No. 3, 2011, pp. 512-516. Ramey, Craig T., and Frances A. Campbell, "Poverty, Early Childhood Education, and Academic Competence: The Abecedarian Experiment," in Aletha C. Huston, ed., Children in Poverty: Child Development and Public Policy, New York: Cambridge University Press, 1994, pp. 190-221. Ramey, Craig T., and Frances A. Campbell, "Preventive Education for High-Risk Children: Cognitive Consequences of the Carolina Abecedarian Project," American Journal of Mental Deficiency, Vol. 88, No. 5, 1984, pp. 515-523. Sparling, Joseph, and Isabelle Lewis, Learning Games for the First Three Years: A Program for Parent/Center Partnership, New York: Walker Educational Book Corp, 1981. Tough, Joan, Listening to Children Talking, London, UK: Word Lock Educational, 1976. Wallach, Lisa, and Michael A. Wallach, The Teaching of All Children to Read Kit, Chicago: University of Chicago Press, 1976.

Last Reviewed

May 2011

Accelerated Academics Academy (AAA)

Program Info

Outcome Areas Children Succeeding in School

Indicators Students graduating from high school

Topic Areas Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Middle School Type of Service Instructional Support Youth Development Type of Outcome Addressed Cognitive Development/School Performance

Evidence Level Promising

- 36 -

Program Overview

The Accelerated Academics Academy (AAA) was an alternative middle school established in the Flint Community Schools school district in Flint, Michigan. Funded and evaluated as part of the federally funded School Dropout Demonstration Assistance Program (SDDAP), it began its operations in 1991 and continued to provide services as of 2008. The primary goal of AAA was to accelerate instruction so that students who are behind one or more grade levels could enter high school with their same-age peers. AAA emphasized small class sizes and a curriculum designed by its staff that compressed two years of middle school learning into one year. AAA teachers placed great emphasis on integrating the curriculum across core academic subjects. In addition to regular teaching activities, the program also included such services as counseling, attendance monitoring, and outreach to families.

Program Participants

AAA served students in grades six through eight who were two or more grade levels behind peers in their age group in Flint, Michigan. Sixty-one percent of the students were African-American, 53 percent came from a household that received public assistance, and 69 percent had discipline problems in their previous schools.

Evaluation Methods

The research study was a randomized controlled trial. Eligible students in the school district were randomly assigned to an intervention group that was offered admission to AAA. Students assigned to the control group generally attended traditional middle schools in the same district. Cohort 1, starting in the 1992-1993 school year, included 46 students in the intervention group and 46 students in the control group. Cohort 2, starting in the 1993-1994 school year, included 66 students in the treatment group and 40 students in the control group. The evaluation design included both pre and post measurements and intervention and control groups. Baseline data and follow-up data were collected annually from school district records and student surveys. Cohort 1 and 2 were each followed for three and two years, respectively. Baseline data collection included students’ demographic characteristics and risk factors (previous attendance records, standardized test scores, and self-esteem). At the baseline, the treatment group and control group students were generally comparable, with two exceptions: First, the treatment group students were less likely than the control group students to have a mother whose educational attainment fell into the "some college" category; second, the treatment group students were less likely than the control group students to have two or more risk factors. The follow-up surveys included academic outcomes (grades, standardized test scores, dropout rates), personal outcomes (self-esteem and locus of control), and social outcomes (alcohol and drug use, arrests, and pregnancies). In addition, the research team gathered information about special services received by students in each group, such as counseling; participation in "special classes" in math, English, or other subjects; and referrals to a social service agency for counseling, health needs, or financial assistance.

Key Evaluation Findings

Findings summarized here are drawn from Dynarski et al. (1998), in which AAA was evaluated along with 15 other federally funded SDDAP programs. Results below only pertain to the AAA program.

• Two years after the AAA program started, the average dropout rate among both cohorts of AAA students (2%) was significantly lower than that of control group students enrolled in traditional middle schools (9%).

- 37 -

• Two years after the AAA program started, the average highest grade completed among both cohorts of AAA students (7.3) was significantly higher than that of control group students enrolled in traditional middle schools (6.8).

• Three years after the AAA program started, the dropout rate among Cohort 1 AAA students (3%) was significantly lower than that of control group students enrolled in traditional middle schools (13%).

• Three years after the AAA program started, the average highest grade completed among Cohort 1 AAA students (8.5) was significantly higher than that of control group students enrolled in traditional middle schools (7.8).

• AAA students and control group students did not demonstrate statistically significant differences in math, English, or reading grades at two or three years into the program. Also, the two groups did not differ in percentage time absent from school.

• In the first follow-up year, AAA students and control group students did not have statistically significant differences in the likelihood of receiving on-campus counseling. However, AAA students were significantly less likely to attend special classes in math, English, or other subjects and to be referred to a Social Service Agency for counseling, health needs, or financial assistance.

• Two years and three years after the program started, AAA students and control group students in traditional middle schools generally did not demonstrate statistically significant differences in self-esteem, locus of control, or personal aspirations. The only exception was that Cohort 1 AAA students reported a significantly lower rate of external control in the third- year follow-up survey.

Probable Implementers

This program could be implemented in public or private school districts either as an independent school or as a special program within an existing school.

Funding

This program was initially supported by grants from the U.S. Department of Education as part of the SDDAP during 1991-1995.

Implementation Detail

Program Design

• The program was self-contained, with its own buildings, staff, and identity within the district.

• Program enrollment was limited to 100 students.

• Teachers were encouraged to use nontraditional approaches, such as cooperative learning groups, instructional technology, and peer tutoring.

• AAA offered five core subjects: language arts, mathematics, science, social studies, and art.

• Each Wednesday, students had opportunities to take a "Wonderful Wednesday" class that included a rotating set of topics chosen based on student interests, including algebra, Spanish, quilting, and science club.

- 38 -

Staffing

AAA instructional members were regular classroom teachers from the Flint Community Schools school district. The school also had a full-time counselor and a full-time social worker. No information was available regarding their training. Curriculum

AAA used a curriculum designed by its staff that compressed two years of middle school learning into one year.

Issues to Consider

This program received a "promising" rating. The research, conducted over 1992-1995, was implemented according to rigorous design standards and included an experimental group of 102 students and a comparison group of 86 students. The research used a randomized controlled trial. However, it is unclear what percentage of the students who were offered AAA admission actually enrolled in the program. If those who declined the offer on average had lower aspirations for education than those who accepted the offer, the true effect of the AAA program might have been overstated. On the other hand, if those who declined the offer were on average more advanced students than those who accepted the offer, the true effect of the AAA program might have been understated. Another potential weakness of this study is that the control group had a higher attrition rate than the intervention group. For example, in the second-year follow-up survey, 96 percent of the treatment-group students responded, compared with 85 percent in the control group. If the lower response rate among control group students was due to a higher dropout rate among these students, the true effect of the AAA program in preventing dropout might have been underestimated. The cost of the AAA program was evaluated in Rosenberg and Hershey (1995). The average per- student monthly cost for AAA was $790, which was 22 percent higher than that of the regular schools in the same district. The AAA program was considered to be relatively inexpensive compared with the other 15 SDDAP programs, where per-student monthly costs could be as much as $382 higher, which was 109 percent higher than that of the regular schools in the same district. Overall, the AAA program helped students stay in school but did not noticeably improve student learning or personal outcomes. It should be noted that all statistical significance tests regarding the impact of the program were conducted at the 0.10 level, rather than the more commonly used 0.05 level.

Example Sites

Flint Community Schools school district in Flint, Michigan

Contact Information

Dr. Mark Dynarski Mathematica Policy Research, Inc. P.O. Box 2393 Princeton, NJ 08543-2393 Tel: (609)799-3535

Available Resources

Additional information about all SDDAP program models and implementation experiences, including those concerning the AAA program, can be found in Hershey, Adelman, and Murray (1995).

- 39 -

Bibliography

Dynarski, Mark, Philip Gleason, Anu Rangarajan, and Robert Wood, Impacts of Dropout Prevention Programs: Final Report—A Research Report from the School Dropout Demonstration Assistance Program Evaluation, Princeton, N.J.: Mathematica Policy Research, Inc., June 1998. Hershey, Alan, Nancy Adelman, and Stephen Murray, Helping Kids Succeed: Implementation of the School Dropout Demonstration Assistance Program, Princeton, N.J.: Mathematica Policy Research, Inc., September 1995. Rosenberg, Linda, and Alan Hershey, The Cost of Dropout Prevention Programs, Princeton, N.J.: Mathematica Policy Research, Inc., February 1995.

Last Reviewed

November 2009

Accelerated Reader

Program Info

Outcome Areas Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School High School Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance

Evidence Level Proven

Program Overview

Accelerated Reader is a computer program designed to facilitate independent reading in the classroom for students in kindergarten through 12th grade. Accelerated Reader monitors students' progress and provides feedback on reading comprehension to the students and their teachers. Accelerated Reader is intended to complement core reading programs used by teachers. Teachers are able to guide student reading through Accelerated Reader by directing their selection of reading materials. After reading a book, students complete quizzes assessing reading comprehension (Magnolia Consulting, 2010).

- 40 -

Program Participants

Accelerated Reader has been used in kindergarten through 12 grade.

Evaluation Methods

There is a large body of research on the effectiveness of Accelerated Reader. One study utilized an experimental design that compared intervention group outcomes with those of a control group (Magnolia Consulting 2010). Other studies conducted do not meet the PPN criteria for inclusion. Only the results of the study that meets PPN criteria are presented here. Note that this study evaluated only Accelerated Reader use for 1st through 4th grades, so we do not present evidence on the effectiveness of Accelerated Reader for other grades. Magnolia Consulting conducted an evaluation of Accelerated Reader among 1st- through 4th-graders during the 2009-2010 school year in three private Catholic schools. The schools were located in a large city in the North-Central region of the United States. Within each school, teachers were randomized to the intervention or control group, such that half of the teachers within each grade were in each group. In total, 19 teachers from the three schools were randomized. There were 344 students within the 19 classrooms. Students were evaluated using the STAR Reading test at pretest, midpoint, and posttest and using Accelerated Reader quizzes, which were administered throughout. The average student took 20 quizzes every week. Student gains in reading from pretest to midpoint and pretest to posttest were compared between intervention and control students. Despite random assignment, there were significant differences on STAR reading test performance at pretest, with control group students performing significantly better than intervention group students on average (mean pretest score: control group 417.91 vs. intervention 372.27). These differences were controlled for in the analyses. Analyses accounted for nesting of students within classrooms.

Key Evaluation Findings

The Magnolia Consulting (2010) evaluation found no significant impact of Accelerated Reader on student performance between pretest and midterm. Between pretest and posttest, however, students in the Accelerated Reader group demonstrated significantly higher gains in reading achievement on average when compared with students in the control group.

Probable Implementers

Elementary school teachers, reading specialists

Funding

Approximately 60 percent of schools using Accelerated Reader fund the program using existing funds. The Accelerated Reader program meets the requirements of No Child Left Behind and qualifies for federal funding under the following programs: Title I, Part A (Improving Academic Achievement of the Disadvantaged, Improving Basic Programs); Title I, Part B (Reading First); Title II, Part D (Enhancing Education Through Technology); Title III, Part A (English Language Acquisition); Title IV, Part A, Subpart I (Safe and Drug-Free Schools); Title IV, Part B (21st Century Community Learning Centers); Title V, Part A (Innovative Programs); and Title VI, Part B (Rural Education Achievement Program). More information regarding funding can be obtained through the Renaissance Learning website: http://www.renlearn.com/fundingcenter/default.aspx.

- 41 -

http://www.renlearn.com/fundingcenter/default.aspx

Implementation Detail

Program Design

The Accelerated Reader program is a computer software program that was designed to facilitate independent reading by students in the classroom and to assist teachers with evaluating student reading performance. Accelerated Reader was not designed to be a core reading program; it is intended to be used in conjunction with core reading programs already in place in classrooms (Magnolia Consulting 2010). Students begin by reading a book at their reading level. After reading the book, students take a quiz using the Accelerated Reader software to assess reading comprehension. The software is designed to provide detailed feedback on student performance that teachers can use to guide each student's reading practice. Students also receive immediate feedback after taking comprehension quizzes.

Staffing

The Accelerated Reader program does not require additional staff as it is designed to be used alongside core reading programs already in place in classrooms.

Issues to Consider

The evaluation conducted by Magnolia Consulting during the 2009-2010 school year was a randomized evaluation in which classrooms were assigned to the Accelerated Reader program (intervention group) or to the standard reading program used by the given teacher (control group). This study included students only in 1st through 4th grades. The evaluation found significantly higher gains in reading, as assessed by the STAR reading test, for students in the Accelerated Reader classrooms, compared with the control classrooms when comparing pretest to posttest performance. The Accelerated Reader program has been used through 12th grade, but the effectiveness of the program beyond 4th grade has not been assessed by studies that meet the PPN evidence criteria. Accelerated Reader is also available with a program management option which includes consultation from Renaissance Learning throughout the year as Accelerated Reader is implemented. This program is available at an additional cost to the basic Accelerated Reader Program. Schools may also purchase on-site or remote consultation on an hourly or daily basis. Two evaluations have assessed the impact of Accelerated Reader with program management through randomized controlled trials (Nunnery et al., 2006 & Ross et al., 2004). These evaluations are not presented in detail here because the use of Renaissance consultants is not a standard part of the Accelerated Reader program The evaluation conducted by Ross et al. (2004) found significant positive effects of the Accelerated Reader program in kindergarten through third grades but not in fourth through sixth grades. The evaluation conducted by Nunnery et al. (2006) found significant positive effects of Accelerated Reader among students in grades three through six. The Magnolia Consulting evaluation used the STAR Early Literacy and STAR Reading tests to evaluation student reading achievement. Both the Accelerated Reader software and the STAR reading tests were designed by Renaissance Learning; the STAR Reading and Early Literacy measures were also validated by Renaissance Learning (Renaissance Learning, 2012, and Renaissance Learning, 2013). Finally, the Magnolia Consulting (2010) evaluation found no significant interaction between gender and program group (i.e., Accelerated Reader or control group) or between ethnicity and program group. This indicates that the effect of the Accelerated Reading program did not differ across children of different genders or ethnicities.

Example Sites

The Accelerated Reader program has been implemented in many different locations such as Pascagoula and Biloxi, Mississippi and school districts throughout Florida.

- 42 -

Contact Information

Renaissance Learning, Inc. PO Box 8036 Wisconsin Rapids, Wisconsin 54495-8036 (800) 338-4204 answers@relearn.com

Available Resources

Renaissance Learning website: http://www.renlearn.com/ar/

Bibliography

Magnolia Consulting, A Final Report for the Evaluation of Renaissance Learning's Accelerated Reader Program, Charlottesville, Va., 2010. As of January 31, 2013: http://www.magnoliaconsulting.org/AR%20Final%20Report%202010.pdf Nunnery, John A., Steven M. Ross, and Aaron McDonald, "A Randomized Experimental Evaluation of the Impact of Accelerated Reader/Reading Renaissance Implementation on Reading Achievement in Grades 3 to 6," Journal of Education for Students Placed at Risk, Vol. 11 No. 1, 2006. pp. 1-18. Renaissance Learning, Renaissance Tools Qualify for Federal Funding, Wisconsin Rapids, Wis., 2012. As of January 31, 2013: http://doc.renlearn.com/KMNet/R004060615GGE424.pdf Renaissance Learning, STAR Early Literacy: Technical Manual, Wisconsin Rapids, Wis., 2012. As of January 31, 2013: Renaissance Learning, STAR Reading: Technical Manual, Wisconsin Rapids, Wis., 2013. As of January 31, 2013: https://resources.renlearnrp.com/US/Manuals/SR/SRRPTechnicalManual.pdf Ross, Steven M., John A. Nunnery, and Elizabeth Goldfeder, "A Randomized Experiment on the Effects of Accelerated Reader/Reading Renaissance in an Urban School District: Final Evaluation Report," University of Memphis, Center for Research in Educational Policy, 2004. http://doc.renlearn.com/KMNet/R004076723GH55D8.pdf

Last Reviewed

April 2013

Adolescent Coping with Depression Course

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children not experiencing anxiety or mood disorders, such as depression

- 43 -

mailto:answers@relearn.com

http://www.renlearn.com/ar/

http://www.magnoliaconsulting.org/AR%20Final%20Report%202010.pdf

http://doc.renlearn.com/KMNet/R004060615GGE424.pdf

https://resources.renlearnrp.com/US/Manuals/SR/SRRPTechnicalManual.pdf

http://doc.renlearn.com/KMNet/R004076723GH55D8.pdf

Topic Areas Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Middle School High School Community-Based Service Provider Health Care Provider Type of Service Health Care Services Health Education Youth Development Type of Outcome Addressed Mental Health

Evidence Level Proven

Program Overview

The Adolescent Coping with Depression Course (CWD-A) is a skills-based small-group treatment program for actively depressed adolescents. The intervention consists of 16 sessions delivered over a period of eight weeks, with six monthly continuation sessions. Adolescents are taught several skills hypothesized to relieve depression, including assertiveness, relaxation skills, cognitive restructuring techniques, mood monitoring, increasing pleasant activities, and communication and conflict- resolution techniques. A parent component helps keep parents aware of what their teens are learning in the program regarding general topics discussed, skills taught, and the rationale for their use. CWD-A has been adapted for adolescents at risk for depression (not currently depressed); that program, the Coping with Stress Course, is also listed as a Proven Program.

Program Participants

Adolescents with depression symptoms

Evaluation Methods

Seven studies of CWD-A meet the PPN inclusion criteria. Kahn et al. (1990) studied the effects of the CWD-A program in a sample of 6th-to-8th-grade students from a large suburban middle school in Utah. A school-wide self-report screening for depression was administered, using an adapted version of the Children's Depression Inventory (CDI) and the Reynolds Adolescent Depression Scale (RADS). One hundred eighty-six students (14.4 percent) scored above the predetermined cutoffs for having depression. To rule out situational or transitory mood disorders, assessment procedures were repeated one month after the initial screening. At the reassessment, 110 students (59 percent) met the criteria for depression. Seven students did not obtain consent to participate and nine students left the study school, leaving a total of 94 students. These students were assessed on a structured interview, the Bellevue Index of Depression (BID), and 79 students met BID cutoff scores for depression. Six of these students moved out of the study area, two were excluded for receiving concurrent outpatient psychological/psychiatric services, and three did not receive parental consent. The final sample was 68 students, who were randomly assigned to one of four groups: (1) CWD-A, (2) relaxation treatment (which involved progressive relaxation skills and basic relaxation training), (3) self-modeling treatment (which involved repeated observation of oneself on edited or rehearsed videotapes showing only desired target behaviors), or (4) a waitlist control group. There

- 44 -

http://www.promisingpractices.net/criteria.asp

were 17 students in each of the four groups. Posttesting on the RADS, CDI, and BID occurred immediately upon completion of the intervention and at a one-month follow-up.

Lewinsohn et al. (1990) studied CWD-A in a sample of 59 Oregon high school students, aged 14-18. A total of 69 adolescents met the inclusion criteria for elevated levels of depression. Participants were randomly assigned to one of three groups: (1) adolescent-only (19 students); (2) adolescent + parent, an identical group for adolescents with their parents enrolled in a separate parent group (21 students); and (3) waitlist control (19 students). Ten students withdrew before or during treatment, leaving a total of 59 study participants (no significant differences were found between completers and noncompleters). When treatment and control groups were compared prior to the intervention, the only significant difference found between groups was that control subjects had a higher average number of siblings than did subjects in the treatment groups. Outcomes were assessed immediately posttreatment and at a one-month and a six-month follow-up. Adolescent outcomes were assessed on the interviewer-rated Schedule for Affective Disorders and Schizophrenia for School-Aged Children, Epidemiological Version (K-SADS-E). Youth self-reports were measured on the Beck Depression Inventory (BDI), the Center for Epidemiological Studies Depression Scale (CES-D), three scales assessing depressogenic cognitions or thoughts, the Pleasant Events Schedule, the Spielberger State Anxiety Questionnaire, and the Issues Checklist (a measure of conflict resolution). Parent reports of their child's behavior problems and depression were assessed on the Child Behavior Checklist (CBCL), and reports of disagreements between parent and teen assessments were measured on the Issues Checklist. CWD-A was evaluated by Rohde, Lewinsohn, and Seeley (1994) in a sample of 14-to-18-year-old students from Oregon. Students were screened and included if they were diagnosed with major depressive disorder (MDD), without concurrent bipolar disorder, panic disorder, generalized anxiety disorder, alcoholism, conduct disorder, or drug use disorder; without current involvement in other treatment for depression; and without a need for immediate treatment or hospitalization. Eighty-four students met the criteria for inclusion, with a subsequent attrition rate of 22 percent. The average participant age was 16.3 years, 74 percent of the students were female, 99 percent were white, and 49 percent had had a previous episode of MDD. Subjects were randomly assigned to one of three groups: (1) CWD-A for adolescents only (31 students); (2) CWD-A for adolescents with a separate group for parents (29 students); or (3) a waitlist control group (24 students). Analyses were conducted separately for low-severity and high-severity groups with regard to past history of MDD. Outcomes were assessed at posttest on the Hamilton Depression Rating Scale (HAM-D) using a structured interview and adolescent self-reports of depression on the BDI and the CES-D. The longer-term effects of CWD-A were studied by Clarke et al. (1999) in a sample of 123 adolescents from Oregon. Three hundred thirty-one adolescents aged 14-18 and at least one of their parents participated in a screening interview. Eligible adolescents were determined by the interview to have MDD or dysthymia (moderate depression); a total of 171 adolescents met these criteria. Twenty- seven adolescents were excluded because they exhibited current mania/hypomania, panic disorder, generalized anxiety disorder, conduct disorder, psychoactive substance abuse/dependence, lifetime organic brain syndrome, mental retardation, or schizophrenia, or because they were determined to need immediate, acute treatment. An additional 21 adolescents were excluded for concurrently receiving other treatment for depression. The final sample of 123 adolescents was randomly assigned to one of three groups: (1) CWD-A (45 students), (2) CWD-A with a separate parent group (42 students), or (3) waitlist control (36 students). Of the initial 123 study participants, 27 did not complete the intervention, leaving a total of 96 (37 in the adolescent-only group, 32 in the adolescent + parent group, and 27 in the control group). Attrition was not significantly related to experimental group, demographic characteristics of adolescents, intake diagnosis of MDD versus dysthymia, intake BDI or HAM-D scores. Of the 96 adolescents, 71 percent were female, and the adolescents had a mean age of 16.2 years. The three experimental groups did not differ significantly on any of the demographic or baseline outcome variables. Immediately after posttest assessment, participants completing the CWD-A groups were randomly reassigned to one of three groups for the two-year follow-up period: (1) booster sessions plus assessments every four months (24 students), (2) assessments only every four months (16 students), or (3) assessments only once per year (24 students). At posttest and follow-up, outcomes were measured on the K-SADS-E interview, the HAM-D interview, the interviewer-rated Global Assessment of Functioning Scale (GAF), the adolescent- reported BDI, and the parent-reported CBCL scales.

- 45 -

Clarke and colleagues (2002) evaluated CWD-A in a sample of depressed youths with depressed parents in a health maintenance organization (HMO). The sampling frame consisted of approximately 410,000 members enrolled in a Kaiser Permanente Northwest HMO, around Portland, Oregon. Recruitment letters were sent to parents of offspring ages 13-18 who had a documented diagnosis of depression. Introductory study letters signed by each parent's treating physician were mailed to 2,995 adults. Of these families, 2,514 declined or were not reached before the baseline assessment. Four hundred sixty-three youth completed the baseline assessment but were not randomized, for reasons such as the family was not eligible, the youth was categorized as "resilient," or the youth declined. In the final sample, 88 eligible adolescents aged 13-18 who met current Diagnostic and Statistical Manual of Mental Disorders (DSM-III-R) criteria for major depression and/or dysthymia were randomly assigned to either usual HMO behavioral health care (47 adolescents) or usual care plus CWD-A (41 adolescents). Assessments were conducted immediately posttreatment (86 youths), at 12-month follow-up (80 youths), and at 24-month follow-up (73 youths). At baseline, the treatment and control groups did not differ with respect to rates of current and past psychiatric disorder, the amount of "usual care" mental health and general health care services consumed during any phase of the day, or on any other key demographic, depression severity, functioning, or psychological measures. Outcomes were assessed via the K-SADS interview, the HAM-D interview, the interviewer-rated GAF scale, parent reports on the CBCL, and parent and youth reports on the CES-D scale. Rohde et al. (2004) assessed the effects of CWD-A in a sample of 93 adolescents aged 13-17. All adolescents were under the supervision of an intake, probation, or parole officer and were not incarcerated at the time of entry into the study. Referred adolescents were screened for depression, and 93 adolescents meeting DSM-IV criteria for MDD and conduct disorder were randomly assigned to either CWD-A (45 youth) or a life skills intervention (48 youth). The life skills intervention consisted of training in life skills, such as filling out a job application and an application for renting an apartment, and academic tutoring. A comparison of baseline demographic and clinical characteristics (including current and past episodes of psychiatric disorders and treatment) found that the only significant difference between groups was gender; the CWD-A group had significantly more female participants than the Life-Skills/Tutoring condition (60 percent versus 38 percent). Participants were assessed posttreatment and at 6- and 12-month follow-up. Outcomes were assessed on the Longitudinal Interval Follow-Up Evaluation (LIFE) interview, the K-SADS-E-5 interview, the HAM-D interview, the interviewer-rated Children's Global Adjustment Scale, the self-reported Social Adjustment Scale, the self-reported BDI, the parent-reported CBCL scales, and youth criminal records. Finally, Garber et al. (2009) conducted a trial of CWD-A across four major U.S. cities in which 316 youth were randomly assigned to CWD-A or usual care. Adolescents were required to be between aged 13-17 and have current depressive symptoms (score of 20 or higher on CES-D) or a prior episode of a DSM-IV depressive disorder. They were also required to have at least one parent who had experienced one episode of MDD during the past three years, or three or more years in a depressive state in their lifetime. Depressive episodes for parents and youth were assessed using the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID-I). The CWD-A intervention consisted of eight weekly 90-minute treatment sessions (the "acute" phase) and six monthly follow-up sessions (the "continuation" phase). Study outcomes were assessed prior to intervention, post-acute phase, and post-continuation phase. The primary study outcome was rate of subsequent depressive episodes, as diagnosed by clinical interviewers using the Depression Symptom Rating (DSR) scale from the LIFE Secondary outcomes included depression symptom ratings using the CES-D and Children's Depression Rating Scale-Revised (CDRS-R). There were no significant differences across treatment groups on any demographic or clinical characteristic at baseline.

Key Evaluation Findings

The study of 68 Utah middle school students by Kahn et al. (1990) found the following:

• Outcomes supported the efficacy of all three treatment groups (CWD-A, relaxation, and self- modeling) compared with the control group. Posttest scores on the self-reported RADS and CDI measures and on the BID interview showed that the majority of students in the CWD-A

- 46 -

and relaxation groups moved from the dysfunctional to the functional range and remained there at the one-month follow-up. More specifically,

o RADS: 88 percent of CWD-A, 65 percent of relaxation, and 70 percent of self-modeling students moved from the dysfunctional to the functional range at posttest. At one- month-follow-up, 88 percent of CWD-A, 65 percent of relaxation, and 50 percent of self-modeling students remained in the functional range.

o CDI: 88 percent of CWD-A, 76 percent of relaxation, and 59 percent of self-modeling students were in the functional range at posttest. At one-month follow-up, 76 percent of CWD-A, 65 percent of relaxation, and 44 percent of self-modeling students remained in the functional range.

o BID: 76 percent of CWD-A, 65 percent of relaxation, and 59 percent of self-modeling students had moved from the dysfunctional to the functional range at posttest. (No one-month follow-up interviews were conducted).

• Students in the control group showed much lower rates of gains from pretest to posttest, with 81 percent of control students continuing to show significant levels of depressive symptomatology on the RADS, 88 percent showing dysfunctional outcomes on the CDI, and 81 percent showing dysfunctional outcomes on the BID.

Lewinsohn et al. (1990) reported that, immediately following the intervention,

• 52 percent of the adolescent + parent teens and 57 percent of the Adolescent-Only teens still met diagnostic criteria for depression on the K-SADS-E; in contrast, there was little change in the control condition, with 95 percent of teens meeting diagnostic criteria. When the two treatment groups were pooled, they were found to have significantly lower rates of depression compared with the control group. Analyses showed no significant differences between the two treatment conditions on the K-SADS-E.

• Adolescent depression scores were significantly lower for the pooled treatment group compared with the control group on both the BDI and the CES-D. No significant differences were found between groups for adolescent reports of conflict on the Issues Checklist.

• No significant differences were found between pooled treatment and control groups for parent- reported CBCL scales or parent reports on the Issues Checklist.

• When the two treatment groups were compared, parents in adolescent + parent condition reported significantly lower problem scores for their children on the CBCL scales than did parents in the adolescent-only condition. No differences were found between the groups on the parent-reported Issues Checklist, and no differences were found between groups on any of the adolescent measures.

• No significant differences were found among groups for anxiety, participation in pleasant activities, or depressogenic thoughts.

• Treatment effects were assessed for the adolescent + parent and adolescent-only groups at one- and six-month follow-up. Results indicated the following:

o There were no significant differences between groups for any of the adolescent variables, including the CES-D, the BDI, and the Issues Checklist.

o Following the initial posttest, the adolescent-only group continued to show improved parent ratings on the CBCL scales, eliminating any significant differences between the two treatment groups by the six-month follow-up.

The study by Rohde, Lewinsohn, and Seeley (1994) found the following:

• No significant program effects were evident for the low-severity group.

- 47 -

• The contrast between the high-severity treatment group and the control group was statistically significant for adolescent reports of depression on the BDI.

• Marginally significant differences between the treatment and control group were found for the adolescent-reported CES-D depression scale and the interviewer-rated HAM-D.

• No significant differences between the adolescent-only and the adolescent+ parent treatment groups were found for any measures.

Clarke et al. (1999) reported the following:

• Recovery from depression was defined as no longer meeting DSM-III-R criteria for either major depression or dysthymia for the two weeks following the posttest. Results indicated that the two active treatments groups had significantly better outcomes when compared with controls, and they did not differ significantly from one another. Recovery rates were 65 percent for the adolescent-only group, 69 percent for the adolescent + parent group, and 48 percent for the control group.

• Compared with the control group, at posttest the combined treatment group was associated with significantly greater reduction in adolescent-reported BDI scores and interviewer-rated GAF scores. Differences in scores between the two treatment groups were not significant.

• At posttest, no significant differences were found between treatment and control groups for the HAM-D interview and the three parent-reported CBCL scales (depression, internalizing behavior, and externalizing behavior).

• Recurrence to depression was examined among the 46 adolescents in the two active treatment groups who had recovered at posttest. By both 12- and 24-month follow-ups, no significant differences in recurrence rates were found among groups. At 12 months, recurrence rates were 14 percent in the annual assessment group, 0 percent in the four-month assessment group, and 27 percent in the four-month booster plus assessment group. At 24- months, recurrence rates were 23 percent in the annual assessment group, 0 percent in the four-month assessment group, and 36 percent in the four-month booster plus assessment group.

• In the longitudinal comparison of the booster condition versus the two assessment-only conditions, the only significant effect found was for CBCL externalizing scores, with parents in the booster condition reporting greater reductions in externalizing symptoms over time.

• When comparisons were made over time between the two assessment-only conditions, a significantly greater score decline was observed among the frequent assessment participants (every four months) for both CBCL depression and internalizing subscales.

The study of children of depressed parents in an HMO (Clarke et al., 2002) found the following:

• There were no significant differences between the treatment and control groups for depression diagnoses or depression measures, including the CES-D, HAM-D, CBCL internalizing and externalizing scales, or the GAF scale.

Rohde et al.'s (2004) study of youth involved with a county department of juvenile corrections reported the following:

• Posttest recovery rates for MDD were significantly greater in the CWD-A group than in the Life-Skills/Tutoring group, (39 percent versus 19 percent, respectively). Recovery rates for conduct disorder did not differ significantly between the groups (9 percent versus 17 percent).

• Compared with the Life-Skills/Tutoring group, the CWD-A treatment was associated with significant improvements on the BDI, the HAM-D interview, and the youth-reported Social Adjustment Scale. No significant differences were found between groups for parent ratings on the CBC, interviewer ratings on the Children's Global Adjustment Scale, or arrest rates.

- 48 -

• At 6- and 12-month follow-ups, group differences in MDD and conduct disorder recovery rates were not significant. Similarly, no significant differences were found between groups for the BDI, the HAM-D, the Social Adjustment Scale, the CBCL, the Children's Global Adjustment Scale, or arrest rates.

Clarke et al. (2009) found improvements on all primary and secondary measures for CWD-A participants as compared with those who received usual care:

• Subsequent rates of depressive episodes were lower for those in CWD-A than for those in usual care at the post-continuation follow-up (21.4 percent versus 32.7 percent of participants).

• Self-reported depressive symptoms declined at a significantly greater rate for CWD-A participants than for those in the control condition. At post-continuation, CES-D scores were 19 percent lower among CWD-A participants versus those in the control group, and CDRS-R scores were 6 percent lower.

Probable Implementers

Public and private middle and high schools, community-based organizations, hospitals, clinics, after- school programs, and departments of correction.

Funding

Funding for the majority of the evaluations of CWD-A has been supported by grants from the National Institute of Mental Health.

Implementation Detail

Program Design

The CWD-A course combines cognitive and behavioral strategies that address the types of problems that commonly characterize depressed individuals (e.g., pessimism, low self-esteem, infrequent engagement in pleasant activities, social withdrawal, anxiety and tension, low social support, and increased conflict). CWD-A is based on the premise that teaching adolescents a variety of coping skills and strategies allows them to counteract the various factors that contribute to their depressive episodes and helps them deal more effectively with the problems they encounter (Rohde et al., 2005).

Curriculum

CWD-A is a group intervention that includes structured intervention sessions, repeated practice of skills, use of rewards and contracts, and homework assignments. The intervention consists of 16 two- hour sessions delivered over eight weeks to groups of 6-10 adolescents, led by one or two therapists trained in the approach. Each participant receives a workbook that provides structured learning tasks, short quizzes, and homework assignments. Components of the CWD-A course include

• increasing social skills (training in basic conversational techniques, planning of social activities, and strategies for making friends)

• increasing pleasant activities (teaching of basic self-change skills, including self-monitoring to establish a baseline, setting realistic goals, developing a plan for behavior change, and self- reinforcement for achieving the goals of their plan)

• decreasing anxiety (progressive muscle relaxation and deep breathing)

• reducing depressive cognitions (cartoon strips illustrate depressive thoughts and alternative positive thoughts that may be used to counter them)

- 49 -

• communication (acquisition of positive behaviors, such as active listening, and the inhibition of nonproductive behaviors, such as accusations)

• four steps for problem solving: (a) defining the problem without criticism, (b) brainstorming alternative solutions, (c) evaluating and agreeing on a solution, and (d) specifying the agreement, including positive and negative consequences for compliance and noncompliance

• planning for the future (integration of skills, anticipation of future problems, and development of a life plan and goals).

A parallel group intervention for the parents of depressed adolescents seeks to inform parents of the CWD-A participants to encourage their support and reinforcement of the adolescent's use of skills, and to teach parents communication and problem-solving skills. Parents meet with a separate therapist weekly for two-hour sessions. Two joint sessions are held in the seventh week, during which the adolescents and the parents practice these skills on relevant family issues. Additional workbooks guide parents through these sessions. Individualized booster sessions are offered at four-month intervals for a two-year period after treatment in an effort to prevent recurrence of depression. After the follow-up assessment, the therapist works with the family and adolescent or the adolescent alone to determine which of the six booster protocols (pleasant events, social skills and communication, relaxation, cognitions, negotiation and problem solving, and maintaining gains and setting goals) would be most appropriate. The booster sessions focus on ways that specific CWD-A skills might be used to cope with the specific problems that the adolescent faces. Staffing

CWD-A leaders undergo 60 hours of training and have a minimum of a master's degree in a mental health field. Leaders use a detailed therapist's manual to help with program implementation.

Issues to Consider

This program received a "proven" rating. All seven studies of the Adolescent Coping with Depression Course employed rigorous evaluation methodology, including randomized assignment, and found significant reductions in interviewer-rated, parent-rated, and self-reported depression symptoms for treatment children when compared with those in a control group or alternative treatment group. The studies by Rohde, Lewinsohn, and Seeley (1994) and Clarke et al. (1999) found no significant benefit of the inclusion of parent sessions to the intervention. These findings were not entirely consistent, as Lewinsohn et al. (1990) reported that at immediate posttest the parent condition reported significantly lower problem scores for CBCL scales than did the adolescent-only condition. However, these differences disappeared by the six-month follow-up. In addition, the study by Rohde, Lewinsohn, and Seeley (1994) suggests that the effectiveness of CWD-A may be apparent only for seriously depressed adolescents, as no program effects were found for adolescents with a low severity of depression symptoms at baseline. No strong conclusions can be drawn at this point, as this was the only study that compared program effectiveness for adolescents with a low versus high severity of depression symptoms. The effects of the addition of booster sessions were assessed by Clarke et al. (1999), who found that parents in the booster condition reported greater reductions in CBCL externalizing symptoms than did parents in assessment-only conditions. Again, since only one study to date has assessed the effect of the booster sessions, and no statistically significant differences were found among groups, no strong conclusions regarding booster-session effectiveness can be made. Another study suggests that psychiatric comorbidity (i.e., the presence of other mental health problems in addition to depression) is generally not a reason to avoid the use of CWD-A for depressed adolescents (Rohde et al., 2001). The authors found that participants who also had anxiety disorders had higher depression measure scores at intake but a greater decrease in scores by posttest. Overall

- 50 -

lifetime comorbidity was unrelated to diagnostic recovery from depression, but lifetime substance abuse/dependence was associated with a slower time to recovery. Study participants with attention- deficit and disruptive behavior disorders were more likely to experience depression recurrence post CWD-A treatment. The Clarke et al. (2009) study also found that among participants with a parent who was currently depressed, CWD-A was not more efficacious than usual care in preventing depressive episodes. Finally, it should be noted that the program developers served as authors on all but one of the evaluations of CWD-A.

Example Sites

Multiple major U.S. cities

Contact Information

Gregory N. Clarke, Ph.D. Kaiser Permanente Center for Health Research 3800 N. Kaiser Center Dr. Portland, OR 97227 phone: (503) 335-6673 greg.clarke@kpchr.org

Available Resources

The Kaiser Permanente Center for Health Research provides free downloads of the CWD-A therapist manual, the CWD-A teen workbook, the CWD-A parent manual, and the CWD-A parent workbook at http://www.kpchr.org/public/acwd/acwd.html.

Bibliography

Clarke, Gregory N., Mark Hornbrook, Frances Lynch, Michael Polen, John Gale, et.al., "Group Cognitive-Behavioral Treatment for Depressed Adolescent Offspring of Depressed Parents in a Health Maintenance Organization," Journal of the American Academy of Child and Adolescent Psychiatry, Vol. 41, No. 3, 2002, pp. 305-313. Clarke, Gregory N., Paul Rohde, Peter M. Lewinsohn, Hyman Hops, and John R. Seeley, "Cognitive- Behavioral Treatment of Adolescent Depression: Efficacy of Acute Group Treatment and Booster Sessions," Journal of the American Academy of Child and Adolescent Psychiatry, Vol. 38, No. 3, 1999, pp. 272-279. Garber J., Gregory N. Clarke, V. Robin Weersing, William R. Beardslee, David A. Brent, et.al., "Prevention of Depression in At-Risk Adolescents: A Randomized Controlled Trial," Journal of the American Medical Association, Vol. 301, No. 21, June 3, 2009, pp. 2215-2224. Kahn, James S., Thomas J. Kehle, William R. Jenson, and Elaine Clark, "Comparison of Cognitive- Behavioral, Relaxation, and Self-Modeling Interventions for Depression Among Middle-School Students," School Psychology Review, Vol. 19, No. 2, 1990, pp. 196-211. Lewinsohn, Peter M., Gregory N. Clarke, Hyman Hops, and Judy Andrews, "Cognitive-Behavioral Treatment for Depressed Adolescents," Behavior Therapy, Vol. 21, 1990, pp. 385-401. Rohde, Paul, Gregory N. Clarke, David E. Mace, Jenel S. Jorgensen, and John R. Seeley, "An Efficacy/Effectiveness Study of Cognitive-Behavioral Treatment for Adolescents with Comorbid Major Depression and Conduct Disorder," Journal of the American Academy of Child and Adolescent

- 51 -

mailto:greg.clarke@kpchr.org

http://www.kpchr.org/public/acwd/acwd.html

Psychiatry, Vol. 43, No. 6, 2004, pp. 660-668. Rohde, Paul, Gregory N. Clarke, Peter M. Lewinsohn, John R. Seeley, and Noah K. Kaufman, "Impact of Comorbidity on a Cognitive-Behavioral Group Treatment for Adolescent Depression," Journal of the American Academy of Child and Adolescent Psychiatry, Vol. 40, No. 7, 2001, pp. 795-802. Rohde, Paul, Peter M. Lewinsohn, and John R. Seeley, "Response of Depressed Adolescents to Cognitive-Behavioral Treatment: Do Differences in Initial Severity Clarify the Comparison of Treatments?" Journal of Consulting and Clinical Psychology, Vol. 62, No. 4, 1994, pp. 851-854. Rohde, Paul, Peter M. Lewinsohn, Gregory N. Clarke, Hyman Hops, and John R. Seeley, "The Adolescent Coping with Depression Course: A Cognitive-Behavioral Approach to the Treatment of Adolescent Depression," in Euthymia D. Hibbs and Peter S. Jensen, eds., Psychosocial Treatments for Child and Adolescent Disorders: Empirically Based Strategies for Clinical Practice, 2nd ed., Washington, D.C.: American Psychological Association, 2005, pp. 219-237.

Last Reviewed

March 2013

Athletes Training and Learning to Avoid Steroids (ATLAS)

Program Info

Outcome Areas Healthy and Safe Children

Indicators Youths not using alcohol, tobacco, or illegal drugs

Topic Areas Age of Child Adolescence (13-18) Type of Setting High School Type of Service Health Education Youth Development Type of Outcome Addressed Mental Health Physical Health Substance Use and Dependence

Evidence Level Promising

Program Overview

Athletes Training and Learning to Avoid Steroids (ATLAS) is a drug prevention and health promotion program designed for male high school athletes. The program, which began in 1994, aims to reduce risk factors for steroid use, alcohol, and other drugs, and to promote healthy nutrition and exercise behaviors among youth. The ATLAS program addresses a variety of potential risk factors related to steroid use and substance abuse. These risk factors include social influences (such as peers, coaches, media), the perceived positive benefits of steroid use, misperceptions about the actual extent of

- 52 -

steroid use among peers, lack of information about the negative effects of steroid use, and beliefs that one is not vulnerable to adverse effects. The ten-week program initially was used with high school football players and currently is used with male athletes participating in a variety of sports. Within a team-based setting, coaching staff and peer leaders present the ATLAS curriculum using a hands-on, interactive approach. Program components include classroom-based educational sessions, weight-room exercise sessions, and an evening session for parents. Since ATLAS first was used, annual "booster" sessions have been added to help maintain early program gains and address risk factors not changed during the program's first year.

Program Participants

The ATLAS program initially was used and studied with male football players at 31 high schools in 12 cities in Washington and Oregon. The average age of these students was 15 to 16 years. Most were from middle-class backgrounds. Approximately 80 percent of study participants were white, with the remainder from African-American, Hispanic, Asian, and other ethnic backgrounds.

Evaluation Methods

Thirty-one high school football teams composed of more than 3,200 athletes were studied in three successive annual cohorts (1994–1996: Year 1, Year2, and Year 3). These schools were assigned randomly either to participate in the program or to not participate, after first being matched on several demographic characteristics, including the following: school size, socio-economic status, participation in a free-lunch program, win-loss record of the school’s football team, school-wide attendance, and number of students attending college. Non-participants tended to come from families with somewhat higher incomes and educational levels; however, no analysis was done to discover why this might be the case. Each of the 2,516 football players across the three cohorts was assigned to one of the two groups (the decrease from the original 3,200 students who were enrolled is due to regular, anticipated attrition throughout the football season): 1,145 attended schools assigned to receive the ATLAS program, and 1,371 students attended schools assigned to the comparison group and received only an anti-steroid informational pamphlet. Year 1 included players from Grades 9 through 12; Years 2 and 3 included players from all these grades but consisted primarily of ninth and tenth graders. Participants in both the program schools and comparison schools completed self-report questionnaires at three points during the study: just prior to the first program session, immediately after the last program session (about ten weeks later), and approximately nine to ten months following program completion (one-year follow-up data is available for Year 1 and Year 2 cohorts and results are combined in the long-term follow-up assessment). Questionnaires assessed steroid, alcohol, and other drug use, risk factors for steroid and other substance use, knowledge/attitudes concerning steroids, sports nutrition and exercise knowledge/behaviors, and intentions to use steroids. Of the original 2,516 participants assessed during the preseason, 78.5 percent were present at the seasons’ end.

Key Evaluation Findings

All phases of the evaluation conducted by Goldberg, et al. (2000) found that compared with non- participants immediately after program participation and nine to ten months later, ATLAS participants:

• Reported significantly less intent to use steroids.

• Reported significantly improved sports nutrition behaviors and significantly greater use of school facilities rather than private gyms – significant, in that young athletes have been shown to have greater vulnerability to influences that encourage usage of steroids and other performance-enhancement products within the private-gym context.

• Demonstrated greater knowledge about the consequences of steroid and alcohol use, nutritional supplements, and exercise.

- 53 -

• Were significantly more likely to believe that their coaches were less tolerant of steroid use and had significantly greater belief in their team and peers as trusted sources of information about drugs.

• Were significantly more skeptical of media advertisements for muscle-building products.

• Had significantly greater feelings of athletic competence, ability to strength train, and ability to refuse steroids and other offers of drugs, and had increased self-esteem.

• Were significantly more likely to believe in the potential risks associated with steroid use and felt more personally vulnerable to negative effects of steroid use.

• Among those athletes who had never used alcohol or drugs, ATLAS participants were more likely to remain alcohol and drug-free at one year after completion of the program.

Compared with non-participants immediately after, and nine to ten months after, program participation, Goldberg, et al. (1996a) found that ATLAS participants reported:

• No significant decrease in actual steroid usage.

Probable Implementers

Male high school and community athletic teams.

Funding

The ATLAS program and its evaluation were funded by the National Institute on Drug Abuse.

Implementation Detail

Program Design

• The ATLAS program takes place in a team setting with peers who share common goals and with coaches who have significant contact time and influence with students.

• The program is implemented during the football season (or other sports season) because this is the period during which students tend to be at greatest risk for steroid usage.

• Periodic "booster" sessions help to make the information offered in the intervention a more- integrated part of the participants’ knowledge base and behavior.

• Parents are involved in prevention efforts.

Curriculum

The ATLAS program is composed of a three-pronged curriculum consisting of classroom-based education, weight-room skill training, and an informational session for parents.

• Classroom Education — Led by peer leaders and coaching staff, a series of ten 45-minute educational sessions is presented that addresses sports nutrition, strength training, and the effects of steroid use and associated risk factors. Participants practice skills for refusing steroid offers by taking part in role-playing exercises, and reviewing bodybuilding magazines to learn about media portrayals of steroid use and potential negative side effects of using steroids and other performance-enhancement products.

• Weight Room Exercises — Participants are provided with printed guides that focus on sports nutrition guidelines and recommendations, and a weight-training booklet that describes strength-training techniques and flexible workout plans designed to be incorporated into any team’s exercise regime. In addition, the importance of working out as a team is emphasized and group-training sessions are implemented. (Note: In the original model of the program,

- 54 -

weight-room sessions were presented at school gyms that were designed to help reinforce the classroom curriculum and demonstrate weight-lifting techniques.)

• Parental Involvement - Parents and guardians of participating students receive a family sports nutrition booklet, which contains information about food shopping and sports nutrition menus. Parents are asked to participate in a voluntary evening informational session that focuses on program goals and content and includes a question-and-answer session.

Staffing

In the original program presentation, coaching staff received a one-day in-service training and curriculum guide with specific lesson plans. Program staff trained peer leaders during a four- to six- hour training session. Each coach and peer leader received curriculum manuals, lesson plans, and activities to use in the classroom. Today, coaches typically are trained using an instructor's manual included in the curriculum package. The package also includes materials for training peer leaders.

Issues to Consider

This program received a "promising" rating. The program was evaluated according to rigorous standards, and included an experimental-control design and a large sample size. The study team was able to follow up with 78.8 percent of the original study population. The evaluations yielded strong, quantifiable, and scientifically credible evidence of a reduction of risk factors associated with steroid usage. The program significantly reduced alcohol and other illicit drug use, and driving under the influence. However, the program was not able to show significant and sizeable changes in the actual use of steroids. While there was evidence of a change in cumulative steroid use in the desired direction, the change was not statistically significant and the effect size was small. This may be related to the fact that an extremely low percentage of young men used steroids at the start of the program/study. Of the total study population, only 1 to 3 percent of the population indicated any steroid usage at baseline, and less than 1 percent initiated steroid use in the follow-up period. Even if the program eradicated a large portion of this usage, it would be unlikely that such a small change would be statistically significant given the sample sizes in the study. As a result, although the program was quite effective in having an impact on attitudes and risk factors, the evaluation was not able to demonstrate how or if these changes are translated into an actual shift in steroid use. ATLAS researchers extended their study by examining how treatment outcomes varied by baseline characteristics (Fritz et al., 2005), and found that those participants that knew less about the effects of steroid use were more likely to benefit from the intervention, that is, to learn more about the harmful effects of steroids. Likewise, those participants with higher intentions to use steroids were more likely to see a decrease in their intentions to use steroids following the intervention. In addition to the identified benchmark, the ATLAS program has been shown to generate improvements in participants’ self-esteem, peer relationships, and general health practices. These accomplishments are as much a focus and benefit of the intervention as is steroid use determent.

Example Sites

Portland, OR, and Vancouver, WA

Contact Information

For more information about the ATLAS program and its research, contact: Linn Goldberg, M.D., FACSM ATLAS Anabolic Steroid Prevention Program Oregon Health Sciences University Mail Code: CR-110

- 55 -

3181 SW Sam Jackson Park Rd. Portland, OR 97201-3098 phone: 503-494-6559 fax: 503-494-1310 e-mail: goldberl@ohsu.edu

Available Resources

A curriculum package is available from Sunburst Communications, Inc. This package consists of an instructor's manual with background information, a 90-minute session for training peer leaders, and the ten-session curriculum. An athlete's pack also is available and includes three booklets: a workbook for the ten program sessions, a training guide, and a sports menu. To order the ATLAS curriculum, contact: Sunburst Communications, Inc. 101 Castleton Street Pleasantville, NY 10570 Phone: 1-800-321-7511 Fax: 1-914-747-4109 Email: service@nysunburst.com

Bibliography

Goldberg, Linn, David McKinnon, Diane Elliot, et al., The Adolescents Training and Learning to Avoid Steroids Program: Preventing Drug Use and Promoting Health Behaviors Archives of Pediatric and Adolescent Medicine, 2000., Vol. 154, pp. 332-338. Goldberg, Linn, Diane Elliot, Greg Clarke, et al., Effects of a Multidimensional Anabolic Steroid Prevention Intervention: The Adolescent Training and Learning to Avoid Steroids (ATLAS) Program Journal of the American Medical Association, 1996a., Vol. 276, pp. 1555-1563. Goldberg, Linn, Diane Elliott, Greg Clarke, et al., The Adolescents Training and Learning to Avoid Steroids (ATLAS) Prevention Program: Background and Results of a Model Intervention Archives of Pediatric and Adolescent Medicine, 1996b., Vol. 150, pp. 713-721.

Last Reviewed

May 2009

Attachment-Based Family Therapy

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children not experiencing anxiety or mood disorders, such as depression

- 56 -

mailto:goldberl@ohsu.edu

mailto:service@nysunburst.com

Topic Areas Age of Child Adolescence (13-18) Type of Setting Health Care Provider Type of Service Family Support Health Care Services Youth Development Type of Outcome Addressed Mental Health

Evidence Level Proven

Program Overview

Attachment-Based Family Therapy (ABFT) is based on the belief that strong relationships within families can buffer against the risk of adolescent depression or suicide and help in the recovery process. ABFT is a psychotherapeutic model, with a foundation in attachment theory. Attachment theory posits that when parents are responsive and protective, children develop a healthy sense of self, trust in others, and better capacity for independence and affect regulation. Ruptures in attachment security can increase the risk for psychopathology. However, as a life-span developmental model, attachment theory posits that attachment ruptures are reparable, and thus children can regain the external and internal resources to promote healthy development. The ABFT model aims to strengthen or rebuild secure parent-child relationships and promote adolescent autonomy. To accomplish this, the therapist helps the family agree to focus on relationship repair as the initial goal of therapy. Then, with the adolescent alone, the therapist helps the adolescent identify perceived attachment ruptures or negative family processes and prepares the adolescent to talk about these problems with his or her parents. In separate sessions with parents, the therapist focuses on reducing parental distress and improving parenting practices. Exploring their own history of attachment rupture helps parents understand their own attachment wounds and builds empathy for the adolescent. When ready, conjoint sessions focus on helping the family successfully discuss these past problems. This process both helps resolve actual problems in the family and allows parents and adolescents to practice new skills related to affect regulation and interpersonal problem solving. As trust begins to reemerge, therapy focuses on promoting adolescent competency outside the home.

Program Participants

Depressed adolescents and their families

Evaluation Methods

Two trials have examined the impact of ABFT. The first study, published in 2002, was conducted among adolescents referred to treatment by their schools or their parents who met the Diagnostic and Statistical Manual III—Revised (DSM-IIIR) criteria for Major Depressive Disorder (MDD). The adolescents were between the ages of 13 and 17, with a mean age of 14.9 years old. Eighty percent of the adolescents in the trial came from single-parent homes, and 69 percent came from households with annual incomes of less than $30,000. Forty-seven percent of adolescents had heard random gunshots in the past six months, 31 percent had family members who used drugs or alcohol, and 19 percent had had unwanted sexual experiences. To enter into the study, adolescents and their parents were required to be willing to participate, and adolescents had to meet the DSM criteria for MDD and not have psychotic symptoms (Diamond et al., 2002).

- 57 -

Eligible adolescents and their families were randomly assigned to the 12-week ABFT intervention (16 participants) or a six-week waitlist control (16 participants). The groups were not statistically significantly different in demographics or baseline measures of depression. Adolescents who were assigned to the waitlist were given weekly 15-minute phone calls restricted to monitoring for deterioration in depressive symptoms. Adolescents on the waitlist were entered into ABFT six weeks after the commencement of the treatment, because it was deemed unethical to withhold treatment from this group for longer than six weeks. ABFT, however, is a 12-week program; thus, it is only possible to test for differences in the outcomes for the waitlist control group with those of the ABFT group at six weeks, one-half of the full duration of ABFT. The authors report the ABFT 12-week outcomes for both groups as well.

The Diamond et al. (2002) study examined the following outcomes:

• Depression, as measured by the Beck Depression Inventory (BDI), a self-reported tool

• Family Functioning, as measured by the Self-Report of Family Functioning, a self-reported measure collected among the adolescents themselves, not among their parents or other family members.

The second Diamond et al. study, published in 2010, examined the effects of ABFT among suicidal adolescents between the ages of 12 and 17. Participants were identified in primary care and emergency departments. Referred adolescents were not eligible for participation if they needed psychiatric hospitalization, were recently discharged from a psychiatric hospital, had current psychosis, or had mental retardation or a history of borderline intellectual functioning. Of 341 adolescents screened, 66 entered the study. Of the 66 who entered the study, 35 were randomly assigned to ABFT and 31 were assigned to the control group, in which they received "enhanced usual care," including a referral to other providers for treatment and weekly check-in phone calls monitoring for deteriorating depression and increased suicidal ideation. Among the 31 adolescents receiving "enhanced usual care" instead of ABFT, 52 percent were referred to and received individual therapy, 19 percent received group therapy, 6 percent received family therapy, 3 percent received case management, and 19 percent did not attend any treatment. The treatment and control groups were not statistically significantly different in demographics or clinical variables prior to treatment. The Diamond et al. (2010) study examined the following outcomes:

• Suicidal ideation, as measured by the Suicidal Ideation Questionnaire, Junior High version (SIQ-JR), a self-reported measure, and the Scale for Suicidal Ideation (SSI), an assessor- administered tool

• Depression, as measured by the Beck Depression Inventory (BDI)

Key Evaluation Findings

In Diamond et al. (2002), adolescents undergoing six weeks of ABFT had a lower prevalence of clinically significant depression (56 percent), defined as a score on the BDI of greater than 9, than did their control group counterparts (19 percent). There were no significant effects of six weeks of the program on adolescent self-report of family functioning as measured by the Self-Report of Family Functioning. Note that the observed effects only demonstrate the impact of ABFT at mid-treatment, prior to the control group entering treatment. The 2010 trial conducted among suicidal adolescents (Diamond et al., 2010) found that ABFT participants improved on several measures of suicidality compared with the control group:

• The ABFT group showed greater improvement on self-reported suicidal ideation (SIQ-R) at the conclusion of the program and at the follow-up administered Scale for Suicidal Ideation (SSI) at 12 weeks after the program's conclusion. At the treatment's conclusion, 87 percent of ABFT participants and 51 percent of control-group participants reported suicidal ideation in the

- 58 -

normal range, and at the follow-up 12 weeks post-treatment, these numbers were 70 percent and 34 percent, respectively.

• Clinical recovery on an assessor-administered suicidal ideation scale (SSI) was greater in the ABFT group at the conclusion of the program and at follow-up. In particular, at the program's conclusion, 69 percent of ABFT participants and 34 percent of control-group participants reported no suicidal ideation in the past week. Twelve weeks after the program's conclusion, these gains continued, with 82 percent of ABFT participants and 46 percent of control group participants reporting no suicidal ideation in the past week.

• Four ABFT participants and seven control group participants made low-lethality suicide attempts during the course of treatment, but these numbers are too small for statistical significance analyses.

In addition to the above measures related to suicidality, fewer ABFT participants had clinically significant depression at the end of treatment (34 percent versus 11 percent). This improvement in the ABFT group relative to the control group was only marginally significant, however, and statistically significant improvements on clinically significant depression did not persist into the 12-week post- treatment follow-up.

Probable Implementers

Therapists working with adolescents at risk for depression or suicide

Funding

Grants for evidence-based mental health treatment may be available from federal and local funding sources, including the Substance Abuse and Mental Health Services Administration (www.samhsa.gov) and the National Institute of Mental Health (www.nimh.gov).

Implementation Detail

Program Design

ABFT treatment has five specific tasks, each of which takes from one to three sessions to accomplish:

1. Relational reframe task: Refocuses the family away from "fixing" the patient toward building better relationships within the family.

2. Adolescent alliance-building task: Focuses on building a bond between therapist and adolescent, identifying core family dynamics that inhibit the parent-child relationship, and encouraging youth to discuss these issues.

3. Parent alliance-building task: Focuses on reducing parental distress and improving parenting practices. It begins with an exploration of the parents' stressors and their own history of attachment failures, with the goal of improving empathy toward the adolescent and making parents more receptive to coaching.

4. Reattachment task: Adolescents explore past and present experiences, thoughts, and feelings that have violated the attachment bond and damaged trust with caregivers. Parents are coached to provide an empathic, supportive stance so that adolescents can explore and express cognitive, emotional responses to these events. This corrective attachment experience helps adolescents begin to reevaluate their internal working models of self and other and promotes change in the interactions between family members.

5. Competency-promoting task: Family members are encouraged to continue practicing improved communication skills while fostering the adolescent's success and connections outside of the home. This helps promote appropriate adolescent autonomy while maintaining the parent-child connection.

- 59 -

http://www.samhsa.gov/

http://www.nimh.gov

Staffing

ABFT can be delivered by therapists who have the minimum qualifications of a master's degree in social work or equivalent. Certification in ABFT is offered by staff of the Children's Hospital of Philadelphia, either on-site at the hospital or off-site at the agencies themselves. Certification generally occurs over a two-year period and includes two three-day workshops and bi-weekly 90- minute group case consultation calls, video supervision, and individual feedback. Therapists must meet fidelity standards to be credentialed. When working with an agency, at least two therapists and a clinical supervisor need to participate. Therapists or agencies interested in discussing credentialing procedures and locations for certification should contact Dr. Guy Diamond or Dr. Suzanne Levy (see contact information). Curriculum

ABFT has a treatment manual that provides a roadmap to the treatment tasks documented above.

Issues to Consider

This program received a "Proven" rating. Both of the evaluations cited used a randomized controlled trial design, which is considered the "gold standard" evaluation methodology. The evaluation published in 2002 included only 16 adolescents in each of the control and treatment groups, which does not meet the PPN "Proven" criteria, but the evaluation published in 2010, which included more than 30 adolescents in each group, does meet the standard. The cited evaluations were both lead-authored by the ABFT developer. Both of the studies cited in this summary were conducted among a specific subset of the population of depressed adolescents. Diamond et al. (2002) was conducted among depressed adolescents at higher risk than the general population based on socioeconomic status. Diamond et al. (2010) was conducted among suicidal adolescents, who experience more severe depression than the general population of depressed adolescents. Thus, the findings of these studies must not be assumed to be generalizable to the population of depressed adolescents at large.

Example Sites

ABFT has been implemented by therapists in a number of agencies nationwide and internationally.

Contact Information

Guy Diamond Ph.D. Director, The Center for Family Intervention Science Email: diamondg@email.chop.edu The Children's Hospital of Philadelphia 3535 Market St. Suite 1230 Philadelphia, PA 19104 215-590-7550 Suzanne Levy Ph.D. Training Director, The Center for Family Intervention Science Email: levysu@email.chop.edu The Children's Hospital of Philadelphia 3535 Market St. Suite 1230 Philadelphia, PA 19104 215-590-7570

- 60 -

mailto:diamondg@email.chop.edu

mailto:levysu@email.chop.edu

Available Resources

For more information on how to implement ABFT, visit: http://www.research.chop.edu/programs/cfis/abft.php

Bibliography

Guy S. Diamond, Brendali F. Reis, Gary M. Diamond, Lynne Siqueland, and Lisa Isaacs, "Attachment- Based Family Therapy for Depressed Adolescents: A Treatment Development Study," Journal of the American Academy of Child and Adolescent Psychiatry, Vol. 41, No. 10, October 2002, pp. 1190-1196. Guy S. Diamond, Lynne Siqueland, and Gary M. Diamond, "Attachment-Based Family Therapy for Depressed Adolescents: Programmatic Treatment Development," Clinical Child and Family Psychology Review, Vol. 6, No. 2, June 2003, pp. 107-127. Guy S. Diamond, Matthew B. Wintersteen, Gregory K. Brown, Gary M. Diamond, Robert Gallop, Karni Shelef, and Suzanne Levy, "Attachment-Based Family Therapy for Adolescents with Suicidal Ideation: A Randomized Controlled Trial," Journal of the American Academy of Child and Adolescent Psychiatry, Vol. 49, No. 2, February 2010, pp. 122-131.

Last Reviewed

July 2011

Be Proud! Be Responsible!

Program Info

Outcome Areas Healthy and Safe Children

Indicators Youths abstaining from sexual activity or not engaging in risky sexual behavior

Topic Areas Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Middle School High School Community-Based Service Provider Health Care Provider Type of Service Health Education Youth Development Type of Outcome Addressed Physical Health Teen Sex/Pregnancy

Evidence Level Promising

- 61 -

http://www.research.chop.edu/programs/cfis/abft.php

Program Overview

Be Proud! Be Responsible! (BPBR, alternatively called Be Proud!) is a five-hour sex-decisionmaking intervention, made up of six 50-minute modules, typically delivered over the course of one day. The goal of the curriculum is to help adolescents change behaviors that put them at risk of transmitting HIV and other sexually transmitted diseases (STDs). To accomplish this, the curriculum is intended to delay the initiation of sex, reduce the frequency of unprotected sex, and support young people in their making responsible decisions about their own sexual behaviors. The intervention covers the ways in which STDs, including HIV/AIDS, are spread. It also covers condom use and the risks posed by unprotected sexual intercourse, multiple partners, vaginal intercourse, and anal intercourse (regardless of gender). The curriculum aims to increase participants' knowledge about the risks of various behaviors and educate participants on how condoms can effectively be used to lower those risks. The BPBR curriculum also targets beliefs about the negative consequences of condom use on sexual enjoyment and works to build decisionmaking and interpersonal negotiation skills to increase the frequency of condom use. The curriculum also attempts to improve self-efficacy regarding condom use—that is, increasing adolescents' belief in the effectiveness of condom use, their ability to use a condom correctly, and their ability to convince a sexual partner to agree to use a condom when he or she originally does not want to use one. Each module incorporates a theme that encourages the participants to be proud of themselves and their community, to behave responsibly for the sake of themselves and their community, and to consider their goals for the future and how unhealthful behavior might impede those goals. The curriculum involves group discussions, videos, games, brainstorming, experiential exercises, and skill- building activities. Participants in the program work in groups of six to eight and are led by a trained facilitator. Be Proud! Be Responsible! is also an umbrella term for a collection of three curricula: the original BPBR curriculum described above, Making Proud Choices! (an eight-hour, multi-module, safer-sex intervention), and Making a Difference! (an eight-hour, multi-module, abstinence-based intervention). Each curriculum is a distinct, stand-alone unit; they need not be implemented in succession. In this summary, we use the term Be Proud! Be Responsible! to refer to the six-module safer-sex intervention only, and not the other curricula. Please see the PPN summary of Making Proud Choices! for more information on that curriculum.

Program Participants

Adolescents in a school setting.

Evaluation Methods

Four evaluations of the BPBR curriculum meet the PPN evidence criteria. The first study (Jemmott, Jemmott, and Fong, 1992) looked at the impact of the one-day, five-hour BPBR curriculum on a male African-American adolescent group immediately following the completion of the intervention and at a three-month follow-up. The intervention was conducted in Philadelphia; 157 participants were recruited from among outpatients at a medical clinic (44 percent), from among students attending 10th-, 11th-, and 12th-grade assemblies at a local high school (32 percent), and from a local YMCA (24 percent). Participation in the program was voluntary. Approximately 97 percent of the participants were currently enrolled in school. The mean age of the participants was 14.6 years. The mean number of years of education of their mothers was 13.8 (as reported by the participants). The chief HIV risk to the population was from heterosexual intercourse; 33.9 percent of the participants reported having multiple partners within the previous three months, and 12.8 percent indicated that they had had heterosexual anal intercourse during that period; 20.9 percent of the respondents who had had any type of sexual intercourse within the previous three months reported that they never used a condom during that period. These data were collected via a self-reporting pre- intervention questionnaire.

- 62 -

Participants were grouped by age and then randomly assigned within age groups to either an experimental or control group. The experimental group received the BPBR intervention. The control group received a presentation that focused on career opportunities and did not include any information on safer-sex practices or HIV prevention. The experimental and control groups were each further subdivided into small groups (14 small groups in the BPBR intervention and 13 small groups in the control condition), each led by a trained facilitator. Eighty-five teens were assigned to the BPBR intervention, and 72 teens were assigned to the control group. Participants completed post- intervention questionnaires immediately following the completion of the intervention. Of the original 157 participants, 96 percent of the total population completed the follow-up. Participants were assessed on their levels of risky sexual behavior during the previous three months, their intentions toward and attitudes regarding risky sexual behavior for the next three months, and their knowledge of HIV/AIDS and STDs. These data were collected via a self-reporting questionnaire. A second study (Jemmott et al., 1999) evaluated a replication of the BPBR curriculum in Trenton, New Jersey. In this evaluation, 496 African American males and females were randomly assigned to receive the BPBR intervention or a general health promotion presentation that focused on behaviors associated with risk of heart disease, stroke, hypertension, and certain cancers. Participants who received the general health promotion presentation made up the control group. The participants were recruited from public schools via announcement by project staff during 7th- and 8th-grade assemblies and lunch periods. Participation in the program was voluntary. The mean age of participants was 13.2 years; 53.8 percent of the population was female, and 100 percent of the population was African- American. Prior to the intervention, 55.3 percent of the participants reported having had vaginal intercourse at least once; 30.2 percent of the participants reported having had vaginal intercourse within the previous three months; 17.7 percent of the participants reported ever having had anal intercourse; and 8.3 percent reported having had anal intercourse in the previous three months. As in the 1992 study, participants in the 1999 study were grouped by age and gender and randomly assigned to treatment conditions and small groups. A total of 269 teens were assigned to the BPBR intervention and 227 teens were assigned to the control group. Measurement procedures were identical to those used in the 1992 study; however, unlike in the 1992 study, the 1999 study included both three- and six-month follow-ups; 96.8 percent of the original sample completed the three-month follow-up, and 92.7 percent completed the six-month follow-up. The third study (Borawski et al., 2009) examined the effects of the BPBR curriculum implemented in five Midwestern schools compared with five matched schools. Schools were matched on location (inner suburb, outer suburb, inner city), percentage of families below the Federal Poverty Line, and the racial composition of student body. Matched pairs of schools were then assigned to the Be Proud! or control conditions based on a coin flip. The intervention participants were 631 9th- and 10th-grade students from five schools, and the control participants included 726 9th- and 10th-grade students from five schools. There were significant differences in demographic features between the intervention and control groups at baseline, which were controlled for in analyses. These differences included the following:

• Treatment group was more likely to be female (55 versus 48 percent).

• Treatment group was more likely to be Hispanic (17 versus 7 percent).

• Treatment group had more neighborhood households in poverty (16 versus 14 percent).

There were no differences in history of sexual intercourse or condom use at baseline. Participants in both groups completed baseline (pre-intervention) questionnaires, questionnaires immediately following completion of the program, and questionnaires at 4- and 12-month follow-ups. Questionnaires assessed the following: (1) knowledge of proper condom use and HIV and other STDs, (2) efficacy, including impulse control, condom negotiation skills, and condom technical skills, (3) beliefs about condom use, including utility, pleasure when using a condom, and the fact that condoms prevent STDs, (4) perceived peer beliefs about the acceptability of sexual activity and condom use, (5) intentions of having sex in the next three or 12 months, and (6) ever had vaginal intercourse and frequency of vaginal intercourse. Clustering of outcomes at the school level was accounted for in analysis.

- 63 -

A final study (Jemmott et al., 2010) examined the implementation of BPBRs among community-based organizations (CBOs) in New Jersey and Philadelphia. Eighty-six CBOs were eligible and agreed to participate in the trial. Selected CBOs served predominately African-American adolescents ages 13-18. Half of participating CBOs were assigned to implement six intervention groups in which they were to deliver the BPBR curriculum. The other half implemented at least six intervention groups involving general health promotion. A total of 1,707 youth were involved across all CBOs, 90 percent of whom were African-American, with a mean age of 14.8 years. Surveys were administered at baseline; immediately after the intervention; and three, six, and 12 months after the intervention. The study assessed self-reported condom use (consistent use, frequency, at last sexual encounter) and frequency of intercourse. Study authors controlled for clustering within CBOs in their analyses for the 12-month follow-up.

Key Evaluation Findings

Jemmott, Jemmott, and Fong (1992) found the following at the three-month follow-up (adjusting for differences in the groups at baseline):

• Total days of sexual intercourse in past three months was significantly lower in the treatment group than the control group (2.15 versus 5.48 days), as was the percentage of treatment group participants having any sexual intercourse in the past three months (48 versus 60 percent).

• Total number of sexual partners in the past three months was significantly lower in the treatment group than the control group (0.85 versus 1.79 partners).

• Frequency of condom use was higher in the treatment group than the control group (4.35 versus 3.50, on a five-point scale).

Jemmott et al. (1999) found the following:

• At the three-month follow-up:

o Participants who received the intervention scored higher on HIV risk-reduction knowledge, expressed more favorable prevention beliefs and hedonistic beliefs, and expressed greater self-efficacy and stronger condom use intentions than the participants in the control group.

o There was no significant difference across groups on subjective norms regarding condoms.

o There were no significant differences on any of the risky sexual behaviors measured.

• At the six-month follow-up:

o BPBR participants scored lower on HIV risk-associated sexual behaviors, higher on HIV risk-reduction knowledge, and expressed more favorable hedonistic beliefs, greater self- efficacy, and greater intentions to use condoms than participants in the control group.

o Treatment participants reported a lower frequency of unprotected sexual intercourse than those in the control group (47 versus 70 percent of sexual acts unprotected).

o The percentage of adolescents who reported engaging in any anal intercourse in the previous three months was significantly lower in the BPBR group than in the control group (3 versus 10 percent).

o There were no significant differences between treatment and control groups in prevention beliefs.

o There was no significant difference across groups on subjective norms regarding condoms.

o There were no significant differences between groups regarding their beliefs that condoms can prevent pregnancy and STDs, including HIV/AIDS.

- 64 -

Borawski et al. (2009) found the following:

• Immediately following the intervention:

o Knowledge of proper condom use was significantly higher in the treatment group than in the control group (4.06 versus 2.83 on a seven-point scale).

o Knowledge of HIV and other STDs was significantly higher in the treatment group than in the control group (5.24 versus 4.81 on seven-point scale).

o Impulse control was significantly higher in the treatment group (3.95 versus 3.84 on a five-point scale).

o Condom negotiation skills were significantly higher in the treatment group (4.13 versus 3.98 on a five-point scale).

o Condom technical skills were significantly higher in the treatment group (4.30 versus 3.99 on a five-point scale).

o Beliefs in condom use were significantly higher in the treatment group (3.42 versus 3.16 on a five-point scale).

o Intentions to use a condom were significantly higher in the treatment group (3.42 versus 3.16 on a five-point scale).

• At the four-month follow-up:

o Knowledge of proper condom use was significantly higher in the treatment group than in the control group (4.14 versus 3.44 on a seven-point scale).

o Knowledge of HIV and other STDs was significantly higher in the treatment group than in the control group (5.07 versus 4.88 on a seven-point scale).

o Condom negotiation skills were significantly higher in the treatment group (4.05 versus 3.93 on a five-point scale).

o Condom technical skills were significantly higher in the treatment group (4.23 versus 4.06 on a five-point scale).

o Beliefs in condom use were significantly higher in the treatment group (4.68 versus 4.59 on a five-point scale).

o Intentions to have sex were significantly lower in the treatment group (3.08 versus 3.20 on a five-point scale).

• At the 12-month follow-up:

o Knowledge of proper condom use was significantly higher in the treatment group (4.18 versus 3.64 on a seven-point scale).

o Knowledge of HIV and other STDs was significantly higher in the treatment group (5.13 versus 4.96 on a seven-point scale).

o Impulse control was significantly higher in the treatment group (3.99 versus 3.87)

o Perception of peer acceptance of sexual activity was lower in the treatment group (2.97 versus 3.11 on a five-point scale).

• Sexual behaviors (sexual initiation, frequency of intercourse, and condom use) were not significantly different between treatment and control groups at any of the follow-ups.

Jemmott et al. (2010) found that, at the 12-month follow-up:

• The treatment group reported significantly higher consistent condom use (use of a condom

during every sexual act) than the control group (56 versus 50 percent).

- 65 -

• The percentage of sexual acts that were condom-protected was significantly higher in the treatment group than in the control group (73 versus 69 percent).

• Condom use at last sexual encounter was significantly higher in the treatment group than in the control group (70.9 versus 71.8 percent).

• There were no significant differences in frequency of sexual intercourse or self-rated frequency of condom use on a five-point scale.

Probable Implementers

Middle and high schools, adolescent health clinics, and youth service organizations

Funding

Funding for initial program implementation and research was provided by the American Foundation for AIDS Research and the U.S. National Institutes of Health.

Implementation Detail

Program Design

• Program materials are culturally and ethnically specific.

• Multiple methods of instruction are used to keep participants engaged in the program.

Curriculum

BPBR (also called Be Proud!) is a five-hour, six-module program intended to increase teens' level of knowledge regarding HIV/AIDS and other STDS, to have a positive impact on attitudes and intentions regarding risky sexual behaviors and condom use, to counter negative beliefs regarding condom use and sexual enjoyment, to improve self-efficacy and confidence so that teens can use prevention methods effectively, and to impart negotiation and refusal skills to help teens decrease risky sexual behaviors and situations. All facilitators are trained in the program curriculum prior to implementation. The level of recommended training varies according to the facilitator's background in HIV/AIDS education and knowledge of teenage sexuality. The length of the training programs ranges from 16 to 24 hours. Training includes proper implementation methods, review of HIV/AIDS knowledge, and review of curriculum content. During the training sessions, facilitators participate in the experience of the curriculum as though they were students. In addition, facilitators are given an opportunity to practice their instruction skills and are provided with feedback on their performance. Staffing

The program is typically staffed by educators, community mental health workers (for example, social workers), or nurses.

Issues to Consider

This program received a "promising" rating. Evaluations indicate that the program produced some positive results; however, the results are somewhat inconsistent. In the 1992, 1999, and 2009 evaluations of Be Proud!, the program was shown to increase the levels of knowledge regarding HIV- associated risk factors and effective prevention measures, as well as improving attitudes and intentions regarding risky sexual behavior. However, this change in knowledge and attitudes did not translate into a sustained improvement in teens' behaviors. The 2010 evaluation did show sustained effects at 12-month follow-up; however, the effects were not large enough to receive a "proven" rating.

- 66 -

Example Sites

Philadelphia, Pennsylvania, and Trenton, New Jersey

Contact Information

Select Media www.selectmedia.org Marketing Department 375 Greenwich St, suite 828 New York, NY 10013 800-707-6334 212-941-3997 (fax) beth@selectmedia.org or tyree@selectmedia.org Ordering Department Sophie Ampel 845-774-7335 845-774-2945 (fax) sophie@selectmedia.org For training information, contact Lynette Gueits at 703-867-9691

Available Resources

Curriculum materials are available from Select Media, Inc. Available materials include program/curriculum manual, activity sets, and program videos. Training for the program is available through the Staff Development Office of the Rocky Mountain Center for Health Promotion and Education at 303-239-6494.

Bibliography

Borawski, Elaine A., Erika S. Trapl, Kimberly Adams-Tufts, Laura L. Kayman, Meredith A. Goodwin, and Loren D. Lovegreen, "Taking Be Proud! Be Responsible! to the Suburbs: A Replication Study," Perspectives on Sexual and Reproductive Health, Vol. 41, No. 1, 2009, pp. 12-22. Jemmott, John B. III, Loretta Sweet Jemmott, and Geoffrey T. Fong, "Reductions in HIV Risk- Associated Sexual Behaviors Among Black Male Adolescents: Effects of an AIDS Prevention Intervention," American Journal of Public Health, Vol. 82, No. 3, 1992, pp. 372-377. Jemmott, John B. III, Loretta Sweet Jemmott, Geoffrey T. Fong, and Knashawn H Morales, "Effectiveness of an HIV/STD Risk-Reduction Intervention for Adolescents When Implemented by Community-Based Organizations: A Cluster-Randomized Controlled Trial," American Journal of Public Health, Vol. 100, No. 4, 2010, pp. 720-726. Jemmott, John B. III, Loretta Sweet Jemmott, Geoffrey T. Fong, and Konstance McCaffree, "Reducing HIV Risk-Associated Sexual Behavior Among Africa American Adolescents: Testing the Generality of Intervention Effects," American Journal of Community Psychology, Vol. 27, No. 2, 1999, pp. 161-175.

Last Reviewed

February 2013

- 67 -

http://www.selectmedia.org/

mailto:beth@selectmedia.org

mailto:tyree@selectmedia.org

mailto:sophie@selectmedia.org

Big Brothers Big Sisters of America

Program Info

Outcome Areas Healthy and Safe Children Children Succeeding in School

Indicators Youths not using alcohol, tobacco, or illegal drugs Students performing at grade level or meeting state curriculum standards Children and youth not engaging in violent behavior or displaying serious conduct problems

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Community-Based Service Provider Type of Service Mentoring Youth Development Type of Outcome Addressed Behavior Problems Cognitive Development/School Performance Physical Health Substance Use and Dependence Violent Behavior

Evidence Level Proven

Program Overview

Big Brothers Big Sisters (BBBS) is a program that matches non-related mentors with children to promote positive development and social responsibility. In existence for more than a century, BBBS is composed of 440 agencies that served more than 220,000 youths across the country in 2005. The BBBS network comprises individual, independent agencies that adhere to very specific BBBS standards and criteria, yet may adjust the program to the specific and unique needs of their communities. The mission of Big Brothers Big Sisters is to provide supportive relationships for young people to assist them in realizing their potential. The program has been shown to impact a variety of behavioral outcomes without providing a behavior-specific intervention or targeting a specific behavior (such as academic improvement, drug use, or violence). Rather, BBBS provides a design for a developmental mentoring program, focusing on providing participants with a positive, caring, and supportive role model.

In the traditional Big Brothers Big Sisters mentoring model, the volunteer mentor commits to spending approximately three to five hours per week with the child for at least one year. Goals for the child are set with the BBBS staff during an initial interview held with the parent and child. A relatively new set of BBBS programs focus on establishing school-based mentoring programs. These programs differ from the traditional BBBS programs in that all contact between the mentor and the youth takes place within a school and in that they adhere to a different set of participant requirements. Owing to these adaptations, the BBBS school-based model seems to be opening the door to a wider range of participants among both volunteers and youth. Big Brothers Big Sisters in School, a mentoring program that takes place in a school environment and allows weekly breaks from regular

- 68 -

programming for the child to take part in one-to-one activities with the mentor, now serves as many children as the traditional community program.

The Big Brothers Big Sisters program received a "proven" rating for the indicators Youths not using alcohol, tobacco, or illegal drugs, Children and youth not engaging in violent behavior or displaying serious conduct problems and Students performing at grade level or meeting state curriculum standards. See Issues to Consider below for further explanation.

Program Participants

Targeted youth are typically between the ages of 6 and 18 and have associated risk factors, such as residence in a single-parent home or a history of abuse or neglect. In most instances, either a parent or guardian initiates contact with BBBS. Prior to acceptance into the program, youths undergo a screening process involving a written application, interviews with both parent(s) and child, and a home assessment. This process is intended to ensure that both child and parent are prepared and equipped to honor the high level of commitment required by the program. Youths participating in the school-based model undergo screening as well; however, because school personnel determine acceptance into the program, children whose parents lack the initiative or time to make contact with program staff are not excluded from eligibility. Mentor participants undergo an extremely rigorous screening process designed to protect youths by identifying and screening out applicants who are unlikely to honor their time commitment or form positive relationships with youths or who pose a safety risk. After acceptance as a volunteer, mentors undergo orientation and training. The specific training requirements vary from site to site but typically involve discussions on program rules, match expectations, relationship building, match activities, and communication skills.

Evaluation Methods

An experimental study design that evaluated the traditional BBBS program was developed by researchers at Public/Private Ventures (Tierney, Grossman, and Resch, 1995) in which youths were randomly assigned to a control or to a treatment group. The sample youths were between 10 and 16 years old; 60 percent were male and more than 50 percent were an ethnic minority. Nearly all lived with one parent, many were from low-income households, and a significant number had a prior history of family violence or substance abuse. At the start of the study, the sample included 1,138 youths from eight BBBS agencies. Over an 18- month period, the research compared youths who participated in BBBS programs with those who did not. Both control and experimental groups were assessed at baseline and at the study’s conclusion via interviews on a range of outcomes, including academic outcomes, use of drug and alcohol, and conduct problems and violent behaviors. Of the 1,138 youths originally randomized to a control or experimental group, 959 (84.3 percent) completed the follow-up interview and were included in the study’s analysis. Analyses examined the BBBS programs’ effect on four demographic groups: boys, girls, whites, and minorities. Herrera et al. (2007) conducted an evaluation of a school-based BBBS program, in which youths were randomly assigned to a treatment or a control group. The sample includes 1,139 youths who were in grades four to nine at the beginning of the 2004-2005 school year (i.e., the baseline), with 565 in the treatment group and 574 in the control group. Surveys were administered to teachers, youths, and (for the treatment group only) mentors at three time points: beginning of the 2004-2005 school year (baseline), end of first school year (first follow-up), and late fall of the second school year (second follow-up). Results are based on intent-to-treat analysis to examine whether offering youths the opportunity of BBBS program involvement affected student outcomes. Additional studies by Public/Private Ventures (Furano et al., 1993; Roaf, Tierney, and Hunte, 1994; Morrow and Styles, 1995) provided descriptive accounts of program implementation, including the

- 69 -

actual mentoring relationship, the relationship between the adult mentor and the youth, and the screening process and characteristics of the volunteers.

Key Evaluation Findings

The results of these studies indicate that the broad developmental program model utilized by BBBS produces a range of positive quantifiable outcomes. The research by Tierney, Grossman, and Resch (1995) compared BBBS youths with nonparticipating youths. The authors reported the following:

• With regard to antisocial behaviors, compared with control group youths, BBBS youths were 46 percent less likely to initiate illegal drug use.

o Analyzed separately, BBBS boys were 55 percent less likely to initiate illegal drug use.

o When examined by racial/ethnic group, no significant differences were found for whites, but minority BBBS boys were 68 percent less likely to start using illegal drugs.

o No significant differences were found for girls as a group, or for white girls as a subgroup. Minority BBBS girls were 73 percent less likely to initiate illegal drug use, a difference that was marginally statistically significant.

• BBBS youths were 27 percent less likely to initiate alcohol use, a marginally significant difference.

o Minority BBBS girls were 54 percent less likely to initiate alcohol use than were control group girls, a difference that was marginally significant.

o No significant differences were found for either boys or girls as individual groups, or for minority boys or white boys or girls.

• BBBS youths were 32 percent less likely to hit someone.

o No significant differences were found for boys as a group or for minority boys as a subgroup. However, white boys in BBBS were marginally less likely to hit someone than were white boys in the control group.

o Girls overall were less likely to hit someone, a difference that was marginally significant, but no significant differences were found for either minority girls or white girls as subgroups.

• No significant effects were found for the BBBS group as a whole or for any of the subgroups for theft, property damage, involvement in fights, cheating on tests, being sent to the principal’s office, or smoking.

With regard to academic outcomes, compared with control group participants,

• BBBS youths attained slightly higher grade point averages (GPAs), with average GPAs of 2.71 versus 2.63, a difference that was marginally significant.

• No significant differences were found for boys as a whole, or for minority boys or white boys as subgroups.

• Girls who participated in BBBS attained significantly higher GPAs than did the comparison girls, with an average GPA of 2.84 versus an average of 2.67.

o Differences in GPAs were marginally significant for minority girls in BBBS, who had an average GPA of 2.83 compared with an average of 2.62 for control group girls.

o No effects were found for white girls.

- 70 -

• BBBS youths were 52 percent less likely to skip a day of school.

o No significant differences were found for boys as a whole, or for minority boys or white boys as subgroups.

o Girls who participated in BBBS skipped 84 percent fewer days of school than did control group girls.

o Effects were significant for both minority girls (78 percent fewer days skipped) and white girls (90 percent fewer days skipped).

Results of the study by Herrera et al. (2007) indicated significant improvement in a range of school- related outcomes. The authors reported the following: At the end of the first school year (first follow-up),

• BBBS youths, relative to their peers in the control group, received significantly higher teacher ratings on

o overall academic performance, with average ratings of 2.73 versus 2.62

o quality of class work, with average ratings of 3 versus 2.89

o number of assignments completed, with average ratings of 3.12 versus 2.98.

• Differences between BBBS youths and control group youths were marginally significant for the teacher ratings on

o performance in science, with average ratings of 2.84 for the BBBS group versus 2.73 for the control group

o performance in written and oral language, with average ratings of 2.77 for the BBBS group versus 2.68 for the control group.

• Teachers reported that, compared with their peers in the control group, a lower percentage of BBBS youths committed a serious school offense, including fighting, principal’s office visits, and suspensions in the past four weeks (14 percent for the treatment group versus 21 percent for the control group), and the difference was marginally significant.

• Teachers reported that a lower percentage of BBBS youths had an unexcused absence in the past four weeks (12 percent for the BBBS group versus 18 percent for the control group), and the difference was marginally significant.

• According to youth surveys, BBBS youths reported feeling better at doing their school work than their control group peers, scoring an average of 2.81 on the academic efficacy measure, compared with the average of 2.74 for the control group.

• According to youth surveys, 11 percent of the BBBS youths reported having started to skip school, compared with 17 percent of their peers in the control group.

• No significant differences were found between the BBBS and control groups for drug and alcohol use, misconduct outside of school, relationship with parents and peers, and self- esteem.

At the late fall of the second school year (the second follow-up), only one outcome was found to be statistically significant, and the rest of the outcomes that were found to have significant effects were no longer statistically significant. According to youth surveys, BBBS youths, compared with their control group peers, had a lower percentage of reporting to have started skipping school, with 20 percent for the BBBS youths versus 28 percent for the control group.

- 71 -

Probable Implementers

To be formally designated a Big Brothers Big Sisters program, a local agency must adopt specific BBBS standards, with minor variations allowed to accommodate local characteristics. Possible implementers include community members or social service organizations, such as family service agencies.

Funding

Big Brothers Big Sisters is traditionally funded through local fundraising efforts from the business, faith, and educational communities, as well as through private and public foundation support. The national average cost of making and supporting a match between a youth and an adult is approximately $1,000. Program cost varies depending on the agency and geographic location of the program.

Implementation Detail

Program Design

• Individual programs adhere to required Big Brothers Big Sisters national guidelines and standards.

• Big Brothers Big Sisters requires extremely rigorous screening of its volunteers and youth.

• Mentors undergo initial and continuing training to aid them in building a successful and supportive relationship.

• Youth-mentor relationships are closely monitored by a Match Support Specialist during the first year of development.

• Participation in the program requires a high level of personal commitment and commitment of time and energy.

Adult volunteers and youths are expected to make a substantial time commitment, agreeing to meet approximately three to five hours per week for at least one year. Rather than targeting specific problems or behaviors, BBBS focuses on developing the whole person. Participants are encouraged to address various issues and participate in a wide range of activities within a variety of settings. The goal is for a relationship to be forged, through which the mentor can provide support to the youth as he or she develops. Big Brothers Big Sisters is predicated on extremely detailed and rigorous volunteer/youth screening, training, and matching practices and standards. In an effort to ensure effective matches, during the first year BBBS Match Support Specialists provide ongoing supervision of the mentor-youth relationship: They are required to contact mentors and youths and/or youths’ parents monthly. Beyond the first year, supervision requirements are reduced: BBBS Match Support Specialists are required to contact mentors and youths and/or youths’ parents four times yearly. These requirements are somewhat unique to BBBS when compared with several other current mentoring programs that give preference to more casual youth-mentor relationships that require far less screening and training, do not necessarily have prescribed standards regarding participant investment, and may not involve intense and rigorous relationship supervision. The newer school-based BBBS program, Big Brothers Big Sisters in School, has slightly different participation requirements. Within the school-based model, all contact between a youth and his or her mentor must take place on school grounds. The constant direct supervision afforded by the school environment has made cross-sex matches a possibility and has permitted younger individuals to volunteer and to become mentors. In addition, the school-based model has a lesser time commitment: It requires that weekly meetings be one to two hours long and take place only during the academic school year. Even in the school setting, this model remains true to BBBS’s broad developmental focus: No greater stress is placed on academic-based interactions.

- 72 -

Curriculum

Big Brothers Big Sisters does not have a prescribed or "set" curriculum. Staffing

Basic staff at each site includes a director, an administrative assistant, and functional staff providing customer relations, enrollment and match services, and match support. The program is staffed by a core of volunteer mentors.

Issues to Consider

Based on the study by Tierney, Grossman, and Resch (1995), this program received a "proven" rating for the indicators Youths not using alcohol, tobacco, or illegal drugs and Juveniles not engaging in violent behavior or displaying serious conduct problems and a "promising" rating for the indicator Students performing at grade level or meeting state curriculum standards. The study employed rigorous standards in evaluating the program, including a randomized sample of nearly 1,000 youths. The significance and size of the effects of the program on the use of illegal drugs meet PPN proven evidence criteria, and the program’s effect on alcohol use was sizeable but at a lower significance level. The effects on the likelihood of BBBS youths hitting someone or on having a higher GPA than non-BBBS youths were marginally significant, and the size of the difference for the GPA outcomes was very small. Based on the study by Herrera et al. (2007), this program received a "proven" rating for the indicators Juveniles not engaging in violent behavior or displaying serious conduct problems and Students performing at grade level or meeting state curriculum standards. The study employed rigorous standards in evaluating the program, including a randomized sample of over 1,000 youths. The significance and size of the effects of the program on academic performance and misconduct meet PPN proven evidence criteria. However, it is worth noting that attrition is substantial and non-random. At the first follow-up, the attrition rate was 6 percent, with 5.8 percent among the treatment youths and 6.8 percent among the control youths. Attrition analysis indicated that those who were not retained for the first follow-up were significantly needier at baseline than those who remained in the sample. The two groups differed on 27 characteristics at baseline. By the second follow-up, the analysis sample further decreased to 968 youths, with a 15 percent attrition rate (12.7 percent among the treatment youths and 17.3 percent among the control youths). Again, attrition analysis revealed non-comparable results between the youths who remained and those who left. While the substantial attrition could impose potential threat to the internal validity of the study, authors conducted baseline analysis and reported no significant differences between the treatment and control group at baseline. It is important to note that findings from the study by Tierney, Grossman, and Resch (1995) are based on evaluations of the traditional BBBS mentoring model only. In contrast, Herrera et al. (2007) evaluated the newer school-based mentoring program.

Example Sites

Houston, Texas; San Antonio, Texas; Columbus, Ohio; Minneapolis, Minnesota; Rochester, New York; Phoenix, Arizona; Philadelphia, Pennsylvania; and Wichita, Kansas. (These sites are those included in Tierney, Grossman, and Resch, 1995.) Columbus, Ohio; Denver, Colorado; Ellsworth, Maine; St. Louis, Missouri; Cleveland, Ohio; Oak Harbor, Washington; Dallas, Texas; Show Low, Arizona; Dalton, Georgia; and Wilkes-Barre, Pennsylvania. (These sites are included in Herrera et al., 2007).

Contact Information

Joseph Radelet Big Brothers Big Sisters of America 230 North 13th Street

- 73 -

Philadelphia, PA 19107 Phone: (215) 567-7000 Fax: (215) 567-0394 national@bbbsa.org http://www.bbbsa.org

Available Resources

All of the following materials are available by contacting Big Brothers Big Sisters:

• Community Needs Assessment and Feasibility Study Guide

• Volunteer Education and Development Manual

• Program Management Manual

Bibliography

Big Brothers Big Sisters of America, BBBS Standards of Practices for One to One Service: Big Brothers Big Sisters of America, Philadelphia, Pa.: Big Brothers Big Sisters of America, 2003. Big Brothers Big Sisters of America, What Will It Take to Achieve Significance by the Year 2001? Philadelphia, Pa.: Big Brothers Big Sisters of America, 1996. Furano, K., P. A. Roaf, M. B. Styles, and A. Y. Branch, Big Brothers Big Sisters: A Study of Program Practices, Philadelphia, Pa.: Public/Private Ventures, 1993. Herrera, C., School-Based Mentoring: A First Look into Its Potential, Philadelphia, Pa.: Public/Private Ventures, September 1999. Herrera, C., C. L. Snipe, and W. S. McClanahan, Mentoring School-Age Children: Relationship Development in Community-Based School-Based Programs, Philadelphia, Pa.: Public/Private Ventures, 2000. Herrera, Carla, et al., Making a Difference in Schools: The Big Brothers Big Sisters School-Based Mentoring, Philadelphia, Pa.: Public/Private Ventures, 2007. Lawler, P.K., S. J. Ellis, and J. Corlett, Volunteer Education and Development, Philadelphia, Pa.: Big Brothers Big Sisters of America, 1991. Morrow, K. V., and M. B. Styles, Building Relationships With Youth in Program Settings: A Study of Big Brothers/Big Sisters, Philadelphia, Pa.: Public/Private Ventures, 1995. Peterson, D. G., and J. Magee, An Evaluation of an Elementary School Based Integenerational Linkages Program: Mentoring for Academic Enrichment, Philadelphia, Pa.: Big Brothers Big Sisters of America, 1994. Roaf, Phoebe A., Joseph P. Tierney, and Danista E.I. Hunte, Big Brothers/Big Sisters: A Study of Volunteer Recruitment and Screening, Philadelphia, Pa.: Public/Private Ventures, 1994. Tierney, Joseph P., Jean Baldwin Grossman, and Nancy L. Resch, Making a Difference: An Impact Study of Big Brothers Big Sisters, Philadelphia, Pa.: Public/Private Ventures, 1995.

Last Reviewed

November 2009

- 74 -

mailto:national@bbbsa.org

http://www.bbbsa.org/

Child Development Project

Program Info

Outcome Areas Healthy and Safe Children Children Succeeding in School

Indicators Youths not using alcohol, tobacco, or illegal drugs Students performing at grade level or meeting state curriculum standards

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Elementary School Type of Service Instructional Support Type of Outcome Addressed Behavior Problems Cognitive Development/School Performance Physical Health Substance Use and Dependence Violent Behavior

Evidence Level Promising

Program Overview

The Child Development Project (CDP) is a comprehensive, elementary school-based intervention program. CDP incorporates class meetings, learning activities for partners and small groups, and open-ended discussions on literature to enhance students’ social, ethical, and intellectual development. CDP is based on the belief that prevention efforts are most likely to be effective when they occur early in a child’s development, before antisocial behavioral patterns have a chance to become firmly established. CDP emphasizes the promotion of positive development rather than the prevention of disorder. The central goal of CDP is to help schools become "caring communities of learners" by offering an environment of caring, supportive, and collaborative relationships to build students’ sense of community in school and to promote school bonding.

Program Participants

Students in kindergarten through grade 6.

Evaluation Methods

Solomon et al. (1988) studied the effects of CDP in a sample of 346 students from six elementary schools in a middle- to upper-middle class suburban school district in the San Francisco Bay area. The sample over the five study years consisted of three treatment schools (173 students) and three control schools (163 students), which included 67 classrooms. The six schools that were selected to participate in the study were divided into two roughly equal groups (of three schools each) based on sociodemographic characteristics, student achievement levels, school size, the faculty’s teaching experience, and teachers’ level of interest in participating in the program. These two groups of schools were then randomly assigned to receive CDP or serve as the control group. The majority of students in

- 75 -

all of the schools were white, and there were no baseline differences between the treatment and comparison groups in terms of school size, student mobility, scores on student achievement tests, teachers’ length of experience, or teachers’ interest in participating in CDP. Children began the program in kindergarten and continued through the fourth grade, although analyses were cross- sectional as opposed to longitudinal (i.e., a different cohort of students provided information during each of the five study years). Students’ scores on the California Achievement Test were compared. Solomon et al. (1996) followed up on the Solomon et al. (1988) sample of students from Northern California. The authors studied learning outcomes among 272 sixth-grade students (157 treatment and 115 control group students) from two treatment and two control schools. (Due to decreased financial support, one treatment school and one control school were not included in the follow-up study.) Although this second study was cross-sectional, many of the students were assessed more than once because the program focused on students in successive grades in each year (e.g., 65 percent of the sixth-grade students had also been assessed in the fourth grade). Extensive baseline assessments of a random sample of students in the treatment and control schools revealed no large or consistent differences between the two groups of students. The outcome measure studied was sixth- grade reading comprehension scores, derived from a test previously developed by the Education Testing Service for the National Assessment of Educational Progress. For the test, students read two brief passages and then wrote responses to general questions about the meaning of the passages. Battistich et al. (2000) conducted a larger study on sixth-grade students at 24 elementary schools (12 treatment and 12 control schools) from six school districts. The analysis sample consisted of approximately 1,600 students (800 each in the treatment and control groups) in each study year. Twelve of the schools were located on the West Coast of the United States, four in the South, four in the Southeast, and four in the Northeast. They included urban, suburban, and rural schools and served diverse student populations (e.g., 2 percent to 95 percent of students received free or reduced-price school lunches, 26 to 100 percent of students were members of ethnic minority groups, and 0 to 32 percent had limited or no English-speaking skills). Districts were selected based on suggestions from district superintendents and other central office administrators. Comparison schools were chosen that matched the treatment schools as closely as possible with respect to various demographic characteristics. The analysis of outcomes comparing treatment and control groups was cross-sectional. Outcome measures included students’ self-reported use of cigarettes, alcohol, and marijuana, and students’ self-reported involvement in ten delinquent behaviors during the past year, including throwing objects at people or cars, carrying a weapon, threatening to harm someone, hurting someone on purpose, or being involved in a gang fight. The authors also conducted a separate analysis for "high-change" schools (treatment schools for which most or all of the teachers showed at least moderately positive changes in their degree of CDP implementation from the baseline year). Solomon et al. (2000) analyzed different program effects in the same sample of students studied by Battistich et al. (2000). Student academic performance was studied through a measure of students’ inductive reasoning skills, as well as through standardized tests of reading and math achievement. Battistich et al. (2004) also conducted a follow-up study to Battistich et al. (2000), tracking students through 6th, 7th, and 8th grades. The sample consisted of six of the original 12 CDP and matched comparison schools. At these schools, an average of 49 percent of students were eligible for free or reduced price school lunches, and an average of 57 percent of students were members of ethnic minorities. By the time the follow-up study was conducted, 35 percent of the original sample of CDP and control group students had graduated from middle school, and an additional 8 percent were unable to be located in the participating school districts. Parental consent to participate in the follow- up study was obtained for 1,246 students, including 700 former CDP students and 546 former control group students. The racial/ethnic background of students was diverse (40 percent White, 32 percent Hispanic, 22 percent African-American, and 5 percent Asian). There were significantly fewer Hispanic students in the CDP group than the control group (28 percent versus 38 percent), and CDP students were significantly more likely than control students to have initiated tobacco use (22 percent versus 13 percent) but significantly less likely to have initiated marijuana use (2 percent versus 6 percent). Outcome measures included past month use of tobacco, alcohol, marijuana, and other elicit drugs; delinquent behaviors (e.g., using weapons in a fight); grade-point average in core academic subjects (language arts, mathematics, science, and social studies), and scores on district-administered achievement tests. Program effects were assessed by comparing the total sample of all treatment and

- 76 -

comparison students, as well as by focusing only on students from high-change schools. Sample sizes for the high-change group were approximately 775 students for the behavior variables, and 900 students for the academic achievement data.

Key Evaluation Findings

Results from the Solomon et al. (1988) study of fourth-grade students found:

• There were no significant differences between treatment and control groups on the California Achievement Test.

The Solomon et al. (1996) follow-up study of sixth-grade students reported:

• The treatment group had significantly higher reading comprehension scores than the control group.

Battistich et al.’s (2000) study of 24 schools, found, for the total sample, that there was little evidence of program effects on students’ involvement in problem behaviors. No significant differences were found between groups in the use of cigarettes or marijuana, carrying weapons, threatening to hurt someone, hurting someone purposely, or involvement in gang fights. The authors reported the following marginally significant findings:

• Compared with control group students, treatment group students exhibited a larger decrease in the use of alcohol.

• Compared with control group students, treatment group students exhibited an increase in the frequency of throwing objects at people or vehicles.

For high-change schools (including five treatment and five control schools), results for the treatment group were somewhat more favorable. The authors reported the following:

• Treatment group students showed a significantly greater decrease in the use of alcohol and marijuana when compared with control group students.

• Treatment group students also showed a marginally significant greater decrease in involvement in gang fights when compared with control group students.

• No significant differences were found between the two groups in cigarette use, throwing objects, carrying weapons, threatening to hurt someone, or hurting someone purposely.

The 24-school study on cognitive effects (Solomon et al., 2000) found the following:

• There was little evidence that CDP had consistent overall effects on achievement.

• The five schools rated as high-change schools showed no significant program impacts on the measure of inductive reasoning.

• Three of the high-change schools showed no significant program impacts on standardized achievement tests, and one of the three schools demonstrated a significantly negative program effect on math achievement. The remaining two schools showed large positive impacts when compared with control schools.

The follow-up study by Battistich et al. (2004), that tracked 1,246 students through middle school, found the following results for the total sample:

• No significant differences between CDP students and control group students for grade point average or achievement test scores.

• No significant differences between groups for use of tobacco, alcohol, cigarettes, or other illicit drugs.

• No significant differences between groups for delinquent behaviors.

- 77 -

For the high-change elementary schools, the authors reported the following:

• A significantly positive effect favoring the treatment group for grade point average of core academic subjects.

• A significantly positive effect favoring the treatment group for achievement test scores.

• Treatment group students scored significantly lower on rates of delinquent behaviors.

• No significant differences were found between groups for use of tobacco, alcohol, cigarettes, or other illicit drugs.

Probable Implementers

Public and private elementary schools

Funding

Recommendations for sources of CDP implementation funding include those in the following list (provided by the Development Studies Center website). CDP grant-writing staff is also available to help schools apply for funds by providing language to describe the CDP teacher training and technical offerings. In addition, CDP staff is available to review school proposals for CDP funding and to suggest improvements to them.

• Title I, Part F: Comprehensive School Reform (CSR) http://www.ed.gov/policy/elsec/leg/esea02/pg13.html

• Title I, Part A, Improving Basic Programs Operated by Local Education Agencies, Section 1114, Schoolwide Programs http://www.ed.gov/legislation/ESEA02/pg2.html#sec1114

• Title I, Part A, Improving Basic Programs Operated by Local Education Agencies, Section 1116, Academic Assessment and Local Education Agency and School Improvement http://www.ed.gov/legislation/ESEA02/pg2.html#sec1116

Implementation Detail

Program Design

The CDP is a whole-school program consisting of an intensive classroom component, a school-wide component, and a family involvement component. The program components seek to build stable, warm, and supportive relationships; address the social and ethical dimensions of learning; teach to the active mind; and recognize self-motivated students. These four program components are developed through:

• Classroom meetings designed to teach students how to act responsibly, show consideration for one another, and communicate effectively;

• Cooperative and noncompetitive school-wide activities that involve students, parents, and school staff;

• Cross-age/cross-grade "buddies" activities designed to build positive relationships and support students' academic goals; and

• Activities for students to do at home so that parents can be involved in their child's schoolwork.

The classroom component consists of five interrelated groups of activities: (1) cooperative activities (e.g., providing children with frequent opportunities to work collaboratively toward common goals); (2) discipline-development activities (e.g., emphasizing students' understanding of the general

- 78 -

http://www.ed.gov/policy/elsec/leg/esea02/pg13.html

http://www2.ed.gov/policy/elsec/leg/esea02/pg2.html#sec1114

http://www.ed.gov/legislation/ESEA02/pg2.html#sec1116

principles behind rules and involving students' input on rule-setting); (3) activities to promote social development (e.g., including role-playing activities and discussions of conflicts among students); (4) activities to promote prosocial values (e.g., discussing the underlying values that characterize classroom behaviors as being prosocial); and (5) helping activities (e.g., encouraging children to be helpful to one another by engaging in classroom tasks and tutoring other students.) Staffing

Implementation of CDP requires purchasing or acquiring program materials and conducting ongoing staff development for two or more years. CDP program materials include teachers' guides on books in the reading curriculum, a student activity book, a book of anecdotal stories about other CDP teachers, and a video on the language arts curriculum. Additional program materials include teachers' guides for building a sense of community in the classroom and for implementing a "buddies" program, a guide to creating a sense of community in the school, and a family activity book. Each of these additional materials is accompanied by a video. To implement CDP, teachers attend a two-day summer institute along with three three-day follow-up visits each year (for purposes of observation, coaching, consultation, and additional staff development). Teachers in each participating school receive follow-up visits for two or three years, depending on the school's success at implementing the program.

Issues to Consider

This program received a "promising" rating for the benchmarks "Increase the percentage of students performing at grade level or meeting state curriculum standards" and "Increase the percentage of youths not using alcohol, tobacco, or illegal drugs." The evaluations of the CDP indicate that in some cases treatment group students have experienced significant improvements in reading, math, science, and social studies skills when compared with control group students. In addition, study results indicate that the CDP may contribute to a short-term decrease in the use of alcohol and marijuana. The program’s "promising" rating for the cognitive benchmark is due to mixed findings on the differences in achievement between treatment and control groups. For example, Solomon et al.’s (1988) study of fourth-grade students did not find any significant academic differences between treatment and control group students, although Solomon et al.’s sixth-grade follow-up study (1996) reported higher reading comprehension scores among treatment group students. Cognitive effects, as seen in Solomon et al. (2000), were very mixed. No differences were found between treatment and control groups on measures of inductive reasoning, and three of the five high--change schools showed no significant positive program effects. Two of the high-change schools did show significant program impacts, although results were mixed among the three program years. Effects on academic measures as reported by Battistich et al. (2004) were also mixed, with no significant differences found between treatment and control groups for the total sample of students, but they found a significantly positive effect favoring the treatment group for the high-change elementary schools. CDP results also seem promising for decreasing students’ short-term use of alcohol and marijuana (Battistich et al., 2000)), although follow-up results (Battistich et al., 2004) reported no significant longer-term program effects for either the total sample of students or the high-change group. Results for CDP effects on violent behavior may be less encouraging. Battistich et al. (2000) noted a marginally greater increase for treatment group students in the frequency of throwing objects at people or vehicles. The authors also did not find any significant differences between groups in their rates of carrying weapons, threatening to hurt someone, or hurting someone purposely. The follow-up study by Battistich et al. (2004), found no significant differences between groups in the total sample for delinquent behaviors, yet reported that CDP students in the high-change elementary schools scored significantly lower on rates of delinquent behaviors. The extent of CDP implementation may be a significant factor in the success of program outcomes. Outcomes for high-change schools were more positive than for other schools, suggesting that schools with a stronger implementation of the program may experience greater cognitive and behavioral changes among their students than schools with weaker implementation of the program.

- 79 -

Finally, it should be noted that the CDP program developers have been authors of all four studies cited in this program description.

Example Sites

San Francisco Bay Area

Contact Information

Eric Schaps Developmental Studies Center 2000 Embarcadero, Suite 305 Oakland, CA 94606-5300 Tel (510) 533-0213 Fax (510) 464-3670 Eric_Schaps@devstu.org http://www.devstu.org

Available Resources

The Developmental Studies Center website (http://www.devstu.org/page/research-child-development- project) provides details on CDP program resources and implementation options for schools. CDP offers school-wide support over several grades, including intensive professional development sessions, leadership support, and technical assistance with data analysis to support continuous improvement of program outcomes.

Bibliography

Battistich, Victor, Eric Schaps, and Nance Wilson, "Effects of an Elementary School Intervention on Students’ "Connectedness" to School and Social Adjustment During Middle School," The Journal of Primary Prevention, Vol. 24, No. 3, 2004, pp. 243-262. Battistich, Victor, Eric Schaps, Marilyn Watson, Daniel Solomon, and Catherine Lewis, "Effects of the Child Development Project on Students’ Drug Use and Other Problem Behaviors," The Journal of Primary Prevention, Vol. 21, No. 1, 2000, pp. 75-99. Solomon, Daniel, Marilyn S. Watson, Kevin L. Delucchi, Eric Schaps, and Victor Battistich, American Educational Research Journal, Vol. 25, No. 4, 1988, pp. 527-554. Solomon, Daniel, Marilyn S. Watson, Victor Battistich, Eric Schaps, and Kevin Delucchi, "Creating Classrooms that Students Experience as Communities," American Journal of Community Psychology, Vol. 24, No. 6, 1996, pp. 719-748. Solomon, Daniel, Victor Battistich, Marilyn Watson, Eric Schaps, and Catherine Lewis, "A Six-District Study of Educational Change: Direct and Mediated Effects of the Child Development Project," Social Psychology of Education, Vol. 4, No. 1, 2000, pp. 3-51.

Last Reviewed

January 2013

- 80 -

mailto:Eric_Schaps@devstu.org

http://www.devstu.org/

http://www.devstu.org/page/research-child-development-project

Child Sexual Abuse Prevention: Teacher Training Workshop

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children not experiencing physical, psychological or emotional abuse

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School High School Type of Service Health Education Type of Outcome Addressed Child Abuse and Neglect

Evidence Level Promising

Program Overview

The Child Sexual Abuse Prevention: Teacher Training Workshop Curriculum was developed in 1988 to train and prepare teachers to more adequately respond to the problem of child sexual abuse. The training was developed with the idea that teachers are in a good position to respond to sexually abused children because they have increased accessibility to children in their classrooms and because they have training in child development. The six-hour program presents classroom teachers from kindergarten through twelfth grade with general knowledge on child sexual abuse and attempts to increase their awareness of the problem. The training provides an opportunity for teachers to (1) explore their own sexual and abuse-related issues and opinions; (2) determine and build upon their knowledge of the symptoms of abuse; and (3) understand their legal and ethical responsibilities as teachers. The program uses a variety of educational tools to help teachers better understand, recognize, and respond appropriately to child sexual abuse. Those tools include lectures, videotapes, role-playing, pencil and paper activities, question-and-answer sessions, and group discussions. The training is conducted by individuals with expertise in child sexual abuse, such as school psychologists or staff from a child abuse center. In addition, community experts—including therapists, pediatricians, social workers, detectives, and attorneys—can be brought in to discuss their involvement with child sexual abuse.

Program Participants

In the studies done so far, classroom teachers in grades K–12 have been trained in the curriculum, with most of the participating teachers in elementary education. The studies were all conducted in the southeastern United States. Participating teachers in one study were randomly selected by age, grade level, and gender, while only female teachers were used in another study. Trained teachers were

- 81 -

responsible for teaching entire classrooms. However, the studies primarily examined outcomes for teachers rather than for students.

Evaluation Methods

In 1988, an evaluation of the program took place with 45 female third and fourth grade teachers from one school district in suburban Atlanta, Georgia (Kleemeier et al.). The volunteer teachers were assigned randomly as either training Program Participants (26 teachers) or nonparticipants (19 teachers). Two psychologists with expertise in child sexual abuse conducted the training. Nonparticipants received written materials about child sexual abuse after the study was completed. In 1994, another study of the program took place with 42 volunteer male and female K–12 teachers from one school district in rural Hendersonville, North Carolina (Randolph and Gold, 1994). The teachers were assigned randomly by gender and grade level to be either training participants (21 teachers) or nonparticipants (21 teachers). Five speakers, including the school system’s psychologist, representatives from a regional child abuse center, and representatives from the local Department of Social Services, presented the training. Teachers not participating in the program had an opportunity to take part in the training the year after the study was completed. A third study (Hazzard, Kleemeier, and Webb, 1990) used a different design and compared three different types of trained instructors. One group contained 15 regular teachers teaching in their own classrooms. A second group was made up of eight lead teachers teaching in unfamiliar classrooms. The third group consisted of expert consultants who taught in ten classrooms with which they were unfamiliar. Only the last study evaluated the outcomes for the children. The other two examined only teacher perceptions.

Key Evaluation Findings

Kleemeier, et al. (1988) found that:

• Trained teachers’ knowledge of the scope of abuse, dynamics of abuse, indicators of abuse, reporting procedures, treatment alternatives, and prevention increased dramatically compared with control teachers.

• Trained teachers were more likely to acknowledge the severity of an abuse problem, more likely to see child protective services as helpful, more supportive of prevention services, and more confident that they could play a helpful role in addressing the problem of abuse.

• The control group teachers went from having a pre-test average of 14.2 on the 30-item knowledge scale to having a post-test average of 13.6, whereas the treatment group teachers went from having a pre-test average of 14.8 to a having post-test average of 23.3.

• On a 25-item opinion scale in which teachers were scored from 0 to 3 on each item, the control teachers’ average scores went from 54.9 to 52.8 whereas treatment teachers’ average scores went from 52.1 to 56.6.

• On an eight-item vignette test that was given only as a post-test, teachers were scored based on the number of behavioral indicators they assessed correctly, the applicability of the recommended action, and amount of warmth and openness that the teacher conveyed. Control teachers scored an average of 25.2 versus an average of 35.7 for treatment teachers.

Hazzard, Kleemeier, and Webb (1990) found that:

• There was no significant difference in the impact of programs conducted by teachers versus child abuse expert consultants.

- 82 -

• Children who were taught by the trained teachers, as compared with those children in the control groups, showed significant increases in knowledge about sexual abuse and these gains were still present at six-week and one-year follow-ups. Third graders went from having a pre- test average of 17.3 on a 25-item scale to having a post-test average of 21.7. Fourth graders went from having a pre-test average of 18.8 to having a post-test average of 22.4.

Randolph and Gold (1994) found that:

• Trained teachers were able to apply their knowledge to hypothetical situations more easily.

• Trained teachers reported spending more time in “discussions of child abuse issues with a friend or colleague,” “classroom activities or discussions about abuse,” and “discussions with individual children about possible abuse” than did control teachers.

• On a 30-item knowledge scale, untrained teachers went from having a pre-test average of 19.62 to a having a post-test average of 19.57. Trained teachers went from having a pre-test average of 18.29 to having a post-test average of 25.43.

• On the opinion scale similar to the one used by Kleemeier et al., untrained teachers scored 50.24 on the pre-test and 50.10 on the post-test, versus the trained teachers who scored 50.52 on the pre-test and 57.00 on the post-test.

• The post-test vignette average score for the untrained teachers was 25.14 versus 44.24 for the trained teachers.

Probable Implementers

Public, private, or parochial schools; social service or public health educators; state and local child abuse prevention agencies.

Funding

The National Institutes of Mental Health funded these studies. The training could be funded by school districts, state governments, federal grants, or community organizations.

Implementation Detail

Program Design

• Teachers are trained with a six-hour training curriculum (the Child Sexual Abuse Prevention: Teacher Training Workshop Curriculum) designed to improve teacher skills in the area of child sexual abuse.

• The training in the studies was a voluntary rather than mandatory part of the teacher’s job so as to minimize any potential trauma for teachers who may have been abused in the past.

• The trainers in the studies tried to make the program as culturally sensitive as possible to be able to reach all participants.

Curriculum

The Child Sexual Abuse Prevention: Teacher Training Workshop Curriculum is a six-hour intervention training that helps teachers to recognize behavioral and physical symptoms of sexual abuse, respond appropriately to disclosures, and report sexual abuse cases. It was designed by a group of teachers from a suburban Atlanta school district in 1988. (See Available Resources.)

- 83 -

Staffing

This program has been tried with regular classroom teachers, lead teachers, and outside experts. The program seemed to work equally well with all groups; therefore, the program can be staffed easily by classroom teachers.

Issues to Consider

This program received a “promising” rating. Evaluations indicate that the teacher trainings have produced some positive results. However, a majority of the research has examined outcomes only for participating teachers and not for the children. Because the result area and benchmark that are affected by this program are related to children being safe, the evaluations only indirectly address the issue of child sexual abuse. Furthermore, the evaluation that included children’s outcomes measured the program’s impact on intermediary outcomes—children’ s knowledge of what constitutes abuse— but did not measure changes in abuse itself or how abuse cases are handled. In general, this program seems to be very effective in making teachers better able to respond to child sexual abuse and the children better able to distinguish between what is appropriate and what isn’t. However, no research has been done to determine how often the indications of abuse are correct or whether children are getting the appropriate help after a real case of abuse is discovered. Any K–12 teacher could be trained on this curriculum, but it is difficult to say whether it would be applicable for all students because student outcomes were studied only to determine whether there were any changes in knowledge about abuse. Two of the three studies used a control group of teachers who were not given the training. However, both of these studies looked only at differences in the amount of knowledge that either teachers or students had on sexual abuse. The third study did not use a control group but looked at differences among the three types of trainers. None of the studies looked at differences in abuse rates or how abuse was handled.

Example Sites

Atlanta, GA., and Hendersonville, NC.

Contact Information

Ann Hazzard, Ph.D. Pediatric Continuity Clinic PO Box 26065 80 Butler Street, SE Atlanta, GA 30303 phone: (404) 616-4875

Available Resources

The Teacher Training Workshop Curriculum is available from Dr. Ann Hazzard (see the Contact Information).

Bibliography

Hazzard, A., Kleemeier, C., & Webb, C. Teacher Versus Expert Presentations of Sexual Abuse Prevention Programs 1990. Journal of Interpersonal Violence, 5: 23-36 Kleemeier, C., Webb, C., Hazzard, A. & Pohl, J. Child Sexual Abuse Prevention: Evaluation of a Teacher Training Model 1988. Child Abuse & Neglect, 12: 555-561. Randolph, M. K., & Gold, C.A. Child Sexual Abuse Prevention: Evaluation of a Teacher Training Program 1994. School Psychology Review, 23(3): 485-495.

- 84 -

Last Reviewed

May 2009

Child-Parent Centers

Program Info

Outcome Areas Healthy and Safe Children Children Ready for School Children Succeeding in School Strong Families

Indicators Students performing at grade level or meeting state curriculum standards Students graduating from high school Children not experiencing physical, psychological or emotional abuse Families increasing economic self-sufficiency Children ages 0 to 5 exhibiting age-appropriate mental and physical development Children and youth not engaging in violent behavior or displaying serious conduct problems Children experiencing good physical health

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Type of Service Family Support Health Care Services Instructional Support Parent Education Type of Outcome Addressed Behavior Problems Child Abuse and Neglect Cognitive Development/School Performance Juvenile Justice Physical Health

Evidence Level Proven

Program Overview

The Chicago Child-Parent Centers (CPCs) provide comprehensive educational support and family support to economically disadvantaged children and their parents. The guiding principle of the program is that by providing a school-based, stable learning environment during preschool, in which parents are active and consistent participants in their child's education, scholastic success will follow. The program requires parental participation and emphasizes a child-centered, individualized approach to social and cognitive development. The CPC program was founded in 1967 to serve families in high-poverty neighborhoods that were not being served by Head Start or similar programs, and it originally served preschool through third-grade

- 85 -

students. The centers are part of the Chicago Public Schools system. They are traditionally housed in preschool facilities. Currently, the Chicago Public Schools operate 11 federal Title I CPC sites.

Program Participants

CPC programming is currently only available to children in preschool. To be eligible to participate in the CPC program, children must reside in school neighborhoods that receive Title I funds. For children to participate in the program, their parents must commit to volunteering at a CPC on a weekly basis. The centers conduct outreach activities to recruit families who are in need.

Evaluation Methods

Evaluations of the CPC program used information from the Chicago Longitudinal Study (CLS), which followed 1,539 low-income minority students (of whom 95 percent are African-American, and 5 percent are Hispanic). The students were scheduled to graduate kindergarten in 26 public elementary schools in Chicago in the spring of 1986. All children resided in neighborhoods eligible for Title I services. Among them were 1,150 children who were enrolled in 20 CPCs that had both preschool and kindergarten programs, and those students served as the "treatment" group. The comparison group consisted of 389 children who were students at six randomly selected schools participating in a full- day kindergarten program for low-income students. For some of the analyses, some of the children in the comparison group had received CPC services in grades one through three. At the start of the CLS, the majority of the two groups’ family and child characteristics were similar (Reynolds, 1997; Reynolds and Temple, 1995; Reynolds et al., 2001). Students were followed for a total of 19 years, after which time the children were an average of 23-24 years old. At the 19-year follow-up, data were available for 91 percent of the original CPC program group and 89 percent of the original comparison group. A range of outcomes for the CPC program and the comparison children were compared at many points in time, beginning at prekindergarten, continuing in multiple school-age grades, and ending when the students were age 24. In addition, outcomes have been examined for children with varying levels of participation in CPCs. For example, a study by Reynolds (2000) assessed program outcomes through grades eight and nine for five different groups. The evaluation first examined students who participated in a CPC program during preschool versus all other children who did not attend preschool (but may have participated in school-age CPC programming). Next, the impact of any participation in the CPC program, regardless of length, was assessed. Third, the author studied the impact of any participation in the follow-on (school-age) program versus no participation in the follow-on program. Fourth, the author assessed the impact of total years of CPC program participation to determine whether there might be a cumulative effect of additional years of program involvement. Finally, the study looked at the impact of extended participation in the CPC program by comparing children who had participated for a total of more than four years up to six years with children who had participated in the program for only four years. The sample sizes in the different studies varied based on the particular outcome measure evaluated and on the year of analysis (kindergarten; grades three, five, six, seven, eight, or nine; or at the 15- year or 19-year follow-ups). However, in most cases, the total sample size (for both the CPC group and comparison group combined) was more than 1,000. Student outcomes were assessed using a variety of measures including the Iowa Test of Basic Skills (ITBS) standardized test of school readiness and math and reading achievement; school records regarding grade retention, graduation, placement in remedial/special education services, and delinquent behavior; arrest, conviction, and incarceration records; and child maltreatment reports. Most of the analyses in the evaluations cited in the next section under Key Evaluation Findings controlled for child, family, and school characteristics when comparing the mean differences between the CPC program children and the comparison group. More recently, researchers have compared the costs of the CPC program with estimates of its outcome, to determine its cost-to-benefit ratio (Reynolds et al., 2002).

- 86 -

Key Evaluation Findings

Overall, the CPC preschool program has shown effectiveness at improving a range of child and adolescent outcomes, with the largest benefits found for participation in the preschool program, and fewer (but still significant) benefits found for school-age participation. Major evaluation findings are as follows: A study by Reynolds (1995) found the following:

• When children who participated in (any amount of) CPC preschool were compared with children without any preschool:

o CPC preschoolers outperformed the nonpreschoolers. The largest effect of preschool was on cognitive readiness at kindergarten school entry—a difference of about three months of learning performance between the two groups.

o Effects of CPC preschool on achievement in reading and math remained statistically significant and meaningful in terms of educational gains through grade six.

o Preschool participants also had consistently lower cumulative rates of grade retention, ranging from 6.6 percentage points to 8.5 percentage points.

o Differences in rates of special education placement did not emerge until the third grade, but they were significant from grade three through grade six.

• Comparing children with two years of CPC preschool with children with one year of CPC preschool produced the following findings:

o Two-year participants significantly surpassed one-year participants in kindergarten cognitive readiness and in reading and math achievement during kindergarten. These differences represent about two months of learning performance.

o Two-year participants had lower rates of grade retention by grade one, but by grade two and later, this difference was no longer significant.

o Two-year participants had lower rates than one-year participants of special education placement during grades three, four, and five. However, the rates of special education placement were relatively low, and only the grade four difference was both statistically significant and sizeable.

Reynolds (2000); Reynolds et al. (2002) found the following at the completion of grade nine:

• Children who attended a CPC preschool program for one year or two years as compared with children who did not attend preschool

o scored higher on ITBS reading and math tests

o were less likely to have ever been retained in a grade (23.0 percent versus 38.4 percent)

o were less likely to ever be placed in special education and, for those who were placed in special education, spent an average of four fewer months in such services

o were not significantly different in rates of school delinquency infractions (i.e., persistent truancy, fighting, theft, carrying weapons, and gang activity).

• Children who had any CPC program participation (in preschool or elementary school) as compared with children who had no CPC exposure

o scored significantly higher on both math and reading achievement

o were significantly less likely to have been retained a grade (23.8 percent versus 34.3 percent)

- 87 -

o were no less likely to have ever been placed in special education but, when placed in special education, spent fewer total years (0.40 years) receiving such services

o were not significantly different in rates of school delinquency infractions.

• Participation in the CPC "follow-on" intervention (provided in primary grades) compared with no participation was

o associated with higher math achievement

o not associated with higher reading achievement

o associated with lower rates of grade retention (22 percent versus 30 percent)

o associated with significantly fewer years spent in special education and marginally significantly lower rates of ever being placed in special education

o associated with significantly lower rates of delinquency infractions at ages 13 to 15 (20.4 percent versus 26.3 percent), but no significant differences remained between groups at ages 15 to 16.

• Participation in the CPC program for more than four and up to six years (the extended intervention) compared with participation for only four years was associated with

o significantly higher reading and math test scores

o significantly lower rates of grade retention (21.9 percent versus 32.3 percent)

o no significant differences in years of special education placement

o no significant differences in school delinquency infractions.

Reynolds et al. (2001, 2002) found the following at the 15-year follow-up (ages 20 to 21):

• The CPC preschool group had significantly better outcomes than the nonpreschool group on

o number of years of special education from ages 6 to 18 (0.73 years versus 1.43 years)

o percentage of children who experienced child maltreatment (reports of abuse/neglect) from ages 4 to 17 (5.0 percent versus 10.3 percent)

o arrests of any type (16.9 percent versus 25.1 percent) and violent arrests (9.0 percent versus 15.3 percent)

o number of petitions to juvenile court by age 18 (an average of 0.45 versus an average of 0.78 petitions)

o high school completion by age 21 (61.9 percent versus 51.4 percent)

o highest grade completed by age 21 (grade 11 [0.23] versus grade 10 [0.87] on average).

• School-age participation (with or without preschool participation) yielded

o no improvement in educational attainment or juvenile arrests

o lower rates of special education enrollment (15.4 percent versus 21.3 percent)

o lower rates of grade retention (23.8 percent versus 34.3 percent).

• The CPC extended intervention group outscored the nonextended group on

o number of years of special education from ages 6 to 18 (0.56 versus 1.23 years, a marginally significant difference)

o child maltreatment from ages 4 to 17 (3.6 percent versus 6.9 percent)

o petitions to juvenile court for violent offenses by age 18 (9.3 percent versus 12.4 percent, a marginally significant difference).

- 88 -

Finally, Reynolds et al. (2007) reported the following outcomes at the 19-year follow-up (ages 23 to 24):

• The CPC preschool group had significantly better outcomes than the nonpreschool group on

o high school completion (71.4 versus 63.7 percent)

o highest grade completed (11.7 versus 11.4)

o four-year college attendance (14.7 versus 10.0 percent)

o rate of felony arrest (16.5 versus 21.1 percent)

o rate of incarceration (20.6 versus 25.6 percent)

o rate of any conviction (20.3 versus 24.7 percent) (marginally significant)

o rate of felony conviction (15.8 versus 19.9 percent)

o rate of violent crime conviction (5.1 versus 7.1 percent) (marginally significant)

o number of months having received any type of public aid (32.1 versus 28.3 months) (marginally significant)

o rate of health insurance coverage (70.2 versus 61.5 percent)

o depressive symptoms (12.8 versus 17.4 percent)

• No significant differences were found between the preschool group and the nonpreschool group for attendance at any college, overall arrests, violent arrests, full-time employment, receipt of food stamps or public aid, birth of a child before age 18, substance use, or smoking.

• School-age participation (with or without preschool participation) yielded

o significantly fewer months receiving public aid, such as Temporary Aid to Needy Families (TANF), food stamps, or Medicaid) from ages 18-24 (33.9 versus 28.3 months)

o marginally significantly lower rates of birth of a child before age 18 (34.1 versus 27.4 percent)

o no significant group differences for any of the educational or crime outcomes, full-time employment, health insurance, substance use, smoking, or depression.

• The CPC extended intervention group outscored the nonextended group on

o high school completion (73.9 versus 65.5 percent)

o highest grade completed (11.82 versus 11.51)

o four-year college attendance (16.7 versus 13.1 percent) (marginally significant)

o rate of arrest for violence (13.9 versus 17.9 percent)

o rate of violent crime convictions (5.5 versus 8.0 percent)

o rate of full-time employment (42.7 versus 36.4 percent)

o rates of public aid receipt (58.8 versus 64.2 percent) (marginally significant).

• No significant differences were found between the CPC extended intervention group and the nonextended group for college attendance, any incarceration, any arrest, felony arrest, any conviction, felony conviction, receipt of food stamps, birth of a child before age 18, any health insurance, smoking, substance use, or depression.

Probable Implementers

Public and private preschools

- 89 -

Funding

The program is funded by federal Title I funds. Reynolds et al. (2002) estimate that the preschool program provided a return to society of $8.47 per dollar invested by increasing the economic well-being of participants and tax revenues from those participants and by reducing public expenditures for remedial education, child welfare, criminal justice treatment, and crime victims. The school-age program provided a return of $1.97 per dollar invested, while the extended intervention program (four to six years of participation) yielded a return to society of $7.25 per dollar invested (all numbers are converted to 2005 dollars). The cost of the CPC program per participant is as follows:

• For each year of the preschool program: $5,219.

• For each year of the extended grade-school program: $1,874.

• The average cost of the extended program (four to six years of participation): $11,862.

Implementation Detail

Program Design

• Each center is directed by a lead teacher, who acts as a program coordinator and has overall responsibility for organizing and implementing program services. This responsibility primarily involves collaborating with the classroom teachers and aides. The lead teacher is also responsible for organizing teacher training and workshops and for coordinating parental participation in the program.

• To maximize individual learning opportunities, preschool class sizes are small, and each classroom has a teacher's aide in addition to a regular classroom teacher. The average teacher-to-child ratio is 1 to 8.

• The smaller class size allows for a child-centered, individualized approach to language development, cognitive development, and improving social relations.

• Each parent is required to volunteer at least one half-day per week at the CPC. Parental participation is designed to accommodate parents' daily schedules and needs.

• A full-time staff member provides outreach services to CPC families. This outreach includes (1) recruiting families from the neighborhood who are most in need of CPC programming; (2) conducting home visits to families upon child enrollment and on a continuing as-needed basis; and (3) referring families to community and social services agencies, such as agencies providing employment training, mental health services, and welfare. The outreach worker provides transportation services to the center for families in need.

• Upon enrollment, all entering children undergo a physical health screening, during which the child’s vision and hearing are tested. This screening is typically done off-site.

• All students receive free breakfast and lunch.

Curriculum

The CPC program has a prescribed literacy curriculum, with other subject areas focusing on developing a particular type of learning style. The CPCs focus on a broad spectrum of activities, including individualized and interactive learning, small-group activities, and frequent teacher feedback. Staffing

The program is staffed by trained, regular classroom teachers and teacher's aides. In addition, each site requires three full-time administrative teachers and a lead teacher. All classroom teachers have

- 90 -

an Illinois Type 04 Early Childhood Certificate. Finally, the centers are supported through the mandatory participation of parents.

Issues to Consider

This program received a "proven" rating, despite the fact that the evaluations used a quasi- experimental design rather than randomly assigning children to treatment or control groups. The sample sizes for the analyses were adequate, and the researchers used rigorous empirical methods. The evaluations demonstrated sizeable and significant gains in several outcome areas over 19 years of follow-up. Potential concerns regarding the comparability of the CPC children and the comparison group are mitigated by the extensive analysis the authors conducted to identify differences between the two groups. The groups were found to be similar on nearly all characteristics at the beginning of the study, the number and type of study participants who dropped out of the study over time were similar in the two groups, and the assignment of participants to the program was largely related to the fact that participating families lived in neighborhoods served by a CPC program (Reynolds et al., 2001; Reynolds and Temple, 1995). Nevertheless, the comparability of the treatment and comparison groups exhibits some weaknesses (Thompson, Reynolds, and Temple, 2001). Families in the CPC "treatment" group voluntarily chose to participate in the program, and parents were required to volunteer in the program regularly. The comparison group included not only these types of families, but perhaps also families who would not have chosen to participate in the program and parents who were not able to provide volunteer time at the CPC. Hence, the comparison group may have included some families who would not have participated in the CPC program had it been offered to them, and these families may be different from the participating families in unmeasured ways. Additional research has found that participation in CPC preschool is associated with significantly better performance on all outcomes noted in this summary, even after site-level factors measuring the influences of the local setting and other family factors were taken into account. Analyses suggest that parent involvement in the program was significantly related to improved outcomes at both the kindergarten and high school levels (Clements, Reynolds, and Hickey, 2004). One area that is left unresolved regarding the CPC program is the question of which services should consistently be delivered as part of the program. Given that the program has no specific guidance mandating that particular activities or curricula are offered, it is not clear which features of the program drive the CPC’s effectiveness.

Example Sites

Chicago

Contact Information

Chicago Public Schools Office of Early Childhood Education 125 South Clark Street, 9th Floor Chicago, IL 60603 Attention: Sonja Griffin Child-Parent Center Program Tel: (773) 553-1958 Fax: (773) 553-2011 Email: sogriffin@cps.k12.il.us Web site: http://www.ecechicago.org/programs/ece/cpc.html

- 91 -

mailto:sogriffin@cps.k12.il.us

http://www.ecechicago.org/programs/ece/cpc.html

Available Resources

Chicago Longitudinal Study, Arthur J. Reynolds, Director, 1998-2002. As of September 29, 2008: www.waisman.wisc.edu/cls/ Newsletter of the Chicago Longitudinal Study, University of Wisconsin-Madison, Waisman Center, Issue 1, August 2000. As of September 29, 2008: www.waisman.wisc.edu/cls/NEWSLETN.PDF Newsletter of the Chicago Longitudinal Study, University of Wisconsin-Madison, Waisman Center, Issue 2, June 2002. As of September 29, 2008: www.waisman.wisc.edu/cls/NEWSLETTER2.PDF Reynolds, Arthur J., Success in Early Intervention: The Chicago Child-Parent Centers, Lincoln, Neb.: University of Nebraska Press, 2000.

Bibliography

Clements, Melissa A., Arthur Reynolds, and Edmond Hickey, "Site-Level Predictors of Children's School and Social Competence in the Chicago Child-Parent Centers," Early Childhood Research Quarterly, Vol. 19, No. 2, 2004, pp. 273-296. Reynolds, Arthur J., The Chicago Child-Parent Centers: A Longitudinal Study of Extended Early Childhood Intervention, discussion paper no. 1126-97, Madison, Wisc.: Institute for Research on Poverty, 1997. As of September 29, 2008: http://www.irp.wisc.edu/publications/dps/pdfs/dp112697.pdf Reynolds, Arthur J., Success in Early Intervention: The Chicago Child-Parent Centers, Lincoln, Neb.: University of Nebraska Press, 2000. Reynolds, Arthur J., "Effects of a Preschool Plus Follow-On Intervention for Children at Risk," Developmental Psychology, Vol. 30, No. 6, 1994, pp. 787-804. Reynolds, Arthur J., "One Year of Preschool Intervention or Two: Does it Matter?" Early Childhood Research Quarterly, Vol. 10, 1995, pp. 1-31. Reynolds, Arthur J., H. Chang, and Judy A. Temple, Evaluation Review, Vol. 22, No. 3, 1998, pp. 341- 372. Reynolds, Arthur J., and Judy A. Temple, "Extended Early Childhood Intervention and School Achievement: Age Thirteen Findings from the Chicago Longitudinal Study," Child Development, Vol. 69, No. 1, 1998, pp. 231-246. Reynolds, Arthur J., and Judy A. Temple, "Quasi-Experimental Estimates of the Effects of a Preschool Intervention: Psychometric and Econometric Comparisons," Evaluation Review, Vol. 19, No. 4, 1995, pp. 347-373. Reynolds, Arthur J., Judy A. Temple, Dylan L. Robertson, and Emily A. Mann, "Age 21 Cost-Benefit Analysis of the Title I Chicago Child-Parent Centers," Educational Evaluation and Policy Analysis, Vol. 24, No. 4, 2002, pp. 267-303. Reynolds, Arthur J., Judy A. Temple, Dylan Robertson, and Emily Mann, "Long-Term Effects of an Early Childhood Intervention on Educational Attainment and Juvenile Arrest," Journal of the American Medical Association, Vol. 285, No. 18, 2001, pp. 2339-2346. Reynolds, Arthur J., Judy A. Temple, Suh-Ruu Ou, et. al., "Effects of a School-Based Early Childhood Intervention on Adult Health and Well-Being: A 19-Year Follow-Up of Low-Income Families," Archives of Pediatric and Adolescent Medicine, Vol. 161, No. 8, August 2007, pp. 730-739.

- 92 -

http://www.waisman.wisc.edu/cls/

http://www.waisman.wisc.edu/cls/NEWSLETN.PDF

http://www.waisman.wisc.edu/cls/NEWSLETTER2.PDF

http://www.irp.wisc.edu/publications/dps/pdfs/dp112697.pdf

Temple, Judy A., Arthur J. Reynolds, and Wendy T. Miedel, "Can Early Intervention Prevent High School Dropout? Evidence from the Chicago Child-Parent Centers," Urban Education, Vol. 35, No. 1, 2000, pp. 31-56. Thompson, Matthew D., Arthur J. Reynolds, and Judy A. Temple, "Letters—Early Childhood Educational Intervention and Long-term Developmental Outcomes," Journal of the American Medical Association, Vol. 286, No. 15, 2001, pp. 1835-1836.

Last Reviewed

September 2008

Class Wide Peer Tutoring Program

Program Info

Outcome Areas Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards

Topic Areas Age of Child Middle Childhood (9-12) Type of Setting Elementary School Middle School Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance

Evidence Level Proven

Program Overview

Class Wide Peer Tutoring (CWPT) was developed during the early 1980s at the Juniper Gardens Children's Project at the University of Kansas, a community-based program devoted to improving the developmental outcomes of children, with or without disabilities, who live in low-income areas. The program addresses both the school and home environments of the children in the program. It is an instructional model based on reciprocal peer tutoring that could be used at any grade level, but has been evaluated primarily for children in kindergarten through sixth grade, with current work being done at the middle school level. On each Monday during the duration of the program, all participating students are individually pre- tested on that week’s classroom material. After pre-testing, students are paired up and each set of partners is assigned to one of two teams. Partners take turns tutoring each other on their spelling, math, and reading passages, and test each other’s learning comprehension by asking questions based on a recited passage. For every correct answer, a tutee is awarded two points. If an incorrect answer is given, the tutor corrects his or her partner. The partner then receives one point for writing the correct answer three times on a tutoring worksheet. After ten minutes, the partners switch roles. At

- 93 -

the end of the daily tutoring session, students report their point totals to the teacher and scores are posted on a Team Point Chart. The team with the most points is announced daily. Team reinforcement is awarded each Friday. CWPT is practiced Monday through Thursday for 30 minutes, including 20 minutes for tutoring and 10 minutes for material preparation. On Friday, students are individually tested on the material presented that week and pre-tested on the material for the upcoming week.

Program Participants

The Class Wide Peer Tutoring program has been used in Grades K through 6. Although developed for regular education students, CWPT has been successfully implemented with learning disabled and educable mentally retarded students. It is now being implemented at the middle school level.

Evaluation Methods

More than 30 evaluations have been done of the Class Wide Peer Tutoring program. The University of Kansas staff and graduate students have conducted most of them, although independent researchers have also evaluated the program. The evaluations have focused primarily on minority inner-city students, although some evaluations have included children from other backgrounds. Several of the evaluations utilized an experimental design that compares treatment group outcomes to those of a control group. Sample sizes for these experiments have ranged from four to more than 400. Other evaluations compared pre- and post-test scores for children without involving a control group. The largest study compared a treatment group on the low end of the socioeconomic scale to a control group of similar socioeconomic status and to a comparison group that was high on the socioeconomic scale.

Key Evaluation Findings

The various project evaluations found that:

• When students began CWPT in the first grade, by the end of the fourth grade they scored more than 11 percentage points higher than control groups on a nationally standardized test in both reading and math (40 percent versus 29 percent in reading, and 49 percent versus 38 percent in math), after test scores were adjusted for differences between the two groups that were determined in the first grade (for example, measured I.Q.).

• CWPT produced average gains of 12 percentage points on spelling tests among third and fourth graders, with 80 percent of the students receiving grades in the A range (90% and higher).

• Children were 20 to 70 percent more likely to stay on task, remain engaged with their lessons, and respond to the teacher during CWPT than they were before the program.

• On average, first graders tested above the second-grade level on comprehension and vocabulary using the Gates-MacGinitie Reading Test, with a class average of second grade, fourth month in comprehension and second grade, seventh month in vocabulary after five months of CWPT.

• An experimental group of children in elementary schools in economically depressed areas performed almost as well as a comparison group of children from higher socioeconomic groups and performed significantly better than a control group of students from other elementary schools in economically depressed areas who did not receive CWPT.

Probable Implementers

• Any school district or classroom—CWPT can be adapted to any basic program (reading, vocabulary, math, and others) currently being utilized by the district.

- 94 -

• School districts looking to improve their regular classrooms, their resource rooms (such as reading laboratories), and their classroom settings specifically for mildly mentally retarded and behaviorally disturbed students.

• School districts struggling with how to teach regular and mainstreamed students who are in same learning environment.

Funding

Juniper Gardens was founded in 1964 and has received continuous funding through competitive research and training grants from the National Institutes of Health and the U.S. Department of Education, among others. The training manual (titled Together We Can) is available at a low cost so that classrooms can implement this program with a minimal outlay of funds. For teachers wishing to undergo the Juniper Gardens training program, funding for program materials may be available from their school or district. However, the program probably can be implemented at little or no cost using existing materials.

Implementation Detail

Program Design

• Pre-tests on Monday and post-tests on Friday covering the material for the week give an indication of a student’s progress and mastery of skills.

• The program can be implemented with existing oral reading comprehension questions, spelling words, reading workbooks, vocabulary words, math problems, or almost any subject matter.

Curriculum

A curriculum book is available from Juniper Gardens that explains how to group the class into pairs of students, one of whom is the tutor and one of whom is tutored, who work together on competing teams. However, the program could be implemented without the prescribed curriculum fairly easily in any classroom setting. Staffing

A Juniper Gardens consultant trains classroom teachers. After teachers are trained, classrooms can implement the program.

Issues to Consider

This program received a “proven” rating. Many controlled experiments have been conducted that demonstrate the effectiveness of Class Wide Peer Tutoring. Most of the experiments have tested the use of the program with minority inner-city students. However, the program has been used with regular education, learning disabled, autistic, and educable mentally retarded students. In all cases, students showed large gains in knowledge as compared with control groups. The program has been shown to be effective with children as young as first graders and those effects were shown to last for at least three years after students received the training. In addition, the program can be implemented at little or no cost to a school district, so it can be a very useful program for school districts with little funding for extra programs.

Example Sites

Contact Barbara Terry at the University of Kansas for specific example sites. phone: (913)321-3143 Internet: http://www.lsi.ku.edu

- 95 -

http://www.lsi.ku.edu/

Contact Information

Barbara Terry, Ph.D. 650 Minnesota Ave, 2nd Floor Kansas City, KS 66101 phone: (913) 321-3143 fax: (913) 371-8522 e-mail: terryb@ku.edu

Available Resources

One-day or two-day workshops are available through the Juniper Gardens Children’s Project. There are two options: one full-day session with two consultants for $1,500 or one half-day session with two consultants for $800. The training workshops provide instruction on how to implement the program and techniques on how to adapt current material. It is recommended that each participant purchase a copy of the CWPT training manual Together We Can. The Together We Can manual is available from SOPRIS West for $29.50 at: SOPRIS West P.O. Box 1890 Longmont, CO 80502-1802 phone: 1-800-547-6747 fax: 303-776-5934 internet: http://www.sopriswest.com Juniper Gardens Children's Project home page can be found at: http://www.jgcp.ku.edu

Bibliography

Delquadri, J., C. R. Greenwood, D. Whorton, J. J., Carta, and R. V. Hall “Classwide Peer Tutoring” Exceptional Children, Vol. 52, pp. 535-542, 1986. Greenwood, C. R., J. Delquadri, and R. V. Hall “Longitudinal Effects of Classwide Peer Tutoring” Journal of Educational Psychology, Vol. 81, pp. 371-383, 1989. Heron, T. E., W. L. Heward, N. L. Cooke, and D. S. Hill, Evaluation of Class-Wide Peer Tutoring System: First Graders Teach Each Other Sight Words, Education and Treatment of Children, Vol. 6, No. 2, pp. 137-152, 1983. Kohler, F. W., and C. R. Greenwood Effects of Collateral Peer Supportive Behaviors Within the Class- Wide Peer Tutoring Program, Journal of Applied Behavior Analysis, Vol. 23, No. 3, pp. 307-322, 1990. Sideridis, G. D., C. Utley, C. R. Greenwood, J. Delquadri, et al., Class-wide Peer Tutoring: Effects on the Spelling Performance and Social Interactions of Students with Mild Disabilities and Their Typical Peers in an Integrated Instructional Setting, Journal of Behavioral Education, Vol. 7, No. 4, pp. 203- 212, 1997.

Last Reviewed

January 2003

- 96 -

mailto:terryb@ku.edu

http://www.sopriswest.com/

http://www.jgcp.ku.edu/

Cognitive Behavioral Intervention for Trauma in Schools (CBITS)

Program Info

Outcome Areas Healthy and Safe Children Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards Children not experiencing anxiety or mood disorders, such as depression

Topic Areas Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School Type of Service Health Care Services Parent Education Youth Development Type of Outcome Addressed Behavior Problems Cognitive Development/School Performance Mental Health

Evidence Level Proven

Program Overview

The Cognitive Behavioral Intervention for Trauma in Schools (CBITS) program is a group intervention for children in grades six through nine. The program is aimed at relieving symptoms of post-traumatic stress disorder (PTSD), depression, and general anxiety among children exposed to trauma. Types of traumatic events that participants have experienced include witnessing or being a victim of violence, experiencing a natural or man-made disaster, being in an accident or house fire, or suffering physical abuse or injury. Symptoms of PTSD, depression, and anxiety among children can include disorganized or agitated behavior, recurrent distressing recollections of an event, nightmares, attempts to reenact an event, intense psychological or physiological distress from cues that symbolize an aspect of the event, diminished interest or participation in certain activities, feelings of detachment or estrangement, difficulty falling asleep or staying asleep, and outbursts of anger. Children in CBITS work on processing traumatic memories, expressing their grief, learning relaxation skills, challenging upsetting thoughts, and improving their social problem-solving. These techniques and skills are learned through the use of drawings and through talking in both individual and group settings.

Program Participants

The CBITS program has been used most commonly for children in grades six through nine. Preliminary versions of CBITS have been used for children as young as eight years old. More recently, CBITS has been used in high school, although these efforts have not been evaluated.

- 97 -

Evaluation Methods

Kataoka et al. (2003) evaluated the CBITS program in a sample of 198 Spanish-speaking immigrant students in grades three through eight, with 152 students in the CBITS group and 46 students in the control group. Eleven public schools in Los Angeles, California, were invited to participate in the study, and nine schools agreed to participate. A total of 970 students met eligibility criteria to participate (i.e., were in grades three through eight, were foreign-born, had immigrated to the United States within the past three years, and spoke Spanish). Ninety-one percent of the sample (879 students) completed a questionnaire regarding exposure to violence and symptoms of trauma. Thirty-one percent of the screened students (276 children) reported clinical PTSD and/or depression symptoms and were recruited for the study, and 83 percent (229 students) of the students who reported these symptoms were given parental permission to participate. A total of 67 students were randomly assigned to the treatment group, and 46 students were assigned to a waitlist comparison group. Waitlist students were given referrals to community mental health agencies, although most subjects did not follow up on these referrals. Later in the school year, an additional 85 eligible students were non-randomly assigned to the intervention, for a total of 152 children participating in the CBITS intervention and 46 in the waitlist control group. The randomized and nonrandomized children did not differ on baseline violence exposure, symptom levels, or socioeconomic characteristics except for a significant difference in parental education (which was higher in the nonrandomized group). All students completed a three-month follow-up assessment for symptoms of childhood PTSD and depression. A second study of the CBITS program was conducted by Stein et al. (2003), in a sample of English- speaking sixth-grade students from two middle schools in East Los Angeles. After parents and children agreed to participate in the study, a self-report questionnaire regarding exposure to violence and symptoms of PTSD was given to 769 students. Students eligible for participation in the study had the following characteristics: substantial exposure to violence, symptoms of PTSD in the clinical range, symptoms of PTSD related to exposure to violence that they were willing to discuss in a group, and were not too disruptive to participate in a group-therapy intervention session. One hundred fifty-nine students were offered the opportunity to participate, and 126 students were randomly assigned to either a ten-session CBITS group (61 students, with 56 eventually participating) or a waitlist control group (65 students). Students were assessed before the intervention and at three and six months post-completion on measures of PTSD, depression, parent-reported psychological dysfunction, and teacher-reported classroom problems. At follow-up, 54 CBITS and 63 control students were included in the three-month analysis (117 total, or 93 percent), and 53 CBITS and 60 controls were included in the six-month analysis (113 total, or 90 percent). At baseline, compared with students who completed all assessments, non-completers (13 students) had significantly higher PTSD and depression scores, acting-out behaviors witnessed in the classroom, and classroom learning problems. Kataoka et al. (2011) further used their sample of sixth graders in East Los Angeles to look at the effects of CBITS on academic performance of students with mental health symptoms. Out of 159 students with PTSD symptoms according to the Child PTSD Symptom Scale (CPSS), 126 were randomly assigned to receive CBITS immediately or with a delay of 4-5 months after initial screening (61 immediate, 65 delayed). The randomized children did not differ on baseline violence exposure, symptom levels, or socioeconomic characteristics. After assigning students to either the immediate or delayed intervention group, their academic performance was recorded at baseline and closely monitored throughout the intervention period. The emphasis was on mean math and language/arts grades, as well as on the percentage of passing grades for each group. Grades were recorded during the spring quarter of the 2001-2002 academic year, as this was the only period when the immediate group had completed the intervention and the delayed group had not started. The study control group also received the program during the same academic year, and there was no longer-term tracking of changes in academic performance for a period exceeding the experiment time frame. Because of these issues, it was impossible to examine long-term differences in academic performance.

- 98 -

Key Evaluation Findings

Kataoka et al.'s (2003) study of 198 Spanish-speaking immigrant students found the following:

• Depressive symptoms in the CBITS group significantly decreased from pre- to post-test (by 17 percent) but did not change in the waitlist group.

• Similarly, PTSD symptoms in the CBITS group significantly decreased from pre- to post-test (by 29 percent), but the reduction in the waitlist group of 13 percent was not statistically significant.

• Of the 83 students with clinical depressive symptoms at baseline (i.e., serious levels of depression), mean depression scores for the CBITS group dropped significantly at post-test (by 22 percent), compared with a non-significant drop of 5 percent in the waitlist group.

• Similarly, of the 180 children with clinically significant PTSD symptoms at baseline (i.e., serious levels of PTSD), follow-up scores declined significantly in the treatment group (by 35 percent), compared with a non-significant decline of 16 percent in the waitlist group.

The study of 117 English-speaking sixth-graders (Stein et al., 2003) reported the following:

• At the three-month follow-up, CBITS students reported a significantly greater reduction of PTSD symptoms than did waitlist control students, with a 64 percent reduction from baseline compared with a 34 percent reduction from baseline.

• At the three-month follow-up, CBITS students also reported significantly lower scores on symptoms of depression than did control students, with a 47 percent reduction from baseline compared with a 24 percent reduction from baseline.

• Parents of CBITS students reported significantly less psychosocial dysfunction of their children at three months compared with parents of waitlist control students. CBITS parents reported a 35 percent reduction in psychosocial dysfunction in their children from baseline, compared with a 2 percent increase for control parents.

• At three months, no significant differences were found between the two groups for teacher- reported classroom problems of acting out, or problems with shyness, anxiousness, or learning.

• At six months, after both the initial group and the waitlist group had received the CBITS intervention, no significant differences were found between groups for symptoms of PTSD, depression, parent-reported psychosocial function, or teacher-reported classroom behaviors.

The study on the academic performance of 126 sixth graders (Kataoka et al., 2011) reported the following:

• Students in the immediate intervention group show higher mean grades in both math (2.0 vs. 1.6) and language/arts (2.2 vs. 1.9), after adjusting for prior test scores, than their counterparts in the delayed intervention group. However, only the difference in math mean scores was statistically significant.

• Students with early CBITS intervention were projected to pass both math (69.5%) and language/arts (79.7%) at significantly higher rates than students with delayed intervention (math: 54.7%; language/arts: 60.9%)

Probable Implementers

Public and private upper elementary, middle schools and high schools

- 99 -

Funding

Previously, schools have successfully implemented CBITS using existing mental health funding, insurance, County Department of Mental Health funding (including Medicaid), or monies from Safe and Drug Free Schools to support the program.

Implementation Detail

Program Design

The CBITS program consists of ten one-hour group sessions with five to eight children, usually conducted once a week in a school or mental health or other office settings. The group sessions include exercises related to six cognitive-behavioral areas:

• education about common reactions to trauma

• relaxation training to combat anxiety

• cognitive therapy (developing an understanding of the link between thoughts and feelings; combating negative thoughts)

• gradual exposure to trauma reminders

• exposure to stress or trauma memory through use of the imagination, drawing, or writing

• social problem-solving.

Additional between-session activities that each student does as homework help strengthen his or her skills and allow group members to apply those skills to real-life problems. In addition to the group sessions, participants receive one to three individual sessions. Furthermore, the CBITS program includes two parent-education sessions and one teacher-education session to help adults to assist children in solidifying the skills learned during the program. Curriculum

Reduction in symptoms of PTSD and depression is accomplished in CBITS via cognitive techniques (e.g., relaxation therapy and real-life exposure to traumatic cues) and trauma-focused memory work using the imagination, writing, and drawing. In each session, a new set of skills is taught to children through the use of age-appropriate examples and games. Participants then use those skills to address their problems through homework assignments collaboratively developed by the child and the CBITS social worker. Staffing

Typically, CBITS group leaders are trained clinicians, including social workers, marriage and family counselors, and psychologists, who attend a two-day training session and receive ongoing supervision from a local clinician with expertise in cognitive-behavioral therapy.

Issues to Consider

This program received a "proven" rating. The first two studies of the CBITS program (Kataoka et al., 2003; Stein et al., 2003) utilized rigorous evaluation methodology, studied groups with low rates of attrition, and found significant reductions in PTSD and depression symptoms for treatment children when compared with a control group. Additionally, the CBITS program demonstrated effectiveness at reducing parent-reported psychosocial dysfunction among participating children. However, teacher- reported classroom behavior problems did not decline as a result of the program.

- 100 -

The third study (Kataoka et al., 2011) examines the impact of CBITS on issues beyond PTSD and looks at the program's effect on academic performance. While results are very promising, there are several limitations to the study design that make the preliminary.

Although the evaluations of CBITS have been limited to Los Angeles, studies indicate that the program is effective with English-speaking students as well as Spanish-speaking immigrant children. There have been various attempts to look at CBITS in different settings, notably post-Katrina New Orleans (Jaycox et al., 2010) and tribal reservations, yet none of these studies fully meet the criteria to be presented in this context (Morsette et al., 2009). While they provide several interesting insights into the external validity of CBITS components beyond the borders of Los Angeles, the results are exploratory and need to be tested further. It should be noted that the program developers served as authors on all three evaluations of the CBITS program.

Example Sites

Los Angeles, California New Orleans, Louisiana Chicago, Illinois San Francisco, California

Contact Information

Lisa H. Jaycox RAND Corporation 1200 South Hayes Street Arlington, VA 22202 Telephone (703) 413-1100 x5118 Fax (703) 414-4726 jaycox@rand.org

Available Resources

The CBITS treatment manual is available for purchase through Sopris West Educational Services at (800) 547-6747 or online at: https://cbitsprogram.org/survey/take/s=244&c=281&f=0#/g=1927&i=1&r=168416&h=298adc&t=22 7

Bibliography

Jaycox, Lisa H., Judith A. Cohen, Anthony P. Mannarino, Douglas W. Walker, Audra K. Langley, et al., "Children's Mental Health Care Following Hurricane Katrina: A Field Trial of Trauma Focused Psychotherapies," Journal of Traumatic Stress, Vol. 23, No. 2, April 2010, pp. 223-231. Kataoka, Sheryl H., Bradley D. Stein, Lisa H. Jaycox, Marleen Wong, Pia Escudero, Wenli Tu, Catalina Zaragoza, and Arlene Fink, "A School-Based Mental Health Program for Traumatized Latino Immigrant Children," Journal of the American Academy of Child and Adolescent Psychiatry, Vol. 42, No. 3, March 2003, pp. 311-318. Kataoka, Sheryl, Lisa H. Jaycox, Marleen Wong, Erum Nadeem, Audra Langley, Lingqi Tang, and Bradley D. Stein, "Effects on School Outcomes in Low-Income Minority Youth: Preliminary Findings from a Community-Partnered Study of a School Trauma Intervention," Ethnicity & Disease, Vol. 21, No. 3, Supplement 1, 2011, pp. S1-71-7.

- 101 -

mailto:jaycox@rand.org

https://cbitsprogram.org/survey/take/s=244&c=281&f=0%23/g=1927&i=1&r=168416&h=298adc&t=227

Morsette, Aaron, Swaney, Gyda, Stolle, Darrell, Schuldberg, David, van den Pol, Richard, and Young, Melissa. "Cognitive Behavioral Intervention for Trauma in Schools (CBITS): School-Based Treatment on a Rural American Indian Reservation," Journal of Behavior Therapy and Experimental Psychiatry, Vol. 40, No. 1, March 2009, pp. 169-178. Stein, Bradley D., Lisa H. Jaycox, Sheryl H. Kataoka, Marleen Wong, Wenli Tu, Mare N. Elliott, and Arlene Fink, "A Mental Health Intervention for Schoolchildren Exposed to Violence: A Randomized Controlled Trial," Journal of the American Medical Association, Vol. 290, No. 5, August 6, 2003, pp. 603-611.

Last Reviewed

June 2013

Cognitive Relaxation Coping Skills

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children and youth not engaging in violent behavior or displaying serious conduct problems Children not experiencing anxiety or mood disorders, such as depression

Topic Areas Age of Child Middle Childhood (9-12) Type of Setting Elementary School Middle School Type of Service Health Care Services Youth Development Type of Outcome Addressed Behavior Problems Mental Health Violent Behavior

Evidence Level Promising

Program Overview

The Cognitive Relaxation Coping Skills (CRCS) program targets heightened cognitive, emotional, and physiological sensations, with a goal of increasing children's ability to control their emotions. Students are taught methods for relaxation and for attitude change and how to use those skills to control their feelings of anger. The premise of the program is that as students apply these skills and learn how to relax, or if they avoid becoming angry in the first place, they are better able to deal effectively with frustrating situations.

Program Participants

Students in grades 6 through 8 who have expressed a high degree of anger

- 102 -

Evaluation Methods

The CRCS program was evaluated in a single study over three years in the 1990s (Deffenbacher et al., 1996). In the first year, all seventh- and eighth-graders in a middle school in Oregon were screened on the Trait Anger Scale (TAS), a self-report measure that asks students to rate how angry they generally feel, e.g., being "hotheaded" or becoming angered by being criticized in front of others. High trait anger is a personality trait that reflects a tendency to experience frequent and intense episodes of anger. In the second year, all sixth- and seventh-graders in the same school were screened on the TAS (eighth-graders were not assessed because they were part of the prior year’s sample). In the third year, all eighth-graders in a junior high in Colorado were screened on the TAS. A total of 694 students were screened over the three-year period. The 178 students who scored in the upper quartile on the TAS were asked for parental permission to participate in the study. Of the 178 potential participants, 11 left the school or were unavailable before the project started, four moved or could not be assessed at follow-up, eight parents refused to consent, and 35 parents did not return the consent form. The final participating sample consisted of 120 high-anger students who were randomly assigned to CRCS (39 students), to an alternative Social Skills Training (SST) program (40 students), or to a no-treatment control group (41 students). The SST group used a curriculum that emphasized social-communication skills as a way to reduce anger, while the no-treatment control students did not receive special programming or special attention. Fifty-three percent of the sample was male, 78 percent was Caucasian, 19 percent was Latino, 2 percent was Asian, and 1 percent was Native-American. Sixteen percent of the students were in the sixth grade, 32 percent were in the seventh grade, and 53 percent were in the eighth grade. Prior to implementation of the treatment programs, students were assessed on a variety of measures to determine their level of anger. Along with the TAS, students completed the Anger Rating Scale (which measured general reactions of anger, such as losing one’s temper or wanting to hit someone); the Anger Situation Rating (in which students described the three situations that had made them the angriest and rated the degree of anger created by each situation); and the Anger-In, Anger-Out, and Anger-Control Scales (which assessed students’ styles of dealing with anger--suppressing feelings of anger, outwardly negative expressions of anger, or calm, controlled, and socially appropriate behaviors). In addition, students’ anxiety levels were assessed on two scales--the Trait Anxiety Inventory (in which students rated how anxious they generally feel, e.g., nervous and restless, and the degree to which they feel like a failure), and the Anxiety Rating Scale, which measures general anxiety and worry. Symptoms of depression were assessed according to the Depression Rating Scale, which asked students whether they felt sad, low, or lonely, for example. General deviant behavior was assessed by asking students how typical it was for them to steal, lie, or cheat, while school deviance was measured by asking students whether any one of seven problems (disciplined by a teacher, disciplined by the principal, suspension, tardy, detention, truant, or getting into a fight at school) had occurred in the past two months. Finally, alcohol use was assessed by asking students how many times in the past month they had used alcohol and how many times they had gotten drunk. Three groups, each consisting of 12 to 14 students, were in each treatment condition. Groups met for nine 45-minute sessions during regular school hours. Attendance rates were high, with students attending eight of nine sessions on average. A follow-up assessment took place approximately eight weeks post-treatment. The content of the posttest assessment was identical to the content of the pretest assessment except that students re- rated the same three Anger Situations that they described at pretest, instead of describing new anger situations. Analysis of pretest scores indicated that there were no significant differences between the two schools; therefore, data for analysis were collapsed across schools. Comparisons of pretest scores also indicated no significant differences among the three treatment groups.

- 103 -

Key Evaluation Findings

The study by Deffenbacher et al. (1996) reported the following:

• Analyses of the anger measures revealed significant positive treatment effects on all anger instruments, with the exception of the Anger Control scale. Significant positive effects for the treatment groups were found on the Trait Anger Scale, the Anger Rating Scale, the Anger Situation Rating, the Anger-In Scale, and the Anger-Out Scale. A consistent pattern of between-group differences was found for the anger measures; in all cases, the CRCS and SST groups differed significantly from the control group but did not differ significantly from one another.

• On the Anxiety Rating Scale, the CRCS and SST groups also scored significantly better than the control group and did not differ from one another.

• On the Trait Anxiety Inventory, CRCS participants reported significantly less trait anxiety than control students, whereas the SST group fell between the CRCS and control groups and did not differ significantly (in a statistical sense) from either group.

• Similarly, for depression, CRCS participants reported significantly less depression than control students, while the SST group did not differ significantly from either group.

• Similarly, for general deviance, CRCS participants reported significantly less general deviance than members of the control group, while SST participants were not significantly different from either group.

• Similarly, for the measure of deviance at school, CRCS students reported significantly less school deviance problems than either SST or control students, who did not differ from one another.

• No significant treatment effect was found for the alcohol-use variables, although it should be noted that students' use of alcohol pretest and posttest was low, making it more difficult to show treatment effects.

Probable Implementers

Public and private elementary and middle schools

Funding

None at this time

Implementation Detail

Program Design

The CRCS program originally was developed by Deffenbacher and colleagues and had targeted late adolescents and adults. For the current study, the program was adapted for middle-school students. To ensure treatment adherence, interventions were based on written outlines that specified the activities for each session and were discussed in detail with program implementers prior to and after each session.

Curriculum

The CRCS program includes nine 45-minutes sessions. The content of the sessions is as follows:

• A portion of the first session is devoted to developing a list of anger-provoking situations and how students had previously reacted angrily to those situations. The remainder of the first session and the second session are devoted to progressive relaxation training and to the development of specific relaxation coping skills, including cue-controlled relaxation, breathing-

- 104 -

cued relaxation, and relaxation without tension. Homework includes anger self-monitoring and practice of relaxation skills.

• The third and fourth sessions focus on how cognitive processes influence anger and how changing those processes can lessen anger. Students are asked to imagine a provocative situation and to become as angry as possible in reaction to the situation. Students report on the thoughts they are having that are eliciting the angry feelings. Students are then asked to visualize the situation again, while trying to remain as calm as possible. Students then report back on their thoughts, which are contrasted with the thoughts they were having in the first part of the exercise. Homework includes self-monitoring of cognitive processes and practicing relaxation skills.

• The first half of each of the last five sessions is spent introducing a specific provocative situation identified in the first session and discussing specific cognitive changes that reduce feelings of anger. These discussions are made to be lively and interactive and include role- playing with the group leader. The second half of each session is spent rehearsing cognitive- adjustment exercises and relaxation-based coping skills. Students visualize an angering event, experience anger arousal, and then rehearse specific cognitive-adjustment exercises and relaxation-based coping strategies.

• At the end of the sixth session, the cognitive strategies are translated into ten basic strategies (e.g., self-instruction to "stop," to "think calmly," to use "no put-downs or name calling," or to "problem-solve"), and a list of those strategies is distributed to the participants. Homework includes self-monitoring and applying cognitive-adjustment exercises and relaxation-based coping skills to anger and other distressing feelings, such as test anxiety.

Staffing

In the program, as it was evaluated, the sessions were led by either a master’s-degree-level school psychologist with more than ten years of experience providing psychological services in schools, or by an advanced doctoral student with several years of experience providing psychological services to youth.

Issues to Consider

This program received a "promising" rating. Significant treatment effects were found for reducing children’s levels of anger, depression, anxiety, and delinquent behavior. While the evaluation had certain strengths, such as the use of a randomized controlled design and a relatively large sample size, it also suffered from several weaknesses. One was that the program developer also served as the program evaluator. In addition, the study did not incorporate any long-term follow-up, so it is unknown whether the observed CRCS benefits lasted beyond eight weeks after program completion. Furthermore, the assessments relied only on student self-reports, rather than also including parent, teacher, or peer ratings. Finally, the program has been evaluated in only two schools, which have a predominantly white population, limiting the generalizability of findings to other populations. It is important to note that the positive findings of the CRCS evaluation are relevant only for high- anger youth. The study sample consisted of children who scored in the upper quartile on the Trait Anger Scale, based on distributions in the participating schools. While study results suggest that the program is successful at reducing levels of anger, anxiety, depression, and delinquent behavior, the program is targeted toward youth displaying high levels anger at baseline, and the program may not be as effective in other populations. Furthermore, while Deffenbacher and his colleagues reported outcomes for both anxiety and depression, they did not discuss the clinical significance of children's pretest scores on the measures of anxiety and depression. It is unknown whether students had clinically significant levels of anxiety or depression to begin with; thus, the observed significant decrease in the scores on these measures may not be meaningful from a practical sense. It is also worth noting that while youth who participated in the CRCS treatment group improved relative to students who received no such services, the outcomes of the SST program group had

- 105 -

improved as much as those for the CRCS group on many measures. The Promising Practices Network plans to review the SST program in the near future.

Example Sites

Oregon and Colorado

Contact Information

Dr. Jerry L. Deffenbacher Department of Psychology Colorado State University Fort Collins, CO 80523 Tel: (970) 491-6871 e-mail: Jerry.Deffenbacher@ColoState.edu

Available Resources

None at this time

Bibliography

Deffenbacher, Jerry L., Rebekah S. Lynch, Eugene R. Oetting, and Calvin C. Kemper, "Anger Reduction in Early Adolescents", Journal of Counseling Psychology, Vol. 43, No. 2, 1996, pp. 149 – 157.

Last Reviewed

June 2006

Cognitively Guided Instruction (CGI)

Program Info

Outcome Areas Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Elementary School Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance

Evidence Level Promising

- 106 -

mailto:Jerry.Deffenbacher@ColoState.edu

Program Overview

Cognitively Guided Instruction (CGI) is a professional development program that increases teachers’ understanding of the knowledge that students bring to the math learning process and how they connect that knowledge with formal concepts and operations. Developed by education researchers Thomas Carpenter, Elizabeth Fennema, Penelope Peterson, Megan Loef Franke, and Linda Levi, CGI is guided by two major theses. The first is that children bring an intuitive knowledge of mathematics to school with them and that this knowledge should serve as the basis for developing formal mathematics instruction in primary school. This thesis leads to an emphasis on assessing the processes that students use to solve problems. The second thesis is that math instruction should be based on the relationship between computational skills and problem solving, which leads to an emphasis on problem solving in the classroom instead of the repetition of number facts (e.g., practicing the rules of addition and subtraction).

Program Participants

Students in kindergarten through sixth grade

Evaluation Methods

In 1989, Carpenter et al. studied the effects of CGI in a sample of 40 first-grade teachers from 24 schools located in Madison, Wisconsin, and in four smaller communities near Madison. Twenty teachers were randomly assigned to the CGI program (by school), and 20 were assigned to the comparison group. Twelve first grade students (six girls and six boys) were selected randomly from each class to serve as target students for analysis (excluding those children with special learning needs). In the two schools in which there were fewer than 12 first grade students in the classroom (due to first/second grade combinations), all first grade students were included in the sample. Student outcomes were evaluated using the computations and problem-solving subtests of the Iowa Test of Basic Skills (ITBS), along with a problem-solving instrument developed by the CGI research team. Pretest results were collected in September and posttests in April/May. A second study (Villasenor and Kepner, 1993) examined CGI in a large Midwestern urban school district. Two first-grade teachers were voluntarily recruited from six schools in which there was at least a 50 percent minority population, resulting in a total of 12 CGI classrooms. Comparison schools were then identified that matched the treatment schools’ population characteristics, and 12 classrooms were selected to match the treatment classrooms. The CGI teachers participated in a 19- hour summer workshop, a 2-hour review in September, and two additional support sessions. Comparison teachers participated in a staff development program that focused on problem solving in elementary school mathematics but did not include CGI principles or research. To compare group outcomes, 12 students were randomly chosen from each class (six boys and six girls), for a total of 144 students in the CGI group and 144 in the control group. Student outcomes were assessed on a 14-item arithmetic word-problem test (Carpenter et al., 1989), which was administered to students in early October and again five months later. In addition, students were interviewed individually to assess the processes and strategies they used to solve both written (word) problems and number- facts problems. An analysis of pretest scores indicated that the CGI group significantly outscored the control group on the written-problem solving pretest; therefore analysis of posttest outcomes were conducted using pretest scores as covariates.

Key Evaluation Findings

The Carpenter et al. (1989) study of 20 CGI teachers and 20 control teachers found the following:

• CGI students scored significantly higher than control students on the complex mathematics addition and subtraction portion of the ITBS; with a mean score of 8.6, compared with 7.8, out of a possible score of 12.

- 107 -

• Similarly, CGI students scored significantly higher than control students on the ITBS problem- solving interview, with an average score of 5.61 versus 5.38, out of a possible score of 6.

• CGI students also significantly outscored controls on the ITBS number-facts problem scale, with a mean score of 2.26 versus 0.78, out of a possible score of 5.

• No significant differences were found between groups for the ITBS problems, advanced problems, or computation scales.

Villasenor and Kepner's (1993) study of 144 CGI students and 144 control students reported:

• CGI students scored significantly higher than control students on the written problem-solving test (9.41 versus 3.18, out of a possible score of 14).

• CGI students significantly outscored control students on the word problems portion of the interview (5.44 versus 2.93, out of a possible score of 6).

• The CGI students also scored significantly higher than control students on the number-facts section of the interview (4.68 versus 3.00, out of a possible score of 5).

Probable Implementers

Public and private elementary schools

Funding

CGI for kindergarteners was originally developed with funding from the National Science Foundation. CGI for older elementary school students (Integrating Arithmetic and Algebra) was developed with funding from the National Science Foundation and the U.S. Department of Education's Office of Educational Research and Improvement through the National Center for Improving Student Learning and Achievement in Mathematics and Science.

Implementation Detail

Program Design

CGI provides a basis for identifying what is difficult and what is easy for students to comprehend in their study of math. It also provides a way for dealing with the common errors students make while learning. The emphasis is on what children can do, rather than on what they cannot do, which leads to a very different approach regarding incorrect answers. With the CGI approach, teachers focus on what students know and help them build future understanding based on present knowledge. The program aims to improve children's mathematical skills by increasing teachers' knowledge of students' thinking, by changing teachers' beliefs regarding how children learn, and by ultimately changing teaching practices. In 1996, CGI was extended into the upper elementary school levels to assist first through sixth grade teachers in integrating the major principles of algebra into arithmetic instruction. The program is based on the premise that children throughout the elementary grades are capable of learning powerful unifying ideas of mathematics that are the foundation of both arithmetic and algebra. Learning and articulating these ideas enhance children's understanding of arithmetic and provide a foundation for extending their knowledge of arithmetic to the learning of algebra.

Curriculum

There is no set curriculum. Teachers use the CGI framework with existing curriculum materials, or they use CGI principles to help develop their own math curriculum.

- 108 -

Staffing

Training for CGI includes an initial workshop for teachers led by program developers or experienced CGI teachers, along with additional support that often lasts for several years in the form of booster workshops, technical assistance, and mentoring.

Issues to Consider

This program received a "promising" rating. Evaluations demonstrated significantly superior mathematics achievement for CGI students when compared with control students on several outcomes. While the Carpenter et al. (1989) study reported significant differences between CGI and control students when looking at performance on smaller subgroups of test problems, there were no significant differences between the treatment and control groups on the total score of the ITBS. CGI students scored significantly higher than control group students on problems identified as number- facts problems or complex addition/subtraction problems, but not on simple addition/subtraction or advanced problems. This finding suggests that CGI may be limited to improving student outcomes only on certain types of mathematics skills. This outcome is of particular interest because the ITBS is the only measure used in any of the evaluations that was not designed by the program developers. The evaluations also imply that successful implementation of CGI requires a substantial amount of training and supervision. The success of the first two pilot studies was linked to intensive involvement with staff trainers, who were often the original program developers. While there have been several attempts at scaling up CGI, none of those efforts has included systematic evaluations of the effectiveness of the program in terms of the children’s math performance.

Example Sites

CGI training programs have been established in Wisconsin, Minnesota, Texas, North Carolina, Arizona, and Ohio. There are also several start-up programs in Phoenix, Arizona; Fargo, North Dakota; Dearborn and East Lansing, Michigan; and other sites in California, Alaska, and New Zealand.

Contact Information

Linda Levi University of Wisconsin-Madison Wisconsin Center for Education Research 1025 West Johnson Street Madison, WI 53706 phone: (608) 263-4267 email: llevi@wisc.edu

Available Resources

Three CGI publications are available to assist teachers with implementation of the program. Available through the National Council of Teachers of Mathematics (NCTM) Web site (http://my.nctm.org/), the first (Carpenter et al., 1999) is designed to help teachers understand children's intuitive mathematical thinking and how students can build up their concepts from within. The guide and accompanying CDs provide a framework for assessing children's thinking in whole number arithmetic and allow readers to look inside real classrooms implementing the CGI technique.

• Carpenter, Thomas P., Elizabeth Fenneman, Megan Loef Franke, Linda Levi, and Susan B. Empson, Children's Mathematics: Cognitively Guided Instruction, The National Council of Teachers of Mathematics, Inc., 1999, Heinemann: Portsmouth, NH.

- 109 -

mailto:llevi@wisc.edu

http://my.nctm.org/

Also available through NCTM, Carpenter et al. (2000) presents a CGI professional development program—a series of workshops along with work in classrooms with children—in which participants learn about the CGI structured framework of mathematics, children's thinking, and the way children's thinking evolves.

• Carpenter, Thomas P., Elizabeth E. Fennema, Linda Levi, Megan Loef Franke, and Susan B. Empson, Children's Mathematics: Cognitively Guided Instruction—A Guide for Workshop Leaders, The National Council of Teachers of Mathematics, Inc., 2000, Heinemann: Portsmouth, NH.

The third publication (Carpenter, Franke, and Levi, 2003) is available through Heinemann (http://books.heinemann.com/products/E00565.aspx), and it provides a foundation for the teaching and learning of algebra.

• Carpenter, Thomas P., Megan Loef Franke, and Linda Levi, Thinking Mathematically: Integrating Arithmetic and Algebra in Elementary School, The National Council of Teachers of Mathematics, Inc., 2003, Heinemann: Portsmouth, NH.

Bibliography

Carpenter, Thomas P., Elizabeth Fennema, Penelope L. Peterson, Chi-Pang Chiang, and Megan Loef, "Using Knowledge of Children's Mathematics Thinking in Classroom Teaching: An Experimental Study," American Educational Research Journal, Vol. 26, No. 4, 1989, pp. 499-531. Villasenor, Albert, Jr., and Henry S. Kepner, Jr., "Arithmetic from a Problem-Solving Perspective: An Urban Implementation," Journal for Research in Mathematics Education, Vol. 24, No. 1, 1993, pp. 62- 69.

Last Reviewed

January 2007

Communities In Schools

Program Info

Outcome Areas Children Succeeding in School

Indicators Students graduating from high school

- 110 -

http://books.heinemann.com/products/E00565.aspx

Type of Service Case Management Family Support Youth Development Type of Outcome Addressed Cognitive Development/School Performance

Evidence Level Promising

Program Overview

Communities In Schools, Inc. (CIS) was founded in 1977 to help adolescents stay in school and graduate from high school. The CIS model assumes that young people in jeopardy of dropping out of school generally have both academic and non-academic problems that need to be addressed. The CIS model is also based on the premise that existing social services are delivered in a fragmented and uncoordinated manner. CIS addresses this problem by operating at school sites and coordinating with various agencies and businesses to deliver needed services to youths and their families. Rather than outlining specific program requirements, the CIS model offers an overall approach for working with youths to help fulfill five basic needs:

• A personal one-on-one relationship with a caring adult.

• A safe place to learn and grow.

• A healthy start and a healthy future.

• A marketable skill to use upon graduation.

• A chance to give back to peers and the community.

Originally, CIS was known as Cities In Schools, and its programs focused on high school students who had already demonstrated behavior problems and who were at high risk of dropping out of school. Over time, this focus has broadened to include prevention efforts aimed at helping young people from elementary through high school stay in school and avoid other related problems. Each local program determines the specific levels and types of services provided. Services may include life skills education, study skills, remedial and grade-level academic education, and field trips. In addition, programs may assess and monitor individual student and family needs, and help put in place individual and group counseling, anger management counseling, health care services, employment counseling, and other related services offered through various community agencies and organizations. Most programs also have individual tutoring and/or mentoring components. While most CIS programs are delivered within traditional schools, CIS programs may also be organized as alternative schools – or “schools within schools” – with separate facilities for CIS students.

Program Participants

CIS serves youths at risk for dropping out of school; programs involve youths in elementary, middle, and high schools. CIS also may direct services to the families of youths in the CIS program. During the 1999 to 2000 school year, CIS programs operated in 2,300 schools and alternative sites in 32 states, providing direct services to 650,000 youths and their families.

Evaluation Methods

A national study conducted by the Urban Institute (Rossman and Morley, 1995) examined 30 CIS school programs in 17 communities across the country from 1991 to 1994. The researchers evaluated student outcomes for 659 youths who participated in CIS during the 1989 to 1990 or 1990 to 1991 academic years. There was no comparison group involved in this study; instead, the authors informally compared CIS results with national dropout rates for at-risk youth. The schools in the study

- 111 -

were predominantly middle and high schools, but a few elementary schools also were included. Four of the schools were “alternative” schools. The schools were selected to be broadly representative based on geography, urban/rural location, program size, services offered, and types of partner organizations. The students in the study were considered at risk of dropping out of school (based on prior academic performance, family income, single-parent household status, and other variables known to be related to dropout risk). They came from diverse racial and ethnic backgrounds: 57 percent were African- American, 25 percent Hispanic, 14 percent Caucasian, and 3 percent from other groups. The students had an average age of 14.6 years, 50 percent were male, and 48 percent lived in single-parent households. Prior to entering the CIS program, 59 percent of the students had been retained in grade at least once. The analysis of school outcomes focused on school attendance, academic performance, promotion or retention in grade, high school graduation, and whether the student had dropped out of school. Specifically, researchers examined how students’ absenteeism and grade point averages changed over a one-year period by comparing data in the year prior to enrolling in CIS with data in the CIS entry year. The study calculated cumulative dropout rates over a period of years, up until the 1992 to 1993 school year. The dropout analysis was based on a sample of 488 students. Other outcomes were analyzed using student self-report data (that is, from interviews and self-administered surveys), including information on students’ behavior, their plans for the future, and their self-esteem.

Key Evaluation Findings

The research study (Rossman and Morley, 1995) found the following:

Among students who were enrolled in CIS during the 1989 to 1990 and 1990 to 1991 school years, 77 percent either were still in school or had graduated during the 1992 to 1993 school year, while 21 percent had dropped out of school. Among CIS enrollees who were expected to be eligible to graduate by 1992 to 1993, 69 percent had graduated and 31 percent had dropped out of school. (These rates appear to compare favorably with cumulative dropout rates for at-risk youth nationally; however, see “Issues to Consider” for more on this topic.)

• Improvement in school attendance was greatest for CIS students who had moderate or severe attendance problems prior to beginning the program. Overall, 50 percent of CIS enrollees (for whom records were available) improved or maintained their school attendance during their first year in CIS, compared with the year before they entered the program. Of the students who entered CIS programs with ten or more days of school absence in the previous year, 68 percent improved their attendance during their first year in CIS. For students with the most- serious attendance problems (at least 21 days of absence in the previous year), 70 percent improved their attendance during their first year in CIS, averaging 6.6 days of increased attendance.

• Improved academic performance was also greatest for CIS students who had moderate or serious problems with their grades prior to beginning the program. Overall, 49 percent of CIS enrollees (for whom records were available) improved their grade point averages (GPAs) by the end of their first year in CIS, compared with the year before they entered the program. Of the students who entered CIS programs with GPAs of 1.99 or lower, 60% percent improved their GPAs by the end of their first year in CIS. For students with the most-serious academic problems (GPAs of 1.0 or lower), 79 percent improved their grades by the end of their first year in CIS, averaging a 1.0 grade point increase in their GPAs.

• Students enrolled in the CIS alternative school programs showed greater improvements than did students in traditional CIS school sites.

Probable Implementers

CIS programs are implemented as partnerships among schools, community health and service organizations, governmental agencies, businesses, and faith-based organizations.

- 112 -

Funding

Each local CIS program is responsible for developing the resources to support its own operation and that of its affiliated school sites. Such resources are acquired through fund-raising, by creating public- private partnerships, by relocating existing service agency personnel to work at school sites, and by recruiting volunteers. The national CIS organization provides training and technical assistance for those interested in creating new CIS programs, and occasionally provides pass-through funding for special projects.

Implementation Detail

Program Design

The national CIS network is composed of a national headquarters and numerous state and local CIS programs. State CIS affiliates help replicate the CIS program throughout their state and secure state- level resources for individual communities. Local CIS programs initiate, implement, and manage the program, adapting CIS principles and methods to the needs and resources within their communities. Local programs also develop partnerships with school districts, service agencies, and local businesses, integrating existing community resources with the needs of students and families. The CIS approach is based on three principal levels of service:

• Level One: All Students and Families. Resources are provided to the entire school and community.

• Level Two: Selected Students and Families (Short Term. When students and families experience a sudden crisis, CIS processes referrals to appropriate service providers for short- term intervention.

• Level Three: A Specific Caseload of Students and Families (Long Term). In some cases, services are directed toward an identified group of students and families on a regular, sustained, case-managed basis.

Curriculum

CIS does not have a prescribed or set curriculum. While some schools with CIS programs offer a “CIS class” as an elective, such a class is not required by the national CIS organization. When offered, “CIS classes” generally focus on teaching basic life skills and encouraging pro-social behavior. Staffing

Staffing teams for local CIS organizations typically include an executive director, an agency coordinator/resource coordinator, an administrative assistant, and the project director for the first school site. The CIS staffing teams at the school sites include the project director, and some mix of the following: repositioned local service agency staff (for example, social workers/case managers or health care providers), teachers or other school personnel, and local volunteers, who typically provide tutoring or mentoring. Training is provided at the CIS Training Institute held nationally in the spring and fall. Training courses span several days and address different needs, including starting a CIS program in a new community and managing a CIS project at the school level. Although centralized training (provided by the national CIS organization) is emphasized, state CIS offices also offer training and provide technical assistance.

Issues to Consider

This program received a “promising” rating. Although a multi-site study was conducted to assess outcomes for CIS participants, there were methodological weaknesses in the research design. Most important, there was no comparison group (that is, similar students who did not participate in the CIS program), which makes it difficult to attribute improvements in CIS student outcomes to program

- 113 -

participation. In addition, as the authors of the study acknowledge, there were difficulties obtaining outcome data on many of the students who were in the study. For example, researchers were able to obtain absenteeism data on only 34 percent of the 659 students in the study and grade point average data on only 44 percent of the students. For this reason, the study’s results regarding absenteeism and grade point averages must be interpreted with caution. The lack of a comparison group makes it particularly difficult to interpret the student outcomes regarding staying in school (or graduating) versus dropping out. The study found that over a two- to three-year period, 77 percent of CIS students either stayed in school or graduated, while 21 percent had dropped out. This dropout rate appears to be at the low end of cumulative dropout rates for at- risk youth nationally; however, this claim is supported only by anecdotal evidence, and not by comparative evidence produced within the study itself. The study also found that CIS participants with the most severe problems with school attendance and academic performance showed the largest gains after completing the CIS program. Specifically, 68 percent of students with moderate attendance problems and 70 percent with severe attendance problems improved their school attendance after one year in the CIS program; similarly, 60 percent of students with low GPAs and 79 percent with very low GPAs improved their grades after one year in CIS. Overall, however, only about one-half of students participating in CIS improved their school attendance or grades. This pattern suggests that, at least for these intermediate measures of success in school, CIS is most effective for those with the greatest need. Whether this means that the program should be more narrowly targeted to those most at risk, or whether less-at-risk students are benefiting in more subtle ways, are questions that remain unanswered. All the findings cited here are from one multi-site study; more research is clearly needed to better assess the effectiveness of the CIS program. Staff at CIS national headquarters indicate that another national research study is underway, with results expected in 2005. If this study uses a more rigorous research design, it may be able to address some of the ambiguities that remain regarding the program’s effectiveness in helping at-risk youth remain in school and graduate.

Example Sites

Communities In Schools programs operate in more than 3,400 schools in 25 states and the District of Columbia, serving nearly 1.3 million young people and their families every year.

Contact Information

Communities In Schools, Inc. (National Office) 277 S. Washington Street, Suite 210 Alexandria, VA 22314 phone: (703) 519-8999 Toll free: 1-800-CIS-4KIDS Fax: (703) 519-7213 e-mail: cis@cisnet.org

internet: www.cisnet.org

Available Resources

CIS offers training and technical assistance to CIS network members. General information about CIS is found on the CIS web site www.cisnet.org

Bibliography

Rossman, S. B. and E. Morley The National Evaluation of Cities in Schools: Executive Summay Washington DC: The Urban Institute, 1995.

- 114 -

mailto:cis@cisnet.org

http://www.cisnet.org/

Rossman, S.B. and E. Morley Helping At Risk Youth: Lessons from Community-Based Initiatives Washington DC: The Urban Institute, 1997.

Last Reviewed

May 2009

Cooperative Integrated Reading and Composition

Program Info

Outcome Areas Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Elementary School Middle School Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance

Evidence Level Promising

Program Overview

Cooperative Integrated Reading and Composition (CIRC) is a school-based program that targets reading, writing, and language arts in grades 2 through 6. The three principle program elements are direct instruction in reading comprehension, story-related activities, and integrated language arts/writing instruction. Each student is paired with another student and then assigned to a group of students at the same or different reading level. These learning teams work cooperatively on program- related activities. All activities follow a cycle that involves teacher presentation, team practice, peer pre-assessment, additional practice, and testing. Students are encouraged to cooperate and help one another, because students’ scores on individual assessments are summed to form team scores. Several years ago, CIRC was adapted to form one component of Reading Wings, a comprehensive reading program in the Roots and Wings whole-school reform model. The Roots and Wings model consists of elementary school age learning programs, reading and language arts instruction, tutoring, family support and integrated services, social studies and science instruction, and mathematics instruction. CIRC has been incorporated into a primer-level reading program called Reading Wings.

Program Participants

Students in grades 2 through 6

- 115 -

Evaluation Methods

Madden et al. (1986a) studied CIRC among 461 third- and fourth-grade students in 21 classes in a suburban Maryland school district. The sample consisted of 11 treatment classes from six schools that were matched on California Achievement Test (CAT) Total Reading scores with ten control classes from four schools. Treatment and control teachers volunteered to participate in the 12-week study, with treatment classes implementing CIRC, and control teachers continuing to use their usual teaching methods and curriculum materials until the following academic year. Teaching methods utilized by the control teachers involved dividing each class into three reading groups by reading level, giving traditional language arts and writing instruction to the whole class, and using workbooks during follow-up time. Outcome measures at 12 weeks included performance on reading comprehension, reading vocabulary, spelling, language expression, and language mechanics subtests on the CAT. In addition, students’ writing samples were analyzed for ideas, the level of organization, and language mechanics (i.e., punctuation and capitalization, spelling, word choice, syntax). An analysis of baseline differences between the groups revealed that the control group had better scores than the treatment group on the CAT Total Language test and on the pretest writing samples that indicated students’ knowledge of language mechanics. To account for these differences, students’ scores on the CAT pretest were used as covariates in all analyses. In a similar study, Madden et al. (1986b) assessed the effects of CIRC in a sample of 450 third- and fourth-grade students from 22 classes in a suburban Maryland school district. The sample included nine treatment classes from four schools and 13 control classes from five schools. The treatment and control classes were matched on CAT Total Reading and Total Language scores, and the research team also attempted to match the groups on ethnic background and socioeconomic status by selecting classes from schools in the same or similar neighborhoods. An analysis of baseline differences between the two groups revealed no significant differences between them. Both treatment and control teachers volunteered to participate in the study, and CIRC was implemented for 24 weeks. Outcome measures included performance on reading comprehension, reading vocabulary, language expression, and language mechanics subtests from the CAT. Student writing samples were also analyzed for ideas, organization, and mechanics. Additionally, a subsample of students was given informal inventories of their reading knowledge (consisting of lists of words to be defined and oral reading passages from the Durrell Analysis of Reading Difficulty test). The student subsample consisted of six pairs of students (one from the treatment group and one from the control group in each pair) from each class. The students in two pairs were randomly selected from the top third of their class according to reading level, students in two pairs were randomly selected from the middle third of their class, and students in two pairs were randomly selected from the bottom third of their class. This random selection of the students in each matched pair provided for representation from all levels of students in each classroom. Stevens et al. (1989) studied CIRC in a sample of 529 students from 29 classes in grades 2 through 6 in a suburban Maryland school district. The sample of 15 treatment classes from three schools was matched on CAT Total Reading scores with 14 control classes from three schools. Zero percent to 10 percent of the student population in these schools were racial/ethnic minorities, and 6 to 13 percent were disadvantaged students as determined by the number receiving free or reduced-price lunches. No significant pretest differences were found between the treatment and control groups on CAT Total Reading or Total Language scores or on an index of reading awareness. Outcome measures included performance on reading vocabulary, reading comprehension, language mechanics, and language expression subscales from the CAT, and scores on an index of reading awareness (which measured students’ awareness of comprehension strategies). CIRC was also assessed by Stevens and Durkin (1992) in a sample of 1,223 sixth-grade students from six middle schools in an urban Maryland school district. The sample included 20 treatment classes from three schools and 34 control classes from three schools, who were matched on CAT Total Reading pretest scores. Racial/ethnic minority students in the schools ranged from 27 percent to 99 percent of the student populations, and the percentage of disadvantaged students ranged from 38 percent to 77 percent. Outcome measures included reading vocabulary and reading comprehension subscales from the CAT.

- 116 -

Students in another state participated in a study by Bramlett (1994), which examined CIRC among 392 third-grade students in eight school districts in rural southern Ohio. The treatment group consisted of 198 students from nine classes, and the control group consisted of 194 students from nine classes. Teachers volunteered to participate in CIRC, and their teaching experience ranged from 4 to 21 years. The teachers were assigned to treatment and control groups based on geographic representation, and an attempt was made to equally distribute teachers according to years of teaching experience. Outcome measures included CAT subscales of reading vocabulary, reading comprehension, Total Reading, and word-analysis. Finally, Stevens and Slavin (1995) studied 1,299 students in grades 2 through 6 in a suburban, working-class school district in Maryland. The sample consisted of 31 treatment classes in three schools, and 32 control classes in four schools. The treatment-group and control-group schools were matched on socioeconomic variables, and the student body in both groups of schools had similar ethnic backgrounds. Minority and disadvantaged students in both schools ranged from zero to10 percent and from 6 to13 percent of the student body, respectively. An analysis of pretest scores between treatment and control groups indicated that the groups differed significantly in language and reading abilities. To account for these differences, pretest scores were used as covariates in the analyses. Outcome measures included reading comprehension, reading vocabulary, language mechanics, and language expression subscales from the CAT, and scores on an index of reading awareness.

Key Evaluation Findings

The Madden et al. (1986a) study of 461 third and fourth graders found the following:

• Treatment-group students scored significantly higher than control group students on four of the five CAT subscales, including reading comprehension, reading vocabulary, language expression, and spelling. No significant differences were found between groups on the fifth subscale, language mechanics.

• As measured by grade equivalents, the CIRC classes outscored the control classes by 0.41 grade equivalents for reading comprehension, 0.34 grade equivalents for reading vocabulary, 0.56 grade equivalents for language expression, and 0.71 grade equivalents for spelling.

• The assessment of student writing samples indicated that the treatment group outscored the control group in organization of writing. No significant differences were found between groups for ratings of writing language mechanics or ideas.

Madden et al.’s (1986b) study of 450 third- and fourth-grade students reported:

• The CIRC group significantly outscored the control group on reading comprehension, language expression, and language mechanics, with effect sizes of 0.35, 0.29, and 0.30, respectively. (An "effect size" is a quantified measure of the effectiveness of a treatment. Conventional values of effect sizes are: small = .20, medium = .50, and large = .80). No significant differences in reading vocabulary were found between the two groups.

• In grade equivalents, the CIRC classes outscored the control classes by 0.66 on reading comprehension, 0.64 on language expression, and 0.66 on language mechanics.

• The CIRC group scored marginally higher on the writing sample analysis of ideas, with an effect size of 0.31. No significant differences in the assessments of writing organization or mechanics were found between the two groups.

• The CIRC subsample of students outscored the control group subsample on all oral reading measures, including word recognition (with an effect size of 0.64), word analysis (effect size 0.47), grade placement (effect size 0.55), time on a common paragraph (effect size 0.62), and number of errors in a common paragraph (effect size 0.44).

The Stevens et al. (1989) study of 529 students in grades 2 through 6 also found favorable results for the treatment group. The authors reported:

- 117 -

• Significant differences favoring the treatment group on three of the four CAT tests, including reading vocabulary (effect size 0.29), reading comprehension (effect size 0.25), and language mechanics (effect size 0.28). There were no significant differences between the treatment and control groups for language expression.

• Treatment effects varied by grade level, i.e., while there were significant program effects for students in the second through fifth grades, there were no significant differences between the two groups in sixth-grade scores.

• The treatment group scored significantly higher than the control group on the index of reading awareness.

Stevens and Durkin’s (1992) study of 1,223 sixth grade students found:

• The treatment group outscored the control group on reading comprehension, with an effect size of 0.11.

• There were no significant differences between the two groups in their reading vocabulary.

• Additional analyses of students’ race and gender did not indicate a differential program impact on racial/ethnic minorities or on students of either gender.

The Bramlett (1994) study of 392 third graders in Ohio schools reported:

• The treatment group scored significantly higher than the control group in reading comprehension, with a small effect size of 0.11.

• There were marginally significant but small effects favoring the treatment group on tests of Total Reading (effect size 0.07) and word analysis (effect size 0.10).

• No significant differences were found between the two groups on their reading vocabulary scores.

For analysis purposes, Bramlett divided students into three levels (the bottom 33 percent, the middle 33 percent, and the upper 34 percent) based on the students’ percentile rankings in pretest CAT Total Reading scores. The Bramlett analysis indicated:

• Scores for the bottom 33 percent of the CIRC group were significantly higher than the scores for the bottom 33 percent of the control group. Treatment-group students in the bottom-third percentile outperformed control group students on all four subtests, including vocabulary (effect size 0.30), reading comprehension (effect size 0.33), CAT Total Reading (effect size 0.35), and word analysis (effect size 0.56).

• Significant differences between the two groups were not found for students in the middle or upper percentile levels.

Finally, the 1995 study by Stevens and Slavin of 1,299 students in grades 2 through 6 reported:

• After the first year of the CIRC program, treatment group classes outscored control group classes on reading vocabulary (effect size 0.22) and reading comprehension (effect size 0.24). No significant program effects were found for language mechanics or language expression.

• After the second year of the program, CIRC students outscored control group students on tests of their reading vocabulary (effect size 0.20), reading comprehension (effect size 0.26), and language expression (effect size 0.26). No significant differences were found between the two groups for language mechanics.

• Treatment-group students outscored control-group students on the index of reading awareness.

Probable Implementers

Public and private elementary schools

- 118 -

Funding

No information on CIRC funding available at this time.

Implementation Detail

Program Design

A unique aspect of CIRC is its focus on cooperative team learning. However, as is also done in traditional reading programs, CIRC teachers use anthologies, basal readers, and/or novels as instructional tools. Students work in pairs on activities that include reading to one another; predicting how stories will end; identifying characters and settings; summarizing stories for each other; writing responses to stories; and practicing spelling, decoding words, and vocabulary. Students also write drafts, revise, and edit each other’s work, and prepare to "publish" their writing. Students then work in teams of four to five to better understand main story ideas and to improve other general comprehension skills. Measurement of individual students’ contributions to their teams are based on quiz scores and independently written compositions.

Staffing

CIRC requires two days of training for classroom teachers and school administrators, plus review of materials including story anthologies, basal readers, and novels. Additional training and follow-up sessions are recommended.

Issues to Consider

This program received a "promising" rating. The evaluations of CIRC are quasi-experimental in design, and utilized reasonably convincing comparison groups along with statistical analyses that attempted to account for any pre-existing differences between the experimental and control groups. The evaluations demonstrate that treatment-group students scored significantly higher than control-group students on standardized tests of reading comprehension, reading vocabulary, language expression, language mechanics, and spelling. Analyses of students’ writing samples indicate that CIRC students exhibited superior performance as compared with control students on organization of their writing and presentation of written ideas. The Bramlett (1994) study suggests that CIRC may have a greater impact on students with poorer reading skills than on students with average or strong reading skills. Study results showed that students in the bottom third of the group at baseline experienced significant program impacts when compared with the control group, whereas students in the middle and upper thirds of the treatment group did not demonstrate any differences when compared with control-group students. One possible issue of concern with several of the evaluations (i.e., Madden et al., 1986a; Madden et al., 1986b; Stevens et al., 1989; Bramlett, 1994; and Stevens & Slavin, 1989) is that classroom teachers volunteered to participate in the experiment. It is possible that these teachers are intrinsically different than teachers who would opt not to participate in the studies, e.g., they may be larger risk-takers, or be particularly concerned about their students’ reading difficulties. Beyond participation in a study on CIRC, such innate differences may lead to teaching practices and classroom styles that differ from other kinds of teachers. The results of these evaluations therefore do not necessarily generalize to classrooms with teachers who would choose not to participate in such an experiment. It should be noted that the program developers were program evaluators on five of the six studies cited in this program description.

- 119 -

Example Sites

Maryland Ohio

Contact Information

Nancy Madden, President Success for All Foundation 200 W. Towsontown Blvd. Baltimore, MD 21204 Phone: 410 616 2330 Fax: 410 324 4440 nmadden@successforall.net

Available Resources

None at this time.

Bibliography

Bramlett, Ronald K., "Implementing Cooperative Learning: A Field Study Evaluating Issues for School- Based Consultants," Journal of School Psychology, Vol. 32, No. 1, 1994, pp. 67-84. Madden, Nancy A., Robert J. Stevens, and Robert E. Slavin, A Comprehensive Cooperative Learning Approach to Elementary Reading and Writing: Effects on Student Achievement (Report No. 2), Center for Research on Elementary & Middle Schools, Baltimore, Md.: The Johns Hopkins University, 1986a. Madden, Nancy A., Robert J. Stevens, and Robert E. Slavin, Reading Instruction in the Mainstream: A Cooperative Learning Approach (Report No. 5), Center for Research on Elementary & Middle Schools, Baltimore, Md.: The Johns Hopkins University, 1986b. Stevens, Robert J., and Robert E. Slavin, "Effects of a Cooperative Approach in Reading and Writing on Academically Handicapped and Nonhandicapped Students," The Elementary School Journal, Vol. 95, No. 3, 1995, pp. 241-262. Stevens, Robert J., and Scott Durkin, Using Student Team Reading and Student Team Writing in Middle Schools: Two evaluations (Report No. 36), Center for Research on Effective Schooling for Disadvantaged Students, Baltimore, Md.: The Johns Hopkins University, 1992. Stevens, Robert J., Robert E. Slavin, and Anne Marie Farnish, A Cooperative Learning Approach to Elementary Reading and Writing Instruction: Long-Term Effects (Report No. 42), Center for Research on Elementary & Middle Schools, Baltimore, Md.: The Johns Hopkins University, 1989.

Last Reviewed

November 2004

- 120 -

mailto:nmadden@successforall.net

Coping Cat

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children not experiencing anxiety or mood disorders, such as depression

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School High School Community-Based Service Provider Health Care Provider Type of Service Health Care Services Youth Development Type of Outcome Addressed Mental Health

Evidence Level Promising

Program Overview

The Coping Cat program is a cognitive-behavioral therapy intervention that helps children recognize and analyze anxious feelings and develop strategies to cope with anxiety-provoking situations. The program focuses on four related components: (1) recognizing anxious feelings and physical reactions to anxiety; (2) clarifying feelings in anxiety-provoking situations; (3) developing a coping plan (for example, modifying anxious self-talk into coping self-talk, or determining what coping actions might be effective); and (4) evaluating performance and administering self-reinforcement. By incorporating adaptive skills to prevent or reduce feelings of anxiety, the Coping Cat therapist uses a workbook to guide the child through consideration of previous behavior in situations in which the child felt anxious, as well as the development of expectations for future behavior in anxious situations. The Coping Cat workbook is used for children aged 8 to 13 years and the C.A.T. Project workbook is used for children aged 14 to 17 years. The C.A.T. Project differs from Coping Cat only in the use of developmentally appropriate pictures and examples for older ages.

Program Participants

Children and youth aged 8-17 years

Evaluation Methods

The first evaluation of Coping Cat was published in 1994 by Philip C. Kendall, in a study of 47 children aged 9 to 13 years. The sample consisted of 27 children receiving the intervention and 20 wait-list control participants. The intervention participants were mostly Caucasian (78 percent), and 22 percent were African American. Of the 47 study participants, clinical interviewers at baseline diagnosed 30 participants with overanxious disorder (64 percent), 8 were diagnosed with separation anxiety

- 121 -

disorder (17 percent), and 9 were diagnosed with avoidant disorder (19 percent; characterized by an excessive avoidance of contact with unfamiliar people). Children were excluded from the study if they had an IQ below 80, a disabling physical condition, displayed psychotic symptoms, or were currently using antianxiety medications. After the intake interview, 60 eligible subjects were randomly assigned to either the 16-week Coping Cat intervention or the wait-list control group. Treated subjects were randomly assigned to therapists. After an 8-week waiting period (i.e., after the first half of the intervention), the wait-list control subjects were randomly assigned to therapists and participated in the intervention. Outcomes were measured after 8 weeks for the control group (i.e., before they underwent the treatment), and after 16 weeks for the intervention group. Outcome measures included child self-reports of anxiety, fear, depression, coping, and negative thoughts; parent reports of child behavior and child anxiety; teacher reports of child behavior; and direct behavioral observations of child anxiety by researchers. Pretreatment differences across groups were not significant. Of the 60 initial subjects, 13 did not complete the study. Attrition analyses revealed no significant differences between study completers and dropouts. In a second study of Coping Cat, Kendall et al. (1997) assessed 94 children aged 9 to 13 years, including 60 intervention and 34 wait-list control participants. The initial pool of 118 participants with primary anxiety disorders had all been referred from community sources, and 24 children subsequently dropped out of the study. Of the 94 children included in the final analysis, 55 were diagnosed at intake with overanxious disorder, 22 with separation anxiety disorder, and 17 with avoidant disorder. Children were excluded if they displayed psychotic symptoms, if their primary diagnosis was simple phobia, or if they were currently using antianxiety medications. After intake, participants who met the eligibility criteria were randomly assigned to either the 16-week Coping Cat program (60 children) or the 8-week waiting list control condition (34 children). Of the study participants, 58 percent of the intervention group was male compared with 68 percent of the control group. The majority of the sample was Caucasian (87 percent of the intervention group versus 82 percent of the control group). Groups were compared for pretreatment differences in terms of age, gender, race, and all dependent variables, and no significant differences were found between groups. Outcomes were assessed via child self-reports of anxiety, fear, depression, coping, and negative thoughts; parent reports of child behavior, anxiety, and coping skills; teacher reports of classroom behavior; and researcher observations of children’s anxious behavior. Outcomes for the treatment group were assessed posttreatment (16 weeks), while outcomes for the wait-list control group were assessed after the waiting-list period (8 weeks). Flannery-Schroeder and Kendall (2000) assessed the effects of Coping Cat in a sample of 37 children aged 8-14 years. Subjects were referred by a clinic and had all been diagnosed with an anxiety disorder, including 21 children with generalized anxiety disorder, 11 with separation anxious disorder, and 5 with social phobia. Exclusion criteria for participation included having a disabling physical condition, psychotic symptoms, or currently using antianxiety or antidepressant medication. The initial study sample included 45 children, and 8 children subsequently dropped out of the study. Subjects were randomly assigned to (a) individual cognitive-behavioral treatment (ICBT, 13 students), (b) group cognitive-behavioral treatment (GCBT, 12 students in four single-gender groups), or (c) wait- list control (12 students). After nine weeks, the 12 control subjects began the Coping Cat program. Analyses of pretreatment group differences in terms of age, gender, race, family income, and parent levels of education revealed no significant differences among groups. In a comparison of pretreatment dependent variable scores across groups, some means on child-reported measures were found to differ significantly. Scores on measures of state anxiety (i.e., situational anxiety) and trait anxiety (i.e., enduring tendency to experience anxiety) scales of the State-Trait Anxiety Inventory were significantly lower in the GCBT group compared to the ICBT and control groups. Additionally, mean scores in the GCBT group were significantly lower than scores in the control group on the measures of loneliness and social anxiety. Treatment outcomes were assessed via clinician interviews; child self- reports of anxiety, depression, coping skills, self-perception, loneliness, friendship, and recall of treatment curriculum; parent reports of child behavior, anxiety, coping skills, social activities, and peer relationships; and teacher reports of classroom behavior.

- 122 -

Key Evaluation Findings

The initial study by Kendall (1994) found:

• Child self-reports: All measures reported by children revealed significantly more positive outcomes for the intervention group participants than for the control group participants. Specifically, the intervention group had better outcome scores than the control group on

o anxiety measures

o fear scores

o perceived ability to cope with most-dreaded situations

o frequency of negative thoughts during the past week

o depression

• Parent reports: Similar to the child self-report measures, all of the parent assessments of child behavior and anxiety showed significant, positive effects for the intervention. Specifically:

o Intervention group parents rated their children more positively on all four behavior scales, including internalizing behavior, social behavior, health, and externalizing behavior.

o Parent reports of their child’s state-trait anxiety also showed a significant effect favoring the intervention group.

• Teacher reports: No significant differences were found for either the internalizing or externalizing behavior scores.

• Behavioral observations: No significant differences were found between groups for the behavioral observation measures when looked at individually. However, when scales were merged into a single score, a significant effect was found that favored the intervention group.

Kendall et al. (1997) reported the following:

• Clinician interviews: Using results from the parent interview used to diagnose the child’s primary anxiety disorder at intake, 71 percent of the treated children at the end of treatment (16 weeks) no longer met criteria for their initial diagnosis as a primary clinical diagnosis. Fifty-three percent no longer met criteria for a primary anxiety disorder at all. Of the control participants, only 6 percent did not qualify for their primary anxiety disorder diagnosis after the waiting-list period (8 weeks).

• Child self-reports: All of the child report measures found either significant or marginally significant positive effects favoring the intervention group. These included

o significant effects for reductions in anxiety

o significant effects for reduction of fear

o significant improvements in ability to cope with dreaded situations

o a significant reduction in the frequency of negative thoughts during the past week

o a marginally significant effect for reduced depression.

• Parent reports:

o Significant outcomes favoring the Coping Cat participants were found for both mothers' and fathers' internalizing behavior scores.

o Significantly more positive scores for the intervention group children were also noted in mothers' anxious-depressed scores and anxiety scores; however, no significant effects were found for fathers' reports on these scores.

- 123 -

o Parent reports of children’s state-trait anxiety were significantly more positive for intervention group children than for control group children.

o Both mothers' and fathers' reports of coping skills revealed significantly superior coping outcomes for intervention group children.

• Teacher reports: No significant group differences were found for the teacher’s reports of children’s internalizing or externalizing classroom behavior problems.

• Behavioral observations: Significantly better outcomes were found for the intervention group for two anxious behaviors—trembling voice and fingers in mouth—while differences between groups were not found for absence of eye contact.

The study by Flannery-Schroeder and Kendall (2000) comparing individual (ICBT) and group cognitive-behavioral therapy (GCBT) with a waiting-list control group found:

• Clinician Interviews: Using parent ratings, at posttest significantly more treated children than control children no longer met diagnostic criteria for their primary anxiety disorder (73 percent for the ICBT group, 50 percent for the GCBT group, and 8 percent for the control group). No significant differences were found between the ICBT and GCBT groups.

• Child self-reports:

o No significant differences were found among groups for anxiety as measured by the Revised Children’s Manifest Anxiety Scale.

o Significant group differences were found on the state anxiety and trait anxiety scales of the State-Trait Anxiety Inventory.

• For trait anxiety, results indicated that both the ICBT and the GCBT groups demonstrated a significant reduction in scores from pretest to posttest, while the control group did not show a significant reduction in scores.

• For state anxiety, only the ICBT group showed significant improvement from pretest to posttest.

o An analysis of the combined child reports of social functioning yielded no significant differences among groups in change scores from pretest to posttest.

o A significant effect was found for coping skills. Both treated groups improved their self-reported coping from pretest to posttest, while the control group did not.

o No significant differences were found among groups for depression scores.

o For recall of content, significant differences were found between the treatment groups, with ICBT participants recalling significantly more of the Coping Cat curriculum than GCBT participants.

• Parent and teacher reports:

o Fathers' reports of child anxiety showed significant differences among groups, with reductions in anxiety found for both the ICBT and GCBT groups, but not for the control group. Differences in mothers' reports of child anxiety were not significant.

o In regard to parent scores for the child’s ability to deal with anxiety-provoking situations, significant differences emerged among groups. Mothers' reports showed that the ICBT and GCBT groups showed increases in coping from pretest to posttest, while no change was shown for the control group. Similar results were found for fathers' reports.

o An analysis of a combination of the measures assessing social functioning showed no significant differences among groups.

o Results showed significant group differences for fathers' reports of internalized distress as rated on the Child Behavior Checklist (CBCL) internalizing scale, with

- 124 -

both the ICBT and GCBT groups showing a decrease in distress scores from pretest to posttest and no significant change for the control group.

o No significant differences were found in either mothers' or teachers' reports of internalizing scores on the CBCL.

Probable Implementers

Community-based child and youth organizations, school psychologists, private therapists, mental health centers and clinics.

Funding

Previous funding for the implementation and evaluation of Coping Cat has been provided by the National Institute of Mental Health. Sources for current funding of program implementation may include mental health agencies, private service providers, and school districts.

Implementation Detail

Program Design

The Coping Cat program provides children and youth with information about anxiety and ways of coping with situations that previously caused anxiety and fear. Behavioral training strategies such as cognitive restructuring, modeling, guided imagery, simulation, real-life exposure, role-playing, relaxation training, and contingent reinforcement are used. Children are taught how to verbally reinforce their own successful coping and are encouraged to practice using the coping skills when anxiety-provoking situations arise.

Curriculum

The first eight sessions of the Coping Cat program involve an introduction of the basic concepts, followed by practice and reinforcement of the skill.

• In Session 1, the therapist builds a rapport with the child and collects specific information about the kinds of situations and experiences during which the child feels anxious, and the ways in which the child responds to that anxiety.

• Session 2 involves teaching the child to identify different types of feelings.

• In Session 3, children construct a hierarchy of anxiety-provoking situations so that they can distinguish anxious reactions from other types of reactions and can identify their own particular somatic responses.

o After Session 3, a meeting is held with the child’s parents to review the treatment goals, share impressions and ideas, receive parental input on particular problem areas for each child, and encourage parental involvement in the treatment.

• In Session 4, children are taught how to relax outside of the sessions by listening to a cassette tape containing personalized relaxation content.

• Session 5 consists of teaching the child to recognize and assess self-talk during anxious situations and to reduce self-talk that is anxiety provoking.

• Session 6 emphasizes coping strategies such as coping self-talk and verbal self-direction, as well as developing appropriate actions to help cope with anxious situations.

• In Session 7, children learn how to self-evaluate and self-reward.

• Session 8 comprises reviewing concepts and skills covered in the previous sessions.

- 125 -

During the second set of eight sessions, the child practices the newly acquired skills by using both imaginary and real life experiences with individualized situations that vary from low stress, low anxiety to high stress, high anxiety.

• In Session 9, the child practices the newly learned skills in nonstressful, low-anxiety situations that begin with imaginary experiences and progress to real-life exposure. Practice includes therapist modeling and role-plays.

• In Sessions 10 to 13, the child is exposed to imaginary and real situations that cause increasing levels of anxiety.

• In Sessions 14 and 15, children practice in high-stress, high-anxiety situations.

• The final session is used to discuss the therapy experience, to review the skills, and to encourage the child to think about how to apply the skills in everyday life.

Staffing Coping Cat therapists are trained providers with advanced degrees in psychology.

Issues to Consider

This program received a "promising" rating. All three studies of Coping Cat found that participants experienced a wide range of significant, positive outcomes when compared with control group participants. Findings for child self-report measures were the most robust across the studies, while parent reports and behavioral observations were somewhat mixed. No significant program impacts were found in any of the studies for teacher reports of classroom functioning. The major limitation in all three of the studies was the use of a short-term wait-list control group. While the authors cite ethical problems with forcing a 16-week or more delay in providing treatment to control group participants, the strength of the study findings is limited by the short-term nature of the follow-up measurement of outcomes. In addition, the studies are limited by the fact that outcomes were assessed at different times for the intervention and control groups. For example, for Kendall (1994) and Kendall et al. (1997), results for the intervention group were assessed after completion of the 16-week program, while results for the control group were assessed after 8 weeks, just before wait-list control group participants began the treatment. Similar timing was used in the study by Flannery-Schroeder and Kendall (2000). The timing of the outcomes assessment calls into question whether the significant differences between groups would have held beyond the initial 8-week interval, as well as whether other factors in the children’s environments at the treatment midpoint may have affected the observed outcomes. The study by Flannery-Schroeder and Kendall (2000) comparing individual (ICBT) and group cognitive-behavioral therapy (GCBT) with a waiting-list control group found few differences between the two treatment groups. The study was limited by numerous pretest differences between groups; however, the results do not suggest the superiority of either method of program delivery. Of interest were the results on the recall of content measure, in which it was determined that GCBT participants recalled significantly less information relating to the treatment protocol, which may suggest that they are less likely to use program skills in the future. An evaluation of a Dutch adaptation of the Coping Cat program suggests that the addition of a cognitive parent-training group to the program does not result in additional benefits. Nauta et al. (2003) studied the effects of Coping Cat in a sample of 79 children from the Netherlands. Participants were randomly assigned to one of the three treatment conditions: cognitive-behavioral training (CBT) only, CBT plus seven sessions of parent training, or a control group. Results indicated significant differences between both treatment groups and the control group, but no differences between the treatment groups themselves. Despite the fact that these findings suggest that additional parent training may not be warranted, it should be noted that the program was a shortened version of Coping Cat, and the study was conducted outside of the United States, thus possibly introducing cultural differences that are not necessarily generally applicable. Furthermore, it is possible that the specific type of parent training is important and that a modified curriculum may have imparted more success.

- 126 -

Canadian and Australian adaptations of the Coping Cat program have been developed, which use variations of the Coping Cat curriculum. In Canada the program is called Coping Bear, while in Australia it has been dubbed Coping Koala or, more recently, FRIENDS. Finally, it should be noted that the program developer served as an evaluator on all three of the Coping Cat evaluations reviewed.

Example Sites

Baltimore, MD Durham, NC Los Angeles, CA New York, NY Philadelphia, PA Pittsburgh, PA

Contact Information

For information about research on the Coping Cat program, contact: Philip C. Kendall, Ph.D., ABPP Department of Psychology Director, Child and Adolescent Anxiety Disorders Clinic Weiss Hall 478, Temple University 1701 North 13th Street Philadelphia, PA 19122-6085 phone: (215) 204-1558 clinic phone: (215) 204-7165 fax: (215) 204-5539 email: pkendall@temple.edu

Available Resources

Coping Cat: Flannery-Schroeder, Ellen C., and Philip C. Kendall, Cognitive-Behavioral Therapy for Anxious Children: Therapist Manual for Group Treatment, 1996, Workbook Publishing: Ardmore, PA. Kendall, Philip C., and Kristina A. Hedtke, Cognitive-Behavioral Therapy for Anxious Children: Therapist Manual, 3rd Edition (Child/Individual treatment manual), 2006, Workbook Publishing: Ardmore, PA. Kendall, Philip C., and Kristina A. Hedtke, Coping Cat Workbook, 2nd Edition, 2006, Workbook Publishing: Ardmore, PA. C.A.T. Project: Kendall, Philip C., Muniya Choudhury, Jennifer Hudson, and Alicia Webb, "The C.A.T. Project" Manual for the Cognitive Behavioral Treatment of Anxious Adolescents, 2002, Workbook Publishing: Ardmore, PA. Kendall, Philip C., Muniya Choudhury, Jennifer Hudson, and Alicia Webb, "The C.A.T. Project" Workbook for the Cognitive Behavioral Treatment of Anxious Adolescents, 2002, Workbook Publishing: Ardmore, PA. To order books or DVDs/videos on Coping Cat, visit: http://www.workbookpublishing.com/anxiety.htm

- 127 -

mailto:pkendall@temple.edu

http://www.workbookpublishing.com/anxiety.htm

phone: (610) 896-9797 fax: (610) 896-1955 Training for Coping Cat therapists is often provided in workshops or at local and national professional meetings. To arrange for therapist training, contact Dr. Philip Kendall (as listed above).

Bibliography

Flannery-Schroeder, Ellen C., and Philip C. Kendall, "Group and Individual Cognitive-Behavioral Treatments for Youth with Anxiety Disorders: A Randomized Clinical Trial," Cognitive Therapy and Research, Vol. 24, No. 3, 2000, pp. 251-278. Kendall, Philip C., "Treating Anxiety Disorders in Children: Results of a Randomized Clinical Trial," Journal of Consulting and Clinical Psychology, Vol. 62, No. 1, 1994, pp. 100-110. Kendall, Philip C., Ellen Flannery-Schroeder, Susan M. Panichelli-Mindel, Michael Southam-Gerow, Aude Henin, and Melissa Warman, "Therapy for Youths with Anxiety Disorders: A Second Randomized Clinical Trial," Journal of Consulting and Clinical Psychology, Vol. 65, No. 3, 1997, pp. 366-380. Nauta, Maaike H., Agnes Scholing, Paul M. G. Emmelkamp, and Ruud B. Minderaa, "Cognitive- Behavioral Therapy for Children with Anxiety Disorders in a Clinical Setting: No Additional Effect of a Cognitive Parent Training," Journal of the American Academy of Child and Adolescent Psychiatry, Vol. 42, No. 11, 2003, pp. 1270-1278.

Last Reviewed

October 2006

Coping with Stress Course

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children not experiencing anxiety or mood disorders, such as depression

Topic Areas Age of Child Adolescence (13-18) Type of Setting Middle School High School Community-Based Service Provider Health Care Provider Type of Service Health Education Youth Development Type of Outcome Addressed Mental Health

Evidence Level Proven

- 128 -

Program Overview

The Coping with Stress Course (CWS) targets adolescents at risk for depression who are experiencing elevated depressive symptoms, or "demoralization." The program involves cognitive-restructuring techniques in which participants learn to identify and challenge negative or irrational thoughts that may contribute to the development of future mood disorders, such as depression. CWS is an adaptation of the Adolescent Coping with Depression Course (Clarke, Lewinsohn, and Hops, 1990), which targets adolescents already experiencing major depression or dysthymia.

Program Participants

Adolescents at risk for depression

Evaluation Methods

The first randomized control trial (Clarke et al., 1995) included 222 ninth and tenth grade students from three suburban high schools in Oregon. The Center for Epidemiological Studies Depression Scale (CES-D) was administered to 1,652 students, and 471 adolescents with elevated CES-D scores were asked to participate in an interview during which their diagnostic outcomes were assessed on the interviewer-rated Schedule for Affective Disorders and Schizophrenia for School-Aged Children, Epidemiological Version (K-SADS-E). Interviews were held with 222 students (those who obtained informed consent from parents); 46 were diagnosed with current affective (mood) disorders and were excluded from the study, while 172 met inclusion criteria for elevated depressive symptoms, according to the Diagnostic and Statistical Manual of Mental Disorders (DSM-III-R) criteria. Of these students, 150 agreed to be randomized to either CWS (76 students) or a "usual care" control condition (74 students). Adolescents were 70 percent female, and the majority was non-Hispanic white (93 percent). Subjects were assessed immediately after the intervention and at 6-month and 12-month follow-up points. The dropout rate at posttest was 17 percent, at 6 months it was 20 percent, and at 12 months it was 27 percent. No significant differences were found between study dropouts and completers on initial depression severity or on demographic variables. Adolescent outcomes were measured on three diagnostic interview tests, the Global Assessment of Functioning Scale (GAF, an assessment of severity of impairment), the K-SADS-E, and the Longitudinal Interval Follow-up Evaluation (LIFE). Items from the K-SADS-E were also extracted to form the Hamilton Depression Rating Scale (HAM-D). Adolescent self-reports of depression were assessed on the CES-D. Clarke et al. (2001) undertook a second randomized control trial of CWS on a sample of 45 experimental and 49 control group youths aged 13-18 from Portland, Oregon. All participants were offspring of adults treated for depression in a health maintenance organization (HMO). Potential adult cases were found by reviewing the HMO pharmacy records for dispensation of antidepressant medication and the mental health appointment system during a two-year period (5,954 adult patients). Medical charts were reviewed for a depression diagnosis, determining that 3,935 of these adults (66 percent) had been diagnosed with depression. Recruitment letters signed by treating physicians were mailed to adults they judged appropriate for the study (2,995 patients), and 458 adults (15 percent) refused. The adolescent offspring of 2,083 of the contacted adults were asked to participate in the study. Interviews were scheduled with 744 families (966 youth) and completed with 481 parents (65 percent) and 551 adolescents (57 percent). Eligible offspring were required to have depressive symptoms insufficient to meet clinical criteria for affective disorder and no history of a past mood disorder, such as depression or bipolar disorder. Of the 551 interviewed youth/parent dyads, 79 did not meet study criteria, and the remaining 472 were classified into one of three mutually exclusive depression severity groups. Those classified as having "medium-severity depression" (123 youth or 26 percent) were the focus of this study. "Medium severity" was defined as the youth having a CES-D score of greater than 24 and/or reporting some level of depressive symptoms on the clinical interview, but at a level that was insufficient to meet full criteria for a DSM-III-R diagnosis. These adolescents were randomized to usual HMO care (49 youth) or usual care plus CWS (45 youth), and 29 youth declined to participate. Follow up assessments were conducted immediately posttreatment, and at 12 and 24 months. Outcomes were assessed on the interviewer-rated K-SADS-E, the HAM-D, and the

- 129 -

http://www.promisingpractices.net/program.asp?programid=152

GAF. Parent reports of child behavior and depression were assessed on the Child Behavior Checklist (CBCL), and adolescent self-reports of depression were measured on the CES-D.

Key Evaluation Findings

The study by Clarke et al. (1995) reported:

• On the K-SADS-E and LIFE interviews, a significant advantage was found for the CWS group at 12 months, with incidence rates for affective disorder of 15 percent compared with 26 percent for the control group. No significant differences were detected on the HAM-D, or for disruptive behavior, anxiety, or substance abuse.

• Analyses of the GAF score found a significant effect favoring the treatment group from pretest to initial posttest, but no significant effects when the entire study period was examined (pretest to 12-month follow-up).

• Outcomes on the self-reported CES-D showed significantly fewer cases of either major depression and/or dysthymia (a more minor form of depression) for the treatment group compared with the control group from pretest to initial posttest. No significant differences were detected when outcomes were measured from pretest to 12-month follow-up.

Clarke et al. (2001) found:

• The CWS group scored significantly better than the control group on the CES-D, the HAM-D, the K-SADS-E suicide symptom total, and the GAF.

• Analyses of major depressive episodes during a 14-month follow-up (on average) found a significant advantage for the treatment group (9 percent incidence) compared with the control group (29 percent incidence).

• No significant effects were found for parent reports of child depression, or behavior problems on the CBCL externalizing or internalizing scales.

Probable Implementers

Public and private elementary, middle, and high schools; community-based organizations; hospitals; clinics; and after-school programs.

Funding

No funding suggestions at this time. A cost-benefit analysis of CWS (Lynch et al., 2005) found that the average cost of the program per participant was $1,632 (in 2005 dollars), with identification and recruitment of participants for the study accounting for 65 percent of the costs. The authors found that although total and indirect costs were $610 higher in the CWS group than in the usual care group, the difference was not statistically significant. This finding suggests that the cost-effectiveness of CWS is comparable to that of other accepted depression treatments.

Implementation Detail

Program Design

The theoretical background of the Coping with Stress Course is that teaching adolescents new coping strategies and strengthening their current coping skills provide them with some measure of "immunity" or resistance against the development of mood disorders later in life. The aim of CWS is to enhance at-risk adolescents' resilience in order to counteract their vulnerability to depression and other mood disorders.

- 130 -

Curriculum

Group size for the adolescent sessions is from six to ten adolescents, and the program consists of fifteen 45- to 60-minute group sessions. CWS uses cartoons, role-plays, and group discussions oriented to the developmental level of the participants. Separate parent information meetings at the beginning, middle, and end of each adolescent course are optional. During these sessions, parents are informed about the general topics discussed, the skills taught in the adolescent groups, and the rationale for the use of the selected techniques.

Staffing

CWS groups are led by specially trained school psychologists and counselors who have a minimum of a master's degree in clinical, counseling, or educational psychology, and who have previous experience in conducting psychoeducational groups with adolescents. Before beginning the group sessions, therapists are provided with 40 hours of training, including mock intervention sessions, role- playing adolescent responses to exercises, homework, and videotaped feedback.

Issues to Consider

This program received a "proven" rating. Both studies of the Coping with Stress Course employed rigorous evaluation methodology involving randomized assignment of study participants and found significant reductions in interviewer-rated and self-reported depression symptoms for treatment children when compared with those in a control group. CWS was not found to be effective in reducing parent-reported psychosocial dysfunction and behavior problems among participating children. In addition, some of the positive results on self-reported and interviewer-rated depression were not maintained in one- and two-year follow-up assessments. Studies of CWS suggest that the program can be successfully implemented in a wide range of settings. Clarke and colleagues have conducted CWS program evaluations in both an after-school setting as well as a clinical hospital setting, and the program was found to be effective in both. A four-site replication study of CWS is currently under way, enrolling 320 youth total. Sites include Nashville, Tenn; Pittsburgh, Penn; Cambridge, Mass.; and Portland, Oreg. Results will be available in early 2007. It should be noted that the program developers served as authors on both evaluations of the CWS.

Example Sites

Eugene, Oregon Portland, Oregon

Contact Information

Gregory N. Clarke, Ph.D. Kaiser Permanente Center for Health Research 3800 N. Kaiser Center Dr. Portland, OR 97227 phone: (503) 335-6673 greg.clarke@kpchr.org

Available Resources

The Kaiser Permanente Center for Health Research provides free downloads of the CWS therapist manual and the CWS teen workbook at http://www.kpchr.org/public/acwd/acwd.html.

- 131 -

mailto:greg.clarke@kpchr.org

http://www.kpchr.org/public/acwd/acwd.html

Bibliography

Clarke, Gregory N., Mark Hornbrook, Frances Lynch, Michael Polen, John Gale, William Beardslee, Elizabeth O'Connor, and John Seeley, "A Randomized Trial of a Group Cognitive Intervention for Preventing Depression in Adolescent Offspring of Depressed Parents," Archives of General Psychiatry, Vol. 58, 2001, pp. 1127-1134. Clarke, Gregory N., Peter M. Lewinsohn, and Hyman Hops, Adolescent Coping with Depression Course, Eugene, Oreg.: Castalia Press, 1990. Clarke, Gregory N., Wesley Hawkins, Mary Murphy, Lisa B. Sheeber, Peter M. Lewinsohn, and John R. Seeley, "Targeted Prevention of Unipolar Depressive Disorder in an At-Risk Sample of High School Adolescents: A Randomized Trial of a Group Cognitive Intervention," Journal of the American Academy of Child and Adolescent Psychiatry, Vol. 34, No. 3, 1995, pp. 312-321. Lynch, Frances L., Mark Hornbrook, Gregory N. Clarke, Nancy Perrin, Michael R. Polen, Elizabeth O'Connor, and John Dickerson, "Cost-Effectiveness of an Intervention to Prevent Depression in At-Risk Teens," Archives of General Psychiatry, Vol. 62, No. 11, 2005, pp. 1241-1248.

Last Reviewed

February 2013

Core Knowledge

Program Info

Outcome Areas Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance

Evidence Level Promising

Program Overview

Core Knowledge (CK) is an educational reform model based on the premise that a solid, specific, shared core curriculum is crucial to ensure a sound elementary education and to help children build

- 132 -

strong foundations of knowledge. The content of this core curriculum is outlined in the Core Knowledge Sequence manual, which describes what students should learn at each grade level in kindergarten through grade 8 (K-8). The Core Knowledge Sequence provides a detailed outline and planned progression of specific content to teach in the language arts, American and world history, geography, math, science, the visual arts, and music. The Core Knowledge Sequence of topics is intended to provide 50 percent of what is taught in a U.S. elementary school, and the content complements the general skills development and learning objectives typically found in state and local curriculum guides. Currently, hundreds of schools are participating in the CK school reform model throughout the United States.

Program Participants

Students in grades K-8

Evaluation Methods

Datnow et al. (2000) evaluated the CK program among students in grades 1 through 5 in four CK schools and four comparison schools in four states-- Maryland, Florida, Texas, and Washington. All CK schools had been implementing the CK sequence for two or more years, and program impacts were assessed for a cohort of students that started CK in the first grade and a cohort of students that started CK in the third grade. In each state, local education officials helped to select a comparison school that was from the same district as the CK school and that was similar in the demographic characteristics of its students. The demographic characteristics of participating students varied across program sites. For example, 82 percent of the Florida CK sample were white, 12 percent were African- American, 5 percent were Latino, and 31 percent received free lunches, while 8 percent of the Texas sample were white, 6 percent were African-American, 85 were percent Latino, and 96 percent received free lunches. Additionally, the level of Core Knowledge implementation at the four sites varied. While the Florida, Washington, and Texas schools implemented CK in 70 percent or more of the observed third-grade and fifth-grade classrooms, only 27 percent of the observed classrooms in the Maryland school showed evidence of CK implementation. Outcome measures included basic skills achievement derived from reading comprehension tests and math concepts and applications tests from the Comprehensive Test of Basic Skills (in the Florida, Texas, and Maryland schools) and basic skills achievement data derived from a locally administered test of functional reading and math skills (in the Washington schools). Also studied were program-developed tests of Core Knowledge achievement, including tests on language arts and social studies achievement and science subtests (in the Florida, Texas, and Maryland schools). The total sample of students analyzed who started CK in the first grade was 343 students assessed for basic skills achievement and 256 students assessed for CK achievement. The total sample of students analyzed that started CK in the third grade was 389 students assessed for basic skills achievement, and 290 students assessed for CK achievement. In another study of CK, Whitehurst-Hall (1999) studied 301 seventh- and eighth-grade students from two public middle schools in central Georgia. The study was conducted over three years, and it compared CK students with students who were taught using a traditional curriculum (comparison students). The comparison school was chosen because it most closely matched the CK school. Prior to the 1996/1997 school year, the students at the comparison school attended the school in which CK was later implemented (they were transferred to the comparison school due to district re-zoning). Forty-seven percent of the 110 CK students were black, 53 were percent white, 58 were percent female, 1 percent were special education students, and 28 percent were gifted students. Of the 191 comparison school students, 51 percent were black, 49 percent were white, 49 percent were female, 2 percent were special education students, and 10 percent were gifted students. Outcome measures included the Iowa Test of Basic Skills (ITBS) battery total and the ITBS reading, language, and math subtests; the proportion of students who failed ninth-grade English and ninth-grade math; and the proportion of students retained in the ninth grade.

- 133 -

Key Evaluation Findings

The Datnow et al. (2000) two-year study on a cohort of first graders found the following by the third grade:

• For the Florida site, the overall scores were significantly better for the CK group than for the control group. Subtest analyses showed that CK students scored significantly higher than control students on the reading test (with an average effect size of 0.61) and scored marginally significantly higher than control students on the math test (with a small effect size of 0.40).

• There were no significant overall or subtest score differences found between CK and comparison students at the Maryland, Texas, or Washington sites.

Basic skills achievement results for the third-grade cohort who participated until the fifth grade indicated the following:

• The Washington CK students significantly outscored the control group students as indicated by the overall score analysis. Subtest analyses showed that the CK students scored significantly higher than the control students on measures of reading skills (with a small effect size of 0.31) and math tests (with a small effect size of 0.44).

• The Maryland CK students scored significantly lower than the control group students as indicated by the overall score analysis. Subtest analyses indicated that the control group outscored the CK group on both the reading test (with a large effect size of -1.31) and the math test (with a large effect size of -1.00).

• No significant overall score differences were found between the CK group and the comparison group at the Florida and Texas sites.

• Subtest analyses at the Florida site revealed that CK students scored significantly higher than control students on the reading test (with a small effect size of 0.26), and marginally significantly higher than control students on the math test (with a small effect size of 0.08).

• Subtest analyses at the Texas site found that CK students scored significantly higher than control students on the math tests (with a small effect size of 0.37), but no significant differences were found between groups on the reading tests.

Results from the Core Knowledge test outcomes in Florida, Maryland, and Texas included the following:

• The overall analysis for the first-grade cohort revealed that Florida and Texas CK students scored significantly higher than did control group students (with an average effect size of 0.54 and a large effect size of 1.11, respectively). No significant differences were found between the two groups at the Maryland schools.

• Similarly, for the third-grade cohort, analyses of overall results showed that Florida and Texas CK students scored significantly higher than control students (with average effect sizes of 0.76 and 0.57, respectively). No significant differences were found between the two groups at the Maryland schools.

The Whitehurst-Hall (1999) study of 311 seventh- and eighth-grade students found that the CK group scored significantly higher than the control group on the

• Reading Advanced Skills subtest of the Iowa Test of Basic Skills

• Reading Total score of the ITBS (which included a test of advanced reading skills and a vocabulary test)

• Mathematics Total score on the ITBS

• ITBS Survey Total scores.

- 134 -

No significant differences were found between the CK and control groups on the Language Advanced Skills test, the Language Total score, or the Mathematics Advanced skills tests of the ITBS. In addition, no significant differences were found in the proportion of CK students versus control students who failed ninth-grade English or math classes or who were retained in ninth grade.

Probable Implementers

Public and private elementary schools

Funding

The cost of implementing Core Knowledge varies significantly from school to school. There are a few nominal fees that each school must pay, including a small fee to become a member of the Core Knowledge Network. In addition, every teacher in the school must have a personal copy of the Core Knowledge Sequence. Many schools also purchase the optional series What Your ... Grader Needs to Know and sign up for optional CK workshops (described below). More significant program expenses may be incurred, depending on the amount of additional student resources and materials that are purchased to supplement the materials already in existence at the school.

Implementation Detail

Program Design

The main component of the CK approach is the Core Knowledge Sequence, a 200-page outline of the specific content that should be taught in each subject and in each year from kindergarten through eighth grade. Although the Core Knowledge Sequence details the content that should be taught at each grade, it does not specify the manner in which the content should be taught. Instructional strategies are left up to the individual teachers.

Curriculum

The Core Knowledge Sequence is designed to take 50 percent of the total classroom instructional time, allowing schools to spend the remaining time to cover topics required by state and local standards. Over time, schools work to align the Core Knowledge Sequence with state and local standards, leaving more time for supplemental work. Core Knowledge produces materials that schools may purchase, including the Core Knowledge Sequence, a series of books for parents and teachers titled What Your ... Grader Needs to Know that covers each grade level, and related reading materials and resources for teachers to use in their classrooms.

Staffing

There are no specific requirements for increased staff at schools implementing the CK program, because regular classroom teachers implement the curriculum. However, because the CK approach emphasizes music and art instruction as well as instruction in language, math, and science, schools that do not have music or art teachers may wish to hire them. Regularly scheduled time set aside for planning the presentation of the CK curriculum is recommended for teachers who are implementing Core Knowledge in their classrooms. Because the material that is taught in each grade builds directly upon what was taught in the previous grade, the CK program recommends that teachers work together to ensure a natural progression of content development from one grade to the next. The Core Knowledge Foundation offers workshops conducted by trained teachers or administrators from CK schools. Training sessions range from one to five days in length and can be given to up to 75 participants at a time. Visits from experts in CK who can provide technical assistance are also available to schools implementing CK.

- 135 -

Issues to Consider

This program received a "promising" rating. While study results were somewhat mixed, the evaluations of the CK program indicate that treatment group students experienced significant improvements in their reading and math skills as compared with no-treatment control-group students. The program's "promising" rating is due to mixed results from the program evaluations, small sample sizes in two of the three evaluation studies, and weakness in the comparability of the control groups used in the analyses. These factors limit our ability to determine with certainty that the program caused the observed positive outcomes. Results from the Datnow et al. (2000) study indicated that there were some positive and significant results for CK students at the Florida, Texas, and Washington sites but no significant program effects for students at the Maryland site. One possible explanation for the differing outcome for Maryland CK students is the substantially lower level of observed CK implementation at the Maryland site as compared with the other three program sites. It is also possible that student and school differences among the four sites were contributing factors for the differences in program outcomes. The socioeconomic and ethic background of the students at the Maryland site differed significantly from that of the students at the other three sites, e.g., while students at the Florida and Washington sites primarily were white (82 percent and 79 percent, respectively), 98 percent of students at the Maryland site were African-American. The CK curriculum also includes a preschool sequence that is being used in many schools across the country, but does not have any rigorous evaluation data available at this time.

Example Sites

Florida Georgia Maryland Texas Washington State

Contact Information

Barbara Garvin-Kester Core Knowledge Foundation 801 East High Street Charlottesville, VA 22902 Tel (434) 977-7550 Toll-free (800) 238-3233 Fax (434) 977-0021

Available Resources

The Core Knowledge website (http://www.coreknowledge.org/) offers information about implementation, professional development, books, and other resources for schools and teachers interested in implementing the program.

Bibliography

Datnow, Amanda, Geoffrey Borman, and Sam Stringfield, "School Reform Through a Highly Specified Curriculum: Implementation and Effects of the Core Knowledge Sequence," The Elementary School Journal, Vol. 101, No. 2, 2000, pp. 167-192.

- 136 -

http://www.coreknowledge.org/

Whitehurst-Hall, Jerri Ann, The Impact of the Core Knowledge Curriculum on the Achievement of Seventh and Eighth Grade Students, Ph.D. dissertation, Faculty of Education, University of Georgia, Athens, 1999.

Last Reviewed

November 2004

Counselors Care (CARE)

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children not experiencing anxiety or mood disorders, such as depression

Topic Areas Age of Child Adolescence (13-18) Type of Setting High School Type of Service Health Care Services Youth Development Type of Outcome Addressed Mental Health Physical Health Substance Use and Dependence

Evidence Level Promising

Program Overview

Counselors Care (CARE) is a school-based intervention for high-school students at risk for suicide. CARE is a two-part, four-hour program, beginning with a personalized computer-assisted assessment of risk and protective factors, and followed by a brief counseling intervention designed to enhance a youth's personal resources and social network connections.

Program Participants

High school students in grades 9-12 who are at risk for dropping out of school and suicidal behavior.

Evaluation Methods

Randell, Eggert, and Pike (2001) and Eggert et al. (2002) studied the effects of CARE in a sample of 341 students in grades 9-12 from seven urban high schools in the Pacific Northwest. Students participated in the study in three cohorts over three academic years. The target population for the program was students at risk for dropping out. Students who were eligible for the study had the following characteristics: below expected credits for grade level, in the top 25th percentile for days absent per semester, a grade point average of less than 2.3, and/or a pattern of declining grades.

- 137 -

Also included as eligible for the study were those students who had previously dropped out of school but later returned, or who were referred by school staff for being at high risk for dropout. Students with parental consent completed the High School Questionnaire: Profile of Experiences (HSQ), a self-report survey instrument that screened suicide risk behaviors and related risk factors such as depression, hopelessness, anxiety, and anger. All youths with specific levels and combinations of indicators (such as prior suicide attempts, high depression, drug involvement, suicide ideation, etc.) were identified as "at suicide risk" and were entered into the study. A total of 381 youths were identified as such (38 percent of the total number of students screened on the HSQ) and were randomized to begin the study in one of three conditions: CARE only (117 students), CARE+CAST (103 students; CAST is a small peer-group life skills-training program), or the "usual care" control group (121 students). The study condition for each school was randomly determined at the start; thereafter all three study conditions were rotated systematically through each school, as well as a "pause" condition during which no interventions were implemented (in order to minimize potential carry-over effects to other students of the interventions within the schools). The control condition was designed to simulate intervention as usual by using procedures that a high school might have in place to respond to youths at risk of suicide. In this condition, a trained interviewer conducted a minimal assessment interview (15-30 minutes) for each student. Notification of parents and designated school personnel was initiated, and immediate assistance was provided in those rare instances when the risk of suicide was imminent. In nonemergency situations, school personnel and parents/guardians identified appropriate resources and obtained assistance as warranted. Of the total sample, 52 percent of the youths were female, 24 percent were in 9th grade, 39 percent in 10th grade, 20 percent in 11th grade, and 17 percent in 12th grade. Students' ethnic/racial representation was 40 percent Caucasian, 13 percent mixed race/ethnicity, 13 percent Asian, 12 percent African American, 7 percent Hispanic, 2 percent American Indian/Alaskan Native, 4 percent another ethnicity, and 9 percent unknown. With the exception of age (CARE+CAST students were slightly older), there were no significant baseline differences among the three groups in terms of background variables; suicide risk; or related risk, protective, and family factors. Outcomes on the HSQ were assessed four weeks after baseline (following implementation of CARE), and ten weeks after baseline (following completion of CAST). Retention rates at ten weeks were 97 percent for the CARE+CAST group, 93 percent for the control group, and 89 percent for the CARE group. Attrition analysis indicated few significant differences among groups for study completers and noncompleters, with the exception that CARE+CAST noncompleters reported higher levels of satisfaction with their families than did CARE and control group noncompleters, and CARE noncompleters reported more suicidal ideation than noncompleters from CARE+CAST and the control group. A second, similar study of CARE was conducted by Thompson et al. (2001) in a sample of youths from seven urban high schools in Pacific Northwest school districts. A two-step process was used to identify youths at risk for suicide. First, each school's database and referrals from school personnel were used to identify the total pool of potential high school dropouts (1,546 students). Second, youths in this pool were randomly sampled and invited to participate in the study; those who consented completed the baseline HSQ survey (1,217 students). Of the surveyed youths, 460 (38 percent) were identified as being at risk for suicide. These students were randomly assigned by the school to one of three conditions: CARE (150 students), CARE+CAST (155 students), or the "usual-care" control group (155 students, using the same minimal assessment interview described previously). The sample was 52 percent female and was ethnically diverse, with 49 percent Caucasian youths, 19 percent African American, 18 percent Asian American/Pacific Islander, 10 percent Hispanic/Latino, and 4 percent Native American. Within schools, the three study conditions were rotated systematically such that each school received each condition, as well as a "pause" semester during which no interventions were implemented. Outcomes on the HSQ were measured four weeks after baseline following completion of CARE, ten weeks after baseline following CAST, and nine months after baseline. Retention at nine-month follow- up was 86 percent for CARE, 93 percent for CARE+CAST, and 90 percent for the control group.

- 138 -

Analyses indicated no significant differences among groups for most of the background variables, although CARE+CAST youths were slightly older than youths in the other two groups, and CARE youths had significantly lower baseline problem-solving coping scores.

Key Evaluation Findings

Eggert et al. (2002) and Randell, Eggert, and Pike (2001) reported the following results:

• Youths in all three groups showed a significant decreasing trend over time in suicide risk behaviors (thoughts, threats, attempts), with no significant differences among groups. Further inspection of findings showed that while reductions in suicide risk behaviors occurred between baseline and four-week assessment (after CARE implementation) for all three groups, effects were sustained at ten-week assessment (after CAST implementation) for youths in the CARE and control conditions (i.e., for youths not in CARE+CAST).

o Changes in suicidal threats showed a significant effect of gender on treatment for female students only. For females, CARE youths showed declining trends from four-week to ten-week assessments, compared with a slight rebounding for those in the control group.

• All three groups showed significant declines in depression over time. Further analyses showed that only CARE+CAST youths had significantly less depression than controls at four-week assessment. At ten-week assessment, both CARE and CARE+CAST youths exhibited significantly lower levels of depression than controls, with controls demonstrating a slight rebound effect.

• A steady decline in anger control problems over time was found for all three groups, with no significant differences among groups.

• Significant reductions were found for youth in all three groups in use of alcohol, marijuana, and hard drugs, drug-use control problems, and adverse drug-use consequences. No significant differences were found among groups.

The study by Thompson et al. (2001) found:

• Compared with the usual-care control group, both CARE+CAST and CARE were associated with faster rates of decline in favorable attitudes toward suicide. No significant differences were found between youths in CARE+CAST and CARE.

o There were no significant differences among groups for suicide threats or attempts within the month preceding the posttest; however the baseline rates for suicide threats or attempts were low and are thus limited as outcome measures.

• For depression, both CARE+CAST and CARE groups showed significantly lower levels of depression at ten weeks and nine months than did the control group.

• Declines in hopelessness were sustained within groups across all time points for the CARE+CAST and CARE groups, with significant effects found for the majority of comparisons between both treatment groups and the control group.

o Following the four-week assessment, the control groups rebounded in reported levels of hopelessness and then showed a decline.

o For CARE+CAST versus controls, hopelessness was significantly lower at the ten- week assessment but not at the nine-month assessment.

o For CARE versus the control group, hopelessness was significantly lower at both the ten-week and the nine-month assessments.

• Significant effects for declines in anxiety were found for females in both CARE+CAST and CARE when compared with controls, but not for males.

- 139 -

o For both CARE and CARE+CAST females, significantly larger declines in anxiety were found at the ten-week and the nine-month assessments when compared with controls.

• Differences in the rates of decline in anger for females in CARE+CAST and CARE were statistically significant when compared with controls. Again, no significant differences were found among groups for males.

o Group comparisons showed significant treatment effects regarding anger for CARE and CARE+CAST females compared with control females at the ten-week but not at the nine-month assessment.

Probable Implementers

Public and private high schools

Funding

Previous studies of the CARE program have been supported in part by the National Institute of Nursing Research.

Implementation Detail

Program Design

CARE is a two-part prevention protocol delivered in two sessions. The first is a two-hour, one-to-one computer-assisted suicide assessment called the Measure of Adolescent Potential for Suicide (MAPS), which provides a comprehensive and individualized assessment of direct suicide risk factors, related risk factors, and protective factors. The MAPS includes a motivational introduction followed by an assessment of the youths' stressors, depression, hopelessness, anxiety, suicidal behaviors, risky behaviors, drug involvement, personal skills and coping strategies, and social support resources. The second part of CARE is a two-hour motivational counseling and social network intervention designed to

• deliver empathy and support

• provide personalized information

• reinforce coping skills and help-seeking behaviors

• increase access to help

• enhance access to social support.

During the one-on-one counseling session, MAPS assessment results are summarized with the youths, accurate perceptions are validated and discrepancies are clarified, positive coping strategies are reinforced, and an action plan for enhancing support resources is developed. Each youth is then personally connected with a case manager in the school (e.g., a counselor or school nurse) and/or the youth's favorite teacher to encourage communication between the youth and school personnel. A telephone contact with the parent/guardian of the youth's choice is also initiated, the intent of which is to enhance social network connections, support, and future accessibility of help. Staffing

CARE leaders are specially trained, advanced level clinicians; generally a master's level high-school teacher, counselor, or nurse.

- 140 -

Issues to Consider

This program received a "promising" rating. The evaluations of CARE found that the program resulted in faster rates of decline in risk factors for suicide and lower levels of depression and hopelessness among participants when compared with a usual-care control group. In addition, the treatment group resulted in lower levels of anxiety and anger for female students. Although studies have been limited to the Pacific Northwest, the CARE interventions were evaluated in schools with diverse ethnic populations. Findings across the evaluations were mixed, however, with the treatment groups demonstrating significant, positive effects compared with the control group for some outcome measures and time periods but not for others. Several limitations should be noted for this program. The results for specific suicide-related behaviors showed few differences between the treatment group and the usual-care control group. For both ethical and legal reasons, the use of a no-intervention control group was not possible, so the CARE participants were compared with a group who underwent a brief assessment interview along with usual-care follow-up that would be expected in schools dealing with students at risk of suicide. Many of the comparisons did not find significant differences between groups for the suicide-related outcomes, and the authors conclude that even the briefest of suicide-risk assessments and provisions of support (as in the control group) produce reductions in suicide risk. While this may be true, an alternate explanation might be that the lack of significant differences coupled with significant declines over time for all groups may have been because neither of the interventions or the usual-care control were responsible for a significant impact on the participants, but that these high-risk students would have demonstrated improved outcomes over time regardless of their participation in the study. Establishing causal effects of the programs is even more difficult given the voluntary basis for program enrollment, which may have resulted in a self-selection bias; i.e., students who were more interested in addressing negative feelings and risky behaviors may have been more likely to sign up for the interventions than students with less interest in change. Few differences were found between the CARE and CARE+CAST groups, and the only differences that emerged favored the CARE-only group. This suggests that the positive outcomes evidenced by the treatment participants were primarily the result of the four-hour CARE intervention, with the added six-week CAST program having little effect. Further support for the benefits of a brief intervention is found in studies of a semester-long and two semester-long early version of CAST (Eggert, Seyl, and Nicholas, 1990; Eggert et al., 1994; and Eggert et al., 1995), which found that the MAPS assessment protocol was just as effective as the longer programs. While CARE and CARE+CAST were found to lower youth substance use, so too did the usual care, and no significant differences were found among groups. Finally, it should be noted that the program developers also served as the evaluators of the program.

Example Sites

Seattle, WA

Contact Information

Reconnecting Youth Inc. P.O. Box 20343 Seattle, WA 98122 V: 425-861-1177 F: 888-352-2819 E: merridy@reconnectingyouth.com

- 141 -

mailto:merridy@reconnectingyouth.com

Available Resources

None at this time

Bibliography

Eggert, Leona L., Christine D. Seyl, and Liela J. Nicholas, "Effects of a School-Based Prevention Program for Potential High School Dropouts and Drug Abusers," The International Journal of the Addictions, Vol. 25, No. 7, 1990, pp. 773-801. Eggert, Leona L., Elaine A. Thompson, Brooke P. Randell, and Kenneth C. Pike, "Preliminary Effects of Brief School-Based Prevention Approaches for Reducing Youth Suicide-Risk Behaviors, Depression, and Drug Involvement," Journal of Child and Adolescent Psychiatric Nursing, Vol. 15, No. 2, 2002, pp. 48- 64. Eggert, Leona L., Elaine A. Thompson, Jerald R. Herting, and Liela J. Nicholas, "Erratum: Reducing Suicide Potential Among High-Risk Youth: Tests of a School-Based Prevention Program," Suicide and Life-Threatening Behavior, Vol. 29, No. 1, 1999, p. 96. Eggert, Leona L., Elaine A. Thompson, Jerald R. Herting, and Liela J. Nicholas, "Reducing Suicide Potential Among High-Risk Youth: Tests of a School-Based Prevention Program," Suicide and Life- Threatening Behavior, Vol. 25, No. 2, 1995, pp. 276-296. Eggert, Leona L., Elaine A. Thompson, Jerald R. Herting, Liela J. Nicholas, and Barbara Garii Dicker, "Preventing Adolescent Drug Abuse and High School Dropout Through an Intensive School-Based Social Network Development Program," American Journal of Health Promotion, Vol. 8, No. 3, 1994, pp. 202-215. Randell, Brooke P., Leona L. Eggert, and Kenneth C. Pike, "Immediate Post Intervention Effects of Two Brief Youth Suicide Prevention Interventions," Suicide and Life-Threatening Behavior, Vol. 31, No. 1, 2001, pp. 41-61. Thompson, Elaine Adams, Leona L. Eggert, Brooke P. Randell, and Kenneth C. Pike, "Evaluation of Indicated Suicide Risk Prevention Approaches for Potential High School Dropouts," American Journal of Public Health, Vol. 91, No. 5, 2001, pp. 742-752.

Last Reviewed

February 2007

Creating Lasting Family Connections

Program Info

Outcome Areas Healthy and Safe Children

Indicators Youths not using alcohol, tobacco, or illegal drugs

Topic Areas Age of Child Middle Childhood (9-12) Adolescence (13-18)

- 142 -

Type of Setting Elementary School Middle School High School Community-Based Service Provider Health Care Provider Type of Service Family Support Health Education Parent Education Youth Development Type of Outcome Addressed Behavior Problems Physical Health Substance Use and Dependence

Evidence Level Promising

Program Overview

Creating Lasting Family Connections is a family-based program developed by the Council on Prevention and Education: Substances, Inc. (COPES) that attempts to reduce alcohol and other drug use among teenagers by increasing family resiliency and community connections. The program was first evaluated as a research project in 1988 under the title Creating Lasting Connections. Initially, as evaluated, parents of teenagers attended 42 to 56 hours of classes and the teenagers attended 14 to 20 hours of classes, spread out over a six- to seven-month period. Parents’ classes, divided into three modules, included training on substance abuse knowledge and issues, family management skills, and communication techniques. Teenagers joined their parents for the communication techniques module. Follow-up care was also provided for one year (up to six months after the end of training) through bimonthly telephone consultations and/or home visits. The program also attempted to increase community strength by involving community members in program implementation and participant recruitment. Based on the results of the evaluation, the program was modified after 1993 and now offers more hours of instruction during a shorter time period. Teenagers and their parents each are engaged in training that lasts two and a half hours a night for 15- to 18 weeks.

Program Participants

Program Participants were high-risk teenagers age 12 to 14 and their parents. Teenagers were considered high risk if they demonstrated certain characteristics associated with high alcohol or other drug use. For the demonstration project studied in the evaluation, six church groups across five sites in the greater Louisville, Kentucky, area were recruited to implement the program. Churches were chosen based on their interest in the project, the numbers of connected at-risk families, the extent to which social services and programs were already offered, and whether church members had any experience administering social service programs. The chosen churches were located in urban, suburban, and rural locations. In most cases, churches recruited participants from their membership, but in one case, a church tried unsuccessfully to recruit from a nearby housing development. Four of the five sites that implemented the program had congregations that were predominantly white and the fifth site (a partnership between two churches of different denominations) had congregations that were predominantly black. Of the families that participated, 16 percent were black and the rest were white, and 30 percent were in low- to medium-income groups and the rest were higher-income.

- 143 -

Evaluation Methods

In the five-year demonstration project, churches recruited participants who were randomly assigned to either an experimental group or a control group (who received the intervention one year later). Parents and teenagers in both groups filled out questionnaires and participated in interviews at the beginning of the program and one year later. In addition, participants in the experimental group were interviewed immediately after the workshops ended (six to seven months after the start). Many interview and questionnaire items were drawn from standard psychological assessments, including the Personal Experience Inventory. A total of 143 parents and 183 teenagers began the program, and 97 parents (49 from the experimental group and 48 from the control group) and 120 teenagers (59 from the experimental group and 61 from the control group) completed all three waves of interviews. An analysis of the attrition rate found evidence of differences between waves that was corrected for in the final analysis. Evaluation findings compared pre- and post-intervention scores as well as scores between the experimental and control groups.

Key Evaluation Findings

The evaluation (Johnson et al., 1996) measured both short-term (six to seven months) and long-term (one year) gains. In addition, data were analyzed to find both direct effects and outcomes that occurred depending on another, intervening factor. Direct effects that were statistically significant included the following:

• Parents and teenagers made greater use of community services.

• Parents' skills in communicating with their children improved.

• Parents involved their children to a greater extent in setting rules relating to alcohol and other drug (AOD) use.

• In the black church community, parents decreased their alcohol use.

• Teenagers reported increased levels of bonding with their mothers.

The following additional effects were found:

• The onset of AOD use was postponed for at least a year as

o parents increased their AOD knowledge

o teenagers reported less conflict between them and their parents

o parents increased their use of punishment for AOD use.

• In the short term, teenagers reduced their alcohol use as

o parents increased their AOD knowledge

o parents decreased their use of cigarettes

o parents decreased their use of punishment for non-AOD use misconduct.

• In the short term, teenagers reduced their drug use as

o teenagers became more involved in setting non-AOD rules

o family communication increased

o parents decreased their use of alcohol

o teenagers reported less family conflict and family pathology.

• Long-term (one year) reductions in teenagers’ alcohol use were found as

- 144 -

o parents increased their AOD knowledge

o parents decreased their use of punishment for non-AOD misconduct

o parents used more community services when necessary

o family pathology decreased

o teenagers increased their communication with their families about AOD use

o teenagers increased their bonding with their fathers

o teenagers decreased their rejection of conventional values.

• Long-term (one year) reductions in teenagers’ other drug use were found as

o teenagers became more involved in setting non-AOD rules

o family pathology decreased

o teenagers increased their communication about AOD use and schoolwork

o teenagers decreased their rejection of conventional values

o parents reported an increase in teens' bonding with their fathers.

• Frequency of teens' alcohol use INCREASED as

o parents' use of punishment for non-AOD infractions increased;

o parents reported increased teen bonding with their mothers

There were no direct effects for the following measures:

• Use of alcohol and other drugs

• Families' involvement in community activities

• The extent to which families used or established AOD rules

• Teenagers' communication with their parents about AOD use and schoolwork

• Parents' reports of bonding with their children.

The program had an adverse affect on families’ use of non-AOD rules. The researchers suggested that this might have been because of the increased emphasis the program placed on AOD rules, perhaps to the unintended exclusion of non-AOD rules.

Probable Implementers

All sites included in the evaluation were church groups. Since the evaluation, the program has been replicated in 40 states by groups including recreation centers, criminal justice systems, school systems, and mental health centers.

Funding

The research demonstration was funded by a five-year federal demonstration grant from the Office for Substance Abuse Prevention (Grant Number 1279), U.S. Department of Health and Human Services. Other sources of funding include public health departments, foundations, and the government. COPES is available to provide free grant-writing assistance.

- 145 -

Implementation Detail

Program Design

• Recruitment of participants relied on already established church communities by having volunteers within each community be responsible for recruitment.

• Volunteers were provided with abbreviated program workshops before beginning recruitment.

• Program sites used the formal curriculum developed by COPES.

Curriculum

A formal curriculum for the workshops was used at the evaluated sites and the current curriculum is available through the COPES Web site. The curriculum as evaluated was developed by COPES between 1978 and 1988 and had three modules. The first module for the parents was titled “AOD Issues Training,” and provided information about substance abuse, including its history and its effects on individuals and families. The second module for parents was called “Not My Child”; it discussed parenting and discipline issues, including how to develop and implement expectations and consequences for children. The third module, which was provided to both parents and teenagers, was called “Straight Communications Training.” The module allowed parents and teens to first meet separately with their peer groups in order to use role playing to learn different communication styles. Parents and their children then met together to practice their newly learned skills. In addition, all families received case management and follow-up. Since the evaluation, the program has been modified, and the curriculum now consists of the following training modules:

• Developing Positive Parental Influences (for adults)

• Raising Resilient Youth (for adults)

• Getting Real (for adults and youth)

• Developing Independence and Responsibility (for youth)

• Developing a Positive Response (for youth)

Staffing

Staffing needs, including the proportion of paid staff and volunteers, varied among sites. Staffing for the demonstration sites included paid staff to recruit churches and church volunteers, groups of volunteers to recruit the families, and paid staff to teach the modules and provide follow-up services.

Issues to Consider

This program received a "promising" rating. The evaluation used experimental and comparison groups and showed several important mediating influences, such as improved parent-child relationships and parental knowledge of AOD, but it did not show that Creating Lasting Connections directly reduced alcohol and drug use among teenagers. Also, it is not clear whether the results of the Creating Lasting Connections demonstration project can be generalized to the overall population. The program as evaluated was church based only, and the characteristics of teenagers and parents who belong to churches could influence the efficacy of the program. In addition, Program Participants were relatively homogenous – most were middle class and the only ethnic groups included were Catholic and Protestant whites and blacks. Multiple evaluations of the replication sites are underway to measure how the program works in other communities, and the summary provided here will be updated as new research information becomes available.

- 146 -

Awards Received

Creating Lasting Family Connections has received recognition as a model program from the Center for Substance Abuse Prevention, OJJDP, and the International Youth Foundation. It also received recognition from the Office of National Drug Control Policy, the National Registry of Evidence-Based Programs and Practices, and the United Nations. It was rated "promising" by the U.S. Department of Education and "likely promising" by the Administration for Children and Families' Responsible Fatherhood Initiative.

Example Sites

The demonstration sites were located in Louisville, Kentucky. Groups from 40 states have since implemented the program.

Contact Information

For training and other information about the program, contact: Ted Strader, Executive Director Council on Prevention and Education: Substances, Inc. (COPES, Inc.) 845 Barrett Avenue Louisville, KY 40204 Phone: (502) 583-6820 Fax: (502) 583-6832 email: tstrader@sprynet.com Internet: www.copes.org and click on “Products”

Available Resources

Curriculum resources, including curriculum guides and participant workbooks are available for purchase at www.copes.org. COPES also provides consultation, including grant-writing assistance, and a master training program.

Bibliography

Johnson, K., D. Bryand, D. Collins, et al., Preventing and Reducing Alcohol and Other Drug Use Among High-Risk Youths by Increasing Family Resilience, Social Work, Vol. 43, pp. 297-308, 1998. Johnson, K., T. Strader, M. Berbaum, et al., Reducing Alcohol and Other Drug Use by Strengthening Community, Family, and Youth Resiliency: An Evaluation of the Creating Lasting Connections Program, Journal of Adolescent Research, Vol. 11, No. 1, pp. 36-67, 1996. Strader, T., D. Collins, T. Noe, and K. Johnson, Mobilizing Church Communities for Alcohol and Other Drug Abuse Prevention Through the Use of Volunteer Church Admocate Teams Journal of Volunteer Administration, Vol. 15, pp. 16-29, 1997.

Last Reviewed

August 2009

- 147 -

mailto:tstrader@sprynet.com

http://www.copes.org/

http://www.copes.org

DARE to be You

Program Info

Outcome Areas Children Ready for School

Indicators Children ages 0 to 5 exhibiting age-appropriate mental and physical development

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Community-Based Service Provider Type of Service Family Support Health Education Parent Education Type of Outcome Addressed Behavior Problems Substance Use and Dependence

Evidence Level Proven

Program Overview

DARE to be You (DTBY) focuses on improving the parenting skills of parents of young children (age 2 to 5) in order promote children’s resiliency to problems later in life, which can, in turn, reduce children's alcohol and drug use as they grow up. The traditional DTBY focuses on reducing alcohol and drug abuse among 5 to 18-year-olds, but the program described here treats only preschoolers and their parents. There were several components of the program: a children's component, a parents' component, training for child-care providers, and training for social service agency workers who work with families. The program was found to be most effective when provided in two-hour-long blocks for each session, with the sessions given over a 10- to 12-week period. Each series of workshops for parents, their young children, and the children's siblings included 10 to 25 parents and their children. Each session began with a joint activity for parents and children and then continued with separate activities for participants based on their age. Each session focused on a different program objective, mostly related to improving parenting techniques. The objectives included:

• Improve parents' self-esteem.

• Increase parents' realization that consequences are brought about by their actions, rather than by fate, chance or a "Powerful Other," thereby changing the "locus of control" of consequences from an external source to an internal source.

• Enhance decision-making skills through effective reasoning.

• Increase communication skills between parents and children, particularly to improve children's self-esteem, decision making, and problem-solving skills.

• Learn effective stress management.

- 148 -

• Learn the speed at which children should develop in order to decrease unrealistic expectations.

• Strengthen peer support and reduce isolation.

Families were also offered annual reinforcement workshops that consisted of four two-hour long sessions. The workshops addressed parents' requests for more information or more skill-building exercises related to the different focuses of the original program objectives. For example, issues surrounding child development was a common topic. Ongoing group activities, including monthly support groups and parent potlucks, were also offered. Participants chose the topics to be covered in those meetings. At all events, child care for the infants of Program Participants was provided and parents were paid a small cash incentive for participating. Concurrent with the workshops for families, DTBY provided training for other caregivers, including teenagers and family-oriented agencies. These caregivers received up to 20 hours of training in working with preschool children.

Program Participants

Program Participants were at-risk children age 2 to 5 and their parents. Risk factors for alcohol use included foster care, child abuse, a parent who dropped out of high school, low annual income, and family history of mental illness or substance abuse. However, to avoid any stigma being attached to Program Participants, some families who were not considered to be high risk were also included in the program. Although all four of the experimental DTBY sites were in Colorado, participants varied by ethnicity, location (urban or rural), and income levels. The sites included a very poor and isolated Native-American community; a poor and primarily Hispanic rural community with high rates of child abuse and drunk-driving arrests; and a largely white, semi-rural community with few available social services and high rates of poverty, unemployment, substance abuse, and child abuse; and an urban location with the state’s highest rates of child abuse and teen pregnancy. New families were recruited into the program each year for five years.

Evaluation Methods

Eligible families in each of the four sites were randomly assigned to either the treatment group or control group. On average, each site included 28 treatment and 17 control families each of the five years. Standard psychological assessments were administered to all participating families before and after treatment. Post-tests were administered to the experimental group immediately after the program's end; post-tests for all families were given one year after the program began and one year again after that. The attrition rate reached 29 percent by the second-year follow-up, but there were few significant differences between the families who dropped out and the families who completed the follow-up tests. In addition, evaluators surveyed participants on their perceptions of the workshops in order to determine whether the same content was taught across sites.

Key Evaluation Findings

The evaluators measured the program according to the program objectives discussed in each workshop (Miller-Heyl et al., 1998). The findings in regard to each program objective included the following:

• Parental self-esteem: Parents in the treatment group, whose self-esteem was measured before and after the workgroup sessions, showed significantly increased feelings of confidence in their parenting skills. In addition, at both the one-year and the two-year mark, parents in the treatment group showed higher levels of self-esteem than parents in the control group.

• Locus of control: Between the pre-test and post-test periods, there was a significant decline in the treatment group's belief that chance controls outcomes and a smaller decline in the

- 149 -

group’s belief that powerful others determine outcomes. However, over time, both the treatment and the control groups showed similar declines in those beliefs in the subsequent follow-ups. Thus, the treatment was shown to be effective in the short-term, but less so over time, as compared to the control group.

• Parents' reasoning skills: Between Year 1 and Year 2 of the program, parents in the treatment group appeared to blame themselves and their children for children’s poor behavior less than parents in the control group.

• Parental communication with their children that leads to effective discipline: Three areas associated with this measure were tested: overly harsh punishment, disciplining effectively, and setting limits for children. Tests measuring these areas showed no change among parents in the control group, but the scores of parents in the experimental group showed improvements through the two year follow-up period.

• Child behavior: The goal of this measure was to determine if children exhibited more age- appropriate behavior after participating in the program and if parents recognized this behavior as such. Evaluators asked parents and other caregivers to note whether a child demonstrated a particular behavior among a checklist of behaviors, and, if so, whether the parents perceived the behavior as being inappropriate. Children exhibited significant increases in their development and age-appropriate behaviors, both compared with their pre-test scores and with the control group. In addition, certain misbehavior – notably "oppositional behaviors" such as engaging in frequent and irrational arguments – decreased significantly as compared with the control group. A notable effect from the program that possibly decreased oppositional behaviors was an increase in parents' knowledge of typical behavior for a preschooler and/or an increase in the use of more-effective discipline on their part.

• Social support networks and peer support: Between the pre-test and the first-year post-test, parents in the experimental group increased their satisfaction with available social support networks more so than parents in the control group did. However, there was no evidence that DTBY changed parents' actual social networks.

Probable Implementers

Community organizations in a variety of high-risk communities.

Funding

Funding for the evaluation was provided by the Center for Substance Abuse Prevention, U.S. Department of Health and Human Services. DTBY is based at Colorado State University's Cooperative Extension, but funding sources and costs vary locally.

Implementation Detail

Program Design

Family service agencies that worked both with the program and with the involved families were asked about the most significant aspects of DTBY. Almost all (98 percent) said that they believed its curriculum was its main strength. More than two-thirds (68 percent) said that its structural support aspects, such as providing infant child care, incentive pay, and follow-up sessions, were also important. Fewer respondents (29 percent) said that program staff was a main strength of the program and even fewer (10 percent) said the same about cultural sensitivity.

Curriculum

The curriculum used by the evaluation sites is available from the Colorado State University Cooperative Extension DARE to be You Program (see Available Resources). The curriculum included joint parent-child workshops, classes for parents, and age-appropriate children's activities. Group discussion and interactive activities predominated, and some lectures also were offered.

- 150 -

Staffing

Staff members for the evaluation sites included a program coordinator and teachers for parents and children. Several teachers were needed at each site because of the multiple age ranges of the children.

Issues to Consider

DTBY received a "proven" rating. Although the program is designated as being proven, it is only proven for socio-emotional development of a child and not for prevention of drug abuse. The affected outcomes are only indirectly related to the professed program goal of reducing substance abuse. There is no evidence that the program prevents drug abuse, as the children were not followed into adolescence. On the other hand, the experimental design, including randomized assignment, a comparison group that received no intervention, and ample sample size, provides evidence that the program positively affects children's socio-emotional development. DTBY runs two other substance abuse prevention programs: one is for teen parents and their children and the other is a school-based program for Grades K–12. These programs are not included in this evaluation; therefore, the findings here regarding DTBY apply only to the program for parents of 2- to 5-year-olds. If evaluation findings for the other DTBY programs become available, they will be reviewed and this summary will be updated as necessary. It should also be noted that the person who developed and implemented the DTBY program conducted the evaluation.

Example Sites

In addition to the four sites included in the evaluation, DTBY has also been implemented in the following communities or among the following groups:

• Pueblo County Cooperative Extension, Colorado

• Navajo Community, BIA, Shiprock Department of Education, New Mexico, Arizona, and Utah

• Asian Association of Salt Lake City, Utah

• African-American and Hispanic communities in Contra Costa County, California

Contact Information

Jan Miller-Heyl DARE to be You Colorado State University Cooperative Extension 1413 N. Mildred Rd. Cortez, CO 81321 Phone: (970) 222-9649 E-mail: jan.miller-heyl@colostate.edu Additional contacts: david.macphee@colostate.edu and aimee.walker@colostate.edu Internet: www.coopext.colostate.edu/DTBY

- 151 -

mailto:jan.miller-heyl@colostate.edu

mailto:david.macphee@colostate.edu

mailto:aimee.walker@colostate.edu

http://www.coopext.colostate.edu/DTBY

Available Resources

The following program materials are available:

• DARE to be You Replication Manual for the DARE to be You Program for Families of Preschool Youth, Caregivers, and Community (2000)

• DARE to be You Parent Training Guide insert packet (1998)

• DARE to be You Preschool Activity Guide (1992)

• DARE to be You Parent Training Guide (1991)

• DARE to be You K-12 Life Skills and Substance Abuse Prevention Curriculum (five-volume set) (1988)

• DARE to be You Leaders' Manual, 2nd ed. (1985).

Bibliography

Miller-Heyl, J., D. MacPhee, and J. Fritz, DARE to be You: A Family-Support, Early Prevention Program, Journal of Primary Prevention, Vol. 18, pp. 257-285, 1998.

Last Reviewed

March 2013

Direct Instruction

Program Info

Outcome Areas Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards Students graduating from high school

Evidence Level Promising

- 152 -

Program Overview

Direct Instruction (DI) is a highly structured approach to instruction designed to accelerate the learning of at-risk students. Previously known as the DISTAR (Direct Instruction Systems for Teaching Arithmetic and Reading) program and Project Follow Through, DI is based on the theory that learning is maximized when instructional presentations are clear, likely misinterpretations are eliminated, and generalizations are facilitated. Classroom teachers learn how to define tasks clearly, build toward more-complex concepts, use interactive lessons with large and small groups, use frequent praise for responses, and recognize and correct errors immediately. To maximize time spent on tasks, students are placed in instructional groups based on similar performance, and grouping may take place across classes and grades. DI can be used as a school-wide program, or the reading/language arts and math portions of the program can be implemented separately. Individual DI programs are currently used in more than 10,000 schools throughout the world, and more than 500 schools use DI on a school-wide basis.

Program Participants

Students in preschool through grade 12, with a focus on kindergarten through grade six (K-6)

Evaluation Methods

Nearly 100 studies of the Direct Instruction program have been undertaken since the late 1960s, although the majority of the evaluations of the program did not utilize methodologies that were rigorous enough to meet Promising Practices Network (PPN) criteria for inclusion in this review. A total of 20 evaluations, representing 17 different study populations, were identified that met PPN criteria for methodological rigor, and characteristics of those studies are summarized below. Student outcomes were evaluated using a wide range of measures assessing skills in vocabulary, reading, language, mathematics, and general cognitive abilities. These instruments included validated measures such as the Wide Range Achievement Test, the Stanford Achievement Test, the Peabody Individual Achievement Test, the Illinois Test of Psycholinguistic Abilities, the Comprehensive Test of Basic Skills, and the Stanford-Binet Intelligence Scale. Other outcome measures included high school graduation rates, grade retention rates, highest grade level achieved, household income and employment, school misconduct, arrest records, and self-reported juvenile delinquency. The 17 studies of the Direct Instruction program reviewed here employed rigorous study designs. Eight of the studies used a quasi-experimental design, while the other nine were randomized trials in which Program Participants were randomly assigned to receive the DI program or a different program. When creating the groups of students to be compared, the researchers attempted to group together students in the DI program and the comparison program who were similar in terms of academic ability. Several of the studies evaluated long-term effects following participation in the DI program, ranging from one to 20 years. The largest of the studies reviewed here included 245 students who participated in the DI program. DI was evaluated as implemented with students over a wide range of grade levels, from preschool through grade 6. Some studies included only students in regular classrooms, while other studies focused on students in special-education programs. Students from different racial and ethnic groups were included in the studies, as were students from different household income levels. The studies were conducted throughout the country, in both urban and rural settings. Specifically, studies were conducted in Alaska, California, Illinois, Michigan, Mississippi, New York, and Washington State. Also included in this review is a study conducted in Great Britain. In many studies, DI was compared with other educational programs, including the regular classroom curriculum and alternative educational programs. Examples of the alternative programs include Phonetic Keys to Reading (PKR) (Williamson, 1970); Johnny Right-to-Read (Summerell and Brannigan, 1997); Integrated Skills Method (Richardson et al., 1978); Palo Alto Reading Program (Stein and Goldman, 1980); Peabody Language Development Kit (Mosley, 1980); Ginn Language Development Program (Mosley and Plue, 1980); Open Court Language Development Program (Mosley and Plue,

- 153 -

1980); Action Reading (Rawl and O’Tuel, 1982); English Colour Code (Lewis, 1982); Mediated Learning (Dale and Cole, 1988; Cole, Mills, and Dale, 1989; Cole, Dale, and Mills,1991; Cole et al., 1993; Mills et al., 1995; Mills et al., 2002); Harcourt Brace Jovanovich (HBJ) Basal Reading Program (Sexton, 1989); Integrated Reading-Writing program (Traweek and Berninger, 1997); and High/Scope (Schweinhart and Weikart, 1997).

Key Evaluation Findings

Studies Finding Positive Effects of Direct Instruction

Four studies reported statistically significant, positive findings for DI in contrast with a comparison group. Outcomes included reading, language ability, auditory language comprehension, math, and high school graduation rate. Stein and Goldman's (1980) study of 53 students age 6 through 8 reported that

• a statistically significant difference favoring the DI students over the Palo Alto Reading Program students was found for reading/recognition scores.

A 15-year longitudinal study (Meyer, 1984) of three cohorts of DI students in Brooklyn, New York found that

• significantly more students in the DI cohorts than in the control cohorts graduated from high school, with average graduation rates of 60 percent and 38 percent, respectively.

• results on the California Achievement Test (CAT) reading and math scores indicated that the average of all DI-group students was significantly higher than the average of the control- group students.

A study comparing students in DI with students in the Harcourt Brace Jovanovich Basal Reading Program (Sexton, 1989) found that

• students in the DI group significantly outscored students in the HBJ group on a measure of language ability.

A study of 45 kindergarten students (Benner et al., 2002) reported that

• statistically significant differences favoring the DI group were found for all four of the Test of Auditory Comprehension of Language subtests, with effect sizes ranging from small to moderate.

Studies Finding Mixed Effects of Direct Instruction

Eight studies reported mixed findings for the effects of DI, with DI students outscoring comparison students on some measures but no significant differences between groups on others. In three of the studies, students in comparison programs (the Peabody Language Development Kit; Action Reading) and/or control groups significantly outscored DI students on some study measures. Williamson’s (1970) study comparing DI students with Phonetic Keys to Reading and control group students reported that

• on three types of achievement tests, DI students scored better than students in the PKR and control groups. The evidence of improvement was strongest for DI students with initial low reading ability, and mixed for students of moderate initial reading ability.

A study of 24 second-graders by Summerell and Brannigan (1977) found that

• student gains from pre-test to post-test on the Paragraph Meaning subtest of the Stanford Achievement Test (SAT) were significantly greater for the DI group than for the Johnny-Right- to-Read group.

- 154 -

• no significant differences were found between the groups on the Word Meaning subtest of the SAT.

A study of 140 students from ten Head Start classes in Mississippi (Mosley and Plue, 1980) reported that

• analysis of student gains on the Illinois Test of Psycholinguistic Abilities between pre-test and post-test indicated that DI students scored significantly higher than the control group (regular reading curriculum with no prescribed manual) and Open Court students.

• No significant differences were found between the DI and Ginn Language Development Program groups.

• Students in the Peabody Language Development Kit group (an alternative curriculum) demonstrated significantly higher gain scores than the DI students.

Rawl and O'Tuel’s (1982) study of 96 Alaskan kindergarten children participating in DI, Action- Reading, or a control group reported that

• there were no significant differences between groups for Comprehensive Test of Basic Skills (CTBS) Alphabet skills.

• both the Action-Reading and the control group scored significantly higher than the DI group on the CTBS Visual Auditory Discrimination subtest.

• results for the CTBS Language subtest indicated no significant differences between the DI students and the control group, but significantly higher scores for the DI students than the Action-Reading students.

• similarly, for scores on the CTBS Total Pre-reading subtest, DI students did not differ significantly from control students but scored significantly higher than Action-Reading students.

• the CTBS Mathematics subtest scores indicated that the control group scored significantly higher than the DI group, but there were no significant differences between the DI group and the Action-Reading group.

A study of 52 British remedial readers (Lewis, 1982) receiving DI, students in the English Colour Code program, and a regular remedial-reading control group found that

• students in the DI group outperformed students in both the English Colour Code program and control group in two of eight assessments of reading—specifically, assessments of reading accuracy and comprehension.

• no significant differences among the DI group, the English Colour Code program group, and the control group were observed for the other six assessments.

A three-study assessment of DI by Kameenui et al. (1986) reported the following:

• Study 1: DI students significantly outscored control students on numerical-problem training scores and maintenance tests and on picture problems.

• Study 2:

o Middle-ability Grade 1 students: Students in the DI group scored significantly higher than students in the control group on both the post-test and the transfer test.

o High-ability Grade 2 students: DI students scored significantly higher than control students on the post-test, but not on the transfer test.

• Study 3: Statistically significant differences favoring the DI group were found for the training- probe measures. No significant differences were noted between groups on the transfer test or on the maintenance measure.

- 155 -

Dale and Cole’s (1988) study of 105 children age three to seven years old reported mixed results, specifically,

• DI students significantly outscored control students on some tests (Test of Early Language Development (TELD) raw score and TELD Language Quotient), while control students performed better on other outcomes (McCarthy Verbal and Memory Scales and McCarthy General Cognitive Index).

Yu and Rachor’s (2000) study of 93 fourth-grade, 71 fifth-grade, and 81 sixth-grade students in DI found that DI had a more positive impact in the higher grades. Specifically,

• fourth-grade students in the control group significantly outperformed DI-group students in gains in reading proficiency scores between pre-test and post-test.

• no significant gains in reading proficiency scores were noted between the DI or control group fifth-grade students.

• A statistically significant difference was found in gains in reading proficiency scores for sixth- graders, with DI students improving by a larger amount than control students.

Studies Finding No Significant Effects of Direct Instruction Four studies found no significant results for students participating in the DI program compared with other students in control groups. Those studies include

• Richardson et al.'s (1978) study of 72 second- through sixth-grade students from New York City

• two-year results by Cole, Mills, and Dale (1989)

• Mills et al.'s (1995) age-9 follow up to Cole et al. (1993)

• Traweek and Berninger's (1997) study comparing first-grade students in the Integrated Reading-Writing program and DI.

Direct Instruction Results for Special-Education Settings Cole, Dale, and Mills’ (1991) study of 107 children in a special-education program in Seattle found

• no significant differences between groups for seven of eight measures of reading and language skills.

• in the case of the Peabody Picture Vocabulary Test Standard Score, Mediated Learning students outscored DI students.

The study by Cole et al. (1993) of 164 kindergarteners enrolled in a special-education program reported that

• there were no significant differences between groups on any of the seven language-skills measures.

Direct Instruction Results for Nonacademic Outcomes The study by Mills et al. (2002), an age-15 follow-up to Cole et al. (1993), found

• no significant differences between groups for any of the self-reported delinquency acts, including personal violence, property damage, stealing, drug abuse, and status offenses.

The age-23 follow-up study of students who received DI in preschool compared with students in High/Scope and a traditional Nursery School program (Schweinhart and Weikart, 1997) reported the following:

- 156 -

• At age 23, the DI group had experienced significantly more felony arrests per person than did the other two groups, significantly higher rates of felony arrests at age 22 and older, and significantly higher rates of arrest for property crime.

• At age 23, no significant differences were found between groups for

o highest year of schooling attained, high school graduation/GED rates, or on-time high- school graduation

o rates of past and present pregnancies

o employment over the past five years, current employment, monthly earnings from work, monthly income from all sources, months on welfare in the past ten years

o number of times suspended or expelled from high school,

o lifetime arrests (juvenile and adult), misdemeanors, arrests for drug-related crimes, violent-crime arrests, felony arrests at age 17 through 21, and felony convictions and prison sentences.

• Between the ages of 5 and 15, virtually no differences were found in intellectual or academic performance among the three groups. The only exception was at age 5, when the DI group scored a significantly higher average IQ on the Stanford-Binet Intelligence Test than did the Nursery School group.

Probable Implementers

Public and private preschools and public K–12 schools

Funding

Schools implementing the DI program frequently use Title I monies, or apply for grant funds. DI is also used in many Reading First schools.

Implementation Detail

Program Design

DI can be used as a school-wide comprehensive reform program, or specific programs (the reading, language arts, and math curricula) can be implemented individually. Although DI’s original focus was on reading, language, and math, the program has since been expanded to include social and physical sciences, fact-learning, and handwriting. DI provides highly scripted and interactive lessons for small, homogeneous groups of students. Groupings may vary across subjects, accounting for the fact that students may be weak in one subject and strong in another. Instruction using the DI curriculum is fast-paced and involves frequent interaction between teachers and students. A placement test is used to assign students to the appropriate group, and the performance level of each group dictates the pace of instruction. Frequent assessment of student progress ensures the continued appropriateness of achievement-level grouping. Teachers monitor student performance every five to ten days, using methods such as calculating reading speed and reading-error ratios. These data, along with weekly grades, are used to regroup students according to performance level.

Materials

When implementing DI, teachers use "presentation books," which are lesson plans that include instructions for monitoring and assessing student progress and for providing immediate feedback to students. Curricular materials, daily lessons, and teachers' guides are available for grades K-6 in reading, language arts, spelling, and math; grades 4-6 in expressive writing; grades 3-6 in science; grades 3-12 in corrective reading; and grades 4-12 in corrective math.

- 157 -

Staffing

The most significant DI organizational requirement for schools is the recommendation that all teachers of reading and English-language arts (and other subject areas, if desired) be scheduled to teach that subject at the same time during the school day. This practice allows for cross-class grouping of students of similar achievement levels. Schools are encouraged to have a peer coach (facilitator) to help teachers implement the program. Classroom paraprofessionals can also be integrated into the program, working as instructional aides, one-on-one tutors, and small-group leaders. Implementation Support/Professional Development

Implementation support and professional development can be contracted from various providers (see Available Resources below). DI’s training program begins with a one-week, on-site, pre- implementation session for all staff. Implementation managers visit each school approximately four days per month for three years and help to model techniques, observe classrooms, address problems that teachers are having, and assist in the grouping of students. Weekly one-hour in-service sessions for teachers are also recommended, during which teachers may learn and practice DI techniques.

Project Costs

In 2005, the first-year cost of adopting school-wide DI is approximately $245,000 for an average school of 500 students. Schools implementing DI will incur costs for training and technical assistance, personnel, and materials. The following costs are for an average school of 500 students and 20 to 25 teachers: Technical assistance will cost $65,000 to $75,000 a year for three to five years, which includes direct costs for teacher training at the start of and during the school year, but does not include the cost of faculty time devoted to training. Training time amounts to five days of pre- implementation at the start of the school year for the entire faculty, plus at least one hour per week (approximately 4.5 days per year) for each teacher. Finally, curricular materials, purchased separately, cost approximately $210 per child in the first year, $165 per child in the second year, and $65 per child in subsequent years.

Issues to Consider

This program received a "promising" rating. Many of the evaluations of Direct Instruction were experimental in design, and those that were quasi-experimental used reasonably convincing comparison groups. The results varied significantly from study to study, with ten of the studies reporting mixed results, six reporting no significant differences between groups, and four reporting solely positive and significant differences for DI students. This lack of consistency among studies suggests that the overall evidence of DI effectiveness is limited. However, when results are considered across all studies, the majority of the evaluations reported at least some significant benefit accruing to students who participated in the Direct Instruction program. These benefits were found on standardized tests of general cognitive skills, reading, and mathematics, and on high school graduation rates. A pattern to note is that more-recent studies of the DI program, conducted in the 1990s and later, were less likely to report significant, positive program effects for students participating in Direct Instruction. It is unclear whether this is an artifact of study methodology, more-effective comparison group programs, changes in student learning patterns, or something else. Two studies assessed the Direct Instruction program in special-education settings, and neither reported significant findings favoring DI. Furthermore, on one measure in the study by Cole, Dale, and Mills (1991), Mediated Learning students significantly outscored DI students. Additionally, two studies assessed the effects of DI on juvenile and young-adult crime and arrest rates, and DI was not shown to be effective at reducing or preventing juvenile delinquency.

- 158 -

Example Sites

The Direct Instruction program has been implemented in thousands of sites around the United States and in other countries. Some examples include Fairbanks, Alaska; Champaign and East St. Louis, Illinois; Flint and Ypsilanti, Michigan; Tupelo, Mississippi; Brooklyn, New York; Dayton. Ohio; Williamsburg County, South Carolina; Smithville, Tennessee; Seattle, Washington; Great Britain; and Canada.

Contact Information

Bryan Wickman Executive Director Association for Direct Instruction P.O. Box 10252 Eugene, OR 97440 Tel (800) 995-2464 Fax (541) 868-1397 info@adihome.org Or Kurt Engelmann National Institute for Direct Instruction PO Box 11248 Eugene, OR 97440 Tel (877) 485-1973 Fax (541) 683-7543 kurt@nifdi.org

Available Resources

There are several key contacts for schools or districts interested in implementing the DI program. The Association for Direct Instruction (http://www.adihome.org/) is a nonprofit organization dedicated to promoting and supporting the use of DI programs. Support includes several annual regional DI conferences, an annual national conference, publications, online networking and assistance, and two semiannual publications. The National Institute for Direct Instruction (NIFDI) is a not-for-profit corporation dedicated to providing school districts with a solid training program and approach for the implementation of DI in districts, schools, and classrooms (http://www.nifdi.org/). DI is a commercially published program, and curriculum materials are published by Science Research Associates (SRA), a division of McGraw-Hill (http://www.sraonline.com/index.php/home/curriculumsolutions/di/9). Materials may be purchased by individual grade and subject, or in a package suitable for school-wide implementations. J/P Associates, Inc. provides access to Direct Instruction materials and staff development resources. They have also developed supplemental programs to support, extend, and enhance the Direct Instruction program. Information is available at www.jponline.com, or by contacting Robert Harris, Executive Director of Programming at info@jponline.com.

Bibliography

Benner, Gregory J., Alexandra Trout, Philip D. Nordness, J. Ron Nelson, Michael H. Epstein, Maria- Louisa Knobel, Alice Epstein, Ken Maguire, and Rodney Birdsell, "The Effects of the Language for Learning Program on the Receptive Language Skills of Kindergarten Children," Journal of Direct

- 159 -

mailto:info@adihome.org

mailto:kurt@nifdi.org

http://www.adihome.org/

http://www.nifdi.org/

http://www.sraonline.com/index.php/home/curriculumsolutions/di/9

http://www.jponline.com/

mailto:info@jponline.com

http://www.nifdi.org/

Instruction, Vol. 2, No. 2, 2002, pp. 67–74. Cole, Kevin N., Paulette E. Mills, and Philip S. Dale, “A Comparison of the Effects of Academic and Cognitive Curricula for Young Handicapped Children One and Two Years Postprogram,” Topics in Early Childhood Special Education, Vol. 9, No. 3, 1989, pp. 110–127. Cole, Kevin N., Philip S. Dale, and Paulette E. Mills, "Individual Differences in Language Delayed Children's Responses to Direct and Interactive Preschool Instruction," Topics in Early Childhood Special Education, Vol. 11, No. 1, 1991, pp. 99–124. Cole, Kevin N., Philip S. Dale, Paulette E. Mills, and Joseph R. Jenkins, “Interaction Between Early Intervention Curricula and Student Characteristics,” Exceptional Children, Vol. 60, No. 1, 1993, pp. 17–28. Dale, Philip S., and Kevin N. Cole, “Comparison of Academic and Cognitive Programs for Young Handicapped Children,” Exceptional Children, Vol. 54, No. 5, 1988, pp. 439–447. Kameenui, Edward J., Douglas W. Carnine, Craig B. Darch, and Marcy Stein, "Two Approaches to the Development Phase of Mathematics Instruction," The Elementary School Journal, Vol. 86, No. 5, 1986, pp. 633–650. Lewis, Ann L., "An Experimental Evaluation of a Direct Instruction Programme (Corrective Reading) with Remedial Readers in a Comprehensive School," Educational Psychology, Vol. 2, No. 2, 1982, pp. 121–135. Meyer, Linda A., "Long-Term Academic Effects of the Direct Instruction Project Follow Through," The Elementary School Journal, Vol. 84, No. 4, 1984, pp. 380–394. Mills, Paulette E., Kevin N. Cole, Joseph R. Jenkins, and Philip S. Dale, "Early Exposure to Direct Instruction and Subsequent Juvenile Delinquency: A Prospective Examination," Exceptional Children, Vol. 69, No. 1, 2002, pp. 85–96. Mills, Paulette E., Philip S. Dale, Kevin N. Cole, and Joseph R. Jenkins, "Follow-Up of Children from Academic and Cognitive Preschool Curricula at Age 9," Exceptional Children, Vol. 61, No. 4, 1995, pp. 378–393. Mosley, Beatrice B., and W. V. Plue, “A Comparative Study of Four Curriculum Programs for Disadvantaged Preschool Children,” Hattiesburg. Miss.: University of Southern Mississippi, 1980. Rawl, Ruth K., and Frances S. O'Tuel, "A Comparison of Three Prereading Approaches for Kindergarten Students," Reading Improvement, Vol. 19, No. 3, 1982, pp. 205–211. Richardson, Ellis, Barbara DiBenedetto, Adolph Christ, Mark Press, and Bertrand G. Winsberg, “An Assessment of Two Methods for Remediating Reading Deficiencies,” Reading Improvement, Vol. 15, 1978, pp. 82–94. Schweinhart, Lawrence J., and David P. Weikart, "The High/Scope Preschool Curriculum Comparison Study Through Age 23," Early Childhood Research Quarterly, Vol. 12, 1997, pp. 117–143. Sexton, C. Waynel, "Effectiveness of the DISTAR Reading I Program in Developing First Graders' Language Skills," Journal of Educational Research, Vol. 82, No. 5, 1989, pp. 289–293. Stein, Claudia L’E., and Jacquelin Goldman, “Beginning Reading Instruction for Children with Minimal Brain Dysfunction,” Journal of Learning Disabilities, Vol. 13, No. 4, 1980, pp. 52–55. Summerell, Sally, and Gary G. Brannigan, "Comparison of Reading Programs for Children with Low Levels of Reading Readiness," Perceptual and Motor Skills, Vol. 44, No. 3, 1977, pp. 743 - 746.

- 160 -

Traweek, Dean, and Virginia Wise Berninger, "Comparisons of Beginning Literacy Programs: Alternative Paths to the Same Learning Outcome,” Learning Disability Quarterly, Vol. 20, No. 2, 1997, pp. 160 - 168.

Williamson, Florence, “DISTAR Reading--Research and Experiment,” Illinois University, Urbana, 1970. Yu, Lei, and Robert Rachor, “The Two-Year Evaluation of the Three-Year Direct Instruction Program in an Urban Public School System,” Paper presented at the Annual Meeting of the American Educational Research Association, 2000, New Orleans, LA.

Last Reviewed

September 2005

Early Childhood Education and Assistance Program (ECEAP)

Program Info

Outcome Areas Strong Families

Indicators Families increasing economic self-sufficiency

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Community-Based Service Provider Health Care Provider Home Visiting Type of Service Family Support Health Care Services Instructional Support Parent Education Type of Outcome Addressed Cognitive Development/School Performance Physical Health Poverty/Welfare

Evidence Level Promising

Program Overview

In 1985, the State of Washington began developing statewide comprehensive early childhood education and assistance services to support the healthy development and success of children in low- income families or children who are otherwise at risk of school failure. Washington's Early Childhood Education and Assistance Program (ECEAP) is administered by the Washington State Office of Community Development (OCD).

- 161 -

ECEAP is composed of four interactive components: education, health and nutrition, parent involvement, and family support. These components collectively identify problems that hinder learning; provide health screenings and immunizations for children; encourage parental involvement in the classroom and in the program itself through local parent-run policy councils; assess family needs and refer families to community resources; and provide adults with training to improve their parenting, leadership, and self-sufficiency skills. ECEAP is a community-based, family-focused, comprehensive, pre-kindergarten program designed to help children and their families who are in poverty. The program focuses on helping three- and four- year-olds prepare for and succeed in school while helping their parents progress toward self- sufficiency. ECEAP operates 260 program sites locally through 35 contractors, including school districts, educational services districts, local governments, nonprofits, childcare providers, and tribal organizations. Throughout the past 16 years, ECEAP has significantly increased the number of children it has served, from 1,000 served in 1986 to 7,879 served during the 2000-2001 program year. The program has served more than 90,000 children and families since its inception.

Program Participants

Eligible participants are children who are at least 3 years old and are not yet enrolled in kindergarten and whose families have been at or below 110 percent of the federal poverty level for the past 12 months. Priority is given to enrolling eligible 4-year-olds, though 3-year-olds can be served as space is available. Ten percent of program slots are available to children from over-income families who are at- risk of school failure for other reasons, such as developmental delays. About 50 percent of the participants are Caucasian, 29 percent are Hispanic, 13 percent are African-American, and the rest are split among various other ethnicities.

Evaluation Methods

The Washington State Early Childhood Assistance Act of 1985 mandated an external evaluation of ECEAP. The Northwest Regional Educational Laboratory (NREL) conducted a longitudinal study from 1988 to 2000 to measure outcomes of enrolled children and families. The study examined ECEAP's effectiveness in preparing these children to achieve educational success, and later, to measure family well-being, changes in social status and family resources, and dependence on public assistance. The longitudinal study completed 12 years of data collection and evaluation, in which it followed 1,358 children drawn from groups selected over three consecutive years beginning in 1988. A comparison group of 322 children who were eligible but not served by the program was also established. The comparison sample was not a random sample, but was composed of children in the same schools as ECEAP children who matched the ECEAP children on age, gender, ethnicity, primary language, and level of poverty. However, a much larger percentage of the ECEAP group was at or below the poverty level at the start of the study than was in the comparison group (95 percent versus 53 percent). In addition, the follow-up rate among the comparison group had been about 65 percent while the follow- up rate among the ECEAP participants had only been about 55 percent. By 2000, evaluators felt that the longitudinal study was no longer providing significant information, and it was ended in favor of developing a yearly outcomes evaluation for enrolled children and families. The new study design is currently being developed and possible outcomes have been identified but not finalized.

- 162 -

Key Evaluation Findings

The Year 8 Longitudinal Study found that:

• The fraction of ECEAP families above the poverty level grew from 5 percent at enrollment to 47 percent. The fraction of control group families above the poverty level grew more modestly over the same period -- from 47 to 61 percent.

The Years 9 and 10 Study showed that:

• Between enrollment and year 9 of the evaluation, the percent of ECEAP participants above the poverty level rose from 5 to 57 percent (a ten-fold increase). Over the same period, the growth in the percent above the poverty level was smaller for the comparison group (47 percent to 68 percent, a 44 percent increase).

Probable Implementers

School districts, local and state governments, nonprofit organizations, community health centers, childcare providers, community colleges, and tribal organizations.

Funding

Programs are encouraged to seek in-kind donations and other funding sources, but no specific target is required. State funds are to be used as a last resort. The program year 2000-2001 state general- fund budget was $30,053,699.

Implementation Detail

Program Design

ECEAP is a "whole child," comprehensive, family-focused, pre-kindergarten program designed to help low-income children succeed in the educational system. The program also assists families in order to make them self-sufficient and able to help the children succeed. The program has four interactive components in each site:

• Education: ECEAP provides a learning environment for pre-kindergarten children that fosters intellectual, social, physical, and emotional growth. It identifies and intervenes early on to correct problems that might interfere with learning and that might prevent children from being successful when they enter the public school system. It is also designed to make the transition into kindergarten easier for children as well as to foster success throughout the children’s primary education. The program is offered for part of the year (for a minimum of 32 weeks out of the year) and is half-day. However, in many cases, it is integrated into childcare settings, so that the children are cared for during the whole day.

• Health and Nutrition: ECEAP provides health screenings for children within the first 90 days of program service. All medical, dental, mental, and nutritional needs are evaluated and brought up to date. If health problems are discovered, the family is referred to the appropriate community provider. ECEAP can also help identify community resources and provide services or funds if necessary. The ECEAP health staff assists with bringing immunizations up to date and with arranging for fluoride treatments in areas where the water is not fluoridated. All children also receive at least one complete meal each day in the classroom. The curriculum also includes nutritional education for children and parents in order to encourage healthy eating habits that last a lifetime.

• Parent Involvement: Parents are encouraged to participate in their children's learning by volunteering in the classroom as well as participating in decision-making through their local program's parent-run policy council. ECEAP also provides parent-skills training and support groups based on community need.

- 163 -

• Family Support: The program staff works with families to assess needs and help families locate and access community resources. In addition, the program provides opportunities for skill-development training in parenting, leadership, and self-sufficiency. Collaborative arrangements with various service providers and community organizations, along with in-kind donations from those groups, allow ECEAP staff to create a network of support for families.

Curriculum

The curriculum for the Washington State program is available from the Early Childhood Education and Assistance Program office (see Contact Information).

Staffing

The program is administered statewide and local programs operate through various organizations including school districts, local government agencies, nonprofit organizations, child care providers, and community colleges. Program staff, community leaders, and parents work together at each site to make sure that the programs offered are appropriate for the individual community.

Issues to Consider

This program received a "promising" rating based on the fact that there is only one study that has evaluated the program so far and the methodology used exhibits some weaknesses. The design was quasi-experimental, but the comparison group may have serious flaws. It is composed of children and families who chose to participate in the comparison group. In some cases, they had the option of participating in ECEAP; in other cases, they may not have had access to the program in their geographical area. Also, the treatment group had much higher rates of poverty at the start of the project than the comparison group. This may make the comparison families different as a group from the families who did participate. Furthermore, both the treatment and the comparison group exhibited relatively high attrition rates over the period of evaluation. In addition, while the results of the study seem promising in that they do show improvement for the children and families who participated in the program, there also seems to be similar improvement in many areas for the families and children who were in the comparison group.

Example Sites

More than 260 sites throughout the state of Washington (see www.ocd.wa.gov/info for a list of sites)

Contact Information

For Program Information: Lynne Shanafelt Managing Director Early Childhood Education and Assistance Program 906 Columbia Street, SW PO Box 48350 Olympia, WA 98504-8350 Phone: (360) 725-2830 Fax: 360-586-0489 E-mail: ECEAP_Admin@CTED.wa.gov For longitudinal study evaluation information: Tim Speth Research Associate Child and Family Program Northwest Regional Educational Laboratory 101 SW Main Street, Suite 500 Portland, OR 97204

- 164 -

http://www.ocd.wa.gov/info/csd/waeceap/Marketing/ECEAP_&_HS_Program_List.pdf

mailto:ECEAP_Admin@cted.wa.gov

Phone: (503) 275-9551 Fax: 503-275-9152 E-mail: SpethT@nwrel.org

Available Resources

Year 8 and Years 9 and 10 ECEAP Longitudinal Study Reports are available at www.ocd.wa.gov/infor. In addition, the ECEAP Web site at www.ocd.wa.gov/info/csd offers information on what the ECEAP is and how to effectively market a program, among other information. The curriculum is available by emailing ECEAP_Admin@cted.wa.gov.

Bibliography

An Investment in Children and Families: Year 8 Longitudinal Study Report Northwest Regional Educational Laboratory, Child and Family Program, December 1999. An Investment in Children and Families: Years 9 & 10 Longitudinal Study Report, Northwest Regional Educational Laboratory, Child and Family Program, December 2000.

Last Reviewed

March 2013

Early Head Start

Program Info

Outcome Areas Children Ready for School

Indicators Children ages 0 to 5 exhibiting age-appropriate mental and physical development

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Home Visiting Type of Service Family Support Health Care Services Instructional Support Parent Education Type of Outcome Addressed Behavior Problems Cognitive Development/School Performance Physical Health

Evidence Level Proven

- 165 -

mailto:SpethT@nwrel.org

http://www.ocd.wa.gov/info/csd/waeceap/Longitudinal_Studies/default_longitudinal_study.html

http://www.ocd.wa.gov/info/csd/waeceap/index.html

mailto:ECEAP_Admin@cted.wa.gov

Program Overview

Early Head Start (EHS) is a federally funded community-based program for low-income pregnant women and families with infants and toddlers up to age 3. Its mission is to promote healthy prenatal outcomes for pregnant women, enhance the development of children age 0-3, and support healthy family functioning. Since its inception in 1994, EHS has become a nationwide effort of 650 community- based programs serving 66,000 children in 2009. EHS programs utilize multiple strategies to provide a wide range of services to participants. Services include child development services delivered in home visits, child care, comprehensive health and mental health services, parenting education, nutrition education, health care and referrals, and family support. EHS offers children and families comprehensive child development services through one or more official program options: (1) center-based, (2) home-based, and (3) combination programs (in which families receive both home visits and center experiences). Children and families enrolled in center- based programs receive comprehensive child development services in a center-based setting, supplemented with home visits by the child's teacher and other EHS staff (a minimum of two home visits per year to each family). In home-based programs, children and their families are supported through weekly home visits and bimonthly group socialization experiences. EHS does not feature a single program model, but asks each grantee to select service delivery options that will best meet the needs of the families and communities it serves. The changing needs and circumstances of families often mean that one program option does not meet the developmental needs of a child over a three-year period. Accordingly, EHS programs often offer multiple program options so that children and families can receive different services as their circumstances change.

Program Participants

The EHS program targets primarily low-income pregnant women and families with children up to 3 years of age. To be eligible for the national research study, the primary family caregiver had to be pregnant or have a child younger than 12 months of age. About 25 percent of the families enrolled while the mother was pregnant. Family caregivers enrolled in EHS in the 17 sites from the national study were on average 23 years old, and 62 percent were first-time parents. The sample was diverse: One-third of families were African-American, one-fourth were Hispanic, and slightly more than one-third were white. Nearly half of the EHS primary caregivers did not have their high school diploma at the time of enrollment, and 45 percent were employed or in some type of school or training. Most families were receiving public assistance of some kind, with 77 percent covered by Medicaid, 88 percent receiving Women, Infants, and Children (WIC) benefits, almost half receiving food stamps, one-third receiving Aid to Families with Dependent Children (AFDC) or Temporary Assistance for Needy Families (TANF), and 7 percent receiving Supplementary Security Income (SSI) benefits.

Evaluation Methods

National Evaluation (Love et al., 2002; Vogel et al., 2011) The EHS national evaluation project was carried out at 17 sites that were purposively selected as being generally representative of all EHS programs. The 17 sites were distributed across the major regions of the country—six in the West, four in the Midwest, four in the Northeast or Mid-Atlantic, and three in the South. About half were in urban areas and half in rural areas, with home-based, center- based, and mixed-approach programs in each. The 17 research programs were not randomly selected. However, the research sites included a wide range of locations and program approaches, the families served by the research programs resembled the families served by other EHS programs, and the research sites differed with respect to their

- 166 -

experience serving infants and their years in operation. Thus, the results from the evaluation of these programs are likely to be applicable to other EHS programs. As soon as program officials from each site determined that applicant families met the EHS eligibility guidelines, families were randomly assigned to either the treatment or the control group. Control group families were able to receive other services in the community, as long as those services were not provided by EHS. Therefore, the comparisons of treatment and control group outcomes represent the effects of EHS services relative to the receipt of all other community services available to families in the absence of EHS. During the sample intake period (between July 1996 and September 1998), 3,001 families were randomly assigned to the treatment groups (1,513 families) and control groups (1,488 families). Most sites included samples of between 150 and 200 families, divided evenly between the two research groups. Researchers examined outcomes for the Early Head Start participants and their families at age 3, age 5, and age 8 (fifth grade). Key outcomes at age 3 included measures of both cognitive skills and social-emotional development. Cognitive measures included the Bayley Mental Development Index (MDI) standard score and the Peabody Picture Vocabulary Test (PPVT). Children who spoke Spanish in the home were assessed using a Spanish version of the PPVT. Social-emotional development was assessed by observations of children's behavior during semi-structured play, as assessed by trained observers of videotaped parent-child interactions. Key measures included engagement of parent (maintained interaction), negativity toward parent, and sustained attention to objects during parent-child play; engagement of parent, persistence, and frustration during a parent-child puzzle challenge task; emotional regulation as measured by the Bayley Behavior Rating Scale (BRS); orientation/engagement on the BRS (a measure of cooperation and positive affect); and aggressive behavior as measured by the Achenbach Child Behavior Checklist. Forty-nine outcomes were assessed at age 8, including social-emotional outcomes, child academic outcomes, parenting and the home environment, family well-being and mental health, and parent self- sufficiency. Additionally, multi-domain indices were constructed from the outcomes to measure cumulative risk and cumulative success. The age-8 follow-up included 1,632 respondents, or 54.4 percent of the baseline sample. Attrition was particularly high among the highest-risk respondents. However, study authors conducted sensitivity testing to determine the effect of attrition on impact estimates and found that findings were similar across different models. Utah Study Roggman et al. (2002) conducted a site-specific study in the Bear River, Utah, EHS site. Bear River EHS was a member of the national study, and families who applied for and who qualified for the Bear River EHS were randomly assigned to either EHS or a control group. The authors assessed whether the developmental path (change over time) for cognitive skills was different for children in EHS versus the control group. The sample included 201 mothers (103 in the EHS group and 98 in the comparison group) who were either pregnant at the time of application or had infants under ten months old. To meet program requirements, more than 90 percent of families were low-income, as defined by federal poverty guidelines, and most families (97 percent) received some sort of public assistance, such as Medicaid, food stamps, or WIC benefits. Most children in the sample were Caucasian (82 percent). Cognitive skills were assessed using the Bayley Scales of Infant Development at 14, 24, and 36 months, and data were collected as part of the national study. Results from a prekindergarten follow-up study are expected to be published shortly. PPN will update this EHS program summary when the new results are available.

- 167 -

Key Evaluation Findings

National Study (Love et al., 2002) Age-3 findings Results indicated that EHS enhanced children's cognitive and language development at age 3. Specifically,

• EHS children earned significantly higher standard scores on the MDI than did control children (a mean score of 91.4 compared with a mean score of 89.9).

• EHS children scored significantly higher on the PPVT than did control children (a mean score of 83.3 versus a mean score of 81.1).

• Significantly fewer EHS children than control children scored below 85 on the PPVT (27.3 percent versus 32.0 percent), a score representing substantially lower language ability than the national average.

• No statistically significant differences were found between the treatment and control groups for the percentage of children scoring below 85 on the MDI (a score lower than the national average).

• No significant impacts were found for the Spanish version of the PPVT.

• Results also indicated significant effects from EHS on children's social-emotional behavior. Compared with control group children, EHS children were significantly

• more engaging of their parents during semi-structured play

• more attentive to objects during semi-structured play

• less negative toward their parents during parent-child semi-structured play

• less aggressive toward their parents, as measured by the Child Behavior Checklist.

EHS programs did not have a significant impact on a child's engagement of his or her parent (i.e., maintaining interaction with the parent), persistence, or engagement during a parent-child puzzle- challenge task. In addition, no significant effects were found for a child's emotional regulation or orientation/engagement, as indicated by ratings by trained observers on the Bayley Behavior Rating Scale. Fifth-grade findings Across the 49 individual outcomes measured for the full sample, only two were found to show improvement for EHS participants, compared with controls. EHS children had significant improvement relative to the control group on

• social-emotional success, as measured by an index that includes the absence of the following risk factors (none of which reached individual significance):

o externalizing behaviors

o internalizing behaviors

o attention problems

o peer bullying

o delinquency

• parent reports of children being anxious or depressed.

- 168 -

Results of subgroup analyses are reported in the Issues to Consider section of this summary. Utah Study Roggman et al. (2002) assessed whether cognitive skills differed among EHS and comparison group children. Results showed that changes in cognitive skills over time were different for those in EHS than for those in the comparison group. Across measurements at 14, 24, and 36 months, EHS participants maintained stable test scores that did not change significantly with age. Conversely, the comparison group children exhibited statistically significant decreases in their standardized cognitive skill scores between the initial and final measurement points.

Probable Implementers

Early childhood programs

Funding

EHS is a federal initiative funded by the Head Start Bureau, the Administration on Children, Youth and Families, and the American Indian Programs Branch. In fiscal year 2009, the total federal EHS budget was $709 million. Note that EHS sites must also provide another source of funding to cover additional indirect costs.

Implementation Detail

Program Design

The framework of EHS includes four required program components:

1. Child Development: EHS programs must support the physical, social, emotional, cognitive, and language development of each child. The services that programs must provide directly or through a referral include early-education services in a range of developmentally appropriate settings, home visits, parent education and parent-child activities, comprehensive health and mental health services, and high-quality child care services.

2. Family Development: Programs must attempt to empower families by developing goals for parents and their children. Staff and parents must develop individualized family development plans that focus on the child's developmental needs and the family's social and economic needs. The services that programs must provide directly or through a referral include child development information; comprehensive health and mental health services (including smoking cessation and substance abuse treatment); adult education, literacy, and job skills training; assistance in obtaining income support, safe housing, or emergency cash; and transportation to program services.

3. Community Building: Programs are required to conduct an assessment of community resources so as to enable them to build a comprehensive network of services and supports for pregnant women and families with young children. The goal of this network is to increase family access to community support, make the most efficient use of limited resources, and improve the service delivery system for all families in the community.

4. Staff Development: EHS staff members must have the capacity to develop caring, supportive relationships with both children and families. Ongoing training, supervision, and mentoring encompass an interdisciplinary approach and emphasize relationship building. Staff development is grounded in established best practices in the areas of child development, family development, and community building. EHS programs must be committed to continuous improvement, through ongoing training and technical assistance provided by the EHS National Resource Center (EHS NRC).

- 169 -

Training and Technical Assistance

The EHS NRC provides training and technical assistance to EHS staff and teachers through leadership meetings, a comprehensive website, and national training events. In partnership with the infant/toddler specialists at the 15 Head Start quality improvement centers across the country, the EHS NRC works to ensure that EHS programs have information and training on best practices in all areas of program services and management. The EHS NRC provides training and technical assistance to EHS staff and teachers in a variety of ways. Specific activities include:

• convening biannual meetings of an expert technical work group

• developing a compendium of quality training resources for consumers

• conducting presentations on best practices at national and regional conferences

• creating technical assistance papers on select topics

• providing orientation training for newly funded EHS programs

• planning Annual Institutes for Head Start birth-to-three programs

• providing Program for Infant Toddler Caregivers (PITC) intensive training

• conducting three national audio-conferences per year featuring expert faculty and EHS program representatives

• developing satellite television training events that are broadcast on the satellite network of the National Head Start Association

• creating partnership and links among EHS programs through the hosting of list serves (web- based discussion groups) for the EHS community.

Issues to Consider

This program received a "proven" rating. The national evaluation of EHS used rigorous standards, including a randomized experimental design with a two-year follow-up. Participants experienced significant and sizable gains across most of the cognitive and social-emotional outcomes. A number of additional evaluations have been conducted for this program, but we included only those studies with methodologies meeting the Promising Practices Network evidence criteria. The significant impacts found for Early Head Start at age 3, however, did not persist into fifth grade, save for a composite measure of social-emotional success, which showed favorable impacts at both age 3 and in fifth grade. Racial/ethnic subgroup analyses were conducted, and authors found that, at both age 3 and in fifth grade, a greater impact was realized among the African-American participants, relative to controls, than children from other racial/ethnic groups in the study. Hispanic children also fared moderately better; among white children, very few of the outcomes were significant at age 3 and none were significant at age 8. Subgroup analyses were also conducted to examine program approaches (center-based, home-based, or mixed-approach). In the age-3 analyses, across all three of the program approaches, EHS had a favorable impact on children's cognitive and language development, on children's levels of aggression, and on children's behavior in relation to their parents during semi-structured play. Nevertheless, the pattern of impacts on children and parents varied somewhat across program approaches. For most child development outcomes, the impact of the program did not differ significantly by program approach. Mixed-approach programs had a somewhat stronger pattern of favorable impacts on children; they demonstrated greater positive impact on children's language development and various aspects of social-emotional development than did the center-based or home-based approach programs. Mixed-approach programs also reduced the proportion of children with receptive vocabulary

- 170 -

scores (a measure of listening comprehension of spoken words) below 85 to a substantially greater extent than did other programs. In the fifth-grade follow-up, home-based programs had generally more favorable impacts than center-based or mixed-approach programs, which showed weak and inconsistent impacts relative to controls. Finally, authors analyzed outcomes for children based on risk profiles. In both the age-3 and fifth- grade studies, the highest-risk group suffered unfavorable outcomes, compared with controls. Authors suggest that this might be due to the fact that high-risk children were more concentrated in programs that were not fully implemented.

Example Sites

The EHS program has been implemented in 650 sites nationwide (as of fiscal year 2009).

Contact Information

Early Head Start National Resource Center @ ZERO TO THREE 2000 M. Street, NW, Suite 200 Washington, DC 20036 Tel (202) 638-1144 Fax (202) 638-0851 ehsnrcinfo@zerotothree.org

Available Resources

The Early Head Start National Resource Center (http://www.ehsnrc.org/), maintained by the Administration for Children and Families, U.S. Department of Health and Human Services, contains many references and resources for learning more about EHS standards, services, and program implementation.

Bibliography

Love, John M., Ellen E. Kisker, Christine M. Ross, Peter Z. Schochet, Jeanne Brooks-Gunn, Diane Paulsell, Kimberly Boller, Jill Constantine, Cheri Vogel, Allison Sidle Fuligni, and Christy Brady-Smith, Making a Difference in the Lives of Infants and Toddlers and Their Families: The Impacts of Early Head Start, Vol. I, Final Technical Report, Princeton, N.J.: Mathematica Policy Research, Inc., and New York, N.Y.: Columbia University's Center for Children and Families at Teachers College, 2002. Roggman, Lori A., Lisa K. Boyce, Gina A. Cook, and Andrea D. Hart, "How Much Better Than Expected? Improving Cognitive Outcomes in Utah's Bear River Early Head Start," in Making A Difference in the Lives of Infants and Toddlers and Their Families: The Impacts of Early Head Start, Volume III: Local Contributions to Understanding the Programs and Their Impacts, Vogel, Cheri A., Yange Xue, Emily M. Moiduddin, Ellen Eliason Kisker, and Barbara Lepidus Carlson, Early Head Start Children in Grade 5: Long-Term Follow-Up of the Early Head Start Research and Evaluation Study Sample, Washington, D.C.: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, OPRE Report #2011-8, 2010.

Last Reviewed

April 2011

- 171 -

mailto:ehsnrcinfo@zerotothree.org

http://www.ehsnrc.org/

Early Intervention in Reading

Program Info

Outcome Areas Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards

Evidence Level Promising

Program Overview

The Early Intervention in Reading Program (EIR) is a small-group intervention for struggling readers that is used within the regular classroom. Regular classroom teachers carry out the program, usually with the help of instructional aides or older students. EIR is designed to help kindergarten and first- grade students succeed in school and to help children continue to make good progress in reading in grades 2 through 4. Kindergarten is a whole class literature-based emergent literacy/oral language program with small group follow-up for children who need extra help. Children in first and second grade receive 20 to 30 minutes a day of instruction as a portion of the school’s regular reading program, with a focus on word recognition, fluency (reading with speed, accuracy and proper expression), and comprehension instruction. In the first-grade program, emphasis is placed on developing students’ phonemic awareness (i.e., their awareness of how to properly sound out words and letters) and their understanding of the alphabet, and on helping students to apply phonics while reading connected text (i.e., grouped words). In grades 3 and 4, students continue to apply their phonics knowledge and word recognition strategies to the reading of connected text and work on fluency, vocabulary, and comprehension.

Program Participants

Students in grades kindergarten through grade 4.

Evaluation Methods

Taylor et al. (1997) evaluated EIR in a sample of second-grade students from a school in St. Paul, Minnesota. Approximately 56 percent of the school’s students were ethnic or racial minorities, and 49 percent received subsidized lunches. Based on scores on the Metropolitan Achievement Test, teachers identified 31 students who they thought would benefit from the reading intervention program. Twelve students who did not have conflicts with other classes participated in a group that received EIR plus tutoring by fourth-grade students, and seven students participated in an EIR-only group. The remaining 12 students made up the control group (no tutoring). Third-grade outcomes studied by Taylor et al. included scores on the reading subtest of the Metropolitan Achievement Test (with

- 172 -

second-grade scores used as a covariate) and reading accuracy (measured by whether or not students were able to read stories with at least 90 percent accuracy). Taylor et al. (2002) studied EIR among first-grade students in two intermediate school districts located in the north-central United States. An average of 43 percent of the students at the schools received subsidized lunches. Teachers identified up to seven students per class who they thought would benefit from the EIR instruction, resulting in a treatment group of 51 children from 11 classrooms in eight schools. A total of 35 students from seven schools participated as control subjects, with teachers identifying up to seven students per classroom who they felt would benefit from an early reading intervention program. Of the seven control schools, two were from the same two districts as the treatment schools, and five were from neighboring districts. Control students received the school’s regular reading program, which included 90 to 120 minutes a day of literacy instruction. Treatment- group students also received 90 to 120 minutes of daily reading instruction, with 20 minutes of EIR included during this block of time. Outcomes that were assessed included reading speed and number of words read correctly in a story passage, accuracy in the retelling of a story passage, the percentage of correct answers to reading comprehension questions related to the passage, and the percentage of children who could read at a primer level or higher with at least 93 percent accuracy. Student’s scores on a phonemic-awareness test given at the beginning of the school year were used as covariates in the analyses.

Key Evaluation Findings

Taylor et al. (1997) reported:

• Significantly more children in the intervention-plus-tutoring group than in the control group were reading at grade level by the end of second grade. By the end of the school year (in May), 75 percent of the EIR-plus-tutoring group, less than 30 percent of the EIR-only group, and none of the control group students could read a selected passage from the second-grade basal reader with at least 90 percent word-recognition accuracy.

• The EIR-plus-tutoring group scored significantly higher than the control group on the Metropolitan Achievement Test, with percentile rankings of 19 and 8, respectively. The EIR- plus-tutoring group scored marginally higher than the EIR-only group, which had a percentile ranking of 11. There were no significant differences between the EIR-only and the control group.

The study of first-graders by Taylor et al. (2002) found the following at the end of first grade:

• EIR and EIR-plus-tutoring students scored significantly higher than control students on the number of words read correctly per minute (effect size 0.34) and on accurate story retelling (effect size 0.69).

• There were no significant differences between EIR and EIR-plus-tutoring students and control students on the percentage of correct answers to reading-comprehension questions.

• When the sample was limited to the most at-risk students at the beginning of the school year (defined by scores on a phonemic awareness test), significantly more EIR and EIR-plus- tutoring students (81 percent) than control students (50 percent) could read at a primer level or higher at the end of the school year.

Probable Implementers

Public and private elementary schools

Funding

The overall program costs of EIR in 2004-2005 are $500 per teacher for training and from $500 to $1000 for books, depending on the grade level.

- 173 -

Implementation Detail

Program Design

The EIR program provides relatively short classroom lessons. Children become familiar with the schedule of lessons on each particular day of the program, and this knowledge may help them to become more efficient in the learning of reading skills.

Curriculum

The EIR kindergarten program uses whole-class instruction with small-group follow-up for the children with the most limited oral language and emergent literacy abilities. Activities include listening to stories for enjoyment, discussion of how the stories relate to students’ lives, creative dramatics, and exposure to letter and sound recognition, phonemic segmentation and blending, and concepts of print and rhyme. EIR for grades 1 and 2 is a classroom intervention program in which the teacher works with a group of five to seven of the lowest-achieving readers for 20 to 30 minutes a day, five days a week. For three days a week, the group engages in repeated reading of and guided writing about a short illustrated book. Students are trained in phonemic awareness and word-recognition strategies to foster independent reading and to become better able to answer high-level reading-comprehension questions. The EIR programs for grades 3 and 4 employ a five-day-a-week routine and focus on deciphering multi-syllabic words, enhancing fluency, and improving reading comprehension. The third-grade program uses both narrative and informational books, while the fourth-grade program uses only informational materials that the children use to practice the reciprocal teaching model (reading to students in lower grade levels) as a reading reading-development technique.

Staffing

The EIR professional development component includes a half-day initial session and nine two-hour monthly meetings of the classroom teachers. In addition to learning new strategies as students make progress in developing their reading abilities, teachers share their successes and concerns with each other and watch videos of themselves teaching the EIR program. Professional development is also provided through an internet-based instructional delivery system, combined with support from an EIR trainer.

Issues to Consider

This program received a "promising" rating. The evaluations of the EIR program indicate that treatment-group students have experienced significant improvements in their reading skills as compared with no-treatment control group students who have not experienced similar improvements. The program's "promising" rating is due to the small samples sizes in the two evaluation studies and the non-equivalent control groups used in the analyses. These factors limit our ability to know for certain that the program alone caused the observed positive outcomes. Results from the Taylor et al. (1997) study suggest that the EIR program may be more effective when used in conjunction with older peer tutors, while an alternative interpretation of the study's results is that EIR may be ineffective without the tutoring component. Furthermore, results from Taylor, et al. (2002) suggest that the EIR program may be most effective for students at a higher risk of failing to acquire reading skills than for lower-risk students. The EIR program is a relatively inexpensive intervention that utilizes small-group instruction instead of more costly one-on-one instruction. It should be noted that the program developer participated in all three program evaluations cited in this program description.

- 174 -

Example Sites

Escanaba Michigan St. Paul Minnesota Staples, Minnesota

Contact Information

Ceil Critchley Early Intervention in Reading Professional Development Program 11293 Hastings Street NE Blaine, MN 55449 Tel (763) 785-0710 Fax (763) 785-0702 ccritchley@comcast.net www.earlyinterventioninreading.com

Available Resources

The Early Intervention in Reading Program website (www.earlyinterventioninreading.com) offers information on professional development and resources for schools and teachers interested in implementing the program.

Bibliography

Taylor, Barbara M., Barbara E. Hanson, Karen Justice-Swanson, and Susan M. Watts, "Helping Struggling Readers: Linking Small-Group Intervention with Cross-Age Tutoring," The Reading Teacher, Vol. 51, No. 3, 1997, pp. 196-209. Taylor, Barbara M., Ceil Critchley, Kristine Paulsen, Kristen MacDonald, and Heidi Miron, Learning to Teach an Early Reading Intervention Program Through Internet-Supported Professional Development, Edina, Minn.: Web Education Company, 2002. Taylor, Barbara M., Ruth A. Short, Brenda A. Shearer, and Barbara J. Frye, "First Grade Teachers Provide Reading Intervention in the Classroom," in Richard L. Allington and Sean A. Walmsley, eds., No Quick Fix: Rethinking Literacy Programs in America’s Elementary Schools, New York, N.Y.: Teachers College Press,1995, pp. 159-176.

Last Reviewed

November 2004

The Effective Learning Program

Program Info

Outcome Areas Children Succeeding in School

Indicators Students graduating from high school

- 175 -

mailto:ccritchley@comcast.net

http://www.earlyinterventioninreading.com/

Topic Areas Age of Child Adolescence (13-18) Type of Setting High School Type of Service Instructional Support Youth Development Type of Outcome Addressed Cognitive Development/School Performance

Evidence Level Proven

Program Overview

The Effective Learning Program (ELP) was a two-year program established at Ballard High School in Louisville, Kentucky. ELP sought to raise students' internal locus of control so that the students were more likely to believe that they can control events that affect them. The program also sought to improve students' skills in building relationships with peers and adults, and to increase graduation rates. ELP students participated in a three-hour block of English, math, and humanities instruction that were taught by trained ELP teachers in small "family" or "team" atmosphere classes, promoted through activities such as recognition of each student's birthday, taking field trips together, and celebrating high attendance. The student-teacher ratio was controlled at 15:1. A crucial component of ELP was regular teacher-led discussions with students about how their interpersonal styles positively or negatively contribute to interactions with others. ELP teachers also maintained close contact with parents, administrators and non-ELP teachers to monitor each student's progress.

Program Participants

ELP participants included junior and senior students at Ballard High School who were identified by teachers, counselors and parents as being at high risk of dropping out based on a variety of student problems (having low grades, poor attendance, learning disabilities, etc.). In general, ELP-eligible students had lower than a 2.0 grade point average (GPA) and had missed more than 15 days of school by the end of their sophomore year.

Evaluation Methods

Among 80 ELP-eligible students, 38 were randomly assigned to the intervention group. Thirty-six remaining ELP-eligible students participated as the first control group. Fifty students not at risk of dropping out, and who had similar demographic characteristics to those of ELP-eligible students, were selected as the second control group. The evaluation design included both pre and post measurements. All participants completed three standardized tests at the beginning and end of the two-year intervention: (a) the Children's Nowicki-Strickland Internal-External Locus of Control Inventory (Nowicki and Strickland, 1973), for which a higher score indicated a greater level of external locus of control; (b) the Interpersonal Adjective Scale (Wiggins, 1996), for which two dimensional scores about interpersonal communication style were calculated: a higher "status" score reflected greater dominance or lower submissiveness and a higher "affiliation" score suggested greater friendliness or less hostility; and (c) the Diagnostic Analysis of Nonverbal Accuracy (Nowicki and Duke, 1996), for which a higher score indicated a better ability to identify emotions in nonverbal communications with others.

- 176 -

In addition, the research team gathered indicators of students' school achievement and engagement, including high school graduation status, GPA, state standard test scores (Kentucky Instructional Results Information System, or KIRIS), other test scores, and daily attendance records.

Key Evaluation Findings

The study by Nowicki et al. (2004) reported the following findings:

• At the baseline of comparison, students who were eligible for ELP (including the ELP intervention group and the first control group) were more externally controlled, less capable of reading nonverbal cues, and characterized by interpersonal styles that were more hostile and more dominant than the second control group of non-ELP-eligible students.

• After the intervention, students in the ELP group had significantly improved their internal locus of control, exhibited reduced levels of hostility, and improved their ability to pick up on nonverbal cues during interactions with others. There were no positive changes among students who were eligible for ELP but did not participate in the program (the first control group).

• At the end of the two-year intervention, students in the ELP intervention group had significantly fewer days of school absence than students who were ELP-eligible but did not participate in the program (the first control group).

• Students in the ELP group improved their GPA and state standardized test scores while both indicators declined among their ELP-eligible comparisons (first control group) who did not participate in the program.

• Ninety-eight percent of students in the ELP group graduated from high school, significantly higher than that of ELP-eligible but nonparticipating students in the first control group (38 percent) and even higher than that of non-ELP-eligible students in the second control group (74 percent).

Probable Implementers

This program could be implemented in private and public middle and high schools.

Funding

The evaluation report did not disclose sources of funding for this program. Nor did it report any information regarding the costs of implementing it.

Implementation Detail

Program Design

• The program was designed as a "school within a school," in which ELP students participated in a three-hour afternoon block of English, mathematics, social studies, and humanities instruction taught by trained regular teachers from the same school.

• The student teacher-ratio in ELP was about 15:1, compared with 31:1 elsewhere at the same school.

• ELP sought to promote a "team" atmosphere by recognizing birthdays, taking field trips, awarding special prizes for high attendance, and celebrating festivals during which each humanities class put on a performance of some type.

• ELP teachers took every opportunity inside and outside classrooms to give students feedback about their interpersonal styles and discuss their consequences.

- 177 -

Staffing

The ELP teaching team included six regular teachers from Ballard High School (two math teachers, two English teachers, and two social studies teachers). One social studies teacher was part of the research team, and the other social studies teacher also served as a fundraiser to finance the lower student- teacher ratio in ELP. In addition, the program was usually aided by one or two student teachers during the spring semester. ELP teachers received professional development on how to effect desired changes in students' behaviors. The professional development included printed materials, lectures, workshops, and examples from outside experts once every three months.

Curriculum

ELP included a noncredit humanities curriculum based on the Waldorf philosophy (Almon, 1992). This education approach emphasizes the role of imagination and seeks to develop children's emotional life, artistic expression, and social responsibilities through collaborative learning.

Issues to Consider

This program received a "proven" rating. The research, which was conducted over two years, was implemented according to rigorous standards and included an intervention group of 38 students, a control group of 36 students, and a comparison group of 50 students. ELP was designed based on existing empirical evidence that locus of control and interpersonal skills are associated with school engagement and that greater school engagement reduces dropout rates. Core features of ELP—including the teaching of a "relationship" language, continuous feedback on students' interpersonal styles, and a "family-style" learning environment—all aim to improve students' locus of control and relationships with others. In the current study, a higher graduation rate among ELP participants was accompanied by positive changes in locus of control, nonverbal processing skill, and affective style, suggesting a potential causal relationship between the two types of changes. However, a reduced student-teacher ratio alone might have also been a significant independent contributing factor to the success of ELP. There is a large body of empirical literature that has established a positive relationship between small class size and better student outcomes (Finn, 1990 and 1999; Mosteller, 1995). Further studies are needed to distinguish which aspects of ELP may be more responsible for the higher graduation rate among at-risk students. The current study included a relatively small sample size of approximately 100 students attending one school in one location. Therefore any generalization from this single case and unique set of circumstances should be made cautiously. It should also be noted that the program evaluation was conducted by the program developers rather than an independent evaluator.

Example Sites

Ballard High School, Louisville, Kentucky

Contact Information

Dr. Stephen Nowicki, Jr. Department of Psychology Emory University 532 N. Kilgo Circle Atlanta, Ga. 30322 404.727.7454 (phone) 404-727-0372 (fax) snowick@emory.edu (email)

- 178 -

mailto:snowick@emory.edu

Available Resources

No information was found regarding additional resources for ELP.

Bibliography

Almon, Joan, "Educating for Creative Thinking: The Waldorf Approach," Re-Vision, Vol. 15, 1992, pp. 3-11. Finn, Jeremy D., "Answers and Questions About Class Size: A Statewide Experiment," American Educational Research Journal, Vol. 27, No. 3, 1990, pp. 557-577. Finn, Jeremy D., "Tennessee's Class Size Study: Findings, Implications, Misconceptions," Education Evaluation and Policy Analysis, Vol. 21, No. 2, 1999, pp. 97-109. Mosteller, Fredrick, "The Tennessee Study of Class Size in the Early School Grades," The Future of Children, Vol. 5, No. 2, Summer/Fall 1995, pp. 113-127. Nowicki, Stephen, Jr., and Bonnie R. Strickland, "A Locus of Control Scale for Children," Journal of Consulting and Clinical Psychology, Vol. 40, No.1, 1973, pp. 148-154. Nowicki, Stephen, Jr., and Marshall P. Duke, "Individual Differences in the Nonverbal Communication for Affect: The Diagnostic Analysis of Nonverbal Accuracy Scale," Journal of Nonverbal Behavior, Vol. 8 No. 1, 1996, pp. 9-35. Nowicki, Stephen, Jr., Marshall P. Duke, Sherleen Sisney, Bridget Stricker, and Mary Ann Tyler, "Reducing the Drop-out Rates of At-Risk High School Students: The Effective Learning Program (ELP)," Genetic, Social, and General Psychology Monographs, Vol. 130, No. 3, 2004, pp. 225-239. Wiggins, Jerry S., "An Informal History of the Interpersonal Circumplex Tradition," Journal of Personality Assessment, Vol. 66, No. 2, 1996, pp. 217-233.

Last Reviewed

October 2009

Family Foundations

Program Info

Outcome Areas Healthy and Safe Children Children Ready for School Strong Families

Indicators Children not experiencing physical, psychological or emotional abuse Fathers maintaining regular involvement with their children Children ages 0 to 5 exhibiting age-appropriate mental and physical development Children and youth not engaging in violent behavior or displaying serious conduct problems

- 179 -

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Community-Based Service Provider Health Care Provider Type of Service Family Support Parent Education Type of Outcome Addressed Behavior Problems Mental Health

Evidence Level Proven/Promising

Program Overview

Family Foundations (FF) is composed of eight pre- and post-natal classes designed for expectant couples who are living together (cohabitating or married). FF classes are interactive and skills-based, focusing on enhancing the "coparenting" relationship. The coparenting relationship is defined as the ways parents organize their parenting, support or undermine each other, and manage conflict regarding parenting. Research shows that coparenting relationship quality has a strong influence on parenting and child outcomes for families regardless of marital status, residential status, and risk level.

Program Participants

Participants include expectant couples who are either cohabitating or married. The evaluation participants were heterosexual couples expecting their first child.

Evaluation Methods

The effectiveness of FF was examined in a trial in which 169 heterosexual adult couples expecting their first child were recruited from hospitals in two small cities. Couples were assigned to either the intervention (89) or comparison (80) groups. Randomization was successful in that both groups were equivalent on all measured variables, including age, income, education, marital status, weeks of gestation, mental health, and relationship quality. Intervention and comparison group couples were assessed before the intervention (pretest) at an average of 22 weeks gestation, and again after the intervention was complete (posttest) when their children were an average of 6.5 months old. Ninety percent of couples who completed the pretest also completed the posttest, and this was not significantly different across groups (Feinberg and Kan, 2008). Two additional follow-up studies were conducted: The 13-month follow-up was conducted when the children were an average of 13.7 months old, with 91 percent of couples completing the 13-month follow-up (Feinberg, Kan, and Goslin, 2009), and a final follow-up was conducted when the children were an average of 3 years old, with 81 percent of couples completing the 36-month follow-up (Feinberg et al., 2010). (Results from a further follow-up when children were an average of 6 years old are being prepared for publication.) At pretest and/or posttest, the following measures were assessed:

• Parental mental health of both mothers and fathers was assessed at pretest and posttest and measured by the following scales:

o Center for Epidemiologic Studies Depression Scale (CES-D)

o Taylor Manifest anxiety scale.

- 180 -

• The Dysfunctional Interaction subscale from the Parental Stress Index was administered to both parents at pretest and posttest.

• Coparenting, a 37-point scale of self-reported coparenting behaviors, was completed by both mothers and fathers at posttest. This scale was developed by the program developers and includes the following three subscales:

o coparental support

o parenting-based closeness

o coparental undermining.

• Infant regulation subscales of the Infant Behavior Questionnaire were administered to both fathers and mothers at posttest only:

o infant soothability

o infant orienting (baby's attention to a single object for a specific length of time).

At the 13-month follow-up, family interactions were videotaped and coded. Parents and the infant engaged in 12 minutes of joint free play on the floor. Parents were then asked to teach their child to accomplish a set of tasks designed to be at the limit of most infants' developmental capacity (e.g., rolling a ball back and forth with a parent, building a tower of blocks). This interaction lasted 6 minutes. Blind coders were trained to rate the videotapes of tasks according to a coding system of 5- to 7-point scales. This coding system was developed for this project by program developers. Measures assessed through videotaped interactions included the following:

• Couple relationship behaviors, including:

o warmth to partner (physical or verbal affection)

o negative communication (contempt, hostility, demandingness)

• Parenting behaviors, including:

o positivity (positive affect, support for exploration)

o negativity (irritability, hostility toward child)

• Coparenting measures, including:

o competition (competition for child attention, love)

o triangulation (use of child as pawn in partner conflict)

o warmth (caring, affection toward partner)

o inclusion (active inclusion of partner in play)

o cooperation (overt cooperation with partner in play)

• Child behaviors, including:

o self-soothing (self-directed comforting: stroking, sucking)

o sustained attention (sustained involvement with objects/people).

At the 36-month follow-up, the following outcomes were assessed by researchers during a home visit:

• The Parenting Sense of Competence scale was administered to both parents, asking parents about their confidence in their parental role.

- 181 -

• The Parenting Stress Index was administered to both parents, asking parents to respond to their agreement with certain questions, such as, "I feel trapped by my responsibilities as a parent."

• Parental depression was assessed for both parents with the Center for Epidemiological Studies Depression Scale (CES-D).

• A Coparenting Scale was administered to both mothers and fathers and assessed items such as coparental agreement, support, undermining, and exposure of the child to conflict.

• The Quality of Marriage Index asked parents to rate their agreement with certain statements about their relationship, such as, "We have a good relationship."

• The Parenting Scale was administered to both parents, assessing discipline practices in parents of children from 18-48 months. Three subscales were used:

o laxness

o over-reactivity

o physical punishment.

• Child outcomes were assessed using the Child Behavior Checklist (CBCL), which was reported by mothers only. Researchers assessed the following dimensions:

o total problems

o externalizing problems

o internalizing problems

o aggression

o attention/hyperactivity.

• Emotional competence was assessed using the Head Start Competence scale, with mothers reporting on the child's interactions.

Key Evaluation Findings

At posttest, study authors found the following:

• Coparenting outcomes:

o Both mother and father report of coparental support was significantly higher for the FF group than for the comparison group.

o Father report of parental-based closeness was significantly higher among fathers in the FF group than those in the comparison group, and no significant difference was found for mothers.

o No significant differences were found in coparental undermining.

• Parental mental health:

o Maternal depression exhibited greater reductions in the FF group than the comparison group, and no significant difference was found for fathers.

o Maternal anxiety exhibited greater reductions in the FF group than in the comparison group, and no significant difference was found for fathers.

• Dysfunctional interaction was significantly improved for both fathers and mothers in the FF group versus parents in the comparison group.

• Infant regulation:

- 182 -

o Combined mother/father report of infant orienting was significantly improved for FF parents versus comparison group parents.

o Parental report of infant soothability was not significantly different across groups.

At the 13-month follow-up, in which family interactions were videotaped, study authors found the following:

• Coparenting outcomes:

o FF mothers and fathers performed significantly better (i.e., lower scores) on parental competition compared with the comparison group.

o FF mothers and fathers performed significantly better (i.e., lower scores) on parental triangulation compared with the comparison group.

o FF fathers performed significantly better on parental warmth than fathers in the comparison group, but there was no significant difference in parental warmth for FF mothers versus comparison group mothers.

o FF mothers performed significantly better than those in the comparison group on parental inclusion, but this was not true for FF fathers.

o There were no significant differences between FF and the comparison group in parental cooperation.

• Parenting:

o Mothers and fathers were both significantly more positive in their parenting practices than those in the comparison group.

o Fathers were significantly less negative in their parenting practices than fathers in the comparison group, but the difference was not significant for mothers.

• Child behavior:

o Infants belonging to couples in the FF group were better at self-soothing than those in the control group.

o There were no differences across groups in sustained attention.

• Dyadic couple behaviors:

o Mothers in the FF group exhibited significantly less negative communication than mothers in the comparison group, but the difference was not significant for FF fathers versus fathers in the comparison group.

o Both mothers and fathers exhibited significantly more warmth to their partner than those in the comparison group.

At the 36-month follow-up, study authors found the following:

• Parenting stress and parenting efficacy were significantly improved for both mothers and fathers in the FF group versus the comparison group on average across all follow-up waves; however, results were not reported for the 36-month follow-up alone.

• Coparenting and Couple Relationship:

o FF parents scored significantly higher on the overall measure of coparenting than parents in the comparison group.

o FF parents were not significantly different than comparison parents on relationship quality; however, relationship quality among parents of boys was significantly improved in the FF versus the comparison group.

- 183 -

• Parenting Scale:

o Parents in the FF group exhibited significantly lower levels of over reactivity than parents in the comparison group.

o FF parents exhibited significantly lower levels of laxness than those in the comparison group.

o FF parents exhibited significantly lower levels of physical punishment than parents in the comparison group.

• Child outcomes Child Behavior Checklist (CBCL):

o There were significantly lower levels of behavior problems among children of couples in the FF group as measured by the Total Problems Scale. Analyses showed that this effect was driven by differences among boys, and, when analyzed separately, girls did not show significantly different levels of problem behaviors.

o Children of couples in the FF group showed significantly lower levels of externalizing behaviors and aggressive behaviors compared with the control group. Again, subgroup analyses revealed that these differences were driven by the boys in the FF group.

o There were no significant differences found for FF versus comparison group children on Internalizing Problems or the Attention/Hyperactivity scale. However, when examining the scores for boys alone, boys in the FF group were significantly improved on both of these measures compared with boys in the comparison group.

Probable Implementers

Many different organizations and individuals might implement Family Foundations, including health care organizations, social service agencies, childbirth educators, teen parenting programs, faith-based organizations, and employee assistance providers.

Funding

Grant writing support is available through the Family Foundations website: http://www.famfound.net/pages/for-professionals

Implementation Detail

Program Design

The goal of FF is to support parents as they adjust to the stress that new parenthood can put on the parental relationship through increased conflict, changes in the division of labor, and reduced couple companionship and sex. FF does this by enhancing positive support and coordination in the coparenting relationship. The program as evaluated was delivered by a trained facilitator over four prenatal and four post-natal in-person sessions, with accompanying worksheets and homework materials.

Staffing

A facilitator is trained in the approach, and the facilitator does not need to be from a particular field.

Curriculum

Curricular materials are available online at http://www.famfound.net/collections/educators

- 184 -

http://www.famfound.net/pages/for-professionals

http://www.famfound.net/collections/educators

Issues to Consider

This program is rated "proven" for the indicators Children ages 0 to 5 exhibiting age-appropriate mental and physical development and Children not experiencing physical, psychological, or emotional abuse. This program is rated "promising" for the indicators Fathers maintaining regular involvement with their children and Children and youth not engaging in violent behavior or displaying serious conduct problems. The program evaluation utilized a rigorous research design. However for the two "promising" indicators, the outcomes measured are considered "intermediate" because while there is existing evidence that they impact the PPN indicators, specific PPN indicators were not directly measured.

Example Sites

The study authors implemented the FF program in two small cities in the United States.

Contact Information

For general questions about the FF program, email Info@FamFound.net. For questions about training, consultation, or large-scale implementation of the classes, contact Jill Zeruth at Jill@FamFound.net, phone: 814-954-0262. For questions about the research and/or program development, contact program developer Mark Feinberg at Mark@FamFound.net.

Available Resources

Materials and information can be found at http://www.famfound.net/.

Bibliography

Feinberg, Mark E., and Marni L. Kan, "Establishing Family Foundations: Intervention Effects on Coparenting, Parent/Infant Well-Being, and Parent-Child Relations," Journal of Family Psychology, Vol. 22, No. 2, April 2008, pp. 253-263. Feinberg, Mark E., Damon E. Jones, Marni L. Kan and Megan C. Goslin, "Effects of Family Foundations on Parents and Children: 3.5 Years After Baseline," Journal of Family Psychology, Vol. 24, No. 5, 2010, pp. 532-542. Feinberg, Mark E., Marni L. Kan, and Megan C. Goslin, "Enhancing Coparenting, Parenting, and Child Self-Regulation: Effects of Family Foundations 1 Year After Birth," Prevention Science, Vol. 10, No. 3, September 2009, pp. 276-285.

Last Reviewed

June 2013

- 185 -

mailto:Info@FamFound.net

mailto:Jill@FamFound.net

mailto:Mark@FamFound.net

http://www.famfound.net/

Family Support and Parenting Education in the Home

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children not experiencing physical, psychological or emotional abuse Children experiencing good physical health

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Community-Based Service Provider Health Care Provider Home Visiting Type of Service Family Support Health Education Parent Education Type of Outcome Addressed Child Abuse and Neglect Physical Health

Evidence Level Proven/Promising

Program Overview

The Family Support and Parenting Education in the Home Program was developed in 1964 to serve poor children in the city of Baltimore. A woman from the participants' community served as a home visitor for new parents, with the goal of encouraging parental compliance with well-child visits, referring parents to support services when necessary, and discussing child development and parenting skills. The first home visit was made within seven to ten days of the child's birth, and nine subsequent visits were made before the child's second birthday. The program was an augmentation of the Children and Youth (C&Y) program, which operated health clinics for families with children ages 0-18 in inner city Baltimore.

Program Participants

Healthy black neonates weighing more than 2,000 grams and their mothers ages 18 or older were involved in the study.

Evaluation Methods

The Children and Youth (C&Y) health clinics provided well-child care for parents in inner-city Baltimore, and they served as the location of the family support and parenting education program study. Parents of healthy infants enrolled in C&Y clinics for health services were randomized to treatment (N=143) and control (N=147) groups. Treatment and control groups were comparable on measures of demographic characteristics, risk status as determined by C&Y staff members at intake, and Medicaid eligibility. The treatment group received 10 total home visiting services focused on parenting, child development, and encouraging timely well-child visits at the C&Y clinics. The control group had access to the same

- 186 -

well-child visits, but did not receive any support through home visitation. To encourage attendance, home visits were scheduled to occur immediately prior to scheduled well-child clinic visits. Six mothers refused participation in the study, and 27 of the 290 participants (12 treatment and 15 control subjects) were lost to follow-up (Hardy and Streett, 1989).

Key Evaluation Findings

Health care service utilization

• There was no significant difference between treatment and control groups in visits to the C&Y clinic for health care.

• Significantly more children in the treatment group had completed the appropriate amount of preventative care for their age (88 percent versus 69 percent).

• Fewer children in the treatment group had delayed preventive care, defined as immunizations given one or more months beyond the proper age (6 versus 14 percent).

Child illness, injury, abuse, and neglect

• There was no significant difference between treatment and control groups in the number of incidents of closed head trauma.

• In the treatment group, the incidence of chronic otitis media, an ear infection commonly associated with upper respiratory tract infections and associated with a lack of compliance with medications and/or follow-up appointments, was less than half that of the control group (21 versus 55 percent).

• Treatment group participants also had significantly lower rates of severe diaper rash than control group participants (21 versus 34 percent).

• Suspected abuse and neglect (as determined by C&Y staff) was significantly lower in the treatment group (1.5 percent in the treatment group versus 9.8 percent in the control group).

• Illness or accident requiring hospitalization was also significantly lower in the treatment group (6.1 versus 15.2 percent).

Probable Implementers

Health clinics or other child and family service organizations

Funding

No information available

Implementation Detail

Program Design

The family support and parenting education program was designed as a way to lend informal support on issues such as child development and parenting strategies, rather than to intervene in psychosocial issues per se. The home visitors linked mothers to additional support services when necessary. The home visits were conducted prior to scheduled well-child visits, with the goal of encouraging attendance.

- 187 -

Staffing

The program employed home visitors who resided in the target community. They worked in collaboration with a social worker and were provided brief trainings on the content of the home visits.

Curriculum

The home visitors used a parenting education curriculum designed to deliver age-appropriate information to parents on parenting, child safety, health care, and immunizations. Mothers were also given calendars with developmental milestones, dates and times of clinic visits, and important telephone numbers included. Pamphlets on child care were also distributed.

Issues to Consider

While the evaluation of this program used a randomized control design, the program improved an intermediate outcome that has an unknown relationship to the benchmark of interest. Specifically, this program received a "promising" rating for the indicator Children not experiencing physical, psychological, or emotional abuse. Suspected abuse was impacted by participation in the treatment group, but the criteria for suspected abuse were not described, nor were the number of cases of actual abuse. A notable aspect of this program is that it appears to have improved outcomes with a very small number of visits (10) and a relatively low-skill workforce, relative to many current models of home visiting. The results are at least suggestive that a low-cost, low-intensity home visiting program might have some impact on important indicators of early childhood well-being. There is limited information about program implementation and the types of training that were offered to the home visitors, and about the content of their home visits.

Example Sites

The program was implemented in Baltimore, Maryland, in the 1960s and is not currently operating.

Contact Information

No information available

Available Resources

There are no replication materials available for this program.

Bibliography

Hardy, J. B., and R. Streett (1989). "Family support and parenting education in the home: An effective extension of clinic-based preventive health care services for poor children." Journal of Pediatrics, 115(6): 927-931.

Last Reviewed

April 2010

- 188 -

Family Thriving Program

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children not experiencing physical, psychological or emotional abuse Children not experiencing anxiety or mood disorders, such as depression Children experiencing good physical health

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Home Visiting Type of Service Family Support Parent Education Type of Outcome Addressed Child Abuse and Neglect Mental Health

Evidence Level Proven

Program Overview

The Family Thriving Program (FTP) uses cognitive reframing as a method for correcting parents' biased understanding of the relationship between themselves and their children. It has been proposed that a skewed view of the parent-child relationship may contribute to child abuse and neglect. FTP is an enhancement to home visitation models that incorporates cognitive appraisal methods to assist parents in becoming "competent and independent problem solvers." To do this, parents receiving the enhancement are asked by home visitors to review recent parenting problems. Using a series of questions aimed at identifying the problem's cause, the home visitor arrives at a strategy for addressing the problems raised by the parent, and the home visitor follows up on the results of the strategy in subsequent home visits. FTP has been tested as an enhancement to the Healthy Start home visitation program.

Program Participants

Parents of infants receiving home visits

Evaluation Methods

A study of FTP among families with moderate risk for child maltreatment (Bugental, Ellerson, et al., 2002) examined the effect of the Healthy Start home visiting program with the FTP enhancement (HS+) when compared with unenhanced Healthy Start (HS only), and the study also compared with a control condition in which the families received no home visiting. The study was conducted among families expecting the birth of a child that were at moderate risk to become abusive. The study excluded families at high risk of child maltreatment, with the justification that a prevention program such as this one may not have the capacity to provide the appropriate services for very high-risk families. Risk was defined in terms of parent history and circumstances (e.g., unemployment, past history of own abuse, lack of support, unstable housing). Seventy-three parents at moderate risk to become abusive who were also eligible for the Healthy Start home visiting program were randomly

- 189 -

assigned either to the control group, the HS only group, or the HS+ group. The three groups were statistically equivalent on all demographic features. Seventy-six percent of families completed the study, and study attrition was statistically equivalent across the three groups. The study examined two types of measures: those intended to measure harsh parenting and those intended to measure the child's health. Harsh parenting was measured using items from the Conflict Tactics Scale (CTS). The CTS asks for frequency of occurrence of different ways of responding to family conflict, including physical abuse, such as shaking or hitting with a fist, and non-abusive use of force, including spanking and slapping. Parents were notified at intake that home visitors were mandated by law to report instances of child abuse. Child health measures included the following parent-reported statistics: (a) frequency of child injuries, (b) frequency of child illnesses, and (c) frequency of child feeding problems. The health measures were combined into a Child Health Scale score. In 2009, researchers conducted another study of FTP among families with infants born at medical risk who were recruited into Healthy Start programs (Bugental and Schwartz, 2009). One-hundred and forty-seven infants and their families were referred to Healthy Start by their physicians, social workers, or public health nurses based on the presence of a medical risk factor. Forty-eight of these infants were preterm, 56 had a medical problem, and 40 were referred for other reasons, such as cesarean delivery. Infants were randomly assigned to receive HS only or HS+. The HS only group was statistically similar to the HS+ group, except in the case of maternal education (which was lower in the HS group) and immigration status (there were more immigrant families in the HS only group compared with the HS+ group). The analysts controlled for these variables in the analysis. The 2009 study examined several measures of parent maltreatment and neglect, including the CTS discussed above; the Framingham Safety Survey, which is focused on household hazards; and the Child Injury Survey, which inquires about the frequency of falls, bruises, etc. Perceived power was also measured using participant (parent and child) drawings of their stature relative to the others in their family. Most recently, FTP was tested in a study conducted among families referred to Healthy Start from Newborn Intensive Care Units (NICUs) (Bugental, Schwartz, et al., 2010). Thirty-three percent were referred based on preterm birth, 28 percent due to birth complications, and 41 percent due to other medical risks. Families were randomly assigned to the Healthy Start plus FTP enhancement (HS+) group or the Healthy Start only group (HS only), and at baseline there were no significant difference across these two groups in gestational age, intake age, birth weight, maternal age, maternal education, child gender, father present in family, percent Latino, or percent twins. There was a statistically significant difference in responses to the Social Desirability Scale, with the HS only group scoring higher than the HS+ group, indicating that the HS only group was providing answers that were more aligned with social desirability, indicating that they might be less likely to report harsh parenting practices. The analysis of parenting practices as measured by the CTS controlled for this variable. In addition to the CTS, this study also measured the child's cortisol levels, which were measured 1, 2, and 3 years following the intervention. Cortisol is associated with the body's stress response system, and children exposed to maltreatment or stressors, such as maternal stress or maternal depression, show elevated cortisol levels. Elevated cortisol levels in early life are associated with reduced capacity for learning and memory later in life (Jameison and Dinan, 2001). The study also employed direct child assessments conducted at the 3-year follow up visit, using the McCarthy Scales of Children's Abilities (MSCA). The MSCA was administered in a lab in English or Spanish. Subscales of MSCA include measures related to visual and verbal short-term memory.

Key Evaluation Findings

In the study of families at risk for maltreatment, parents in the Healthy Start plus FTP (HS+) group participated in significantly fewer instances of harsh parenting, and their children experienced significantly improved health related to both of the other two groups, unenhanced Healthy Start (HS only) and the control group (Bugental, Ellerson, et al., 2002). Among the women participating in HS+, 4 percent reported physically abusing their child, compared with 23 percent in the HS only group and

- 190 -

26 percent in the control group. The HS only group was not statistically different from the control group on harsh parenting practices. The HS+ group yielded the highest score on the Child Health Scale, statistically significantly higher than either the home visitation alone or control conditions, with mean Child Health Scale scores of 0.25 for HS+, -0.05 for HS only, and -0.30 for the control group (Bugental, Ellerson, et al., 2002). In the study of medically at-risk children, no significant effects were found across HS only and HS+ for CTS measures of physical abuse. Significant improvements were found in HS+ compared with HS only in the area of corporal punishment: 21 percent of parents reported some corporal punishment in HS+, compared with 35 percent in the HS only group. Injury and home safety maintenance were significantly improved in the HS+ group relative to the HS only group. Additionally, in drawings of themselves and their children, mothers in the HS+ group depicted themselves as larger in size relative to their children than mothers in the HS group (average difference of 4 cm), indicating that their sense of self-efficacy in the parent-child relationship was elevated. In their study of infants referred from the NICU, Bugental, Allen, et al. (2010) found that the HS+ group's cortisol levels were significantly lower than those of the HS only group at the 1- and 3-year assessments. Cortisol levels were not significantly different in year 2. Child verbal short-term memory was significantly improved in the HS+ group relative to the HS only condition (0.48 in HS+ versus - 0.10 in HS only); however, child visual short term memory was not significantly different across HS+ and HS only groups. No significant effects were found for the subscale of the CTS concerning harsh parenting practices; however, avoidance tactics were significantly lower among HS+ parents (0.01 in HS+ versus 0.32 in HS only), and there was a significant effect of the intervention on the combined use of both harsh parenting and avoidance tactics, with HS+ parents scoring lower on the combination of these scales than the HS only group.

Probable Implementers

Organizations already implementing intensive home visitation programs

Funding

Services provided within the program were funded by the State of California.

Implementation Detail

Program Design

FTP enhancement incorporates all of the features of the home visitation program (in this study Healthy Start), adding an additional component incorporated at the start of each of the home visits. The program protocol states, "In implementing the Family Thriving Program, the basic method involved assistance to parents in the cognitive and motivational re-framing of commonly-occurring caregiving challenges. That is, parents were assisted in rethinking the causes of caregiving challenges, and in becoming their own information seekers and problem solvers... In the [enhanced] condition, better ways of interpreting caregiving problems were suggested to parents, along with the direct provision of information regarding community resources and developmental issues, along with recommendations regarding potential ways to solve existing problems" (Bugental, 2010).

Staffing

FTP enhancement does not require additional staffing beyond the home visitation staff. Additional training is needed, and training materials are provided on the program developer's website, http://www.psych.ucsb.edu/~bugental/.

- 191 -

http://www.psych.ucsb.edu/%7Ebugental/

Issues to Consider

The Family Thriving Program received a "proven" rating. The initial program evaluation and the two additional replications were randomized controlled trials, and the participants in these trials experienced significant improvement in a range of outcomes in the areas of physical and emotional health. FTP is a home visitation program enhancement and has not been evaluated as, nor is it intended to be, a stand-alone program. Only the initial study of FTP compared the FTP enhancement with Healthy Start alone and with no home visitation services. All subsequent studies evaluated the FTP enhancement to Healthy Start in comparison with Healthy Start alone. FTP has not been evaluated as an enhancement to any other home visitation programs beyond Healthy Start, and the results may differ when the enhancement is applied to other programs.

Example Sites

The program has been implemented as an enhancement to established Healthy Start program sites. FTP was first evaluated among an immigrant population at risk for child maltreatment in Santa Barbara County, California. It has since been tested among populations of medically at-risk infants and their parents.

Contact Information

Daphne B. Bugental bugental@psych.ucsb.edu

Available Resources

Resources can be found on the program developer's website: http://www.psych.ucsb.edu/~bugental/

Bibliography

Bugental, D. "FTP Program Protocol," 2010. As of November 22, 2010: http://www.psych.ucsb.edu/~bugental/ Bugental, D. B., A. Schwartz, et al., "Effects of an Early Family Intervention on Children's Memory: The Mediating Effects of Cortisol Levels," Mind, Brain, and Education, Vol. 4, No. 4, 2010, pp. 159- 170. Bugental, D. B., and A. Schwartz, "A Cognitive Approach to Child Mistreatment Prevention Among Medically At-Risk Infants," Developmental Psychology, Vol. 45, No. 1, 2009, pp. 284-288. Bugental, D. B., P. C. Ellerson, et al., "A Cognitive Approach to Child Abuse Prevention," Journal of Family Psychology, Vol. 16, No. 3, 2002, pp. 243-258. Jameison, K., and T. G. Dinan, "Glucocorticoids and Cognitive Function: From Physiology to Pathophysiology," Human Psychopharmacology, Vol. 14, No. 4, 2001, pp. 293-302.

Last Reviewed

December 2010

- 192 -

mailto:bugental@psych.ucsb.edu

http://www.psych.ucsb.edu/%7Ebugental/

Father/Male Involvement Preschool Teacher Education Program

Program Info

Outcome Areas Strong Families

Indicators Fathers maintaining regular involvement with their children

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Type of Service Parent Education

Evidence Level Promising

Program Overview

The Father/Male Involvement Preschool Teacher Education Program pilot was designed in 1995 to help teachers increase father/male involvement in state-funded preschool programs for at-risk students. The pilot ran for two full academic years prior to the evaluation. The focus of the Teacher Education Program was to help teachers develop the knowledge and skills needed to successfully plan, implement, and evaluate specific activities that encourage program involvement by fathers and other males who serve as father figures for school children. Specialists in child development provided teacher training on topics such as staff development; planning and implementing events such as father/child picnics, gym nights, and classroom nights; and developing other outreach initiatives to encourage father/male involvement. The teachers also had the opportunity to participate in individual consultations and group discussion sessions to discuss questions they may have had on a variety of issues, including the appropriateness of involving fathers/males in preschool programs. Currently, the program's developers are working with the Illinois State Board of Education to implement the program statewide.

Program Participants

Preschool teachers in a state-funded preschool for 3- to 4-year-olds from economically disadvantaged homes participated in the program and study. These students were considered at risk for later school failure based on family income and on other risk indicators such as the students having teenage parents, limited education of their parents, or their living in single-parent households. The target pre- kindergarten program enrolled approximately 300 children. Approximately 60 percent of the participants were African-American, 35 percent were Caucasian, and 5 were percent were from other ethnic groups. The average age of the teachers was 33. The teachers had been in the profession for an average of nine years and had an average of three years of higher education. There were 14 teachers at the treatment site.

- 193 -

Evaluation Methods

The purpose of the evaluation was to answer the question: What is the impact of an indirect intervention program revolving around issues of father/male involvement in early childhood programs designed to provide support services for staff members on the proportion of “parent involvement contacts” and activities that include fathers/men? (McBride, Rane, and Bae, 2000, p. 79). This program evaluation design included a comparison pre-kindergarten program with a post-test, surveys, and attitudinal instruments. The treatment site was self-selected as a result of an informal discussion between one of the teachers and a member of the program development team. Teams of teachers at the treatment site collaborated with the program team to develop the program. The program was piloted at the treatment site and was later implemented school-wide. The comparison site was selected because it was similar to the treatment site in all aspects (funding base, criteria for enrollment, services provided to enrolled children and their families, staff training and backgrounds, families being served, and other factors). The comparison site also was in proximity to the treatment site, which helped to ensure that populations with similar demographics were being served by both programs. At the beginning of the academic year, both the treatment and comparison teachers completed a packet of questionnaires, which included items on demographic backgrounds and two attitudinal measures on parent and father involvement in early-childhood settings. The student and teacher demographics were similar in the comparison and treatment programs. There were 7 teachers at the comparison and 14 teachers at the treatment site. The comparison pre-kindergarten program enrolled approximately 175 children age 3 and 4 and 300 children at the treatment site. The two attitudinal measures were the Attitudes Toward Father Involvement (ATFI) scale and an adapted version of the General Attitudes Toward Parent Involvement (GATPI) scale. Both scales were used to assess teachers’ attitudes toward father involvement in early childhood programs. During the academic year, detailed information tracking parent involvement activities and contacts with teachers was gathered from both sites. Types of information recorded for each contact included the method of contact, such as phone call, school visit, home visit, or written note; the nature/focus of the contact, such as developmental progress, behavior, or health issues; who initiated the contact— school or family/home; gender of person contacting the school when the contact is initiated from home; and gender of the person contacted at home by the school. Parent-involvement contacts ranged from one-on-one parent-teacher conferences to open house events. Data was collected for the evaluation at both sites in 13 consecutive two-week segments. Research assistants visited each teacher at the end of each two-week period to provide assistance. At the end of the 26-week period, each teacher received a $250 stipend. Information from the data-recording sheets for each of the 13 two-week periods was collapsed into a single record. Proportional scores were used because of the unequal number of teachers at the treatment and control sites.

Key Evaluation Findings

The study had the following findings in comparing the treatment site with the preschool that did not have the training program:

• Fathers/men participated in parent involvement activities at a significantly higher rate at the preschool with the training program.

• Prior to the start of the program, 5 percent of the total parent involvement at the preschool with the training program included fathers/males. By the program's third year, this participation rate was up to 23 percent at the treatment school and 12 percent at the comparison school was.

- 194 -

• Teachers participating in the training program reported significantly more parent involvement contacts with fathers/males than did teachers at comparison program schools, regardless of their level of commitment to involving fathers/males.

• Of the teachers who participated in the training program, those teachers who were highly committed to involving fathers/males reported significantly more contact with fathers/males than did teachers in the training program who were not highly committed to involving fathers.

• A significantly higher proportion of fathers/males initiated family-member contacts at the treatment site than at the control site.

Probable Implementers

Public, private, and religious preschool programs; elementary schools; and Head Start.

Funding

The U.S. Department of Agriculture funded the program evaluation. The local school system funded the program implementation. The Illinois State Board of Education currently is funding the statewide teacher training.

Implementation Detail

Program Design

The focus and intensity of program activities varied over the course of three years based on the identified needs and interests of individual teachers and school teams. Work with one teaching team, for example, focused on actively involving fathers/males in the classrooms. Another teaching team focused on discussions of why it is important to involve fathers in their children's preschool programs. The research and development group that assisted in the development of this program worked with the three teams of four classrooms each at the treatment site on a regular basis for all aspects of the program. The focus of the program was to allow enough flexibility to meet the teachers’ needs. In addition, the program sought to incorporate ways to deal with the resistance that some teachers have to involving fathers/males in school activities.

Curriculum

This program does not have a prescribed or set curriculum.

Staffing

The research and development group from the University of Illinois provided staff support. This group consisted of the lead program evaluator/article author and two graduate assistants. The group has expertise and experience with similar programs. During the three-year evaluation period, program staff held monthly training and technical assistance meetings with each preschool teaching team and bimonthly meetings for all teachers in the program. In addition, trainers provided individual assistance to teachers. The statewide training program for pre-kindergarten teachers in Illinois has made some adaptations to the original program and schedule.

Issues to Consider

This program received a “promising” rating. The evaluation design had several limitations, including a small sample size of just 21 teachers, a post-test-only design, and lack of a long-term follow-up. The disadvantage to the post-test-only design was that it was not entirely certain that differences between the groups resulted from the program. Absent a long-term follow-up, it is not possible to comment on the program’s sustainability over time. Although the results of the evaluation were encouraging, the limitations of the study design make it difficult to generalize the findings to other similar intervention

- 195 -

programs. It should also be noted that there were no outside evaluators of this program; the evaluation was conducted solely by the program designer.

Example Sites

Urbana, IL.

Contact Information

Dr. Brent McBride Childhood Development Laboratory University of Illinois at Urbana/Champaign 1105 W. Nevada Urbana, IL 61801 phone: (217) 333-0971 fax: (217) 333-0961 e-mail: brentmcb@uiuc.edu

Available Resources

Program developers currently are working on training and technical assistance materials for use by preschool teachers around the country.

Bibliography

McBride, B. A., T. R. Rane, Father/Male Involvement in Early Childhood Programs: Training Staff to Work with Men, in J. Fagan and A. J. Hawkins, eds., Clinical and Educational Interventions with Fathers, Binghampton, NY: The Haworth Clinical Practice Press, 2001. McBride, B. A., T. R. Rane, and J. Bae, Intervening with Teachers to Encourage Father/Male Involvement in Early Childhood Programs, Early Childhood Research Quarterly, Vol. 16, pp. 77-93, 2000.

Last Reviewed

April 2004

FluText

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children experiencing good physical health

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Health Care Provider

- 196 -

mailto:brentmcb@uiuc.edu

Type of Service Health Care Services Health Education Type of Outcome Addressed Physical Health

Evidence Level Proven

Program Overview

FluText is a text messaging intervention that sent automated vaccination reminders to parents of children and adolescents aged six months to 18 years. The goal was to encourage seasonal influenza vaccination by educating parents about influenza vaccine safety and the seriousness of influenza infection among children and adolescents, providing recommendations for influenza vaccine uptake among children and adolescents, and informing parents about upcoming influenza vaccine clinics at their child's health care facility (Stockwell et al., 2012).

Program Participants

FluText has been used among parents of children and adolescents aged six months to 18 years, though it has been found to be effective at increasing influenza vaccine uptake only among children aged six months to less than five years. (Stockwell et al., 2012).

Evaluation Methods

Stockwell et al. (2012) assessed the effect of FluText on receipt of seasonal influenza vaccine. Researchers utilized EzVac, an institutional immunization information system that automatically collects electronic vaccine administration on patients from the four study sites as well as from New York's Citywide Immunization Registry. The four study sites were four clinics affiliated with New York-Presbyterian Hospital and Columbia University Medical Center (Stockwell et al., 2012). The clinics serve a population that is predominantly Latino and eligible for free vaccines through the federal Vaccines for Children Program. Children and adolescents aged six months to 18 years as of September 28, 2010, who had attended one of the four clinics in the past 12 months and had a parental cell phone number recorded in the hospital registration system were eligible for the evaluation. All eligible children aged six months to less than five years were randomized to intervention or control groups. A random sample of eligible children aged five to 18 years was also selected for randomization to intervention or control groups. (Not all children age 5-18 years were included due to clinic capacity constraints.) Randomization was stratified by age group and clinic to ensure an equal distribution of children in the intervention and control groups by age and clinic. The baseline intervention consisted of a series of five weekly text messages sent to the cell phones of parents of children in the intervention group. The first three messages provided educational information on vaccine safety and the seriousness of influenza infection in children. The last two messages recommended that the target child receive the influenza vaccine and provided information about flu vaccine clinics held each Saturday from October through March at their health care clinic. If the target child was vaccinated before all five text messages were sent, the text messages were discontinued. Finally, if the target child remained unvaccinated by January 2011, two additional text messages were sent regarding the recommendation to receive the influenza vaccine and the remaining Saturday vaccine clinic dates. All text messages were sent in either English or Spanish based on the parent's preferred language recorded in the clinic electronic medical record. Both parents of children in the control group and the intervention group received standard care, which included an automated telephone message sent at the beginning of November 2010 providing information on the seriousness of influenza, vaccine safety, and information on the Saturday vaccine clinics. Flyers advertising the Saturday vaccine clinics were also posted at the four clinic locations.

- 197 -

Randomized children who had received the influenza vaccine before October 2010 (the initiation of the study) were excluded from the analyses. In total, 9,213 children and their parents were included in the evaluation. There were 4,607 participants randomized to the intervention group and 4,606 participants randomized to the control group. Of these, 817 intervention group participants and 822 control group participants received the influenza vaccine prior to the start of the intervention, leaving 3,790 participants in the intervention group and 3,784 participants in the control group. There were no significant differences between intervention and control group participants on demographics (age, sex, race, insurance status, primary language, or clinic site) or prior influenza vaccination.

Key Evaluation Findings

Stockwell et al. (2012) found the following:

• Among children aged 6-23 months, significantly more intervention children were vaccinated for influenza by the end of the influenza season (March 31, 2011) compared with children in the control group (58.5 versus 52.3 percent).

o In addition, among this age group, significantly more children in the intervention group were vaccinated for influenza by the fall target date established by the Centers for Disease Control and Prevention's (CDC's) recommendations (November 30, 2010; 38.6 versus 32.0 percent).

• Among children aged two years to less than five years, significantly more children in the intervention group were vaccinated for influenza by the end of the influenza season compared with children in the control group (46.0 versus 42.2 percent) and by the fall target date established by the CDC's recommendations (December 15, 2010; 28.5 versus 23.9 percent).

o Within this age group, significantly more children in the intervention group were vaccinated for influenza by the fall target date (December 15, 2010; 28.5 percent versus 23.9 percent).

• Among children aged five to 18 years, there was no statistically difference in influenza vaccine uptake between intervention and control group participants by the end of the influenza season (27.8 versus 25.7 percent).

Probable Implementers

Health care providers, local public health agencies

Funding

The evaluation conducted at New York-Presbyterian Hospital and Columbia University was funded by a research grant from the Health Resources and Services Administration's Maternal and Child Health Bureau. Other potential funding sources include the National Institutes of Health (NIH) and the CDC, as well as foundation sources.

Implementation Detail

Program Design

FluText is a text messaging intervention that involves sending a series of five weekly text messages to parents at the beginning of the influenza season. The text messages provide information regarding vaccine safety and the seriousness of influenza among children and adolescents, a recommendation for the target child to be vaccinated for influenza, and information on Saturday vaccine clinics at the child's health care provider.

- 198 -

Staffing

Staff may be required to develop the content of the reminders to ensure that the messages are appropriate to the target community. For study purposes, these were developed with community input through focus groups and in-depth interviews. However, once developed, FluText messages are automated and thus additional staff are not required to send them. The configuration of the text messaging system does require a programmer to set up the automated messages and to monitor the system.

Issues to Consider

The evaluation (Stockwell et al., 2012) was conducted among families attending clinics at New York- Presbyterian Hospital and was a randomized evaluation in which children and their parents were assigned to receive FluText text messages or standard care. This study included children aged six months to 18 years. The intervention was found to effective in increasing influenza vaccine uptake among children at greatest risk for poor outcomes following infection with seasonal influenza: children aged six months to less than five years. However, the intervention was effective among children aged 5-18 years. This study was conducted in an urban setting with a predominantly low-income population. Ninety-five percent of the population treated was eligible for free vaccines through the Vaccines for Children program. FluText has not been evaluated in other settings. It is important to note that this intervention was not accompanied by any apparent expansion of capacity for influenza vaccination at the four clinic sites. Therefore, to avoid overwhelming the clinic, not every family was made aware of every clinic date. This may have resulted in a lower observed effect of this intervention. Finally, parents were notified about special Saturday flu clinics. However, these special clinics were not necessarily held at children's regular clinic site. Improved alignment in this area may have an impact on the effect of this intervention. The evaluation of FluText described here was conducted by the same group of researchers who designed the Text4Health text messaging intervention.

Example Sites

New York-Presbyterian Hospital and Columbia University

Contact Information

Melissa Stockwell Columbia University mss2112@columbia.edu

Available Resources

http://www.columbia.edu/cu/text4health/flutext_b.html

Bibliography

Stockwell, Melissa S., Elyse Olshen Kharbanda, Raquel Andres Martinez, Celibell Y. Vargas, David K. Vawdrey, and Stewin Camargo, "Effect of a Text Messaging Intervention on Influenza Vaccination in an Urban Low-Income Pediatric and Adolescent Population," Journal of the American Medical Association, Vol. 307, No. 16, 2012, pp. 1702-1708.

- 199 -

mailto:mss2112@columbia.edu

http://www.columbia.edu/cu/text4health/flutext_b.html

Last Reviewed

March 2013

Gang Resistance Education and Training (G.R.E.A.T.)

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children and youth not engaging in violent behavior or displaying serious conduct problems

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Elementary School Middle School Out of School Time Type of Service Family Support Mentoring Parent Education Youth Development Type of Outcome Addressed Behavior Problems Juvenile Justice Physical Health Substance Use and Dependence Violent Behavior

Evidence Level Promising

Program Overview

The Gang Resistance Education and Training (G.R.E.A.T.) program is a school-based curriculum facilitated by law-enforcement officers. The program’s primary objective is to prevent delinquency, youth violence, and gang membership. The G.R.E.A.T. program was originally developed in 1991 by police officers from the Phoenix metropolitan area and agents of the Bureau of Alcohol, Tobacco, and Firearms. Since then, the gang prevention program has been replicated numerous times in cities across the country. By 1997, G.R.E.A.T. had been established in all 50 states and the District of Columbia with more than 2,400 participating officers. The G.R.E.A.T. program consists of four components—a middle school curriculum, an elementary school curriculum, a summer program, and a families training component. The core, required portion of the program is the middle school curriculum, which seeks to reduce gang activity, delinquent behavior, and violence by teaching students the consequences of gang involvement and by teaching them life skills and problem-solving techniques.

- 200 -

Program Participants

Middle school students, elementary school students, and their families

Evaluation Methods

Esbensen and Osgood (1999) conducted a retrospective one-year study of the effects of G.R.E.A.T. among eighth graders who had and had not participated in the middle school program during seventh grade. Eleven sites were included in the study, identified through records indicating that at least two police officers had been trained to teach G.R.E.A.T. in recent years, including Kansas City, Missouri; Las Cruces, New Mexico; Milwaukee, Wisconsin; Omaha, Nebraska; Orlando, Florida; Philadelphia, Pennsylvania; Phoenix, Arizona; Pocatello, Idaho; Providence, Rhode Island; Torrance, California; and Will County, Illinois. In each of the sites, researchers selected schools that had offered G.R.E.A.T. programs during the past two years. All eighth-graders who attended school on the day the surveys were administered became part of the sample, with attendance ranging from 75 to 93 percent among the schools. The final sample was 5,935 eighth-graders in 315 classrooms from 42 schools. Researchers randomly selected treatment classrooms in which the G.R.E.A.T. program had been offered and selected comparison classrooms from among those that had not implemented the G.R.E.A.T. curriculum (or any other gang-related curriculum). No pretests of students’ attitudes or behavior were conducted. Analysis of students’ demographic variables revealed few significant differences between groups, with the exception of treatment students being significantly younger than comparison students and being marginally more likely to live only with their fathers. Survey questions contained items measuring self-reports of delinquency and gang membership. Esbensen et al. (2001) conducted a second evaluation of the middle school G.R.E.A.T. program. Schools in six cities were selected for the study, based on the existence of a G.R.E.A.T. program in the city, geographic location, and the cooperation of the local school districts and police departments. The study sites included an East Coast city (Philadelphia), a West Coast city (Portland), the original G.R.E.A.T. city (Phoenix), a Midwestern city (Omaha), a "non-gang" city (Lincoln), and a small border city with chronic gang problems (Las Cruces). Classrooms were randomly assigned to the treatment or control group, and the analysis sample consisted of more than 3,500 students in 153 classrooms from 22 schools, roughly evenly divided between the treatment and control group. Because the G.R.E.A.T. program was implemented differently at each site (varying by class scheduling and structure of the schools), random assignment was implemented differently at each district and/or school. Pretest surveys indicated that the treatment group held more negative beliefs about gangs, higher rates of self-reported status offenses (offenses specific to juveniles, e.g., school truancy, curfew violations, or running away from home), higher rates of delinquency among peers, and lower rates of prosocial behavior among peers. The initial post-test survey was conducted two weeks after the program was completed. Subsequently, students were surveyed once a year for three consecutive years. The follow-up surveys required active consent from the parents in all sites; the final rate for parental consent was 57 percent of the total initial sample (2,045 students). Eighty-six percent of children who obtained parental consent completed the initial post-test (1,761 students), with follow-up rates of 76 percent, 69 percent, and 67 percent in the first, second, and third follow-up years, respectively. The surveys included questions on self-reported gang activity, drug use, and delinquent behavior.

Key Evaluation Findings

Esbensen and Osgood (1999) reported the following:

• Students who participated in the G.R.E.A.T. program reported significantly less drug use, total delinquency across all offense types, and minor offenses (e.g., drinking alcohol, petty theft, fighting) than did students who did not participate in G.R.E.A.T.

- 201 -

• No significant differences were found between groups for current or previous gang membership, number of offenses against persons or property, rates of selling drugs, or number of status offenses.

Esbensen et al. (2001) reported the following:

• At initial post-test, there were no significant differences between the G.R.E.A.T. and comparison groups for gang membership, drug use, total delinquency, minor delinquency, person delinquency (e.g., assault, robbery, harassment), property delinquency (e.g., arson, auto theft, burglary, larceny), or status delinquency (e.g., school truancy, curfew violations, or running away from home).

• As found from the longitudinal follow-up surveys, G.R.E.A.T. participants were significantly less likely to demonstrate property delinquency, while they were marginally less likely than the control group to have engaged in minor delinquency or to have exhibited higher scores on the measure of total delinquency. No significant differences were found between the groups for gang membership, drug use, person delinquency, or status delinquency.

Probable Implementers

Middle schools and elementary schools in cooperation with police departments serving high-risk/high- gang-activity areas

Funding

Federal funding for local agencies to implement the G.R.E.A.T. program is available through grants from the Bureau of Justice Assistance (BJA). In 2004, 215 BJA grants were given to municipal or state agencies to assist them in providing G.R.E.A.T. in their local schools and communities. Early in 2005 continuation grants were approved for most of those agencies, and later in 2005 BJA will hold open competition for remaining grant funds, with all law enforcement agencies eligible to apply. Interested applicants can find information about the grant application on the G.R.E.A.T. website, or on BJA’s website at http://www.ojp.usdoj.gov/BJA/grant/great.html. Previous funding for G.R.E.A.T. programs has been obtained through the Safe and Drug-Free Schools Act, the Office of Juvenile Justice Delinquency Prevention, Community Oriented Policing Services (COPS), the YMCA, the YWCA, and the Boys and Girls Clubs of America.

Implementation Detail

Program Design

The G.R.E.A.T. program requires formal partnerships between schools and local police departments, and a written agreement from a school is a required element of the school’s enrollment in the program. Trained police officers are in charge of the implementation and delivery of the curriculum. As such, G.R.E.A.T. schools must commit to allowing officer(s) to interact with the children both in a formal classroom setting and informally on the playground or in the cafeteria. These informal sessions are intended to enhance the bonding process between the officer(s) and students.

Curriculum

The G.R.E.A.T. program includes four components: (1) a middle school curriculum, (2) an elementary school curriculum, (3) a summer program, and (4) a families training component.

(1) The 13-week middle school curriculum, which is skills based, is designed to enhance students’ knowledge and produce changes in their attitudes and behavior through the use of facilitative teaching (which encourages active participation by students), positive-behavior rehearsal, cooperative and

- 202 -

http://www.ojp.usdoj.gov/BJA/grant/great.html

interactive learning techniques, and extended teacher activities. The curriculum is designed to provide students with the skills they need to avoid gang pressure and youth violence. The curriculum consists of 13 lessons, each 45-minutes to 60-minutes long, which cover topics such as "Relationship Between Gangs, Violence, Drugs, and Crime" and "Community, Roles and Responsibilities." (2) The elementary school curriculum, designed for fourth- and fifth-grade students, is a skills-based program that is similar to the middle school curriculum. The program has the added goals of early prevention of antisocial behavior and the promotion of positive relationships with law enforcement for children at an early age. The curriculum consists of six 30-minute to 45-minute lessons, each of which is accompanied by a letter to parents that explains the lesson and encourages student/parent interaction. The lessons cover topics such as "Bullying, Victim and Bystander" and "Identifying Adults When We Need Help." (3) The summer program builds on the school-based curricula by offering students an opportunity to enhance their social and cognitive skills, providing them with alternatives to gang involvement, and adding structure to the summer months. (4) The families program has the goal of strengthening communities and families by engaging youths age 10 to 14 and their parents in cooperative lessons designed to facilitate better communication among family members and enhance family decision-making skills. The curriculum uses group interaction, activities, and skills practice, and each of the six sessions is led by a facilitator trained in the G.R.E.A.T. families component. Staffing

The G.R.E.A.T. middle school and elementary school curricula are facilitated by uniformed police officers in the school. Guidelines are offered G.R.E.A.T. Program Coordinators to assist schools in the selection of officers being considered for training. G.R.E.A.T. requires a 40-hour training component for officers who have previous experience in school- based instruction or an 80-hour training component for officers who have no previous experience in this area. A two-day training session is available for certified G.R.E.A.T. officers who are interested in leading G.R.E.A.T. families programs. Certified facilitators are authorized to train members of their co- facilitation teams, who may include teachers, youth counselors, parents, or other qualified adults who are committed to working with families.

Issues to Consider

The G.R.E.A.T. program received a "promising" rating. Both evaluations of the program indicate that treatment-group students had more positive outcomes than comparison students on self-reported measures of total delinquency and minor delinquency. The two studies of the G.R.E.A.T. program exhibit methodological weaknesses. For example, while Esbensen and Osgood (1999) used sophisticated statistical techniques, their comparison group had shortcomings. They analyzed outcomes among groups of students who had not been randomly assigned to the treatment group and the researchers did not have baseline (pretest) data available to ensure group comparability. Additionally, while the study by Esbensen et al. (2001) made use of a randomized design and a pretest, randomization was not implemented uniformly across sites, and the requirement for active parental consent resulted in a baseline participation rate of only 57 percent of the eligible students. Thus, generalizations to the entire study population are somewhat limited. While the central goal of the G.R.E.A.T. program is prevention of gang involvement, no significant findings on gang-related activity measures were noted in either study. Similarly, neither study reported significant program effects on rates of offenses against person. Conflicting results were found for drug use, with Esbensen and Osgood (1999) reporting significantly reduced drug use among treatment students, but Esbensen et al. (2001) reporting no significant differences between groups for any of the four follow-up measurements. Similarly, conflicting results between the studies were found

- 203 -

for rates of property delinquency, with Esbensen and Osgood (1999) reporting no differences between treatment and control groups, and Esbensen et al. (2001) reporting significantly higher rates of property offenses among control students. The study by Esbensen et al. (2001) suggests that the G.R.E.A.T. program may have a delayed impact on participants, because study results indicated no significant differences between treatment and control groups immediately following program completion, but there were a handful of significant findings from the one-, two-, and three-year follow-ups.

Example Sites

• Boston Police Department

• Philadelphia Police Department

• Phoenix Police Department

• Portland Police Bureau

• Tucson Police Department

Contact Information

G.R.E.A.T. Program Coordinator Institute for Intergovernmental Research P.O. Box 12729 Tallahassee, FL 32317 Web site: http://www.great-online.org Telephone: (800) 726-7070 Fax: (850) 386-5356 Email: information@great-online.org

Available Resources

Training is available through one of five regional training sites for law enforcement officers wanting to become qualified G.R.E.A.T. program facilitators:

• Midwest Region: La Cross Police Department, La Cross, Wisconsin

o (877) 86-GREAT

• Northeast Region: Philadelphia Police Department, Philadelphia, Pennsylvania

o (215) 686-1477

• Southeast Region: Orange County Sheriffs Office, Orlando, Florida

o (800) 363-5569

• Western Region: Portland Police Bureau, Portland, Oregon

o (800) 823-7188

• Southwest Region: Phoenix Police Department, Phoenix, Arizona

o (800) 24-GREAT

More information and a downloadable application form are available through the G.R.E.A.T. website at http://www.great-online.org.

- 204 -

http://www.great-online.org/

mailto:information@great-online.org

http://www.great-online.org/

Bibliography

Esbensen, Finn-Aage, and D. Wayne Osgood, "Gang Resistance Education and Training (G.R.E.A.T.): Results from the National Evaluation," Journal of Research in Crime and Delinquency, Vol. 36, No. 2, 1999, pp. 194-225. Esbensen, Finn-Aage, D. Wayne Osgood, Terrance J. Taylor, Dana Peterson, and Adrienne Freng, "How Great is G.R.E.A.T.? Results from a Longitudinal Quasi-Experimental Design," Criminology and Public Policy, Vol. 1, No. 1, 2001, pp. 87-118.

Last Reviewed

July 2005

Get Real About AIDS

Program Info

Outcome Areas Healthy and Safe Children

Indicators Youths abstaining from sexual activity or not engaging in risky sexual behavior

Topic Areas Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School High School Community-Based Service Provider Type of Service Health Education Youth Development Type of Outcome Addressed Physical Health Substance Use and Dependence Teen Sex/Pregnancy

Evidence Level Promising

Program Overview

The Get Real About AIDS program was first developed in 1988 to help prevent the spread of sexually transmitted diseases, HIV, and AIDS among high-school-aged young people. It was updated in 1992 and further revised in 1994 when the curriculum was designated by the U.S. Centers for Disease Control and Prevention (CDC) as a "Program That Works." The 14-session program includes training in basic sexuality, abstinence, and contraception education as well as training in behavioral skills. The program encourages teens to delay having sex, or, for those who do become sexually active, to reduce the risk of HIV by adopting such behaviors as being monogamous, avoiding drugs and alcohol, practicing safer sex, and not sharing needles.

- 205 -

The program is based on the cognitive and reasoned action theories of human behavior. These theories view actions as based on thinking style and intention. Thus, the program aims to change the sexual behavior of teens by providing information and by helping to develop and change the participants' understandings and beliefs.

Program Participants

The curriculum is used predominantly in school-based settings, but also has been implemented by community organizations. In addition to a third edition of the program for high school students, two other curricula have been developed for younger children in grades 4 through 9.

Evaluation Methods

In a study done in Colorado in the fall of 1991, ten schools were selected to receive the Get Real About AIDS curriculum. Students in these schools were compared to students at seven demographically matched schools that either continued to offer their current HIV-related curricula or where no HIV-related curricula were used. The treatment and comparison groups were very similar, with the only significant difference being that the average age of the treatment students was slightly higher (14.99 years for the treatment group versus 14.73 years for the comparison group). More than 2,800 students participated, filling out surveys before the program, after the first semester, and six months after completing the program. The researchers were not allowed to use names or code numbers on the surveys, so they matched surveys across 16 school and demographic variables to make comparisons over time. Approximately half of the students completed the survey at the six- month follow-up. Students were asked about their number of sexual partners, their use of contraceptives, how often they had sex, if at all, and their use of drugs and alcohol before sex. The matched follow-up sample for the treatment and comparison groups had only two differences in their backgrounds that were significant. As mentioned before, the treatment group was older. In addition, the comparison group was significantly more likely to have had more than three lifetime sex partners. None of the other differences between the groups were significant.

Key Evaluation Findings

The Colorado evaluation (Main et al., 1994) found that: - Compared with the group that didn’t receive the training, Program Participants who were sexually active indicated that six months after completing the program they had fewer sexual partners, purchased and used condoms more frequently, intended to engage in sex less frequently with fewer partners, and planned to use condoms more often when they did engage in sex. All of these differences were significant. - Students in the program group also had significantly greater knowledge of HIV, increased commitment to engage in safer sexual practices, and were more likely to believe that someone their age who engaged in risky behaviors could become infected with HIV than did students who did not participate in the program. - The program did not delay the start of sexual activity or reduce the frequency of sex or drug and alcohol use prior to sex.

Probable Implementers

Schools and community organizations.

- 206 -

Funding

Federal and local health department HIV/AIDS funds, generally from sources such as the Centers for Disease Control and Prevention (CDC) of the U.S. Department of Health and Human Services (HHS) and the Substance Abuse and Mental Health Services Administration (SAMHSA); local school districts and community organizations.

Implementation Detail

Program Design

The program uses a variety of approaches to reach teens, including group discussions, lectures, role- playing, games, books, work sheets, simulations, and videos. Many groups offering the program also supplement the program with activities designed to reinforce program content (for example, by using HIV posters or distributing wallet-sized HIV information cards). In addition, the program package includes activities for schools, families, and communities. The program also has a parent component that includes activities the children and parents can do together and a parent newsletter.

Curriculum

This program has a 14-lesson curriculum that uses entertaining activities, discussions, role-playing, simulations, and videos to give teens the knowledge and skills needed to reduce their risk of HIV infection. It is generally offered over the course of a semester. During the first half of the program, students study HIV transmission and prevention, teen vulnerability to HIV, and what factors cause risky behavior. In the second half of the program, students learn and practice social skills to help them recognize, manage, avoid, and leave risky situations that they are likely to encounter.

Staffing

Classroom teachers conduct the program in a school setting, preferably in health or science classes. In other settings, youth or community organization staff direct the program. At some sites, peer leaders now assist with program implementation. Teachers who participated in the program by teaching the Get Real About AIDS curriculum were trained in a 5-day, 40-hour training session. They were also observed periodically during the study to ensure that their teaching was faithful to the design of the curriculum.

Issues to Consider

This program received a “promising” rating. There is only one evaluation, and while the sample size is very large, the data have a number of limitations. The attrition across the waves of the survey is very high, and those students who did not complete follow-up surveys were more likely to engage in risky behavior. In addition, matching of students’ responses over time is imperfect because they were not allowed to include any individual identifiers. Although the results of the evaluation do tend to favor the program, it is difficult to assess the size of the impact. There is some evidence of effects from the program, but the effects were measured only until the end of the school year. Also, evidence suggests that there were changes in knowledge and intentions among Program Participants but fewer changes in actual behaviors, such as the onset of sexual intercourse.

Example Sites

Colorado; San Diego

- 207 -

Contact Information

For a preview of or to purchase program materials, contact:

Discovery Education info@unitedlearning.com For staff training, contact: Comprehensive Health Education Foundation (CHEF) 22419 Pacific Highway South Seattle, WA 98198 1-800-323-2433 fax: 206-824-3072 chefstaff@chef.org www.chef.org/curricl.htm

Available Resources

Get Real About AIDS program materials are available from AGC Educational Media for three age groups: high school students and young people in grades 4–6 and in grades 6–9. The program package includes a teacher’s guide, lesson plans, curriculum materials, and a parent newsletter. In addition, special training for instructors and "train the trainer" sessions are available from CHEF. A Get Real About AIDS program package developed by Sociometrics includes a user's guide, implementation guidelines, staff training and evaluation materials, a directory of evaluation consultants, and telephone technical support for a year (contact Sociometrics at 1-800-846-DISK).

Bibliography

Main, D. S., et al., “Preventing HIV Infection Among Adolescents: Evaluation of a School-Based Education Program,” Preventive Medicine, Vol. 23, 1994, pp. 409–417.

Last Reviewed

April 2004

Go Grrrls

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children not experiencing anxiety or mood disorders, such as depression Children experiencing good physical health

Topic Areas Age of Child Middle Childhood (9-12) Adolescence (13-18)

- 208 -

mailto:info@unitedlearning.com

http://www.chef.org/curricl.htm

mailto:chefstaff@chef.org

Type of Setting Out of School Time Type of Service Youth Development Type of Outcome Addressed Mental Health

Evidence Level Promising

Program Overview

Go Grrrls is a program designed to promote healthy psychosocial development in female adolescents. The program has been delivered to girls in grades six through nine and is expected to occur as a series of 12 sessions held outside of traditional school hours. The 12-session curriculum covers six topics related to female adolescent development: Being a girl in today's society, Establishing a positive self-image, Establishing independence, Making and keeping friends, When it all seems like too much, and Planning for the future. Sessions are expected to last 60 minutes and occur once per week. They are made up of groups of 8-10 female adolescents, led by two female graduate psychology students or professionals with master's degrees in social work (MSWs). The sessions include didactic instruction, class discussion, group exercises, completion of worksheets, role-playing, and weekly journal assignments. The Go Grrrls program materials include a Go Grrrls curriculum guide for group leaders and a Go Grrrls workbook for use by group attendees. In addition, the program developers offer one-day trainings for group leaders on topics such as an introduction to Go Grrrls and female adolescent development, being a group leader, and advanced topics related to leading a Go Grrrls group.

Program Participants

Adolescent females, grades six through nine.

Evaluation Methods

LeCroy (2004) evaluated the effectiveness of the Go Grrrls program in promoting healthy psychosocial development among 118 adolescent females attending a middle school in a semi-urban location in Arizona. Females were recruited through parent orientation sessions held at their school over the summer. A total of 118 female middle school students and their parents consented to participate in the study. Fifty-nine girls were randomly assigned to participate in the Go Grrrls program, and 59 were randomly assigned to serve as control study participants. The average age of the girls in the study was 13.5 years. About 62 percent of the girls were Caucasian, about 14 percent were Hispanic, about 33 percent were Native American, and less than 2 percent were African-American. About 51 percent of the girls lived in single-parent homes, and about 37 percent received free lunch at school. There were no differences in demographic variables between the control group of girls and the Go Grrrls program group before the intervention began. The Go Grrrls program was administered to groups of 8-10 female adolescents, each led by two female co-leaders with either an MSW or graduate-level experience in psychology. Group sessions lasted for one hour and occurred weekly over a period of 12 weeks. The co-leaders were supervised weekly through separate small group meetings, lasting 30 to 60 minutes, to ensure that the curriculum was delivered similarly across groups. All girls participating in the study completed self- report questionnaires just prior to and just after the Go Grrrls program 12-week series. The questionnaires included scales designed to rate the girls' competency in areas such as self esteem, acceptance of body image, attitude toward attractiveness, self-efficacy, self-liking and self-confidence, and hopelessness. Two girls who were assigned to receive the Go Grrrls program were excluded from the study analysis: One of these girls dropped out of the program, and the other girl did not receive "enough" of the program, in the study author's estimation. Receiving 75 percent of sessions was

- 209 -

considered sufficient exposure to the program to detect an effect, but the study author does not note a specific threshold used to include girls in the analysis.

Key Evaluation Findings

LeCroy (2004) found that, compared with girls in the control group, girls who completed the Go Grrrls program showed significantly greater improvement (at p < .05) in:

• body image — as measured by an Acceptance of Body Image Scale developed by Simmons and Blythe (1987).

• assertiveness — as measured by an Assertiveness Scale developed by Center for Substance Abuse Prevention (1993).

• attractiveness — as measured by an Attitude Toward Attractiveness Scale developed by LeCroy and Daley (1997a) that assesses healthy attitudes toward attractiveness by asking girls questions such as, "I think girls need to be skinny to be attractive." The scale was standardized so that an increase in scores indicates an increase in healthy attitudes.

• self-efficacy — as measured by a Girl's Self-Efficacy Scale developed by LeCroy and Daley (1997b).

Girls in the program also showed marginally significantly (p = .09) greater improvement on the Hopelessness Scale (representing a general reduction in feelings of hopelessness) developed by Kazdin, Rogers, and Colbus (1986).

Probable Implementers

Community organizations, including schools, with a commitment to healthy female development.

Funding

The authors received funding from the Center for Substance Abuse Prevention.

Implementation Detail

Program Design

The program is designed as an out-of-school-time group series. Female adolescents attend group sessions of about 8-10 adolescent females, led by two adult female co-leaders. Sessions occur over 12 weeks and last one hour per week, with girls completing weekly journal assignments between sessions.

Staffing

The groups are led by two adult females, with master's degrees in social work or graduate-level training in psychology.

Curriculum

The curriculum covers six major topics that build on each other over the course of 12 weeks. Each week, girls receive didactic instruction and engage in role-plays, exercises, group discussion, and journal assignments. The topics covered are:

• Being a girl in today's society — focuses on pressure placed on females through media and teaches positive perceptions of gender.

- 210 -

• Establishing a positive self-image — helps girls maintain a positive body image, establish a stable sense of self-worth, and understand that attractiveness is about more than physical appearance.

• Establishing independence — teaches problem-solving skills and focuses on mature, independent thinking.

• Making and keeping friends — encourages development of meaningful friendships and teaches girls how to use friendships for support and well-being.

• When it all seems like too much — teaches girls how to recognize when situations are too much to handle alone and how to find available resources, such as helpful peers or hotlines and crisis centers.

• Planning for the future — teaches girls to establish educational, vocational, and other goals.

Issues to Consider

Craig LeCroy, author of the LeCroy (2004) study, is one of the Go Grrrls program developers; thus, readers should be aware that the study was not conducted by an independent, third-party evaluator. In addition, many of the scales used to assess female outcomes were developed by LeCroy, and the validity of these scales has not been well established. Further, there is no mention of intervention fidelity monitoring. Also, each group of girls was supervised by two co-leaders, and these co-leaders varied from group to group. This study did not make any statistical adjustments to account for group leader effects on variability in program outcomes. Although the random assignment and pre/post-test methods used are elements of a strong research design, this program has been designated as a promising program due to the above-listed factors.

Example Sites

The program has been implemented in some middle schools in Arizona.

Contact Information

Craig W. LeCroy, M.S.W., Ph.D. 340 N. Commerce Park Loop Suite 250 Tucson, AZ 85745-2700 Phone: (520) 884-5507 ext. 15 Email: craig.lecroy@asu.edu Jan Daley, M.S.W. Consultant Email: jan.daley@yahoo.com

Available Resources

A brief description of the Go Grrrls program, training programs, resources, and the curriculum can be found at http://www.public.asu.edu/~lecroy/gogrrrls/body.htm In addition, links to the "Go Grrrls Workbook" ($12 in 2011 dollars) and the book version of the Go Grrrls framework, which includes the step-by-step curriculum for group leaders, "Empowering Adolescent Girls" ($35 in 2011 dollars), can be found under the "Curriculum" section of this site: http://www.public.asu.edu/~lecroy/gogrrrls/curriculum.htm

- 211 -

mailto:craig.lecroy@asu.edu

mailto:jan.daley@yahoo.com

http://www.public.asu.edu/%7Elecroy/gogrrrls/body.htm

http://www.public.asu.edu/%7Elecroy/gogrrrls/curriculum.htm

Bibliography

Center for Substance Abuse Prevention, "Knowledge, Attitude, and Behavior Instrument (KAB)," unpublished manuscript, Washington, D.C., 1993. Kazdin, Alan E., Antoinette Rogers, and Debra Colbus, "The Hopelessness Scale for Children: Psychometric Characteristics and Concurrent Validity," Journal of Consulting and Clinical Psychology, Vol. 54, No. 2, April 1986, pp. 241-245. LeCroy, Craig Winston, "Experimental Evaluation of 'Go Grrrls' Preventive Intervention for Early Adolescent Girls," Journal of Primary Prevention, Vol. 25, No. 4, December 2004, pp. 457-473. LeCroy, Craig Winston, and J. Daley, "Girls' Attitude Toward Attractiveness Scale," unpublished, 1997a. LeCroy, Craig Winston, and J. Daley, "Girls' Self Efficacy Scale," unpublished manuscript, 1997b. Simmons, R. G., and D. A. Blythe, Moving into Adolescence: The Impact of Pubertal Change and School Context, New York: Aldine De Gruyter, 1987.

Last Reviewed

November 2011

Guiding Good Choices

Program Info

Outcome Areas Healthy and Safe Children

Indicators Youths not using alcohol, tobacco, or illegal drugs

Topic Areas Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School High School Community-Based Service Provider Health Care Provider Type of Service Family Support Health Education Parent Education Youth Development Type of Outcome Addressed Physical Health Substance Use and Dependence

Evidence Level Proven

- 212 -

Program Overview

Guiding Good Choices (GGC) (formerly known as Preparing for the Drug Free Years) is a program designed to teach parents skills that can help prevent drug and alcohol abuse in their families. Dr. David Hawkins and Dr. Richard Catalano developed the program based on the social development model, which holds that strong bonding to positive influences reduces problem behaviors, such as delinquency and substance abuse. This is combined with research that has identified both risk and protective factors in the development of behavior problems. Risk factors include a low level of communication between parents and children, poorly defined and communicated expectations for children’s behavior, excessively severe and inconsistent discipline, and high levels of negative interaction or family conflict. Protective factors include regular communication or parental warmth and affection, presentation of clear and pro-social expectations, monitoring of children, and consistent and moderate discipline. The primary goal of GGC is to increase family involvement that is rewarding and enhances parent- child bonds. The program is offered in a series of sessions, each designed to focus on one of five areas. The program begins with increasing parents’ knowledge of the risk factors associated with drug abuse. It then focuses on teaching parents the skills that help mitigate these risk factors, such as how to clearly communicate expectations for behavior, how to reduce family conflict, and how to encourage the expression of positive feelings and love. One of the sessions teaches both parents and children various ways to resist peer and social pressures to engage in inappropriate behavior.

Program Participants

The GGC curriculum was designed for a general public, and is primarily intended for parents with children age 9 to 14. The program was pilot-tested in ten Seattle public schools. Among the participating students, 52 percent were minorities, 48 percent were from low-income families, and 39 percent were from single- parent homes. The evaluation studies looked at 209 families in the central Midwest, where the families were predominantly white. It has been used to train more than 275,000 urban, suburban, and rural families in all 50 states, DC, PR, VI and Canada.

Evaluation Methods

Note: GGC was evaluated when the program was called Preparing for the Drug Free Years (PDFY), and this section uses the PDFY name to match the language found in the evaluation studies. The first evaluation (Kosterman and Hawkins, 1997) recruited families of all sixth and seventh graders from six schools districts in the central Midwest. Of the 387 families identified, 209 completed the pretest assessment. The families were then randomly assigned to either the program or to a wait- listed control group. The measures included surveys and videotapes of two family activities, Task 1 and Task 2, conducted at the families’ homes before and following the intervention. Before each videotaping, family members independently filled out a survey identifying the primary causes of family disagreements, such as chores, curfews, or finances. Then in Task 1, the family was asked to answer some questions concerning general family life, such as how chores were handled. In Task 2, the family was asked to discuss and resolve an issue they had identified in the earlier survey. Using the videotapes, the families were then scored on a five-point scale across 60 different individual dimensions. For analysis, these dimensions were grouped into three main areas: proactive communication, negative interaction, and relationship quality. The analysis only included parents participating in both the pretest and posttest (174 mothers and 157 fathers), regardless of how many individual sessions the parents had attended. The second evaluation (Park et al., 2000) recruited all families of sixth graders enrolled at 33 rural schools in 19 contiguous, economically stressed counties in a Midwestern state. Grouped by school size and proportion of lower-income students, 11 schools were assigned to the control group, 11 to

- 213 -

the PDFY program, and 11 to a second intervention condition. Of the 883 families in the 22 PDFY or control schools, 424 families agreed to participate and completed the initial pretest assessment (217 PDFY and 207 control families). Of these, 85 percent completed the posttest, 73 percent completed the one-year follow-up, 67 percent completed the two-year follow-up, and 70 percent completed the three-year follow-up (144 PDFY and 151 control families). Information was collected up to two months prior to the intervention with family questionnaires and a videotaped family task, although the videotapes were not used in the evaluation. A similar posttest assessment was completed approximately nine months later, with follow-up assessments at about 1, 2, and 3.5 years after the intervention. The evaluation team created four constructs based on items from the parent and child surveys. The first measured parents’ norms against substance use (e.g., How wrong would it be for children who are the same age as your child to drink alcohol?). The second measured family management, assessing parents’ vigilance in monitoring the child. The third assessed family conflict -- the amount and the ways in which the family dealt with conflict. The fourth assessed the child’s refusal skills -- his or her ability to refuse peer pressure to drink. The measure for alcohol use was constructed from the child’s self-reporting of initial alcohol use and frequency of drinking. In the scales that were developed, a higher score represents a higher level of the construct -- i.e., stronger parental norms, better family management, more family conflict, better refusal skills, and more alcohol use.

Key Evaluation Findings

• The program was effective in promoting proactive communication from parent to child. After the program, mothers in Task 1 and both parents in Task 2 showed much more proactive communication.

• In Task 1, mothers were significantly less likely to engage in negative interaction, specifically antagonistic behavior.

• Both mothers and fathers appeared to be positively influenced by the program, but the behavior appeared to vary by the type of family interaction context. The mothers’ improvements were most noticeable in Task 1 while the fathers’ improvements were significant only in Task 2.

The Park et al. (2000) study found that:

• Over a two-year period, PDFY parents showed significantly more improvement than the control group in norms against alcohol and other drug use.

• There was significantly less growth in alcohol use among youths in the PDFY group over time compared with controls. At the three-and-a-half-year follow-up, 65 percent of the control students reported initiating alcohol use compared with 52 percent of the PDFY students.

• There was also less total alcohol use among youths in the PDFY group. At the three-and-a- half-year follow-up, 40 percent of the control students reported using alcohol in the past month compared with 24 percent of the PDFY students, and 42 percent of the control students reported being drunk compared with 32 percent of PDFY students.

A cost-benefit analysis of PDFY (Roth, Guyll & Day, 2002) estimates a $5.85 benefit for every dollar invested in the program.

- 214 -

Probable Implementers

Schools and community organizations.

Funding

The National Institute of Drug Abuse funded the original research and program development by Hawkins and Catalano. The U.S. Department of Education's Drug Free Schools and Communities Program sponsored the field-testing and evaluation study. The program implemented in the Midwest costs approximately $730 (in 2003 dollars) per participant (Roth, Guyll & Day, 2002).

Implementation Detail

Program Design

• The goal of the program is to reduce drug abuse and behavioral problems by

o increasing involvement and interaction between parents and children

o reducing family conflict

o increasing the promotion of good behavior through better and more consistent family management.

• The program targets parents with children age 9 to 14, a time when they are facing increasing numbers of outside risk factors.

• Recruiting and retaining parents for the full five sessions can be challenging. The most successful approaches use multiple strategies focused on overcoming the common barriers to participation and that are specifically tailored to a community’s needs.

Curriculum

The program is divided into five two-hour sessions, or ten one-hour sessions. Each session has a particular theme and parents are given activities to complete at home with the entire family. Session 1, "Getting Started: How to Prevent Drug Abuse in Your Family," provides an overview of the risk factors associated with adolescent substance abuse and the theory of why social bonding may help mitigate those factors. Session 2, "Setting Clear Family Expectations on Drugs and Alcohol," focuses on parenting skills. Parents learn how to develop family guidelines, how to effectively convey their expectations for children’s behavior, and how to establish consequences for bad behavior. Parents also involve their children in a family meeting to develop a family policy about alcohol and other drugs. Session 3, "Avoiding Trouble," is the one session that requires the children's attendance with the parents. The session discusses the social and peer pressures adolescents will face regarding substance abuse, and then teaches skills to resist such pressures. The session focuses on the risk factors of peers who use drugs. Together, parents and children practice skills to resist peer influences using the five steps of "Refusal Skills." Session 4, "Managing Family Conflict," is aimed at reducing risks related to family conflict, negative interaction, and rebelliousness. Among other things, parents are taught ways to properly express and control anger without damaging family bonds and how to teach their children these same skills. In Session 5, "Strengthening Family Bonds," parents learn ways to expand family involvement. The session also emphasizes the benefits of expressing positive feedback and love.

- 215 -

Staffing

The sessions are usually led by two trained community volunteers who attend a three-day training workshop where they are taught to used a structured, standardized curriculum for conducting interactive workshops.

Issues to Consider

This program received a "proven" rating. The two evaluations used random assignment designs with large sample sizes and produced scientifically credible and significant program effects. There is, however, an issue regarding participation. The families all participated in the program voluntarily in the intervention. In the Kosterman and Hawkins evaluation, 57 percent of the families agreed to participate, whereas in the Park et al. study, only 48 percent of the eligible families agreed to participate. Although it is not clear whether the program results would be similar in cases in which families did not participate voluntarily, it is likely that participation would be voluntary in most replication settings. While these two evaluations suggest the program would only reach about half of the potential families, the results are positive for those families who do participate. Furthermore, the program seems to have positive effects even though many families did not attend every session. For example, in the Park study, only 61 percent of the families attended all five sessions. In addition to affecting the substance use benchmark, the program also had positive impacts on other outcomes such as communication between parents and their children. Finally, when considering the evaluations and outcomes, it is important to note that the evaluation team included a developer of the program.

Example Sites

Seattle, Washington School District; Iowa State University, Project Family; Florida, Broward County Commission on Substance Abuse

Contact Information

Channing Bete Company One Community Place South Deerfield, MA 01373-0200 Phone: (877) 896-8532 Fax: (800) 499-6464 E-mail: custsvcs@channing-bete.com Web site: www.channing-bete.com/ggc

Available Resources

Training is available from certified, experienced trainers who will assist in the implementation of the program and help ensure its fidelity. The training contact for Guiding Good Choices is: Dr. Dorothy Ghylin 81 NW Doncee Dr. Bremerton, WA 98311-9110 (360) 649-8903 (mobile) or (360) 692-9986 (360) 613-0726 (fax) dr.d.ghylin@comcast.net

- 216 -

mailto:custsvcs@channing-bete.com

http://www.channing-bete.com/ggc

mailto:dr.d.ghylin@comcast.net

The Office of Juvenile Justice and Delinquency Prevention (OJJDP) provides information on Guiding Good Choices (formerly known as Preparing for the Drug Free Years) at http://www.ncjrs.org/html/jjbulletin/9907/contents.html.

Bibliography

Kosterman, R., and J. David Hawkins, "Effects of a Preventive Parent-Training Intervention on Observed Family Interactions: Proximal Outcomes from Preparing for the Drug Free Years," Journal of Community Psychology, Vol. 25, No. 4, 1997, pp. 337-352. Park, J., Kosterman, R., Hawkins, J. David, et al., "Effects of the 'Preparing for the Drug Free Years" Curriculum on Growth in Alcohol Use and Risk for Alcohol Use in Early Adolescence," Prevention Science, Vol. 1, No. 3, 2000, pp. 125-138. Spoth, R.L., Guyll, M., and Day, S.X., “Universal Family-Focused Interventions in Alcohol-Use Disorder Prevention: Cost Effectiveness and Cost-Benefit Analyses of Two Interventions,” Journal of Studies on Alcohol, Vol. 63, No. 2, 2002, pp. 219-228.

Last Reviewed

August 2009

Head Start

Program Info

Outcome Areas Healthy and Safe Children Children Ready for School

Indicators Children ages 0 to 5 exhibiting age-appropriate mental and physical development Children experiencing good physical health

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Type of Service Family Support Instructional Support Parent Education Type of Outcome Addressed Behavior Problems Cognitive Development/School Performance Physical Health

Evidence Level Promising

- 217 -

http://www.ncjrs.org/html/jjbulletin/9907/contents.html

Program Overview

Head Start is a federal matching grant program that was initiated in the mid-1960s as part of President Lyndon B. Johnson's "War on Poverty." The mission of the Head Start program is to "promote school readiness by enhancing the social and cognitive development of children through the provision of educational, health, nutritional, social and other services to enrolled children and families" (ACF, 2008). For eligible low-income children, Head Start provides free access to preschools, which are provided primarily in centers and are expected to conform to a specific set of guidelines laid out in the Head Start Program Performance Standards and other regulations (45 CFR 1301-1311, 2006). In addition to early childhood education, Head Start incorporates parental involvement and facilitates access to health care services; most Head Start programs also provide at least one meal to children during the day. The program requirements are flexible to meet the needs of the community, allowing individual programs to determine such program characteristics as the number of hours per day, or months per year, that a participation must attend; curriculum; teacher salary; and mode of delivery (in a home, a school, or a center) (GAO, 1998; ACF, 2008).

Program Participants

Head Start programs primarily serve low-income children ages 3 to 5 and their families. Early Head Start programs, another federal program that is summarized on PPN, serve children prenatal to age 3, pregnant women, and their families.

Evaluation Methods

Head Start has been studied extensively for the past 40 years, with hundreds of studies of the program having been completed since its inception. However, the great majority of these studies utilized methods that do not meet PPN criteria for inclusion in our program review. Beyond issues with study design, researchers studying Head Start face unique difficulties when evaluating the program that result from the program's structure as well as its national scope. We have summarized the difficulties in evaluating Head Start in the PPN Issue Brief “Head Start: What Do We Know?”, and will not detail them here. Because of these difficulties, only analyses conducted on data from one study out of the hundreds identified have been included in this program summary. The study is a randomized controlled trial funded by the U.S. Department of Health and Human Services (DHHS) (ACF, 2010b) and performed within Westat. The authors conducted a randomized controlled trial of a representative national sample of Head Start programs using a wait-list control—that is, the authors randomly assigned children to either Head Start or the Head Start wait list. The authors investigated outcomes for approximately 5,000 newly entering 3- and 4-year-olds in 383 Head Start centers in 84 randomly selected Head Start sites across 23 states. The group assignment was done separately for 3-year-olds (2,559 children) and 4-year-olds (2,108 children). No statistically significant differences were found between the characteristics of children randomly assigned to the Head Start and non-Head Start groups. Of the original sample of 4,667 children, 87 percent completed two or more assessments, and 81 percent completed three or more assessments. Attrition rates were larger for the control group. The data were statistically weighted so that the longitudinal sample was representative of the newly entering Head Start population. The control group was eligible to receive any other non-Head Start services available in the community, as chosen by their parents. The 3-year-old control group children were allowed to attend any preschool at age 4, and half of them did attend Head Start programs the subsequent year. There was some degree of treatment and control group crossover: Fourteen percent of children assigned to the Head Start treatment group did not end up attending Head Start, and 18 percent of children assigned to the control group attended a different Head Start program in their area. Due to the wait-list control methodology, the randomization could occur only in those sites that had an excess of Head Start applicants. These sites and their participants differ significantly from sites that

- 218 -

did not experience an excess of applicants in the following ways: These programs have less Hispanic enrollment, less total enrollment, and are more often elementary school-based. The researchers used statistical weighting procedures to account for these differences, so the weighted outcomes are for a nationally representative sample of Head Start enrollees. Outcomes were assessed immediately following program participation, after one year of kindergarten, and again at the end of first grade. Outcomes were measured across a range of 64 variables assessing outcomes in four areas: parenting practices and child cognitive, social-emotional, and health outcomes. A developmentally relevant subset of these 64 measures, which included variables from the four outcome areas, was measured at each point in time. It is important to note that this study evaluated children attending Head Start during the 2002-2003 program year, which is four years prior to Head Start's reauthorization in the Improving Head Start for School Readiness Act of 2007 (P.L. 110-134).

Key Evaluation Findings

Below we summarize the evaluation findings for each cohort of participants. The majority of the favorable outcomes identified for participants relative to the control group were in measures of language, literacy, and pre-writing, but this is also the category in which the most outcomes were measured. Please note that we only report program gains that were statistically significant at the 10 percent level or less. While the size of the effects are an important consideration, we do not report the effect sizes of the dozens of outcome measures reported in this summary. We encourage readers to consult the two study publications for detailed information about the scales and magnitudes of the measures. Findings for the 3-Year-Old Cohort Among children and their parents who entered the Head Start study when they were 3 years old and participated in Head Start for up to two years, the DHHS study found significant improvements in 14 of the 32 outcome measures relative to children assigned to the control group at the end of the first year of Head Start. Most of these positive impacts were on measures of language, literacy, and pre- writing, but improvements were also seen in some parenting, health, and social-emotional measures. One year later, these children and their parents showed significant improvement in 5 of the 37 measures assessed. Two of these were in language and literacy, one in parent reports of the child's social skills, one in the child's dental coverage, and one in parenting. At the end of kindergarten, 54 outcome measures were assessed, and significant improvements were seen in 7 of the measures, while an unfavorable outcome among Head Start participants was observed in a measure of math ability. The positive outcomes included a Spanish-language assessment of vocabulary, parent-reported social skills and hyperactivity, and health insurance coverage. Parent's disciplinary use of time-out and spanking also decreased. Finally, at the end of first grade, Head Start participants and their parents who were assigned to Head Start at 3 years of age showed significant improvements relative to the control group in 5 of 58 measures. These included improvements in oral comprehension, parent reports of the child's closeness and positive relationships, reductions in parent's utilization of time-out, and reductions in parents reporting that they had an authoritarian parenting style. In sum, among the 3-year-old cohort, positive effects were measured at the end of the first year of Head Start, and the number of positive effects, as a percentage of the total number of outcomes measured, decreased over time (44% of the outcomes assessed had significant improvements after the year of Head Start, 14% at age 4, 13% after the kindergarten year, and 9% after first grade). These positive outcomes were not consistent over time.

- 219 -

Findings for the 4-Year-Old Cohort

Among children and their parents who were assigned to participate in Head Start when the children were 4 years old, the DHHS study documented statistically significant improvements relative to the control group in 9 out of 40 outcomes assessed at the end of their year of Head Start. Seven of these were in the language, literacy, and pre-writing category. Participants saw significant improvements in dental coverage, and parents reported less disciplinary use of time-out in the previous week. At the end of their kindergarten year, these children showed improvement relative to the control group in 2 out of 55 outcomes assessed. Both of these were in the health category, with children more likely to have health insurance coverage and parents more likely to report their child's health status as excellent/good. At the end of their first grade year, 4-year-old Head Start participants showed statistically significant gains in 3 out of 53 outcomes, which included improvements in vocabulary, reductions in parent- reported withdrawn behavior, and improvements in health insurance coverage. However, participants also showed unfavorable outcomes in 2 of these measures related to teacher-reported socio-emotional outcomes. Teachers were more likely to report Head Start participants as shy/reticent and were more likely to identify that they had problems with teacher interaction. In other words, the results for 4-year-old Head Start participants were similar to those for 3-year-old Head Start attendees—relatively few outcomes exhibited significant improvements, and those that did were not consistently sustained past the initial period. Subgroup Analyses The evaluation included analyses of differential effects by child subgroup. In general, the findings by subgroup were consistent with those for the entire study population, with a few notable exceptions. Children from the 3-year-old cohort with special needs showed significant benefits in math and social- emotional areas at the end of first grade, as well as displaying reductions in behavior problems as reported by their teachers. Dual-language learners from the 4-year-old cohort experienced improved ratings of overall health and improved health insurance coverage at the end of kindergarten and improved health and more dental care at the end of first grade. Children from the 4-year-old cohort in the lowest academic quartile at baseline showed several social- emotional benefits that persisted through first grade, including reduced oppositional behavior, conflict, and problems interacting with peers. Program Quality Head Start children in the 3- and 4-year-old cohorts attended classrooms of good quality as rated by the Early Childhood Environment Rating Scale—Revised Edition. Head Start classrooms were of higher quality than classrooms in other center-based programs, including many state-funded pre- kindergarten programs (USDHHS, 2010).

Probable Implementers

Public and private preschools and day care centers

Funding

In Fiscal Year 2010, regular-appropriations funded enrollment for Head Start and Early Head Start is 904,153 children and pregnant women, at a cost of $7.2 billion per year to the federal government. In addition, the American Recovery and Reinvestment Act provided additional funds during Fiscal Years 2009 and 2010 to increase Head Start and Early Head Start enrollment by an additional 64,000

- 220 -

children and pregnant women (ACF, 2010a). Head Start and Early Head Start programs have enrolled more than 27 million children since Head Start and Early Head Start began in 1965 and 1996, respectively (National Head Start Association estimates, 2010). The Head Start program is administered by the Office of Head Start, which is part of the Administration on Children, Youth and Families (ACYF), which in turn is part of DHHS's Administration for Children and Families (ACF). There are 2,569 Head Start and Early Head Start grantees and delegate agencies that run about 20,000 Head Start and Early Head Start centers (excluding family child care homes) and 51,933 classrooms. Grantees and delegate agencies include:

• Community Action Agencies (31.4%)

• School systems (16.2%)

• Private/Public Non-Profits (non-Community Action Agencies), e.g., church or non-profit hospitals, (38.2%)

• Private/Public For-Profits, e.g., for-profit hospitals (.4%)

• Government Agencies (non-Community Action Agencies) (6.4%)

• Tribal Government or Consortium (6.9%)

• No data available (.5%)

(U.S. Department of Health and Human Services, 2010)

Implementation Detail

Program Design

Based on a community needs assessment, Head Start programs tailor their program design to suit their community's needs. Programs are generally either center-based or home-based, or use a combination of a center- and home-based approach. Programs must follow the Head Start Program Performance Standards and Other Regulations. These standards and regulations help to ensure that program quality is consistent. Having program consistency aids the evaluation of the program. Although no set curriculum exists and programs vary greatly between individual providers, the Head Start Act (P.L. 110-134) requires that Head Start programs use a comprehensive research-based curriculum that promotes school readiness and is aligned with the Head Start Child Outcomes Framework. The Head Start Child Outcomes Framework addresses the following development domains: language development, literacy, mathematics, science, creative arts, social and emotional development, approaches to learning, and physical health and development (U.S. Department of Health and Human Services, 2003). Head Start and Early Head Start programs institutionally encourage parental involvement: Parents participate on program policy councils and policy committees and, in concert with the program directors and board of directors, make decisions on the programming and services of their program. During the 2008-2009 program year, 850,000 former or current Head Start and Early Head Start parents volunteered in Head Start and Early Head Start programs (U.S. Department of Health and Human Services, 2010).

Staffing

Detailed information on required staff qualifications is provided in the Head Start Act (P.L. 110-134).

Cost

The Head Start program is a federal-to-local program and is free to participants. In Fiscal Year 2009, the cost-per-child for Head Start was approximately $7,300 (National Head Start Association estimates based on Office of Head Start data).

- 221 -

Issues to Consider

This program received a "promising" rating, because, although some outcomes for participants were better immediately after program participation, these gains were not sustained by the end of first grade. Furthermore, only a small number of the total outcome measures improved in the initial follow- up, and there were not clear patterns in the types of outcomes affected. There were also a small number of negative impacts observed. In the first (2005) wave of the DHHS study, around 14 percent of children assigned to the Head Start treatment group did not, in fact, attend Head Start. Furthermore, 18 percent of children assigned to the control group ended up attending a Head Start program. Using the first wave of DHHS data, Ludwig and Phillips (2007) applied statistical adjustments to calculate the effects of Head Start after one year of program participation on those children that actually attended the program, compared with children that did not actually attend the program. Controlling for Head Start program participation, Ludwig and Phillips found that, for cognitive outcomes that were significantly improved in the first wave of the DHHS study, the magnitude of the effects were larger. This analysis examined only cognitive impacts. In a technical appendix, the DHHS (2010) study provided statistical adjustments, calculated using a slightly different approach, of their findings to account for this "crossover" effect, and similarly found that, for all of the outcomes that were significantly improved in all waves of the study, the magnitude of the effects were greater. The negative effects found for the 3-year-old cohort on math skills were also greater in magnitude, as were the two negative effects found for the 4-year-old cohort on teacher-reported social-emotional behaviors. There is a substantial body of quasi-experimental literature on, as well as randomized controlled trials of, Head Start's effectiveness. These studies have not been included here for reasons explained in the PPN Issue Brief Head Start: What Do We Know?. In general, these studies find no effect to some positive effect; but, despite research designs intended to control for selection bias, it is impossible to guarantee that they have been successful in controlling such bias. It is important to note that findings from the DHHS study represent a national average, which conceals dramatic variation in program design and implementation at the local level. There is very limited literature describing the Head Start programmatic elements that lead to improved outcomes. Furthermore, the findings based on the randomized trial reflect the characteristics of recent Head Start programs that adhere to the DHHS program standards in effect at the time of the trial, and hence these results do not necessarily apply to Head Start programs implemented in earlier time periods.

Example Sites

Head Start programs are operational in every state, the District of Columbia, and various U.S. territories. Head Start programs can be located online at the DHHS Administration for Children and Families website: http://eclkc.ohs.acf.hhs.gov/hslc/HeadStartOffices

Contact Information

Ben Allen, Ph.D. National Head Start Association 1651 Prince St. Alexandria, VA 22314 Tel (703) 739-0875 or (800) 687-5044 Fax (703) 739-0878 http://www.nhsa.org/ Dorothy V. Harris The National Head Start Training and Technical Assistance Resource Center PaL-Tech, Inc. 1901 North Fort Myer Drive

- 222 -

http://www.promisingpractices.net/briefs/briefs_headstart.asp

http://eclkc.ohs.acf.hhs.gov/hslc/HeadStartOffices

http://www.nhsa.org/

Suite 301 Arlington, VA 22209 Tel (703) 243-0495

Available Resources

The Office of Head Start, a division of the Administration for Children and Families at the U.S. Department of Health and Human Services, is the primary source for Head Start-related information. The Office of Head Start also supports a federal information clearinghouse and archive, the Early Childhood Learning and Knowledge Center (http://eclkc.ohs.acf.hhs.gov/hslc), which provides information on Head Start products and services, conference and meeting support, publication distribution, training guides, online forums, email alert services, and marketing and outreach efforts. Another comprehensive resource for Head Start is the National Head Start Association (http://www.nhsa.org/), a private not-for-profit organization dedicated to meeting the needs of Head Start children and their families. The Association represents more than 900,000 children, 200,000 staff, and nearly 2,600 Head Start programs in the United States (NHSA website, 2005). The NHSA supports the Head Start community by providing training and professional development to Head Start staff; undertaking and disseminating research, information, and resources related to program delivery; and advocating for policies that strengthen Head Start services for children and families. Detailed information about grants and funding can be obtained at: http://www.acf.hhs.gov/programs/hsb/grant/index.htm.

Bibliography

ACF — See U.S. Department of Health and Human Services, Administration for Children and Families. Code of Federal Regulations, Title 45, Part 1301-1311, Head Start Program Performance Standards and Other Regulations, 2006. As of March 12, 2010: http://eclkc.ohs.acf.hhs.gov/hslc/Program%20Design%20and%20Management/Head%20Start%20Re quirements/Head%20Start%20Requirements GAO — See U.S. Government Accountability Office. Ludwig, Jens, and Deborah A. Phillips, "The Benefits and Costs of Head Start," Social Policy Report (Society for Research on Child Development), Vol. 21, No. 3, 2007. Public Law 110-134, Improving Head Start for School Readiness Act of 2007, December 12, 2007. As of March 12, 2010: http://eclkc.ohs.acf.hhs.gov/hslc/Program%20Design%20and%20Management/Head%20Start%20Re quirements/Head%20Start%20Act/headstartact.html#648A U.S. Department of Health and Human Services, 2008-2009 Head Start Program Information Report (PIR) Survey Summary Report—National Level, Washington, DC, 2010. U.S. Department of Health and Human Services, Administration for Children and Families, Head Start Program Fact Sheet Fiscal Year 2008, 2008. As of March 12, 2010: http://eclkc.ohs.acf.hhs.gov/hslc/About%20Head%20Start/dHeadStartProgr.htm U.S. Department of Health and Human Services, Administration for Children and Families, FY 2011 Justification of Estimates for Appropriations Committees, 2010a. As of March 12, 2010: http://www.acf.hhs.gov/programs/olab/budget/2011/cj2011.html U.S. Department of Health and Human Services, Administration for Children and Families, Head Start Impact Study: Final Report, Washington, DC, 2010b. As of March 12, 2010: http://www.acf.hhs.gov/programs/opre/hs/impact_study/reports/impact_study/hs_impact_study_final .pdf

- 223 -

http://eclkc.ohs.acf.hhs.gov/hslc

http://www.nhsa.org/

http://www.acf.hhs.gov/programs/hsb/grant/index.htm

http://eclkc.ohs.acf.hhs.gov/hslc/Program%20Design%20and%20Management/Head%20Start%20Requirements/Head%20Start%20Requirements

http://eclkc.ohs.acf.hhs.gov/hslc/Program%20Design%20and%20Management/Head%20Start%20Requirements/Head%20Start%20Act/headstartact.html%23648A

http://eclkc.ohs.acf.hhs.gov/hslc/About%20Head%20Start/dHeadStartProgr.htm

http://www.acf.hhs.gov/programs/olab/budget/2011/cj2011.html

http://www.acf.hhs.gov/programs/opre/hs/impact_study/reports/impact_study/hs_impact_study_final.pdf

http://www.nhsa.org/

U.S. Department of Health and Human Services, Administration on Children, Youth and Families, Head Start Bureau, The Head Start Leaders Guide to Positive Child Outcomes. Washington, DC, 2003. U.S. Government Accountability Office, Head Start Programs: Participant Characteristics, Services, and Funding, Washington DC, GAO/HEHS-98-65, 1998. U.S. Government Accountability Office, Head Start: Curriculum Use and Individual Child Assessment in Cognitive and Language Development, Washington, DC, GAO-03-1049, 2003.

Last Reviewed

March 2010

Healthy Families New York (HFNY)

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children not experiencing physical, psychological or emotional abuse Babies born weighing more than 5.5 pounds and improving outcomes for low birth weight babies

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Home Visiting Type of Service Family Support Health Care Services Health Education Parent Education Type of Outcome Addressed Child Abuse and Neglect Physical Health Substance Use and Dependence

Evidence Level Proven

Program Overview

Based on the national Healthy Families America (HFA) critical program elements, Healthy Families New York (HFNY) is a community-based prevention program that seeks to improve the health and well- being of children at risk for abuse and neglect by providing intensive home visitation services. The target population consists of expectant parents and parents with an infant less than three months old who are considered to be at high risk for child abuse and neglect. Specially trained paraprofessionals are assigned to the participating families to deliver home visitation services until the child reaches five or is enrolled in Head Start or kindergarten. Home visitors provide families with support, education, and referrals to community services aimed at addressing the following goals: 1) promoting positive parenting skills and parent-child interaction; 2) preventing child abuse and neglect; 3) ensuring optimal prenatal care and child health and development; and 4) increasing parents' self-sufficiency.

- 224 -

HFNY began operation in 1995 as an initiative of the New York State Office of Children and Family Services in collaboration with the New York State Department of Health, Prevent Child Abuse New York, and the Center for Human Services Research at the University at Albany. As of February 2011, HFNY was operating in 37 sites throughout New York State.

Program Participants

Through community health and social service agencies, and hospitals, the HFNY program screens expectant parents and parents with an infant less than three months old for risk factors that are associated with child abuse, neglect, and poor developmental outcomes, including, but not limited to: single parenthood, teen pregnancy, poverty, poor education, unstable housing, substance abuse, and mental health problems. Parents who screen positive are referred to the HFNY program, which conducts an assessment interview to determine their eligibility using the Family Stress Checklist, a tool that measures the mother's risk of abusing or neglecting her children. Parents who score above a pre-determined cutoff on the Family Stress Checklist are offered the opportunity to receive home visitation services.

Evaluation Methods

The researchers evaluated the HFNY program in a sample of 1,173 at-risk women who were pregnant or who had an infant less than three months old (DuMont et al., 2008). Families were selected for the evaluation based on the same criteria used to determine eligibility for HFNY. Women who were pregnant or who had an infant less than three months old in New York's Erie, Rensselaer, and Ulster counties were screened and then assessed for risk of child abuse and neglect. A total of 1,254 women met the eligibility criteria for the program and agreed to participate in the study. Among those women, 1,173 (94 percent) completed an intake interview and became study participants. The sample is comparable to the population of families enrolled in HFNY programs across the state on a range of measures such as demographics, employment, prior child abuse and neglect reports, substance use, and depression. Just over half of the participants were first-time mothers. Nine percent of the sample had substantiated child abuse or neglect reports prior to intake, and 20 percent had been physically abused in the previous year by a partner or spouse. About two out of every five participants also scored above the clinically relevant cutoff point on a depression scale. A total of 579 women were randomly assigned to the treatment group (i.e., the HFNY group) and 594 were randomly assigned to the control group. Nearly two-thirds of the women were pregnant at the time they were assigned to the treatment or control group. The control group was given information and referrals to other appropriate services available in the community. Baseline statistics on the measures mentioned above showed no significant differences between the HFNY group and the control group, one indication that the random assignment was successful. DuMont et al. (2008) evaluated the effects of HFNY on child abuse and neglect during the first two years of life. Of the 1,173 participants, 1,060 (90 percent of the sample) completed the Year 1 follow- up interview, and 992 (85 percent of the sample) completed the Year 2 follow-up interview. To maintain data quality, interviewers were independent of the HFNY program and were not informed of a participant's group assignment. Attrition analyses showed no significant differences between the HFNY group and the control group at the time of the Year 1 and Year 2 interviews. However, HFNY and control group members who remained in the study (regardless of whether they continued to participate in the HFNY program) were more likely than those who dropped out of the study to be first-time mothers, but less likely to have been randomly assigned at a gestational age of 30 weeks or less. In addition, at Year 1, rates of attrition from the study were slightly lower for non-Latina white women than for Latina women and non-Latina African-American women. However, by Year 2 these differences had disappeared. No other significant differences were found between those who continued in the study and those lost to attrition. DuMont et al. (2008) conducted analysis on the Year 1 and Year 2 follow-up samples, using statistical adjustments to control for demographic and other risk-related factors. The researchers examined the outcomes of the entire sample and also two subgroups: "a high prevention opportunity" (HPO) subgroup (i.e., first-time mothers under age 19 who were randomly assigned at a gestational age of

- 225 -

30 weeks or less, about 15 percent of the sample) and a "psychologically vulnerable" subgroup: (i.e., women with a low sense of mastery and high levels of depressive symptoms at baseline, about 10 percent of the sample). Both subgroups included women in the HFNY group and the control group. The study reported retention rates for the home visiting program consistent with those for other HFA programs. Specifically, at baseline, about 90 percent of the women who were assigned to the intervention group enrolled in the HFNY program, and only one-half and one-third of the HFNY participants remained in the program by Year 1 and Year 2, respectively. However, to the extent possible, those who dropped out of the HFNY program were retained in the intervention group of the study. Lee and colleagues (2009) reported on HFNY's ability to impact low birth weight. The researchers obtained birth certificate data from the New York State Department of Health for mothers who were randomly assigned prenatally and provided informed consent. The match was performed using the names and dates of birth of the child and the mother, and was successful for 99 percent of the cases searched. Analyses were conducted on mothers who had a single birth and were randomized at a gestational age of 30 weeks or less in order to allow enough time prior to the target child's birth for participants to benefit from home visiting services. Rodriguez et al. (2010) examined the effects of HFNY on the promotion of parenting competencies and the prevention of harsh parenting at the Year 3 follow-up, which included videotaped observational assessments of mother and child interactions. At two of the study sites, all of the mothers who met eligibility criteria were asked to participate in the Year 3 observational assessments. At the largest study site, a random sample of eligible mothers was asked to participate in the Year 3 observations. In order to be eligible for these observational assessments, the mothers had to have custody of their child, live within a two-hour driving distance of the interview location, and have completed the Year 2 interview. Of 643 selected mother-child pairs, 522 (81 percent) participated in the observations. This sample was comparable to the overall sample on most baseline characteristics, with the exception of a few significant differences between mothers who participated in the observational assessments and mothers who did not participate, in part due to the substantially reduced representation of participants from the largest (inner-city) site. Mothers who participated in the observations were less likely to be African-American, less likely to be Latina, more likely to be white, less likely to have been receiving welfare at baseline, and more likely to have healthy parenting attitudes at baseline. Mothers in the HFNY group were more likely to receive welfare at baseline, and more likely to report having a partner or spouse at baseline, when compared with the control mothers. These differences were between HFNY and control mothers were adjusted for using statistical controls. The observational assessments were administered in participants’ homes by trained interviewers, who were independent of the HFNY program and blind to group assignment. Mothers and children were observed and videotaped while they interacted on three tasks: Puzzle Problem Solving, Delay of Gratification, and Cleanup. In the puzzle task, the mother is asked to work with the child on two puzzles, one of which is difficult for the child’s age, and the mother is asked to provide instruction when necessary. In the delay task, a snack is placed in front of the child and the mother is asked to fill out a questionnaire while also making sure the child does not eat or touch the snack. In the cleanup task, the mother and child play with toys for a time and then the mother is asked to have the child clean up the toys while she holds the bag. The data were also analyzed for the "high prevention opportunity" (HPO) subgroup, which included first-time mothers who were under the age of 19 and were randomized at a gestational age of 30 weeks or less. DuMont et al. (2010) reported the findings from the Year 7 interview, including prevention of child maltreatment and risks for delinquency. The Year 7 follow-up included in-home interviews of mothers and the target children. Mothers were eligible to participate in this interview if both they and the target child were still alive, and if women in the control group had not received any HFNY services between the time of randomization and two weeks prior to the Year 7 interview. Target children also had to live within driving distance of an interviewer and had to be under their mother’s care and custody. Of the 1,173 original participants, 942 (80 percent) completed the Year 7 follow-up interview. Interviews were also conducted with 800 (71 percent) of the 1,128 children who were eligible for the Year 7 interview. Respondents and non-respondents at Year 7 were similar on the majority of baseline characteristics, with a few exceptions. Respondents were more likely than non-respondents to be

- 226 -

African-American (48 percent vs. 36 percent), less likely to be Latina (16 percent vs. 28 percent), less likely to report moving in the prior year (54 percent vs. 63 percent), and presented with higher risk for child abuse or neglect initially as assessed by the Kempe Family Stress Checklist (5.77 vs. 5.39). The HFNY and control groups within the Year 7 mother and child samples were comparable on most attributes at baseline, but a few significant differences were found. Significantly more mothers interviewed had target children who were male in the HFNY group than in the control group (57 percent vs. 50 percent), mothers in the HFNY group had significantly lower annual earnings ($4,887 vs. $6,603) and were less likely to have a high school degree or equivalent (44 percent vs. 51 percent), and HFNY mothers were assessed with a higher level of risk for child abuse and neglect based on the Kempe Family Stress Checklist (5.86 vs. 5.68). Mothers of interviewed children in the HFNY group had significantly lower earnings ($4,733 vs. $6,190) and were less likely to have a high school degree or equivalent (44 percent vs. 51 percent). These differences between the HFNY and control group participants were adjusted for using statistical matching controls.

Key Evaluation Findings

The study has reported overall positive program findings in terms of childbirth outcomes (Lee et al., 2009) and parenting practices (DuMont et al., 2008; Rodriguez et al., 2010; DuMont et al., 2011). Compared with their counterparts in the control group, HFNY mothers were less likely to deliver low birth weight babies, less likely to engage in abusive, neglectful, or harsh parenting practices, and more likely to use positive parenting skills. According to standard practice in the public health field, Lee and colleagues (2009) considered babies born weighing less than 2,500 grams to be low birth weight. Focusing on women who had a single birth and were randomized into the study at a gestational age of 30 weeks or less, Lee et al. (2009) reported significant findings based on birth certificate data provided by the New York State Department of Health:

• HFNY group mothers were only about half as likely as control group mothers to deliver low birth weight babies (5.1 percent vs. 9.8 percent). Rates were measured while controlling for other factors including race/ethnicity, program site, welfare receipt, age, and smoking during pregnancy.

• The program was particularly effective in reducing low birth weight among African-American and Latina mothers, groups that persistently experience high levels of poor birth outcomes. For example, African-American mothers who were assigned to the HFNY group were 70 percent less likely than African-American mothers in the control group to deliver low birth weight babies (3.1 percent vs. 10.2 percent).

• The earlier in their pregnancies that women were offered HFNY, the greater the impact of the program on low birth weight. Among women who entered the study at a gestational age of 16 weeks or less, the rate of low birth weight experienced by the HFNY group was one-quarter as high as the rate for the control group (3.6 percent vs. 14.1 percent)

DuMont et al. (2008) and Mitchell-Herzfeld et al. (2005) used two measures to assess the effects of HFNY on child abuse and neglect from birth to age 2: substantiated (or confirmed) Child Protective Services (CPS) reports and parents' self-reported parenting behaviors as measured by the revised Conflict Tactics Scale (CTS). They found the following findings at the Year 1 interview:

• Compared with mothers in the control group, HFNY participants reported committing fewer acts of very severe physical abuse (0.01 vs 0.08), minor physical aggression (2.40 vs. 3.46), harsh parenting in the past week (1.21 vs. 1.81), and psychological aggression against their children (3.34 vs. 4.74) (DuMont et al., 2008).

• Among mothers in the psychologically vulnerable subgroup, the frequency of psychological aggression was significantly lower for the HFNY group (1.95) than for the control group (8.57) (DuMont et al., 2008).

• No significant differences were observed between the HFNY and control groups with respect to the rate or average number of substantiated CPS abuse and neglect reports. The evaluators

- 227 -

note that the discrepancy in the findings for official indicators of abuse and neglect and self- reported measures may be due to greater surveillance of HFNY parents by home visitors and the providers to which they refer families. HFNY parents who admitted to having committed acts of serious abuse and neglect were nearly twice as likely to have a CPS report than were control parents who self-reported serious abuse or neglect, suggesting that actual incidents of abuse and neglect committed by HFNY parents were more likely to be detected and reported to CPS than were those committed by control parents (Mitchell-Herzfeld et al., 2005).

At the Year 2 interview, DuMont et al. (2008) found the following:

• No program effects on the prevalence or frequency of substantiated CPS reports were detected at Year 2.

• The average number of reported acts of severe physical abuse in the past year was lower among mothers in the HFNY program than mothers not participating in the program (0.01 vs. 0.04).

• Among mothers who were "psychologically vulnerable" (mothers who had a low sense of mastery and high levels of depressive symptoms), mothers in the HFNY program were less likely to report acts of serious abuse or neglect (5 percent vs. 19 percent). Among psychologically vulnerable mothers, the average number of self-reported incidents of serious abuse and neglect in Year 2 was significantly lower for the HFNY group (0.02) than for the control group (0.62).

• Among mothers in the "high prevention opportunity" (HPO) subgroup (first-time mothers under age 19 who were randomized at a gestational age of 30 weeks or less), mothers in the HFNY program were less likely to report acts of minor physical aggression in the past year (51 percent vs. 70 percent), and harsh parenting behaviors in the past week (41 percent vs. 62 percent).

The following is a summary of the Year 3 findings from Rodriguez et al. (2010):

• Significantly more HFNY mothers used positive parenting strategies (e.g., maternal responsivity and cognitive engagement) during all three tasks (Puzzle: 95.5 percent vs. 92.8 percent, Delay: 17.2 percent vs. 10.6 percent, Cleanup: 85.3 percent vs. 78.3 percent).

• Among the overall sample, there were no significant differences between mothers who participated in the program and mothers who did not participate in the program on harsh parenting or role-reversed parenting.

• Mothers in the "high prevention opportunity" (HPO) subgroup who participated in the program were significantly less likely to use harsh parenting during the Puzzle and Delay tasks (Puzzle: 5.3 percent vs. 21.5 percent, Delay: 5.3 percent vs. 23.8 percent) than HPO subgroup mothers who did not participate in the program.

Data were analyzed at Year 7 for mothers participating in the program and mothers not participating in the program. The data were also analyzed for two subgroups: the "recurrence reduction opportunity" (RRO) subgroup and the HPO subgroup. The RRO group included women involved in a substantiated CPS report within five years prior to randomization and the HPO group, again, included first-time mothers who were under the age of 19 and were randomized at a gestational age of 30 weeks or less. At the Year 7 interview, DuMont et al (2010) found the following:

• For the sample as a whole, there was no difference between intervention and control parents in the rate or cumulative number of CPS abuse or neglect reports.

• Mothers in the HFNY group reported using serious physical abuse less frequently in the past year than mothers in the control group (0.03 vs. 0.15).

- 228 -

• Mothers in the HFNY group reported using non-violent discipline strategies more frequently in the past year than mothers in the control group (an average of 49 times vs. 45 times) and reported higher rates of using non-violent discipline (100 percent vs. 98.6 percent).

• Children in the HFNY group were less likely to report that their mothers engaged in minor physical aggression against them than children in the control group (70.1 percent vs. 77.2 percent).

• Significantly more children who took part in the HFNY program participated in gifted programs within their schools than children in the control group (5.38 percent vs. 1.99 percent).

• Intervention children were less likely to report skipping school, but, according to maternal reports, there was no difference in skipping school.

• There were no significant differences in problem behavior, socio-emotional difficulties, or self- regulation.

• Among children in the HPO subgroup, children in the HFNY group were significantly less likely to score below average on the PPVT-IV receptive vocabulary assessment (59.40 percent vs. 77.61 percent).

• Compared with mothers in the RRO subgroup who did not participate in the program, mothers in the RRO subgroup who participated in the program had lower rates of initiation of child welfare cases for preventative, protective, or placement services (38.02 percent vs. 60.02 percent).

• HFNY mothers in the RRO subgroup had a smaller number of total confirmed CPS reports for mothers than the control subjects (0.8 vs. 1.6).

• No significant differences were found among mothers in the HPO subgroup on confirmed CPS reports of abuse or neglect, reports of physical abuse, or initiation of preventative, protective, and placement services.

Probable Implementers

Public health and social welfare services

Funding

The project is funded through the state budget, and the allocation for the program was $23.3 million in 2010. The cost per family ranges from $3,500 to $4,500 per year, with slightly higher costs in New York City.

Implementation Detail

Program Design

HFNY is a variant of the HFA home visiting model, and this variant is shown to have significant and positive impacts. HFNY participants may receive home visiting services until the child reaches the age of five or is enrolled in Head Start or kindergarten. Families are served at different service levels that correspond to different frequencies of home visits based on families' needs. Home visits are scheduled one or more times per week during pregnancy (Level 1), and families usually remain on Level 1 until the child is at least six months old. As families progress through the service levels, home visits occur on a diminishing schedule, from biweekly (Level 2), to monthly (Level 3), and then quarterly (Level 4). The content of home visits is individualized and culturally appropriate to address the unique needs of each family. HFNY is managed by the New York State Office of Children and Family Services, which contracts with public and community-based agencies to provide home visitation services. Funded programs are required to follow HFA and HFNY standards and participate in the HFA credentialing process.

- 229 -

Curriculum

All new home visitors attend a one-week core training program provided by approved in-state HFA trainers. They receive training on parent-child interaction, child development, strength-based service delivery, and on such a range of topics such as domestic violence, substance abuse issues, abuse and neglect, and well-baby care. HFNY supervisors receive an additional four days of training on their role in promoting quality services. New home visitors are mentored by experienced home visitors. Supervisors meet with each home visitor for at least 1.5 hours every week and observe one home visit per quarter.

Staffing

The program selects paraprofessionals from the community being served to be home visitors. Personal attributes such as warmth, the ability to establish trusting relationships, the ability to work effectively with children and families, and nonjudgmental attitudes are the primary selection criteria. Although post-secondary education is not a requirement, many home visitors (43 percent) have attended college, and about one-third (34 percent) are college graduates.

Issues to Consider

The HFNY program received a "proven" rating. The study utilized random assignment with a fairly large sample size of 1,173 women. To maintain data quality, interviewers were independent of the HFNY program and were not informed of a participant's group assignment. Parents who were randomly assigned to the HFNY program were kept in the analysis, regardless of whether or not they actually received any home visiting services. DuMont et al. (2008) reported a 10 percent attrition rate at Year 1 and a 15 percent attrition rate at Year 2; 2 percent of the study sample completed the Year 2 but not the Year 1 interview. DuMont et al. (2010) reported a 20 percent attrition rate at Year 7. At Years 1, 2, and 7, attrition analyses indicated no significant differences between mothers in the control and HFNY groups with respect to attrition. However, the analyses at all three years indicated significant differences in baseline measures between parents lost to attrition and those retained in the sample. At Year 1 differences were found on race and parenting attitudes, at Year 2 differences were found on randomization before 30 weeks of gestational age and being a first time mother, and at Year 7 differences were found on race, report of moving in the past year, and initial risk of child abuse or neglect. At Year 3, a reduced sample of mothers were observed interacting with their children on a series of semi-structured tasks. All eligible mothers who were initially interviewed at two study sites were asked to participate in the observational assessments, and at one study site a random subsample of eligible mothers were selected. There were significant differences between mothers who participated in the Year 3 interview and mothers who did not participate. The study found significant and positive program effects on a range of outcomes, including child abuse and child neglect and childbirth outcomes. Although, some of the program effects found in Year 1 did not persist into Year 2, the reduction in the average number of maternal-reported acts of severe physical abuse found at Year 1 and Year 2 was sustained at Year 7. Dozens of evaluations of HFA have been conducted, but most lacked a comparison group or used quasi-experimental designs, and most failed to find significant positive effects. Possible reasons for the evaluations not finding significant effects could be the relatively small sample sizes and a low base rate for child abuse and neglect reports. The studies summarized here are the only studies of an HFA model that have met the Promising Practices Network (PPN) evidence criteria, including such factors as study design, effect size, and statistical significance. HFNY is very similar to many other HFA programs in that it satisfies all the critical elements required by the HFA model, including participation eligibility, service content, home visitor selection and training, and service monitoring and supervision. HFNY differs from other programs only in some program details that are allowed under the HFA structure. Previous studies have shown larger program effects from the Nurse Family Partnership (NFP) home visiting model than the HFA model. One hypothesis forwarded to explain the differences focuses on the training of the provider of the home visitation services —the NFP model uses nurse home visitors,

- 230 -

whereas the HFA model employs paraprofessional home visitors. However, DuMont et al. (2008) found more pronounced program effects among the first-time mothers under age 19 than for the entire sample, and this subgroup of first-time mothers resembled the type of participants typically served by NFP programs. Therefore, the HFNY evaluation proposed that the characteristics of recipients might be a key factor in explaining the differences in the program effects between the NFP and HFA model. In fact, consistent with these findings from the HFNY subgroup analysis, the NFP data have shown that higher-risk mothers also benefit more from NFP services (see the discussion in Karoly et al., 2001). Further studies that examine this hypothesis would be valuable. A cost-benefit analysis of HFNY over the first seven years of life found that the program did not pay for itself through generation of government savings in public assistance and food stamp payments, Medicaid-funded births, CPS investigations, preventative services, and foster care placements. For the sample as a whole, for every dollar spent on the program, $0.15 were returned in savings to the government due to decreased use of government supported programs. Among women in the recurrence reduction opportunity (RRO) subgroup, the program generated a savings of $3.16 for every dollar spent on the program, and among women in the high prevention opportunity (HPO) subgroup, the recovery was $0.25 in savings for every dollar spent on the program. One of the members of the study team, Susan Mitchell-Herzfeld, is on the PPN Board of Advisors.

Example Sites

Erie, Rensselaer, and Ulster Counties in New York State

Contact Information

Bernadette Johnson Program Coordinator Healthy Families New York New York State Office of Children and Family Services 52 Washington Street, 334N Rensselaer, NY 12144 (518) 402-6770 Fax: (518) 402-6824 Email: bernadette.johnson@ocfs.state.ny.us

Available Resources

Healthy Families New York website: http://www.healthyfamiliesnewyork.org/ New York State Office of Children and Family Services website: http://www.ocfs.state.ny.us Center for Human Services Research, University at Albany website: http://www.albany.edu/chsr

Bibliography

DuMont, K., K. Kirkland, S. Mitchell-Herzfeld, S. Ehrhard-Dietzel, M. Rodriguez, E. Lee, C. Layne, and R. Greene, Final Report: A Randomized Trial of Healthy Families New York (HFNY): Does Home Visiting Prevent Child Maltreatment? 2010. As of February 2011: http://www.ncjrs.gov/pdffiles1/nij/grants/232945.pdf DuMont, K., S. Mitchell-Herzfeld, R. Greene, E. Lee, A. Lowenfels, M. Rodriguez, and V. Dorabawila, "Healthy Families New York (HFNY) Randomized Trial: Effects on Early Child Abuse and Neglect," Child

- 231 -

mailto:bernadette.johnson@ocfs.state.ny.us

http://www.healthyfamiliesnewyork.org/

http://www.ocfs.state.ny.us/

http://www.albany.edu/chsr

http://www.ncjrs.gov/pdffiles1/nij/grants/232945.pdf

Abuse and Neglect, Vol. 32, 2008, pp. 295-315. Lee, E., S. Mitchell-Herzfeld, A. Lowenfels, R. Greene, V. Dorabawila, and K. DuMont, "Reducing Low Birth Weight Through Home Visitation: A Randomized Controlled Trial," American Journal of Preventive Medicine, Vol. 36, No. 2, 2009. Mitchell-Herzfeld, S., C. Izzo, R. Greene, E. Lee, and A. Lowenfels, Evolution of Health Families New York (HFNY): First Year Program Impacts, 2005. As of February 2011: http://www.ocfs.state.ny.us/main/reports/#hfny Rodriguez, M. L., K. Dumont, S. Mitchell-Herzfeld, N. J. Walden, and R. Greene, "Effects of Healthy Families New York on the Promotion of Maternal Parenting Competencies and the Prevention of Harsh Parenting," Child Abuse and Neglect, Vol. 34, 2010, pp. 711-723.

Last Reviewed

March 2011

Healthy Start

Program Info

Outcome Areas Healthy and Safe Children

Indicators Babies born weighing more than 5.5 pounds and improving outcomes for low birth weight babies

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Community-Based Service Provider Health Care Provider Type of Service Family Support Health Care Services Health Education Parent Education Type of Outcome Addressed Physical Health Substance Use and Dependence

Evidence Level Promising

Program Overview

The Healthy Start Initiative was established in 1991 by the Health Resources and Services Administration (HRSA) of the U.S. Public Health Service. The initiative's primary purpose was to reduce infant mortality by 50 percent and generally improve maternal and infant health in at-risk communities.

- 232 -

http://www.ocfs.state.ny.us/main/reports/%23hfny

Originally, 15 demonstration programs were funded for a five-year period; they included 13 urban and 2 rural programs located around the country (e.g., in the Northeast, South, Midwest, and West). Congress subsequently continued the funding for those programs and other additional programs. As of fiscal year 2002, 96 Healthy Start programs were being funded. The programs are currently administered through the Division of Perinatal Systems and Women's Health, which is part of the Maternal and Child Health Bureau within HRSA.

Each grantee (i.e., participating service area) has a great deal of flexibility in designing its specific program, although several common models have emerged. Certain elements were required of the programs, including a focus on reducing infant mortality, inclusion of the local community in program planning, assessment of local needs, efforts to increase public awareness, implementation of an infant mortality review, development of a package of innovative health and social services for pregnant women and for infants, and evaluation of the initiative.

Program Participants

The grantees are typically city, county, or state health departments. The program is designed to serve pregnant women (particularly women at high risk of poor pregnancy outcomes) and infants, with each service area determining its specific program recipients.

Evaluation Methods

Mathematica Policy Research, Inc. conducted a national evaluation of Healthy Start focusing on the 15 demonstration programs (see Devaney et al., 2000). Two comparison sites were selected for all but one of the demonstration programs -- a single comparison site was identified for the final program. The comparison sites were selected for their similarity in terms of race and ethnic composition, infant mortality rate, and trends in the infant mortality rate between 1984 and 1992 (the years leading up to implementation of Healthy Start). The number of clients served in the demonstration program sites ranged from a low of 1,000 in the Pee Dee region of South Carolina to a high of 7,000 in Cleveland, Ohio. The other sites were in Baltimore, Birmingham, Boston, Chicago, Detroit, the District of Columbia, New Orleans, New York City, Oakland, Philadelphia, Pittsburgh, cities in the Northern Plains states (including 19 American Indian tribal organizations in Iowa, Nebraska, North Dakota and South Dakota), and in several cities in Lake County in Northwest Indiana (specifically East Chicago, Gary, Hammond, and Lake Station). The evaluation examined both implementation issues and program outcomes. To study implementation, the researchers (Devaney et al.) conducted site visits, telephone interviews, focus groups, document review, observations, and home visits, as follows:

• The site visits involved interviews with program staff, service providers, and consortia members. Researchers used semi-structured protocols to gather comparable information across the programs.

• Telephone interviews were conducted with the grantees as follow-ons to the site visits and included project directors, consortium members, and outreach and case management staff. These interviews addressed any changes in the community affecting their projects, changes they had made to their organizational structures, and what progress they had made in implementing the program. The researchers also solicited thoughts on what lessons the grantees had learned and how to sustain the program past the life of the grant.

• The focus groups were conducted with Healthy Start clients and providers and addressed a range of topics, such as the quality of the clients' lives, the health and well-being of their children, clients' experiences during pregnancy, how they learned about Healthy Start, and why they became involved in Healthy Start and about their experiences with the program.

• The researchers reviewed a range of documents, including documents on case management services received by Healthy Start clients (e.g., number of visits, type of help received, if

- 233 -

mother's health was checked, if child's health was checked, ranking of how helpful the services were) and health education materials.

• The researchers observed activities such as consortia meetings and health education classes to assess their content and quality.

• Researchers also accompanied program staff on home visits to observe how staff members interacted with clients.

The outcomes studied included prenatal care utilization, pre-term birth rate, low- and very-low-birth- weight rates, and infant mortality rate. The researchers compared infant birth and death rates for Healthy Start project areas with matched comparison sites from 1984 to 1996. They used several measures to assess various dimensions of prenatal care. One of those instruments is called the "Kotelchuck index," which is used to rate prenatal care depending on when it is initiated, how often it occurs, and other factors. The five categories in the Kotelchuck index are: no care, inadequate care, intermediate care, adequate care, and adequate-plus care. Vital statistics data were collected for the target population in each program area and include information on maternal and paternal characteristics, the timing and extent of prenatal care, pregnancy history, basic demographic characteristics, and birth outcomes. These data helped the researchers gain a better understanding of the Healthy Start target population. The authors (Devaney et al.) offer two caveats -- the study did not involve random assignment, and the comparison areas likely had interventions similar to those of Healthy Start; therefore, the researchers were probably detecting differences in impact between interventions, not differences in outcomes between an intervention and no intervention.

Key Evaluation Findings

Devaney et al. (2000) found the following: Outcomes

• Three Healthy Start program sites had significantly lower rates of low-birth-weight babies than their comparison sites. Similarly, three Healthy Start program sites had significantly lower rates of very-low-birth-weight babies.

• The Healthy Start program was associated with a significantly lower pre-term birth rate in four program sites (with decreases ranging from 1.3 percent to 2.9 percent); the differences in the pre-term birth rate in the other (11) sites were not statistically significant.

• The Healthy Start program areas rated significantly higher on the Kotelcheck index than the comparison areas on several dimensions:

o In 8 of the 15 program sites, a higher percentage of women in Healthy Start received adequate or better prenatal care as compared with women in the comparison areas. One Healthy Start program area was higher than the comparison areas on the Kotelcheck index, but not significantly so. The other 6 program sites had less positive outcomes; the comparison sites had higher percentages of women who received adequate or better prenatal care than did those 6 sites.

o In 4 of the 15 program sites, a higher percentage of women in Healthy Start experienced adequate initiation of prenatal care-with adequate initiation defined as the first prenatal care visit occurring by the fourth month of pregnancy. In the remaining programs, the percentage of Healthy Start women experiencing adequate initiation of prenatal care was lower but not significantly lower in 6 programs sites, was the same in two program sites, and was higher but not significantly so in 3 program sites.

o In 9 of the 15 program sites, a larger percentage of women in Healthy Start had an adequate or better number of prenatal care visits than women in the comparison areas. In the other 6 sites, a higher percentage of women in the comparison group had an adequate or higher number of prenatal care visits.

- 234 -

Implementation

• Implementation generally took longer than expected.

• Programs were generally successful in developing case management programs (case management included initial contact or outreach; intake; assessment, care planning, and referrals; and ongoing contact and tracking).

• Community involvement was considered a key goal of Healthy Start. While all of the programs did encourage community involvement, it was very difficult to secure.

Probable Implementers

City, county, or state health departments serving at-risk communities

Funding

Grants from HRSA for the demonstration programs ranged from $13.8 million (Birmingham) to $30 million (New York). Funding continued, but was reduced, following the demonstration phase.

Implementation Detail

Program Design

Although grantees have flexibility in how they structure their programs, there are nine typical intervention models: community-based consortia, outreach and client recruitment, care coordination/case management, family resource centers, enhanced clinical services, risk prevention and reduction, facilitating services, training and education, and adolescent programs. The researchers documented several characteristics of successful programs:

• Strong program organization and administration, with stable program leadership, is associated with better program implementation and improved outcomes.

• Programs that focus on service coordination, with close links to the existing clinical care system, appear to be more successful than others.

• Community involvement through the employment of community residents is associated with improved outcomes in some but not all programs.

Staffing

The Healthy Start Initiative described the roles and responsibilities of staff, but not their actual titles or positions. For example, substance abuse counseling and health education can be conducted by staff or contracted out. The case manager positions can be filled by the following types of staff: lay workers (community residents without professional training, but who received on-the-job training or training from Healthy Start), social workers or similarly trained professionals, and public health nurses. The Healthy Start grant money can also be used to hire clinic staff, including obstetrics/gynecology providers, pediatricians, nurses (particularly nurse midwives and nurse practitioners), nutritionists, phlebotomists, nursing assistants, and clerks.

Curriculum

No set or prescribed curriculum.

Issues to Consider

This program received a "promising" rating. While the researchers used a matched comparison design across many sites over several years, the methodology had some limitations and the findings were

- 235 -

mixed. Three program sites did experience statistically significant decreases in very-low-birth-weight babies. However, no single program consistently showed improvements across all of the outcome areas, and it is not clear which program or implementation features led to which outcomes. The lack of understanding about which features lead to which outcomes makes it difficult to replicate the successful features of the demonstration programs. In addition, the comparison sites likely had interventions similar to Healthy Start previously in place that may affect one’s assessment of the influence of Healthy Start compared with doing nothing at all (i.e., the Healthy Start results might have been more positive).

Example Sites

Researchers (Devaney et al.) evaluated the original 15 demonstration programs. Since then, the number of programs has increased to 96. (For more information, see the National Healthy Start Association Web site at www.healthystartassoc.org.)

Contact Information

Karen Hench Director, Division of Perinatal Systems and Women's Health Health Resources and Services Administration 5600 Fishers Ln., Room 11A-55 Rockville, MD 20857 301-443-0543 fax: 301-594-0186 email: khench@hrsa.gov

Available Resources

Harrington, M., B. Foot, and E. Closter, Using Health Education to Reduce Infant Mortality: The Healthy Start Experience, Washington, D.C.: Mathematica Policy Research, Inc., September 1998. Howell, E. M, B. Devaney, M. McCormick, and T. T. Raykovich, "Back to the Future: Community Involvement in the Healthy Start Program," Journal of Health Politics, Policy and Law, Vol. 23, April 1998, pp. 291-317.

Bibliography

Devaney, B., E. M. Howell, M. McCormick, and L. Moreno, Reducing Infant Mortality: Lessons Learned from Healthy Start, Final Report, Princeton, N.J.: Mathematica Policy Research, Inc., July 2000.

Last Reviewed

September 2008

Healthy Steps for Young Children

Program Info

Outcome Areas Healthy and Safe Children Children Ready for School

- 236 -

mailto:khench@hrsa.gov

http://www.healthystartassoc.org

Indicators Children ages 0 to 5 exhibiting age-appropriate mental and physical development Children experiencing good physical health

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Health Care Provider Home Visiting Type of Service Health Care Services Health Education Parent Education Type of Outcome Addressed Physical Health

Evidence Level Promising

Program Overview

Healthy Steps for Young Children is a national initiative that incorporates developmental specialists into primary care pediatric visits with the aim of meeting families' needs related to their young children's development and behavior. The program also aims to improve the relationships between parents and children, between parents and pediatric practices, and between pediatric practice members. The program targets families with newborns between birth and four weeks. Participating families receive up to six home visits and extended developmental services provided by a Healthy Steps Specialist (HSS) from birth to age three. The HSSs participate in the well-child office visits with the child's health care provider, answer parents' questions about child development, assess the children's developmental status, and identify family health risks. Participating families are also provided with written materials on preventative safety measures and community resources, and the families are given access to a child development telephone information line staffed by an HSS and parent groups facilitated by an HSS.

Program Participants

Currently, more than 50 sites nationwide, including pediatric offices and family practices, provide Healthy Steps services to families with young children starting at birth through age three.

Evaluation Methods

The Women's and Children's Health Policy Center of the Johns Hopkins Bloomberg School of Public Health evaluated the impact of the Healthy Steps program in a sample of 2,235 parents utilizing care at one of 15 pediatrics practices who had a newborn between birth and four weeks old. Results from the 6 randomized sites are reported herein; data also are published from 9 quasi-experimental sites. Families were either enrolled in the hospital at birth or at the practice within the first four weeks after birth. Families were ineligible if the newborn was going to be adopted or placed in foster care, the newborn was too ill to come to an office visit by four weeks old, the newborn's custodial parent did not speak English or Spanish, or the family intended to leave the practice within six months. Families were enrolled at six sites between September 1996 and November 1998: Allentown, PA; Pittsburgh, PA; Amarillo, TX; Florence, SC; Iowa City, IA; and San Diego, CA. At each of the six sites, approximately 400 families were randomized to receive either standard pediatric care plus Healthy Steps services or standard pediatric care only. A total of 1,133 families were randomized to receive Healthy Steps

- 237 -

services, and 1,102 were randomized to receive standard pediatric care. At baseline, there were no significant differences in maternal characteristics or newborn's birth weight between the randomization and control families. Outcomes were compared at two to four months, at three years, and at 5.5 years. Of the 2,235 participating families, 1,987 (89 percent) completed the two-to-four-month interview, 1,593 (71 percent) completed the three-year interview, and 1,308 (59 percent) completed the 5.5-year interview. Mothers who participated in the three-year and 5.5-year interviews were older; more educated; more likely to be white, non-Hispanic, married, and employed; and less likely to be on Medicaid than nonrespondents. There was no evidence of selective attrition based on intervention status. The percentages of nonresponders in the intervention and control groups were similar for mothers younger than 20 or older than 30; who had less than high school education; and who were black, Hispanic, or employed during the last month of pregnancy. Minkovitz et al. (2001, 2003, and 2007) conducted analyses on the baseline sample, the three-year follow-up sample, and the 5.5-year follow-up sample. Baseline data were collected through a family demographic questionnaire at enrollment and a telephone interview at two to four months; the questionnaire and interview included questions on demographic information and on services and care received at the Healthy Steps site. The telephone interview at 30-33 months of age included questions on use of services, satisfaction with care, perceptions of child's behavior, parenting activities to promote safety and development, mother's health status, and depressive symptoms. Medical record abstraction was also completed after the child reached 32 months to collect data on well-child visits and vaccinations. The telephone interview at 61-66 months of age included questions on the child's misbehavior; perceptions of the child's health, behavior, development, and social skills; practices that promote development and safety; and experiences seeking care for the child. Analyses were conducted comparing intervention and control families controlling on baseline characteristics, including low birth weight; source of payment for care; mother's age, education, race, and employment status; father's employment status; mother's marital status; presence of the father in the household; number of siblings; and whether the family owned its own home. The researchers also controlled for enrollment site (e.g., Allentown, PA), enrollment location (hospital or office), and age of infant at the time of the interview.

Key Evaluation Findings

The Healthy Steps program increased the receipt of services among intervention families at two to four months (Minkovitz et al., 2001). The Healthy Steps program provided services (denoted as Healthy Steps services), including parent support groups, discussion and assessment of development at office visits, discussion of caring for the infant at office visits, a telephone information line, informational materials about development and health, letters to help parents prepare for office visits, and home visits. Some or all of these services were also provided to non-intervention families if the services were part of the practices' standard of care. Significantly more intervention families than control families received four or more Healthy Steps services at office visits (75% versus 24%) and received a home visit (76% versus 32%). Intervention families were more likely to report discussing all five developmentally important topics (calming the baby, sleep position, routines, solid foods, and car seats) with their provider (44% versus 28%). Parental perceptions of care were more positive among intervention families than among control families. Intervention families were significantly more likely to report that someone at the practice went out of their way to help them (66% versus 49%). Satisfaction with care may be an important indicator in this population because it has been linked to improved adherence with medical advice (Cameron, 1996; Winefield, Murrell, and Clifford, 1995). At two to four months, there were also reported differences in some of the safety practices of intervention and control families (Minkovitz et al., 2001). Intervention families were less likely to report use of an incorrect sleeping position for their infants (11% versus 14%). There were no differences in use of car seats or lowering of water temperature at home. There were also no differences in reported feeding practices (including breast feeding and timing of solid foods) between intervention and control families. At three years, intervention families were still receiving more services than control families (Minkovitz et al., 2003). Significantly more intervention families than control families reported receiving four or

- 238 -

more Healthy Steps services at office visits (79% versus 21%) and reported receiving a home visit after six months (79% versus 25%). Additionally, significantly more intervention families reported discussing more than six developmentally important topics with their provider (86% versus 44%), receiving a development assessment (82% versus 43%), and receiving information about community resources (45% versus 20%). Intervention families reported more frequent timely care than control families. Significantly more intervention families reported on-time, age-appropriate visits than control families, including visits at one month (98% versus 96%), two months (92% versus 88%), six months (87% versus 81%), 12 months (89% versus 84%), and 24 months (87% versus 77%). There was no difference in on-time visits at month four. Significantly more intervention families reported receiving vaccinations on time, including receipt of the DTP1 (94% versus 91%), DTP3 (82% versus 77%), and MMR1 (91% versus 87%) vaccinations, as well as receipt of all recommended vaccinations at 24 months (80% versus 72%). Parents in the intervention cohort were more likely to report receiving patient-centered care at three years (Minkovitz et al., 2003). Significantly more intervention families reported that someone in the practice went out of their way to help them (68% versus 51%). Significantly more intervention families agreed that the pediatrician or nurse practitioner provided support (91% versus 81%) and agreed that the pediatrician or nurse practitioner listened to the parent (91% versus 88%). There was no difference between intervention and control families in perceptions of the providers' respect for the parent. At three years, the care received by intervention families was more efficient in some ways than the care received by control families (Minkovitz et al., 2003). Intervention families were significantly more likely to report attending their last visit after 20 months than control families (70% versus 57%). There were no differences among intervention and control families in number of hospitalizations in the past year, emergency department use in the past year, and emergency department use for injury- related causes in the past year. Parents in the intervention and control cohorts did not differ in their use of discipline, perceptions of their child's behavior, promotion of their child's development, and use of safety practices, with the exception of use of covers on electrical outlets. Intervention parents were significantly more likely to use covers on their electrical outlets (92% versus 89%). Healthy Steps services were provided to intervention families through age three. Outcomes at 5.5 years were assessed to determine whether there were sustained effects of the Healthy Steps program. A significantly higher number of parents in the intervention group received anticipatory guidance and parenting education (59.3% versus 53.7%). Intervention parents reported a higher frequency of discussing four or more age-appropriate topics with their provider (53.7% versus 48.9%), but this difference was marginally significant. There were no differences between intervention and control families on other care-seeking experiences. There were also no differences between intervention and control families in care utilization, including use of the emergency department in the past year for an injury and hospitalizations in the past year. At 5.5 years, the discipline practices, safety practices, promotion of development, and reported behavior, developmental, and social skills of the child were also assessed. There was a marginally significant difference in parents' reports of negotiating with their children. Parents in the intervention group reported often or almost always negotiating with their child more frequently than parents in the control group (58.9% versus 54.3%). There were no significant differences on use of other discipline techniques, including slapping in the face or spanking with an object, using harsh discipline, and often or almost always ignoring misbehavior. There were no differences in reported significant concern for child's development or in reported social skills. There were also no differences in behaviors that promote development, including bedtime routine and looking at or reading books in the past week. There were no differences in reported safety practices, including use of seat belt without booster seat most of the time and always wearing a bike helmet.

Probable Implementers

Pediatric clinics, primary care providers, family practices, residency training programs.

- 239 -

Funding

Funding for the program evaluation was provided by The Commonwealth Fund (New York), the Agency for Healthcare Research and Quality, local funders, and health care providers nationwide. The Healthy Steps program is funded individually by each site. Examples of funders include local funders, philanthropies and national funders, Medicaid, and Title V. The cost of the Healthy Steps program is estimated at $463 per family per year (inflated to 2010 dollars) (Zuckerman et al., 2004). Costs are site-dependent and vary based on such factors as the number of families served, the number of HSSs needed, the needs of the population served, the need for administrative support, and the site's overhead rate.

Implementation Detail

Program Design

The Healthy Steps for Young Children program was designed by a multidisciplinary team of pediatricians, pediatric nurse practitioners, and child development faculty at Boston University School of Medicine to meet families' needs regarding their young children's development and behavior. The Healthy Steps program incorporates developmental specialists (Healthy Step specialists) into the pediatric care team by including developmental and behavioral services at pediatric well-child visits and at home visits. Implementation of the Healthy Steps program may differ by site, as the Healthy Steps team tailors the program to fit within their clinical practice. The implementation of the program in the national evaluation discussed above is a guideline for sites. In the national evaluation, the Healthy Steps team at each site provided seven services to families (Minkovitz et al., 2001):

• Enhanced well-child care: The pediatrician/family medicine physician, nurse/physician assistant, and HSS conduct well-child visits jointly, answer questions about development, and assess family health risks. A main goal of these visits is to take advantage of teachable moments in an effort to inform and support parents about development.

• Home visits by the HSS: The program offers up to six home visits during the three years. At home visits, HSSs promote safety, support interactions between the parents and the child, and discuss topics that arise in the home, where parents may feel more comfortable bringing up sensitive topics.

• Child development telephone information line: An HSS staffs this information line to answer parents' questions regarding development and behavior.

• Child development and family health check-ups: These check-ups identify early signs of developmental or behavioral problems in the child and assess family health risks, such as depression and domestic or community violence.

• Written information materials: These materials emphasize prevention and span a variety of issues relevant to young children.

• Parent groups: These meetings are facilitated by an HSS, are intended to provide parents with social support, and are a forum for informing parents through interactive exercises.

• Links to community resources: The Healthy Steps team provides information and links parents to valuable resources within the community, such as early intervention, housing, food supports, and library activities for children.

Staffing

The Healthy Steps team includes the pediatrician/family medicine physician, nurse/physician assistant, and Healthy Steps specialist. An HSS is trained as a nurse, early childhood specialist, or social worker with experience in child development. Each HSS serves approximately 100-150 families. Staff at each site were trained at three annual training sessions sponsored by the Healthy Steps training team at Boston University School of Medicine, which covers topics in child development, parenting, and clinical strategies. Program and training manuals were also provided to the Healthy Steps sites. Training and

- 240 -

implementation materials are available through the Healthy Steps website or through the Healthy Steps program office.

Issues to Consider

The Healthy Steps for Young Children national evaluation included a fairly large sample size of 2,235 families at six pediatric practices. The families were randomly assigned to intervention (standard care plus receipt of Healthy Steps services) or control (standard care only). The national evaluation found that the Healthy Steps program led to significantly more services delivered during the first three years, but in general was associated with only mixed gains in health outcomes during the first three years and no improvements in health outcomes by 5.5 years; this may be attributable to ceiling effects. This program is listed as "promising," because there is evidence that it improves some health behaviors, but the evidence that it promoted health outcomes per se is weak. The greater services delivered in the first three years included significantly higher frequencies of on-time vaccinations, developmental assessments, and discussing age-appropriate developmental topics with a provider. The evaluation also found that families receiving intervention services were more satisfied with the care received during the first three years. The program did not have an impact on the number of emergency room visits or hospitalizations. Although the program had an impact on certain intermediate outcomes during the first three years, lasting impacts on health outcomes were not seen at 5.5 years. Additional evaluations of the Healthy Steps program have been conducted, including quasi- experimental sites in the national evaluation. These evaluations found improvements at two to four months and at three years in parenting practices and delivery of services and, in some cases, found differences in health outcomes, such as on-time immunizations. An evaluation of impact by income level found no differences in some measures of care utilization at three years (including on-time well- child visits) or in perceptions of care. This study included only families receiving Healthy Steps services and did not include a control group. The national evaluation presented here is the only study of the Healthy Steps program that meets the Promising Practices Network (PPN) evidence criteria, including study design, effect size, and statistical significance.

Example Sites

For a complete listing of Healthy Steps sites, visit http://www.healthysteps.org/

Contact Information

Margot Kaplan-Sanoff, Ed.D. Associate Professor of Pediatrics Director, Healthy Steps National Office Boston University School of Medicine Vose Hall #419 72 E. Newton Street Boston, MA 02118 617 414-4767 617 414-7915 (fax) healthysteps@healthysteps.org

Available Resources

http://www.healthysteps.org/

- 241 -

http://www.healthysteps.org/

mailto:healthysteps@healthysteps.org

http://www.healthysteps.org/

Bibliography

Cameron, C., "Patient Compliance: Recognition of Factors Involved and Suggestions for Promoting Compliance with Therapeutic Regimens," Journal of Advanced Nursing, Vol. 24, No. 2, 1996, pp. 244- 250. Kaplan-Sanoff, M., "Healthy Steps: Delivering Developmental Services for Young Children Through Pediatric Primary Care," Infants and Young Children: An Interdisciplinary Journal of Special Care Practices, Vol. 13, No. 3, 2001, pp. 69-76. Minkovitz, C. S., D. Strobino, K. B. Mistry, D. O. Scharfstein, H. Grason, W. Hou, N. Ialongo, and B. Guyer, "Healthy Steps for Young Children: Sustained Results at 5.5 Years," Pediatrics, Vol. 120, No. 3, 2007, pp. e658-e668. Minkovitz, C. S., N. Hughart, D. Strobino, D. Scharfstein, H. Grason, et al., "A Practice-Based Intervention to Enhance Quality of Care in the First 3 Years of Life: The Healthy Steps for Young Children Program," Journal of the American Medical Association, Vol. 290, No. 23, 2003, pp. 3081- 3091. Minkovitz, C., D. Strobino, N. Hughart, D. Scharfstein, B. Guyer, and the Healthy Steps Evaluation Team, "Early Effects of the Healthy Steps for Young Children Program," Archives of Pediatric and Adolescent Medicine, Vol. 155, No. 4, 2001, pp. 470-479. Winefield, H. R., T. G. Murrell, and J. Clifford, "Process and Outcomes in General Practice Consultations: Problems in Defining High Quality Care," Social Science and Medicine, Vol. 41, No. 7, 1995, pp. 969-975. Zuckerman, B., S. Parker, M. Kaplan-Sanoff, M. Augustyn, and M. C. Barth, "Healthy Steps: a Case Study of Innovation in Pediatric Practice," Pediatrics, Vol. 114, No. 3, 2004, pp. 820-826.

Last Reviewed

April 2011

HighScope Perry Preschool Program

Program Info

Outcome Areas Healthy and Safe Children Children Ready for School Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards Students graduating from high school Children ages 0 to 5 exhibiting age-appropriate mental and physical development Children and youth not engaging in violent behavior or displaying serious conduct problems

Topic Areas Age of Child Early Childhood (0-8)

- 242 -

Type of Setting Child Care/Preschool Home Visiting Type of Service Instructional Support Parent Education Type of Outcome Addressed Cognitive Development/School Performance Juvenile Justice Poverty/Welfare Violent Behavior

Evidence Level Proven

Program Overview

The HighScope Perry Preschool Curriculum, an early childhood education program, is an open framework of educational ideas and practices based on the natural development of young children. Drawing on the child development theories of Jean Piaget (Piaget and Inhelder, 1969) and Lev Vygotsky (1934), the progressive educational philosophy of John Dewey (1938), and more recent work in cognitive-developmental psychology (e.g., Clements, 2004; Gelman and Brenneman, 2004; National Research Council 2005) and brain research (Shore, 1997; Thompson and Nelson, 2001), the program recognizes children as active learners, who learn best from activities that they themselves plan, carry out, and reflect on. Adults use complex language, as appropriate, as they observe, support, and extend the work of the child. Adults arrange interest areas in the learning environment; maintain a daily routine that permits children to plan, carry out, and reflect on their own activities; and join in children's activities, engaging in conversations that scaffold and extend children's plans and help them think through their ideas. The adults encourage children to make choices, solve problems, and otherwise engage in curriculum activities that contribute to their learning on key developmental indicators that encompass all areas of intellectual, social, and physical development. The HighScope Perry Preschool Program, conducted from 1962 to 1967, was provided to 3- and 4- year-old African-American children from low-income neighborhoods in Ypsilanti, Michigan. The teachers conducted daily classroom sessions for two and one-half hours on weekday mornings for children and weekly home visits to each mother and child for one and one-half hours on weekday afternoons during the course of a 30-week school year. The home visits were intended to involve the mother in the educational process and to help her to provide her child with education support and implement the curriculum within the child’s home. The Perry Preschool Program researchers have followed the children from the initial study for four decades.

Program Participants

The HighScope Perry Preschool Program study involved 123 African-American children ages 3 to 4 with IQs between 70 and 85 (between one and two standard deviations below the mean) from families of low socioeconomic status. Children with diagnosed physical handicaps were excluded. All participants were drawn from the geographic area from which children attended the Perry Elementary School in Ypsilanti, Michigan.

Evaluation Methods

Researchers from the HighScope Educational Foundation (Schweinhart et al., 2005; Berrueta-Clement et al., 1984, Schweinhart, Barnes, and Weikart, 1993) have been conducting a long-term study of the impact of participation in the Perry Preschool Program. The program utilized the HighScope Perry Preschool curriculum. Staff of the Perry Preschool Program identified children for the longitudinal study from a census of the families of students then attending Perry Elementary School, referrals by neighborhood groups, and door-to-door canvassing. All selected families were of low socioeconomic status and had children showing low intellectual performance as measured by the Stanford-Binet

- 243 -

Intelligence Scale Test at program entry. At the time of entry, the mothers had completed an average of 9.7 years of schooling, and the fathers had completed an average of 8.0 years of schooling. Forty- seven percent of the families were in single-parent households, and 40 percent of the parents were unemployed. Forty-nine percent of the households were on welfare. Children from entering families were matched on initial IQ test scores, mean socioeconomic status, and the percentage of boys versus girls. One student in each matched pair was then assigned to the treatment or control condition. Children from the same family were assigned to the same treatment status. The total study population consisted of 123 children—58 in the program group and 65 in the control group. When participants were 15 years old, socioeconomic data (such as parental educational and occupational achievement and population mobility) on the control and program groups were collected to determine whether the groups were still characteristically similar. Differences across the two groups were controlled for in the analysis. Over time, attrition in the study sample has been low, with a 9 percent attrition rate for the age-40 (37-year) follow-up. Researchers assessed the status of the two groups annually from ages 3 to 12 and at ages 14, 15, 19, 27, and 40. Information was collected via a variety of methods, including through an examination of school records; standardized testing; aptitude testing; juvenile, police, and court records; life competency testing; and interviews to reveal personal characteristics, abilities, attitudes, and social relations, among other things.

Key Evaluation Findings

Scholastic and Cognitive Outcomes

Program Participants scored significantly higher on nonverbal intellectual performance tests at the end of their first preschool year (a group average score of 97.0 versus 72.0 for the control group) and second preschool year (89.8 versus 77.9). In subsequent years, the control population narrowed this gap; however, the program participant group continued to maintain a slight edge, and the difference again achieved statistical significance at age 9, the final year of this type of testing, with the program group scoring 89.3 and the control group scoring 84.8 (Weikart et al., 1970)

Program Participants significantly outscored their control counterparts on vocabulary tests at the end of the first preschool year (a group average score of 74.5 versus 63.6 for the control group) and the second preschool year (81.0 versus 62.9). The program group maintained a slight edge in subsequent years; however, the difference was not significant (Weikart et al., 1970).

Program Participants' high school grade-point averages were significantly higher than those of the control group (2.08 versus 1.71), and control students received nearly twice as many failing grades per year as did their program counterparts (Berrueta-Clement et al., 1984).

Perry participants spent a significantly lower percentage of all their years of education in special education (16 percent for Program Participants versus 28 percent for control students), and participation in the program reduced the likelihood of being classified as mentally retarded by more than half (Berrueta-Clement et al., 1984).

Perry participants spent a significantly higher percentage of all their years of education receiving remedial services (such as speech/language services) other than special education services (8 percent for Program Participants versus 3 percent for the control population) (Berrueta-Clement et al., 1984).

Program Participants at age 14 significantly outscored their control counterparts in the total score and all subtests of the California Achievement Tests. The effect size for each of the score differences was moderate to large (Schweinhart and Weikart, 1980).

• Program students gave a positive response more frequently than did control students on 14 of 16 items measuring the students' attitudes toward high school (Berrueta-Clement et al., 1984).

- 244 -

• On the age-19 Adult Performance Level Survey (APL), the program group significantly outscored the control group in general literacy (which indicates total score), occupational knowledge, health information, and reading skills. On the age-27 APL survey, the program group significantly outscored the control group in health information and problem-solving but not general literacy. This is reflective of larger gains in general literacy among the control population as compared with the program participant group (Berrueta-Clement et al., 1984).

By age 27, the program group had completed a significantly higher level of schooling than had the control group (11.9 years for the program group versus 11.0 years for the control group) and had a sizably higher rate of high school graduation, or its equivalent, than did the control participants. Seventy-one percent of Program Participants versus 54 percent of control participants had earned a high school diploma or General Educational Development Test (GED) (Schweinhart, Barnes, and Weikart, 1993).

• The group differences in levels of schooling completed and high school graduation rates are due to differences between females in the two groups. By age 27, as compared with control females, program females completed a significantly higher level of schooling (12.5 years versus 10.5 years) and had a significantly higher rate of regular high school graduation or its equivalent (84 percent versus 35 percent). (Schweinhart, Barnes, and Weikart, 1993).

Socioeconomic Outcomes

• A significantly higher percentage of program students were working at the time of their age- 19 follow-up interview (50 percent of program students versus 32 percent of control students) (Berrueta-Clement et al., 1984).

• The control population had spent, on average, twice as many months without work since leaving school than had the program population (Berrueta-Clement et al., 1984).

Program Participants were nearly twice as likely to be economically self-sufficient and nearly half as likely to be receiving money from welfare at the time of the age-19 follow-up interview (Berrueta- Clement et al., 1984).

At ages 27 and 40, Program Participants had higher average monthly earnings than nonparticipants ($1,020 versus $700 at age 27 and $1,856 versus $1,308 at age 40) (Schweinhart, Barnes, and Weikart, 1993; Schweinhart et al., 2005).

• At age 27 there were no significant differences between the groups in terms of employment over the previous five years or in months of unemployment in the previous two years (Schweinhart, Barnes, and Weikart, 1993).

• At age 27, a significantly lower percentage of program males had received social services (such as welfare assistance services and public housing) in the previous ten years (52 percent of program males versus 77 percent of control males) (Schweinhart, Barnes, and Weikart, 1993).

Nearly three times as many Program Participants owned their own homes at age 27 (36 percent versus 13 percent of the control population) (Schweinhart, Barnes, and Weikart, 1993).

Criminal Outcomes

• At age 19, as a group, the program population had a total of 47 property or violence arrests versus 74 such arrests among the control population (Berrueta-Clement et al., 1984).

At age 19, Program Participants were half as likely to have been arrested for a non-minor offense (Berrueta-Clement et al., 1984).

- 245 -

At age 19, Program Participants were nearly half as likely to have been involved in a serious fight, caused someone an injury requiring medical attention, or have been in trouble with the police. The control population was more than twice as likely to have been involved in a group or gang fight (Berrueta-Clement et al., 1984).

By age 27, as compared with the control group, the Program Participants averaged a significantly lower number of lifetime arrests (2.3 versus 4.6) and a significantly lower number of adult arrests (1.8 versus 4.0) (Schweinhart, Barnes, and Weikart, 1993).

By age 27, 7 percent of Program Participants versus 35 percent of control participants had been arrested five or more times in their lifetimes (Schweinhart, Barnes, and Weikart, 1993).

• At age 40, 36 percent of participants versus 55 percent of nonparticipants were arrested five or more times (Schweinhart et al., 2005).

Health Outcomes

At age 40, Program Participants were more likely to be insured than were nonparticipants, and this insurance coverage led to an increase in the utilization of preventive health services; however, there were no significant differences in any measured physical health outcomes (Muennig et al., 2009).

Probable Implementers

Preschool teachers, planners, directors, and administrators.

Funding

The Ford Foundation and the U.S. Administration of Children, Youth, and Families funded the Perry Preschool Program and the longitudinal study of Program Participants. Additional funding was provided by the U.S. Department of Education's Office of Special Education Programs, the National Institute of Mental Health, the Rosenberg Foundation, the Levi-Strauss Foundation, Carnegie Corporation of New York, the Spencer Foundation, and the Robert McCormick Tribune Foundation. Current program funding comes from a variety of sources including student tuition fees, foundation grants, federal Head Start monies, and funds from state and local governments. Cost savings analyses of the program (Barnett, 1996; and Karoly et al., 1998 and 2001) have shown that over the lifetime of the participants, the Perry Preschool Program saves the government considerably more than the program originally cost.

Implementation Detail

Program Design

The Perry Program emphasizes active learning, in which children learn through self-initiated and directed activities. This framework uses a set of key developmental indicators derived from child development theory. These indicators help teachers support and extend children's activities, as well as monitor their progress. The key developmental indicators are important to the growth of rational thought in children throughout the world, regardless of nation or culture. These indicators are very simple and pragmatic. Preschool key developmental indicators have been identified in the following domains, which parallel the dimensions of school readiness identified by the National Education Goals Panel (Kagan, Moore, and Bredekamp, 1995): approaches to learning; language, literacy, and communication; social and emotional development; physical development, health, and well being; mathematics; science and technology; social studies; and the arts. Each of these categories is divided into specific types of experiences. For example, the key developmental indicators in language, literacy, and communication are "talking with others about personally meaningful experiences"; "describing objects, events, and relations"; "having fun with language by listening to stories and poems and

- 246 -

making up stories and rhymes"; "writing in various ways: drawing, scribbling, and using letter-like forms, invented spellings, and conventional forms"; "reading in various ways: reading storybooks, signs, symbols, and one's own writing"; and "dictating stories." The role of adults in the Perry Program model is to observe, guide, support, and help to extend the children's activities by arranging and equipping a variety of interest areas within the learning environment, maintaining a daily routine that permits children to plan and carry out their own activities, and joining in with children's activities as active participants and helping children to think about their play. The curriculum does not teach predefined lessons, but instead teachers listen closely to what students plan and then actively work with and question them to extend their activities to developmentally appropriate experiences. In addition to the classroom-based activities, throughout the school year the program included a weekly home-visit component by Perry Preschool classroom teachers. This component was intended to involve and integrate parents into their children's educational activities, and to promote the use of the program's methods within the home environment.

Curriculum

The program uses the HighScope Curriculum as recently described in HighScope publications, especially Epstein (2007) and Hohmann, Weikart, & Epstein (2008).

Staffing

The program as evaluated was staffed by teachers certified to teach in elementary, early childhood, or special education settings. The Perry Preschool Program had a teacher-to-student ratio of one teacher for every 5.7 students.

Issues to Consider

This program received a "proven" rating. It had rigorous evaluation standards, including a randomized experimental design and longitudinal follow-up, and the participants experienced significant and sizeable gains. The small sample size involved in this study is often cited as a study weakness. This concern is countered by the many large group differences that were found favoring the program group. It is also countered to some extent, by the high level of internal consistency in data over the years, the low level of attrition in the longitudinal sample, the existence of long-term longitudinal data, the rigorous standards of true random assignment to treatment groups, and a periodic "rematching" of the comparison groups over time to verify the continued comparability of samples and to determine whether any significant differences have arisen (for example, population mobility or increased level of parental educational or occupational attainment). The Perry Preschool study compared various outcomes for individuals who had participated in the Perry Preschool model to the same outcomes for individuals with no preschool participation. In contrast, an additional long-term study, the HighScope Preschool Curriculum Comparison study (Schweinhart and Weikart, 1997) looked at outcomes yielded from participation in the Perry Preschool program's High/Scope curriculum versus participation in two alternative preschool programs. Each curriculum model was implemented similarly, with classes two and one-half hours in duration five days a week, and one and one-half hour-long home visits weekly when the students were 3 and 4 years old. Except for the use of different curriculum models, all aspects of implementation across the three groups were identical. Although somewhat more heterogeneous, the HighScope comparison study population, like the Perry study sample, was uniformly of low socioeconomic status and faced many of the same risk factors as the Perry study group. The study found that the High/Scope curriculum was indistinguishable from a traditional Nursery School preschool curriculum through age 23. However, at age 15 and above, the group that participated in the third curriculum, Direct Instruction, reported 2 1/2 times as many acts of misconduct as the High/Scope and Nursery School groups. Participants in the Direct Instruction curriculum also participated in sports and social activities less often, and were thought of less positively by their families. Finally, when considering the evaluations and outcomes, it is important to note that all evaluative

- 247 -

efforts to date have been conducted by the HighScope Educational Research Foundation, the original designers and founders of the program. However, Heckman et al. (2009) reanalyzed the HighScope data and found similar results.

Example Sites

Ypsilanti, Michigan

Contact Information

Dr. Lawrence J. Schweinhart HighScope Educational Research Foundation 600 North River Street Ypsilanti, Mich. 48198-2898 (734) 485-2000 (phone) (734) 485-0704 (fax) LSchweinhart@highscope.org (email) www.highscope.org (website)

Available Resources

For High/Scope Curriculum materials or information or to reach the training network, call 1-800-40- PRESS. The program has a demonstration classroom in Ypsilanti, Michigan. In addition, the HighScope Educational Research Foundation offers assistance to preschool programs in the areas of training, curriculum and materials development, and research.

Bibliography

Barnett, W. Steven, Lives in the Balance: Age-27 Benefit-Cost Analysis of the High/Scope Perry Preschool Program, Ypsilanti, Mich.: High/Scope Press, Monographs of the HighScope Educational Research Foundation, 11, 1996. Berrueta-Clement, J. R., L. J. Schweinhart, W. Steven Barnett, et al., Changed Lives: The Effects of the Perry Preschool Program on Youths Through Age 19, Ypsilanti, Mich.: High/Scope Press, Monographs of the HighScope Educational Research Foundation, 8, 1984. Clements, D. H. "Major Themes and Recommendations," in D. H. Clements, J. Samara, and A. M. DiBiase, eds., Engaging Young Children in Mathematics: Standards for Early Childhood Mathematics Education, Mahwah, N.J.: Lawrence Erlbaum Associates, Inc., 2004, pp. 7-72. Dewey, J., Experience and Education, New York: Macmillan, 1938 (1973 edition). Epstein, A.S. Essentials of Active Learning in Preschool: Getting to Know the High/Scope Curriculum. Ypsilanti, MI: High/Scope Press, 2007. Gelman, R., and K. Brenneman, "Science Learning Pathways for Young Children," Early Childhood Research Quarterly, Vol. 19, No. 1, 2004, pp. 150-158. Heckman, J. J., S. H. Moon, R. Pinto, P. A. Savelyev, and A. Q. Yavitz, "A Reanalysis of the HighScope Perry Preschool Program," unpublished manuscript, University of Chicago, Department of Economics, 2009. Hohmann, M., & Weikart, D. P., & Epstein, A. S. Educating Young Children: Active Learning Practices for Preschool and Child Care Programs (3rd ed.) Ypsilanti, MI: HighScope Press, 2008.

- 248 -

mailto:LSchweinhart@highscope.org

http://www.highscope.org/

Kagan, S.L., Moore, E. & Bradekamp, S. (Eds.). Reconsidering Children's Early Development and Learning: Toward Common Views and Vocabulary. Report of National Education Goals Panel, Goal 1 Technical Planning Group. Washington D.C.: U.S. Government Printing Office, 1995. Karoly, Lynn A., M. Rebecca Kilburn, James H. Bigelow, Jonathan P. Caulkins, and Jill S. Cannon, Assessing Costs and Benefits of Early Childhood Intervention Programs: Overview and Application to the Starting Early, Starting Smart Program, Santa Monica, Calif.: RAND Corporation, MR-1336-CFP, 2001. As of August 20, 2009: http://www.rand.org/pubs/monograph_reports/MR1336/ Karoly, Lynn A., Peter W. Greenwood, Susan S. Everingham, Jill Hoube, M. Rebecca Kilburn, C. Peter Rydell, Matthew Sanders, and James Chiesa, Investing in Our Children: What We Know and Don't Know About the Costs and Benefits of Early Childhood Interventions, Santa Monica, Calif.: RAND Corporation, MR-898-TCWF, 1998. As of August 20, 2009: http://www.rand.org/pubs/monograph_reports/MR898/ Muennig, Peter, Lawrence Schweinhart, Jeanne Montie, and Matthew Neidell, "Effects of a Prekindergarten Educational Intervention on Adult Health: 37-Year Follow-Up Results of a Randomized Controlled Trial," American Journal of Public Health, Vol. 99, 2009, pp. 1431-1437. National Research Council, Mathematical and Scientific Development in Early Childhood, Washington, D.C.: National Academy Press, 2005. Piaget, J., and B. Inhelder, The Psychology of the Child, New York: Basic Books, 1969. Schweinhart, L. J., and D. P. Weikart, Young Children Grow Up: The Effects of the Perry Preschool Program on Youths Through Age 15, Ypsilanti, Mich., HighScope Press., Monographs of the HighScope Educational Research Foundation, 7, 1980. Schweinhart, L. J., and D. P. Weikart, Lasting Differences: The HighScope Preschool Curriculum Comparison Study Through Age 23, Ypsilanti, Mich.: High/Scope Press, Monographs of the HighScope Educational Research Foundation, 12, 1997. Schweinhart, L. J., H. V. Barnes, and D. P. Weikart, Significant Benefits: The High/Scope Preschool Study Through Age 27, Ypsilanti, Mich.: High/Scope Press, Monographs of the HighScope Educational Research Foundation, 10, 1993. Schweinhart, L. J., J. Montie, Z. Xiang, W. S. Barnett, C. R. Belfield, and M. Nores, Lifetime Effects: The HighScope Perry Preschool Study Through Age 40, Ypsilanti, Mich.: HighScope Press, Monographs of the HighScope Educational Research Foundation, 14, 2005. Shore, R., Rethinking the Brain: New Insights into Early Development, New York: Families and Work Institute, 1997. Thompson, R. A., and C. A. Nelson, "Developmental Science and Media: Early Brain Development," American Psychologist, Vol. 56, No. 1, 2001, pp. 5-15. Vygotsky, L., Thought and Language, Cambridge, Mass.: MIT Press, 1934 (1962 edition). Weikart, D. P., D. Deloria, S. Lawser, and R. Wiegerink. 1970. Longitudinal Results of the Ypsilanti Perry Preschool Project, Ypsilanti, Mich.: High/Scope Press.

Last Reviewed

August 2009

- 249 -

http://www.rand.org/pubs/monograph_reports/MR1336/

http://www.rand.org/pubs/monograph_reports/MR898/

Hip-Hop to Health Jr.

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children experiencing good physical health

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Type of Service Health Education Type of Outcome Addressed Physical Health

Evidence Level Promising

Program Overview

Hip-Hop to Health Jr. was a community-based program that aimed to promote healthy eating and physical activity habits in young children ages 3-5 years. The 14-week intervention was implemented within existing Head Start programs in Chicago and included 45-minute instructional sessions three times each week. The sessions began with a five-minute transitional period, followed by a 20-minute hands-on activity related to healthy eating and exercise, and concluding with a 20-minute aerobic activity. Parents of the participating children were sent weekly newsletters related to the topic being reviewed in class, and they were also sent weekly homework assignments related to the newsletter content. Parents were compensated $5 for completing each homework assignment. The intervention also included free, voluntary, 30-minute low-impact aerobic sessions for the parents twice each week (Fitzgibbon et al., 2002).

Program Participants

Latino and African American preschool students between 3 and 5 years of age

Evaluation Methods

Fitzgibbon et al. (2005) evaluated Hip-Hop to Health Jr. in 12 primarily African American Head Start programs administered through the Archdiocese of Chicago. The Head Start centers were paired according to class size, and one center in each pair was randomly assigned to the 14-week Hip-Hop to Health Jr. intervention. The other center in each pair was assigned to a general health intervention that involved one 20-minute session each of the 14 weeks on general health topics and did not include any information on physical activity or nutrition. All children attending each of the centers were eligible to participate in the intervention offered at their site. A total of 197 children enrolled in the Hip-Hop to Health Jr. intervention across the six intervention schools, while 212 children were enrolled in the general health intervention in the six control schools. Outcomes were assessed at baseline and at 1 and 2 years post-intervention and included children's

• absolute body mass index (BMI, weight in kilograms/height in meters)

- 250 -

• BMI z-scores (standard deviation) for age and sex based on 2000 Centers for Disease Control and Prevention population estimates

• 24-hour dietary intake as reported by parents to a trained dietician

• physical activity and television viewing as reported by the parent.

At baseline, control and intervention students were comparable on demographic measures, BMI, television viewing, exercise intensity, and calories from fatty acids. However, control students were statistically significantly older (average of 2 months), and were different in terms of exercise frequency: Significantly more control students than intervention students exercised 7 or more days per week (54 versus 44 percent). Finally, there were significantly more African American students in the intervention group than in the control group (99 versus 81 percent). Differences in age and BMI at intake were adjusted for in the analyses.

Key Evaluation Findings

Children participating in Hip-Hop to Health Jr. had significantly lower increases in BMI compared with control children at the 1- and 2-year follow-ups (1-year follow-up: 0.06 versus 0.59 kg/m2 increase; 2-year follow-up: 0.54 versus 1.08 kg/m2 increase). Additionally, children participating in Hip-Hop to Health Jr. had a lower percentage of calories from saturated fat at year 1 (11.6 versus 12.8 percent); however, these results were not sustained at year 2. Other measures of dietary intake showed no significant differences at either follow-up. There were no significant differences in physical activity or television viewing across the groups at either follow-up.

Probable Implementers

Public and private preschools

Funding

Funding for initial program implementation and research was provided by the National Institutes of Health.

Implementation Detail

• The intervention was delivered in three 45-minute sessions per week for 14 weeks.

• Each session included a 20-minute lesson that introduced a new concept related to healthy eating or exercise, and 20 minutes of physical activity.

• Parents in the Hip-Hop to Health Jr. intervention received weekly newsletters related to the curriculum the children were studying. Parents also received short homework assignments related to the newsletters, which they could turn into the Head Start instructor for a $5 incentive.

Issues to Consider

This program received a "promising" rating. This is due to the small number of Head Start centers (6 in each condition) as well as the baseline differences across treatment and control groups in the Fitzgibbon et al. (2005) study, which indicates some lack of comparability in the two groups. Also note that baseline levels of dietary intake and physical activity were not reported in this study. A similarly designed evaluation of the Hip-Hop to Health Jr. program among primarily Latino Head Start Centers in Chicago found no impact of the program on any measured outcomes at the year 1 and year 2

- 251 -

follow-ups (Fitzgibbon et al., 2006). These findings indicate either that the program may not be effective in different populations, or that it cannot be replicated to produce similar results, further supporting the "promising" designation.

Example Sites

Chicago Head Start Centers

Contact Information

Marian L. Fitzgibbon Institute for Health Research and Policy University of Illinois at Chicago 1747 West Roosevelt Road, Room 558 Chicago, IL 60608 E-mail: mlf@uic.edu

Available Resources

See Fitzgibbon et al. (2002) for a detailed description of the curriculum development process. For information on curriculum materials, contact the author.

Bibliography

Fitzgibbon, Marian L., Melinda R. Stolley, Alan R. Dyer, Linda VanHorn, and Katherine Kauferchristoffel, "A Community-Based Obesity Prevention Program for Minority Children: Rationale and Study Design for Hip-Hop to Health Jr.," Preventative Medicine, Vol. 34, No. 2, February 2002, pp. 289-297. Fitzgibbon, Marian L., Melinda R. Stolley, Linda Schiffer, Linda Van Horn, Katherine Kauferchristoffel, and Alan Dyer, "Two-Year Follow-Up Results for Hip-Hop to Health Jr.: A Randomized Controlled Trial for Overweight Prevention in Preschool Minority Children," Journal of Pediatrics, Vol. 146, No. 5, May 2005, pp. 618-625. Fitzgibbon, Marian L., Melinda R. Stolley, Linda Schiffer, Linda VanHorn, Katherine Kauferchristoffel, and Alan R. Dyer, "Hip-Hop to Health Jr. for Latino Preschool Children," Obesity, Vol. 14, 2006, pp. 1616-1625.

Last Reviewed

February 2012

Incredible Years

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children and youth not engaging in violent behavior or displaying serious conduct problems

- 252 -

mailto:mlf@uic.edu

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Child Care/Preschool Elementary School Out of School Time Community-Based Service Provider Type of Service Family Support Instructional Support Parent Education Youth Development Type of Outcome Addressed Behavior Problems Mental Health

Evidence Level Proven

Program Overview

The Incredible Years series is a set of comprehensive curricula targeting children ages 0-12, their parents, and teachers. The curricula are designed both to promote emotional and social competence and to prevent, reduce, and treat children's behavioral and emotional problems. Since the program's inception in 1984, Incredible Years curricula have been modified and updated. Parent Training Program The Incredible Years Series offers five different parent training programs, grouped by child's age, plus a school readiness program:

• The Incredible Years Baby and BASIC Toddler programs target parents of children ages 0-2. The Incredible Years Baby program teaches parents to read their child's cues, use effective verbal communication, and provide physical, tactile, and visual stimulation. The BASIC Toddlers program teaches a variety of positive and nurturing parenting skills.

• The Preschool BASIC program targets parents of children ages 3-6 and emphasizes parenting skills to promote children's social competence and reduce behavior problems, and it teaches parents how to play with children, help children learn, give effective praise and incentives, use limit-setting, and handle misbehavior.

• The School Age BASIC program targets parents of children ages 6-12. The School Age BASIC series focuses on promoting positive behaviors and reducing inappropriate behaviors. The ADVANCED series targets parents of children ages 4-12 and teaches effective communication and problem-solving skills.

• The School Readiness Program is a supplement to the BASIC programs and is split into two parts. Program one demonstrates child-directed play techniques using 22 DVD vignettes of adults and children in unrehearsed situations. Group leaders use these vignettes to teach parents how to promote children's academic, social, and emotional skills through child- directed play. Program two demonstrates interactive reading using 28 DVD vignettes of adults and children in unrehearsed situations, and it teaches parents how to promote children's academic, social, and emotional skills with interactive reading techniques.

All of the parent training programs are designed to enhance parenting skills and school involvement, helping parents boost their children's academic, social, and emotional competencies. In each of the five programs, group leaders trained in the Incredible Years workshops use Incredible Years manuals

- 253 -

to teach the age-appropriate curriculum to parents in group sessions, which are typically held once per week over the course of 14 or more weeks. Child Training Programs The Incredible Years child training programs use the "Dinosaur Social Skills and Problem Solving" curriculum to teach social, emotional, and academic skills to children ages 3-8. The curriculum may be taught in small group therapy sessions (about six children per group) or in the classroom. Small group therapy sessions using the Dinosaur curriculum are called "Dina Dinosaur Child Training Programs" and, for optimal performance, are taught along with the parent training program. When taught by a licensed therapist or in a therapeutic setting, child training is sometimes referred to as small group therapy or child therapy. Group therapy typically lasts between 18 and 22 weeks, with one two-hour session per week. Incredible Years offers a three-day workshop for therapists interested in using this approach for group therapy with children ages 4-8. The Dinosaur classroom program is called "Dina Dinosaur Classroom Curriculum" and is ideally taught by classroom teachers 2-3 times per week. Each session starts with a 20- to 30-minute lesson taught to the whole class. Students are then broken into small groups and given activities designed to reinforce the lesson. The skills learned should also be promoted throughout the day. The classroom curriculum includes versions to be used with children in preschool, kindergarten, or primary school. Incredible Years offers a three-day workshop for teachers and school counselors interested in using this approach with children ages 3-8. Teacher Training Program The training program for teachers emphasizes classroom-management skills, such as the effective use of praise and encouragement, proactive teaching strategies, and ways to manage inappropriate classroom behavior and build positive relationships with students. The teacher classroom-management program includes seven DVDs and program manuals delivered in a six-day training that should be facilitated by a group leader trained in the Incredible Years classroom-management workshop.

Program Participants

The Incredible Years series targets newborns and children up to age 12 who are at risk for, or who are exhibiting, conduct problems and their parents and teachers. Parents may be self-referred to the program or referred by a professional.

Evaluation Methods

Several studies from around the world that have evaluated Incredible Years components meet the Promising Practices Network (PPN) criteria and have found Incredible Years to be a promising or proven program. These studies are listed below, by the component that was tested in the study. Parent Training Program Evaluations Nine studies that met PPN criteria targeted the effects of the Incredible Years parent training program in its standard form. Webster-Stratton (1982) studied the BASIC parent training program for families living in the United States with children ages 3-5. Thirty-five mothers and their children were randomly assigned to either the BASIC parent training component group (16 mothers) or a waiting-list control group (19 mothers). Mothers, whose average age was 33 years, were recruited to the study through flyer advertisements. Data were collected at baseline prior to the intervention and again just after the intervention. Mothers in the intervention group attended four two-hour videotape modeling discussion sessions over four weeks. Intervention mothers attended the sessions in groups of eight or nine and were encouraged to participate in facilitated group discussions about their observations of the role-modeling videotapes. Mothers could practice the skills they observed unsupervised at home. Researchers collected parental- reported child behavior measures through the Eyberg Child Behavior Inventory (ECBI). They also

- 254 -

directly observed parents with their children in 30-minute sessions in a laboratory setting, during which the researchers measured children's positive affect behavior, negative affect behavior, non- acceptance behavior, and also the dominance in the parent-child relationship. Another study by Webster-Stratton (1992) looked at the effects of Incredible Years parent training in families living in the United States with children ages 3-8. In this study, 100 families, including 167 parents and their children, were randomly assigned to either the BASIC parent training component group (59 mothers and 37 fathers) or to a waiting-list control group (41 mothers and 30 fathers). The average age of the mothers was 33.7 years, and the average age of the fathers was 35.8 years. After families were assigned to receive the parent training or serve as a control, intervention families were given a 10-week individually administered Incredible Years parent training program via videotape. Parents participating in the individually administered program would visit the treatment clinic once a week to watch the videotape sessions in a private room. These parents were also given individual reading and homework assignments to complement the videotape material. The authors measured the family's socioeconomic status and positive and negative life experiences in the past year, marital status and satisfaction in marriage, parental depression, and parental intellectual impairment. In addition, using established scales, parents rated their levels of stress on the Parenting Stress Index (PSI) and child behaviors on the Child Behavior Checklist (CBCL) and ECBI. Teachers rated child behavior in preschool on the Preschool Behavior Questionnaire (PBQ), and 30-minute home observations by study staff were used to measure parent-child interactions. In addition, mothers completed daily reports of child behavior throughout the duration of the study. Differences between the intervention and control group were measured by comparing changes in the study measures that happened between the initial assessment and the assessment just after the intervention timeframe. Scott et al. (2001) looked at the effect of Incredible Year parent training in families living in England with children ages 3-8. In this study, 141 parents and their children were randomly assigned to either the BASIC parent training component group (90 parents) or to a waiting-list control group (51 parents). Parents in the parent training group were given 16 weeks of video training in two-hour weekly sessions in groups of 6-8 parents. Outcomes measures included parent reports of conduct problems and hyperactivity, conduct problems and deviance as measured in a strengths and difficulties questionnaire, behaviors measured through the CBCL, parent reports of problems, parent daily child reports, oppositional defiant disorder ratings, and observations of parental behavior. The study compared changes from before to after the intervention time period between the intervention and the control group. Patterson et al. (2002) studied the effect of Incredible Years parent training in a similar age group in England. For this evaluation, 116 parents of children ages 2-8 were assigned to one of two groups with similar background characteristics. Of 1,788 parents who received ECBI survey invitations to participate in the study, 800 families returned the survey and 391 families had children who scored above the median for behavior problem intensity. All 391 qualifying families were invited to participate in the study, and 116 families consent to participate. Then the groups were randomly assigned to BASIC parent training (60 parents) or to a control group (56 parents). The parent trainings occurred in two-hour meetings of groups of ten parents for ten weeks. Measured outcomes were child behavior ratings on the ECBI and the Goodman Strengths and Difficulties Questionnaire, health on the Goldberg General Health Questionnaire, parental stress on the PSI, and parental mental health on the Rosenberg Self-Esteem Scale. Questionnaires were administered just before the intervention timeframe, just after the timeframe, and again six months later. Gardner, Burton, and Klimes (2006) also studied the Incredible Years parent training in England. Parents of 76 British children ages 2-9 referred for conduct problems were randomly assigned to BASIC parent training (44 parents) or a no-treatment comparison group (32 parents). Families were referred to the study by health care providers, social workers, or other professionals and had children above clinical cut-off levels for problem behaviors. Of 158 referred families, 37 did not meet the study criteria and 34 were unwilling to participate. The parent training program was delivered in groups of 10-12 parents in two-hour weekly sessions over 14 weeks. Measurements were taken in the family home, just before the intervention time period and again six months later (i.e., several months after the parent training program was done). Researchers recorded parent-child interactions in six structured home setting observations, to capture child problem behaviors as well as positive and negative parenting. In addition, questionnaires were used to measure child behavior (ECBI), parental

- 255 -

discipline techniques (parenting scale), and parental depression (Beck Depression Inventory). Hutchings et al. (2007) evaluated the Incredible Years parenting program with a set of families living in Wales with preschool-age children at risk for developing conduct disorder. In 11 Sure Start areas in north and mid-Wales, 153 families were eligible to participate in the study, based on having a child age 3-4 who tested above clinical cut-off levels for conduct disorder and who lived with his or her primary caretaker. Of these, 104 families were block-randomized to receive the Incredible Years intervention, and 49 were assigned to a control group. Incredible Years parent training sessions occurred in groups of 12 or fewer, in 2- to 2.5-hour sessions over 12 weeks. Parents and children were measured when they entered the study, assigned to either receive intervention or serve as a control, then measured again every six months for up to 21 months. Measures of interest were the parent's report of child behavior as assessed through the ECBI; a parenting stress index that measured stress levels, depression, and competencies; and 30-minute home observation sessions that captured parenting behaviors and deviant child behaviors. Eighty-six intervention families and 47 control families had data for all measures taken at both the pre-test and a follow-up test. There were no significant observable differences between families who remained in the study and families who did not provide follow-up data. Jones et al. (2007) evaluated the effectiveness of the Incredible Years parent training program at treating preschool children at risk for developing both conduct problems and attention deficit hyperactivity disorder (ADHD). Children came from a larger study conducted in Wales that included 255 families that lived in designated Sure Start center areas and had been referred by their health visitor. In the Jones et al. (2007) study, 79 children were included because parents had rated their behavior above clinical cut-off levels of hyperactivity and early onset of externalizing problems. These children had been randomized on a two-to-one basis to the Incredible Years parent training intervention group or to a control group in the larger study; 50 of the children belonged to the intervention group and 29 belonged to the control group. There were no significant demographic or behavior and symptomatic differences between the control and intervention groups prior to the intervention. Child behavior and ADHD symptoms were measured just before intervention and again six months after intervention. The measures included a parent questionnaire; the ECBI, which assessed child behavior; the Strength and Difficulties Questionnaire, which assessed behaviors associated with conduct problems; the Conner's abbreviated Parent/Teacher Rating Scale, which assessed the incidence of ADHD symptoms; and in-home observations, which measured both parenting behavior and child behavior. Ninety percent of families completed assessments at both the pre- and post-tests. Webster-Stratton and Herman (2008) looked at the effect of the Incredible Years parent training program on child depressive and internalizing symptoms in children ages 3-8 living in the United States. This study relied on data from 181 families that had participated in an earlier study of young children with oppositional defiant disorder or conduct disorder. Families that had requested treatment from the University of Washington Parenting Clinic were recruited for participation in the study. Interested families were screened for eligibility based on child's age, conduct problems, and having no major physical or psychological impairments. Families that met eligibility criteria completed intake assessments and were then randomly assigned to receive the parent training program or to a control group that was waitlisted to receive the treatment. A total of 111 children participated in the parent training group, and 70 children participated in the waiting-list control group. Pre- and post-test mother and father reports were recorded using the CBCL to measure child internalizing behavior, a child mood scale from the PSI to measure child depressive symptoms, and the ECBI to measure child conduct problems. Mother-provided data was available for all children at the pre- and post-test. However, only 128 fathers provided data at the second time point. The treatment and control groups were similar in baseline characteristics, and characteristics of fathers who dropped out did not differ between the two groups. Changes in the measures over time were compared between the treatment and control groups. McIntyre (2008) measured the impact of Incredible Years parent training on preschool-age children with developmental disabilities. Parents were recruited for the study through flyers made available in their child's preschool program or through an early intervention provider. Families eligible for the study had at least one child age 2-5, and the child had to have an Adaptive Behavior Scale rating between 45 and 85, be neither deaf nor blind, and have been living with their primary caregiver

- 256 -

(guardian) for at least six months. Of 57 families who were screening, 49 met eligibility criteria for participating in the study. After meeting eligibility criteria, families were randomly assigned to the treatment group (12 weeks of 2.5-hour group sessions of Incredible Years parent training, adapted for children with developmental disabilities) or to the comparison group, which did not receive Incredible Years training. Twenty-one families in the treatment group completed the study, and 23 families in the comparison group completed the study. At baseline, the groups did not differ demographically, nor did they differ on child behavior ratings. All families in the study continued to receive their usual care in early childhood education programs. All families in the study participated in an initial pre-assessment and a post-assessment 14-16 weeks later. Assessments included a parent-completed CBCL that measured child behavior problems, a family impact questionnaire that measured negative and positive impacts the child has on the family, and observations of parent-child interactions in the home, which measured parenting behavior. Changes over time in these measures were compared between the treatment group and the comparison group. In contrast to the studies above that compared the traditional form of BASIC parent training to business as usual for families, six studies compared the effects of variations on the parent training component or compared Incredible Years parent training with other options. Webster-Stratton (1984) compared the Incredible Years parent training program with higher-cost individual therapy and with a waiting-list control group. In this study, 35 mothers of conduct-problem children ages 3-8 were randomly assigned to one of the three groups. Families were recruited from the psychiatric and behavioral clinic in a pediatric hospital and had been referred because of a child with oppositional behaviors. Families assigned to the parent training program were asked to attend nine weeks of therapist-led sessions in groups of 8-10 parents. Families in the individual treatment group met with therapists in one-on-one sessions, where the therapist modeled behavior, based on the Incredible Years curriculum, with the child and parent present. Assessments were conducted at baseline just before the intervention and three months later. Assessments included parental reports on ECBI and CBCL and parent telephone reports that measured negative and pro-social child behaviors. Thirty-minute home observation sessions were also conducted to measure parent and child behavior. Finally, teachers completed the PBQ to assess children's behavior in preschool. Webster-Stratton, Kolpacoff, and Hollinsworth (1988) looked at three variations of the Incredible Years program. In this study, 194 parents of 114 conduct-problem children ages 3-8 were randomly assigned to one of four groups: individually self-administered BASIC videotape-based modeling treatment (29 mothers and 20 fathers), a group-discussion treatment (28 mothers and 19 fathers), a therapist-led group discussion and BASIC videotape-based modeling treatment (28 mothers and 20 fathers), or a waiting-list control group (29 mothers and 21 fathers). Families that entered the study were both self-referred and referred by professionals. Families in the group discussion video modeling group were asked to meet weekly in groups of 10-15 parents for 10-12 weeks. Families in the individually administered video modeling group were asked to come to the clinic once a week for 10- 12 weeks of video sessions. Group discussion families met with a therapist-led group of 10-15 parents for 10-12 weeks, to cover the same topics covered in the Incredible Years parent training videos. Measurements were taken just before the intervention and one month after the intervention ended. Measures used were parent ratings of child behavior on the CBCL and ECBI, mother observations of child behavior in a parent daily report, parental stress on the PSI, teacher behavior ratings in the PBQ, and home observations of parent and child behavior. Webster-Stratton (1990a) looked at self-administered Incredible Year parent training, with and without therapist consultation. Forty-three families with conduct-problem children ages 3-8 were randomly assigned to one of three groups: individually administered BASIC videotape-based modeling treatment without therapist consultation (17 mothers and nine fathers), individually administered BASIC videotape-based modeling treatment with therapist consultation (14 mothers and seven fathers), or a waiting-list control group (12 mothers and seven fathers). Spaccarelli, Cotler, and Penman (1992) compared two variations of the Incredible Years parent training curriculum with a waiting-list control group. Fifty-three parents (47 mothers and six fathers) were randomly assigned to one of three groups: BASIC videotape-instruction parenting-skills group with training in problem-solving (21 parents), BASIC videotape-instruction parenting-skills group with therapist-facilitated discussion (16 parents), a waiting-list control group (16 parents).

- 257 -

Taylor et al. (1998) compared the Incredible Years parent training program with a "typical" treatment approach. In this study, 110 parents with children ages 3-8 who contacted a family center for assistance related to child conduct problems were randomly assigned to receive either the BASIC parent training component (46 families); the typical "eclectic" treatment approach offered at the center, which was not limited in the duration or number of sessions and did not follow a therapy manual (46 families); or postponed treatment in a waiting-list control group (18 families). Approximately 85 percent of the families assigned to the two treatment groups and 78 percent of families assigned to the control group met clinical cut-off levels for child behavior problems on the ECBI. Families in the eclectic treatment met with their therapist at times and frequencies decided by them. Families in the Incredible Years parent training group met in groups of about seven families to receive 11-14 weeks of therapist-led discussion, plus individual meetings with the group therapist. Therapists in this group might also make home phone calls or visits to troublesome families. Assessments were done at pre-intervention and post-intervention. Measures used were the ECBI, CBCL, parent daily telephone reports, teacher-reported child behaviors (Achenbach Teacher Report Form), Matson Evaluation of Social Skills with Youngsters (MESSY), parental depressions (Beck inventory), relationship quality, familial support, and parental anger and aggression. Lau et al. (2011) examined the impacts of Incredible Years parent training, modified to target Chinese American parents, on children's internalizing and externalizing behaviors and positive and negative parenting practices. The study enrolled 54 parents who had immigrated from China after age 18 and spoke either Mandarin or Cantonese and were parents of children ages 5-12. Parents were referred to the study by community professionals because of concerns about parental discipline and child behavior problems. Thirty-two families were randomly assigned to receive parent training, and 22 families were randomly assigned to serve as a waiting-list control group. The treatment families were split into six groups of 5-10 parents each, who attended parent training together. Parent training was offered in 14 sessions that included components of the BASIC and ADVANCED Incredible Years parent training programs. Children's internalizing and externalizing behaviors were assessed, using parental reports on the CBCL, just prior to and just after treatment. Positive and negative parenting behaviors were assessed using the Alabama parenting questionnaire just prior to and just after treatment. Child Training Program Evaluations Only one study, Webster-Stratton, Reid, and Hammond (2001b), looked at the effects of delivering the Dinosaur curriculum to children in isolation of other Incredible Years program components. In this study, 99 children ages 4-8 with early-onset conduct problems and a diagnosis of oppositional defiant disorder or ADHD were randomly assigned to the Dinosaur child training program group (51 children) or a waiting-list control group (48 children). Children in the intervention group were asked to attend 18-22 weekly sessions with two child therapists in groups of 5-6 six children. Study assessments were done just before the intervention and two months after the treatment period ended. Child conduct problems were measured using the parent-reported CBCL, a teacher report (Teacher Assessment of Social Behavior) that measures prosocial and aggressive behavior, and 30-minute home observations of child-parent interactions. Children's social problem-solving skills were measured with the WALLY test. Hyperactivity was measured using the parent-reported CBCL, the teacher-reported PBQ, and test-administrator ratings of child behavior during test-taking. Parenting behavior was measured through a daily discipline interview and in-home observations, while parenting stress was measured with a marital adjustment scale, the Beck Depression Inventory, a life experiences survey, socioeconomic status, and an anger assessment survey. See the studies on delivery of multiple components of Incredible Years for evaluations of child training in conjunction with other Incredible Years program components. Teacher Training Program Evaluations Two studies examined the impacts of delivering Incredible Years teacher training to teachers of young children. Williford and Shelton (2008) studied the effects of the Incredible Years teacher training program when delivered using a consultation model. Head Start teachers attended one group session that introduced the Incredible Years program, then met weekly with a graduate student pursuing his or her degree in

- 258 -

http://www.promisingpractices.net/program.asp?programid=134%23multicomponents

clinical psychology and trained in the Incredible Years program. In these weekly one-on-one sessions, teachers were shown Incredible Years-recommended effective classroom management, effective discipline, positive attention, and strengthening the teacher-child relationship through hands-on training tailored to the teacher's classroom needs. Specific topics covered and length of consultation period varied from teacher to teacher. In addition to the teacher training, parents of the children in the classes taught by these teachers were encouraged to attend a 10-week Incredible Years parent training series. Head Start centers were randomly assigned to either receive the treatment describe above (includes two centers with 21 classrooms in total) or to serve as a comparison group that did not receive any Incredible Years training but was given resources for mental health services (includes one large center and four small centers with 19 classrooms in total). After centers were assigned to treatment status, 59 children from the treatment group were recruited for the study, while 37 were recruited from the comparison group. The average age of children in the study was about four and a half years. Recruitment criteria were the child's score on a behavior rating scale and the family's willingness to participate in the study. The two groups were statistically equivalent on demographic characteristics, but the authors do not report whether the groups differed on pre-test measures. The study measured changes in teacher reports of disruptive behavior by the child (measured through three scales: an ADHD rating scale, an oppositional defiant disorder scale, and the Behavior Assessment System for Children, a behavior assessment tool) between the treatment group and comparison group over time. The study also measured changes in effective teaching strategies, parenting stress, and parenting skills between treatment and comparison groups over time. Approximately 66 percent of all children in the study had data for all measures for both the pre- and post-tests. Raver et al. (2008) examined the effect of Incredible Years group teacher training plus in-class coaching from a master's-level mental health consultant one day a week. Thirteen neighborhoods in Chicago were determined eligible for study participation, based on poverty rates, Head Start eligibility, crime rates, and ethnic composition. Of these, seven were selected for study participation, based on proximity to each other and to the research facility. The study included 35 classrooms from 18 preschool sites in the neighborhoods that volunteered to participate. The 18 sites were grouped into pairs, based on similarity to each other, and randomly assigned to receive the treatment described above or to a comparison group that received weekly in-class staffing support. An average of 91 percent of families of children enrolled in classrooms at these sites agreed to participate in the study. The study measured classroom quality using the Classroom Assessment Scoring System, with graduate student observers (blind to treatment status) rating each classroom on positive climate, negative climate, teacher sensitivity, and behavior management. For this study, observations were taken in September and March; comparisons were made between the treatment and comparison group scores on March observations, with September observations used as a control. Four of 87 teachers dropped out of the study between September and March. Across the 35 classrooms, 455 children entered the study in September and stayed until March; 93 children entered in September but dropped out before March; and 59 children entered the study after September and stayed until March. Differences in child gender dropout rates between the treatment and control groups could lead to a lower estimated treatment effect. Multiple-Component Evaluations The Incredible Years components are designed to be used simultaneously to treat children. Several studies have studied the impacts of delivering multiple components of the Incredible Years program to the same families. Webster-Stratton and Hammond (1997) studied the impact of delivering both the parent training component and the child training component on children meeting criteria for diagnosis of oppositional defiant disorder or conduct disorder. In this study, families of 97 children ages 4-8 with early-onset conduct problems were randomly assigned to receive either BASIC parent training (43 parents of 26 children), Dinosaur child training (27 children), combined BASIC parent training and Dinosaur child training (36 parents and 22 children), or postponed treatment as a waiting-list control group (40 parents and 22 children). Children in the child training group were asked to attend 22 weekly sessions of Dinosaur school in groups of five or six, led by two therapists. Parents in the parent training group met with other parents in groups of 10-12 for 22-24 sessions of parent training led by a therapist. Children and parents in the child training plus parent training group attended each, respectively.

- 259 -

Study assessments occurred just before the treatment period and two months after the treatment period. Parent and teacher perceptions of child behavior were measured using the ECBI, the PSI (child stress), parent daily reports, the PBQ, and the CBCL. The WALLY test was also used to measure children's social problem-solving skills, and home observations were conducted to assess parent-child interactions. Gross et al. (2003) studied Incredible Years parent training and teacher training in families with younger children. Here, 264 low-income parents of children ages 2-3 in the Chicago area were randomly assigned to one of four conditions: parent training only (75 parents), teacher training using the BASIC parent training program (52 parents), parent training delivered to both parents and teachers in separate groups (78 parents), or no intervention in a waiting-list control group (59 parents). Approximately 21 percent (56) of families dropped out of the study before it was completed, with more families dropping out of the parent training groups than the other groups. Of 112 teachers who participated in the study, 31 percent (35 teachers) left the schools and were not included in the analysis. Parents in the parent training intervention groups were asked to attend 12 weeks of Incredible Years parent training in groups of 8-12 parents. Teachers in teacher training groups were asked to attend 12 Incredible Years teacher training sessions in groups of 4-12 teachers. Parent and teacher groups were led by nurses, and homework assignments were added to the program to encourage parent-teacher collaboration. Measurements were taken just before the intervention period and just after. Measurements included parent-reported self-efficacy, discipline strategies, depression, and neighborhood and parent stress. In addition, observers rated parenting behaviors in 15-minute videotaped sessions of child-parent interactions. Child behavior was measured through the parent- reported ECBI, teacher-reported Kohn's Problem Checklist, and observer ratings of child behavior in the 15-minute video sessions. Webster-Stratton, Reid, and Hammond (2004) looked at several combinations of Incredible Years component delivery to families living in Washington. Families of 159 children ages 4-8 with oppositional defiant disorder were randomly assigned to receive either BASIC parent training (31 children); combined BASIC parent training and Incredible Years teacher training (24 children); Dinosaur child training (30 children); combined Dinosaur child training and Incredible Years teacher training (23 children); combined BASIC parent training, Dinosaur child training, and Incredible Years teacher training (25 children); or postponed treatment as part of a waiting-list control group (26 children). Child training was given to groups of 6-7 children once a week for 18-19 weeks. Parent training was conducted with groups of 10-12 parents, in weekly sessions, for 22-24 weeks. Parent and child training was led by master's- or doctoral-level therapists. Teachers received four full days of training, spread out over the intervention period. The study was conducted over multiple years, with new families recruited and assessed each year. Pre-testing was done about one to two months before the intervention period, and post-testing was completed within a few months of the end of the intervention period. Testing included in-home observations of 30-minute sessions of parent-child interacts and in-class observations of children during structured and unstructured time. In addition, parents were surveyed on different measures of parenting behaviors and strategies and parent- reported child behaviors, and teachers rated children on a variety of child behavior measures. Drugli and Larsson (2006) studied the effects of combined Incredible Years parent training with Incredible Years child training in Norway on child behavior at home and in day care or school. Study participants were 127 children ages 4-8 and their families that had been referred to clinical treatment due to oppositional or conduct problems. Eighty percent of the children were male, 31 percent were in day care and the other 69 percent were in school, and 87 percent had confirmed diagnoses of oppositional defiant disorder. Three families dropped out of the study prior to completion. Incredible Years parent training occurred in groups of 10-12, in two-hour sessions over 12-14 weeks. In addition, the child's teacher was invited to participate in one meeting with the parents and a therapist to learn about the treatment program. Incredible Years child training used the Dinosaur classroom curriculum in groups of six children who met with two therapists for two-hour sessions over 18 weeks. Forty-seven families received the parent training intervention only, and 52 families received parent training and child training. Twenty-eight families served as waiting-list control participants and did not receive any Incredible Years intervention during the study. Study data were collected just before the intervention and six months later. Multiple teacher-reported data collection tools were used to assess changes. Teacher-reported data included the PBQ, which measured conduct problems in school. Teachers also rated each child's academic performance and completed a social competence and

- 260 -

behavior evaluation of study children, which asked teachers to assess patterns of social competence and peer interactions. A student-teacher relationship scale asked teachers questions about their perception of their relationship with study children. Finally, the INVOLVE-T, a questionnaire in which teachers assess the degree of parental involvement in school, was used, and children completed the WALLY problem-solving test, which is designed to measured children's problem-solving abilities in social situations. Drugli, Larsson, and Clifford (2007) examined changes in social competence for a group of Norwegian children ages 4-8 whose families received either Incredible Years parent training or both Incredible Years parent training and Incredible Years child training. Families participating in the study were the same as those from Drugli and Larsson (2006), as were the treatment conditions. Multiple data collection tools were used to assess changes. The ECBI was used as a parent-reported measure of frequency of conduct problems. The CBCL was also used as a parent-reported and measured aggression, attention, and internalizing. A final parent-reported source of data was the KIDDIE-SADS, which is a semistructured diagnostic interview used to assess psychopathology in children. Children completed the WALLY problem-solving test, which is designed to measured children's problem-solving abilities in social situations. Using the same set of families, Larsson et al. (2009) measured the effects of either the Incredible Years parent training or both Incredible Years parenting training and Incredible Years child training on parenting strategies and parental stress. The measures of interest in this study were the parent practices interview (PPI), a questionnaire that assesses harsh discipline, inconsistent discipline, and positive parenting. In addition, the PSI, which measures parent perceived stress, was used. Webster-Stratton, Reid, and Stoolmiller (2008) studied the effects of the Incredible Years classroom management and Dinosaur school curriculum on children enrolled in Head Start, kindergarten, and first grade. Matched pairs of Head Start programs and elementary schools were randomly assigned, so that one of each pair was assigned to receive the Incredible Years intervention and the other served as a control group following its usual school curriculum. Head Start programs (33 classrooms) and elementary schools (42 kindergarten classrooms and 45 first-grade classrooms) from Seattle participated in the study. The study ran for four years, with pre-test data collected in the fall of the school year and post-test data collected in the spring. To assess impacts on teachers, data were collected for each of 153 teachers' first year of participation in the study, as control teachers were given the intervention in their second year. Since students changed classrooms each year, classroom data used to assess student impacts were collected across multiple years. Student outcomes data were available for 1,746 students located in 160 classrooms, taught by 119 teachers. There were no school-level, teacher-level, or student-level demographic differences between the control and intervention schools. Teachers in the intervention group received an average of 3.73 of four possible days of Incredible Years training to teach and an average of 27 of 30 possible lessons of the Dinosaur curriculum in their classes. Independent observers, blind to the classroom's treatment status, observed structured and unstructured classrooms sessions to record child and teacher behaviors. Teacher behaviors were recorded using the Multiple Option Observation System for Experimental Studies (MOOSES), which measures positive reinforcement, critical statements made by teachers, and the amount of interaction with students. MOOSES was also used measure child behaviors in class, including conduct problems, emotional self-regulation, and social competence. In addition, the study authors developed a teacher coder impressions inventory to evaluate the teacher's classroom management style and strategies. Observers also measured children's emotional self-regulation skills, social skills, and conduct problems using the School Readiness and Conduct Problems: Coder Observation of Adaptation-Revised tool (COCA-R). Classroom atmosphere was also rated by observers, including classroom levels of student cooperation and problem-solving, interest in subject matter, focus, responsiveness, on-task behavior, and classroom support. Two tests were administered to assess child and parent outcomes: the WALLY problem-solving and feelings test, developed by Webster-Stratton, which measures children's problem-solving skills in social situations; and the INVOLVE-T, a questionnaire in which teachers report parental involvement in school activities. Baker-Henningham et al. (2009) studied the effects of the Incredible Years teacher training and Dinosaur classroom curriculum used with preschool children in Jamaica. Five preschools—three that served inner-city communities and two that served wider areas—were selected for study participation. The schools were matched by school type, and one school of each pair was randomly assigned to

- 261 -

receive the intervention or serve as a control group. Fifteen classrooms spread across three schools were assigned to receive the Incredible Years intervention, and 12 classrooms spread across two schools were assigned to the control group. Most classrooms had one teacher, and the average classroom size was 21 children. There were no significant differences in teacher characteristics, teacher behavior, or classroom behavior between the intervention and control classrooms prior to the intervention. Teachers in the intervention group received Incredible Years teacher training, delivered in seven sessions between October and April. In addition, each intervention teacher received a one- hour consultation session each month. Classrooms in the intervention group also received 14 sessions of the Dinosaur classroom curriculum. Some components of the intervention were modified for appropriateness in a Jamaican setting. Measurements were done at the classroom level through direct observations conducted just prior to intervention in the fall and just after the intervention, in the spring of the same school year. At each observation time, four 15-minute intervals were observed to measure positive teaching behaviors, negative teaching behaviors, teacher commands, and promotion of children's social and emotional competence. In addition, classroom-level child behavior was observed for appropriate behavior and interest and enthusiasm. Herman et al. (2011) examined the impact of combinations of the Incredible Years components on improvements in internalizing symptoms among 159 families with children ages 4-6 living in the United States. The study compared families randomly assigned to one of six experimental conditions: 30 families were assigned to receive child training; 31 families were assigned to receive parent training; 24 families were assigned to receive parenting training and teacher training; 23 families were assigned to receive child training and teacher training; 25 families were assigned to receive teacher training, parent training, and child training; and 26 families were assigned to serve as a waiting-list control group that did not receive any treatment during the study period. In this study, child training occurred as 18-19 weeks of "Dinosaur School," with a single two-hour session offered at the study parenting clinic each week. Parent training occurred as 22-24 weeks of parent training, with a single two-hour session offered at the study parenting clinic each week. Teacher training was offered as four day-long (8-hour) sessions at the study parenting clinic. All study children were assessed for internalizing symptoms, using maternal reports on the CBCL, in the fall prior to treatment and in the spring, after treatment had ended. Webster-Stratton, Reid, and Beauchaine (2011) examined the effectiveness of parent training combined with child training on children's behavior and parenting behaviors among families with children ages 4-6 showing signs of ADHD. In this study, researchers randomly assigned 49 children to a treatment group that received parent training and child training and 50 children to a control group that was waitlisted to receive treatment after the study. Parents and children were assessed for child behavior and parenting behaviors, using multiple assessment tools, in the fall before the treatment period began and again in the spring, after treatment had ended. Treatment consisted of 20 weeks of Incredible Years preschool parent training and child training. Both trainings were offered in a single two-hour session per week and were scheduled to occur at the same time. Assessment tools included the Parenting Practices Inventory (parental reported parenting behaviors), the CBCL (parental reported child behaviors), Conner's Parent Rating Scale-Revised (parental reported child hyperactivity, inattentiveness and oppositional behavior), the ECBI (parental reported problem behaviors and intensity), the Social Competence Scale (parental reported positive social behaviors in child), the Teacher Report Form (teacher reported internalizing and externalizing behaviors in children), Conner's Teacher Rating Scale (teacher reported child hyperactivity, inattentiveness and oppositional behavior), and researcher-coded scales to assess child and parent behavior in a lab setting and child behavior in the school setting.

Key Evaluation Findings

Parent Training Program Fifteen studies evaluated the Incredible Years parent training component in isolation of other components and found positive effects on the parent-child relationship and/or child behavior. Webster-Stratton's (1982) study found the following:

- 262 -

• BASIC parent training group mothers reported significantly less-frequent problem behaviors than did control group mothers.

• Independent observations of children's behavior found that treatment group children showed significantly fewer submissive behaviors (e.g., approval-seeking or help-seeking) and negative behaviors (e.g., pouting, ridicule) and higher rates of positive-affect behaviors (e.g., smiling, expressions of affection) than control group children.

• There were no significant differences between the groups in the frequency of non-acceptance behaviors (e.g., frustration, ignoring) and dominance behaviors (e.g., criticizing, refusing to comply), nor in the total number of problem behaviors reported.

Another study by Webster-Stratton (1984), comparing mothers in BASIC parent training videotape group sessions with mothers in higher-cost individual therapy and mothers in a control group (no therapy), found the following:

• Children whose parents participated in BASIC parent video modeling exhibited significantly fewer occurrences and lower intensity of problem behaviors and negative behaviors. BASIC parent training video modeling mothers also reported less use of spanking than control group mothers.

• Children in the BASIC parent training video modeling group were observed to have higher rates of prosocial behavior than control group children.

• There were no significant differences between the individual therapy and video modeling groups in parent attitudes, total parent-reported child behavior problems, or total independent observations of child deviancy.

Webster-Stratton, Kolpacoff, and Hollinsworth (1988) reported results for three types of treatment groups, which included a self-administered videotape treatment modeling treatment group (IVM), a group-discussion treatment group (GD), and a therapist-led group discussion with videotape modeling treatment group (GDVM), in comparison to a control group. The authors reported the following:

• Parents' reports: Compared with parents in the control group, parents in all three treatment groups reported significantly superior results. This included both mothers' and fathers' reports of total problem behaviors, mothers' reports of occurrence and frequency of problem behaviors, mothers' reports of negative behaviors, and both mothers' and fathers' use of spanking.

• Teachers' reports: Teachers rated the GDVM and GD children as showing larger decreases in behavior problems than control group children. No significant differences were found between the IVM and control group.

• Independent observations: Children's total deviance with mothers and fathers was significantly less for the GDVM and GD groups than for the control group. When compared with the control group, total deviance for children of IVM fathers was also significantly less, but this was not the case for children of IVM mothers.

When comparing parents in the IVM, GDVM, and control groups, Webster-Stratton's (1990a) study found the following:

• Both mothers and fathers in the IVM group and GDVM group reported significantly less- frequent problem behaviors and less spanking than the parents in the control group reported.

• Observations of parents found a significantly larger decrease in the measure of total child deviance with GDVM mothers (but not with GDVM fathers) compared with the control group.

- 263 -

• When the two treatment groups were compared with each other, the GDVM children showed a significantly larger decrease in home observations of total child deviance with mothers (but not with fathers).

Spaccarelli, Cotler, and Penman (1992) reported the following:

• Parents in the videotape-based parenting skills group with additional parent training in problem-solving showed greater reductions than the control group showed in the occurrence and frequency of problem behaviors and in the intensity of the three most troubling behaviors (as identified by parents).

• The videotape-based parenting-skills group with therapist-facilitated discussion showed greater reductions than the control group showed in the frequency of problem behaviors and in the intensity of the three most troubling behaviors.

• There were no significant differences between the two videotape-based parenting skills treatment groups (additional parent training group and therapist-facilitated discussion) for any of the outcome measures.

A later study by Webster-Stratton (1992) found the following:

• Mothers in the BASIC parent training group reported significantly better scores than did control group mothers for reductions in the occurrence and frequency of child problem behaviors, total behavior problems, negative versus prosocial behaviors, and use of spanking.

• Similarly, fathers in the BASIC parent training group reported significantly better scores than did control group fathers for reductions in the frequency of problem behaviors (but not total occurrence), total behavior problems, and use of spanking. Home observations of children with their fathers showed lower rates of child deviance (i.e., whining, crying, smart talk, and noncompliance) for treatment group children than for control group children.

• No significant differences were found between the groups for teachers' reports of total behavior problems or for home observations of child deviance with mothers.

Taylor et al. (1998) found the following:

• Relative to the control group, the BASIC parent training group had significantly lower rates of occurrence and frequency of problem behaviors and total problem behaviors.

• There were no significant differences between the BASIC parent training group and the control group in parents' daily reports of negative behaviors or in teachers' reports of total problems and inappropriate social behaviors.

• When compared with the "eclectic" treatment group, the BASIC parent training mothers reported significantly greater reductions in the occurrence of behavior problems and negative behaviors.

• No significant differences were found between the two treatment groups in the frequency of problem behaviors and total problem behaviors or in teachers' reports of inappropriate social behaviors.

Scott et al. (2001) compared mothers in the BASIC parent training group with control group mothers and found that BASIC group mothers had significantly fewer reports of children's antisocial behavior, hyperactivity, deviance, total behavior problems, the three most serious behavior problems for each child (determined by parents), and daily reports of total problems.

- 264 -

Patterson et al. (2002) reported the following:

• Intensity of conduct problems did not differ significantly across intervention and control groups at post-test. By the six-month follow-up, scores in the treatment group had dropped significantly more than the scores in the control group (26.1 points compared with 9.3 points).

• Intervention-group children with baseline conduct-problem intensity scores in the normal range had significantly lower scores from pre-test to six-month follow-up (9.2 points lower), while children in the control group with baseline scores in the same range did not exhibit a statistically significant change (5.9 points lower).

• Negative outcomes on the conduct-problem frequency score were also significantly reduced in both control and intervention groups at six-month follow-up. Differences between groups were significant, with the intervention group exhibiting greater improvement than the control group.

• Differences between groups on the Strengths and Difficulties Questionnaire were statistically significant for conduct problems at both initial post-test and six-month follow-up, with outcomes favoring the intervention group.

• No significant differences were found between groups for emotional problems, hyperactivity, peer problems, or prosocial behaviors.

The study by Gardner, Burton, and Klimes (2006) found:

• Six months after program completion, child problem behaviors as reported both by parents and through direct observations were significantly more apparent in the control group than in the treatment group.

Jones et al. (2007) found the following results for Welsh families:

• Intervention children had significantly improved parental scores on the Conner's rating of ADHD symptoms, relative to control children.

Hutchings et al. (2007) also studied the Incredible Years parent training program in Welsh families and found the following:

• Intervention group children showed significant reductions in conduct problems, compared with control group children, according to the child behavior inventory test.

• Intervention group children showed significant reductions in Conner' test ratings of hyperactivity, compared with control group children.

• Intervention group children showed significant improvements in self-control ratings, compared with control group children.

• Intervention group children showed significant improvements in hyperactivity ratings and marginally significant improvements in conduct problems and deviance on the Strengths and Difficulties Questionnaire, compared with control children.

• Siblings of intervention children showed significant improvements in conduct problem intensity on the child behavior inventory test, compared with control siblings.

• Parents in the intervention group showed significant improvements in parental stress scores, significant reductions in depression, and significant improvements in rating on the parenting scale, compared with control parents.

• Observations of parents showed significant improvement in positive parenting and marginally significant reductions in critical parenting in the intervention parents, compared with control parents.

- 265 -

McIntyre (2008) found the following results related to Incredible Years parent training:

• Parents in the intervention group displayed significantly fewer inappropriate or negative parenting behaviors and marginally significantly greater child-directed praise statements than control group parents.

• Children in the intervention group had significantly lower overall parent-reported behavior problems on the CBCL, compared with control children.

• Intervention children showed significant reduction in internalizing behavior problems, compared with the control children.

• There were no significant differences between the intervention and control group in reported changes in child impact on the family over time.

Webster-Stratton and Herman (2008) found the following results with Incredible Years parent training for child depressive symptoms:

• Children in the intervention group showed a significantly greater decrease in mothers' ratings of internalizing symptoms than control group children.

• Fathers' ratings of internalizing symptoms did not differ significantly between intervention and control groups, but when the analysis was restricted to children that began the study with elevated depression levels, intervention children showed a significant decrease compared with control children.

• Mothers' ratings of child's depressed mood significantly lowered for the intervention group, compared with the control group, and the fathers' ratings showed a marginally significant decrease, with a stronger effect for children with elevated depression levels prior to treatment.

• Significant improvements were also found for mothers' parental competence, mothers' attachment, and mothers' role restriction in intervention group mothers, compared with control group mothers.

• Marginally significantly fewer intervention children (48 percent) remained in the borderline clinical range of depression, compared with control children (70 percent), by either the mother's or father's report.

• Significantly fewer intervention children (26 percent) remained in the severe-clinical range of depression, compared with control children (78 percent), by either the mother's or the father's report. Along the same lines, significantly more intervention children (31 percent) that started in the severe-clinical range returned to normal levels at post-test than control children (0 percent), by either the mother's or the father's report.

• A significantly greater percent of intervention children (23.1 percent) showed a 20 percent or greater reduction in symptoms than control children (3.3 percent), according to the mothers' ratings.

Lau et al. (2011) found the following results associated with a version of the Incredible Years parent training program modified for Chinese American parents:

• Treatment children's internalizing and externalizing behaviors, as reported by parents, were improved significantly compared with children in the waiting-list control group.

• Parents in the treatment group showed significantly greater improvement in positive and negative discipline than parents in the waiting-list control group.

- 266 -

Child Training Programs One study evaluated the Incredible Years child training or classroom curriculum component in isolation of other components and found positive effects on child behavior. Webster-Stratton, Reid, and Hammond (2001b) found:

• Child training group children had larger reductions than did control group children in parent- reported total problem behaviors, teachers' reports of aggression toward peers, and independent observations of child deviance and noncompliance.

• Similarly, child training group children outscored control group children on measures of social problem-solving, including positive responses to hypothetical conflict situations, and the variety of positive strategies (versus negative strategies) identified on a children's problem- solving test.

Teacher Training Program Two studies evaluated the Incredible Years teacher training component in isolation of other components and found positive effects on classroom environment and/or child behavior. Raver et al.'s (2008) study of Incredible Years teacher training with coaching found the following:

• Classrooms in the intervention group had significantly higher improvement in positive climate and significant reductions in negative climate, compared with the control group classrooms.

• Teachers in the intervention group demonstrated significant improvements in sensitivity and marginally significant improvements in behavior management, compared with control group teachers.

Williford and Shelton (2008) found the following effects of the Incredible Years teacher training program with consultation:

• Teachers who received the Incredible Years intervention reported significantly greater reduction in disruptive behavior by children, compared with the control teachers (who reported an increase in disruptive behavior).

• Intervention teachers also reported that a greater percentage (55 percent) of children in their classes improved at least one standard deviation on at least one measure of child disruptive behavior, compared with control children (30 percent).

• Teachers who received the Incredible Years intervention reported significantly greater use of effective teacher strategies compared with control teachers.

• Parents did not report a significant improvement in overall child disruptive behavior compared with the control group, but the researchers found that a significantly greater percentage of parents in the intervention group (64 percent) reported that their child improved at least one standard deviation on at least one measure of child disruptive behavior, compared with the control group (33 percent).

• Parents in the intervention group reported significantly greater improvement in the parenting skill area of verbosity and significantly greater improvement in child behavior management, compared with control group parents. (Parents in the teacher training intervention group were given the opportunity to take parent training, and 35 percent of parents participated in at least half of these sessions.)

- 267 -

Multiple Components Ten studies evaluated multiple components of Incredible Years series and found positive effects on the parent-child relationship, teacher-student relationship, and/or child behavior. Webster-Stratton and Hammond (1997) reported the following post-treatment results for the comparison of the Dinosaur child training (CT) group, the parent training (PT) group, the combined BASIC parent training plus Dinosaur child training (PT+CT) group, and the control group:

• For the PT and the PT+CT group, mother and father ratings on both measures of improvement in child behavior problems were significantly greater than for control children. For the CT group, significantly better scores were found for both father ratings and for one out of two mother ratings.

• Mothers in the PT group, the PT+CT group, and the CT group observed significantly fewer negative behaviors and significantly more prosocial behaviors than did control mothers.

• For child social problem-solving scores, both the PT+CT and CT children showed a significantly greater improvement in the number of different positive solutions than did control children.

• Independent observations revealed that PT, PT+CT, and CT children exhibited significantly greater improvement in negative conflict management skills with peers than did control children.

• Home observations of child behaviors found that PT+CT children significantly outscored controls in terms of positive affect (mood) with mothers (but not fathers), while PT group children scored higher than control children for positive affect with father and had a marginally significant improvement with mothers. CT children and controls did not differ in ratings of positive affect.

• No significant differences were found among the CT, CT+PT, and control groups for teacher ratings of behavior problems or for home observations of child deviance.

• PT children demonstrated no significant group differences from control children for child social problem-solving scores, teacher-rated problem behavior scores, or home observations of child deviance with mothers or with fathers.

Gross et al. (2003) found the following:

• For teacher-reported behavior problems, results showed that both parent training (PT) and teacher training (TT) led to a significant decrease from pre-test to initial post-test in classroom behavior problems when compared with the no-treatment control group. Children in the combined PT+TT group did not fare any better than children in either the PT-only or the TT- only group.

• From initial post-test to the one-year follow-up, significantly lower teacher-reported conduct- problem scores were found for the PT-only and TT-only groups but not for the control group. Furthermore, when the PT and TT groups were compared with the combined PT+TT, children in the latter group fared significantly worse than children in both the PT-only and TT-only groups.

• No significant differences were found among groups for parent-reported conduct-problem scores or for observer-rated negative child behavior.

Webster-Stratton, Reid, and Hammond (2004) reported the following:

• The CT group, the PT group, the CT+TT group, the PT+TT, and the CT+PT+TT group all showed significant and positive treatment effects on child conduct problems at home with mothers when compared with the control group. For child conduct problems at home with

- 268 -

fathers, the CT+PT+TT group, the PT group, and the PT+TT group had a significant positive effect compared with the control group, while the CT+TT group had a marginally significant effect.

• For child social competence with peers, both the CT and CT+PT+TT groups had significant effects compared with the control group, while the CT+TT group had a marginally significant effect.

In their study of Incredible Years parent training and child training with families in Norway, Drugli, and Larsson (2006) found the following:

• Children in the combined PT+CT group significantly reduced their aggression levels in day care and school settings, compared with both the PT-only and waiting-list control group children.

• Significantly more children in the PT+CT group reduced from clinical levels of conduct problems to nonclinical levels than both the PT-only and waiting-list control group children.

• Children in the PT+CT group had significantly improved level of social problem-solving strategies than PT-only children, but neither group differed significantly from the wait list control group.

• There were no significant differences between any groups in attention and internalizing problems, social competence, peer interactions, student-teacher relationships, or parent- teacher involvement.

In this same sample of families, Drugli et al. (2007) found:

• Fathers in the PT group and the PT+CT group rated their children as significantly improving in social competence on the CBCL, compared with control children.

• Mothers in the PT+CT group rated their children as marginally significantly improving in social competence on the CBCL, compared with control children.

• The number of prosocial strategies used by children in the PT+CT group on the WALLY test was significantly improved compared with PT-only children, but not compared with control children.

Also with this Norwegian sample of families, Larsson et al. (2009) found the following:

• Mothers in both the PT and PT+CT groups showed significant reductions in harsh disciplining practices and inconsistent practices and significant improvements in positive parenting, compared with control mothers.

• Fathers in both the PT and PT+CT groups showed significant improvements in positive parenting, compared with control fathers.

• Parental stress was significantly lower among mothers in both the PT and PT+CT groups, compared with control mothers.

• Parental stress was significantly lower among fathers in the PT-only group, compared with control fathers.

Webster-Stratton, Reid, and Stoolmiller's (2008) study of classroom management and Dinosaur curriculum training found the following results:

• Observers ratings showed that teachers in the intervention group had significant improvements in warmth, social/emotion, inconsistent, and harsh/critical compared with control teachers; Head Start intervention teachers showed significant improvements in

- 269 -

effective discipline compared with Head Start control teachers, but no effect was found for first-grade and kindergarten teachers.

• On the MOOSES scale, intervention teachers became significantly less critical compared with control teachers, but no differences were found for teacher-child involvement and teacher praise.

• Intervention classrooms had significantly improved classroom atmosphere ratings compared with control classrooms.

• Children in the intervention group showed significantly greater improvements in school readiness scores than did control children.

• There were no significant differences in classroom-level child conduct problems.

• On the WALLY test, children in the intervention group showed significantly greater improvement in the number of positive strategies generated and on the number of positive feelings they could identify, compared with control children.

• Parents in the intervention group showed marginally greater improvement in parent involvement in school, compared with control parents.

Baker-Henningham et al. (2009) found the following results when Incredible Years teacher training and Dinosaur classroom curriculum were used in Jamaica:

• Teachers in the intervention group showed significantly greater levels of positive behaviors, significantly lower levels of negative behaviors, significantly higher levels of warmth, and marginally significant improvement in teacher command, compared with control teachers.

• Intervention classrooms had significantly improved levels of children's appropriate classroom behavior and significantly improved interest and enthusiasm, compared with control classrooms.

• In addition, intervention classrooms showed significant improvement in opportunities for children to share and help each other, compared with control classrooms.

Herman et al. (2011) found the following effecting of multiple Incredible Years training components on children's internalizing behaviors:

• Children in the treatment group that included all three components (child training, parent training, and teacher training) showed significantly greater improvement in internalizing behaviors, as reported by mothers, than the waiting-list control children.

• Children in the treatment group that included teacher training and parent training showed marginally significantly greater improvement in internalizing behaviors, as reported by mothers, than the waiting-list control children.

• No other treatment conditions showed significant improvement in children's internalizing behaviors, compared with the waiting-list control group.

Webster-Stratton, Reid, and Beauchaine (2011) found the following results associated with families of children with signs of ADHD that received both Incredible Years parent training and Incredible Years child training:

• Children in the treatment group showed significant improvement over waitlisted children in the externalizing behavior scale, according to both mother and father reports on the CBCL.

• Children in the treatment group showed significant improvement over waitlisted children in aggression and attention scales, according to mother reports on the CBCL.

- 270 -

• Children in the treatment group showed significant improvement over waitlisted children in opposition, inattentiveness, and hyperactivity scales, according to both mother and father reports on the Conner's Parent Rating Scale-Revised.

• Children in the treatment group showed significant improvement over waitlisted children in behavior problem and intensity scales, according to both mother and father reports on the ECBI.

• Children in the treatment group showed significant improvement over waitlisted children in emotion regulation and social competence, according to both mother and father reports on the Social Competence Scale.

• Children in the treatment group showed significant improvement over waitlisted children in the externalizing behavior scale, according to teacher report on the Teacher Report Form.

• Parents of children in the treatment group showed significant improvement over waitlisted parents on laboratory observations of critical/negative statements (reduced) and praise (increased).

• Children in the treatment group showed significant improvement over waitlisted children on laboratory observations of child deviance.

• Children in the treatment group showed significant improvement over waitlisted children on researcher observations of social behaviors (i.e. concentration, acceptance of authority) in school.

Probable Implementers

Public and private preschool programs and elementary schools; small group therapists, Head Start and other child care centers.

Funding

Note that all funding figures reported here are in 2011 dollars. The BASIC and Baby parent training programs range from $1,300 to $1,895. The ADVANCED parent training program costs $995, and the BASIC supplement program costs $595. The teacher training program can be ordered for $1,250, and the child training series can be purchased for $1,150 and $1,250. Supplemental DVDs may be purchased for $195 to $595. A variety of extra and supplemental materials are available for $1 to $360. Prices do not include shipping or bulk-order discounts. Seattle-based workshops (including parent and teacher group leader training) cost $400 for a three- day workshop and $300 for a two-day workshop. Prices do not include travel fees. To receive on-site workshops, the cost is $1,500 to $2,000 per day for a trainer, plus an additional $750 to cover the trainer's travel ($1,750 to $2,000 for overseas travel). Consultation fees are $75 per hour for videotape review, $150 per hour for phone consultations, $200 for a consultation day in Seattle, and $450 for a certification (which includes two videotape reviews). For detailed cost information about the suggested budgets for each program component, visit the "Cost Planning for Administrators—Implementing the Incredible Years Programs" page on the Incredible Years website at: http://www.incredibleyears.com/WI/hosting_costplanning.asp

Implementation Detail

Program Design

The parenting, child, and teacher training programs target different precursors of conduct problems in the home, classroom, and school setting and with the child individually or in his or her peer group. Additionally, the programs are tailored to meet the needs of specific age groups or may target specific

- 271 -

http://www.incredibleyears.com/WI/hosting_costplanning.asp

skill sets. Implementers of Incredible Years can select the program and components that most closely meet their needs.

Staffing

Training is not required but is highly recommended by Incredible Years staff for effective program delivery. Workshops in the parent and child programs are offered regularly in Seattle and on request elsewhere in the United States and abroad. Training for Incredible Years programs includes the content of the programs as well as the processes and methods of delivery. Trainers model group- leader skills and use videotape examples of actual interventions. Six program workshops are offered:

• parent group leader BASIC workshop, for ages 2-8 or 6-12 years (three days)

• parent group leader ADVANCE workshop, for ages 4-12 (two days)

• parent group leader Baby workshop, for babies one month to one year (two days)

• small group therapist Dinosaur social skills and problem solving training for children workshop, for ages 4-8 (three days)

• teacher and school counselor teacher classroom management and classroom-based Dinosaur Curriculum workshop, for ages 3-8 (three days)

• teacher classroom management group leader training, for ages 3-8 (three days).

Incredible Years also offers a one-day "Consultation Day" workshop for professionals that have led at least one parenting group. The consultation day focuses on in-depth issues related to leading groups. In addition, Incredible Years program staff are available for consultation and offer critical review of videotaped sessions that are submitted by clients for feedback. Curriculum

Curriculum: Parent Training Program The parent training programs use a collaborative process, with group discussions guided by trained facilitators. Program materials include DVDs, detailed group-facilitator manuals (including questions for group discussion), parents' books, and materials for home-based activities, such as conversation card games, stickers, and detective books. Curriculum: Child Training Program The Dinosaur Child Training Program emphasizes understanding and communicating feelings, friendship development, anger management, interpersonal problem-solving (e.g., waiting, taking turns), and doing well in school. The small-group therapy materials include a therapist manual, three DVDs, stickers, books, a home activity manual, and other tools, such as a feelings wheel and a detective kit box. The classroom curriculum materials include teacher lesson plans, books, home activities for children, three DVDs, and other tools, such as a feelings wheel and detective kit box. Curriculum: Teacher Training Program The teacher training program teaches how to effectively manage a classroom, how to build school readiness and prosocial skills in children, and how to reduce child aggression and noncooperation. A supplemental DVD is available that teaches how to help children cope with conflict. The classroom management program materials include seven DVDs, a group leader's manual, Dina Dinosaur's Wheel of Fortune, a Teaching Pyramid poster, a teacher's book, and a Teaching Pyramid refrigerator magnet.

Issues to Consider

The Incredible Years series received a "proven" rating for the studies evaluating the parent training program and the child training program combined. More-recent evaluations have looked at the teacher

- 272 -

training and the child training Dinosaur classroom programs. These studies present promising results for the effectiveness at managing child behavior in the United States and proven results abroad. Few studies have assessed the effects of the Incredible Years programs versus the outcomes for a comparison group beyond two months after program completion (only three studies that went beyond a two-month follow-up were identified). Because of the waiting-list control group design that was employed by most of the studies' authors (due to ethical concerns about not offering treatment to children assigned to control groups), longitudinal follow-up often was not possible because the untreated control group participated in the program(s) shortly after completion by the original participants. The findings from the evaluations of the Incredible Years programs are thus limited to primarily short-term results, and it is unknown whether the differences noted between treatment and control groups would be maintained in the longer run. However, the handful of longer-term studies reviewed did show some significant extended program effects. Using outcomes data from 21 groups of families participating in six clinical trials, Foster, Olchowski, and Webster-Stratton (2007) estimated the cost-effectiveness of combining the child training, parent training, and teacher training components of Incredible Years. The study found that if policymakers are willing to pay at least $3,000 (in 2007 dollars) to treat child behavior problems, combining parent and teacher training is most likely to be cost-effective, based on teacher-reported problem behaviors. For parent-reported behaviors, the most likely cost-effective option is to combine all three program components.

Example Sites

Multiple states (e.g., Ohio, North Carolina, Texas, Kansas, Oklahoma, California, Colorado, Delaware, Florida, Oregon, Maine, Massachusetts, and Minnesota) and numerous international sites (e.g., Australia, Canada, England, Jamaica, Wales, Scotland, Ireland, Denmark, New Zealand, Norway, Sweden, Russia, Portugal, and the Netherlands).

Contact Information

Lisa St. George Administrative Director The Incredible Years, Inc. 1411 8th Avenue West Seattle, WA 98119 Phone: (206) 285-7565 Toll-free phone and fax: (888) 506-3562 Email: LisaStGeorge@comcast.net

Available Resources

The Incredible Years website, www.incredibleyears.com, is a comprehensive resource that provides information on Incredible Years programs, training, workshops, evaluation, implementation, and materials.

Bibliography

August, Gerald J., George M. Realmuto, Joel M. Hektner, and Michael L. Bloomquist, "An Integrated Components Preventive Intervention for Aggressive Elementary School Children: The Early Risers Program," Journal of Consulting and Clinical Psychology, Vol. 69, No. 4, 2001, pp. 614-626. Baker-Henningham, H., S. Walker, C. Powell, and J. Meeks Gardner, "A Pilot Study of the Incredible Years Teacher Training Programme and a Curriculum Unit on Social and Emotional Skills in Community Pre-Schools in Jamaica," Child: Care, Health and Development, Vol. 35, No. 5, September 2009, pp. 624-631.

- 273 -

mailto:LisaStGeorge@comcast.net

http://www.incredibleyears.com/

Baydar, Nazli, M. Jamila Reid, and Carolyn Webster-Stratton, "The Role of Mental Health Factors and Program Engagement in the Effectiveness of a Preventive Parenting Program for Head Start Mothers," Child Development, Vol. 74, No. 5, 2003, pp. 1433-1453. Brotman, Laurie Miller, Rachel G. Klein, Dimitra Kamboukos, Elissa J. Brown, Stephanie Irby Coard, and Laura Stout Sosinksy, "Preventive Intervention for Urban, Low-Income Preschoolers at Familial Risk for Conduct Problems: A Randomized Pilot Study," Journal of Clinical Child and Adolescent Psychology, Vol. 32, No. 2, 2003, pp. 246-257. Drugli, May Britt, and Bo Larsson, "Children Aged 4-8 Years with Parent Training and Child Therapy Because of Conduct Problems: Generalization Effects to Day-Care and School Settings," European Child and Adolescent Psychiatry, Vol. 15, No. 7, 2006, pp. 392-399. Drugli, May Britt, Bo Larsson, and Graham Clifford, "Changes in Social Competence in Young Children Treated Because of Conduct Problems as Viewed by Multiple Informants," European Child and Adolescent Psychiatry, Vol. 16, No. 6, 2007, pp. 370-378. Foster, E. Michael, Allison E. Olchowski, and Carolyn H. Webster-Stratton, "Is Stacking Intervention Components Cost-Effective? An Analysis of the Incredible Years Program," Journal of the American Academy of Child and Adolescent Psychiatry, Vol. 46, No. 11, 2007, pp. 1414-1424. Gardner, Frances, Jennifer Burton, and Ivana Klimes, "Randomised Controlled Trial of a Parenting Intervention in the Voluntary Sector for Reducing Child Conduct Problems: Outcomes and Mechanisms of Change," Journal of Child Psychology and Psychiatry, Vol. 47, No. 11, 2006, pp. 1123-1132. Gross, Deborah, Louis Fogg, and Sharon Tucker, "The Efficacy of Parent Training for Promoting Positive Parent-Toddler Relationships," Research in Nursing and Health, Vol. 18, No. 6, 1995, pp. 489- 499. Gross, Deborah, Louis Fogg, Carolyn Webster-Stratton, Christine Garvey, Wrenetha Julion, and Jane Grady, "Parent Training of Toddlers in Day Care in Low-Income Urban Communities," Journal of Consulting and Clinical Psychology, Vol. 71, No. 2, 2003, pp. 261-278. Hartman, Renee R., Scott A. Stage, and Carolyn Webster-Stratton, "A Growth Curve Analysis of Parent Training Outcomes: Examining the Influence of Child Risk Factors (Inattention, Impulsivity, and Hyperactivity Problems), Parental and Family Risk Factors," Journal of Child Psychology and Psychiatry, Vol. 44, No. 3, 2003, pp. 388-398. Herman, Keith C., Lindsay A. Borden, Wendy M. Reinke, and Carolyn Webster-Stratton, "The Impact of the Incredible Years Parent, Child, and Teacher Training Programs on Children's Co-Occurring Internalizing Symptoms," School Psychology Quarterly, Vol. 26, No. 3, 2011, pp. 189-201. Hutchings, Judy, Dave Daley, Karen Jones, Pam Martin, Tracy Bywater, and Rhiain Gwyn, "Early Results from Developing and Researching the Webster-Stratton Incredible Years Teacher Classroom Management Training Programme in North West Wales," Journal of Children's Services, Vol. 2, No. 3, 2007, pp. 15-26. Hutchings, Judy, Eleanor Lane, Richard Ellis Owen, and Rhiain Gwyn, "The Introduction of the Webster-Stratton Classroom Dinosaur School Programme in Gwynedd, North Wales: A Pilot Study," Educational and Child Psychology, Vol. 21, No. 4, 2004, pp. 4-15. Hutchings, Judy, Tracey Bywater, Dave Daley, Frances Gardner, Chris Whitaker, Karen Jones, Catrin Eames, and Rhiannon Edwards, British Medical Journal, Vol. 334, March 31, 2007, pp. 678-682. Jones, K., D. Daley, J. Hutchings, T. Bywater, T., and C. Eames, "Efficacy of the Incredible Years Basic Parent Training Programme as an Early Intervention for Children with Conduct Problems and ADHD," Child: Care, Health and Development, Vol. 33, No. 6, November 2007, pp. 749-756. Larsson, Bo, Sturla Fossum, Graham Clifford, May Britt Drugli, Bjorn Helge Handegard, and Willy-Tore

- 274 -

Morch, "Treatment of Oppositional Defiant Disorder and Conduct Problems in Young Norwegian Children: Results of a Randomized Controlled Trial," European Child and Adolescent Psychiatry, Vol. 18. No. 1, 2009, pp. 42-52. Lau, Anna S., Joey J. Fung, Lorinda Y. Ho, and Lisa L. Liu, "Parent Training with High-Risk Immigrant Chinese Families: A Pilot Group Randomized Trial Yielding Practice-Based Evidence," Behavior Therapy, Vol. 42, 2011, pp. 413-426. Lees, Dianne G., and Kevin R. Ronan, "Parent Management Training for Solo Mothers of Children Diagnosed with Attention Deficit Hyperactivity Disorder: An Effectiveness and Multiple Baseline Evaluation," unpublished, 2005. As of January 31, 2013: http://www.incredibleyears.com/library/paper.asp?nMode=1&nLibraryID=439 Linares, L. Oriana, Daniela Montalto, MinMin Li, and Vikash S. Oza, "A Promising Parenting Intervention in Foster Care," Journal of Consulting and Clinical Psychology, Vol. 74, No. 1, 2006, pp. 32-41. McIntyre, Laura Lee, "Parent Training for Young Children with Developmental Disabilities: Randomized Controlled Trial," American Journal of Mental Retardation, Vol. 113, No. 5, 2008, pp. 356-368. Morch, Willy-Tore, Graham Clifford, Bo Larsson, Per Rypdal, Torill Tjeflaat, Jim Lurie, May Britt Drugli, Sturla Fossum, and Charlotte Reedtz, The Incredible Years: The Norwegian Webster-Stratton Programme 1998-2004, 2004. As of January 31, 2013: http://www.incredibleyears.com/library/paper.asp?nMode=1&nLibraryID=420 Olchowski, Allison, E. Michael Foster, and Carolyn Webster-Stratton, Implementing Behavioral Intervention Components in a Cost-Effective Manner: An Assessment of the Incredible Years Program, 2006. As of January 31, 2013: http://www.incredibleyears.com/Library/items/cost-effectiveness-of- the-incredible-years-program_9-13-06.pdf Patterson, Jacoby, Jane Barlow, Carole Mockford, Ivana Klimes, Cecilia Pyper, and Sarah Stewart- Brown, "Improving Mental Health Through Parenting Programmes: Block Randomised Controlled Trial," Archives of Diseases in Children, Vol. 87, 2002, pp. 472-477. Raver, C. Cybele, Jones, Stephanie M., Li-Grining, Christine P., Metzger, Molly, Champion, Kina M., and Sardin, Latriese, "Improving Preschool Classroom Processes: Preliminary Findings from a Randomized Trial Implemented in Head Start Settings," Early Childhood Research Quarterly, Vol. 23, 2008, pp. 10-26. Reid, M. Jamila, Carolyn Webster-Stratton, and Mary Hammond, "Follow-Up of Children Who Received the Incredible Years Intervention for Oppositional-Defiant Disorder: Maintenance and Prediction of 2- Year Outcome," Behavior Therapy, Vol. 34, No. 4, 2003, pp. 471-491. Reid, M. Jamila, Carolyn Webster-Stratton, and Nazli Baydar, "Halting the Development of Conduct Problems in Head Start Children: The Effects of Parent Training," Journal of Clinical Child and Adolescent Psychology, Vol. 33, No. 2, 2004, pp. 279-291. Reid, M. Jamila, Carolyn Webster-Stratton, and Theodore P. Beauchaine, "Parent Training in Head Start: A Comparison of Program Response Among African-American, Asian American, Caucasian, and Hispanic Mothers," Prevention Science, Vol. 2, No. 4, 2001, pp. 209-227. Scott, Stephen, Quentin Spender, Moira Doolan, Brian Jacobs, and Helen Aspland, "Multicentre Controlled Trial of Parenting Groups for Childhood Antisocial Behaviour in Clinical Practice," British Medical Journal, Vol. 323, 2001, pp. 1-7. Spaccarelli, Steve, Sheldon Cotler, and Doris Penman, "Problem-Solving Skills Training as a Supplement to Behavioral Parent Training," Cognitive Therapy and Research, Vol. 16, No. 1, 1992, pp. 1-18.

- 275 -

http://www.incredibleyears.com/library/paper.asp?nMode=1&nLibraryID=439

http://www.incredibleyears.com/library/paper.asp?nMode=1&nLibraryID=420

http://www.incredibleyears.com/Library/items/cost-effectiveness-of-the-incredible-years-program_9-13-06.pdf

Taylor, Ted K., Fred Schmidt, Debra Pepler, and Christine Hodgins, "A Comparison of Eclectic Treatment with Webster-Stratton's Parents and Children Series in a Children's Mental Health Center: A Randomized Controlled Trial," Behavior Therapy, Vol. 29, 1998, pp. 221-240. Webster-Stratton, Carolyn, "Advancing Videotape Parent Training: A Comparison Study," Journal of Consulting and Clinical Psychology, Vol. 62, No. 3, 1994, pp. 583-593. Webster-Stratton, Carolyn, "Early-Onset Conduct Problems: Does Gender Make a Difference?" Journal of Consulting and Clinical Psychology, Vol. 64, No. 3, 1996, pp. 540-551. Webster-Stratton, Carolyn, "Enhancing the Effectiveness of Self-Administered Videotape Parent Training for Families with Conduct-Problem Children," Journal of Abnormal Child Psychology, Vol. 18, No. 5, 1990a, pp. 479-492. Webster-Stratton, Carolyn, "Individually Administered Videotape Parent Training: Who Benefits?" Cognitive Therapy and Research, Vol. 16, No. 1, 1992, pp. 31-35. Webster-Stratton, Carolyn, "Long-Term Follow-Up of Families with Young Conduct Problem Children: From Preschool to Grade School," Journal of Clinical Child Psychology, Vol. 19, No. 2, 1990b, pp. 144- 149. Webster-Stratton, Carolyn, "Predictors of Treatment Outcome in Parent Training for Conduct- Disordered Children," Behavior Therapy, Vol. 16, No. 2, 1985, pp. 223-243. Webster-Stratton, Carolyn, "Preventing Conduct Problems in Head Start Children: Strengthening Parent Competencies," Journal of Consulting and Clinical Psychology, Vol. 66, No. 5, 1998, pp. 715- 730. Webster-Stratton, Carolyn, "Randomized Trial of Two Parent-Training Programs for Families with Conduct-Disordered Children," Journal of Consulting and Clinical Psychology, Vol. 52, No. 4, 1984, pp. 666-678. Webster-Stratton, Carolyn, "Systematic Comparison of Consumer Satisfaction of Three Cost-Effective Parent Training Programs for Conduct-Problem Children," Behavior Therapy, Vol. 20, No. 1, 1989, pp. 103-115. Webster-Stratton, Carolyn, "Teaching Mothers Through Videotape Modeling to Change Their Children's Behavior," Journal of Pediatric Psychology, Vol. 7, No. 3, 1982, pp. 279-294. Webster-Stratton, Carolyn, and Keith C. Herman, "The Impact of Parent Behavior-Management Training on Child Depressive Symptoms," Journal of Counseling Psychology, Vol. 55, No. 4, 2008, pp. 473-484. Webster-Stratton, Carolyn, and Mary Hammond, "Treating Children with Early-Onset Conduct Problems: A Comparison of Child and Parent Training Interventions," Journal of Consulting and Clinical Psychology, Vol. 65, No. 1, 1997, pp. 93-109. Webster-Stratton, Carolyn, Jamila Reid, and Mary Hammond, "Social Skills and Problem-Solving Training for Children with Early-Onset Conduct Problems: Who Benefits?" Journal of Child Psychology and Psychiatry and Allied Disciplines, Vol. 42, No. 7, 2001b, pp. 943-952. Webster-Stratton, Carolyn, M. Jamila Reid, and Mary Hammond, "Preventing Conduct Problems, Promoting Social Competence: A Parent and Teacher Training Partnership in Head Start," Journal of Clinical Child Psychology, Vol. 30, No. 3, 2001a, pp. 283-302. Webster-Stratton, Carolyn, M. Jamila Reid, and Mary Hammond, "Treating Children with Early-Onset Conduct Problems: Intervention Outcomes for Parent, Child, and Teacher Training," Journal of Clinical Child and Adolescent Psychology, Vol. 33, No. 1, 2004, pp. 105-124.

- 276 -

Webster-Stratton, Carolyn, M. Jamila Reid, and Mike Stoolmiller, "Preventing Conduct Problems and Improving School Readiness: Evaluation of the Incredible Years Teacher and Child Training Programs in High-Risk Schools," Journal of Child Psychology and Psychiatry, Vol. 49, No. 5, 2008, pp. 471-488. Webster-Stratton, Carolyn, M. Jamila Reid, and Ted Beauchaine, "Combining Parent and Child Training for Young Children with ADHD," Journal of Clinical Child and Adolescent Psychology, Vol. 40, No. 2, 2011, pp. 191-203. Webster-Stratton, Carolyn, Mary Kolpacoff, and Terri Hollinsworth, "Self-Administered Videotape Therapy for Families with Conduct-Problem Children: Comparison with Two Cost-Effective Treatments and a Control Group," Journal of Consulting and Clinical Psychology, Vol. 56, No. 4, 1988, pp. 558- 566. Williford, Amanda P., and Terri L. Shelton, "Using Mental Health Consultation to Decrease Disruptive Behaviors in Preschoolers: Adapting an Empirically-Supported Intervention," Journal of Child Psychology and Psychiatry, Vol. 49, No. 2, 2008, pp. 191-200.

Last Reviewed

February 2013

Infant Health and Development Program

Program Info

Outcome Areas Healthy and Safe Children Children Ready for School Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards Children ages 0 to 5 exhibiting age-appropriate mental and physical development Babies born weighing more than 5.5 pounds and improving outcomes for low birth weight babies

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Health Care Provider Home Visiting Type of Service Family Support Health Care Services Health Education Parent Education Type of Outcome Addressed Behavior Problems Cognitive Development/School Performance Physical Health Substance Use and Dependence

- 277 -

Evidence Level Proven/Promising

Program Overview

The Infant Health and Development Program (IHDP) was an evaluation of a comprehensive early childhood intervention for low birth weight (less than or equal to 2,500 grams or about 5.5 pounds), and premature (less than or equal to 37 weeks) infants designed to reduce the infants’ health and developmental problems. The intervention was operated in eight medical institutions in Little Rock, Arkansas; New Haven, Connecticut; Miami, Florida; Cambridge, Massachusetts; Bronx, New York; Philadelphia, Pennsylvania; Dallas, Texas; and Seattle, Washington from 1985 to 1988. The IHDP was designed as a randomized clinical trial, and the participating sites were selected through a national competitive review. The intervention conducted by IHDP combined early child development and family support services with pediatric follow-up. The program was initiated upon infants’ discharge from the neonatal nursery and continued until 36 months of age (child age was corrected for prematurity). The intervention services, provided free to participating families, consisted of three components: home visits, child attendance at a child development center, and parent group meetings. Infants participated in pediatric follow-up, which was comprised of medical, developmental, and social assessments, with referral for pediatric care and other services as indicated.

Program Participants

Participants in the IHDP study included low birth weight newborns. The mothers' average age at study enrollment was 25 years, and roughly 40 percent of the mothers had less than a high school education. Approximately half of the infants were male, 52 percent were African-American, 11 percent were Hispanic, and 37 percent were white or another race.

Evaluation Methods

The IHDP evaluated a sample of participants from which infants were randomly assigned to intervention and control groups. The sample was selected according to the following process (IHDP, 1990): Infants were screened at birth for eligibility, including a total of 4,551 infants who would reach 40 weeks' post-conception age between January and October 1985 and whose birth weights were less than or equal to 2,500 grams (g). Of this sample, 3,249 infants were excluded by criteria related primarily to residence outside of the catchment area defined by distance from the early childhood education center, gestational age greater than 37 weeks, or hospital discharge before or after the study recruitment period. Nearly 45 percent of LBW infants are born after 37 weeks, and these infants are at an increased risk of health complications compared to full-term infants at a healthier weight (Hediger, Overpeck, et al. 2002). Infants with an illness or neurological deficit so severe as to prevent participation in the IHDP intervention (61 such infants) were also excluded from the study. Of the 1,302 infants who met the eligibility criteria for participation, the parents of 274 (21 percent) infants refused to give consent to be randomized into intervention or control groups. Among the 1,028 remaining infants who had both parental consent and were randomized, 43 withdrew before participating in the study. The remaining 985 infants constituted the primary analysis group. The 985 infants were categorized into two birth weight groups: greater than 2,000g = "heavier" and less than or equal to 2,000g = "lighter." One-third of the sample came from the heavier group and two-thirds from the lighter group. Within each weight group, one-third of the subjects were randomized to the intervention group and two-thirds to the control group. The randomization procedures monitored balance between groups for birth weight, gender, maternal education (less than high school; high school graduate; some college, or more), maternal race (African-American, Hispanic, and white/other), primary language in the home, and infant participation in another study. Random assignment procedures resulted in 377 infants (142 heavier, 235 lighter) in the intervention group and 608 infants (220 heavier, 388 lighter) in the control group.

- 278 -

Children in both of the study groups were assessed by the project staff at eight clinic visits occurring at 40 weeks' post-conceptional age and at 4, 8, 12, 18, 24, 30, and 36 months. At each clinic visit, mothers were asked about their child's health and developmental functioning and about social and demographic characteristics of the family. Each clinic visit also included measurements of the child's growth. Cognitive assessments were completed at 12, 24, and 36 months, and behavioral data were measured at 24 and 36 months. In addition, the quality of each child's home environment was assessed at 12 and 36 months using the HOME inventory. Although the baseline characteristics of the study sample varied greatly among the eight sites, the randomization procedure overall resulted in comparable intervention and control groups at study entry. Retention in the study was high; of the 985 infants in the primary analysis group, 908 were seen at 12 months, 875 at 24 months, and 913 at 36 months. Key child outcomes measured included cognitive development (e.g., IQ, vocabulary, and visual- motor/spatial skills); behavioral competence (e.g., mother's reports of behavior problems, observations of child's behavior during problem-solving tasks, and observations of mother/child interaction); school achievement (e.g., school attendance and results from standardized achievement tests in reading and math); and health (e.g., mother's perception of child health, morbidity [presence or absence of health conditions], and functional status [limitations in activities of daily living due to health problems]). Study children have been followed through age 18. The initial IHDP study reported on 36-month cognitive (IQ), behavioral (mother's reports of behavior problems), and health outcomes (morbidity, functional status, and maternal perception of child's health) for the full sample of infants (IHDP, 1990). Additional studies using the same national dataset examined two, three, five, eight and 18-year cognitive, behavioral and health outcomes for the same cohort (Brooks-Gunn, Liaw, and Klebanov, 1992; Brooks-Gunn et al., 1993; Blair, Ramey, and Hardin, 1995; Brooks-Gunn et al., 1994; McCarton et al., 1997; McCormick et al., 2006). The above studies all adjusted for attrition and stratified results by heavier low birth weight (2001-2499 g) and lighter low birth weight (<=2000g). A number of studies used the IHDP data to assess the effects of the intervention on various subsamples of the original sample of infants. One study (McCormick et al., 1993) analyzed 36-month outcomes for the 280 very-low birth weight (between 1,001g and 1,500g) and extremely low birth weight (less than or equal to 1,000g) infants who were selected for the intervention group. No differences in demographic factors were found between intervention and control groups. Outcome measures included cognitive development, behavioral competence, and health. Spiker, Ferguson, and Brooks-Gunn (1993) studied children's behavioral competence and mother-child interactions at 30 months of age. Of the 985 families in the IHDP study, videotapes of 683 mother/infant pairs (69 percent) were obtained, including 271 dyads from the intervention group and 412 dyads from the control group. Children and mothers were videotaped during an eight-minute free- play period, a clean-up period, and three different problem-solving tasks. Comparisons of initial status characteristics between the two treatment groups did not identify any significant differences. Child outcome measures included persistence with task (goal-directedness), percentage of time off-task, enthusiasm for task (quality of interest and enjoyment), and overall child behavior (summary of enthusiasm, persistence, cooperation with mother, and enjoyment of reward). Mother/child interactions were also assessed, including overall experience (overall quality of the problem-solving situation for affecting the child's sense of self as competent and enthusiastic) and mutuality (how well the mother and child worked together). Bradley et al. (1994) focused on low-income infants enrolled in the study and classified children as "resilient" or not, based on whether they were functioning within acceptable ranges with respect to health and development. The sample consisted of the 410 infants from the larger study who were living in poverty (as defined by the 1985 US Census), including 167 in the intervention group and 243 in the control group. The authors assessed cognitive, behavioral, and health outcomes at 36 months. Hollomon and Scott (1998) assessed school performance, attendance, and special education placement of 299 children at age 9 in the Miami site of the IHDP study. The sample consisted of three groups of children: (1) children born low birth weight and preterm who received no intervention

- 279 -

(control group; 49 children); (2) children born low birth weight and preterm who participated in the intervention (intervention group; 42 children); and (3) children born full term and normal birth weight (NBW) (>2,500g) who received no intervention (NBW group; 360 children). The control and intervention group consisted of all those children from the Miami site of the IHDP whose 1993/1994 academic records could be tracked in the school system. Of the original 60 children in the control group, 82 percent were located, and of the initial 50 children who received the intervention, 84 percent were located. The sample of NBW children was chosen from hospital birth logs by selecting the next three births after the birth of an infant with low birth weight whose home address fell within the IHDP catchment area; 58 percent of the NBW children were tracked in the school system. Approximately 66 percent of the total sample was African-American, 26 percent was Hispanic, and 8 percent was white. Finally, Bradley, Burchinal, and Casey (2001) assessed whether the children's home environment influenced the effects of the intervention on the children's 36-month IQ and behavior. The HOME Inventory was used to measure the quality and quantity of stimulation and support available to a child in his or her home environment. Of the 985 children constituting the primary analysis group for the IHDP, 819 were administered the HOME Inventory at 36 months (including 86 percent of those receiving the intervention at 36 months and 82 percent of those in the control group at 36 months). There were no significant differences in the demographic characteristics of those receiving HOME assessments and those with missing data.

Key Evaluation Findings

The IHDP study included children from eight different program sites. Multiple research studies have analyzed a variety of participant outcomes at different time intervals, ranging from 12 months to 18 years of age. The studies have shown effect sizes for cognitive development at 24 and 36 months that are "proven" according to Promising Practices criteria, and "promising" for behavioral changes. Through age 18, the intervention showed effects that are "promising" on cognitive development and achievement. There were no effects on health status, including growth. The following findings are based on the national-level data from the IHDP study:

• 12-month outcomes

o No significant intervention effect was found for IQ (Brooks-Gunn et al., 1993, Blair, Ramey, and Hardin, 1995)

• 24-month outcomes

o Significant, positive effects were found across all cognitive domains assessed, which were IQ, vocabulary, receptive language, and visual-motor skills. (Brooks-Gunn, Liaw, and Klebanov, 1992; Brooks-Gunn et al., 1993)

o Results indicated the intervention was more efficacious for the heavier infants than for the lighter infants, and for African-American children compared to white or Hispanic children. (Brooks-Gunn, Liaw, and Klebanov, 1992; Brooks-Gunn et al., 1993)

• 36-month outcomes

o Mean IQ scores were significantly higher for the intervention children than for the control children in both the heavier-birth-weight group (mean score of 98.0 versus 84.8), and lower-birth-weight group (mean score of 91.0 versus 84.4) (IHDP, 1990; Brooks-Gunn et al., 1993; Brooks-Gunn et al., 1994; Bradley, Burchinal, and Casey, 2001)

o Follow-up IQ comparisons suggest that the intervention was most effective for infants with poorer quality home environments (i.e., children derive more benefit from the intervention if it offers supports not currently available in the child’s home environment). The association between quality of home environment and improved outcomes was not found for behavioral outcomes (Bradley, Burchinal, and Casey, 2001).

- 280 -

o The intervention group had significant, positive effects on all aspects of cognitive functioning, including vocabulary, receptive language, visual-motor/spatial skills, reasoning, the picture vocabulary test, and the Motor Integration Test (Brooks-Gunn, Liaw, and Klebanov, 1992; Brooks-Gunn et al., 1994)

o The intervention group had significantly lower behavioral problem scores than the control group (IHDP, 1990; Brooks-Gunn et al., 1993; Brooks-Gunn et al., 1994; Bradley, Burchinal, and Casey, 2001), and 36-month behavior scores indicated that the IHDP was more effective for African-American infants than for white infants (Brooks-Gunn et al., 1993)

o No significant differences were found between groups for functional status or for maternal perceptions of the child’s health. (IHDP, 1990)

Lighter birth weight children in the intervention group reported higher morbidity scores (i.e., a higher presence of health conditions over three years) than did lighter birth weight children in the control group. No significant differences were found between groups for the heavier birth weight infants. (IHDP, 1990; Brooks-Gunn et al., 1993; Brooks-Gunn et al., 1994). McCormick et al. (1993) further stratified lighter low birth weight infants and found that the association was strongest among extremely low birth weight infants (less than or equal to 1,000g). The authors note that these results were due to an increase in brief illnesses and conditions, rather than a change in serious morbidity (IHDP, 1990). This issue is discussed further below, in the Issues to Consider section.

o The intervention was more effective for Hispanic children with respect to receptive language and for African-American children with respect to the picture vocabulary test and visual-motor/spatial skills (Brooks-Gunn, Liaw, and Klebanov, 1992).

• 5-year outcomes

o No significant differences between intervention and control groups overall were found for full-scale IQ, or for verbal or performance IQ subscales.

o When stratified by birth weight, significant differences in full-scale IQ were demonstrated for the heavy low birth weight intervention group (2001-2499g) compared with the treatment group (95 versus 92), and significant differences were also found in verbal IQ (94 versus 90). The same differences were not found for the lighter low birth weight group (Brooks-Gunn et al., 1994).

o There was a significant difference favoring the intervention group on the picture vocabulary test (Brooks-Gunn et al., 1994).

o No significant differences in behavioral measures were observed across intervention and control groups (Brooks-Gunn et al., 1994).

o There were no significant differences across control and intervention groups in number of hospitalizations (Brooks-Gunn et al., 1994).

• 8-year outcomes

o In the full sample and in the lighter low birth weight subgroup, there were no significant differences between the intervention and control groups for any of the cognitive, school performance, or behavioral outcome measures (McCarton et al., 1997).

o Differences favoring the intervention group were found within the heavier low birth weight subgroup, including full-scale IQ (mean score of 97 versus 92), verbal IQ (mean score of 99 versus 94), performance IQ (mean score of 95 versus 91), mathematics achievement, and the picture vocabulary test. No significant differences were found between groups for reading scores, or for the number of children that were required to repeat a grade (McCarton et al., 1997).

- 281 -

Few significant differences were found between groups for health outcomes, with two exceptions. In the full sample, the intervention group received significantly fewer favorable ratings than the control group on the measure of physical functioning, and marginally significant lower maternal ratings assessing social limitations caused by physical health. (McCarton et al., 1997). These counterintuitive findings are discussed in the Issues to Consider section.

• 18-year outcomes

o The lighter low birth weight intervention group showed statistically significantly lower scores on the Woodcock-Johnson Test of Achievement in reading but no other significant differences in any of the outcome measures were observed for lighter low birth weight intervention children compared with the control group (McCormick et. al., 2006)

o The heavier low birth weight group showed significantly higher scores in math, vocabulary, and reduced scores on a scale of youth risk behaviors (McCormick et. al., 2006).

Additional studies examined outcomes for different subsets of IHDP participants. A study by Bradley et al. (1994) focused on low-income infants enrolled in the IHDP. At 36 months, the authors found:

• Significantly more intervention group than control group infants were classified as being resilient (39 percent compared with 12 percent).

• Significantly more intervention group than control group infants scored above 85 on an IQ test (57 percent compared with 26 percent).

• Significantly more intervention group than control group infants scored less than 65 on the Total Problems scale of the behavioral outcome measure (85 percent versus 73 percent).

• There were no significant differences between groups with regard to health status or growth status.

Hollomon and Scott (1998) assessed school performance at age 9 among children that were in the Miami site of the IHDP. The authors reported the following:

• There were no significant differences between the three groups in terms of attendance records, rates of special education placement, and tests of reading, math computation, and math application.

Probable Implementers

Public and private early childhood programs

Funding

Funding for the Infant Health and Development Program was provided by the

• March of Dimes Foundation

• National Institute of Child Health and Human Development

• Pew Charitable Trusts

• Robert Wood Johnson Foundation

• Stanford Center for the Study of Families, Children, and Youth

• U.S. Department of Health and Human Services.

- 282 -

While cost data for implementing the intervention for the IHDP across the eight sites were not obtained, program expenses were assessed at the Miami site (Fewell and Scott, 1997). The cost of delivering the three programmatic components was estimated at that time to be $15,146 per year per child. The investigators suggested that this high cost could have been reduced to $8,806 per year per child if the child care centers were located in the community rather than at a central location, if transportation costs were reduced, and if the teacher-to-child ratio at the child care center were increased from two-to-six to two-to-eight.

Implementation Detail

Program Design

The specific curricula used in the IHDP were adapted for low birth weight infants from an early intervention with socially disadvantaged normal birth weight children called the Carolina Abecedarian Project. The Carolina Abecedarian Project is a "Proven" program on PPN. For more information, read the Carolina Abecedarian Project program summary. In addition, IHDP incorporated other treatment modalities such as parent support, parent problem solving, nurse support, and developmentally appropriate play materials.

Curriculum

Home Visits: The IHDP protocol specified weekly home visits for the first year, and biweekly visits thereafter. The home visitor provided parents with health and developmental information, along with family support. In addition, the home visitor implemented two specific curricula, the first of which emphasized cognitive, linguistic, and social development through games and activities for the parent to use with the child, while the second involved a systematic approach to help parents manage self- identified problems. Child Development Centers: Beginning at 12 months and continuing until 36 months, the IHDP intervention children attended a Child Development Center five days a week for at least four hours a day. The teaching staff continued to implement the curriculum learning activities used by the home visitors and tailored the program to each child’s needs and developmental levels. Teacher-child ratios were one-to-three for children age 12 to 23 months and one-to-four for those age 24 to 36 months. Class sizes were six children for those under 24 months of age and eight children for those 24 to 36 months of age. Each site provided children with (optional) transportation in IHDP-operated vans. Parent Groups: Beginning at 12 months, bimonthly parent group meetings provided parents with information on child rearing, health and safety, and other parenting concerns, along with some degree of social support.

Issues to Consider

This program received a "proven" rating for the indicators "Children ages 0 to 5 exhibiting age- appropriate mental and physical development," and "Babies born weighing more than 5.5 pounds and improving outcomes for low birth weight babies." The program received a "promising" rating for the indicator "Students performing at grade level or meeting state curriculum standards." The IHDP study used rigorous standards, including a randomized experimental design and longitudinal follow-up. Participants experienced significant gains across most of the cognitive outcomes through age 3 and across some of the behavioral outcomes, however the program's impacts at later ages are less consistent.

The studies (e.g., Brooks-Gunn Liaw, and Klevanov, 1992, and Brooks-Gunn et al., 1993) suggest that the intervention was more effective for African-American (and sometimes Hispanic) children than for white children. Another study (Bradley, Burchinal, and Casey, 2001) found that IQ improvements were greatest for infants with poorer quality home environments. In addition, several of the IHDP evaluations (e.g., Brooks-Gunn, Liaw, and Klevanov, 1992, Brooks-Gunn et al., 1993, and Brooks- Gunn et al., 1994) suggest that the intervention was less successful for children at greater biological

- 283 -

risk (i.e., very-low birth weight infants) than for lower-risk children (the heavier low birth weight group).

At age 3, authors found that IHDP participants were more likely to have experienced illnesses than their counterparts in the control group (IHDP, 1990; Brooks-Gunn et al., 1993; Brooks-Gunn et al., 1994; McCormick et al., 1993). The authors speculate that this may be due to the fact that children who regularly attend child care are more likely to contract childhood illnesses than other children, or due to the fact that IHDP participants were in more frequent contact with professionals who might identify health conditions. Note that during the course of the trial there were no accidents or serious infectious epidemics noted by program staff. At age 8, the intervention group received lower ratings than the control group on a measure of physical limitations in behavior, and the lighter low birth weight group received lower ratings than their control counterparts on measures of social limitations due to behavior. Similar to the findings at age 3, the authors speculate that mothers in the intervention group are "more accurate observers and reporters of their children's health-related behaviors" (McCarton 1997). No differences in illness or number of hospitalizations were noted for this group. The success of the IHDP on cognitive outcomes may have been associated with the frequency with which families participated in program interventions. For example, Ramey et al. (1992) found that more-frequent participation resulted in significantly higher IQ scores than did less-frequent participation. Similarly, Blair, Ramey, and Hardin (1995) found that higher 36-month IQ scores in the intervention group were associated with the number of home visits received in the first year and with cumulative participation in the second and third years of the study. Importantly, the effects of the IHDP on IQ for lighter low birth weight children appeared to fade after age 3 (e.g., Brooks-Gunn et al., 1994, McCarton et al., 1997, and Hollomon and Scott, 1998). Brooks- Gunn et al. (1994) found that, for the lighter low birth weight subsample of infants, differences between groups that were present at age 3 faded by age 5 for IQ, behavior, and morbidity outcomes. For the subsample of heavier low birth weight infants, significant, positive program effects were present at both ages 3 and 5 for IQ and behavior scores. As participants aged, fewer significant differences remained between the intervention and comparison groups. At age 8 years, McCarton et al. (1997) reported no significant differences for cognitive, behavioral, or academic outcomes for the lighter low birth weight subsample of infants, while the heavier low birth weight subsample of IHDP infants retained significant gains in IQ scores and math achievement scores and had a lower percentage of children requiring special education classes. In addition, Hollomon and Scott (1998) found no significant differences between the intervention and control groups at age 9 for attendance records, rates of special education placement, reading scores, or math achievement. A number of additional evaluations have been conducted for this program (all of which included at least one of the original study authors), but we include here only those studies with methodologies meeting the Promising Practices Network criteria.

Example Sites

Little Rock, Arkansas New Haven, Connecticut Miami, Florida Cambridge, Massachusetts Bronx, New York Philadelphia, Pennsylvania Dallas, Texas Seattle, Washington

- 284 -

Contact Information

Jeanne Brooks-Gunn Columbia University Teachers College 525 West 120th Street New York, NY, 10027 Tel (212) 678-3369 Fax (212) 678-3676 brooks-gunn@columbia.edu Marie McCormick, MD, ScD Department of Maternal and Child Health Harvard School of Public Health 677 Huntington Avenue, 6th Floor Boston, MA 02115 Tel (617) 432-3759 mmccormi@hsph.harvard.edu

Available Resources

None at this time

Bibliography

Blair, Clancy, Craig T. Ramey, and J. Michael Hardin, "Early Intervention for Low Birthweight, Premature Infants: Participation and Intellectual Development," American Journal on Mental Retardation, Vol. 99, No. 5, 1995, pp. 542-554. Bradley, Robert H., Leanne Whiteside, Daniel J. Mundfrom, Patrick H. Casey, Kelly J. Kelleher, Sandra K. Pope, "Contribution of Early Intervention and Early Caregiving Experiences to Resilience in Low- Birthweight, Premature Children Living in Poverty," Journal of Clinical Child Psychology, Vol. 23, 1994, pp. 425-434. Bradley, Robert H., Margaret R. Burchinal, and Patrick H. Casey, "Early Intervention: The Moderating Role of the Home Environment," Applied Developmental Science, Vol. 5, No. 1, 2001, pp. 2-8. Brooks-Gunn, Jeanne, Cecilia M. McCarton, Patrick H. Casey, Marie C. McCormick, Charles R. Bauer, Judy C. Bernbaum, Jon Tyson, Mark Swanson, Forrest C. Bennett, David T. Scott, James Tonascia, and Curtis L. Meinert, "Early Intervention in Low-Birth-Weight Premature Infants: Results Through Age 5 Years from the Infant Health and Development Program," Journal of the American Medical Association, Vol. 272, No. 16, 1994, pp. 1257-1262. Brooks-Gunn, Jeanne, Fong-ruey Liaw, and Pamela Kato Klebanov, "Effects of Early Intervention on Cognitive Function of Low Birth Weight Preterm Infants," Journal of Pediatrics, Vol. 120, No. 3, 1992, pp. 350-359. Brooks-Gunn, Jeanne, Pamela Kato Klebanov, Fong-ruey Liaw, and Donna Spiker, "Enhancing the Development of Low-Birthweight, Premature Infants: Changes in Cognition and Behavior over the First Three Years," Child Development, Vol. 64, 1993, pp. 736-753. Fewell, Rebecca R., and Keith G. Scott, "Cost Analysis Decisions for IHDP," in Ruth T. Gross, Donna Spiker, and Christine W. Haynes, eds., Helping Low Birth Weight, Premature Babies: The Infant Health and Development Program, Palo Alto, Calif.: Stanford University Press, 1997.

- 285 -

mailto:brooks-gunn@columbia.edu

mailto:mmccormi@hsph.harvard.edu

Hediger ML, Overpeck MD, Ruan WJ, Troendle JF. "Birthweight and gestational age effects on motor and social development." Pediatric and Perinatal Epidemiology, 2002;16(1):33-46.

Hollomon, Holly A., and Keith G. Scott, "Influences of Birth Weight on Educational Outcomes at Age 9: The Miami Site of The Infant Health and Development Program," Developmental and Behavioral Pediatrics, Vol. 19, No. 6, 1998, pp. 404-410. Infant Health and Development Program, "Enhancing the Outcomes of Low-Birth-Weight, Premature Infants: A Multisite, Randomized Trial," American Medical Association Journal, Vol. 263, No. 22, 1990, pp. 3035-3042. Liaw, Fong-ruey, and Jeanne Brooks-Gunn, "Cumulative Familial Risks and Low-Birthweight Children’s Cognitive and Behavioral Development," Journal of Clinical Child Psychology, Vol. 23, No. 4, 1994, pp. 360-372. McCarton, Cecilia M., Jeanne Brooks-Gunn, Ina F. Wallace, Charles R. Bauer, Forrest C. Bennett, Judy C. Bernbaum, Sue Broyles, Patrick H. Casey, Marie C. McCormick, David T. Scott, Jon Tyson, James Tonascia, and Curtis L. Meinert, "Results at Age 8 Years of Early Intervention for Low-Birth-Weight Premature Infants," Journal of the American Medical Association, Vol. 277, No. 2, 1997, pp. 126-132. McComick, Marie C., Jeanne Brooks-Gunn, Stephen L. Buka, Julie Goldman, Jennifer Yu, Mikhail Salganik, David T. Scott, Forrest C. Bennett, Libby L. Kay, Judy C. Bernbaum, Charles R. Bauer, Camilia Martin, Elizabeth R. Woods, Anne Martin and Patrick Casey. "Early Intervention in Low Birth Weight Premature Infants: Results at 18 Years of Age for the Infant Health and Development Program," Journal of Pediatrics, Vol. 117, No. 3, March 2006, pp. 771-780. McCormick, Marie C., Cecelia McCarton, James Tonascia, and Jeanne Brooks-Gunn, "Early Educational Intervention for Very Low Birth Weight Infants: Results from the Infant Health and Development Program," Journal of Pediatrics, Vol. 123, No. 4, 1993, pp. 527-533. Ramey, Craig T., Donna M. Bryant, Barbara H. Waski, Joseph J. Sparling, Kaye H. Fendt, and Lisa M. LaVange, "Infant Health and Development Program for Low Birth Weight, Premature Infants: Program Elements, Family Participation, and Child Intelligence," Pediatrics, Vol. 89, 1992, pp. 454-465. Spiker, Donna, Joan Ferguson, and Jeanne Brooks-Gunn, "Enhancing Maternal Interactive Behavior and Child Social Competence in Low Birth Weight, Premature Infants," Child Development, Vol. 64, 1993, pp. 754-768.

Last Reviewed

January 2009

Know Your Body

Program Info

Outcome Areas Healthy and Safe Children

Indicators Youths not using alcohol, tobacco, or illegal drugs

- 286 -

Topic Areas Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School Type of Service Health Education Youth Development Type of Outcome Addressed Physical Health Substance Use and Dependence

Evidence Level Promising

Program Overview

Know Your Body (KYB), as evaluated and reported here, was a comprehensive, skills-based health promotion program that targeted students in the fourth to ninth grades. The goal of KYB was to teach students the necessary knowledge, attitudes, skills, and experience to practice positive health behaviors and reduce their risk of future illness. The program has since been modified to target kindergarten through sixth-grade students, but the modified program has not been evaluated and the information reported here is only for the evaluated, fourth-grade to ninth-grade program. The KYB curriculum focused on voluntary behavioral changes in the areas of cigarette smoking, physical fitness, and diet, and was taught by the regular classroom teacher for approximately two hours per week throughout the school year. The cigarette smoking prevention component taught students the skills necessary to resist pressures to smoke cigarettes, while the physical fitness component helped students to begin a regular program of endurance exercise designed to improve cardiovascular fitness. The nutritional component of the curriculum focused on encouraging students to adopt the American Heart Association’s "prudent diet," which consists of decreasing the consumption of total and saturated fat, cholesterol, sodium, and refined sugar; increasing the consumption of complex carbohydrates and fiber; increasing the consumption ratio of polyunsaturated fat to saturated fat; and maintaining an ideal body weight.

Program Participants

As evaluated, the program targeted fourth-grade to ninth-grade students. Currently the program targets elementary school students in grades K-6.

Evaluation Methods

The Know Your Body program has been evaluated in four major studies, all of which focused on the effects of the program on chronic disease, cardiovascular risks, and cancer-related risk factors, including smoking. Only smoking-related results from two major studies are reported here because the other studies of KYB experienced methodological problems that prevented their inclusion. A brief description of the two evaluations follows: Bronx, New York (Walter et al., 1985, Walter et al., 1988): A total of 1,563 4th-grade students in 22 elementary schools from a single school district were randomly assigned to either a treatment group (in 14 schools as part of the KYB program) or to a control group (non-participants in 8 schools). The school district was located in an inner-city urban district in the New York City borough of the Bronx.

- 287 -

The intervention program lasted a total of five years, from the fourth to eighth grades. Participants were assessed one year after the start of the program and again at five-years following completion of the intervention program. After one year, 1,115 students remained in the sample (an attrition rate of 29 percent), and 1,036 students remained after five years (a 34-percent attrition rate). The outcome measure studied was students’ levels of a biochemical marker (serum thiocyanate) that indicates cigarette smoking. Westchester County, New York (Walter et al., 1986; Walter et al., 1987; Walter et al., 1988; Walter et al., 1989): 1,105 fourth-grade students from 15 elementary schools in four school districts were randomly assigned to a treatment group (in 8 schools as part of the KYB program) or a control group (non-participants in 7 schools). The intervention lasted six years, while the students completed grades 4 through 9. Westchester County is a predominantly white, suburban, middle- to upper-middle-class area in proximity to New York City. One-, three-, five, and six-year follow-up assessments (from the beginning of the study) were conducted on this sample of children. The attrition rate at the one-year follow-up was 21 percent, 18 percent at the three-year follow-up, 20 percent at the five-year follow- up, and 35 percent at the six-year follow-up. The study outcome measures of interest were students’ levels of serum thiocyanate and their levels of saliva cotinine (another biomarker for cigarette smoking). The authors reported no significant differences between those students who left the study and those who completed it.

Key Evaluation Findings

The Bronx study (Walter et al., 1985; Walter et al., 1988) found:

• Significant differences between the intervention and control groups at the one-year follow-up. From baseline to follow-up, the average level of serum thiocyanate detected in KYB students decreased by 5 percent, while levels of serum thiocyanate for control group students increased by 9 percent.

• No significant differences between the intervention and control group at the five-year follow- up.

The Westchester County study (Walter et al., 1986; Walter et al., 1987; Walter et al., 1988; Walter et al., 1989) found:

• Significant differences between the intervention group and control students at the one-year follow-up, with intervention students demonstrating significantly lower levels of serum thiocyanate than control students. From baseline to follow-up, serum thiocyanate levels for the intervention group increased by 2 percent, while the control group showed an increase of 15 percent.

• No statistically significant differences between the intervention and control groups at the three-year or five-year follow-up.

• At the six-year follow-up, when the students were in the ninth grade, the rate of cigarette smoking as determined by thiocyanate levels was 73 percent less among intervention group students than among control group students. While 13.1 percent of participants in the control group were biochemically classified as current cigarette smokers, only 3.5 percent of participants in the KYB group were classified as such.

• When analyzed separately for gender, the intervention was found to be effective only for male students.

Probable Implementers

Public and private elementary schools

- 288 -

Funding

Developmental funding for the project included the American Health Foundation; the National Heart, Lung, and Blood Institute; the National Cancer Institute; and the W.K. Kellogg Foundation.

Implementation Detail

Program Design

Program materials for KYB were specific to each grade, and included teacher guides, student workbooks and worksheets, videotapes, posters, and calendars.

Curriculum

The curriculum was developed according to the principles of social learning theory, and each activity was designed to incorporate learning strategies to encourage behavior change, including modeling of desired behaviors, behavioral rehearsal, goal specification, feedback of results, and positive reinforcement for favorable behavior change.

Staffing

Teachers were trained in the KYB curriculum in three half-day teacher workshops. Adherence to the curriculum and teaching protocols was ascertained through teacher monitoring, which included documentation of attendance at training workshops and the number of classroom lessons taught, along with intermittent classroom visits by the research staff.

Issues to Consider

This program received a "promising" rating. The evaluations used a randomized experimental design and longitudinal follow-up, and participants in the majority of the studies experienced significantly reduced levels of smoking (as measured by serum thiocyanate levels) compared with the control group. However, findings were mixed at different evaluation times between the two major studies. While serum thiocyanate levels are one indicator of cigarette smoking, there is a concern that this measure might not fully capture experimental smoking. None of the evaluations used any other non- biochemical indicator of smoking, e.g., self-reported smoking behavior, so it is not possible to check on the validity of serum thiocyanate for measuring actual smoking levels. In addition, one longitudinal study found the program to be more effective with males than with females.

Example Sites

Bronx, New York; Los Angeles, California; Santa Monica, California; Washington, D.C.; Westchester County, New York

Contact Information

Kendall Hunt Publishing Company PreK-12 Division 4050 Westmark Drive P.O. Box 1840 Dubuque, Iowa 52004-1840 phone: 800-542-6657 fax: 800-772-9165 e-mail: orders@kendallhunt.com

- 289 -

mailto:orders@kendallhunt.com

Available Resources

A description of current K-6 program components for each grade level, and order forms for program materials, are available on the Kendall Hunt website: http://kendallhunt.com/kyb.

Bibliography

Taggart, Virginia S., Patricia J. Bush, Alan E. Zuckerman, and Patricia K. Theiss, "A Process Evaluation of the District of Columbia ‘Know Your Body’ Project," Journal of School Health, Vol. 60, No. 2, 1990, pp. 60-66. Walter, Heather J., Albert Hofman, Linda T. Barrett, Patricia A. Connelly, Kathryn L. Kost, Ellen H. Walk, and Rebecca Patterson, "Primary Prevention of Cardiovascular Disease Among Children: Three- Year Results of a Randomized Intervention Trial," in B. Hetzel and G. S. Berenson, eds., Cardiovascular Risk Factors in Childhood: Epidemiology and Prevention, New York, NY: Elsevier Science Publishers B.V. (Biomedical Division), 1987. Walter, Heather J., Albert Hofman, Patricia A. Connelly, Linda T. Barrett, and Kathryn L. Kost, "Coronary Heart Disease Prevention in Childhood: One-Year Results of a Randomized Intervention Study," American Journal of Preventive Medicine, Vol. 2, No. 4, 1986, pp. 239-245. Walter, Heather J., Albert Hofman, Patricia A. Connelly, Linda T. Barrett, and Kathryn L. Kost, "Primary Prevention of Chronic Disease in Childhood: Changes in Risk Factors After One Year of Intervention," American Journal of Epidemiology, Vol. 122, No. 5, 1985, pp. 772-781. Walter, Heather J., Albert Hofman, Roger D. Vaughan, and Ernst L. Wynder, "Modification of Risk Factors for Coronary Heart Disease," The New England Journal of Medicine, Vol. 318, No. 17, 1988, pp. 1093-1100. Walter, Heather J., Roger D. Vaughan, Ernst L. Wynder, "Primary Prevention of Cancer Among Children: Changes in Cigarette Smoking and Diet after Six Years of Intervention," Journal of the National Cancer Institute, Vol. 81, No. 13, 1989, pp. 995-999.

Last Reviewed

January 2013

LifeSkills Training

Program Info

Outcome Areas Healthy and Safe Children

Indicators Youths not using alcohol, tobacco, or illegal drugs Youths abstaining from sexual activity or not engaging in risky sexual behavior Children and youth not engaging in violent behavior or displaying serious conduct problems

Topic Areas Age of Child Middle Childhood (9-12) Adolescence (13-18)

- 290 -

http://kendallhunt.com/kyb

Type of Setting Elementary School Middle School High School Type of Service Health Education Youth Development Type of Outcome Addressed Physical Health Substance Use and Dependence Teen Sex/Pregnancy

Evidence Level Proven

Program Overview

The LifeSkills Training (LST) program is a school-based substance abuse prevention curriculum. LST originally targeted middle and junior high school students, but is now used with elementary and high school students as well. The LST program was developed in the late 1970s and aims to modify drug- related knowledge, attitudes, and norms; teach skills for resisting social influences that encourage drug use; and foster the development of general personal and social skills. The LST program originally focused on preventing cigarette smoking, and the curriculum was later expanded to include preventing the use of alcohol and other drugs. The LST curriculum is taught over three consecutive school years, beginning in the 6th and 7th grade. The program consists of 15 lessons in the first year, followed by 10 "booster" lessons during the program's second year and 5 booster lessons in the third year. The booster lessons are designed to reinforce earlier material and to provide additional opportunities for skill development and practice. There is also an LST curriculum for elementary school children, beginning in 3rd or 4th grade, and for high school students, beginning in 9th or 10th grade. Regular classroom teachers usually implement the LST curriculum; however, the program can be implemented by outside health professionals or older student peers.

Program Participants

The LST program may be used with elementary, middle and junior high, or high school students. It has been studied extensively with white, middle-class participants from suburban and rural areas of New York State, as well as with African-American and Hispanic youth in urban New York City.

Evaluation Methods

The LST program has been evaluated in various formats and for effectiveness at reducing risky behaviors in several samples of students since its inception. Program impacts have been assessed immediately following program completion and at intervals up to six years later. Implementation of the program has varied somewhat in length (12 to 20 sessions, with the average being 15) and format of program delivery (e.g., teacher-led versus peer-led, intensive mini-course versus regular weekly sessions, and implementation feedback for teachers versus no feedback). More recently, the program has been modified for use with elementary students. In addition, studies have tested the effects of an added violence-reduction component and whether LST concepts are effective when infused into the content of students' daily classroom curricula. Outcomes measured include impacts on driving behavior, HIV-risk behavior, aggression, and substance use—cigarette use only, alcohol use only, or cigarettes, alcohol, and marijuana use. A study by Botvin, Renick, and Baker (1983) sought to provide a test of the effectiveness of the LST program at preventing the initiation of cigarette smoking, compare the relative efficacy of two different scheduling formats of the curriculum, and test the effectiveness of booster sessions. The

- 291 -

sample consisted of 902 7th graders from seven public schools in suburban New York, and the majority of participating students were white and from middle- to upper-middle-class families. Schools were randomly assigned to (1) LST using an integrated weekly scheduling format (two schools), (2) LST using an intensive mini-course scheduling format (two schools), or (3) control group (three schools). We report the combined experimental group, short-term results only. After excluding pretest smokers, 831 students remained in the analysis sample. Botvin, Batson, et al. (1989) conducted a study of smoking among 608 7th grade students from nine urban junior high schools in northern New Jersey. The sample was predominantly African-American (87 percent) and 10 percent Hispanic. Three schools were randomly assigned to the treatment group, and six schools were assigned to the control group. Three months following the pretest, a total of 520 students completed the posttest. Botvin, Dusenbury, et al. (1989) assessed the effects of the LST program on smoking among 471 7th graders from eight public schools in the metropolitan New York area. Students were predominantly Hispanic (74 percent). Schools were randomly assigned to treatment and control group groups, and 73 percent (345) of students completed posttest data 3.5 months after the pretest. Based on an analysis of program implementation data—i.e., researcher observations of the extent to which the curriculum was taught and how effectively participating teachers taught it—a high-implementation subgroup and a low-implementation subgroup were formed from the original treatment group. Botvin, Baker, et al. (1990) report results from a three-year longitudinal study initiated in 1985. This study assessed LST program effects on cigarette smoking among 3,684 students, who started the study in 7th grade, from 56 predominantly white middle/junior high schools in New York State. The intervention began with 15 class sessions in year one, followed by 10 class booster sessions in year two and five sessions in year three. Schools were randomly assigned to one of three groups: (1) a prevention program with annual one-day teacher workshops and implementation feedback by project staff (workshop group, 18 schools); (2) a prevention program with teacher training provided by videotape and no implementation feedback (videotape group, 16 schools); and (3) a comparison control group (22 schools). Posttest data were collected following completion of the intervention program and analyses were conducted on only those experimental schools with implementation scores of 60 percent. A six-year follow-up study was conducted on 3,597 12th-grade students (60 percent of the initial 7th grade sample); it assessed cigarette, alcohol, and marijuana use (Botvin, Baker, et al., 1995). No differential attrition effects were found among treatment and control groups. To test the efficacy of the program on Hispanic students in a larger sample, Botvin, Dusenbury, et al. (1992) assessed smoking among 3,153 predominantly low-income, Hispanic 7th graders from 47 schools (11 public and 36 parochial schools) in New York City. Schools were randomly assigned to either a treatment group (25 schools with 1,795 students) or control group (22 schools with 1,358 students). Posttest data were collected four months after the pretest. A study by Botvin, Epstein, et al. (1997) assessed cigarette, alcohol, marijuana, and multidrug use among 833 predominantly minority (70 percent Hispanic and 26 percent African-American) 7th grade students from seven urban schools in New York City. Schools were assigned to either a treatment (LST) or control group (the latter received the standard drug prevention program in New York City schools). Posttest data were collected three months after program implementation and included 721 students (87 percent of the initial sample). A fourth LST evaluation using the Botvin, Griffin, et al. (2001a) sample focused on the effects of LST on cigarette smoking among female students only (Botvin, Griffin, et al., 1999). The sample consisted of 2,690 low-income, minority (60 percent were African-American and 23 percent Hispanic) 7th grade girls. Schools were randomly assigned to treatment and control groups, and posttest data were collected after one year. The final sample contained 2,209 students (82 percent), including 1,278 treatment group students and 931 control group students. Botvin, Griffin, et al. (2001a) examined the effects of LST among a sample of 5,222 predominantly low-income minority students in the 7th grade attending 29 New York City schools. The sample was 47 percent male, 61 percent African-American, and 22 percent Hispanic, and 62 percent of the students qualified for a free lunch at school. Prior to randomization, schools were divided into high,

- 292 -

medium, or low smoking prevalence and randomly assigned to treatment (16 schools) and control (13 schools) groups. Treatment group students received the LST program in 7th grade, followed by booster sessions in the 8th grade, while control group students received the regular public school substance abuse prevention program. Pretest data were collected in 7th grade, initial posttest data were collected three months later, and one-year follow-up data were collected when students were in 8th grade. A total of 3,621 students (69 percent of the initial sample) provided data at both initial posttest and one-year follow-up, including 2,144 students from the treatment group and 1,477 students from the control group. Attrition analyses found that pretest smokers, drinkers, and marijuana users were more likely to drop out of the study than nonusers (rates were equivalent across both experimental groups). Botvin, Griffin, et al. (2001b) followed up the same sample in the 8th and 9th grade, assessing program effects on binge drinking among 3,041 students (including 1,713 treatment group and 1,328 control group students). Of the 5,222 7th grade students who completed the pretest survey, 4,190 (80 percent) completed the one-year follow-up, and 3,041 (58 percent) completed the survey at both the one- and two-year follow-up assessments. Attrition rates were similar across both experimental conditions. A third study (Griffin et al., 2003) using the Botvin, Griffin, et al. (2001a) sample examined the effects of LST among a subsample of youth at high risk for substance use initiation. Students were identified as high "social risk" if they had friends that smoked cigarettes and drink alcohol, and at high "academic risk" if they reported poor academic achievement in school (average grades equivalent to a C or less). A total of 758 (15 percent) of the original sample of 5,222 students were classified as high risk for substance use initiation based on social and academic risk, including 426 students in the treatment group and 332 students in the control group. The sample was 49 percent male, 58 percent African-American, and 29 percent Hispanic, and 61 percent of students qualified for free lunch at school. Spoth et al. (2002) studied the effects of the LST program on student use of alcohol, cigarettes, and marijuana among 1,664 white 7th graders from 36 rural Midwestern schools. The authors combined the Strengthening Families Program for Parents and Youth 10-14 (SFP) parent and youth skills- building program with the LST curriculum, then randomly assigned 12 schools each to the (1) SFP + LST group (549 students), (2) LST-only group (621 students), or (3) control group (494 students). Participants were also given LST booster sessions when students were in the 8th grade, including four sessions for the SFP + LST group and five sessions for the LST-only group. Outcomes were examined at program completion and one year after the first posttest. At the first posttest, 517 of the SFP + LST students (94 percent), 583 of the LST-only students (94 percent), and 463 of the control group students (94 percent) remained in the study. At the one-year follow-up, 453 (83 percent) of the SFP + LST, 503 (81 percent) of the LST-only, and 416 (84 percent) of the control group students remained in the sample. Botvin, Griffin, et al. (2003) examined LST program effects on early stage tobacco and alcohol use in elementary students. A total of 426 students located in nine schools were randomly assigned to receive the LST intervention over a three-year period, with eight 30-45 minute LST sessions per year, in grades 3 to 5. The randomly assigned control group, 664 students across 11 schools, received no psychosocial prevention programming during this time. Using data from those students who were tested just before and three months after the prevention period, changes over time were compared between control group and LST group students on measures of substance use (tobacco and alcohol), attitudes toward substance use (student and peer attitudes), knowledge (substance use knowledge, advertising knowledge, decisionmaking, social skills, refusal skills), normative expectations (smoking and drinking behavior of peers, teens and adults), and psychosocial factors (risk-taking and self- esteem.) Pre- and post-intervention surveys were able to be matched for approximately 56 percent of students participating in the study. These students had lower rates of having ever smoked than students whose pre- and post-intervention surveys were unable to be matched. Of the matched survey students, 70 percent came from two-parent families, and 73 percent of children's fathers had completed high school. Forty-eight percent of these students were white, 26 percent were Hispanic and 13 percent were African-American. Prior to the intervention, students in the LST intervention group who had matched surveys were more likely to report having ever used chewing tobacco, were less likely to be white, and were more likely to come from two-parent families than control students

- 293 -

with matched surveys. Trudeau et al. (2003) collected data at three time points to measure effects of LST on 7th grade students in 36 schools in the rural Midwest. To be included in the study, schools must have had at least 20 percent of students eligible for free or reduced-price lunch, must have had a district average of less than 1,200 students per school, and must have been middle schools where grades six, seven, and eight were located on the same campus. Using a randomized block design, the study assigned schools to an LST intervention group, a control group, or another experimental condition not included in this analysis. Students in the LST intervention group received 15 40-45 minute sessions in the 7th grade and five booster sessions in the 8th grade. A total of 847 intervention and control group students across 24 schools were measured at three time points: just prior to intervention in the fall of their 7th grade year, just after intervention in the spring of their 7th grade year, and just after the booster sessions in the spring of their 8th grade year. Students who completed all three testing sessions were less likely to report substance use before the intervention study began. Seventy-seven percent of analyzed students lived with both parents, 97 percent were white, and 21 percent were eligible for free or reduced-price lunch. The study measured differences between the LST intervention group and control group in changes over time for the following measures: expectancies (negative social and personal consequences of substance use), refusal intentions (self-reported likelihood of saying no to tobacco, alcohol or marijuana), and substance initiation (has student ever used tobacco, alcohol or marijuana). Students assigned to the LST intervention group were more likely to report having ever used tobacco, alcohol or marijuana and had lower levels of negative expectancies and refusal intentions than control students before the intervention began. Griffin et al. (2004) performed an after-the-fact analysis to test whether students who had received LST programming were less likely to engage in risky driving behavior. The study reviewed DMV records for 2,042 students that had participated in a randomized LST study approximately six years earlier. Of these, 1,360 students had been in the LST intervention group and 682 had been in the control group. Using the DMV records, the study compared the presence of traffic violations and "points" on students' driver's licenses accumulated by students from the LST and control groups. Points on the driver's license accumulated based on traffic violations. For example, a student would receive 11 points on his/her driver's license for speeding 40 miles per hour above the speed limit and 3 points for driving 10 miles per hour above the speed limit. Points were cleared from DMV records 18 months after the violation occurred. Of the 2,042 students for which DMV records were available, 79 percent had no points on their DMV record and 77 percent showed no traffic violations on their record. Smith, Swisher, and Vicary (2004) compared the standard LST intervention with an I-LST, a version that infused the LST curriculum into teacher's regular classroom lessons. The study cites doing an exercise in math class, such as graphing the percentage of students who do not use marijuana, as an example of infused LST. In this study, 84 percent of LST topics were covered in the I-LST version, while 81 percent of topics were covered in the standard LST condition. To test program effectiveness, nine schools in rural Pennsylvania were randomly assigned to one of three groups: a group that would receive the standard LST intervention, a group that would receive I-LST, and a control group that would not receive any intervention. To be included in the study, schools must have been classified as having low socioeconomic status and must have had a student enrollment less than 1,000. Students were 96.6 percent white and 54.4 percent male. All students in the study were surveyed just prior to the intervention at the beginning of their 7th grade year, just after the standard LST first year session period at the end of 7th grade, and then at the end of students' 8th grade year. The total number of students who participated in the study was 234 in the LST group, 297 in the I-LST group, and 201 students in the control group; however, by the third testing period, only 659 total students had completed surveys. The study measured differences between the LST, I-LST, and control groups in changes in tobacco, alcohol, marijuana and inhalant use over time. Vicary et al. (2006) also evaluated the infused version of LST (I-LST), in rural settings with nine middle schools that had student enrollments less than 1,000 and only one middle school per district. This study compared the standard LST (which covered an average of 90 percent of lessons) with the I- LST (which covered an average of 95 percent of LST topics) and a control group whose students did not receive any LST until they were in 10th grade. Students were assigned to conditions by school, with 234 students in the standard LST condition, 297 students in the I-LST condition, and 201 students in the control condition. All students participating in the study were surveyed just before the

- 294 -

intervention period, at the end of 7th grade, at the end of 8th grade, and at the end of 9th grade. Surveys measured the students' self-reported use of alcohol, cigarette and marijuana, skills in communication, decision-making, substance refusal, media resistance, coping and assertiveness, and students' norms, attitudes and knowledge about alcohol, tobacco and marijuana. Approximately 78.9 percent of students completed surveys at all four time points, with no significant differences in response rates between experimental groups. Taking initial substance use into account, the study compared self-reported differences in survey responses between LST, I-LST and control group students. Botvin, Griffin, and Bichols (2006) studied the effects of an LST intervention adapted to address violence among adolescents. This version of LST consisted of 15 classroom sessions that included material related to violence, such as anger management and conflict resolution skills, in addition to the standard substance use curriculum. In this study, 41 New York City public and parochial schools were assigned to receive the adapted LST (a total of 2,374 students in 20 schools) or to serve as a control group (a total of 2,484 students in 21 schools). Students were surveyed in the sixth grade, before the intervention period and then once again three months after the intervention period ended. The study compared the difference in change over time between students receiving any part of the LST intervention and control students and between students receiving at least half of the LST intervention and control students. The surveys measured students' self-reported levels of verbal aggression, physical aggression, fighting behavior, and delinquency (destroying property, throwing objects, stealing or vandalizing). Of the students participating in the study, 51 percent were male, 39 percent were African-American, 33 percent were Hispanic, and 10 percent were white, and 55 percent received free or reduced-price lunch and 30 percent lived with their mothers only. While students in the LST intervention group did not differ significantly from the control group on any of the baseline measures of aggression and delinquency, there was a higher percentage of Hispanic students and a lower percentage of African-American students in the LST intervention group. Using a sample of youth from New York middle class suburban and rural areas, Griffin, Botvin, and Nichols (2006) studied the effectiveness of the standard LST intervention on reducing behaviors that put youth at risk for contracting the HIV virus. In 1985, 56 schools were randomized to a "treatment as usual" control group or to an LST intervention group, which received 15 LST sessions in 7th grade, 10 booster sessions in 8th grade, and five booster sessions in 9th grade. In 1998, 2,042 youth (63.3 percent of the original sample) who had participated in the study completed 10-year follow-up surveys; of these, 66.6 percent were from the LST intervention group. In this sample of 2,042 youth, 77.6 percent had lived in two-parent families while in junior high, 91.2 percent were white, 49.6 percent had graduated college, and the median age at 10-year follow-up was 24.6 years. In addition, students who completed the 10-year follow-up surveys were less likely to have reported smoking, drinking, or marijuana use in the initial survey, taken prior to the LST intervention. Controlling for youth characteristics present in the baseline 1985 survey, this study measured the difference in HIV risk behavior between students who had received LST and those who had not. Spoth et al. (2008) examined the long-term effects of LST through surveys taken when students were in their 12th grade year, five and a half years after baseline measurement. Students in the study came from 36 schools in the rural Midwest that enrolled less than 1,200 students and were located in districts with at least 20 percent of students eligible for the free or reduced-price lunch program. Prior to baseline measurements, schools were split into groups according to authors' ratings of risk factors present at the school. Twelve groups of three schools each were created, with one school in each group assigned to one of three experimental conditions: (1) standard LST program; (2) standard LST program plus seven sessions of another program, called the Strengthening Families Program (SFP); or (3) a control condition in which students' parents were mailed brochures on teen development. Approximately 622 students assigned to the LST group and 489 students assigned to the control group completed baseline surveys in 7th grade, prior to the study intervention. Five and a half years later, when students were scheduled to be in 12th grade, 428 students assigned to the LST group and 347 students assigned to the control group completed follow up surveys. There were no significant differences in study dropout rates between students assigned to LST group and those assigned to control group. Surveys measured whether students had ever used alcohol, tobacco or marijuana, how often students used each of the three substances and whether students used at least two substances at the same time. Using all students that had completed surveys, regardless of their participation in

- 295 -

their assigned intervention, this study measures the difference between LST and control students in substance use change over time.

Key Evaluation Findings

Tobacco Use LST program studies that measured tobacco use included analyses of the program's direct and mediating effects on adolescent cigarette smoking or tobacco use. Cigarette smoking was usually assessed in terms of past-month, past-week, or past-day smoking. Mediating effects were typically assessed by surveying students' skills—for example, their ability to resist media influences—and students' norms and attitudes, such as expectations about smoking or alcohol use. The majority of the findings indicate that the LST program significantly lowers the risk that participating students will become new or frequent users of cigarettes. Short-Term Effects:

Botvin, Renick, and Baker (1983) found that, shortly after LST program completion, significantly more control group students (13 percent) than the combined experimental group students (6 percent) reported smoking during the past month. There were no significant differences among groups for weekly or daily cigarette use. Botvin, Batson, et al. (1989) found that, following program completion, significantly fewer students in the LST treatment group than in the control group reported smoking in the past month, but they found no significant differences between groups for "current" smoking or smoking in the past week or past day. Botvin, Dusenbury, et al.'s (1992) four-month posttest found that there was significantly less past- month smoking for the LST treatment group than control group students (5.2 percent versus 7.2 percent). There were no significant differences between groups for smoking during the past week or the past day. Botvin, Epstein, et al. (1997) reported that, at the three-month posttest, students in the LST treatment group used cigarettes significantly less frequently than students in the control group. Among elementary students, Botvin, Griffin, et al. (2003) found that a significantly lower proportion of students at LST schools reported smoking in the past year and LST students smoked less often than control students. Long-Term Effects:

Botvin, Griffin, et al.'s 1999 study of female students one year after program initiation found that significantly more control group than intervention group students reported:

• "ever" having smoked a cigarette (34.5 percent versus 28.3 percent)

• having initiated cigarette smoking (23.9 percent versus 19.6 percent)

• having smoked during the past month (12.3 percent versus 8.8 percent)

• having increased smoking frequency to "monthly" smoking (9.9 percent versus 6.7 percent).

While Botvin, Griffin, et al. (2001a) found no significant differences between treatment and control groups for frequency or quantity of cigarette smoking at three-month posttest, at one-year follow-up, treatment group students showed a lower frequency and a lower quantity of smoking than did control group students.

- 296 -

Griffin et al. (2003) reported significantly lower rates of smoking at one-year follow-up among LST treatment group students than control group students. Alcohol Use Several LST program studies also included analyses of program effects on alcohol use. The majority of the findings indicate that the program significantly lowers the risk that participating students will become new or frequent users of alcohol. Short-Term Effects:

Botvin, Epstein, et al.'s 1997 study found that, at three-month posttest, students in the LST treatment group had significantly more positive outcomes than control group students for less frequency of alcohol use, drank significantly less alcohol when drinking, and had lower rates of drunkenness. In addition, Botvin, Griffin, et al. (2001a) that, at three-month posttest, after taking into account the effects of differences between the 29 schools participating in the study, LST treatment group students exhibited a lower frequency of drunkenness than did control group students. No significant differences were found between groups for frequency of drinking or for quantity of alcoholic beverages consumed when drinking. Among elementary students, Botvin, Griffin, et al. (2003) found marginally significant positive effects of the LST program on students' anti-drinking attitudes, marginally significant positive effects on students' beliefs about drinking "norms" among peers, and marginally significant reductions in the proportion of students reporting drinking alcohol in the past year, compared to control students. Botvin, Griffin, et al. (2003) found significant positive effects on student's friends' anti-drinking attitudes for students who participated in LST schools, compared with control students. Long-Term Effects:

Botvin, Griffin, et al.'s (2001b) study on binge drinking found:

• At one-year follow-up (8th grade), the proportion of binge drinkers in the LST treatment group was significantly less than the proportion in the control group, with rates of 1.8 percent and 4.3 percent, respectively.

• At two-year follow-up, the proportion of binge drinkers in the LST treatment group remained significantly lower than that of the control group (2.2 percent compared with 5.2 percent).

At one-year follow-up, Botvin, Griffin, et al. (2001a) found that LST treatment group students reported significantly less frequent drinking, less frequent drunkenness, and a lower quantity of alcohol consumed per occasion than did control group students. Finally, Griffin et al. (2003) reported significantly lower rates of drinking at one-year follow-up among LST treatment group students than among control group students. Marijuana Use LST program studies also measured the program's impact on students' marijuana use. Overall, results suggest the program was moderately effective at reducing marijuana use initiation and frequency of use, although findings were not as consistent as those for cigarettes and alcohol. Short-Term Effects:

Botvin, Epstein, et al.'s 1997 three-month posttest found that students in the LST treatment group reported significantly less frequent use of marijuana than students in the control group (mean score of 1.16 versus 1.26 on a nine-point scale assessing frequency of use).

- 297 -

Spoth et al. (2002) found significant program effects for rates of new users of marijuana. The authors reported significant differences between the LST + SFP and control groups (4.1 percent versus 7.9 percent) and between the LST-only and control groups (4.3 percent versus 7.9 percent). After taking into account the effects of differences between the 29 schools participating in the study, Botvin, Griffin, et al. (2001a) did not find any differences between treatment and control groups for frequency of marijuana use at either three-month posttest or at one-year follow-up. Similarly, Griffin et al. (2003) did not report any significant differences between groups for frequency of marijuana use. Long-Term Effects:

The 6.5-year longitudinal study (Botvin, Baker, et al., 1995) reported that, at six years, there were no significant differences among groups for monthly or weekly marijuana use. "Any" Substance Use and Other Outcomes

Self-Esteem: Among elementary students, Botvin, Griffin, et al. (2003) found that students assigned to LST schools scored significantly higher on self-reported measures of self-esteem than control group students. Any substance use: Trudeau et al. (2003) found that, by the spring semester of 8th grade, rural Midwestern students who received LST treatment showed significantly higher growth in substance use refusal intentions than control students and significantly lower growth in substance use initiation. In addition, by 8th grade LST treatment students had marginally higher growth in negative expectancies of substance use than control students. Violence: Botvin, Griffin, and Nichols (2006) found that, shortly after the intervention period, youth assigned to the modified (for violence prevention) LST program reported significantly lower rates of delinquency, frequent fighting and frequent delinquency in the past year. HIV-Risk Behavior: Griffin, Botvin, and Nichols (2006) found that, ten years after receiving the standard LST intervention, young adults who had participated in the LST group reported significantly lower rates of behavior putting them at risk for contracting HIV than control group young adults (10.3 percent versus 13.6 percent.) Risky Driving: Griffin et al. (2004) found that significantly fewer students who had participated in the LST group in middle school had driving violations on their DMV records six years after the intervention (25 of control students versus 20 percent of LST students). Also, significantly fewer students who had participated in the LST group had negative points on their driver's licenses six years after the intervention (23 percent of control students versus 18 percent of LST students). Implementation Effects High Implementation:

Botvin, Dusenbury, et al. (1989) found that the high-implementation group reported marginally significantly lower rates of smoking during the past month compared with control group students (with a probability of error of 0.10). No significant differences were found in past-month smoking between the low-implementation group and the control group, or among any of the groups for smoking rates over the past week or past day. Program Format:

Teacher training mode: Botvin et al. (1990) found that, at the end of the third year of intervention, students in both experimental conditions were significantly less likely to report smoking than students

- 298 -

in the control condition. Additionally, students in the teacher workshop condition were significantly less likely to report marijuana use than control students. The 6.5-year longitudinal study (Botvin, Baker, et al., 1995) found significantly less cigarette smoking among adolescents in the workshop and videotape treatment groups than in the control group for the monthly smoking measure, as well for as the weekly smoking measure. Additionally, the prevalence of heavy smoking (pack-a-day) was significantly lower for the videotape group (but not the workshop group) than the control group. Also, at six years, the prevalence of monthly drunkenness was significantly lower for adolescents in both the workshop and videotape intervention groups than those in the control group. There were no significant differences among groups for monthly, weekly, or heavy drinking. Life Skills Training Plus Strengthening Families Program:

Spoth et al. (2002) found that the LifeSkills Training + Strengthening Families Program (LST + SFP) treatment group reported significantly lower rates of alcohol use than the control group (25.7 percent versus 36.7 percent) and the LST-only group (25.7 percent versus 35.2 percent). There were no significant differences between the LST-only students and the control group students. Spoth et al. (2008) found that, five and a half years after baseline, the rate of change in alcohol initiation was significantly lower for students in the LST and LST + SFP groups, compared with control students. Both of these treatment groups also showed significantly lower rates of increase in drunkenness initiation, compared with control students. Further, the LST group students showed overall lower rates of drunkenness initiation than both the control students and students in the LST + SFP. Overall rates of cigarette initiation and rates of growth in cigarette initiation were significantly lower for both LST and LST + SFP students than control students, at the five and a half year follow up. Overall rates of marijuana initiation were also lower for both LST and LST + SFP students at the five and a half year follow up. LST students also had significantly lower rates of growth in cigarette initiation. I-LST:

Smith, Swisher, and Vicary (2004) found no program effects of LST or I-LST on any tested variables. For females, the LST program was found to significantly lower alcohol use, binge drinking, marijuana use, and inhalant use one year after baseline. One year after baseline, females in the I-LST group showed significant improvements in binge drinking, smoking, and marijuana use. Two years after baseline, no significant effects from LST were detected for males or females. However, females in the I-LST program showed significantly lower cigarette use than control females two years after baseline. At the end of 7th grade, Vicary et al. (2006) found that females in both the LST and I-LST groups reported significantly lower binge drinking and marijuana use than control students. The LST program females also reported significantly lower overall alcohol use than both I-LST and control students at the end of 7th grade, while I-LST females reported lower overall smoking than the other two groups at this time. At the end of students' 8th grade year, I-LST females continued to show significantly lower overall smoking than both LST and control females. By the end of students' 9th grade year, neither I-LST nor LST programs showed significant effects on substance use for any of the students. For mediating factors, the study found that females in LST showed significantly improved decisionmaking, communication, coping, norms, attitudes, and knowledge regarding substance use at

- 299 -

the end of 7th grade; significantly improved media resistance, assertiveness, and coping at the end of 8th grade; and significantly improved communication and coping by the end of 9th grade. LST males did not show any significant improvements on any of the mediating factors at any time point. I-LST females showed significant improvements in substance use knowledge at the end of 7th grade, significant improvement in coping and norms regarding substance use at the end of 8th grade, and significant improvement in substance use knowledge at the end of 9th grade. I-LST males showed significant improvement in substance use norms at the end of 7th grade only.

Probable Implementers

Elementary schools, middle schools, and high schools

Funding

Funding for LST programs comes from a variety of sources, including the U.S. Department of Education (Safe and Drug-Free Schools); the U.S. Office of Juvenile Justice; the U.S. National Guard; state and local school, mental health, and parks and recreation funds; and from PTAs and other local organizations. The complete LST curriculum consists of a Teacher's Manual, Student's Guide, and a relaxation tape available through the National Health Promotion Associates (NHPA). A version of the program is available for both middle and elementary school students. Materials can be found on the Life Skills Training website (http://www.lifeskillstraining.com/), under "Programs." Cost of materials range from $50 to over $1,000 (in 2011 dollars). The NHPA also provides online, on-site or one- or two-day open training workshops. Prices begin at $235 (in 2011 dollars) for online or one-day open training.

Implementation Detail

Program Design

The LST program uses social-resistance skills training and techniques designed to develop adolescents' key personal and social skills. The prevention curriculum addresses the major cognitive, attitudinal, psychological, and social factors related to adolescent cigarette smoking. LST incorporates five major components that:

• Provide information concerning the short-term consequences of substance use, prevalence rates, and the current social acceptability of smoking and other drug use

• Facilitate critical thinking and independent decisionmaking

• Help students develop skills for coping with anxiety

• Teach social skills and assertiveness skills

• Facilitate self-improvement and a sense of personal control.

Curriculum

A combination of teaching techniques is used to teach substance use-prevention skills to students, including group discussion, demonstration, modeling, behavioral rehearsal, feedback and reinforcement, and behavioral "homework" assignments for out-of-class practice. In addition to teaching skills for the enhancement of personal and social competence, the LST program teaches students specific skills related to resisting pressures for substance use, including how to apply

- 300 -

http://www.lifeskillstraining.com/

assertiveness skills in situations in which they might experience pressure to smoke, drink, or use drugs. Unlike traditional prevention approaches, only minimal information concerning the long-term health consequences of drug use is provided. Instead, information hypothesized to be more relevant to prevention is given, such as information concerning the immediate negative consequences of drug use, the addictive nature of certain substances, the decreasing social acceptability of use, and the actual prevalence rates among adults and adolescents. LST program materials and curriculum length vary by targeted population: elementary, middle or junior high, and high school. Program components for all three include a teacher's manual with detailed lessons plans and student guide. High school materials also include a companion website, while the junior high program components include a stress management CD-ROM in addition to the website, teacher manual, and student guide. Staffing

Regular classroom teachers usually teach the LST program. Peer leaders may be used to assist classroom teachers in presenting the program, or they may implement the curriculum themselves. Training is provided online, on-site, or through open training workshops provided by National Health Promotion Associates. Open training workshops are one- or two-day training sessions that familiarize teachers with the program and its rationale. Teachers also have an opportunity to learn and practice the skills necessary to deliver the curriculum.

Issues to Consider

This program received a "proven" rating. The majority of the evaluations of the LST program used a randomized experimental design, which accounted for the grouping of students in classrooms, and some studies conducted long-term follow-ups. Participants in most of the studies experienced significantly reduced levels of cigarette smoking, alcohol use, and marijuana use when compared with control groups. However, it should be noted that the program developer has been an author on the majority of evaluations of the program. A number of additional evaluations have been conducted for this program, but we included only those studies with methodologies meeting the Promising Practices Network criteria.

Example Sites

Albany, New York Syracuse, New York Long Island, New York New York City New Jersey Rural Midwest

Contact Information

National Health Promotion Associates, Inc. 711 Westchester Avenue White Plains, NY 10604 1-800-293-4969 Phone: (914) 421-2525 or (800) 293-4969 fax: (914) 421-2007 e-mail: lstinfo@nhpamail.com Internet: http://www.lifeskillstraining.com

- 301 -

mailto:lstinfo@nhpamail.com

http://www.lifeskillstraining.com/

Available Resources

Information about the LifeSkills Training program, program materials, and information about provider training can be found on the LifeSkills Training website: http://www.lifeskillstraining.com

Bibliography

Botvin, Gilbert J., Eli Baker, Linda Dusenbury, Elizabeth M. Botvin, and Tracy Diaz, "Long-Term Follow- up Results of a Randomized Drug Abuse Prevention Trial in a White Middle-Class Population," Journal of the American Medical Association, Vol. 273, No. 14, 1995, pp. 1106-1112. Botvin, Gilbert J., Eli Baker, Linda Dusenbury, Stephanie Tortu, and Elizabeth M. Botvin, "Preventing Adolescent Drug Abuse through a Multimodal Cognitive-Behavioral Approach: Results of a 3-Year Study," Journal of Consulting and Clinical Psychology, Vol. 58, No. 4, 1990, pp. 437-446. Botvin, Gilbert J., Horace W. Batson, Sylvia Witts-Vitale, Valerie Bess, Eli Baker, and Linda Dusenbury, "A Psychosocial Approach to Smoking Prevention for Urban Black Youth," Public Health Reports, Vol. 104, No. 6, 1989, pp. 573-582. Botvin, Gilbert J., Jennifer A. Epstein, Eli Baker, Tracy Diaz, Michelle Ifill-Williams, "School-Based Drug Abuse Prevention with Inner-City Minority Youth," in Gilbert J. Botvin and Steven Schinke, eds., The Etiology and Prevention of Drug Abuse Among Minority Youth, Binghamton, N.Y.: The Haworth Press, Inc., 1997, pp. 5-19. Botvin, Gilbert J., Kenneth W. Griffin, and Tracy Diaz Nichols, "Preventing Youth Violence and Delinquency through a Universal School-Based Prevention Approach," Prevention Science, Vol. 7, 2006, pp. 403-408. Botvin, Gilbert J., Kenneth W. Griffin, Elizabeth Paul, and Araxi P. Macaulay, "Preventing Tobacco and Alcohol Use among Elementary School Students through Life Skills Training," Journal of Child and Adolescent Substance Abuse, Vol. 12, No. 4, 2003, pp. 1-17. Botvin, Gilbert J., Kenneth W. Griffin, Tracy Diaz, and Michelle Ifill-Williams, "Drug Abuse Prevention among Minority Adolescents: Posttest and One-Year Follow-Up of a School-Based Preventive Intervention," Prevention Science, Vol. 2, No. 1, 2001a, pp. 1-13. Botvin, Gilbert J., Kenneth W. Griffin, Tracy Diaz, and Michelle Ifill-Williams, "Preventing Binge Drinking During Early Adolescence: One- and Two-Year Follow-Up of a School-Based Preventive Intervention," Psychology of Addictive Behaviors, Vol. 15, No. 4, 2001b, pp. 360-365. Botvin, Gilbert J., Kenneth W. Griffin, Tracy Diaz, Nicole Miller, and Michelle Ifill-Williams, "Smoking Initiation and Escalation in Early Adolescent Girls: One-Year Follow-up of a School-Based Prevention Intervention for Minority Youth," Journal of the American Medical Women's Association, Vol. 54, 1999, pp. 139-143. Botvin, Gilbert J., Linda Dusenbury, Eli Baker, Susan James-Ortiz, and Jon Kerner, "A Skills Training Approach to Smoking Prevention among Hispanic Youth," Journal of Behavioral Medicine, Vol. 12, No. 3, 1989, pp. 279-296. Botvin, Gilbert J., Linda Dusenbury, Eli Baker, Susan James-Ortiz, Elizabeth M. Botvin, and Jon Kerner, "Smoking Prevention among Urban Minority Youth: Assessing Effects on Outcome and Mediating Variables," Health Psychology, Vol. 11, No. 5, 1992, pp. 290-299. Botvin, Gilbert J., Nicole L. Renick, and Eli Baker, "The Effects of Scheduling Format and Booster Sessions on a Broad-Spectrum Psychosocial Smoking Prevention Program," Journal of Behavioral Medicine, Vol. 6, No. 4, 1983, pp. 359-379. Griffin, Kenneth W., Gilbert J. Botvin, and Tracy R. Nichols, "Effects of a School-Based Drug Abuse

- 302 -

http://www.lifeskillstraining.com/

Prevention Program for Adolescents on HIV Risk Behavior in Young Adulthood," Prevention Science, Vol. 7, No. 1, 2006, pp. 103-112. Griffin, Kenneth W., Gilbert J. Botvin, and Tracy R. Nichols, "Long-Term Follow-Up Effects of a School- Based Drug Abuse Prevention Program on Adolescent Risky Driving," Prevention Science, Vol. 5, No. 3, 2004, pp. 207-212. Griffin, Kenneth W., Gilbert J. Botvin, Tracy R. Nichols, and Margaret M. Doyle, "Effectiveness of a Universal Drug Abuse Prevention Approach for Youth at High Risk for Substance Use Initiation," Preventive Medicine, Vol. 36, 2003, pp. 1-7. Smith, Edward A., John D. Swisher, Judith R. Vicary, Bechtel, Lori J., Minner, Daphne, Henry, Kimberly L., and Palmer, Raymond, "Evaluation of Life Skills Training and Infused-Life Skills Training in a Rural Setting: Outcomes at Two Years," Journal of Alcohol and Drug Education, Vol. 48, No. 1, 2004, pp. 51- 70. Spoth, Richard L., Cleve Redmond, Linda Trudeau, and Chungyeol Shin, "Longitudinal Substance Initiation Outcomes for a Universal Preventive Intervention Combining Family and School Programs," Psychology of Addictive Behaviors, Vol. 16, No. 2, 2002, pp. 129-134. Spoth, Richard L., G. Kevin Randall, Linda Trudeau, Chungyeol Shin, and Redmond, Cleve, "Substance Use Outcomes 5-1/2 Years Past Baseline for Partnership-Based, Family-School Preventive Interventions," Drug and Alcohol Dependence, Vol. 96, 2008, pp. 57-68. Trudeau, Linda, Richard Spoth, Catherine Lillehoj, Cleve Redmond, and K. A. S. Wickrama, "Effects of a Preventive Intervention on Adolescent Substance Use Initiation, Expectancies and Refusal Intentions," Prevention Science, Vol. 4, No. 2, 2003, pp. 109-122. Vicary, Judith R., Edward A. Smith, John D. Swisher, Abigail M. Hopkins, Elvira Elek, Lori J. Bechtel, and Kimberly L. Henry, "Results of a 3-Year Study of Two Methods of Delivery of Life Skills Training," Health Education and Behavior, Vol. 33, 2006, pp. 325-338. Vicary, Judith R., Kimberly L. Henry, Lori J. Betcchel, John D. Swisher, Edward A. Smith, Richard Wylie, and Abigail M. Hopkins, "Life Skills Training Effects for High and Low Risk Rural Junior High School Females," Journal of Primary Prevention, Vol. 25, No. 4, 2004, pp. 399-416.

Last Reviewed

December 2011

Making Proud Choices!

Program Info

Outcome Areas Healthy and Safe Children

Indicators Youths abstaining from sexual activity or not engaging in risky sexual behavior

Topic Areas Age of Child Middle Childhood (9-12) Adolescence (13-18)

- 303 -

Type of Setting Middle School High School Out of School Time Community-Based Service Provider Health Care Provider Type of Service Health Education Youth Development Type of Outcome Addressed Physical Health Teen Sex/Pregnancy

Evidence Level Promising

Program Overview

Making Proud Choices! (MPC) is an eight-hour, multi-module, sex decisionmaking intervention which was designed to be educational and, at the same time, entertaining and culturally sensitive. It was designed to provide young adolescents with the knowledge, confidence, and skills necessary to reduce their risk of sexually transmitted diseases (STDs), HIV, and pregnancy by abstaining from sex or using condoms if they choose to have sex. It is based on cognitive-behavioral theories, findings from focus groups, and the authors' extensive experience working with youth. It is an adaption and extension of the Be Proud! Be Responsible! (BPBR) curriculum (also listed on PPN), integrating teen pregnancy prevention in addition to the HIV/STD prevention components. The curriculum involves group discussions, videos, games, brainstorming, experiential exercises, and skill-building activities. Participants in the program work in groups of six to eight teens and are led by a trained facilitator. The curriculum incorporates the BPBR theme, which encourages the participants to be proud of themselves and their community, to behave responsibly for the sake of themselves and their community, and to consider their goals for the future and how unhealthful behavior might impede those goals.

Program Participants

Adolescents in a school or after-school setting.

Evaluation Methods

Jemmott et al. (1998) examined the relative effects of MPC compared with a control population that received a health promotion intervention. The participants were 659 African American adolescents recruited from 6th-grade and 7th-grade classes in three middle schools serving low-income African American communities in Philadelphia. Participation in the program was voluntary. Fifty-three percent of the sample was female, and 26.8 percent of the sample lived with both parents. At baseline, 22 percent of participants in the MPC group and 24 percent of participants in the control group reported ever having had sexual intercourse. Just over 15 percent of all the participants reported having had intercourse in the previous three months. Participants were grouped based on gender and age and randomly assigned to one of the three groups. Participants in all three groups completed pre- intervention questionnaires, questionnaires immediately following completion of the program, and questionnaires at 3-, 6-, and 12-month follow-ups. The questionnaires assessed sexual behaviors and attitudes during the previous three months. Analyses of baseline data showed that baseline condom-use knowledge was significantly higher in the MPC group than in the control group. No other differences among groups on baseline measures of conceptual variables, sexual behavior, or demographic variables were significant. Baseline condom-

- 304 -

http://www.promisingpractices.net/program.asp?programid=30

use knowledge was associated with three outcome variables: attitude toward sexual intercourse, intention to have sexual intercourse, and HIV risk-reduction knowledge. Researchers used baseline condom-use knowledge as a covariate in analyses of these outcome variables to control for this incomparability across the groups. Of the original sample, 96.5 percent completed the 3-month follow-up, 94.4 percent completed the 6- month follow-up, and 92.6 percent completed the 12-month follow-up. Attrition analysis indicated that returnees at the 3-month follow-up scored higher in pre-intervention condom beliefs than did non- returnees, indicating that they had stronger beliefs that condoms could prevent the spread of HIV and STDs as well as pregnancy when used correctly. Returnees at the 6-month follow-up scored higher in abstinence prevention beliefs and scored lower in condom-use knowledge than did non-returnees. There were no significant differences in attrition rates between the experimental and control groups.

Key Evaluation Findings

Research by Jemmott et al. (1998) found:

• At 3-months post-intervention:

o Participants in the MPC group were significantly less likely than their control counterparts (4.0 percent versus 11.6 percent) to report having had unprotected sexual intercourse in the previous three months.

o Participants in the MPC group were significantly more likely than controls to report consistent condom use (65.6 percent versus 36.1 percent of sexually active participants) and a higher frequency of condom use (4.22 versus 3.56 on a scale from 1 (never) to 5 (always)) among sexually active participants.

• At 6-months post-intervention:

o Among teens who were sexually experienced at pre-intervention, those in the MPC group reported having sexual intercourse on significantly fewer days (0.6 days) in the previous three months than the control group (4.5 days).

o MPC participants reported a significantly higher mean frequency of condom use than controls (3.99 versus 3.25 on a scale from 1 (never) to 5 (always)).

• At 12-months post-intervention:

o MPC participants reported a significantly higher mean frequency of condom use than controls (4.15 versus 3.16 on a scale from 1 (never) to 5 (always)).

o Among participants who were sexually experienced at baseline, MPC participants were significantly less likely than controls to report unprotected intercourse (9.7 percent versus 31.6 percent).

o Among teens who were sexually experienced at pre-intervention, those in the MPC group reported having sexual intercourse on significantly fewer days (1.3 days) in the previous three months control group (3.8 days).

Probable Implementers

Middle and high schools, adolescent health clinics, and youth service organizations.

Funding

Funding for initial program implementation and research was provided by the American Foundation for AIDS Research and the U.S. National Institutes of Health.

- 305 -

Implementation Detail

Program Design

• Program materials are culturally and ethnically specific.

• Multiple methods of instruction are used to keep participants engaged in the program.

Curriculum

Making Proud Choices! is a safer-sex intervention. The scope and goals of this program are essentially the same as the original Be Proud! Be Responsible! curriculum. The primary difference is that the Making Proud Choices! curriculum places a heavier emphasis on safer sex than the other curricula. It is an eight-hour program designed to be implemented over multiple sessions. All facilitators are trained in the program curriculum prior to implementation. The level of recommended training varies according to the facilitator's background in HIV/AIDS education and knowledge of teenage sexuality. The length of the training programs ranges from 16 to 24 hours. Training includes proper implementation methods, review of HIV/AIDS knowledge, and review of curriculum content. During the training sessions, facilitators participate in the experience of the curriculum as though they were students. In addition, facilitators are given an opportunity to practice their instruction skills and are provided with feedback on their performance.

Staffing

The program is typically staffed by educators, community mental health workers (for example, social workers), or nurses.

Issues to Consider

This program received a "promising" rating. The 1998 study examined the impact of participation in MPC and found that while MPC does seem to decrease risky behaviors, it appears to have had a consistent and meaningful effect only on teens who were sexually active at the start of the program. Additionally, it is important to note that the MPC program designers conducted the evaluation discussed above. The Jemmott et al. (1998) study not only compared MPC recipients with a control group but also with recipients of an abstinence-based intervention called Making a Difference!, which is an abstinence- oriented intervention. The positive outcomes from Making a Difference! were minimal and disappeared after the initial 3-month follow-up.

Example Sites

Philadelphia, Pennsylvania, and Trenton, New Jersey

Contact Information

Select Media www.selectmedia.org Marketing Department 375 Greenwich St, suite 828 New York, NY 10013 800-707-6334 212-941-3997 (fax) beth@selectmedia.org or tyree@selectmedia.org

- 306 -

http://www.selectmedia.org/

mailto:beth@selectmedia.org

mailto:tyree@selectmedia.org

Ordering Department Sophie Ampel 845-774-7335 845-774-2945 (fax) sophie@selectmedia.org For training information, contact Lynette Gueits at 703-867-9691.

Available Resources

Bibliography

Jemmott, John B. III, Loretta Sweet Jemmott, Geoffrey T. Fong, and Konstance McCaffree, "Abstinence and Safer Sex: HIV Risk-Reduction Interventions for African American Adolescents," Journal of the American Medical Association, Vol. 279, No. 19, 1998, pp. 1529-1536.

Last Reviewed

February 2013

Midwestern Prevention Project/Project STAR

Program Info

Outcome Areas Healthy and Safe Children

Indicators Youths not using alcohol, tobacco, or illegal drugs

Topic Areas Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Middle School Type of Service Parent Education Youth Development Type of Outcome Addressed Physical Health Substance Use and Dependence

Evidence Level Proven

- 307 -

mailto:sophie@selectmedia.org

Program Overview

The Midwestern Prevention Project (MPP) is a comprehensive, community-based prevention program that is intended to reduce the use of cigarettes, alcohol, and marijuana among adolescents. MPP began in 1984 in Kansas City, Kansas and Missouri, where it is known as Project STAR (Students Taught Awareness and Resistance). The program was later implemented in Indianapolis and Marion County, Indiana. The primary component of MPP is a school program offered to sixth and seventh graders, supported by four other components: a parent program, a community organization program, a program aimed at changing local health policy, and mass-media events. About 30 mass-media events, ranging from news articles to conferences, occurred every year. Further explanation of the program components is available in the Implementation Detail section below. The program's components are designed to address the multiple influences that adolescents face related to drug use. On the demand side, the program tries to change behavior through teaching adolescents resistance skills in the school program. On the supply side, the program tries to change the environment by involving the entire community in drug-prevention activities. The different components of Project STAR are introduced sequentially over the course of several years, beginning with the school program. Currently, program replication materials are not available for purchase.

Program Participants

The program targets all students within middle schools, with the intention of preventing drug abuse during the early adolescent risk period.

Evaluation Methods

Several evaluations draw from the sample of schools participating in Project STAR when it was first implemented with a cohort of sixth and seventh graders in 1984 and 1985. These students attended a total of 50 middle and junior high public schools in the Kansas City area. Of the 50 schools, 16 had scheduling flexibility that allowed them to be randomly assigned to serve as a control school or to receive treatment. Half of these 16 schools were randomly assigned to receive the intervention. Thirty-four schools were not able to accommodate random assignment. Of these, 15 schools implemented the intervention and 19 schools served as controls. Between 1985 and 1986, six schools closed and two missed data collection times, leaving 42 schools for which data was collected at baseline. This left 24 schools that implemented the MPP intervention and 18 schools that served as a comparison control group. Students in all schools were potentially exposed to the multimedia events and to community leader organization efforts, because they were implemented throughout the Kansas City area. All of the Kansas City evaluations relied on students' self-reported drug use through 133-item questionnaires, which asked about alcohol, cigarette, and marijuana use; demographic characteristics; and psychosocial variables. Questionnaires were administered in Kansas City at baseline—either sixth or seventh grade in 1984—and annually thereafter until 1987. A carbon monoxide breath kit was used at each questionnaire administration to verify accuracy of self-reported drug use. In eight of the 42 Kansas City schools, questionnaires were collected from all students in the relevant grade cohort each year. In the remaining 34 schools, a random sample of 25 percent of the relevant grade cohort was collected each year, but this sample did not consist of the same students each year. No significant differences were found between program and control schools for lifetime drug use or demographic characteristics at baseline. Based on the eight schools in which all of the students within the relevant grade level were tracked over time, no significant differences in the number of students who failed to complete questionnaires at one year follow-up were found between control and program schools.

- 308 -

http://www.promisingpractices.net/program.asp?programid=72%23implementation

Using schools as the unit of analysis, Pentz et al. (1989a) examined overall program effects in all 42 Kansas City schools after one year, controlling for urbanicity, socioeconomic status, and race. By 1985, one year after baseline, mass media and the 10-week classroom and homework components of MPP had been implemented. Program effects were measured using the difference between changes in drug use observed in the program schools and in the control schools, one year after baseline measurements. Because all students across experimental conditions had been exposed to media events, the measured differences capture the effect of only the classroom and homework components of MPP. Pentz et al. (1989c) studied the effects of MPP on cigarette smoking two years after Kansas City students began the program in middle school or junior high. Using district feeder patterns, 47 high schools were merged with baseline schools to test program effects after two years. By this time, students in the intervention had received 10 sessions of the classroom program plus homework, a five-session booster classroom program plus homework, parent training program for parents, and 70 media items and events. Students who were not in the MPP schools were also exposed to the MPP mass media component. Program effects on the proportion of students at each school who smoked cigarettes monthly, weekly, and daily were measured. Pentz et al. (1989d) examined the effects of MPP on cigarette smoking in the eight panel schools at six months, one year, and two years after baseline. A total of 1,122 students in the panel schools were surveyed at all time points. By two years after baseline, the MPP intervention had delivered the 10 classroom sessions, five booster sessions, parent training and parent organization, and media campaigns. In the eight panel schools, the control group had a significantly higher number of students who had ever smoked than the intervention group. Pentz et al. (1989b) tested whether MPP had an effect on cigarette smoking, alcohol use, or marijuana use in students attending the 42 study schools at six months, one year and two years after baseline. Substance use was measured as the change in the percentage of students who reported using each substance in the past week and past month. MacKinnon et al. (1991) used questionnaire data collected at baseline (1984) and at one year follow- up (1985) from the original 42 Kansas City schools to identify program impacts on factors mediating drug use, such as intentions to use, beliefs about positive and negative consequences of drug use, communication skills, and friends' reactions to drug use. Tests of baseline characteristics showed that students in program schools had greater resistance skills and stronger beliefs about negative consequences of marijuana use than students in control schools, indicating that the two groups of students were not entirely comparable before they received treatment. Two more recent studies have looked at specific aspects of the program in the Indiana replication. Fifty-seven schools located within 12 districts in Marion County, Indiana were in the study. In the fall of 1986, 32 schools were randomly assigned to use the MPP program for sixth and seventh graders and 25 schools were randomly assigned to serve as a comparison group. Treatment and comparison group sizes are unequal because some districts did not include an even number of schools. In these districts, the odd school was included in the treatment group. Classroom and media components of MPP began in the treatment group schools during fall 1987, while the remaining three components— parent education, policy change, and community organization—began in the 1988-1989 school year. All schools received the parent component in 1988-1989, regardless of whether the schools were in the treatment or control group. In addition, control schools began receiving the school-based program in the 1988-1989 school year. Thus, any student entering sixth or seventh grade in 1986-1987 would not go through the MPP curriculum, a student entering sixth or seventh grade in 1987-1988 within a treatment group school would go through the MPP curriculum, and any student entering sixth or seventh grade in any of the 57 schools in 1988-1989 would go through the MPP curriculum. In a study of the Indiana replication, Chou et al. (1998) tested whether the Prevention Project reduced use of cigarettes, alcohol, and marijuana among students who were already using one or more of these substances. Data were collected from a random sample of classrooms representing one-third of all students across the 57 schools, using a 100-item questionnaire that measured substance use, demographics, attitudes, and social influences. Students were tracked for four follow-up periods: six months after baseline, one and a half years after baseline, two and a half years after baseline, and

- 309 -

three and a half years after baseline. This study examined whether students in the program showed greater decreases than control students in their substance use over time. Effects were measured by comparing an individual's reported substance use at each baseline to reported use at each follow-up. The researchers compared changes in self-reported substance use over time among the control and program students. It should be noted that control students who reported alcohol use at baseline failed to complete questionnaires at higher rates than program students at the second follow-up. Chou et al. (2004) examined the effects of MPP on cigarette use among students in the Indianapolis study schools. This study compared the growth in cigarette use of students who smoked cigarettes in the past month between the MPP intervention and control groups. Measurements were taken at baseline, six months later, and then annually for five years.

Key Evaluation Findings

Pentz et al. (1989a) found that:

• While prevalence of substance use had increased for both groups at the one-year follow-up, the increase in proportion reporting past month substance use among students exposed to the MPP program was lower than for students not exposed. The reported increases in past month use was as follows: cigarettes (15% vs. 22%), alcohol (9% vs. 12%) and marijuana (4% vs. 7%).

• Preliminary results from students who were tracked over time suggest program effects were maintained at the two-year follow-up questionnaire.

The Pentz et al. (1989c) study examined the proportion of monthly, weekly, and daily cigarette smokers and found that:

• The rate of increase in smoking at schools not in the MPP program was about 1.5 times the rate of increase in program schools. For example, after two years, students in control schools had increased their monthly smoking by 15.4 percent, compared with a 9.1 percent increase among students in program schools.

Pentz et al. (1989d) study of the eight panel schools found that:

• Significantly fewer students in the MPP intervention group reported smoking cigarettes in the past month than control students at six months, one year, and two years after baseline.

• Significantly fewer students in the MPP intervention group reported smoking cigarettes in the past week than control students at six months and one year after baseline; only marginally significantly fewer MPP intervention students reported smoking in the past week at two years.

• At two years after the MPP intervention, students in the control group were 1.4 times more likely to report having ever smoked in their lifetime, 1.7 times more likely to report smoking in the past month, and 1.6 times more likely to report smoking in the past week than intervention students.

Pentz et al. (1989b) found that:

• Students in the MPP intervention had a significantly lower increase in reported cigarette use in the past month and in the past week than students in the control group.

• Students in the MPP intervention had a significantly lower increase in reported alcohol use in the past month and in the past week.

• Students in the MPP intervention had a significantly lower increase in reported marijuana use in the past month and in the past week.

- 310 -

The MacKinnon et al. (1991) study reported that:

• Significantly fewer program students reported an intention to use cigarettes, alcohol, and marijuana within the next two months.

• Program students were significantly less likely to believe that there were positive consequences from drug use.

• Program students were more likely to report that their friends had negative views toward drug use.

In their study of substance use behavior among Indiana students already using cigarettes, alcohol, and/or marijuana, Chou et al. (1998) found that:

• Program students who reported smoking at baseline showed small but significant decreases in their use after six months. However, no significant effects were found at the next three follow- up tests.

• Program students who reported drinking alcohol at baseline showed small but significant decreases in their use after six months, small decreases again at 1.5 years, and no significant changes at the last two follow-up tests.

• Program students who reported marijuana use at baseline did not significantly differ in their use from non-program students at any follow-up.

Chou et al.'s (2004) study of past-month cigarette use among students in Indianapolis over six follow- up time periods found that:

• Students in the MPP intervention group showed a lower rate of growth in past-month cigarette use during junior high than control group students. This lower rate was only marginally statistically significant.

Probable Implementers

Middle and junior high schools

Funding

National Institute on Drug Abuse, E. M. Kauffman Foundation

Implementation Detail

Program Components

• School Program. This is a two-year program presented to students in the transition year to middle or junior high school (typically sixth or seventh grade). The first year of the school program consists of about 10 classroom sessions that focus on increasing skills to resist pressures to use drugs. A five-session booster program follows in the second year. Peer leaders help teachers facilitate the program. Methods include group discussion, role-playing, and homework to be completed with the family.

• Parent Program. This is implemented in the second year of the program and includes workshops on parenting skills and neighborhood activities, with some parents becoming involved in parent committees to organize and plan prevention activities.

• Community Organization. In the third year, the program brings together and trains community and government leaders to plan and implement drug-prevention services for youth.

- 311 -

• Policy Program. With the relevant leaders from the third component, the goal of this component is to change local ordinances to further restrict students' access to cigarettes, alcohol, and marijuana and to support prevention efforts.

• Media Program. An ongoing component of the multiyear effort, this component uses television, radio, and newspapers to introduce the program to the community, inform the community about its activities, and reinforce the program components.

To reach as many students as possible, the program is designed to be introduced in middle school or junior high school because absenteeism and dropout rates are lower in those grades than in high school.

Staffing

Regular school teachers are trained in a two-day workshop for the school program, followed by annual half-day training sessions. Principals and volunteer parents receive training during a one-day workshop to become leaders for the parent program component. The training for the community leaders takes place over several sessions.

Issues to Consider

This program received a "proven" rating. The program was studied in two locations, with thousands of sixth and seventh graders who were assigned to program and control groups. Several separate studies have shown significant effects on several positive outcomes over differing lengths of time. One potential concern regarding the methodology used in the Kansas City area is that only eight of the 42 schools were randomly assigned to either the program or control group. The others were assigned based on the flexibility of the school to incorporate the program. Although a random-assignment study is ideal, control and program groups were observably similar in student demographics and in grade level-adjusted student drug use rates at baseline. This limitation does not apply to the schools in Indiana. In Indiana, all 57 schools in the study participated in either the treatment or control group because they were randomly assigned to that group. It should also be noted that the program developer has been involved in every MPP evaluation which met PPN criteria.

Example Sites

Kansas City, Missouri; Kansas City, Kansas; and Indianapolis, Indiana

Contact Information

For more information about MPP/Project STAR, contact: Mary Ann Pentz, Ph.D. USC Norris Comprehensive Cancer Center University of Southern California 1441 Eastlake Avenue, MS-44 Los Angeles, CA 90089-9175 Phone: (626) 457-6691 Fax: (626) 457-4012 Email: pentz@usc.edu or

- 312 -

mailto:pentz@usc.edu

Gaylene Gunning USC Department of Preventive Medicine Division of Health Behavior Research 1000 South Fremont Ave., HAS 5235 Alhambra, CA 91803 Phone: (626) 457-6600 Fax: (626) 457-4012 Email: gunning@usc.edu

Available Resources

A Blueprints for Violence Prevention monograph on MPP with information on implementing the program is available from the Center for the Study and Prevention of Violence at the University of Colorado, Boulder. An overview of the project is available at http://www.colorado.edu/cspv/blueprints/modelprograms/MPP.html, and the full book may be purchased for $12 (in 2011 dollars).

Bibliography

Chou, Chih-Ping, Dongyun Yang, Mary Ann Pentz, and Yih-Ing Hser, "Piecewise Growth Curve Modeling Approach for Longitudinal Prevention Study," Computational Statistics & Data Analysis, Vol. 46, 2004, pp. 213-225. Chou, Chih-Ping, Susan Montgomery, Mary Ann Pentz, Louise Rohrbach, C. Anderson Johnson, Brian R. Flay, and David P. MacKinnon, "Effects of a Community-Based Prevention Program on Decreasing Drug Use in High-Risk Adolescents," American Journal of Public Health, Vol. 88, No. 6, 1998, pp. 944- 948. MacKinnon, David P., C. Anaderson Johnson, Mary Ann Pentz, James H. Dwyer, William B. Hansen, Brian R. Flay, and Eric Yu-I Wang, "Mediating Mechanisms in a School-Based Drug Prevention Program: First-Year Effects of the Midwestern Prevention Project," Health Psychology, Vol. 10, No. 3, 1991, pp. 164-172. Pentz, Mary Ann, C. Anderson Johnson, James H. Dwyer, David M. MacKinnon, William B. Hansen, and Brian R. Flay, "A Comprehensive Community Approach to Adolescent Drug Abuse Prevention: Effects on Cardiovascular Disease Risk Behaviors," Annals of Medicine, Vol. 21, 1989b, pp. 219-222. Pentz, Mary Ann, David P. MacKinnon, Brian R. Flay, William B. Hansen, C. Anderson Johnson, and James R. Dwyer, "Primary Prevention of Chronic Diseases in Adolescents: Effects of the Midwestern Prevention Project on Tobacco Use," American Journal of Epidemiology, Vol. 130, No. 4, 1989d, pp. 713-724. Pentz, Mary Ann, David. P. MacKinnon, James H. Dwyer, Eric Y. I. Wang, William B. Hansen, Brian R. Flay, and C. Anderson Johnson, "Longitudinal Effects of the Midwestern Prevention Project on Regular and Experimental Smoking in Adolescents," Preventive Medicine, Vol. 18, 1989c, pp. 304-321. Pentz, Mary Ann, James H. Dwyer, David P. MacKinnon, Brian R. Flay, William B. Hansen, Eric Yu I. Wang, C. Anderson Johnson "A Multi-Community Trial for Primary Prevention of Adolescent Drug Abuse: Effects on Drug Use Prevalence," Journal of the American Medical Association, Vol. 261, 1989a, pp. 3259-3266.

Last Reviewed

December 2011

- 313 -

mailto:gunning@usc.edu

http://www.colorado.edu/cspv/blueprints/modelprograms/MPP.html

Multisystemic Therapy (MST)

Program Info

Outcome Areas Healthy and Safe Children Strong Families

Indicators Youths not using alcohol, tobacco, or illegal drugs Children living in a permanent home Children and youth not engaging in violent behavior or displaying serious conduct problems

Topic Areas Age of Child Adolescence (13-18) Type of Setting High School Community-Based Service Provider Health Care Provider Type of Service Family Support Parent Education Youth Development Type of Outcome Addressed Behavior Problems Juvenile Justice Physical Health Substance Use and Dependence Violent Behavior

Evidence Level Proven

Program Overview

Multisystemic Therapy (MST) is an intensive, family-based treatment approach for improving the antisocial behavior of serious juvenile offenders. MST seeks to reduce youth criminal activity and other kinds of negative behavior (for example, drug abuse) in a cost-effective manner by limiting the need for incarceration or other types of out-of-home placement. Adaptations of MST have also been developed for child abuse and neglect, psychiatric issues, substance abuse, and problem sexual behavior. Developed in the 1970s by MST Services board director Dr. Scott Henggeler and Dr. Charles Borduin, the MST model is based on the belief that youth behavior is determined by multiple factors, such as youth's social and cognitive development, family relations, peer interactions, and community influences, and that each of these factors can be targeted to promote positive behavioral change. Thus, depending on the youth's individual circumstances, MST treatment may aim to improve a caregiver's discipline practices, decrease the youth's interaction with deviant peers, improve the youth's school performance, or aim to produce other positive results. The MST approach is guided by nine principles:

• Finding the Fit—how youth problems relate to youth's environment

- 314 -

• Focusing on Positives and Strengths—build on strengths currently present in youth's lives

• Increasing Responsibility—promote responsible behavior

• Present-Focused, Action-Oriented and Well-Defined—focus on actions that can happen immediately and have clear outcomes that can be measured

• Targeting Sequences—target the interaction between youth and external influences

• Developmentally Appropriate—appropriate to youth's age and developmental needs

• Continuous Effort—families are expected to show effort on a daily or weekly basis

• Evaluation and Accountability—the MST team is responsible for overcoming barriers to success, and intervention effects are monitored continuously

• Generalization—youth's caregivers are equipped to handle all family issues after intervention ends.

MST treatment is conducted in natural settings (for example, in the youth's home, school, or community) under the premise that youths and their families must learn how to function more effectively within their natural environment if they are to sustain improvements after treatment concludes. Specific systems to target for treatment are determined by each youth's situation; however, the focus of MST is to teach parents how to be more effective at managing their child's activities and develop positive support systems. Therapists are trained in the MST model and supervised by an MST-trained mental health professional. MST treatment typically lasts between three and five months, but can be shorter or longer than this, and involves several hours of contact per week.

Program Participants

The original version of MST targets juvenile offenders, age 12-17, and their families. The targeted youth are chronic, violent, and/or substance-abusing juvenile offenders at high risk of out-of-home placement. More recently, adaptations of MST have also been developed for youth that have experienced abuse and neglect, psychiatric issues, substance abuse, and problem sexual behavior.

Evaluation Methods

In one study (Henggeler, Melton, and Smith, 1992), 84 serious juvenile offenders and their families were randomly assigned to two groups: One group received MST, and the other received the usual services provided by the local Department of Youth Services. Going into the study, the youths averaged 3.5 previous arrests and 9.5 prior weeks of incarceration. The youths lived in South Carolina; they had an average age of 15.2 years; 77 percent were male; and 56 percent were African- American, 42 percent were white, and 2 percent were Hispanic. Before the treatment began, both groups completed a battery of tests evaluating family relations, self- reported delinquency, and other items. The same tests were given after treatment, approximately 13 weeks later. Fifty-six out of the original 84 participants completed both tests and were included in the analysis of test results. (A separate analysis determined that rates of dropout from the study did not affect the comparability of the two groups.) Fifty-nine weeks after the start of treatment, researchers examined arrest and incarceration records for all 84 youth to determine whether there were differences between the two groups. Still later, in a follow-up study (Henggeler et al., 1993), researchers analyzed arrest data for the two groups, 2.4 years after initial treatment began. In another study (Borduin et al., 1995), 176 serious juvenile offenders and their families were randomly assigned to two groups: One group received MST, and the other received individual therapy that mimicked typical community treatment for juvenile offenders in the area. Individual therapy was provided by master's-level therapists and used either a psychodynamic, client-centered, or behavioral approach. All individual therapy focused on the adolescent and ignored the external factors, or

- 315 -

"systems," surrounding youth. Going into the study, the youths averaged 4.2 previous arrests. The youths lived in Missouri; they had an average age of 14.8 years; 68 percent were male; and 70 percent were white and 30 percent were African-American. Each group completed a battery of tests prior to and approximately four months after the treatment. Of the original 176 participants, 126 completed the two sets of tests and were included in the analysis of test results. (A separate analysis determined that attrition from the study did not affect the comparability of the two groups.) Researchers also analyzed arrest data for all youths in the two groups, comparing differences four years after the study began. In a related analysis, the two groups were compared to see whether there were differences in the number of substance-related arrests (Henggeler et al., 1991). In a four-year follow-up to an earlier study, Henggeler et al. (2002) looked at the long-term effects of MST. The researchers contacted 90 percent (106) of the original 118 study participants, 80 of whom provided information at the four-year follow-up. The study examined criminal behavior, substance abuse, and psychiatric symptoms. Participants completed the Self-Report Delinquency Scale, which measures aggressive and property crimes committed in the past 12 months. The study also looked at law enforcement records for convictions that occurred in the previous 2.5 years. Four years after treatment ended, substance abuse was reported by participants through a scale using established measures from the Young Adult Self-Report, the Addiction Severity Index, and the Youth Risk Behavior Survey. Because self-reported measures may not be reliable by themselves, the study used biological drug tests to supplement the self-reported questionnaire. For these drug tests, the study gathered hair and urine samples from participants at the same time the questionnaires were administered and once again, during the next year. Psychiatric symptoms were measured using the Externalizing and Internalizing scales of the Young Adult Self-Report. Early studies of MST were conducted by teams that included at least one of the MST program developers in a controlled setting. Timmons-Mitchell et al. (2006) tested the effectiveness of MST when administered in a community setting without direct oversight from a principal developer of the approach, and this study team did not include any program developers. Youth that appeared in a Midwestern family court were randomly assigned to receive either MST or treatment as usual. In this case, treatment as usual was primarily centered on referrals to youth for such services as anger management courses and individual counseling. The study collected baseline information from both groups on the demographic characteristics, criminal history, and scores on the Child and Adolescent Functional Assessment Sale (CAFAS). CAFAS measures youth functioning in several areas, including school, work, community, behavior toward others, and substance abuse. The study collected two more sets of CAFAS scores, once just after completing MST and again six months after treatment ended. Criminal histories of youth were tracked until 18 months after treatment ended. CAFAS scores and criminal histories from the treatment-as-usual group were collected at time intervals approximately equal to those for the MST group.

Key Evaluation Findings

Three related studies of serious juvenile offenders and their families (Henggeler et al., 1991, 1992, and 1993) found the following:

• After 13 weeks of treatment:

o MST youth reported significantly less peer aggression than did non-MST youth.

o MST youth reported significantly lower use of alcohol and marijuana than did non-MST youth.

o MST families showed significantly more family cohesion than did non-MST families.

• Fifty-nine weeks after treatment began:

o Significantly fewer MST participants had been arrested than had non-MST participants (42 percent versus 62 percent).

- 316 -

o MST participants had spent significantly fewer days in incarceration than had non-MST participants. Twenty percent of MST participants had been incarcerated, compared with 68 percent of non-MST participants.

• Almost 2.5 years after treatment began, significantly fewer MST participants had been arrested than had non-MST participants (61 percent versus 80 percent).

Another set of related studies of serious juvenile offenders and their families (Borduin et al., 1995; Henggeler et al., 1991) found the following:

• After about four months of treatment, MST families reported significantly greater family cohesion and fewer youth behavior problems than did non-MST families.

• Four years after treatment began:

o Significantly fewer MST youths had been arrested compared with non-MST youths (26 percent versus 71 percent).

o Significantly fewer MST youths had been arrested for substance-related offenses compared with non-MST youths (4 percent versus 16 percent).

• Among the youths that were arrested following treatment, those in the MST group were arrested for significantly fewer serious and violent crimes than were those in the non-MST group.

• MST was equally effective with youths of different genders and ethnic backgrounds.

The Henggeler et al. (2002) four-year follow-up study reported these results:

• Youth that had participated in MST reported fewer aggressive crimes and were convicted of fewer aggressive crimes than the treatment as usual participants.

• MST participants showed significantly higher rates of marijuana abstinence on biological tests, but did not self-report significantly higher abstinence rates.

• Youth that had participated in MST did not show significant reductions in property crimes, cocaine use, or psychiatric symptoms compared to treatment-as-usual participants.

The Timmons-Mitchell study (2006) of MST in a community setting identified these significant outcomes:

• Immediately after treatment, youth in the MST group showed substantial improvement in school/work, home, community and mood/emotional functioning.

• In the 18 months after treatment, youth participating in MST were three times less likely to be re-arrested and were arrested and arraigned for fewer new offenses.

Probable Implementers

MST programs are typically housed within public mental health organizations or in private provider organizations that offer mental health services. MST programs typically interact with multiple local agencies (for example, juvenile justice, mental health, and social welfare agencies) as well as with schools and family courts.

Funding

Funding for MST may come from a variety of sources, including Medicaid reimbursement, state children's services funding, program-level grants to reduce recidivism, funds diverted as an alternative to out-of-home placements for youth, and state reimbursements to managed care organizations that treat emotionally disturbed youth.

- 317 -

Training and comprehensive program support for MST can be obtained from MST Services or any of the more than 20 MST Network Partner organizations, which are fully certified to deliver MST training. More information about training in MST and MST Network Partner organizations is available at www.mstservices.com. Books, training materials, and DVDs can be purchased from the MST online store. Prices range from $15 (in 2011 dollars) for a DVD to $115 (in 2011 dollars) for the MST startup kit. While MST does not publish the cost to treat individuals on its website, a review of national research on MST (Aos et al., 1999) found that the average program cost is about $4,500 per MST participant (in 1998 dollars). A more recent study (Sheidow et al., 2004) estimated the average cost to treat one individual for psychiatric problems with MST at about $8,200 (in 2004 dollars).

Implementation Detail

Program Design

MST uses techniques from cognitive, behavioral, and family therapies. Each MST treatment plan is designed by an MST-trained team, usually composed of three master's-level counselors and a clinical supervisor. Guided by the nine principles, the team develops a treatment plan in collaboration with the youth's family members; the plan is family-driven rather than therapist-driven. The overall objectives of MST treatment are to empower caregivers with the skills and resources they will need to address the inevitable difficulties associated with raising teenagers, and to empower youths with skills and resources for coping with family, peer, school, and neighborhood problems. Over the course of treatment, MST therapists place developmentally appropriate demands on the youths and family members so that they behave in an increasingly responsible manner.

Curriculum

MST does not have a prescribed or set curriculum.

Staffing and Training

Master's-level therapists, who work for the MST program, provide treatment. Each full-time therapist carries a caseload of four to six families. The therapists are organized into "teams" of two to four; each team of therapists receives on-site supervision, usually by a Ph.D.-level mental health professional. MST therapists are required to track weekly progress and outcomes on each case by completing case paperwork and participating in clinical supervision and MST consultation. Staff training and program development assistance are provided by MST Services, through licensing agreements with the Medical University of South Carolina and the Family Services Research Center. The core services for program development and training include the following:

• organizational assessment and assistance

• an initial five-day training session

• weekly MST clinical consultations

• quarterly "booster" training sessions

• ongoing monitoring for treatment fidelity and adherence.

Issues to Consider

This program received a "proven" rating. Two rigorous studies found that MST was effective in reducing arrests and incarceration among serious juvenile offenders. Furthermore, positive outcomes were shown to persist for two to four years after treatment began. While the program developer was involved in all the major studies of MST, the research was conducted and reported according to high scientific standards.

- 318 -

http://www.mstservices.com/

http://store02.prostores.com/servlet/mst/StoreFront

Additional research has examined how MST performs with other populations, such as substance- abusing juvenile offenders (Henggeler et al., 1999a) or youths in psychiatric crisis (Henggeler et al., 1999b). Henggeler et al. (1999b) found evidence that a modified form of MST was more effective than hospitalization at reducing externalizing psychiatric symptoms and improving self-esteem. Henggeler et al. (1999a) supplemented traditional MST with psychopharmacological treatment and found this combination was more effective at reducing substance use than non-MST "service as usual." Timmons-Mitchel et al. (2006) also looked at substance use as an outcome possibly affected by MST. Although they found improvement in substance use outcomes, the effect was not statistically significant and it is not clear whether the improvement was due to multisystemic therapy. However, the Timmons-Mitchel et al. (2006) study was not specifically targeting substance-abusing youth, so they may not have had enough of these youth to detect program effects on substance use. These studies suggest that MST may need to be adapted if it is to serve populations with specific needs beyond those of "typical" juvenile offenders. Other research has investigated MST's effectiveness when delivered under "real-world" conditions, under which study authors did not have control over hiring MST therapists and could not ensure the integrity of treatment delivery (Henggeler et al., 1997). Again, the results were mixed, suggesting that adherence to strict MST protocols (including regular, expert supervision) may be essential to program success. Timmons-Mitchell et al. (2006) found positive effects when MST was delivered in the community without direct oversight from the program developer, but the effects were smaller than those shown in more-controlled studies. These less-favorable studies do not take away from the demonstrated success of MST when delivered under standard protocols and with serious juvenile offenders. Rather, these later studies indicate the difficulties that program implementers may face as they attempt to expand the program.

Example Sites

MST programs operate in 30 states and in 11 countries outside of the United States, including Australia, Canada, Iceland, Norway, New Zealand, England, the Netherlands, Northern Ireland, Sweden, Switzerland, and Denmark. All licensed sites are listed on MST's website: www.mstservices.com.

Contact Information

For further information about program development, treatment model dissemination, and training, contact: Marshall E. Swenson, MSW, MBA Manager of Program Development MST Services, Inc. 710 Johnnie Dodds Blvd., Suite 200 Mt. Pleasant, SC 29464 Tel: 843-856-8226, ext. 215 Direct: 843-284-2215 Fax: 843-856-8227 Email: marshall.swenson@mstservices.com or Melanie Duncan, Ph.D. Program Development Coordinator MST Services, Inc. 710 Johnnie Dodds Boulevard, Suite 200 Mt. Pleasant, SC 29464 Office: 843.284.2221 Fax: 843.856.8227 Email: melanie.duncan@mstservices.com

- 319 -

http://www.mstservices.com/

mailto:marshall.swenson@mstservices.com

mailto:melanie.duncan@mstservices.com

For further information about research-related issues, contact: Dr. Scott W. Henggeler Family Services Research Center Department of Psychiatry and Behavioral Sciences Medical University of South Carolina 326 Calhoun St. Charleston, SC 29425-0742 Phone: (843) 876-1800 Fax: (843) 876-1808

Available Resources

For general information on MST program design, training, research, and publications, see the MST Services website: www.mstservices.com. For research related issues, see: www.musc.edu/fsrc.

Bibliography

Aos, Steve, Polly Phipps, Robert Barnoski, and Roxanne Lieb, The Comparative Costs and Benefits of Programs to Reduce Crime, Olympia, Wash.: Washington State Institute for Public Policy, Version 3.0, May 1999. As of March 9, 2011: http://www.wsipp.wa.gov/pub.asp?docid=01-05-1201 Borduin, Charles M., Barton J. Mann, Lynn T. Cone, Scott W. Henggeler, Bethany R. Fucci, David M. Blaske, and Robert A. Williams, "Multisystemic Treatment of Serious Juvenile Offenders: Long-Term Prevention of Criminality and Violence," Journal of Consulting and Clinical Psychology, Vol. 63, No. 4, 1995, pp. 569-578. Henggeler, Scott W., Charles M. Borduin, Gary B. Melton, B. Mann, Linda A. Smith, J. Hall, Lynn Cone, and Bethany Fucci, "Effects of Multisystemic Therapy on Drug Use and Abuse in Serious Juvenile Offenders: A Progress Report from Two Outcome Studies," Family Dynamics of Addiction Quarterly, Vol. 1, No. 3, 1991, pp. 40-51. Henggeler, Scott W., Gary B. Melton, and Linda A. Smith, "Family Preservation Using Multisystemic Therapy: An Effective Alternative to Incarcerating Serious Juvenile Offenders," Journal of Consulting and Clinical Psychology, Vol. 60, 1992, pp. 953-961. Henggeler, Scott W., Gary B. Melton, Linda A. Smith, Sonja K. Schoenwald, and Jerome H. Hanley, "Family Preservation Using Multisystemic Treatment: Long-Term Follow-Up to a Clinical Trial with Serious Juvenile Offenders," Journal of Child and Family Studies, Vol. 2, 1993, pp. 283-293. Henggeler, Scott W., Gary B. Melton, Michael J. Brondino, David G. Scherer, and Jerome H. Hanley, "Multisystemic Therapy with Violent and Chronic Juvenile Offenders and Their Families: The Role of Treatment Fidelity in Successful Dissemination," Journal of Consulting and Clinical Psychology, Vol. 65, No. 5, 1997, pp. 821-833. Henggeler, Scott W., Melisa D. Rowland, Jeff Randall, David M. Ward, Susan G. Pickrel, et al., "Home- Based Multisystemic Therapy as an Alternative to the Hospitalization of Youths in Psychiatric Crisis: Clinical Outcomes," Journal of the American Academy of Child and Adolescent Psychiatry, Vol. 38, No. 11, pp. 1331-1339, 1999b. Henggeler, Scott W., Susan G. Pickrel, and Michael J. Brondino, "Multisystemic Treatment of Substance-Abusing and -Dependent Delinquents: Outcomes, Treatment Fidelity, and Transportability," Mental Health Services Research, Vol. 1, No. 3, pp.171-184, 1999a. Henggeler, Scott W., W. Glenn Clingempeel, Michael J. Brondino, and Susan G. Pickrel, "Four-Year Follow-Up of Multisystemic Therapy with Substance-Abusing and Substance-Dependent Juvenile

- 320 -

http://www.mstservices.com/

http://www.musc.edu/fsrc

http://www.wsipp.wa.gov/pub.asp?docid=01-05-1201

Offenders," Journal of the American Academy of Child and Adolescent Psychiatry, Vol. 41, No. 7, 2002, pp. 868-874. Sheidow, Ashli J., W. David Bradford, Scott W. Henggeler, Melisa D. Rowland, Colleen Halliday- Boykinds, Sonja K. Schoenwald, and David M. Ward, "Treatment Costs for Youth Receiving Multisystemic Therapy or Hospitalization After a Psychiatric Crisis," Psychiatric Services, Vol. 55, No. 5, 2004, pp. 548-554. Timmons-Mitchell, Jane, Monica B. Bender, Maureen A. Kishna, and Clare C. Mitchel, "An Independent Effectiveness Trial of Multisystemic Therapy with Juvenile Justice Youth," Journal of Clinical Child and Adolescent Psychology, Vol. 35, No. 2, 2006, pp. 226-236.

Last Reviewed

March 2011

National Guard Youth ChalleNGe Program

Program Info

Outcome Areas Healthy and Safe Children Children Succeeding in School

Indicators Students graduating from high school Children and youth not engaging in violent behavior or displaying serious conduct problems Children experiencing good physical health

Topic Areas Age of Child Adolescence (13-18) Type of Setting High School Out of School Time Type of Service Instructional Support Mentoring Youth Development Type of Outcome Addressed Cognitive Development/School Performance Juvenile Justice Physical Health Poverty/Welfare Substance Use and Dependence Teen Sex/Pregnancy

Evidence Level Proven

- 321 -

Program Overview

The National Guard Youth ChalleNGe Program (ChalleNGe) is an intensive residential program that aims to "reclaim the lives of at-risk youth" who have dropped out of high school and give them the skills and values to succeed as adults. Developed by the National Guard Bureau in the U.S. Department of Defense, ChalleNGe operates in more than half of the states in the country. Over 100,000 young people have completed the program since it was launched in 1993. The program is 17 months long and divided into three phases: a two-week Pre-ChalleNGe Phase, which is a demanding orientation and assessment period; a 20-week Residential Phase; and a one- year Post-Residential Phase. The participants live at the program site, often a military base, during the first two phases. The curriculum for the Residential Phase focuses on eight core components of positive youth development: leadership/fellowship, responsible citizenship, service to community, life- coping skills, physical fitness, health and hygiene, job skills, and academic excellence. At the end of the Residential Phase, participants work with staff to arrange post-residential placement, such as employment, education, or military service. During the Post-Residential Phase, participants return to their families and receive structured mentoring from qualified mentors identified by themselves within their own community. While the program environment is described as "quasi-military," participation in ChalleNGe is voluntary, and there are no requirements for military service during the program or afterward (Millenky et al., 2011).

Program Participants

ChalleNGe serves youths between the ages of 16 and 18 who have dropped out of school and are unemployed, drug-free, and not heavily involved with the justice system. The program is open to both males and females. There are no income-based eligibility criteria.

Evaluation Methods

ChalleNGe was evaluated using a randomized controlled research design. Because the number of applicants to ChalleNGe was greater than the number of spaces available, program admittance was decided by a lottery in which eligible applicants were randomly assigned either to an intervention group that was offered admission to ChalleNGe or a control group that was not offered admission. During the study period of 2005-2007, 2,320 applicants across 10 program sites were assigned to the treatment group and 754 applicants were assigned to the control group. Program sites were not selected randomly, but rather were selected for stable staffing and a tendency to receive more applications to participate than spaces available. Among applicants assigned to the ChalleNGe group, 1,575 (67.9 percent) actually enrolled. The analyses included those applicants who did not enroll in or finish the ChalleNGe program. Baseline data were collected via a two-page questionnaire shortly before the applicants were randomly assigned. These data provide a snapshot of the study participants' demographic information. Three follow-up studies have been conducted:

• The first wave follow-up surveys were administered an average of nine months after the participants had entered the study, shortly after the treatment group had completed the Residential Phase. The response rate was 85 percent (Bloom et al., 2009).

• The second wave follow-up surveys were administered approximately 21 months after the participants entered the study. The follow-up surveys included education outcomes (high school enrollment, graduation rate, college attendance), employment outcomes (work status), health outcomes (self-reported overall health and body mass index, or BMI), and social outcomes (arrest rate and conviction rate). The response rate for this follow-up was 79 percent (Millenky et al., 2010).

• The third wave follow-up surveys were conducted three years after program enrollment, more than one year after the Post-Residential Phase was complete for participants. The response

- 322 -

rate for this survey was 78 percent. These surveys assessed the same outcomes that were included in the second wave follow-up (Millenky et al., 2011).

In addition, the research teams gathered information about program implementation and participation through site visits and the ChalleNGe Data Management and Reporting System (DMARS), the national Web-based program tracking system used by all ChalleNGe programs.

Key Evaluation Findings

First wave (nine month) findings (Bloom et al., 2009) Nine months after the participants entered the study, the treatment group (14.6 percent) was significantly more likely than the control group (2.6 percent) to have obtained a high school diploma. Similarly, the treatment group (30.9 percent) was also significantly more likely than the control group (7.5 percent) to have earned a General Education Development certificate (GED). The control group (35.5 percent) was significantly more likely than the treatment group (16.3 percent) to have returned to high school but not to have obtained a diploma or GED.

• The treatment group (51.2 percent) was significantly more likely than the control group (42.1 percent) to be employed and to be taking college courses (10.9 percent versus 2.7 percent)

• The treatment group (14.2 percent) was significantly less likely than the control group (20.0 percent) to have been arrested since the random assignment. Similarly, the treatment group (6.5 percent) was also significantly less likely than the control group (11.0 percent) to have been convicted.

• The treatment group (76.7 percent) was significantly more likely than the control group (68.3 percent) to report their overall health as being "very good" or "excellent." Further, the treatment group (8.4 percent) was significantly less likely than the control group (12.8 percent) to be obese (BMI of 30 or more), though the two groups did not demonstrate any statistically significant differences in the likelihood of being overweight (BMI of 25 to 29).

• The treatment group (11.0 percent) was significantly more likely than the control group (7.0 percent) to report high self-efficacy and social adjustment scores (defined as one standard deviation above the average score).

Second wave (21-month) findings (Millenky et al., 2010)

• Twenty-one months subsequent to enrollment in the study, ChalleNGe participants were significantly more likely than their control group counterparts to have earned a high school diploma or GED (60.5 percent versus 36.4 percent). They were also significantly more likely to have earned some college credit (24.8 percent versus 9.6 percent) and to have received vocational training (29.7 versus 22.9 percent)

• A higher percentage of participants were currently working relative to the control group (55.0 percent versus 50.1 percent), and a greater proportion of participants were currently working full-time (43.7 versus 38.8 percent). Participants were more likely to be earning an hourly wage of $8 to $9.99 (22.6 percent versus 18.2 percent) or to be earning $10 or more per hour (9.1 percent versus 6.2 percent).

• There was no difference between program and control groups on likelihood of being arrested and charged with a crime in the past year, but the ChalleNGe group was less likely than the control group to have been convicted of a crime (8.9 percent versus 13.2 percent).

• On self-reported delinquency, ChalleNGe participants were equally as likely as the control group to report any violent incidents in the past year, but the average number of violent incidents was lower in the ChalleNGe group compared with the control group (2.0 versus 2.3 incidents). ChalleNGe participants were less likely to report any property damage incidents (27.2 percent versus 35.1 percent), and, similarly, the average number of property damage incidents was lower in the ChalleNGe group (0.9 versus 1.3 average incidents per participant)

- 323 -

• There were no significant differences between the groups on health, sexual activity, or drug use outcomes, with the exception of binge drinking. ChalleNGe participants were less likely to report binge drinking in the last 14 days (2.8 percent versus 4.7 percent).

• Significantly more ChalleNGe than control group participants reported living in their own home or apartment (versus with their parents or a friend; 19.0 percent versus 15.0 percent).

• The 21-month survey also asked questions related to life-coping and leadership. While most of the questions showed no significant differences between the ChalleNGe and control groups, the ChalleNGe group showed positive impacts relative to the control group on the following measures: "Has learned about organizing time and not putting things off" (ChalleNGe participants ranked this an average of 3.6 out of 4, compared with 3.5 for the control group); "Has learned how to better control temper" (3.5 out of 4 versus 3.4); "Has learned the challenges of being a leader" (3.7 out of 4 compared with 3.5); "Encourages different points of view without worrying about agreement (ChalleNGe participants scored 3.7 out of 5, versus 3.5 out of 5 in the control group).

Third wave (3-year) findings (Millenky et al., 2011)

• At three years subsequent to enrollment in the program, a range of positive outcomes related to academic performance and employment were observed:

o ChalleNGe participants were still significantly more likely than their control group counterparts to have earned a high school diploma or GED (71.8 percent versus 55.5 percent). They were also significantly more likely to have earned some college credit (34.9 percent versus 18.8 percent) and to have received a college degree including associate's degrees (0.9 percent versus 0.0 percent).

o Participants were also more likely than the control group to have been employed in the last 12 months (88.4 percent versus 84.5 percent) and to be currently employed (57.8 percent versus 50.7 percent). Participants had been employed for more total months on average (8.1 months versus 7.2 months) and earned more in the previous year on average ($13,515 versus $11,248, averages include zero amounts for those who do not work).

o More ChalleNGe than control group participants reported living in their own home or apartment (versus with their parents or a friend, 25.0 percent versus 20.0 percent), but there were no other differences observed across the groups in living arrangements, marital status, or parental status.

o In questions related to civic engagement, ChalleNGe participants reported more collective civic efficacy than the control group, averaging 3.31 out of 5 compared with 3.23 in the control group. According to study authors, "this aspect of civic engagement was measured based on the respondent's strength of endorsing beliefs that, by working with fellow members of his or her community, he or she could be effective in addressing community problems" (Millenky et al., 2011).

• The study evaluated a range of health and lifestyle-related outcomes. For several of these outcomes, the ChalleNGe group appears to be doing statistically significantly worse. Study authors did not comment on possible reasons for these differences:

o While there was no significant difference across the groups in terms of obesity (BMI of 30 or above), the ChalleNGe group was significantly more likely to be overweight than the control group (defined as BMI between 25 and 29.9; 32.1 percent in the ChalleNGe group versus 25.9 percent in the control group). There were no other significant differences in overall physical or mental health.

o ChalleNGe participants were significantly less likely to report that they "always use birth control" than the control group (49.8 percent versus 57.8 percent) and were significantly more likely to report never using birth control (16.1 percent versus 12.4 percent). There

- 324 -

were no significant differences in the proportion of ChalleNGe or control group members who reported being sexually active.

o Finally, a higher proportion the ChalleNGe group reported ever having used illegal drugs other than alcohol or marijuana (28.2 versus 23.2 percent). Other drug and alcohol measures were not significantly different.

• There was no difference across the treatment and control groups in enlistment history or current enlistment.

• There were no differences across the two groups in being charged or convicted of a crime, or self-reported incidents of delinquency.

• Twelve questions related to leadership and life-coping skills were asked of both groups; there were no significant differences across the groups on responses to any of the questions.

Probable Implementers

This program is implemented by states under a Master Cooperative Agreement with the National Guard Bureau.

Funding

Perez-Arce et al. (2012) conducted a cost-benefit analysis of ChalleNGe, finding that ChalleNGe program generates increased earnings and other benefits which result in a $2.66 return for every dollar expended on the program. Researchers found that the ChalleNGe program is estimated to increase the present discounted lifetime earnings of ChalleNGe admittees by $45,231 (in 2012 dollars).

Implementation Detail

Program Design

• Although there is considerable room to tailor the program model to local conditions, the basic structure of the ChalleNGe program is the same in all states.

• Most states operate a single "100-bed" ChalleNGe program, serving a total of about 200 participants per year in two class cycles (starting in January and July).

• During the two-week Pre-ChalleNGe Phase, candidates are introduced to the program's rules and expectations; learn military bearing, discipline, and teamwork; and begin physical fitness training.

• The 20-week Residential Phase environment is quasi-military: Participants are called "cadets" and are divided into platoons and squads, live in barracks, have their hair cut short, wear uniforms, and are subject to military-style discipline. The cadets are closely supervised by staff at all times.

• Cadets who successfully complete the Residential Phase move into the one-year Post- Residential Phase, during which they receive structured mentoring from a mentor who is nominated by the cadet and screened and trained by the program staff.

Staffing

Typically, the ChalleNGe program staff includes a Program Director, who oversees all elements of the program and focuses particularly on external matters, such as marketing, fundraising, and government and community relations; a Program Deputy Director, who oversees all internal affairs, including program management, human resources, and cadet affairs; cadre or team leaders, who directly supervise all daily activities of the cadets; academic instructors, who teach GED courses, responsible citizenship, and job skills; counselors, who conduct psychological counseling and career counseling; and recruitment, placement, and mentoring (RPM) coordinators, who are responsible for

- 325 -

recruiting and screening applicants, screening and training mentors, and interacting with cadets and mentors during the Post-Residential Phase. Most administrative staff are military veterans, military retirees, or members of the National Guard and reserves. Academic instructors and counselors are hired through various channels, such as the local school district, community colleges, or direct hiring.

Curriculum

During the Residential Phase, academic instructors teach the GED subject courses, including math, science, writing and language arts, and computer skills. The curricula vary by program site and no particular example is mentioned.

Issues to Consider

This program received a "proven" rating. The research, conducted during 2005-2007, was implemented according to rigorous design standards and included a treatment group of 2,320 participants and a comparison group of 754 participants. The program evaluation used a randomized controlled trial. However, several issues regarding the study sample should be noted. First, the 10 programs—roughly half of the state programs in existence at the time—included in the evaluation study were not chosen randomly. Rather, they were programs that had stable staffing and tended to have more applicants than they could serve, a prerequisite for conducting a random assignment. Thus the results presented in this study do not represent the overall impact of ChalleNGe nationally but rather the impact of selected ChalleNGe programs. Second, applicants who were under age 16 and a half were excluded from random assignment (though not from the program) so that they would not be barred from reapplying for ChalleNGe in the future if they were originally assigned to the control group. As a result, the results reported in the evaluation study pertain only to ChalleNGe participants who were age 16 and a half or older at the time of application. It is possible that younger participants of ChalleNGe have had different experiences from their older counterparts. Finally, in most cases the first wave of follow-up surveys, from which conclusions about the early impacts of ChalleNGe were drawn, were administered only to the first cohort of random assignment for each site despite the fact that random assignments had been conducted for two or more cohorts at most sites. It should also be noted that, in several areas related to health and lifestyle, the ChalleNGe group appears to perform significantly worse than their control group counterparts. Study authors did not comment on reasons for these differences, which included a higher likelihood of overweight among ChalleNGe participants, a higher proportion of ChalleNGe participants ever having used illegal drugs compared to their control group counterparts, and a lower proportion of ChalleNGe participants reporting that they always use birth control.

Example Sites

Arizona, California, Florida, Georgia, Illinois, Michigan, Mississippi, New Mexico, North Carolina, Texas, Virginia, and Wisconsin

Contact Information

MDRC Evaluation Study Contact Dan Bloom and Megan Millenky MDRC 19th Floor 16 East 34 Street New York, NY 10016-4326 Phone: 212-532-3200 Email: dan_bloom@mdrc.org

- 326 -

mailto:dan_bloom@mdrc.org

National ChalleNGe Contact Joe Padilla Air National Guard Office of Athletics and Youth Development Jefferson Plaza 1, Room 2456 1411 Jefferson Davis Highway Arlington, VA 22202-3231

Available Resources

The National Guard Youth ChalleNGe Program website (www.ngycp.org) offers comprehensive information about the program models and implementation experiences for youth, parents, government agencies, and the public.

Bibliography

Bloom, Dan, Alissa Gardenhire-Crooks, and Conrad Mandsager, Reengage High School Dropouts: Early Results of the National Guard Youth ChalleNGe Program Evaluation, New York: MDRC, 2009. Millenky, Megan, Dan Bloom, and Colleen Dillon, Making the Transition: Interim Results of the National Guard Youth ChalleNGe Evaluation, New York: MDRC, 2010. Millenky, Megan, Dan Bloom, Sara Muller-Ravett, and Joseph Broadus, Staying on Course: Three-Year Results of the National Guard Youth ChalleNGe Evaluation, New York: MDRC, 2011. Perez-Arce, Francisco, Louay Constant, David S. Loughran, and Lynn A. Karoly, A Cost-Benefit Analysis of the National Guard Youth ChalleNGe Program, Santa Monica, CA: RAND Corporation, 2012.

Last Reviewed

February 2012

New Hope Project

Program Info

Outcome Areas Healthy and Safe Children Children Succeeding in School Strong Families

Indicators Students performing at grade level or meeting state curriculum standards Families increasing economic self-sufficiency Children and youth not engaging in violent behavior or displaying serious conduct problems

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18)

- 327 -

http://www.ngycp.org/

Type of Setting Child Care/Preschool Community-Based Service Provider Health Care Provider Type of Service Family Support Type of Outcome Addressed Behavior Problems Cognitive Development/School Performance Poverty/Welfare Violent Behavior

Evidence Level Proven

Program Overview

The New Hope Project operated from 1994 to 1998 in two inner-city areas of Milwaukee, Wisconsin. New Hope offered low-income individuals and families the opportunity to use a comprehensive set of integrated program services designed to increase income, financial security, and access to full-time employment. In the two target locales, all adults whose earnings were below 150 percent of the federal poverty level and who were willing to work full-time were eligible to apply for enrollment in the program. Applicants need not have been welfare recipients nor have children. New Hope offered two program benefits to all participants:

1. Community service-based full-time job opportunities for participants unable to find full-time work (or part-time job opportunities to supplement an existing part-time job) in the private job market

2. Personalized services assisting participants in job searches, child care, and other employment- related needs.

For participants who worked full-time (30+ hours per week) New Hope also offered:

1. A monthly earnings supplement designed to raise participants’ income to exceed the poverty threshold for the household

2. Subsidized health insurance

3. Child care subsidies.

New Hope operated outside the existing public assistance system, and many participants continued to receive various types of public assistance while participating. The program was funded by a consortium of local, state, and national organizations interested in work-based antipoverty policy, as well as by the state of Wisconsin and the federal government.

Program Participants

The program targeted low-income individuals ages 18 or older, including both male and female adults with and without children. Eligible participants had earnings at or below 150 percent of the federal poverty level and were willing and able to work a minimum of 30 hours per week.

- 328 -

Evaluation Methods

The New Hope Project enrolled 1,367 low-income adults drawn from two inner-city areas in Milwaukee. Applicants who met the eligibility criteria (i.e., had earnings which did not exceed 150 percent of the federal poverty level and were willing to work at least 30 hours per week) were randomly assigned to participate in New Hope or serve as a control group. From the initial 1,367 participants, 10 cases lacked baseline data, resulting in a total of 678 individuals in the treatment (New Hope) group and 679 individuals in the control group. Very few statistically significant differences were found between the treatment and control groups on a range of baseline measures, suggesting that randomization had been successful. Individuals who were randomized to the treatment group were eligible to receive New Hope's benefits upon providing verification that they met the 30-hour minimum work requirement. Program benefits were available for three years, and treatment group participants were eligible to access any and all New Hope benefits and services for any portion of that time, subject to the requirements outlined above. Applicants randomized to the control group received no program benefits but were enrolled in the study to serve as a comparison group and remained eligible for other programs in the community that were not related to New Hope. Evaluations of the New Hope program collected data on both treatment and control group participants during and after the program. Three assessments of the New Hope Project have been conducted by the Manpower Demonstration Research Corporation and members of the MacArthur Foundation on Successful Pathways Through Middle Childhood. The studies were conducted two, five, and eight years after program applicants were randomized into treatment and control groups (Bos et al., 1999; Huston et al., 2003; Miller et al., 2008). The two-year evaluation was conducted when treatment group participants were still eligible to receive program benefits. The five- and eight-year evaluations were conducted only for families with children ages 1-11 at the time parents applied for the program. These assessments occurred after the program services had been terminated. The evaluations investigated numerous short- and long-term outcomes among treatment and control group adults as well as their children. The treatment group was defined as those adults (and their children) who were randomly assigned to be eligible for the New Hope program benefits, whether or not they actually fulfilled the eligibility requirements (30 hours of verified work per week) or accessed the benefits available. Most treatment group members (87 percent) did use at least one program benefit, but the treatment can most precisely be interpreted as eligibility for (rather than use of) New Hope benefits. Information used in the evaluations came from five sources:

• a baseline enrollment form

• administrative records (i.e., unemployment insurance earnings, public assistance benefit records, tax records, and New Hope administrative records)

• in-person surveys with parents and children conducted two, five, and eight years after parents were randomly assigned to the program or control group

• teacher reports of children's school performance and social behavior collected two, five, and eight years after parents were randomly assigned

• additional data gathered for 44 families taking part in an ethnographic study.

Applicants to the program, i.e., the full study sample, were 72 percent female. The racial/ethnic breakdown was 51 percent black, 27 percent Hispanic, 13 percent white, and 9 percent other ethnicities. Approximately 70 percent of the full study population lived in a household with children. Thirty-eight percent of the full sample was currently employed, 97 percent had earned less than $15,000 in the 12 months prior to program intake, and 71 percent had received Aid to Families with Dependent Children (AFDC) benefits or some other type of welfare or Medicaid within the same 12- month period. Approximately 57 percent of the sample had received a high school diploma or GED. At the beginning of the study, 418 individuals (roughly one-third) were employed 30 or more hours per week, while 935 worked part time or not at all.

- 329 -

An expanded survey was conducted in the homes of participants who had children. The Child and Family Study (CFS) targeted 745 adult sample members who were living in households with at least one child ages 1-11 (55 percent of the total sample). The sample excluded 67 Asian-American families—most of whom were Southeast Asian refugees—because of language barriers and because many of the measurement instruments were culturally inappropriate for them. Overall response rates at each assessment ranged from 76 to 82 percent. The five-year survey was completed by 591 adults and 927 of their children. In addition, teachers of children who were living in New Hope households were sent a questionnaire to assess school behavior and performance. Data from teachers were collected on 420 children who were between the ages of 6 and 16 at the time of the survey. The eight- year survey was completed by 595 families and 866 focal children between the ages of 9 and 19. Teacher reports on 540 children were also received. The evaluation findings reported below present outcomes for the CFS sample only, since these results focus on outcomes related to children and families. If a family had more than one child in the survey age range, two children were identified as focal children. If there were more than two eligible children per family, the focal children were randomly selected, with the restriction that opposite-sex siblings were given preference over same-sex siblings. The evaluation included a total of 1,140 focal children. The parents in the sample were in many respects similar to those in other studies in which samples were drawn from individuals receiving welfare. When they applied for New Hope, over half were not employed, and about 80 percent were receiving AFDC, general assistance, food stamps, and/or Medicaid. The majority had never been married. Slightly over 10 percent were married and were living with their spouse, and almost half had three or more children. Slightly over half were African- American, and over one-quarter were Hispanic. Results, for the most part, are presented for the sample of families who responded to the two-year, five-year, and eight-year follow-up surveys; however, some findings are presented for the entire set of 745 families in the sample. Children's ages at the two-year follow-up ranged from 3 to 12; at the five-year follow-up, they were 6 to 15; and at the eight-year follow-up, they were 9 to 18. The child- reported outcomes presented are based on interviews with the roughly 900 children who responded to the surveys. Teacher-reported outcomes are based on the reports of teachers in the teacher-survey samples.

Key Evaluation Findings

Poverty, parental earnings and employment

• During the two years following the intervention, the treatment group worked an average of 3.0 quarters per year, compared with 2.6 and 2.7 quarters for the control group. More than two years after randomization, statistically significant differences in the number of quarters worked were no longer observed (Huston et al., 2005).

• Employment effects varied depending on the services participants utilized. During the two- year period following randomization, the only New Hope benefit that was associated with an increase in the number of quarters worked per year was placement in a community service job. Adults in the treatment group who were placed in community service jobs worked on average 2.6 more quarters (for an increase of 11 percent) relative to the comparison group. In contrast, New Hope's benefits that were intended to increase employment through indirect means (i.e., through child care vouchers, health insurance, and/or supplemental income) were found to have no effect on the number of quarters beneficiaries worked (Gibson, 2003).

• Two years after randomization, treatment group participants earned 19 percent more than control group participants. This gap could be traced to a one-time increase in earnings in the first quarter after randomization, which was attributable to job placement and earnings supplements. In the subsequent quarters, trends in earnings growth did not differ between groups, but the initial advantage of the treatment group was maintained (Dunifon, 2005).

• Five years after randomization, the percentage of families with records-based earnings and transfer income below the poverty line was 59.3 and 64.6 percent in the treatment and control

- 330 -

groups, respectively. Eight years after randomization, the gap had narrowed to a statistically insignificant 63.1 and 67.1 percent, respectively (Duncan et al., 2008).

• A study by Huston et al. (2003) concluded that New Hope was responsible for a 20 percent reduction in the number of families living below the poverty line five years after the intervention.

• Huston et al. (2003) found that the program had no significant impact on welfare receipt but did increase earnings-related income by 10.5 to 23.3 percent one and three years after randomization, respectively. However, no such significant increases were observed four and five years after randomization.

• During the three years in which program benefits were available, significantly more treatment group families (42 percent) than control group families (34 percent) had incomes above the poverty threshold (Duncan et. al., 2009).

Children's academic achievement, cognitive skills and social behavior

• Children in New Hope families had significantly higher academic achievement than those in control families two years after the intervention. Of note, boys in the program group families scored one-third of a standard deviation higher than boys in control group families on the SSRS Academic subscale (Huston et al., 2001).

• Boys in treatment group families received teacher ratings of classroom behavior (e.g., attention, independent work) and social skills that were 10 percent higher than their peers in control group families two years after randomization (Bos et al., 1999).

• Similarly, Huston et al. (2001) found that boys in treatment group families scored higher on classroom behavior by 0.38 of a standard deviation two years after the intervention.

• Two years after randomization, boys in treatment group families had fewer behavior problems (including aggression) relative to their counterparts in control group families, and they had more positive behaviors as measured by the Positive Behavior Scale (PBS). The PBS is an assessment tool that was developed to assess behavior (e.g., social competence and sensitivity, compliance and self-control, and autonomy) among children from economically disadvantaged families (Epps and Huston, 2007).

• For both problem behavior and positive behavior, treatment group boys scored about one- third of a standard deviation better than did control group boys two years after randomization (Huston et al., 2001).

• Five years after randomization, boys in treatment group families continued to score 0.3 standard deviations higher on teacher-reported overall achievement compared with control group boys. Significant effects were not found among girls (Huston et al., 2005; Duncan et. al., 2009).

• The New Hope study included measures of the families' and children's environments that might account for program effects on children. Among treatment group families, increases in income improved parents' effective child management, which in turn was associated with improvements in children's test scores and teacher- and parent-rated school performance, as well as children's behavior (e.g., internalizing problems, externalizing problems, delinquent acts) five years after the intervention (Mistry et al., 2006).

• The significant effects on teacher ratings of boys' positive social behavior remained apparent at the five-year assessment. The treatment group boys scored about one-fourth of a standard deviation higher than the control group boys, but there was not a significant difference on behavior problems. Teachers rated treatment group girls about one-fourth of a standard deviation lower (at the 10 percent significance level) than control group girls on positive behavior (Huston et al., 2005).

• As part of the program evaluation, children in treatment and control group families completed several assessments of their psychological well-being. One such test required children to listen

- 331 -

to vignettes about peers' physical or social actions and interpret the characters' intentions. Both five and eight years after randomization, boys in treatment group families were significantly less likely than boys in control group families to perceive hostile intent in vignettes. There was no program effect for girls (Huston, et al., 2005).

• A study by Huston, Gupta, et al. (2008) found only modest effects on academic performance— including reading scores, parental ratings of literacy, and overall achievement—eight years after randomization.

• Another study by Huston, Walker, et al. (2008) found slightly more pronounced effects on boys' reading scores and overall achievement and math scores among younger children (under 13 years of age) eight years after randomization.

• Boys in treatment group families averaged 2.6 percent higher on reading scores eight years after the intervention (Miller et al., 2008).

• Eight years after the intervention, youth of both genders in New Hope families had significantly higher parent ratings of positive social behavior than did those in control group families. However, there were no differences in teacher ratings of positive behavior or problem behavior, nor were there differences in self-reported delinquent behavior (Miller et al., 2008).

• Similarly, Huston, Gupta, et al. (2008) found no significant differences in the delinquent behaviors of children from treatment and control group families by age, gender, or on the whole, eight years after the intervention.

Probable Implementers

Charitable and health and human service organizations, foundations, and state and local governments

Funding

New Hope was funded by a large consortium of local, state, and national organizations. Donors who contributed $250,000 or more to the program included the State of Wisconsin Departments of Workforce Development and Health and Human Services, the U.S. Department of Health and Human Services, the National Institute for Child Health and Human Development, the John D. and Catherine T. MacArthur Foundation, the Mott Foundation, the Helen Bader Foundation, the Rockefeller Foundation, the Ford Foundation, the William T. Grant Foundation, Ameritech/Wisconsin Bell, Northwestern Mutual Life, the Wisconsin Energy Corporation, and the Annie E. Casey Foundation. The five-year and eight-year evaluations were funded by the National Institute of Child Health and Human Development. The annual taxpayer cost of the program was approximately $7,250 per family (in 2009 dollars).

Implementation Detail

Program Design

New Hope required that participants work a minimum of 30 hours per week. Participants were required to verify fulfillment of the work requirement by submitting wage stubs. For those individuals who fulfilled the work requirement, the following benefits were available:

• Earnings Supplement: The supplement was designed to complement the state and federal Earned Income Tax Credits (EITCs), that is, the refundable tax credits for low-income working families, so as to raise the income of full-time workers above the poverty level. The Earnings Supplement was implemented to ensure that additional work effort by participants or higher wages earned would lead to a true increase in overall income.

- 332 -

• Subsidized Health Insurance: Health insurance was available to any New Hope participant who did not already have access to coverage through an employer or a government-provided health plan, or who wished to supplement an employer-provided plan. New Hope required a small co-payment by participants that increased with income.

• Child Care Subsidy: A child care subsidy was available to parents with children under age 13. The program allowed parents to select their own licensed child care facility and then paid a rate comparable to the state-provided subsidy, but differed from the state subsidy by paying providers directly. New Hope participants were required to pay a small co-payment, which increased with income.

• Community Service Jobs: Job opportunities were made available to participants who were unable to find full-time employment after two months. The community service jobs provided wage-paying positions with local nonprofit organizations. Participants were required to apply for these positions and could be removed from the positions if job performance was unreliable or unsatisfactory. Each placement in a community service job was limited to a six-month stint, but participants could take up to two community service jobs over the course of the program. The hours worked in these jobs were counted toward New Hope’s full-time work requirement, on which eligibility for other program benefits was based.

In addition, New Hope provided limited traditional employment services. New Hope participants were assigned a project representative who assisted the participants in identifying their interests and aptitudes, job searching, employment maintenance, and acquiring job training, as well as finding child care services and other job-related needs. Staffing

The direct service providers came from a variety of backgrounds (e.g., some were trained in social work) and had professional or volunteer experience in human services or community action. In addition to the direct service providers, New Hope was staffed by management personnel, persons responsible for accounting and information services, and individuals providing administrative support.

Issues to Consider

This program received a "proven" rating. The evaluations employed rigorous standards, including a randomized sample of nearly 1,400 individuals and an analysis sample of 561 to 745 families (depending on the particular outcome measure). Furthermore, the evaluations were conducted by the Manpower Demonstration Research Corporation (MDRC), an organization that was independent of the implementing agencies, and members of the MacArthur Foundation. The evaluations found significant benefits for the New Hope group when compared with the control group in the realms of employment, earnings, poverty, and children's cognitive and social skills and academic achievement. Outcomes suggest that the majority of the income and employment effects were concentrated in the early years of the study and tended to fade out within five years after randomization. The New Hope program operated for three years, during which time participants were eligible to receive a range of work supports (i.e., earnings supplement, child care subsidies, and employment). It is therefore not entirely surprising that the program's effects were largest during the early years and faded after these supports ended. There is some evidence of lasting effects for the sample as a whole, such as the statistically significant reductions in poverty that occurred throughout the eight-year period and the higher wages earned by the treatment group after eight years, particularly among vulnerable subgroups. Several studies compared results among ex-ante sub-groups. Participation in New Hope was found to have had a greater impacts on individuals who were not employed full-time at baseline (see Bos et al., 1999), faced one or more barriers to employment (see Yoshikawa et al., 2003), or had children (see Duncan et al., 2009). Among the subgroups defined by the type of New Hope benefit utilized, positive effects on future employment were observed only among participants who had been placed in community service jobs.

- 333 -

The impacts of New Hope were larger and more consistent for boys of New Hope parents than for girls, especially when teacher reports were considered. In fact, teachers rated girls from New Hope families more negatively than they did control group girls, on a range of measures including appropriate classroom behavior, positive social behavior, and problem behaviors. The most recent study of the effects of New Hope was completed by MDRC and members of the MacArthur Foundation in 2008. It confirms that many of the program effects continue to fade, but several remain as much as eight years after the intervention. The most widespread lasting effects are related to children's academic achievement. Since 2005, New Hope has administered a new program called Supporting Families, which provides services to help individuals who are among the most difficult to employ (e.g., those who have minimal work histories, criminal records, or no high school diploma) find and retain employment. Supporting Families is not a demonstration project like the earlier New Hope program.

Example Sites

New Hope was implemented in Milwaukee, Wisconsin, from 1994 to 1998.

Contact Information

For information on the New Hope program: The New Hope Project, Inc. 2620 West North Avenue Milwaukee, WI 53210 Tel (414) 270-7580 Fax (414) 270-7579 info@newhopeproject.org http://www.newhopeproject.org For information on the evaluations of New Hope: Manpower Demonstration Research Corporation 16 East 34th Street, 19th Floor New York, NY 10016-4326 Tel (212) 532-3200 Fax (212) 684-0832 information@mdrc.org http://www.mdrc.org Greg Duncan University of California-Irvine gduncan@uci.edu Aletha C. Huston University of Texas-Austin achuston@mail.utexas.edu

Available Resources

The New Hope Project website (http://www.newhopeproject.org) provides more detailed information about the New Hope program, links to available publications and reports, and a listing of recent news and events.

- 334 -

mailto:info@newhopeproject.org

http://www.newhopeproject.org/

mailto:information@mdrc.org

http://www.mdrc.org/

mailto:gduncan@uci.edu

mailto:achuston@mail.utexas.edu

http://www.newhopeproject.org/

Bibliography

Bos, J. M., A. C. Huston, et al. (1999). New Hope for People with Low Incomes: Two-Year Results of a Program to Reduce Poverty and Reform Welfare. MDRC. Duncan, G. J., A. C. Huston, and T. Weisner. (2009). Higher Ground: New Hope for the Working Poor and Their Families. New York: Russell Sage Foundation. Duncan, G., C. Miller, et al. (2008). New Hope's Eight-Year Impacts on Employment and Family Income. MDRC. Dunifon, R. (2005). "The Labor-Market Effects of an Anti-Poverty Program: Results from Hierarchical Linear Modeling," Social Science Research, 34(1): 1-19. Epps, S. R., and A. C. Huston (2007). "Effects of a Poverty Intervention Policy Demonstration on Parenting and Child Behavior: A Test of the Direction of Effects," Social Science Quarterly, 88(2): 344- 365. Gibson, C. M. (2003). "Privileging the Participant: The Importance of Sub-Group Analysis in Social Welfare Evaluations," American Journal of Evaluation, 24(4): 443-469. Huston, A. C., A. E. Gupta, et al. (2008). New Hope's Effects on Social Behavior, Parenting, and Activities at Eight Years. MDRC. Huston, A. C., C. Miller, et al. (2003). New Hope for Families and Children: Five-Year Results of a Program to Reduce Poverty and Reform Welfare. MDRC. Huston, A. C., G. J. Duncan, et al. (2001). "Work-Based Antipoverty Programs for Parents Can Enhance the School Performance and Social Behavior of Children," Child Development, 72(1): 318- 336. Huston, A. C., G. J. Duncan, et al. (2005). "Impacts On Children Of A Policy To Promote Employment And Reduce Poverty For Low-Income Parents: New Hope After 5 Years," Developmental Psychology, 41(6): 902-918. Huston, A. C., J. T. Walker, et al. (2008). Long-Term Effects of New Hope on Children's Academic Achievement and Achievement Motivation. MDRC. Miller, C., A. C. Huston, et al. (2008). New Hope for the Working Poor: Effects After Eight Years for Families and Children. MDRC. Mistry, R. S., E. D. Lowe, et al. (2006). "What Earnings and Income Buy—The 'Basics' Plus 'A Little Extra': Implications for Family and Child Well-Being." In H. Yoshikawa, T. S. Weisner, and E. D. Lowe, ed., Making It Work: Low-Wage Employment, Family Life, and Child Development. New York: Russell Sage Foundation, 173-205. Yoshikawa, H., K. A. Magnuson, et al. (2003). "Effects of Earnings-Supplement Policies on Adult Economic and Middle-Childhood Outcomes Differ for the 'Hardest To Employ.'" Child Development, 74(5): 1500-1521.

Last Reviewed

April 2010

- 335 -

New York City's Small Schools of Choice

Program Info

Outcome Areas Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards Students graduating from high school

Topic Areas Age of Child Adolescence (13-18) Type of Setting High School Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance

Evidence Level Promising

Program Overview

"Small Schools of Choice" (SSCs) are public high schools in New York City. They do not select students based on academic criteria and are small in size, relative to other high schools. Students are admitted based on student choice on the High School Application and seat availability, with priority given to students who attend a school information session, open house, or a high school fair. Some schools give priority to students or residents of a specific borough. Each school has about 400 students total, with 100 students per grade in grades 9-12. The SSCs initially replaced failing neighborhood high schools around New York City through a competitive process in which prospective school leadership teams submitted proposals. SSCs began to open in 2002, starting with the 9th grade, and added an additional grade level each year until they served 9th through 12th grade. New SSCs are opened on a yearly basis. The SSCs received resources that included start-up grants or partnerships with organizations that had started new schools previously. Principals were selected through a competitive application process and then hired a mix of teachers from the old school and ones who were new to the building. Ultimately, the SSC schools brought small, academically rigorous schools to areas where previous schools had struggled.

Program Participants

The SSCs serve high school students, especially disadvantaged students who live in areas previously served by failing high schools that were closed. While there were 123 SSCs in New York City at the time of the study, only the 105 schools that were oversubscribed and required a lottery for admission were included in the evaluation.

- 336 -

Evaluation Methods

Bloom, Thompson, and Unterman (2010) evaluated the impact of SSCs on various academic outcomes for the period 2002-2008. Bloom and Unterman (2012) extended the analysis from the 2010 evaluation an additional two years for two of the cohorts of students. Both evaluations employed a natural experiment design that relied on the city's High School Admissions algorithm, which functions like a lottery, assigning "winning" students to SSCs according to a stepped process. The algorithm first tries to match the student with his or her top school choice. If the top-choice school is full, then the next step is to scan each applying student's school priority status (which is based on whether the student resides in the same borough as the school and whether the student attended a school information session or was otherwise "known" to the school). If there are students who have lower school priority, they get "bumped" in favor of student with the higher priority. If all students have the same school priority, the student is then placed into their next choice school using the same method. Students are placed in the comparison group when they "lose" the lottery and are not placed in an SSC through the lottery. Lottery winners placed in an SSC of their choice thus become a part of the treatment group. The study sample included only students who chose SSCs that were oversubscribed and, through the High School Admissions Process lottery, were randomly assigned either to the SSC or another school on their list. About 10 percent of students who applied through the lottery were not ultimately assigned to an SSC and then applied through other means; such students were not included in the study sample. The SSCs included in the evaluation are only those for which applicants exceeded admissions. These selection criteria yielded 21,085 students across four cohorts to be included in the evaluation. The treatment group consists of students who were offered a seat in an SSC, and the comparison group consists of students who applied through the same lottery and were not offered a seat in that particular SSC. The school assignment process yielded treatment and comparison groups that were comparable on baseline characteristics such as test scores and student demographics. The evaluation found that the students in the study sample (both treatment and comparison groups) were the same as the broader New York City student population, with a few exceptions. The study sample was more likely to live in the Bronx than all students enrolled in SSCs and all 9th-grade students in New York City public schools. They were also less likely to be special education students (as a result of the fact that researchers could not collect data on non-mainstreamed special education students). Outcome data were tracked for four cohorts beginning in 9th grade, starting with the 2005-2006 school year. The 2010 study tracked outcomes for four years and included the number of credits earned, the core subjects that were failed, on-time graduation from high school, and type of diploma received. The 2012 study tracked outcomes for six years of two of the cohorts and included on-time graduation from high school, type of diploma received, five-year graduation rate, and college readiness. College readiness was measured by New York State Regents Exams scores in math and English.

Key Evaluation Findings

Bloom, Thomson, and Unterman (2010) found the following:

• Approximately 58 percent of the treatment group (lottery winners enrolled in an SSC) versus 51 percent of the comparison group (lottery losers who did not receive an offer to an SSC) were on track to complete high school after their first year. This represents a statistically significant difference.

• The treatment group earned significantly more credits than the comparison group (1.4 and 1.3 more) in the third and fourth years of high school, and they had a significantly higher graduation rate (about 5 percentage points higher) four years after their scheduled start of 9th grade.

- 337 -

• At the end of each year of high school, the treatment group earned significantly more credits than comparison students, ranging from 0.9 to 2.6 more credits, depending on the cohort.

• At the end of the first year in high school, treatment students were significantly less likely (7.8 percent less likely) to have failed more than one semester of a core subject.

• Treatment students that enrolled in an SSC were significantly more likely (6.8 percent more likely) to have graduated from high school within four years.

• Students who graduate get one of three diplomas, from most basic to most stringent requirements: local, Regents, Advanced. There were no statistically significant differences in the type of diploma students received.

Bloom and Unterman (2012) found the following:

• Treatment students were significantly more likely (8.6 percent more likely) to have graduated from high school within four years.

• The difference in graduation rates was mainly driven by receipt of Regents diplomas, for which treatment students were significantly more likely (6.5 percent more likely) to receive. There were no statistically significant differences in receiving local or advanced diplomas.

• Treatment students were significantly more likely (7.6 percent more likely) to be college-ready in English (i.e., get a 75 or higher on the English Regents Exam).

• There was no statistically significant difference for college readiness in math.

• Subgroup analyses were conducted on levels of reading and math proficiency, low-income status, race/ethnicity and gender, and whether students had contact with the SSC before enrolling. All analyses indicated that treatment students had significantly higher graduation rates than comparison students, with the exception of Hispanic males and the race/ethnicity category of "other" for both males and females. The estimated effects for each subgroup ranged from 6.9 to 11.4 percent.

• There was no statistically significant difference in the five-year graduation rate (i.e., the difference between the percentage of students graduating after four and five years).

Probable Implementers

School districts or other school operators

Funding

The schools received start-up funding from philanthropic foundations such as the Bill and Melinda Gates Foundations, which provided grants for four- and five-year startup time periods, totaling around $400,000, in addition to support from the New York City Department of Education.

Implementation Detail

Program Design While the specific features of the small schools vary, the common design principles for each school that were emphasized in the proposal process include academic rigor, personalization, and community partnerships. There are also smaller educational units within each school (e.g., student advisory, small learning communities) that help teachers and other adults get to know students better.

The schools received technical assistance from the New York City Department of Education and teachers' and principals' unions. Additionally, schools received certain policy protections during their

- 338 -

start-up period to allow them to open with only one grade level, access supports to select and hire staff, and have greater control over budgets and programming.

Staffing SSCs are staffed with high school teachers and administrators. Curriculum SSCs do not use a specific curriculum.

Issues to Consider

SSCs received a "promising" rating. Students enrolling in SSCs are 10 percent more likely to graduate from high school on time, and this effect size meets the PPN requirements for a "promising" rating. It should also be noted that about three-quarters of the students were assigned by a random process and one-quarter by a process that might not have been random. The potential for non-randomness stems from students who apply to multiple school lotteries, where the probability of winning an earlier school lottery depended on the lotteries to which the student applied. As is common in studies of experiments, the study authors included an analysis of those students offered a seat in a small school (the group which the program intended to treat) and, among those offered, students who enrolled in a small school (the group that actually received the treatment, which is the subset of students that chose to take up the offer and therefore has greater degree of self- selection). The reported effects above are for those who enrolled. The effects on those who were offered a seat are approximately 20-30 percent smaller than the effects on those who actually received a seat. It is worth nothing that there is variation in the design of the schools. Additionally, the comparison group attended a variety of schools, including regular schools, other SSCs, and other innovative schools in New York. The schools described in this report were created under an early iteration of the New York City Department of Education's New School Development process. To learn more about what the New York City Department of Education's current process looks like, visit http://schools.nyc.gov/community/newschools/default.htm.

Example Sites

There are 123 Small Schools of Choice in New York City.

Contact Information

New York City Department of Education High School Enrollment Office HS_Enrollment@schools.nyc.gov 718-935-2399

Available Resources

New York City Department of Education website: http://schools.nyc.gov/ChoicesEnrollment/High/Choices/default.htm

- 339 -

http://schools.nyc.gov/community/newschools/default.htm

mailto:HS_Enrollment@schools.nyc.gov

http://schools.nyc.gov/ChoicesEnrollment/High/Choices/default.htm

Bibliography

Bloom, Howard S., and Rebecca Unterman, Sustained Positive Effects on Graduation Rates Produced by New York City's Small Public High Schools of Choice, New York City: MDRC, 2012. Bloom, Howard S., Saskia Levy Thompson, and Rebecca Unterman, Transforming the High School Experience: How New York City's Small Schools Are Boosting Student Achievement and Graduation Rates, New York City: MDRC, June 2010.

Last Reviewed

April 2013

Newborn Individualized Developmental Care and Assessment Program (NIDCAP)

Program Info

Outcome Areas Healthy and Safe Children Children Ready for School

Indicators Children ages 0 to 5 exhibiting age-appropriate mental and physical development Babies born weighing more than 5.5 pounds and improving outcomes for low birth weight babies Children experiencing good physical health

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Health Care Provider Type of Service Case Management Family Support Health Care Services Health Education Type of Outcome Addressed Cognitive Development/School Performance Mental Health Physical Health

Evidence Level Proven

Program Overview

The Newborn Individualized Developmental Care and Assessment Program (NIDCAP) offers an individualized and nurturing approach to the care of infants in neonatal intensive care unit (NICU) and special care nurseries (SCN). It is a relationship-based, family-centered approach that promotes the idea that infants and their families are collaborators in developing an individualized program of

- 340 -

support to maximize physical, mental, and emotional growth and health and to improve long-term outcomes for preterm and high medical risk newborns.

The therapeutic framework and method of NIDCAP provides early developmental support and preventive intervention, beginning immediately with birth. Numerous premature infants are born during or before the last trimester of gestation (beginning around 24 weeks), which is an exceedingly critical period for brain development. The infant's sensory experience in the environment of the NICU and SCN, including exposure to bright lights, high sound levels, frequent stressful and painful interventions, and diminished positive experiences, presents unexpected challenges to the immature brain during this sensitive period. The goal of the NIDCAP approach is to minimize the mismatch between the immature brain's expectations and the overstimulating environment. In turn, NIDCAP seeks to improve brain development and long-term outcomes. The NIDCAP approach uses methods of detailed documentation of an infant's ongoing communication to teach parents and caregivers skills in observing an individual infant's behavioral signals. These sometimes subtle signals provide the basis for interpreting what the infant is trying to communicate and can be used to guide parents and caregivers to adapt all interaction and care to be supportive of the infant's behavior. Suggestions for care are made in support of the infant's self-regulation, calmness, well-being, and strengths and the infant's sense of competence and effectiveness. Such suggestions begin with support, nurturance, and respect for the infant's parents and family, who are the primary co-regulators of the infant's development; and the suggestions extend to the atmosphere and ambiance of nursery space, the organization and layout of the infant's care space, and the structuring and delivery of specific medical and nursing care procedures and specialty care. These practices ensure that a developmental perspective and an infant's environment are incorporated into the infant's care (see Als, 1995).

Program Participants

Infants and their families at risk due to premature delivery, low birth weight, and/or requiring care in a NICU and SCN for various risk reasons.

Evaluation Methods

Many studies have evaluated NIDCAP. Resnick et al. (1987) studied 255 low-birth-weight babies (124 in a treatment group and 131 in a control group) and assessed them using the Bayley Scales of Infant Development in a blind evaluation. Becker et al. (1991) tracked the developmental and medical outcomes for preterm infants by looking at 21 infants prior to staff training and 24 infants after staff training. Als et al. (1994) investigated the medical and neurodevelopmental effects of NIDCAP on very low-birth-weight infants, and the randomly assigned treatment and control groups were followed up at eight years of age by McAnulty et al. (2009). Fleisher et al. (1995) randomly assigned 40 very low- birth-weight preterm infants (weighing less than 1250 g) who required mechanical assistance in breathing (ventilation) to one group that received NIDCAP or one that did not, and then they compared the developmental results for the two groups. Buehler et al. (1995) tested the program on a randomly assigned sample of 24 medically healthy preterm infants at low risk and compared them with a group of 12 healthy, full-term infants. Als et al. (2003) randomly assigned 92 very low-birth- weight preterm infants, weighing less than 1250 g and aged less than 28 weeks, to treatment and control groups. Als et al. (2004) randomly assigned 30 low-risk preterm infants to treatment and control groups in Boston, Mass., and conducted tests using magnetic resonance imaging to determine whether brain structures changed as a result of treatment. Kleberg et al. (2008) examined the effects of NIDCAP on 68 infants' behavioral and pain responses to painful and stressful eye examinations following premature delivery by randomly assigning infants to a treatment group that received NIDCAP treatment and then the subsequent eye examination. Van der Pal et al. (2008) conducted a randomized trial of Dutch preterm infants, examining the effect of NIDCAP on parent reports of health-related quality of life. Finally, Maguire et al. (2009) examined the effects of NIDCAP at one and two years corrected age by randomly assigning 148 infants born at less

- 341 -

than 32 weeks to treatment and control groups. (Corrected age is the age the child would be if the pregnancy had gone to full term.)

Key Evaluation Findings

Physical and health outcomes

• Infants in NIDCAP had shorter stays on respirators, supplemental oxygen, and feeding tubes (Becker et al., 1991; Als et al., 1994).

• NIDCAP babies started oral feeding sooner and had better average daily weight gains, shorter hospital stays, and improved overall behavioral functioning (Fleisher et al., 1995; Als et al., 1994).

• NIDCAP babies showed reduced need for the following: tube feeding (23 days for babies given treatment versus 37 days for babies not given treatment), positive airway pressure (38 days for the treatment group versus 60 days for the nontreatment group) and length of hospitalization (92 days versus 115 days) (Fleisher et al., 1995).

• Hospital costs were dramatically lower for infants in the NIDCAP group (an average of $362,000 versus $491,000 for the non-NIDCAP infants) (Fleisher et al., 1995).

• In a sample of very low-birth-weight preterm infants, NIDCAP participants had a shorter duration of intravenous feeding, a shorter duration of transition to full oral feeding, and shorter durations of intensive care and hospitalization; a lower incidence of necrotizing enterocolitis (a medical condition that primarily occurs in premature infants in which portions of the bowels undergo tissue death); reduced discharge ages and hospital charges; and improved weight, length, and head circumferences (Als et al., 2003).

• Infant pain profiles during eye examination were not different between the NIDCAP treatment and control groups in Kleberg et al. (2008).

• A study of infants born preterm (less than 32 weeks) found no improvement in neurological outcomes at one or two years (corrected age) (Maguire et al., 2009).

• Regarding a sample of preterm Dutch infants at one year of age, van der Pal et al. (2008) found no improvements in health-related quality of life as reported by parents among those infants who participated in NIDCAP compared with those in a control group.

Cognitive and behavioral outcomes

• The NIDCAP babies had a significantly lower incidence of developmental delay and scored significantly higher than the control group on mean mental and physical indexes at 12 and 24 months of corrected age (Resnick et al., 1987).

• Infants in NIDCAP had improved behavioral organization at two weeks corrected age and at nine months (Becker et al., 1991).

• In a sample of very low-birth-weight preterm infants, NIDCAP participants had enhanced autonomic, motor, state, attention, and self-regulatory functioning following the intervention compared with the control group (Als et al., 2003).

• A preterm NIDCAP group had better outcomes than a group of preterm non-NIDCAP infants in terms of behavioral performance and in the amount of activity found in the frontal lobe of the brain (Beuhler, 1995). This was also true among low-risk infants (Als et al., 2004).

NIDCAP Program Participants had improved behavior scores during a painful eye examination compared with control group participants (Kleberg et al., 2008).

• A study of infants born preterm (less than 32 weeks) found no improvement in mental or psychomotor outcomes at one or two years of corrected age (Maguire et al., 2009).

- 342 -

• A follow-up study of NIDCAP infants at eight years old found significantly better right hemisphere and frontal lobe function in the experimental group than in the control group (McAnulty et al., 2009).

Probable Implementers

Neonatal intensive care nurseries, special care nurseries, chronic infant care facilities, and lead agencies for special-education services (for example, departments of education or departments of health), lead agencies for the quality of neonatal care implementation (for example, the American Academy of Pediatrics and the Joint Commission on the Accreditation of Hospitals), and others.

Funding

Federal Special Education funds (Part C of the Individuals with Disabilities Education Act) and state departments of health; private funds or endowments accessed through hospitals.

Implementation Detail

Program Design

• All materials emphasize consistent caregiving by a primary team designated for each infant when the infant arrives in the NICU; all infants were observed before any hands-on interaction by the caregiver.

• Equipment is arranged aesthetically and in close proximity to ensure ready access to the infant at all times. Comfortable chairs are provided for parents' extended naps and overnight stays, and families are encouraged to personalize and decorate their infant's bed space with personal items (incubator cover, photos, stuffed animals).

• A peaceful and quiet care area is maintained for the infant—e.g., the infant is spoken to with a soft voice at all times, and NICU staff wear quiet shoes.

• Darkness is assured for the infant during sleep and low, muted light levels are maintained at all other times in order to support alertness. All light that falls on the infant's face is indirect.

• In timing caregiving, the infant's sleep-wake cycle alertness, medical needs, and feeding ability are considered.

• Trained developmental specialists are made aware of both infant and family development, and they support the primary care team. These specialists are full-time staff members.

• The infant is consistently supported and facilitated physiologically in well-aligned positions whether on the back, tummy, or side. Blankets, nesting, and swaddling help assure proper positioning. Supports were used to increase and build on the infant's ability to support him or herself, and the use of supports diminished as stability increased.

• The parents are supported from the beginning of the feeding process in their role as the infant's most important nurturer and provider of nutrition. Feedings are timed to be supportive of the infant's sleep-wake cycles so that the infant may learn to recognize feelings of hunger and satiation. Parents are supported to breastfeed.

In observing the infants, NIDCAP developmental specialists focus on the "stability" and/or "stress" signals communicated by the infant in relation to the following:

• Autonomic nervous system (signals related to breathing, heart rate, skin color, or body temperature or if the infant experiences tremors or is easily startled)

• Motor system (signals related to muscle tone, posture, and movement patterns)

• "State" organization system (signals such as robustness, sleep patterns, alertness, or crying)

- 343 -

• Attention/interaction system (processing and responding to the environment)

• Self-regulatory system (balancing of the various systems).

These observations are used to develop and implement a plan for each infant that addresses issues such as lighting, noise levels, positioning, touch, and timing of interventions. Curriculum

The specific NIDCAP system is proprietary but involves intensive and structured training of staff. Staff determine an appropriate method of interacting with each family on a case by case basis. Staffing

NIDCAP designates a primary care team for each infant within hours of the infant's arrival in the NICU. This team includes a developmental specialist, social worker, respiratory specialist, nurse, physician, and key family members. Some hospitals also have a developmental-care nurse/educator, other health care specialists, and/or parent representatives. Intensive training is needed to become a NIDCAP developmental specialist. Cost

The process of starting a NIDCAP center usually takes about five years and can involve up to 15 people at a site. The required training costs vary depending on training location. The costs are approximately $1,500 per day for each trainee, plus expenses, if the training is done on site.

Issues to Consider

This program received a "proven" rating. Several controlled experiments, including two with reasonable sample sizes (40 very low-birth-weight babies in one group and 255 low-birth-weight babies in another), have shown the program to be effective at improving outcomes for low- and very low-birth-weight babies. The program has been shown to be effective in various sites throughout the country, and several studies indicated that low- and very low-birth-weight babies who participated in NIDCAP showed gains similar to those of full-term babies. Not all of these improvements have, however, been shown to persist beyond the first months of life. Several studies examining outcomes at one year of age and beyond found no significant effect of the program, and critics have cited this as a reason to question the effectiveness of the program (Jacobs, Sokol, and Ohlsson, 2002). A meta-analysis of the NIDCAP randomized controlled trials published to date concluded that there was not as yet sufficient evidence to broadly recommend NIDCAP as a standard of clinical care, stating that not only did the randomized controlled trials all involve small sample sizes, but moreover they lacked follow-up into school age (Jacobs, Sokol, and Ohlsson, 2002).

Example Sites

Boston Children's Hospital, Colorado Consortium of Intensive Care Nurseries, Phoenix Children's Hospital, St. Vincent Hospitals (Indiana)

Contact Information

National NIDCAP Training Center Enders Pediatric Research Laboratories Room EN107 Children's Hospital Boston 320 Longwood Avenue Boston, Mass. 02115 617.355.8249 (phone)

- 344 -

617.730.0224 (fax) nidcap@childrens.harvard.edu (email) www.nidcap.org (Web site)

Available Resources

Resources include a NIDCAP Program Guide and training through NIDCAP training centers across the United States. Contact Boston Children's Hospital for ordering information at the address above or the NIDCAP Web site.

Bibliography

Als, H., Manual for the Naturalistic Observation of the Newborn (Preterm and Fullterm), Boston, Mass.: Children's Hospital, 3rd revision, 1981, rev. 1995. Als, H., Program Guide—Newborn Individualized Developmental Care and Assessment Program (NIDCAP): An Education and Training Program for Health Care Professionals, Boston, Mass.: Children's Medical Center Corporation, 11th revision, 1986, rev. 2008. Als, H., "A Synactive Model of Neonatal Behavioral Organization," Physical and Occupational Therapy in Pediatrics, Vol. 6, No. 3-4, 1986, pp. 3-55. Als, H., "Earliest Intervention for Preterm Infants in the Newborn Intensive Care Unit," in M. J. Guralnick, ed., The Effectiveness of Early Intervention, Baltimore, Md.: Paul Brookes Publishing, 1997, pp. 47-76. Als, H., F. H. Duffy, G. B. McAnulty, M. J. Rivkin, S. Vajapeyam, R. V. Mulkern, et al., "Early Experience Alters Brain Function and Structure," Pediatrics, Vol. 113, 2004, pp. 846-857. Als, H., G. Lawhon, F. H. Duffy, G. B. McAnulty, R. Gibes-Grossman, and J. G. Blickman, "Individualized Developmental Care for the Very Low Birthweight Preterm Infant: Medical and Neurofunctional Effects," Journal of the American Medical Association, Vol. 272, 1994, pp. 853-858. Als, H., L. Gilkerson, F. H. Duffy, G. B. McAnulty, D. M. Buehler, K. A. VandenBerg, et al., "A Three- Center Randomized Controlled Trial of Individualized Developmental Care for Very Low Birth Weight Preterm Infants: Medical, Neurodevelopmental, Parenting and Caregiving Effects," Journal of Developmental & Behavioral Pediatrics, Vol. 24, 2003, pp. 399-408. Becker, P. T., P. C. Grunwald, J. Moorman, and S. Stuhr, "Outcomes of Developmentally Supportive Nursing Care for Very Low Birthweight Infants," Nursing Research, Vol. 10, 1991, pp. 150-155. Browne, J. V., and S. Smith-Sharp, "The Colorado Consortium of Intensive Care Nurseries: Spinning a Web Support for Colorado Infants and Families," Zero to Three, Vol. 15, No. 6, 1995, pp. 18-23. Buehler, D. M., H. Als, F. H. Duffy, G. B. McAnulty, and J. Liederman, "Effectiveness of Individualized Developmental Care for Low-Risk Preterm Infants: Behavioral and Electrophysiological Evidence," Pediatrics, Vol. 96, 1995, pp. 923-932. Fleisher, B. F., K. A. VandenBerg, J. Constantinou, et al., "Individualized Developmental Care for Very- Low-Birthweight Premature Infants," Clinical Pediatrics, Vol. 34, 1995, pp. 523-529. Gilkerson, L., and Heidelise Als, Infants & Young Children, Vol. 7, No. 4, 1995, pp. 20-28. Gilkerson, L., and P. Gorski, "Hospital-Based Intervention for Preterm Infants and Their Families," in P. S. Panitz, J. Meisels, and J. P. Shonkoff, eds., Handbook of Early Childhood Intervention, New York: Cambridge University Press, 1990.

- 345 -

mailto:nidcap@childrens.harvard.edu

http://www.nidcap.org/

Harrison, H., "The Principles for Family-Centered Neonatal Care," Pediatrics, Vol. 92, No. 5, 1993, pp. 643-650. Jacobs, S. E., J. Sokol, and A. Ohlsson, "The Newborn Individualized Developmental Care and Assessment Program Is Not Supported by Meta-Analyses of the Data," Journal of Pediatrics, Vol. 140, No. 6, 2002, pp. 699-706. Kleberg, A., I. Warren, E. Norman, E. Mörelius, A. Berg, E. Mat-Ali, K. Holm, A. Fielder, N. Nelson, and L. Hellström-Westas, "Lower Stress Responses After Newborn Individualized Developmental Care and Assessment Program Care During Eye Screening Examinations for Retinopathy of Prematurity: A Randomized Study," Pediatrics, Vol. 121, No. 5, May 2008, pp. e1267-e1278. Maguire, C. M., Frans J. Walther, Paul H. T. van Zwieten, Saskia Le Cessie, Jan M. Wit, and Sylvia Veen, on behalf of the Leiden Developmental Care Project, "Follow-Up Outcomes at 1 and 2 Years of Infants Born Less Than 32 Weeks After Newborn Individualized Developmental Care and Assessment Program," Pediatrics, Vol. 123, No. 4, 2009, pp. 1081-1087. McAnulty, G. B., F. H. Duffy, S. C. Butler, J. H. Bernstein, D. Zurakowski, and H. Als, "Effects of the Newborn Individualized Developmental Care and Assessment Program (NIDCAP) at Age 8 Years: Preliminary Data," Clinical Pediatrics, May 15, 2009. Parker, S. J., L. K. Zahr, J. G. Cole, and M. Brecht, "Outcome After Developmental Intervention in the Neonatal Intensive Care Unit for Mothers of Preterm Infants with Low Socioeconomic Status," Journal of Pediatrics, Vol. 120, 1992, pp. 780-785. Resnick, M., F. Eyler, R. Nelson, et al., "Developmental Intervention for Low Birth Weight Infants: Improved Early Developmental Outcomes," Pediatrics, Vol. 80, No. 1, 1987, pp. 68-74. van der Pal, S. M., C. M. Maguire, J. Bruil, S. Le Cessie, J. M. Wit, F. J. Walther, and S. Veen, "Health- Related Quality of Life of Very Preterm Infants at 1 Year of Age After Two Developmental Care-Based Interventions," Child Care Health Development, Vol. 34, No. 5, September 2008, pp. 619-625. Wieder, S., R. Drachman, and T. DeLeo, "A Developmental/Relationship In-Service Training Model for Public Health Nurses Serving Multi-Risk Infants and Families," Zero to Three, Vol. 10, No. 1, 1989, pp. 16-20.

Last Reviewed

July 2009

Nurse Family Partnership

Program Info

Outcome Areas Healthy and Safe Children Children Succeeding in School Strong Families

Indicators Youths not using alcohol, tobacco, or illegal drugs Students performing at grade level or meeting state curriculum standards Children not experiencing physical, psychological or emotional abuse Families increasing economic self-sufficiency Babies born weighing more than 5.5 pounds and improving outcomes for low birth weight babies

- 346 -

Children and youth not engaging in violent behavior or displaying serious conduct problems Children experiencing good physical health

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Health Care Provider Home Visiting Type of Service Family Support Health Care Services Health Education Parent Education Type of Outcome Addressed Behavior Problems Child Abuse and Neglect Cognitive Development/School Performance Juvenile Justice Physical Health Poverty/Welfare Substance Use and Dependence Teen Sex/Pregnancy Violent Behavior

Evidence Level Proven

Program Overview

The Nurse Family Partnership program (previously named the Prenatal and Infancy Nurse Home Visitation Program) provides home visits by registered nurses to first-time mothers, beginning during pregnancy and continuing through the child's second birthday. The program has three primary goals: (1) to improve pregnancy outcomes by promoting health-related behaviors; (2) to improve child health, development, and safety by promoting competent caregiving; and (3) to enhance parent life- course development by promoting pregnancy planning, educational achievement, and employment. The program also has two secondary goals: to enhance families' material support by providing links with needed health and social services, and to promote supportive relationships among family and friends. The program was originally developed to address the underlying causes of antisocial behavior. Antisocial behavior is defined as behavior that violates social rules or harms others. When this behavior begins at an early age, it is likely to be more severe and is more likely to persist than antisocial behavior that begins in adolescence. Three main factors have been found to be associated with early onset of antisocial behavior:

• Neurodevelopmental impairment of the fetus. Children of women who engage in risky behaviors (cigarette, alcohol, or drug use) are more at risk for this kind of impairment.

• Dysfunctional care giving, which generally refers to inadequate parental provision of material and emotional care.

• Maternal life-course development. Children of women who are on welfare, are unmarried, are high school dropouts, or who have three or more children are more likely to have children with reported behavioral problems.

- 347 -

The three primary goals of the program directly address these three main risk areas. The content of the program is grounded in three theories: human ecology, human attachment, and the theory of self-efficacy.

• Human ecology theory emphasizes the importance of social context in human development. The program is introduced to first-time mothers because a first child represents a major change in the mother's life. The program continues into the early years of the child's life when the parent is still learning the parental role.

• Attachment theory argues that a caregiver's level of responsiveness to her child can be traced to her own childrearing history and attachment experiences. By helping the parent to see herself as someone who deserves support and attention, she begins to see her child as deserving the same.

• The theory of self-efficacy posits that differences in motivation and behavior are due to an individual's beliefs about how his or her efforts and the desired results are interconnected. Based on this theory, the program emphasizes helping mothers to set small achievable objectives that involve behavioral change and that will help them in dealing with similar problems in the future. Researchers conducting this intervention have observed that the women with the most success in the program originally felt they had very little control over their lives.

Since the first trial study in 1977, the program has evolved to put greater emphasis on the attachment and self-efficacy theories.

Program Participants

The program is designed to serve first-time mothers. It is particularly aimed at new mothers who have additional risk factors, such as low socioeconomic status, being unmarried, or being young (under 19).

Evaluation Methods

The program has undergone three randomized trials with different study populations. A study in Memphis, Tennessee, was mostly composed of African-American women; another in Elmira, New York, was primarily composed of white women. A third study in Denver, Colorado, involved women from a range of backgrounds, and this study was primarily focused on determining the program's effectiveness when delivered by paraprofessionals as compared with registered nurses. In the Memphis study, the researchers actively recruited women from prenatal clinics who had no previous live births, were fewer than 25 weeks pregnant, and who had at least two of the following sociodemographic risk conditions: were unmarried, completed fewer than 12 years of education, and were unemployed. Of the 1,290 eligible women, 1,139 (88 percent) consented to participate and were randomly assigned to one of four treatment conditions. Randomization was stratified by the following characteristics in order to have roughly comparable groups: maternal race, maternal age, gestational age at enrollment, employment status of head of household, and geographic region of residence. Group 1 (n = 166) received free prenatal care appointments and round-trip taxicab service, but they did not receive any postnatal treatment. Group 2 (n = 515) received the free screening plus transportation for prenatal and well-child care at 6, 12, and 24 months of age. These two groups were combined in all the evaluations to form the control group. Group 3 (n = 230) received the prenatal services offered to group 1, and nurse home visiting services during pregnancy. The women in group 4 (n = 228) received the pre- and post-natal services offered to women in group 2 plus the nurses continued the home visits through the child's second birthday. Ninety-two percent of the women enrolled in the Memphis study were black, 98 percent were unmarried, and 85 percent had household incomes at or below the federal poverty level.

- 348 -

The Elmira, New York, study used the same methods, but it had a much smaller treatment group. This study identified 500 eligible women and enrolled 400 in the program study. The Denver study was slightly different in its study design. Researchers recruited first-time mothers with no private insurance or who qualified for Medicaid. Of the 1,178 eligible women, 734 agreed to participate. These women were then assigned to one of three groups: a comparison group that received the usual screening (n = 255), a second group that received nurse visits through the child's second birthday (n = 235), and a third group that received visits from paraprofessionals through the child's second birthday (n = 244). Dropout rates in the three evaluations ranged from 15 to 21 percent. The main reason for dropouts was passive refusal (the mother could not be located or would not respond). Other reasons included refusing services, moving out of the area, or the death of the child. The trials looked at several indicators, including women's health-related behaviors during pregnancy, the quality of parental caregiving (including state-verified reports of child abuse and neglect), rates of subsequent pregnancy, educational achievement, participation in the workforce, and the use of welfare. Interviews and assessments were carried out at several points during the program with all those initially randomized. At 34 and 46 months, the staff carried out interviews and assessments in the home. The Caldwell and Bradley Home Inventory scales were used to measure different aspects of parental caregiving. Observation checklists assessed the mother's warmth for, control over, and involvement with her child and the child's exposure to various hazards in the home. At 36 and 48 months, children went to the project offices for standardized testing, such as the Stanford-Binet test of intelligence. The evaluations included interviews with the mothers. The 15-year follow-up of the Elmira study also included interviews with the adolescents who had been in the study and recorded measures of truancy; running away from home; criminal activity; pregnancy; and drug, alcohol, or cigarette use. These measures were matched with school data and teacher reports.

Key Evaluation Findings

Olds et al. (1986) found significant effects in the Elmira study:

• women who were visited by registered nurses were more aware of the community services available to them, attended childbirth education classes more frequently, indicated that the fathers of their babies showed a greater interest in their pregnancies, and were more frequently accompanied by someone to the labor room

• nurse-visited young adolescents (those 14 to 16 years old) had babies who were an average of 395 grams heavier than the babies of adolescents in the comparison group (3,423 grams compared with 3,028 grams)

• nurse-visited smokers made greater reductions in the number of cigarettes smoked than did smokers assigned to the comparison group (a four cigarettes-per-day difference by the end of pregnancy).

Olds et al. (1994) found for the period during which the Elmira children were between 25 and 50 months of age

• homes of nurse-visited families had significantly fewer hazards for children at the 34- and 46- month assessment

• nurse-visited children had 40 percent fewer notations of injuries and accidental ingestions and 45 percent fewer notations of child behavioral and/or parental coping problems in physicians' records

• during this period, nurse-visited children made 35 percent fewer visits to the emergency room.

- 349 -

Olds et al. (1997) found that compared with unmarried women of low socioeconomic status in the control group, nurse-visited women in Elmira of low socioeconomic status

• received significantly fewer months of public assistance

o 60.4 months of Aid to Families with Dependent Children versus 90.3 months among the control group

o 46.7 months of food stamps versus 83.5 months among the control group.

• had 43 percent fewer subsequent pregnancies

• delayed the birth of their second child an average of 12 months longer.

The 15-year evaluation (Olds et al., 1998a) of the Elmira trial found that compared with the control group

• children in group 4 had significantly fewer arrests, convictions, and violations of probation

• among adolescents born to unmarried women with low socioeconomic status, those in group 4 reported

o significantly fewer instances of running away

o significantly fewer sexual partners

o consuming alcohol on fewer days during the six months before the interview.

The evaluation of the Memphis trial (Kitzman et al., 1997), found

• nurse-visited mothers reported attempted breast-feeding significantly more frequently than comparison women and were more likely to use other community services

• by the 24th month of the child's life, nurse-visited women

o had significantly fewer health care encounters in which injuries and accidental ingestions were detected, and their children were hospitalized for fewer days

o had significantly fewer beliefs about childrearing that were associated with child abuse and neglect (such as lack of empathy, belief in physical punishment, unrealistic expectations for infants)

o reported having significantly fewer subsequent live births than women in the comparison group (36 percent versus 47 percent).

Olds et al. (2004) completed a follow-up of the children and mothers involved in the Memphis program at six years. They found

• nurse-visited women had

o significantly fewer subsequent pregnancies (1.16 versus 1.38) and births (1.08 versus 1.28)

o significantly longer relationships with their current partners (54 versus 45 months)

o fewer months using welfare (7.2 versus 9.0) and food stamps (9.7 versus 11.5)

• nurse-visited children had

o higher intellectual functioning as measured by the Kaufman Assessment Battery for Children

o higher vocabulary scores as measured by the Peabody Picture Vocabulary Test

o fewer behavior problems as measured by the Achenbach Child Behavior Checklist

- 350 -

• nurse-visited children born to women with low psychological resources (limited intellectual functioning, poor mental health, a low sense of control over their life circumstances) had

o higher arithmetic achievement test scores

o lower levels of aggression in their response to stories

o told less incoherent stories.

Olds et al. (2007) completed a nine-year follow-up of the mothers and children involved in the Memphis study. They found that nurse visited women compared to the control group had

• significantly fewer subsequent low birth weight infants (0.18 versus 0.27)

• significantly fewer subsequent births (0.8 versus 0.9)

• significantly longer relationships with their current partners (51 versus 44 months)

• fewer months per year using welfare (5.2 versus 5.92) and food stamps (6.9 versus 7.8)

• children that were less likely to die by preventable causes

• children with higher grade-point averages (2.7 versus 2.4) and higher achievement test scores in math and reading in grades one to three.

Probable Implementers

Nurses; public health, social welfare, and criminal justice officials; obstetricians; and pediatricians. Most typically, public health departments, visiting nurse associations, and hospitals will carry out the program.

Funding

The Elmira trial was originally funded from the research division of the Maternal and Child Health Bureau, with later funding from a variety of public and private resources. The Memphis trial was supported in 1987 by the Maternal and Child Health Bureau, the National Institute for Nursing Research, the Robert Wood Johnson Foundation, the W. T. Grant Foundation, the Pew Charitable Trusts, and the Carnegie Corporation. The Colorado Trust was the primary funder of the Denver trial. Dissemination sites are self-funded, most often using Temporary Assistance for Needy Families, Medicaid, child abuse prevention, maternal and child health, substance abuse prevention, and county funds. A RAND cost-benefit analysis (Karoly et al., 1998) evaluated the Elmira trial through the 15-year follow-up. The average cost of the program was $6,083. Savings accrued from four areas: increased tax revenues due to increased employment; decreased welfare outlays; reduced expenditures for education, health, and other services; and lower criminal justice system costs. Karoly et al. found that for high-risk families (unmarried mothers with low socioeconomic status), the overall savings is more than four times the cost of the program (average total savings of $24,694). They did not find a net savings for lower-risk families (average total savings of $3,775).

Implementation Detail

Program Design

• The home visitors focus on improving maternal health, promoting competent parenting, and enhancing parental life-course development.

• The program serves first-time, low-income mothers and their families.

• The home visits begin during pregnancy and continue through the second year of the child's life.

- 351 -

• The home visitors are highly trained registered nurses who follow specific protocols during each visit and carry a maximum of 25 cases.

• Home visitors involve family members and friends in the program and help families to use other community health and human services.

The nurse visits occur approximately every one to two weeks through most of the intervention, depending on the mother's delivery date or the age of the child. During the prenatal period, the nurses help women complete 24-hour diet histories and plot weight gains; try to facilitate a reduction in the use of cigarettes, alcohol, and drugs through behavioral analysis; and help women to identify the signs and symptoms of pregnancy complications and other health problems (urinary tract infections, sexually transmitted diseases, and hypertensive disorders). After delivery, the nurses also educate women on how to recognize health problems with their child and whom to contact when their child becomes ill. They promote parent-child interactions by helping parents to understand a child's communicative signals, enhancing parents' interest in playing with their children, and creating safer households. Nurses also help women clarify their goals and solve problems that may have interfered with completing their education, finding work, or planning future pregnancies.

Curriculum

The nurses are trained to follow a very specific set of protocols and home visit guidelines, which they then adapt to each family's strengths and needs.

Staffing

Home-visiting registered nurses provide the core staff of the program. Program nurses typically are hired through local health departments, community health centers, or Visiting Nurses Associations of America. In some cases, the program has been administered through local hospitals or nonprofit, private organizations.

Issues to Consider

This program received a "proven" rating. The program has undergone three randomized studies using large sample sizes (ranging from 400 to 1,189 women) and up to 15 years of longitudinal follow-up for the initial Elmira study. Comparisons between women who were visited by nurses and those who were not demonstrated significant effects from nurse visits on several measures of maternal health, maternal life-course development, child health and safety, and adolescent measures of delinquency. In all the trials, the program was most effective for first-time mothers who exhibited multiple risk factors, most commonly being unmarried, young, and of low socioeconomic status. Compared with lower risk women in the program, they were more likely to show increased employment and fewer subsequent births. Their children also showed greater gains; the children were less likely to run away from home, they had fewer sexual partners, and they consumed less alcohol. One implementation issue that remains unresolved is the comparison between nurses and paraprofessionals. The evaluation in Denver did show that there were some program differences between the two. Nurses concentrated on issues of personal health and parenting, whereas paraprofessionals emphasized environmental health and safety, social supports, and the mother's life- course development. Based on the goals of the program, the program developers suggest using nurses rather than paraprofessionals, but questions regarding cost-effectiveness and long-term effects still remain. In addition, nurses were more likely to retain contact with the mother and complete the program through the second year of the child's life than were paraprofessionals. The latter issue may be a factor in trying to keep the mothers in the program. Note that all of the evaluations of this program have been conducted by the developers of the program, Dr. David Olds and his associates, and NFP has not been studied by an external evaluator.

- 352 -

Example Sites

Original research sites: Elmira, New York; Memphis, Tennessee; and Denver, Colorado. The program also is being replicated in over 20 states in the United States.

Contact Information

Nurse-Family Partnership National Office 1900 Grant Street, Suite 400 Denver, CO 80203-4307 866.864.5226 (toll free) 303.327.4240 (main) 303.327.4260 (fax) info@nursefamilypartnership.org (email) www.nursefamilypartnership.org (Web site)

Available Resources

Detailed visit-by-visit protocols, training, and technical assistance are made available to organizations that commit to conduct the program in accordance with the program model and that have the financial resources and organizational capacity to do so. Specific training modules have been developed around issues of staff cultural competence. Further, program teaching materials have been translated into Spanish. An extensive list of publications is available on the research findings and the theoretical and clinical foundations of the program.

Bibliography

Karoly, Lynn A., Peter W. Greenwood, Susan S. Everingham, Jill Hoube, M. Rebecca Kilburn, C. Peter Rydell, Matthew Sanders, James Chiesa, Investing in Our Children: What We Know and Don't Know About the Costs and Benefits of Early Childhood Interventions, Santa Monica, Calif.: RAND, MR-898- TCWF, 1998. As of May 26, 2009: http://www.rand.org/pubs/monograph_reports/MR898/ Kitzman, H., et al., "Effect of Prenatal and Infancy Home Visitation by Nurses on Pregnancy Outcomes, Childhood Injuries, and Repeated Childbearing: A Randomized Controlled Trial," Journal of the American Medical Association, Vol. 278, No. 8, 1997, pp. 644-652. Olds, D., et al., "Does Prenatal and Infancy Nurse Home Visitation Have Enduring Effects on Qualities of Parental Caregiving and Child Health at 25-50 Months of Life?" Pediatrics, Vol. 93, 1994, pp. 89-98. Olds, D., et al., "Effects of Nurse Home Visiting on Maternal Life Course and Child Development: Age 6 Follow-up Results of a Randomized Trial," Pediatrics, Vol. 114, 2004, pp. 1550-1559. Olds, D., et al., "Effects of Nurse Home Visiting on Maternal Life Course and Child Development: Age 9 Follow-up Results of a Randomized Trial," Pediatrics, Vol. 120, 2007, pp. e832-e845. Olds, D., et al., "Effects of Prenatal and Infancy Nurse Home Visitation on Surveillance of Child Maltreatment," Pediatrics, Vol. 95, 1995, pp. 365-372. Olds, D., et al., "Improving the Delivery of Prenatal Care and Outcomes of Pregnancy: A Randomized Trial of Nurse Home Visitation," Pediatrics, Vol. 77, 1986, pp. 16-28.

- 353 -

mailto:info@nursefamilypartnership.org

http://www.nursefamilypartnership.org/

http://www.rand.org/pubs/monograph_reports/MR898/

Olds, D., et al., "Improving the Life-Course Development of Socially Disadvantaged Parents: A Randomized Trial for Nurse Home Visitation," American Journal of Public Health, Vol. 78, November 1988, pp. 1436-1445. Olds, D., et al., "Long-Term Effects of Nurse Home Visitation on Children's Criminal and Antisocial Behavior: 15-Year Follow-up of a Randomized Controlled Trial," Journal of the American Medical Association, Vol. 280, No. 14, 1998a, pp. 1238-1244. Olds, D., et al., "Long-Term Effects of Nurse Home Visitation on Maternal Life Course and Child Abuse and Neglect: Fifteen-Year Follow-up of a Randomized Trial," Journal of the American Medical Association, Vol. 278, No. 8, 1997, pp. 637-643. Olds, D., et al., "Reducing Risks for Antisocial Behavior with a Program of Prenatal and Early Childhood Home Visitation," Journal of Community Psychology, Vol. 26, 1998b, pp. 65-83.

Last Reviewed

May 2009

Parent-Child Home Program

Program Info

Outcome Areas Children Ready for School

Indicators Children ages 0 to 5 exhibiting age-appropriate mental and physical development

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Home Visiting Type of Service Parent Education Type of Outcome Addressed Cognitive Development/School Performance

Evidence Level Promising

Program Overview

The Parent-Child Home Program (PCHP) is a home visitation program that supports positive parenting practices and helps parents learn how to stimulate their children's cognitive and social-emotional development. PCHP home visitors meet with parents and their children in the home for a half hour, twice per week. On the first visit of each week, the home visitor brings a book or educational toy (known as verbal interaction stimulus materials, or VISM), which is the curricular material for the week, and demonstrates how to use the book or toy through reading, play, and conversation activities to promote school readiness, early literacy, and parent-child verbal interaction. The home visitor then leaves these materials with the parents so that they will be able to continue these behaviors with their

- 354 -

children throughout the week and after the program is concluded. PCHP home visitors meet with families for two 23-week cycles (46 home visits in each cycle) over a two-year period, for a total of 92 of sessions over 46 weeks (many sites operate on a school year calendar over the two years). By the end of two years, it is expected that PCHP will have provided families not only with a library of high- quality children's literature and educational toys, but also with the skills for using these items to stimulate children's cognitive development.

Program Participants

The Parent-Child Home Program is targeted at families that are challenged by poverty, isolation, and language, literacy, and cultural barriers and that have children aged two at the time of program initiation (however, the curriculum can be used with children as young as 16 months old). The program serves non-native-English-speaking families (the program has been delivered in more than 50 different languages), homeless families, grandparents raising grandchildren, teen parents, foster parents, and families in high-need communities.

Evaluation Methods

Two studies of the Parent-Child Home Program have met PPN criteria for evidence of a promising program. Madden, O'Hara, and Levenstein (1984) examined the impacts of PCHP on children's IQ scores and maternal behaviors. The study assessed PCHP with a sample of families with children aged 21 to 33 months who lived in one of four suburban areas surrounding New York City from 1973 to 1976. Ninety-five percent of families spoke only English at home, and 88 percent were African-American. There were four cohorts of families, with one cohort per study year. While PCHP program implementation remained consistent across the cohorts, there were some variations in the study methodology from year to year. For example, in 1973, families who agreed to participate in the study were assigned by lottery to receive PCHP (at the time, PCHP was called the "Mother Child Home Program") or to receive no intervention (18 children were in the intervention group, 16 in the control). In 1974, families were assigned by lottery to receive PCHP (N=22 children) or to receive the educational toys and books without any home visiting services (N=26 children). In 1975, the lottery reverted to contrasting PCHP relative to no intervention other than the home study assessments by the researchers (17 children were in the 1975 intervention group, 12 in the control). And in 1976, study families were recruited for an "early screening" program, which included an IQ test, and were then randomly assigned to receive an offer to participate in PCHP (N=29) or to not receive an offer (N=26). This last addition was intended to reduce the chances that comparison families would become aware that they were being compared against families who were receiving PCHP services and thus try to overcompensate for their lack of services in their behavior or responses during the study assessments. Despite the differences in the study design across the four years of the study, all families in the treatment group were delivered 92 sessions of PCHP over two years, regardless of which of the four parent cohorts they belonged to. All children in the study were administered the Stanford-Binet IQ test at the end of the intervention period. Children in the 1973 to 1975 cohorts were also administered the Peabody Picture Vocabulary Test (PPVT) test after the intervention period. All cohort children except the 1975 children were administered the Cattell IQ test prior to the intervention period. Children in the 1973 and 1974 cohorts were also administered the PPVT test before the intervention period. In addition, beginning with the 1974 cohort, the quality of mother-child interactions was assessed after the intervention period using the Maternal Interactive Behavior (MIB) record in an unstructured laboratory setting: Mothers were left alone in a room with their child at a table with small educational props and instructed that the child and mother may choose to play with the props as desired. In addition, the study measured differences between comparison group and PCHP group mean IQ assessment scores, adjusting for pretest differences in IQ assessment score. Scarr and McCartney (1988) studied a pilot implementation of the PCHP program in Bermuda that began in 1978. In this study, one Bermuda parish offered PCHP to families with children aged 24 to 30 months old. Eligible families were randomly assigned to receive the PCHP intervention or to serve as a

- 355 -

control family. The PCHP intervention was offered in 92 sessions (two sessions per week, 23 weeks per year) over the course of two years; however, some families opted to participate in only one session per week in the second year of the program, for a total of 69 sessions. Assessment outcomes included children's Stanford-Binet IQ scores, children's motivation to learn and cooperate, children's ability to delay gratification, children's enthusiasm and ability to respond to their mother's teaching, mothers' discipline techniques, and mothers' teaching abilities. The researchers compared the differences in mean scores on assessment measures between PCHP families and control families, accounting for pretest scores when available. Assessments were conducted prior to the intervention period, when children were 24 to 30 months old, and after the intervention period, when children were 42 to 48 months old. A total of 125 families participated in the study, but due to various circumstances (e.g., family refusing assessment or moving away) only 117 families provided assessments at both time points. Of these, 78 received the PCHP intervention and 39 served as control families. Only 66 of the 78 PCHP families participated in the full program for two years.

Key Evaluation Findings

Madden, O'Hara, and Levenstein (1984) found that, among children in the 1976 cohort, PCHP children scored 5.3 points higher than control children on the Stanford-Binet IQ test at the conclusion of the program (104.9 versus 99.6 mean IQ score). In addition, among 1974 and 1976 cohort families, PCHP mothers had significantly higher Maternal Interactive Behavior scores than comparison mothers. These improvements in parents' verbal behavior as measured by the MIB persisted on follow-up one and two years later. Despite these differences in total frequency of verbal interactions, there were no significant differences between PCHP and the control groups either in nonverbal demonstrations of affection or in verbal praise in any year. Scarr and McCartney (1988) found that, at the conclusion of the program, children in Bermuda who participated in PCHP had significantly higher scores than the comparison group children on the sorting task taught by their mothers and scored significantly higher on the communication skills test than control children. The study also found that mothers in the Bermuda PCHP group had higher post-test scores on self-reported activity sharing than Bermuda control mothers and were less prone to use pleading as a discipline technique than Bermuda control mothers. The study did not find that PCHP led to improvements in discipline techniques, mothers' teaching abilities, or parents' reporting of child behaviors.

Probable Implementers

Local partner agencies with an interest in home visiting models, early childhood development, school readiness, and particular target populations, including immigrant families, homeless families, teen parents, and grandparents raising grandchildren.

Funding

PCHP local agencies are primarily responsible for obtaining their own funding, with assistance in identifying funding sources, conducting outreach, and writing proposals from the PCHP National Center and PCHP regional staff. PCHP sites access funding from federal, state, and local public agencies, as well as from foundations, corporations, United Way, local service organizations, and local businesses.

Implementation Detail

Program Design

Home visitors meet with families twice per week for 23 weeks per year over two years, for a total of 92 sessions. In the first visit per week, the home visitor brings an educational toy or book for the family to keep. The home visitor uses the toy or book to model reading, play, and conversation interactive behaviors that will help stimulate children's intellectual and social-emotional development. The materials remain in the home so that parents can continue these interactive behaviors with their children throughout the week and after the program is completed. At the end of two years, PCHP will

- 356 -

have equipped parents with a library of educational toys and books as well as developed the parents' skills for stimulating their own children's intellectual and social-emotional development. In addition to the content of the home visits, as families make their needs and concerns known to program staff, the site coordinator takes an active role in providing referrals and facilitating connections to other community resources and identifying children who have developmental delays and are in need of early intervention and other services. Play groups, parent groups, workshops, and field trips may be offered to program families, although they are not required.

Staffing

Parent-Child Home Program sites are staffed with a site coordinator (typically a certified early childhood teacher or a social worker) who has completed an initial three-day Site Coordinator Training Institute and one day of follow-up training (during the first year of implementation) with the PCHP National Center. Site coordinators hire, train, and supervise home visitors on PCHP's core values, curriculum, best practices, and fundamentals. All home visitors must complete a minimum of 16 hours of training prior to beginning home visits and must participate in a two-hour weekly group supervision meeting, where the site coordinator provides ongoing professional development. In addition, all home visitors receive one-on-one supervision from the coordinator. A full-time home visitor can serve up to 16 families, although caseloads of 8-12 families per home visitor are more typical.

Curriculum

The core of the PCHP curriculum is the high-quality books, educational toys, and interactive activities that are the focal point of each home visit and follow an age-related developmental progression. Each site coordinator chooses the curricular materials based on detailed guidelines learned in the Site Coordinator Training Institute. These guidelines allow for customization based on particular language or cultural needs. Each toy, book, or activity has an accompanying curricular guide sheet that is given to the parent, which provides suggestions for additional ways to interact using the curricular material and enrichment activities. The curriculum is designed to guide home visitors in employing strategies that build on parent strengths, increasing positive parenting and the quality and quantity of parent- child verbal interaction. Coordinators are provided by the PCHP national center with a training/curricular toolkit of program manuals, DVDs, and training materials for home visitors.

Issues to Consider

The PCHP program has been examined in at least 15 studies since the late 1960s. Only two studies reviewed by PPN met the criteria for a promising program. The Madden, O'Hara, and Levenstein (1984) study found that PCHP children in only one of four cohorts showed significant increases in IQ test scores; the Scarr and McCartney (1988) study found no program effects on child IQ. In addition, Scarr and McCartney tested 17 child outcomes and 44 maternal outcomes (including sub-scales) and found significant differences on only four of the tested outcomes, two for mothers and two for children.

Example Sites

The PCHP program is currently offered in 14 U.S. states and internationally in Bermuda, Canada, and Ireland.

Contact Information

PCHP National Office: 1415 Kellum Place, Suite 101 Garden City, New York 11530-1690 Email: info@parent-child.org

- 357 -

mailto:info@parent-child.org

Phone: 516-883-7480 Fax: 516-883-7481

Available Resources

General information about the Parent-Child Home Program is available at its website, www.parent- child.org. The website includes a Program Overview, summaries of research supporting the program model, and information about starting up a new PCHP site, becoming a site coordinator, and becoming a PCHP home visitor. The website also provides a list of "favorite" books and educational toys.

Bibliography

Madden, John, John O'Hara, and Phyllis Levenstein, "Home Again: Effects of the Mother-Child Home Program on Mother and Child," Child Development, Vol. 55, No. 2, April 1984, pp. 636-647. Scarr, Sandra, and Kathleen McCartney, "Far From Home: An Experimental Evaluation of the Mother- Child Home Program in Bermuda," Child Development, Vol. 59, No. 3, June 1988, pp. 531-543.

Last Reviewed

March 2012

Parents as Teachers

Program Info

Outcome Areas Healthy and Safe Children Children Ready for School Children Succeeding in School

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Community-Based Service Provider Home Visiting Type of Service Family Support Health Care Services Parent Education Type of Outcome Addressed Behavior Problems Child Abuse and Neglect Cognitive Development/School Performance

- 358 -

http://www.parent-child.org/

Evidence Level Promising

Program Overview

The overarching philosophy of Parents as Teachers (PAT) is to provide parents with child development knowledge and parenting support. Through a four-part intervention model known as the Parents as Teachers Born to Learn® model, trained and certified parent educators offer support to families from pregnancy to the time the children entered kindergarten. The goals of PAT are to increase parent knowledge of early childhood development, improve parenting practices, detect developmental delays and health issues early, prevent child abuse and neglect, and increase children's school readiness and success. PAT program services include home visits to families; health, hearing, vision, and developmental screenings of children; parent group meetings; and a resource network that links families with needed community resources. The PAT Born to Learn model is a universal-access model, adaptable for families of all configurations, in all life circumstances, and from all types of communities. With funding from the Missouri Department of Elementary and Secondary Education and the Danforth Foundation, New Parents as Teachers (NPAT) began in 1981 as a pilot project for first-time parents of newborns. The program was expanded statewide and renamed Parents as Teachers in 1984. Since 1985, programs offering PAT services have expanded to all 50 states and to other countries. In the 2006-2007 program year, approximately 3,000 PAT programs served close to a quarter of a million families and over 300,000 children.

Program Participants

Families from pregnancy through age 5

Evaluation Methods

Pfannenstiel and Seltzer (1985) evaluated the effects of NPAT in the four original program sites in Missouri, where program services were provided to families from pregnancy to the time the children were three years old. The NPAT sample consisted of 75 families randomly selected from among participating families in each of the four sites, and these families were matched with 75 comparison families randomly selected from a governmental agency list of first-born children who were born during a specified time frame in the same geographic area as the PAT participants. Despite matching, the NPAT and control groups differed significantly on several characteristics, including parents' ages and educational levels at the time their children were born. The evaluation assessed the children's cognitive achievement, language ability, and social/developmental skills at age three. No pretest measures were available. A 1989 follow-up study to Pfannenstiel and Seltzer (1985) evaluated child outcomes at the completion of first grade (Pfannenstiel, 1989). Ninety-two percent of the original NPAT group (69 children) and 65 percent of the original comparison group (49 children) were located for the follow-up evaluation. Analyses comparing the follow-up samples to those in the original evaluation did not find any significant differences between groups on family-related variables, suggesting that the follow-up sample was representative of the original study sample of NPAT and comparison children. Student outcomes were compared on reading and math achievement, teacher assessments of academic progress and social/behavioral adjustment in the classroom, and parent perceptions of their child's academic and social performance. Drazen and Haust (1993) assessed a replication of the PAT program, Parents and Children Together (PACT), which was implemented in Binghamton, New York. PACT was initiated in 1987 and served 509 primarily Caucasian families. In the study, the cognitive, language, and physical development of the 20 oldest, most at-risk Program Participants at ages four to five were compared to control children (who had not participated in PACT) matched on various risk factors such as low income. Analyses

- 359 -

confirmed no significant differences between the two groups on these risk factor variables. Outcomes were compared on assessments of physical development, cognitive achievement, language skills, report of suspected abuse or neglect, and family income in each group. Drazen and Haust (1995), in a follow-up to Drazen and Haust (1993), studied a larger sample of PACT participants. The study sample involved all children born in 1987 who entered kindergarten in 1992 in the Binghamton district and were still enrolled in the system in 1996. Of this group, close to 10 percent (47 children) had participated in PACT until age three, while 412 had not (and formed the comparison group). Only 41 of the 47 PACT children completed outcome measures. Children in both groups were similar on all demographic measures, except that the PACT children were more likely to have at least one parent employed, were more likely to have had mothers with complications of pregnancy or delivery, and were significantly more likely than control children to have attended prekindergarten and/or other preschool programs. Child outcomes included a measure of kindergarten readiness (given to students at the beginning of kindergarten), tests of reading and mathematics readiness at the end of kindergarten, grade point average, and referrals for special education services. Wagner, Cameto, and Gerlach-Downie (1996) studied the effects of a two-year PAT intervention among a sample of teen parents in California. The study involved four groups: a PAT group, a group that was given comprehensive case management, a group that combined PAT services with comprehensive case management, and a control group that received no direct services, but did receive regular mailings of child toys. A total of 717 teen mothers were recruited, more than half of whom were Latina, 20 percent were African American, and 20 percent were Caucasian. Almost one- third of the participants received Aid to Families with Dependent Children, two-thirds were enrolled in a high school program, and 30 percent were high school dropouts. Child outcomes were assessed at birth (for weight), and at one and two years on developmental scales (including physical, self-help, social, academic/cognitive, and communication) and Children's Protective Services data on child abuse/neglect reports. In addition, school dropout rates and pregnancy rates of teen mothers were compared. Overall, 402 teens dropped out of the demonstration between enrollment and the child's second birthday, for an overall attrition rate of 57 percent. The dropout rate was fairly evenly distributed among the groups, and the total sample of participants remaining in the study was 315. When the characteristics of program dropouts and those who stayed in the study (participants) were compared, a significant difference was found for rate of high school dropout (35 percent among program dropouts, and 24 percent among Program Participants). The effects of PAT were studied by Coleman, Rowland, and Hutchins (1997) in a sample of three groups from a rural Southeastern county. The first group included 21 families who had entered the PAT program within four months of their child's birth, who had remained in the program until their child was 30 to 36 months of age, and who had enrolled their child in public school kindergarten at age five. These 21 families were from an original starting sample of 97 families (i.e., a five-year retention rate of 22 percent). The second group consisted of 22 children who did not receive PAT services, but whose parents requested and received quarterly educational newsletters during the first year following their child's birth. The newsletters were written by PAT staff and included identical information as that provided to the PAT families. The third group of 22 children served as the no- treatment control group, receiving neither PAT services nor newsletters. Demographic variables were used to match PAT families as closely as possible to newsletter-only families and control families. Child outcomes included a cognitive profile, language profile, gross motor profile, and a self help profile assessed at kindergarten entry. Another study by Wagner and colleagues assessed the effects of a four-year PAT intervention in a sample of parents from Northern California (Wagner et al., 1999). Families with infants up to six months old were randomly assigned to PAT or to a control group that did not receive PAT services but did receive age-appropriate toys through the mail at regular intervals. A total of 497 families [does PAT deal with "families" or does it sometimes specifically target "mothers?"] enrolled in the study (199 in the PAT group and 298 in the control group), of which 81 percent were Latina. Both groups received annual child assessments at or around the children's birthdays until the children turned three. Of the 497 who enrolled, a total of 315 families participated in the age-one assessment (175 PAT group and 140 control). One year later, a total of 375 families participated in the age-two assessment (220 PAT and 155 control), and after another year 363 participated in the age-three assessment (210 PAT and 153 control). The PAT and control groups did not differ significantly from each other on any dimension

- 360 -

that was measured at enrollment, however at the one-year assessment the control group had significantly fewer Latina mothers than the PAT group (74 percent versus 84 percent). Child outcomes were assessed on the Peabody Picture Vocabulary Test, the Bayley Scales of Infant Development, and a child developmental profile that included physical, self-help, social, academic/cognitive, and communication dimensions. Wagner et al. (2001a) studied the effects of PAT in a randomized control trial of 667 children from three geographically dispersed sites, including one on the Eastern Seaboard (Site 1), one in a midsize Southern city (Site 2), and one in a large Western city (Site 3). Selected sites had been in operation for at least two years, served a high proportion of low-income families, and offered home visits at least monthly. Families with infants up to eight months old were recruited and randomly assigned to PAT or to a control group. Most of the background characteristics of treatment and control children were similar. However, in Site 1, significantly more PAT children than control children had Caucasian mothers, while in Site 2 significantly more PAT children than control children had siblings. In Site 3, PAT parents reported much more knowledge about infants than control parents did. Outcomes were assessed at or around the children's second birthdays, while participants were still receiving PAT services. Sixty-four percent of families in both treatment and control groups participated in at least one of the assessments—a child developmental profile that included physical, self-help, social, academic/cognitive, and communication dimensions. Wagner et al. (2001b) conducted an age-three follow-up in Site 1 of the original study Wagner et al. (2001a). Site 1 originally included a sample of 206 families, and 181 families were retained in the sample at the age-three assessment. No significant differences in demographic or background variables were seen between the treatment and control groups. Child outcomes were assessed on a child developmental profile that included physical, self-help, social, academic/cognitive, and communication dimensions and a parent report of child pro-social behaviors. Finally, Wagner et al. (2002) used data from Wagner et al. (2001b) to determine whether the impact of PAT on certain parental and child development measures was different when data was stratified by family income. Family income was divided into two categories, very low income (less than $15,000 per year for the household) and moderate income (more than $15,000 per year for the household).

Key Evaluation Findings

The original study of 75 NPAT children and 69 comparison children (Pfannenstiel and Seltzer, 1985) found the following:

• NPAT children scored significantly higher than comparison children in their ability to organize components to solve problems, intellectual functioning, verbal intelligence, and general knowledge. No significant differences were found between groups on the measure of sequential processing (e.g., word order and number recall).

• NPAT children also significantly outscored comparison children on all three measures of language skills, including auditory comprehension, verbal ability, and language ability.

• In terms of social/developmental skills, NPAT children were rated more positively than comparison children on 8 of the 21 social development items rated by researchers. On 13 of the 44 parental assessment items, NPAT parents rated their children significantly higher than did comparison parents, including, for example, on "separating easily from parents," "describing his/her feelings," and "differentiating present and future social roles." Comparison parents rated their children significantly higher than did NPAT parents on one item measuring how frequently their child enjoyed playing with other children. In all instances the effect size, while significant, was small. On the remaining 30 parental assessment measures there were no significant differences

In the follow-up study examining NPAT children at the end of first grade, Pfannenstiel (1989) reported:

• NPAT children scored significantly higher than did comparison children on longitudinal math achievement, with an average math score in the 82nd percentile compared with the 74th

- 361 -

percentile. No significant differences were found between NPAT and comparison children for reading achievement.

• There were no significant differences between groups in teacher assessments of children's academic progress and social/behavioral adjustment. The one exception was spelling, on which teachers rated NPAT children significantly higher than they did comparison children.

• There were no significant differences between groups in parental assessment of academic and social performance.

Drazen and Haust's (1993) study of 40 at-risk PACT (Parents and Children Together, a replication of the PAT program) and control group participants found:

• There were no significant differences between groups in average scores on the measure of cognitive abilities (only 13 PACT and 11 control students were tested).

• PACT students scored significantly higher than control students on language skills with significantly fewer PACT graduates (30 percent) than control students (65 percent) scoring below their age level on the test.

• Significantly more control participants than PACT participants had gross motor developmental delays on the total score of the developmental screening test.

• No significant differences were found between groups for the family report of suspicions of child abuse or neglect.

• Welfare dependence (receipt of AFDC) in both groups doubled between the time of the children's birth and one year later. From a child's first to second birthday, marginally significant differences emerged between groups, with welfare dependence declining in the PACT group (from 40 percent to 30 percent), and increasing among control families (from 30 percent to 50 percent).

The larger evaluation of PACT including 41 treatment and 412 comparison children (Drazen and Haust, 1995) found the following:

• Children whose families participated in PACT had significantly higher school readiness scores on the three tests that were used (kindergarten, math, and reading readiness) than those whose families did not participate.

• PACT children also had significantly higher grades in kindergarten than did control children, with computed averages of 95 percent versus 93 percent.

• A significantly lower proportion of PACT participants than control participants were enrolled in remedial special education in first grade (14 percent versus 31 percent).

The four-group randomized control trial by Wagner, Cameto, and Gerlach-Downie (1996) found that:

• There were no significant differences among teen parents in the four groups for high school dropout rates.

• While no significant differences were found among groups for having a subsequent pregnancy during the study period, significantly fewer PAT-only mothers than control group mothers had multiple pregnancies during this period (1.4 percent versus 4.8 percent). No significant differences were found among the PAT-only, case management, or combined intervention (PAT plus case management) groups.

• For those who entered the study while pregnant, mothers in the PAT-only group had marginally lower rates of low birth weight babies than did mothers in the control group (4 percent versus 8 percent). No significant differences were found among the PAT-only, case management, or combined intervention groups.

• There were no significant differences among groups for Child Protective Services reports of child abuse or neglect, although the combined intervention group was significantly less likely

- 362 -

than the control group to have an opened case of child abuse or neglect (0 percent versus 2 percent). No significant differences were found among the PAT- only, case management, or combined intervention groups.

• No significant differences were found among groups at the one-year assessment for child scores on the developmental tests. At the two-year assessment, both the case management and combined intervention groups had significantly higher scores for cognitive development than did PAT-only or control groups. Additionally, the case management, combined intervention, and PAT groups scored significantly higher than the control group on the social development scale at age two. No significant differences were noted among groups regarding the children's physical, self-help, or communication development.

• When the sample was limited to teens who received the expected or recommended level of home visits or direct case management during their child's second year, those in the PAT-only group had more positive outcomes. Effects were again strongest for cognitive development, with both the combined intervention and the case management group (but not the PAT-only group) scoring significantly higher than the control group. The combined intervention, case management, and PAT-only groups also scored significantly higher than the control group on social development. No statistically significant differences were found among groups for physical, self-help, or communication development.

The study by Coleman, Rowland, and Hutchins (1997) of the PAT, newsletter-only, and control groups reported the following:

• PAT children scored significantly higher on the average language age equivalencies than children from the newsletter-only families and children from the control group. No significant differences were found between the newsletter-only and control groups.

• PAT children scored significantly higher on the average self-help/social age equivalencies than children from either of the other groups. In addition, the average age equivalency of children from newsletter-only families was significantly higher than the average of children from control families.

• No significant differences were found between groups on the cognitive or the motor development profiles.

The Northern California PAT study (Wagner et al., 1999) reported the following:

• For the total sample (including children of both Latina and non-Latina mothers), scores on the researcher-assessed mental development scale indicated that one-year-olds in the PAT group significantly outscored one-year-olds in the control group. No significant differences were noted for two-year-olds or three-year-olds.

o Analyses comparing subgroups by ethnicity found that while one-year-old PAT children of Latina mothers significantly outscored controls, three-year-old control children of non- Latina mothers significantly outscored their PAT group counterparts.

• For the total sample, results for the self-help scale found that children in the PAT group significantly outscored children in the control group at both the one-year and three-year assessments.

o Further analysis suggests these findings may have been primarily attributable to the children of Latina mothers, as no significant findings were noted for children of non-Latina mothers.

• For the social development scale, children in the control group significantly outscored children in the PAT group at the age-two assessment.

o Subgroup analysis suggests that the above finding was consistent for children of non- Latina mothers only. Conversely, three-year-old children of Latina mothers in the PAT group significantly outscored controls on the measure of social development.

- 363 -

• On the physical development scale, children in PAT significantly outscored children in the control group at the one-year assessment. These results were consistent for both children of Latina mothers and children of non-Latina mothers.

• For the total sample, no significant differences were found between the PAT and control groups on the communication portion of the child development profile, the parent-reported cognitive development scale, or the Peabody Picture Vocabulary Test.

o For children of Latina mothers, significant differences were found on the parent-reported cognitive development scale for three-year-olds.

Wagner et al.'s (2001a) randomized control trial reported:

• No significant differences between PAT and the control groups on either the measure of cognitive and physical development or the parent report of adaptive social behavior inventory at age two.

o When the subgroup of very-low-income children was assessed separately, a significant difference favoring the PAT group was found for adaptive social behavior.

Similarly, Wagner et al.'s (2001b) age-three follow-up of one site from Wagner et al. (2001a) reported:

• PAT children showed significant, positive effects relative to control group children in the area of self-help development at the three-year assessment.

• There were no significant differences between PAT and control groups on the measures of cognitive development, physical development, or the parent report of adaptive social behavior inventory.

Using the same data as Wagner et al (2001b), Wagner et al (2002) found:

• Very low income PAT children showed positive effects of PAT on child and parental development outcomes compared to the control group; however these differences were generally not statistically significant.

• "Moderate" income PAT children showed mixed results on child development and parenting outcomes relative to the control group, with those differences generally not being statistically significant.

Probable Implementers

School districts, family resource centers, nonprofit organizations, public health and social service agencies, government agencies, and child care centers.

Funding

The NPAT program was originally funded by the Missouri Department of Elementary and Secondary Education in cooperation with the Danforth Foundation. The PAT expansion was legislated and partially funded by the Missouri General Assembly. Funds for other replications of the program have been generated from a variety of sources, including federal funds from Goals 2000: Educate America Act, Early Head Start, Even Start, and Title I. State and local dollars from school districts, corporate funders, private foundations, and social organizations can also be used. Parents as Teachers National Center (PATNC) estimates program costs of $1,400 to $1,500 per family annually. Costs vary depending on service intensity, as well as on program location and the availability of in-kind contributions. Program start-up costs for implementing the PAT Born to Learn® Curriculum, Prenatal to 3 Years are approximately $595 for each parent educator to attend the one-week initial

- 364 -

training, plus an additional $295 for the curriculum materials, not including the cost of the parent educator's time or cost of transportation, lodging, and meals during the training.

Implementation Detail

Program Design

Programs offering PAT services are local implementations that may differ somewhat depending on their sponsoring agency, but all PAT programs provide the same four core services:

• Personal Visits: Personal home visits are the major service delivery component of the program. During these visits, parent educators share age-appropriate child development information with parents, help them learn to observe their own children, address their parenting concerns, and engage the family in activities that provide meaningful parent-child interaction. Visits are usually one hour long and are scheduled bimonthly, biweekly, or weekly, depending on family needs and local program budgetary restrictions.

• Group Meetings: Parent group meetings provide opportunities for parents to share information about parenting issues and child development. Parents learn from and support each other, observe their children with other children, and practice parenting skills. Many programs also offer informal drop-in and play times for families.

• Screening: Periodic child screening provides for early identification of developmental delays and health, vision, and hearing problems. Each child’s developmental progress is also reviewed regularly.

• Resource Network: Parent educators help families identify and connect with community resources. Programs take an active role in establishing ongoing collaborative relationships with other organizations that serve families.

Curriculum

The core of the program is the Born to Learn® Curriculum, Prenatal to 3 Years. It is an age-related curriculum that progresses through the mother’s pregnancy and monthly throughout the child’s first three years. The curriculum is designed to promote optimal child development and positive parent- child relationships. The content is adaptable, and parent educators can individualize the curriculum to the particular strengths, weaknesses, and needs of the participating families. It includes parent handouts to reinforce learning between visits. Staffing

Programs are implemented by trained parent educators. The majority of parent educators have at least a four-year college degree. Implementers tend to be nurses, teachers, social workers, and, in some communities, paraprofessionals. All parent educators must complete the Born to Learn Institute which requires 33 hours of pre-service training for initial certification, and continuing professional development for annual recertification.

Issues to Consider

This program received a "promising" rating. The three quasi-experimental evaluations find, generally, that PAT participants significantly outscored control group children on measures of language skills, cognitive abilities, physical development, and social development. In addition, one of the randomized controlled trials (Wagner et al., 1999) reports several significant program effects on social, self-help, physical, and cognitive dimensions—although effects varied by child's age and a significant difference was found for only one of three cognitive tests. No significant effects were found in either of the two studies that assessed child abuse and neglect outcomes, although Wagner (1999) did find an effect on these outcomes when PAT was combined with case management among teen mothers.

- 365 -

Despite the generally positive findings for the three quasi-experimental studies and one randomized control trial, it is important to consider that the Wagner et al. (2001a, 2001b) randomized controlled studies did not find any significant differences between children who participated in the PAT group and those in the control group. Further analyses of these studies by Wagner et al. (2002) found significant effects on PAT children as compared to the control group for three of the 45 outcomes measured by the study; however, given the large number of outcomes measured, a finding of only three significant outcomes may in fact be due to random chance rather than program success. Furthermore, although the three quasi-experimental studies show promising results for the PAT program, none used baseline (pretest) data on outcomes of interest to control for group differences at the start, two studies had small sample sizes, and two studies reported baseline demographic characteristics showing PAT participants were more advantaged than control participants.

Example Sites

All school districts in the state of Missouri currently offer PAT services. In addition, the program has been implemented in the 49 other states, as well as Australia, Belize, Canada, China, Germany, Mexico, New Zealand, and the United Kingdom.

Contact Information

Karen Guskin, Ph.D. Parents as Teachers National Center 2228 Ball Drive St. Louis, Mo. 63146 Tel (314) 432-4330 or 1-866-PAT-4YOU (1-866-728-4968) Fax (314) 432-8963 info@parentsasteachers.org http://www.parentsasteachers.org/

Available Resources

Parents as Teachers National Center (PATNC) is a not-for-profit organization established in 1987 to provide leadership and training for Parents as Teachers programs. A variety of curricular and training materials are available through PATNC. These materials are for organizations and early childhood professionals; they are not for use by individual parents. Available resources include: Born to Learn® Curricula — Each contain detailed personal visit plans, activities for parent/child engagement, child development information, professional resources and parent handouts. Some have been translated into other languages, including Spanish, French, German and Mandarin.

• Born to Learn® Curriculum Prenatal to 3 Years

• Born to Learn® Curriculum 2 Years to Kindergarten Entry

• Born to Learn® Curriculum 3 Years to Kindergarten Entry

Other Born to Learn resources

• A Closer Look: The PAT Standards and Self-Assessment Guide, which provides standards and quality indicators for implementing the PAT Born to Learn® model, as well as assistance for conducting a self-assessment.

• Supervisor's Manual and Program Administration Guide, a desktop resource for PAT program supervisors that details program administration, supervision, advocacy, fund development, marketing, and evaluation strategies.

- 366 -

mailto:info@parentsasteachers.org

http://www.parentsasteachers.org/

• Supporting Military Families provides resources and parent materials for programs serving military families. It addresses issues faced by military parents and families including relocation, deployment, reunification, and combat stress.

• Nutrition and Fitness of Young Children focuses on nutritional requirements, feeding behaviors, and fitness issues for children birth to age three. Personal visit plans, educator resources and parent handouts follow the Born to Learn curriculum format. (Parent handouts have been translated for Spanish-speaking families.)

In addition, PATNC offers numerous professional development courses and materials in traditional and distance/self-study education formats.

Bibliography

Coleman, Mick, Bobbie Rowland, and Betty Hutchins, Parents as Teachers: Policy Implications for Early School Intervention, paper presented at the Annual Conference of the NCFR Fatherhood and Motherhood in a Diverse and Changing World (59th), Arlington, Va., November 7-10, 1997. Drazen, Shelley M., and Mary Haust, Raising Reading Readiness in Low-Income Children by Parent Education, paper presented at the Annual Convention of the American Psychological Association (101st), Toronto, Ontario, Canada, August 23, 1993. Drazen, Shelley M., and Mary Haust, The Effects of the Parents And Children Together (PACT) Program on School Achievement, Binghamton, N.Y.: Community Resource Center, 1995. Pfannenstiel, Judy C., New Parents as Teachers Project: A Follow-Up Investigation, Overland Park, Kan.: Research and Training Associates, 1989. Pfannenstiel, Judy C., and Dianne A. Seltzer, Evaluation Report: New Parents as Teachers Project, Overland Park, Kan.: Research and Training Associates, 1985. Pfannenstiel, Judy C., and Dianne A. Seltzer, "New Parents as Teachers: Evaluation of an Early Parent Education Program," Early Childhood Research Quarterly, Vol. 4, 1989, pp. 1-18. Wagner, Mary, and Serena Clayton, "The Parents as Teachers Program: Results from Two Demonstrations," The Future of Children, Vol. 9, No. 1, 1999, pp. 91-115. Wagner, Mary, Donna Spiker, and Margaret Inman Linn, "The Effectiveness of the Parents as Teachers Program with Low-Income Parents and Children," Topics in Early Childhood Special Education, Vol. 22, No. 2, 2002, pp. 67-81. Wagner, Mary, Donna Spiker, Frances Hernandez, Julia Song, and Suzanne Gerlach-Downie, Multisite Parents as Teachers Evaluation: Experiences and Outcomes for Children and Families, Menlo Park, Calif.: SRI International, 2001a. Wagner, Mary, Elizabeth Iida, Donna Spiker, Frances Hernandez, and Julia Song, The Multisite Evaluation of the Parents as Teachers Home Visiting Program: Three-Year Findings from One Community, Menlo Park, Calif.: SRI International, 2001b. Wagner, Mary, Renee Cameto, and Suzanne Gerlach-Downie, Intervention in Support of Adolescent Parents and Their Children: A Final Report on the Teen Parents as Teachers Demonstration, Menlo Park, Calif.: SRI International, 1996. Wagner, Mary, Serena Clayton, Suzanne Gerlach-Downie, and Mary McElroy, An Evaluation of the Northern California Parents as Teachers Demonstration, Menlo Park, Calif.: SRI International, 1999.

- 367 -

Last Reviewed

November 2008

Parents' Fair Share

Program Info

Outcome Areas Strong Families

Indicators Fathers maintaining regular involvement with their children

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Community-Based Service Provider Type of Service Family Support Parent Education Type of Outcome Addressed Poverty/Welfare

Evidence Level Promising

Program Overview

The Parents' Fair Share (PFS) demonstration program, implemented from 1994 through 1996 (with an initial pilot phase from 1992 to 1994), was a national demonstration project authorized by the Family Support Act of 1988. PFS was designed and evaluated by the Manpower Demonstration Research Corporation (MDRC). The goals of the program included helping unemployed, noncustodial parents (primarily fathers) to secure employment, pay child support, and participate more fully and responsibly as parents. The PFS program was designed as an alternative to standard child support enforcement. The program offered services in four areas: employment and training, modified child- support enforcement, peer support, and voluntary mediation services with the custodial parent. PFS services were provided through newly developed coalitions of governmental child-support enforcement agencies, employment and training agencies at the state and community level, and private community service organizations. The PFS program was originally implemented in seven states, including Dayton (OH), Grand Rapids (MI), Jacksonville (FL), Los Angeles (CA), Memphis (TN), Springfield (MA), and Trenton (NJ). Most of the sites that continued the PFS program through to the end of the demonstration project adapted the original program model based on their initial experiences.

- 368 -

Program Participants

PFS was targeted at underemployed or unemployed noncustodial fathers who owed child support and had children who were receiving welfare. Specific eligibility criteria for PFS required that the noncustodial parent: (1) have a child-support order in place, (2) be behind in making payments, (3) be the parent of a child whose custodial parent had received public assistance for the child, (4) be unemployed or underemployed, and (5) have attended a court-ordered hearing for not paying child support. Almost all participants (98 percent) were men living in or on the edge of poverty, and who had histories of sporadic employment. Their average age was 31 years, and over 80 percent were African American or Hispanic. Two-thirds of the participants had never been married, and 60 percent lived with a relative. Nearly 70 percent of the participants reported that since the age of 16, they had been arrested at least once on a charge unrelated to child support. Half of the participants did not have a high school diploma or general equivalency diploma, and only 2 percent had attended some college.

Evaluation Methods

MDRC began the seven-site, large-scale evaluation of PFS in 1994. In three sites, the demonstration project involved random assignment of parents to treatment groups at two different stages. The first stage took place when nonpaying, noncustodial parents in welfare-related cases were identified as potential PFS referrals, at which point over 6,800 parents were randomly assigned to either an extra outreach group that was subject to extra outreach and case review or a standard group subject to the site's usual child support enforcement practices. In the second stage, noncustodial parents in the extra outreach group who appeared at a hearing or case review were judged as eligible for PFS and were randomly assigned to either the PFS program group (subject to PFS services and mandates) or a control group that did not receive those services and was subject to normal child support enforcement practices. Members of the control group were free to participate in other services in the community on their own initiative. In the remaining four sites, random assignment took place only at the second stage, after which noncustodial parents appeared for hearings or case reviews and were deemed eligible for PFS. Across all sites, approximately 5,600 noncustodial parents were randomly assigned to either the PFS program group or the control group. Knox and Redcross (2000) continued the evaluation of PFS, assessing program effects on paternal child support and father's contact with youngest child. Levels of both "formal" child support (child support that is required by and paid directly to the child support enforcement system) and "informal" child support (support that is either cash or in-kind support that the noncustodial parent provides directly to the custodial parent) were measured. Data used in the analysis consisted of a sample of 2,005 custodial parents who responded to a follow-up survey. The survey sample included parents who were named in the child support cases of noncustodial parents who had been randomly assigned to either PFS or the control group. The survey was conducted approximately 12 months after the noncustodial parent's month of random assignment and had a response rate of 90 percent. In addition, a separate survey was administered to a random subset of noncustodial parents 12 months after program entry. A total of 553 noncustodial fathers responded to the survey, for a response rate of about 78 percent. The final MDRC evaluation report (Martinez and Miller, 2000) assessed the effects of PFS on noncustodial parents' employment and earnings. The program effects were estimated for the full sample of noncustodial parents using data from the unemployment insurance system, as well as for the noncustodial parent survey results used in the Knox and Redcross (2000) report (which included a sample of 553 noncustodial parents).

- 369 -

Key Evaluation Findings

Knox and Redcross (2000) reported the following:

• Noncustodial parents in the PFS program group were significantly more likely to provide formal child support than were members of the control group (50.2 percent versus 43.4 percent, respectively). Furthermore, the average value of support provided during the six months prior to the survey was higher for the PFS parents ($397 versus $313).

• PFS did not change the likelihood that noncustodial parents would provide any informal child support. However, PFS did lead to a small reduction in the average value of informal support given during the six-month follow-up period, with control group parents giving significantly more ($149 versus $112).

• Overall, from the perspective of the custodial parents, the net result of PFS did not produce a detectable change in their total income as a result of child support payments.

• With respect to child contact, PFS did not lead to increases in the frequency or length of contact that noncustodial parents had with their children. However, site-specific analyses indicated that PFS was effective at increasing the occurrence of regular visits when it served families with relatively low visitation rates, i.e., significant results were seen in the two sites whose level of noncustodial parental involvement had significant room for improvement.

Key findings of Martinez and Miller (2000) were reported as follows:

• For the full sample, PFS did not significantly increase noncustodial parents' employment or earnings during the two years after they entered the program. However, PFS did increase employment among noncustodial parents who might be characterized as "less employable"— those without a high school diploma and those with little recent work experience.

o For more-employable men, the program had little effect on average earnings and somewhat reduced employment among those who would have worked in part-time, lower- wage jobs.

Probable Implementers

State and local governmental child-support enforcement agencies, job placement and training agencies, private nonprofit service providers, and county welfare departments.

Funding

Demonstration project funders: U.S. Department of Health and Human Services, The Pew Charitable Trusts, W.K. Kellogg Foundation, Charles Stewart Mott Foundation, U.S. Department of Agriculture, The Annie E. Casey Foundation, U.S. Department of Labor, Smith Richardson Foundation, Ford Foundation, The McKnight Foundation, and the Northwest Area Foundation.

Implementation Detail

Program Design

Key PFS program services include the following:

• Peer support groups

• Mediation to improve relations with custodial parents

• Employment and training services

- 370 -

• Enhanced child support enforcement

• Reduced child support obligations during the period of program participation.

All program services were designed to accomplish specific results. Peer support and mediation services were designed to affect family relationships and improve fathers’ involvement with their children. Enhanced child support enforcement was designed to modify the procedures and services for low- income noncustodial parents, for example by expediting the modification of child support awards and reducing child support awards while parents participated in PFS. Lastly, employment and training services were included because they might serve to increase a father’s income, and thus increase his capacity to pay child support, improve his sense of parenting abilities, increase his interest in visiting with his children, or increase the willingness of the mother to allow the children to visit with him. All of the program services were provided at each of the seven sites, but with some degree of variation, as the PFS program is designed to be adjustable to meet the needs of the particular target population. In the demonstration sites, participation in the PFS services was greatest in the peer-support component (with about a 64 percent participation rate), followed by the job-search workshops (with a participation rate of approximately 57 percent). Peer support was judged by program staff to be the most effective component of the program. In contrast, mediation was rarely used, and only one site achieved a participation rate for mediation that exceeded 5 percent. Curriculum

The PFS peer-support component was built around The Responsible Fatherhood Curriculum. All activities in the curriculum were designed to help encourage positive parental behavior and to inform participants about their rights and obligations as noncustodial parents. Each session was conducted in three parts and included activities that presented some new ideas to group members while helping them to think about these ideas through a discussion structured around a set of questions. The three parts of the session included:

1. A beginning exercise called "What's New?"

2. One or several activities that required the active involvement of the entire group.

3. A closing activity called "Feedback/Wrap Up."

Program activities included role-playing, problem solving, and sharing information within small groups, giving and receiving feedback in pairs, brainstorming, and other similar activities. Staffing

PFS program services were administered by the child support services agency at each site, a partnership of local community organizations, and employment and training providers. Across the seven PFS sites, two programs (in Florida and Massachusetts) were state administered through regional offices, and the remaining five sites were county administered. In four of the seven sites, local child support enforcement staff identified a special worker to monitor child support payments on all cases and to take appropriate enforcement actions.

Issues to Consider

This program received a "promising" rating. Noncustodial parents in the PFS program, who were almost exclusively fathers, were significantly more likely to pay child support than were noncustodial parents in the control group. Evaluation results suggest that the demonstrated increases in child support were more likely to be of a formal rather than informal nature. MDRC researchers suggest that one possible consequence of the design of the PFS program is that since many fathers provide support for their children through informal or in-kind payments, it is possible that fathers who are forced to pay through the formal system will reduce their informal contributions, resulting in no increase in total benefits to custodial parents. Despite the increase in the percentage of noncustodial parents who paid child support, overall, parents

- 371 -

who were referred to the PFS program did not pay higher dollar amounts than did parents in the standard child-support group. Additionally, the PFS program had no significant effects on employment rates and earnings for the full sample of noncustodial fathers, although it did increase earnings for the least employable men. An important caveat to the results for the full sample is that they were obtained using data from jobs covered by the unemployment insurance system, which may miss any effects PFS might have had on informal employment or unreported earnings. Overall results of the evaluations indicate that PFS was not successful in increasing the frequency or length of contact that fathers had with their children. Findings suggest that the program may be more successful at increasing the occurrence of regular parental visits for families with relatively low visitation rates at baseline. It should be noted that MDRC was responsible for both the design and the evaluation of the Parents' Fair Share program.

Example Sites

Duval County (Jacksonville), Florida; Hampden County (Springfield), Massachusetts; Kent County (Grand Rapids), Michigan; Los Angeles County, California; Mercer County (Trenton), New Jersey; Montgomery County (Dayton), Ohio; and Shelby County (Memphis), Tennessee. Missouri and Minnesota were among those states that had sites in the pilot study for this program, but the states did not continue with the full demonstration.

Contact Information

Cynthia Miller Manpower Demonstration Research Corporation 16 East 34th Street New York, NY 10016 phone: (212) 532-3200 fax: (212) 684-0832 email: cynthia.miller@mdrc.org

Available Resources

MDRC provides curriculum materials for The Responsible Fatherhood Curriculum used in the peer support groups, available online at http://www.mdrc.org/publications/40/abstract.html. More details on the demonstration study and early stages of implementation of the project are available in the following publication:

Doolittle, Fred, and Suzanne Lynn, Working with Low-Income Cases: Lessons for the Child Support Enforcement System from Parents' Fair Share, Oakland, Calif.: Manpower Demonstration Research Corporation, May 1998. Available online at http://www.mdrc.org/publications/144/overview.html (as of August 30, 2006). Final lessons and an overall summary of the PFS program model and evaluations are provided in: Miller, Cynthia, and Virginia Knox, The Challenge of Helping Low-Income Fathers Support Their Children: Final Lessons from Parents' Fair Share, Oakland, Calif.: Manpower Demonstration Research Corporation, November 2001. Available online at http://www.mdrc.org/publications/104/overview.html (as of August 30, 2006).

- 372 -

mailto:cynthia.miller@mdrc.org

http://www.mdrc.org/publications/40/abstract.html

http://www.mdrc.org/publications/144/overview.html

http://www.mdrc.org/publications/104/overview.html

Bibliography

Doolittle, Fred, Virginia Knox, Cynthia Miller, and Sharon Rowser, Building Opportunities, Enforcing Obligations: Implementation and Interim Impacts of Parents' Fair Share, Oakland, Calif.: Manpower Demonstration Research Corporation, December 1998. Knox, Virginia, and Cindy Redcross, Parenting and Providing: The Impact of Parents' Fair Share on Paternal Involvement, Oakland, Calif.: Manpower Demonstration Research Corporation, October 2000. Accessed August 30, 2006 at http://www.mdrc.org/publications/38/abstract.html. Martinez, John M., and Cynthia Miller, Working and Earning: The Impact of Parents' Fair Share on Low-Income Fathers' Employment, Oakland, Calif.: Manpower Demonstration Research Corporation, October 2000. Accessed August 30, 2006 at http://www.mdrc.org/publications/39/abstract.html.

Last Reviewed

January 2007

Partners in Reading

Program Info

Outcome Areas Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Elementary School Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance

Evidence Level Promising

Program Overview

The Partners in Reading program (PIR) is a school-based tutoring program for beginner readers. PIR is designed to increase the number of books that students read independently, to enhance the difficulty level of the books that students read, and to improve students' ability to recognize and pronounce words correctly. Tutors help students read and reread instructional texts that are appropriate to a student's reading level, provide students with immediate and frequent feedback, and employ activities to encourage students' use of sophisticated word recognition strategies and to motivate students to learn. PIR was implemented in a single school that requested help in meeting the needs of its

- 373 -

http://www.mdrc.org/publications/38/abstract.html

http://www.mdrc.org/publications/39/abstract.html

struggling readers, because the number of these students exceeded the resources of the school's Reading Recovery (RR) program (the regular tutoring program).

Program Participants

PIR targets struggling first-grade readers in schools with limited financial and personnel resources.

Evaluation Methods

Miller (2003) evaluated the PIR program by studying first graders from 15 classrooms in a Title I elementary school in the coastal Carolinas. Approximately 65 percent of the school’s students were racial minorities, primarily African-American, and 75 percent received free or reduced-price lunches. The sample was assembled in two groups over two consecutive years, and consisted of 54 PIR students, 62 RR students, and 58 control students. During the first year (Group 1 of two groups), 12 first-grade classrooms containing 263 students took part in the study. Nineteen first-grade students out of this first cohort were selected for tutoring. During the second year (Group 2), an additional three first-grade classrooms were added for a total participation of 15 classrooms consisting of 334 students. Thirty-five students out of the second cohort were selected for tutoring. Students in the treatment groups were selected based on their scores on tests of word recognition and spelling from the Howard Street Training Manual (Morris, 1992). Scores on these two assessment measures were compared with teachers’ rankings of student reading ability. Students whose scores were at the bottom third of the measures as well as being in the bottom third of the teachers’ rankings were selected for tutoring. The selection of students for PIR occurred after the RR students were selected (because RR teachers were able to provide tutoring before the PIR tutors were ready to start). The control group included students who did not receive any special assistance from PIR, RR, or special education, and whose pretest scores on the word recognition and spelling measures were within one standard deviation of the PIR and RR students’ average scores. (Subsequent analysis indicated that the three groups were statistically equivalent in word recognition and spelling at the start of the school year.) From the initial sample, 18 of the 54 PIR students (33 percent), 15 of the 62 RR students (24 percent), and 29 of the 58 control students (50 percent) left the school prior to testing at the end of second grade. No attrition analysis was conducted to determine whether the remaining students in the three groups were equivalent at the outset of the study. Due to the high number of students who left the study sample, an analysis of data on Metropolitan Achievement Test scores for the combined group was done at the end of second grade.

Key Evaluation Findings

At the end of first grade, students were compared on their performance on word recognition and developmental spelling tests from the Howard Street Training Manual. At the end of second grade, the students’ performance was compared using the Metropolitan Achievement Test for reading and language arts, which includes subtests on language, word recognition, vocabulary, and comprehension. Word Recognition and Spelling at the End of First Grade Group 1:

• At year’s end, students in PIR (19 students) and RR (30 students) were statistically equivalent and significantly superior to the control group (29 students) on word recognition scores.

• Similarly, PIR and RR students were statistically equivalent and significantly superior to control group students on scores of developmental spelling.

- 374 -

Group 2: • At year’s end, students in PIR (35 students) and RR (32 students) were equivalent and

significantly superior to control group students (29 students) on the measure of word recognition.

• PIR and RR students scored similarly on the developmental spelling test, with both groups significantly outscoring the control group.

Metropolitan Achievement Test at the end of Second Grade

• Significant differences were found among the three groups on the word recognition subtest, with PIR and RR students scoring higher than control students.

• Differences were also found among the groups on the comprehension subtest, with PIR students significantly outperforming control students, but no differences were found between PIR and RR students.

• No significant differences were found among the groups on the vocabulary or language subtests.

Probable Implementers

Public and private elementary schools

Funding

The school discussed in the Miller (2003) study utilized professional development funds to implemented PIR training.

Implementation Detail

Program Design

Each tutor was assigned to multiple students and completed a progress chart for each student every six weeks. The charts included information on the number of tutoring sessions, the number of books a student had read, the reading difficulty level of each book, word-sort activities, and notes about a student’s successes or difficulties. Every six weeks, classroom teachers received a note from the tutor in the form of a greeting card, which stated the number of books each student had read and the associated reading difficulty levels. Students received at least four 40-minute tutoring sessions each week. Each session consisted of two phases: First, a student read a book that was familiar from either a previous session or from a nightly homework selection. Second, tutors allowed the student to select one book from among several new books that were at a reading level slightly above the student’s instructional level. After selecting a book, the student would choose to read it independently, read it along with the tutor, or have the tutor read it to him or her. Students were encouraged to read the books independently.

Staffing

The classroom tutors attended two half-day workshops prior to the start of the school year, where they learned how to organize a typical tutoring lesson.

Issues to Consider

This program received a "promising" rating. For both Group 1 and Group 2, PIR students significantly outscored control group students on measures of word recognition and spelling at the end of first grade. Results from the Metropolitan Achievement Test at the end of second grade indicated that PIR students had outscored control students on word recognition and reading comprehension.

- 375 -

An area of concern regarding the results of the Miller study is the high rate of student attrition prior to the end of second grade. Given that 33 percent of PIR students, 24 percent of RR students, and 50 percent of control students left the study prior to testing on the Metropolitan Achievement Test, it is possible that the remaining students were no longer directly comparable. In other words, because an analysis was not conducted to determine the pre-study comparability of these three groups, it may be that they differed significantly from each other in baseline characteristics due to the differential in their dropout rates. It should be noted that the sample sizes in the Miller study were quite small, and that the program has been evaluated in only a single site. Therefore, the Miller study results may not be applicable to schools having a student body with markedly different socioeconomic characteristics. Results from this study suggest that the Partners in Reading program may be an effective way for financially challenged schools to help struggling first-grade readers improve their language skills. Although no significant differences in achievement scores were found between students in the PIR program and those in the school’s regular Reading Recovery tutoring program, the PIR program is much less expensive to implement.

Example Sites

The coastal Carolinas

Contact Information

Dr. Samuel D. Miller P.O. Box 26170 School of Education University of North Carolina Greensboro, NC 27402-6170 Ph: (336) 334-3445 Fax: (336) 334-4120 Email: sam_miller@uncg.edu

Available Resources

No resources were available as of fall 2004.

Bibliography

Miller, Samuel D., "Partners-in-Reading: Using Classroom Assistants to Provide Tutorial Assistance to Struggling First-Grade Readers," Journal of Education for Students Placed at Risk, Vol. 8, No. 3, 2003, pp. 333-349. Morris, D., Case Studies in Teaching Beginning Readers: The Howard Street Tutoring Manual, Boone, N.C.: Fieldstrip, 1992.

Last Reviewed

September 2004

- 376 -

mailto:sam_miller@uncg.edu

Peer-Assisted Learning Strategies (PALS)

Program Info

Outcome Areas Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards

Evidence Level Promising

Program Overview

Peer-Assisted Learning Strategies (PALS) is a 25-to-35-minute math or reading activity implemented two to four times per week. It is designed to complement, not replace, existing reading and math curricula. PALS supplements traditional peer tutoring with instructional principles and practices. Teachers identify and pair children who require help with specific skills ("players") with children who are the most appropriate to help other children learn those skills ("coaches"). The pairs of students are changed regularly, so, over a period of time, the students work on a variety of skills and all students have the opportunity to be "coaches" and "players." Approximately 13 to 15 pairs of students are created in the classroom, and each pair is geared to the individual student's needs (as opposed to a single, teacher-directed activity meant for all students, which may not address the specific problems that children face). The PALS peer-tutoring strategy enables teachers to circulate around the classroom and observe students, providing feedback and remedial lessons where necessary. Note: This program is similar to the Class Wide Peer Tutoring program.

Program Participants

Originally, PALS was designed for use in 2nd- through 6th-grade classrooms. More recently, both upward and downward extensions of PALS have been developed, resulting in PALS Kindergarten Math and Reading, PALS First-Grade Math and Reading, and PALS High-School Reading.

- 377 -

Evaluation Methods

Four studies were identified that met Promising Practices Network criteria for inclusion. The majority of the studies indicated that PALS students significantly outperformed control group students on tests of reading and mathematics skills.

The effects of a 15-week PALS program were assessed in a sample of 120 students from 40 classrooms in grades 2 through 6 in 12 schools in a southern state (Fuchs et al., 1997a). Twenty-two elementary and middle schools were first stratified into three groups (high, middle, and low) based on student achievement and family income (with a "high" level signifying populations with relatively high average reading scores and a comparatively low proportion of students qualifying for free or reduced lunch). Schools were then randomly assigned to PALS (20 classrooms) or No-PALS (20 classrooms) groups. The 12 schools were equally divided between PALS and No-PALS assignments and were equally divided across high-, mid-, and low-level socioeconomic designations. To determine the sample of students, each of the 40 participating teachers identified the following three students in his or her reading class: (1) a learning-disabled student, (2) a non-learning-disabled but low-performing student (in the lowest quartile in reading in the class), and (3) a student estimated to be an average- achieving reader. Pre-test analyses revealed no significant demographic differences among the groups, no significant differences in Comprehensive Reading Assessment Battery (CRAB) scores among the groups, and no significant interaction effects between treatment group and student type. The CRAB was used to assess student outcomes. Fuchs et al. (1997b) studied an 18-week PALS program that included 120 students from 40 classrooms in grades 2 through 4. Teachers were randomly assigned to one of three groups: (1) Peer- Mediated Instruction (PMI) with training in how to offer and receive elaborated help (helpful, conceptual explanations) (ten classrooms); (2) PMI with training in both elaborated help and methods for providing conceptual mathematical explanations (ten classrooms); or (3) a control group (ten classrooms). Each teacher identified the following four students: (1) a student who was chronically low-achieving and had been classified as learning disabled, (2) a student who was chronically low- performing but had never been identified as disabled, (3) a student with average math performance, and (4) a student whose math performance was near the top of the class. Analysis of group comparability found no significant differences among the three groups and no significant interactions between treatment and type of student. Student outcomes were assessed on the Operations and Concepts/Applications subscales of the Comprehensive Math Test. The effects of PALS were assessed by Fuchs et al. (1999) in a sample of 72 students from 24 classrooms in grades 2 through 4. While stratifying by grade level, classrooms were randomly assigned to PALS (16 teachers) or a control group (eight teachers). Half of the PALS teachers were then randomly assigned to one of two treatment groups: PALS plus collaborative reading activities or PALS only. Three students from each class—including one at-risk student, one average-achieving student, and one high-achieving student—were selected by their teachers to form the analysis sample. No significant differences were found among the control and two treatment groups on demographic variables or on the pre-test for the Stanford Diagnostic Reading Test (SDRT). After completion of the 21-week program, student outcomes were assessed with the Comprehension subtest of the SDRT. Fuchs et al. (2001) studied 168 kindergarten students in five schools in a metropolitan public school district in the Southeast. Twenty classrooms within the schools were randomly assigned to PALS (84 students) or control (84 students) groups. Treatment effects were estimated on a subset of students, who were identified by scores on a pre-test of mathematics achievement, and outcomes analysis was done separately for special education students and for non-disabled low-, middle-, and high-achieving students. No statistically significant differences among the groups were found for any demographic variables. Student outcomes were assessed using the mathematics portion of the Stanford Early School Achievement Test and the math portion of the Primary I level of the Stanford Achievement Test.

- 378 -

Key Evaluation Findings

Fuchs et al.'s (1997a) study of 120 students from 40 classrooms reported the following:

• Significant growth effects were found on all three CRAB scores (words read correctly, questions answered correctly, and number of items correctly replaced in a maze-based activity). For each CRAB, growth in test scores among students in PALS classrooms, averaged across student type, was greater than among students in No-PALS classes.

• No statistically significant effects were found for any combination of treatment group, student type, or school type, suggesting that the effectiveness of the PALS treatment was not affected by student type.

Fuchs et al.'s (1997b) study of 120 students from 40 classrooms reported the following:

• Growth in math scores for the Peer-Mediated Instruction plus Elaborated help plus Conceptual help (PMI+E+C) group significantly exceeded that of both the PMI+E group and the control group, while the growth in math scores of the PMI+E group significantly exceeded that of the control group.

• Analysis indicated that the PMI+E+C treatment was more effective than the no-treatment control for low-performing and high-achieving students but not for learning-disabled or average-achieving students.

• Similarly, the PMI+E+C treatment was more effective than the PMI+E treatment for low- achieving and high-achieving students but not for learning-disabled or average-achieving students.

In their study of 72 students in 24 classrooms in the PALS, PALS plus elaborated help giving (PALS+HG), and control groups, Fuchs et al. (1999) reported the following:

• Primary-level PALS students significantly outperformed PALS+HG and control students on the number of questions answered correctly on the CRAB, while PALS+HG and control students performed comparably. At the intermediate level, PALS+HG students outperformed PALS and control students, while PALS and control students performed comparably.

• No significant effects for treatment group were found on the number of words read correctly on the CRAB.

Fuchs et al.'s (2001) study of 168 kindergarten students found the following:

• No significant differences between treatment and control groups on Stanford Early School Achievement Test (SESAT) scores, and no significant interaction of treatment group by student type.

• The growth on SESAT scores of PALS students from pre-test to post-test significantly exceeded that of control students.

• No significant differences were found between treatment and control groups on any of the Stanford Achievement Test measures.

Probable Implementers

Public and private elementary, middle, and high schools

- 379 -

Funding

Many schools use Title I funds to cover PALS training and program materials. More recently, schools have begun using federal Reading First monies to cover the implementation costs of the program. Costs include an initial on-site workshop ($1,500 plus presenter travel expenses, in 2013 dollars) and teacher manuals and student material packets ($15-$40 per item, in 2013 dollars). Ordering information is available on the PALS website, under Ordering Materials: http://kc.vanderbilt.edu/pals/order.html.

Implementation Detail

Program Design

Materials needed for PALS implementation are minimal and inexpensive. They include the PALS teacher's manual, a box of blank transparencies, an overhead projector, a transparency pen, pocket folders for each pair of students (to hold the PALS materials), and access to a photocopier to make copies of lessons for students. In addition, a kitchen timer is recommended to help teachers with appropriate pacing during the lesson. The actual presentation of the lessons is left up to each teacher. The PALS teacher's manual includes both outlines of the material and detailed scripts. Teachers may use the scripts to present the material, or use only the outlines to guide the presentations in their own words. The amount of class time required for program implementation varies with grade level:

• K-PALS: Three to four times per week for approximately 30 minutes per session.

• First-Grade PALS: Three to four times per week for approximately 35 minutes per session.

• Grades 2 through 6 PALS Reading: Three times per week for approximately 35 minutes per session.

• Grades 2 through 6 PALS Math: Two times per week for approximately 40 minutes per session.

• High-School PALS: Five times per week every two weeks for approximately 35 minutes per session.

Curriculum

PALS Reading is a structured activity that can be implemented at any grade level, from preschool through high school. The program does not require special reading material, and teachers may use library books or short segments of text. In Kindergarten PALS, children concentrate on letter-sound correspondence, decoding (making sense of text), and phonological awareness, while students in First-Grade PALS practice decoding and reading fluently. The Grades 2 through 6 program makes use of three PALS activities that help promote reading fluency and comprehension: (1) reading with a partner, (2) paragraph "shrinking" (stopping at the end of each paragraph to identify the main idea), and (3) prediction relay (requires students to formulate and confirm or disconfirm hypotheses). There is a Preschool PALS in development, which will focus on letter names, letter sounds, first-sound identification (proper identification of the first sound when reading a word), and phonological awareness. PALS Math uses two basic learning procedures: coaching and practice. During coaching, students complete a worksheet of problems in a specific skill area (e.g., adding, subtracting with regrouping, number concepts, charts and graphs). For 15 to 20 minutes, the "coach" uses a worksheet with a series of questions, which differ by type of problem, that are designed to guide the "player" toward skill comprehension. During the practice sessions, each student receives a worksheet listing the type of problem covered in the coaching session and less-challenging types of problems. Students work independently for five to ten minutes, then exchange their papers with a partner and score each

- 380 -

http://kc.vanderbilt.edu/pals/order.html

other's practice sheets. Cooperative learning is encouraged because students earn points for forming good explanations during coaching and for answering problems correctly during practice. Staffing

Regular classroom teachers implement PALS in the classroom, without the need for teaching aides or administrative assistants.

Issues to Consider

This program received a "promising" rating. The evaluations of PALS are mostly randomized control trials, which used convincing comparison groups along with statistical analyses that attempted to account for any pre-existing differences between the experimental and control groups. The statistical analyses of student-level effects do not, however, account for clustering of students at the classroom or teacher levels. The evaluations demonstrate that treatment-group students, in various versions of PALS, scored significantly higher than control-group students on tests of reading and mathematics skills. Several of the evaluations described above involved teachers selecting students (with differing performance levels) from their classes for inclusion in the research sample. Although the strength of the randomized control designs compensate for the student-selection strategy, the possibility of sample bias is introduced through having teachers select students for participation. Another issue to note is that teachers who implemented PALS in their classrooms often volunteered to do so, and volunteered to participate in the study. This raises questions about how typical these teachers may be, and therefore how generalizable the study results may be to other teachers not as eager to participate in a new program or a research study. Finally, at least one of the program developers is an author on all of the studies included in this program description.

Example Sites

Nashville, Tennessee

Contact Information

Loulee Yen PALS Outreach Vanderbilt University Peabody Box 328 110 Magnolia Circle, Suite 418 Nashville, TN 32703-5701 Tel (615) 343-4782 Fax (615) 936-5803 PALS@vanderbilt.edu

Available Resources

The PALS website provides detailed information on the program and its implementation, training resources, and a summary of research: http://kc.vanderbilt.edu/pals/. Training and implementation assistance for PALS reading and math programs are available to educators nationwide. PALS staff recommend a one-day training workshop for interested schools (see http://kc.vanderbilt.edu/pals/workshops.html). Depending on the particular workshop chosen, the presenter's fee in 2013 is about $1,500 plus travel expenses. All workshop participants require a PALS

- 381 -

mailto:PALS@vanderbilt.edu

http://kc.vanderbilt.edu/pals/

http://kc.vanderbilt.edu/pals/workshops.html

teacher's manual, which can be purchased through Vanderbilt University prior to the training workshop (see http://kc.vanderbilt.edu/pals/pdfs/PALS-Order-Form-NEW.pdf).

Bibliography

Fuchs, Douglas, Lynn S. Fuchs, Patricia G. Mathes, and Deborah C. Simmons, "Peer-Assisted Learning Strategies: Making Classrooms More Responsive to Diversity," American Educational Research Journal, Vol. 34, No. 1, 1997a, pp. 174-206. Fuchs, Lynn S., Douglas Fuchs, and Kathy Karns, "Enhancing Kindergarteners' Mathematical Development: Effects of Peer-Assisted Learning Strategies," Elementary School Journal, Vol. 101, No. 5, 2001, pp. 495-510. Fuchs, Lynn S., Douglas Fuchs, Carol L. Hamlett, Norris B. Phillips, Kathy Karns, and Suzanne Dutka, "Enhancing Students' Helping Behavior During Peer-Mediated Instruction with Conceptual Mathematical Explanations," Elementary School Journal, Vol. 97, No. 3, 1997b, pp. 223-249. Fuchs, Lynn S., Douglas Fuchs, Sarah Kazdan, and Shelley Allen, "Effects of Peer-Assisted Learning Strategies in Reading with and Without Training in Elaborated Help Giving," Elementary School Journal, Vol. 99, No. 3, 1999, pp. 201-220.

Last Reviewed

March 2013

Postponing Sexual Involvement/Human Sexuality Educational Series

Program Info

Outcome Areas Healthy and Safe Children

Indicators Youths abstaining from sexual activity or not engaging in risky sexual behavior

Topic Areas Age of Child Adolescence (13-18) Type of Setting Middle School Health Care Provider Type of Service Health Education Youth Development Type of Outcome Addressed Physical Health Teen Sex/Pregnancy

Evidence Level Promising

- 382 -

http://kc.vanderbilt.edu/pals/pdfs/PALS-Order-Form-NEW.pdf

Program Overview

The Postponing Sexual Involvement (PSI) program is a school-based program designed to delay sexual activity among adolescents. The curriculum was widely introduced into the Atlanta public schools in 1985 and was taught to eighth graders by health educators, nurses, and adolescent counselors using teens as primary presenters. The origins of PSI began in the mid-1970s, with the initiation of the knowledge-based sex education course "Human Sexuality," which consisted of five classroom sessions covering basic human sexuality, decisionmaking, and contraceptive methods. Evaluations of the Human Sexuality program, however, showed that simply providing information was not effective in changing behavior. The PSI program was developed to complement the Human Sexuality program and is based on the "social influence" theory, which holds that young people are more likely to become sexually involved because of social and peer pressures than because of a lack of knowledge about sexuality and sexual activity. PSI uses activities that help identify the origins of pressure to engage in sexual activity, examines the motivations behind that pressure, and helps students develop skills to respond to that pressure effectively. The program is also built on research that shows that teenage leaders produce greater and more lasting effects on other teens' behavior than do adults.

Program Participants

Junior high students in seventh and eighth grades.

Evaluation Methods

In an evaluation of the pilot PSI program in Atlanta, Georgia, Howard and McCabe (1990) studied a population of 536 low-income minority students, including 395 from the 24 schools that received PSI and 141 students from 29 control schools in the area (that received those schools' regular sexual education program). Ninety-nine percent of all students in both groups were African American. Students were included in the study sample if they were born at Atlanta’s large public hospital (Grady Memorial Hospital) in 1972, if they were entering eighth grade in 1985, and if they or their mothers had received care from the public hospital within the past five years (an indicator of poverty). The majority of the baseline measures did not differ significantly between treatment and control groups, however a greater percentage of PSI program students were from families in the lowest income category (56 percent versus 45 percent), and a greater percentage of control students lived with two parents (57 percent versus 36 percent). These demographic differences suggest that the PSI program group was from a higher-risk population than the control group. For analyses, students were divided into two groups: those who had not had sexual intercourse before the program, and those who had. Analyses were limited to those who had not had sexual intercourse at baseline. Of the 536 students who completed all interviews, 131 (24 percent) reported that they had had sexual intercourse before baseline, while 387 students (72 percent) said they had not (18 students had indeterminate answers and were dropped from the analysis). Of the 387 baseline virgins, 278 attended PSI schools with 256 actually attending the program, and 109 attended control schools. Five telephone health surveys were administered to the students over a five-year period, including three during the eighth grade and two during the ninth grade. The questionnaire was part of a larger study on health habits, so students were unaware they were participating in an evaluation of the PSI program. At the end of ninth grade, retention rates were 85 percent for the PSI students and 81 percent for the control students. Howard (1992) conducted a twelfth grade follow-up to Howard and McCabe (1990). By the end of twelfth grade, 56 percent of the treatment group and 43 percent of the comparison group were retained in the study (237 treatment and 66 comparison group children, respectively). Analyses included all students, regardless of sexual activity at baseline. Based on PSI, a similar program was implemented in the UK, consisting of 25 to 30 one-hour lessons given to students in curriculum years nine (ages 13-14) and ten (ages 14-15). The evaluation of the British PSI program (Mellanby et al., 1995) matched schools based on size. Depending on school schedules, the schools were then assigned to either the treatment (1,175 students) or control group

- 383 -

(5,398 students), which received the school's existing sex education program. The evaluation surveyed three consecutive years of eleventh year students (age 15-16). The first group of students did not receive any portion of the program, the second group received half of the program while they were in year ten, and the third group received both years of the program. The outcome measures used in the study included changes in attitudes, knowledge of risk factors, sexual activity, and age at first intercourse. Aarons et al. (2000) conducted a randomized controlled evaluation of the effect of the PSI program in a sample of six Washington, D.C., junior high schools. The schools were paired according to class size, location, and racial/ethnic distribution, and then randomly assigned to the treatment or control group. The treatment in this study was an expanded version of PSI, consisting of reproductive health classes, the PSI curriculum, health risk screening, and "booster" educational activities during the following school year. Of 812 eligible students, 582 enrolled in the six schools at the beginning of the study obtained written parental consent to participate. Students were surveyed at baseline (522 students), with the first follow up at the end of the seventh grade (503 students). The following year 459 students had useable surveys at the beginning of eighth grade, and 422 students had useable surveys at the end of eighth grade. The surveys were cross-sectional and anonymous, i.e., results indicate the average of the group of students at each individual point in time. A comparison of baseline characteristics between treatment and control groups revealed some significant differences between groups, however the way the differences might have positively or negatively affected the results is not obvious. At baseline, 44 percent of the seventh grade males and 81 percent of the seventh grade females reported being virgins.

Key Evaluation Findings

Howard and McCabe's (1990) evaluation of the pilot PSI program found that, among students who had not had sexual intercourse prior to the program:

• By the end of eighth grade, students in the control group were five times more likely to have begun having sex than were PSI students (20 percent versus 4 percent). By the beginning of ninth grade, the difference between these rates was still significant at 27 percent and 12 percent, respectively. By the end of ninth grade, the difference between groups was still significant, with rates of 39 percent versus 24 percent.

• For females in the study, by the end of eighth grade students in the control group were significantly more likely than students in the PSI group to have engaged in sexual intercourse (15 percent versus 1 percent). The differences between groups remained statistically significant in the ninth grade, with rates of 18 percent versus 7 percent at the beginning of the school year, and 27 percent versus 17 percent by the end of the school year.

• For male students, those in the control group were significantly more likely to have engaged in sexual intercourse by the end of eighth grade (29 percent versus 8 percent), with rates of 42 percent versus 22 percent at the beginning of ninth grade, and 61 percent versus 39 percent at the end of ninth grade.

Howard's (1992) twelfth grade follow-up study reported the following results at the end of twelfth grade:

• No statistically significant differences remained between groups in rates of initiation of sexual intercourse.

• There was no significant difference between groups with respect to use of birth control; 90 percent of the PSI group and 91 percent of the control group said they used a method of birth control at the time of last intercourse.

Mellanby et al. (1995) evaluated a British version of the PSI program and found that:

• In each year, PSI students increased their knowledge related to contraception, sexually transmitted diseases, and the true prevalence of sexual activity among their peers. Within

- 384 -

control populations, there was no annual increase in correct answers when asked questions to test their knowledge of these issues.

• There was a significantly lower rate in sexual activity among PSI students compared with control students. In year three, control students were 1.45 times more likely to have had sex than program students.

Finally, the randomized controlled study by Aarons et al. (2000) reported that:

• For virginity status among female participants,

o at the end of seventh grade, treatment group females were significantly more likely than control group females to report they were virgins.

o at the beginning of eighth grade no significant differences were found between groups.

o at the end of eighth grade, significantly more treatment group females than control group females reported that they were virgins.

• With regard to the use of birth control/condoms at the time of last intercourse for female, nonvirgin students, treatment group females were significantly more likely to report usage than were control group females at all three measurement points.

• No significant differences in virginity or use of birth control/condoms were observed among male participants at any time during the study.

Probable Implementers

Public and private junior high schools, community-based health organizations, and faith-based groups.

Funding

The Ford Foundation funded the pilot Atlanta implementation. Nationally, programs are supported with federal, state, local, foundation, and agency funds.

Implementation Detail

Program Design

The PSI program in combination with the Human Sexuality program covers four main areas: an emphasis on abstinence from or delay of sexual activity, life-skills training, sexuality education, and contraceptive education. Abstinence is presented as the best way to prevent unintended pregnancy and sexually transmitted diseases. The program also teaches that young teens are not yet mature enough to handle the consequences of sexual activity. The life-skills component includes activities that help students build decisionmaking skills, set goals for their lives, learn how to say no to sex, and negotiate within relationships. Sexuality education refers to a broad-based curriculum covering physical growth and the development of healthy sexual attitudes and values. Contraceptive education covers methods of contraception, how such methods are used, and their effectiveness in preventing pregnancy and sexually transmitted diseases.

Curriculum

The PSI program uses three principles—experiential learning (active involvement on the part of the students); a single, consistent message (postponing sexual activity); and repetition and reinforcement. When implemented in Atlanta in 1985, students received both the Human Sexuality and the PSI programs. The five Human Sexuality sessions were taught by hospital nurses and health educators. PSI was delivered in five classroom sessions, each 45 to 60 minutes long, was directed by two student leaders, and involved group discussion and role-playing activities. The first four sessions were given

- 385 -

either in the same week or weekly over four weeks. The fifth session, which was designed to reinforce the material, was given one to three months later. More specifically, session 1 addressed students' beliefs about why teenagers have sex and presented reasons for why they should postpone sexual activity. Session 2 focused on the influence of the media, while session 3 focused on peer pressures and how to respond to them, as well as the need to set a "stopping point" in physically expressing affection. Session 4 helped students develop assertiveness skills in resisting pressures to have sex. Lastly, session 5 reinforced the concepts from the previous sessions.

Staffing

The student leaders were recruited from among eleventh and twelfth graders, and each received 20 hours of initial training, followed by monthly two-hour training sessions. Student leaders learned to present information, conduct discussions, teach assertiveness skills, and lead role-playing.

Issues to Consider

This program received a "promising" rating. While the PSI program did have several positive effects related to postponing sexual activity for both male and female students, results from Howard (1992) suggest that the delay was not maintained by the end of twelfth grade. Study results were mixed with respect to use of birth control, with Howard (1992) reporting no program effects, and the study by Aarons et al. (2000) reporting positive findings for female students only. An area of potential concern is program replication. The program in Great Britain was based on the PSI program and seems to have been successful in obtaining similar results regarding youths delaying their sexual activity. However, a large-scale replication in California did not produce any of the positive effects seen in the Atlanta study. The California program replication, called the Education Now and Babies Later initiative, took place from 1992 to 1994. An evaluation of 10,600 students (Kirby et al., 1997) found that youths in treatment and control groups were equally likely to have become sexually active or to have reported a pregnancy or sexually transmitted disease 17 months after the program. The California program, however, may not have been a faithful replication for several reasons. The program was randomly delivered to classrooms within the same school, with some students receiving the program and some receiving the standard sexuality curriculum offered by the school. It is possible that, given the peer pressure associated with sexual activity, providing only some of the students with the program makes it less effective. Second, the PSI program component did not follow the Human Sexuality program offered in the Atlanta schools, although the students were required to participate in a similar program. The study did not find any significant effects with the students who were taught by peer leaders; in addition it was found that some of the teen leaders were not sufficiently trained or lacked experience. Furthermore, 90 percent of the classes were taught by adults. Personal observations of the sessions revealed that some of the adults did not like the program's emphasis on postponing sexual involvement and the exclusion of information about contraception and disease prevention. This identifies a problem with the replication given that the Human Sexuality program in Atlanta provided that information.

Example Sites

The program was originally piloted in Atlanta. It has been used in several locations throughout the United States, Great Britain, Canada, and New Zealand.

Contact Information

Marian Apomah Resources Coordinator: Jane Fonda Center Emory University School of Medicine

- 386 -

Building A Briarcliff Campus 1256 Briarcliff Road Atlanta GA 30306 phone: (404) 712-4710 fax: (404) 712-8739

Available Resources

The Postponing Sexual Involvement Educational Series was updated with a new 15 segment video and leader's guide in 2006. The updated materials are available for purchase from the Jane Fonda Center at Emory University (www.janefondacenter.emory.edu). A series of training materials is available for use in orienting teen leaders, consisting of a leader's guide, a live-action video, and five teen-leader survival guides. Other training also is available through Emory University's Jane Fonda Center.

Bibliography

Aarons, Sigrid J., Renee R. Jenkins, Tina R. Raine, M. Nabil El-Khorazaty, Kathy M. Woodward, Rick L. Williams, Marc C. Clark, and Barbara K. Wingrove, "Postponing Sexual Intercourse Among Urban Junior High School Students: A Randomized Controlled Evaluation," Journal of Adolescent Health, Vol. 27, No. 6, 2000, pp. 236-247. Frost, Jennifer J., and Jacqueline Darroch Forrest, "Understanding the Impact of Effective Teenage Pregnancy Prevention Programs," Family Planning Perspectives, Vol. 27, No. 5, 1995, pp. 188-195. Howard, Marion M., "Delaying the Start of Intercourse Among Adolescents," Adolescent Medicine, Vol. 3, No. 2, 1992, pp. 181-193. Howard, Marion M., and Judith B. McCabe, "Helping Teenagers Postpone Sexual Involvement," Family Planning Perspectives, Vol. 22, No. 1, 1990, pp. 21-26. Kirby, Douglas, Meg Korpi, Richard P. Barth, and Helen H. Cagampang, "The Impact of the Postponing Sexual Involvement Curriculum Among Youths in California," Family Planning Perspectives, Vol. 29, No. 3, 1997, pp. 100-108. Mellanby, Alex R., Fran A. Phelps, Nicola J. Crichton, and John H. Tripp, "School Sex Education: An Experimental Programme with Educational and Medical Benefit," British Medical Journal, Vol. 311, 1995, pp. 414-417.

Last Reviewed

October 2006

Primary and Secondary Control Enhancement Training

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children not experiencing anxiety or mood disorders, such as depression

- 387 -

http://www.janefondacenter.emory.edu/

http://www.janefondacenter.emory.edu

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Elementary School Middle School Type of Service Health Care Services Youth Development Type of Outcome Addressed Mental Health

Evidence Level Promising

Program Overview

The Primary and Secondary Control Enhancement Training (PASCET) program is an eight-session child depression treatment program. The theoretical background for the program is based on managing depression through a two-process model of control. Primary control involves rewarding oneself by identifying enjoyable activities and making objective, modifiable goals (such as school achievement or relationships with peers) that conform to one's wishes. Secondary control involves rewarding oneself or avoiding negative thoughts or self-punishment by adjusting one's beliefs or interpretations in response to objective, nonmodifiable conditions (such as parental divorce). The approach of PASCET is that depression may be addressed, in part, by applying primary control when faced with distressing conditions that are modifiable and using secondary control when confronted with conditions that are not.

Program Participants

Elementary school-age children in grades 3–6.

Evaluation Methods

PASCET has been evaluated in a single comparison group study conducted in the early 1990s (Weisz et al., 1997). The sample of 48 students was drawn from three elementary schools with a total population of about 500 students in grades 3 through 6. A three-step selection procedure was used. First, all children for whom researchers had parental consent completed the Children’s Depression Inventory (CDI), a self-report of depression symptoms. Second, teachers and counselors were asked to identify any children whom they believed had significant problems involving depression. In the last step, children who had been nominated by their teachers or counselors, or scored 10 or higher on the CDI, participated in individual clinical interviews on the Revised Children’s Depression Rating Scale. The semistructured interviews were conducted by trained interviewers and covered such depression- related symptoms as unhappiness, guilt, and low self-esteem. Children included in the final sample met one or both of the following criteria: (1) a CDI score of 11 or higher, suggesting elevated depression; and/or (2) a clinical interview score of 34 or higher (i.e., above the 80th percentile for elementary school-age children in previously established scale norms). A total of 51 children met these criteria, and 3 chose not to participate in the study. The resulting sample included 26 boys and 22 girls, with 30 Caucasian (63 percent) and 18 ethnic minority (primarily African American) children. The average age of the students was 9.6 years, and the average CDI score was 18.1, which was the 91st percentile of previously established scale norms.

- 388 -

Within each school, 16 children who could be accommodated in treatment (given available therapists, a room or appropriate space in the school to conduct the intervention, and scheduling availability) were randomly selected from the pool of 48 students to receive the intervention, with the remaining 32 students assigned to the no-treatment control condition. At pretest, the treatment and control groups did not differ significantly in gender, ethnic composition, age, CDI scores, or clinical interview scores. PASCET was implemented during school hours in rooms in, or adjacent to, the school. Children were treated in four small groups of fewer than six children per group, with each group led by two cotherapists. Posttest outcomes were assessed on the CDI and clinical interview within 18 days after the last treatment session. A nine-month follow-up assessment was also conducted for 29 children (60.4 percent) from the original sample. No significant differences were found between study dropouts and completers on pretest variables such as gender, ethnicity, age, CDI, or clinical interview scores.

Key Evaluation Findings

At the initial posttest, Weisz et al. (1997) reported the following:

• From pretest to posttest, children’s self-reported CDI scores dropped about twice as much in the treatment group as in the control group, a difference that was statistically significant.

• Comparisons of posttest CDI scores, with the analysis controlling for pretest scores, showed that scores were significantly lower (i.e., children were less depressed) in the treatment group than in the control group.

• Assessment of the clinical interview scores revealed that the treatment group decline of 12.06 from pre- to posttest was significantly larger than the control group decline of 3.94. However, a different statistical test, which controlled for pretest scores, did not indicate a significant difference in mean posttest scores between groups.

• The authors also assessed the extent to which children moved from above the normal range on the CDI and the clinical interview at pretest to within the normal range after treatment. Treatment-control group differences were significant for the CDI, but not for the interview scores. From pretest to posttest, 50 percent of the treatment group, compared with 16 percent of the control group, moved from above to within the normal range on the CDI.

At the nine-month follow-up, Weisz et al. (1997) found:

• CDI scores dropped about three times as much from pretest to follow-up in the treatment group as in the control group, a statistically significant difference.

• Comparisons of CDI scores at follow-up, controlling for pretest scores, also showed that scores were significantly lower in the treatment group than in the control group.

• Analysis of the clinical interview scores found that the treatment group decline from pretest to follow-up was statistically significant and over three times as large as the control group decline. Despite this difference, an alternative statistical approach that examined nine-month interview scores, while controlling for pretest scores, showed no significant difference between groups.

• With regard to children moving from above the normal range to within the normal range on the CDI and the clinical interviews, the differences between the treatment and control groups were significant for both the CDI and the interview scores. From pretest to posttest, 62 percent of the treatment group, compared with 31 percent of the control group, moved from above to within the normal range on the CDI. Similarly, 69 percent of the treatment group, compared with 24 percent of the control group, moved from above to within the normal range on the clinical interview.

- 389 -

Probable Implementers

Public and private elementary and middle schools

Funding

Previous research on the effects of PASCET has been supported by grants from the National Institute of Mental Health.

Implementation Detail

Program Design

PASCET consists of eight 50-minute sessions guided by a detailed therapist’s manual. Session activities (e.g., role-play, games, and a video) and weekly homework for children are outlined in a children’s practice book containing illustrations and reminders of key points.

Curriculum

Sessions 1-6 of PASCET focus on teaching and practicing primary and secondary control coping methods. The sessions emphasize two primary control skills:

• identifying and conscientiously engaging in activities that the child finds enjoyable

• skill building through goal setting and practice in activities that the child values.

The sessions also teach three secondary control skills:

1. identifying and modifying depressive thoughts

2. cognitive techniques for mood enhancement (e.g., finding a "silver lining" in an otherwise bad experience)

3. relaxation and positive imagery.

In Session 7, a therapist meets with each child individually to discuss themes of the program in relation to specific characteristics of the child and his or her situation. In Session 8, children serve as contestants in a "quiz show," in which they review the overall lessons of PASCET. Staffing

The program requires training of therapists prior to beginning the intervention. Additionally, in presession meetings, therapists role-play the sessions with other therapists, providing the intervention using the manual, practice book, and a written list of specific session objectives. The program is delivered by two therapists, both of whom are present at each session. During each session, one therapist serves as group leader and the other plays a supporting role, including monitoring adherence to the program and ensuring that each section of the material is covered. In the program as evaluated, six therapists implemented the program, including five doctoral-level students in clinical psychology and one licensed faculty member.

Issues to Consider

This program received a "promising" rating, due in large part to the small sample size used in the evaluation. Despite the small sample size, the study used a randomized control design and retested participants nine months after completion of the program, finding significant and sizable program effects on reducing levels of child self-reported depression. While results for PASCET are promising, the sample used to investigate program effects was not severely depressed to begin with, so it would not be appropriate to claim that the intervention is

- 390 -

effective at alleviating clinical levels of depression. Furthermore, although posttest and nine-month follow-up scores on the CDI (controlling for pretest scores) were significantly lower for treatment than control group children, no significant differences were found between groups for the clinical interview scores. The authors suggest that this discrepancy may reflect the CDI’s greater sensitivity to recent changes in symptoms, given that children are asked to report on their symptoms and feelings over the past two weeks. It should be noted that the program developer also served as the evaluator for the program.

Example Sites

Los Angeles, CA

Contact Information

John R. Weisz President and CEO, Judge Baker Children’s Center Professor of Psychology, Harvard Medical School 53 Parker Hill Avenue Boston, MA 02120-3225 phone: (617) 278-4299 fax: (617) 730-5440 email: jweisz@jbcc.harvard.edu

Available Resources

Previously, Dr. Weisz has offered workshop training sessions for mental health professionals interested in implementing the PASCET program. Training includes instruction on program delivery, video illustrations, group exercises, and role-play.

Bibliography

Weisz, John R., Christopher A. Thurber, Lynne Sweeney, Valerie D. Proffitt, and Gerald L. LeGagnoux, "Brief Treatment of Mild-to-Moderate Child Depression Using Primary and Secondary Control Enhancement Training," Journal of Consulting and Clinical Psychology, Vol. 65, No. 4, 1997, pp. 703- 707.

Last Reviewed

February 2007

Project ALERT

Program Info

Outcome Areas Healthy and Safe Children

Indicators Youths not using alcohol, tobacco, or illegal drugs

- 391 -

mailto:jweisz@jbcc.harvard.edu

Topic Areas Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Middle School Type of Service Health Education Parent Education Youth Development Type of Outcome Addressed Physical Health Substance Use and Dependence

Evidence Level Proven

Program Overview

Project ALERT is a school-based prevention program for middle or junior high school students that focuses on alcohol, tobacco, marijuana, and inhalant use. The main goals of the program are to prevent adolescent non-users from experimenting with drugs and to prevent youths who are already experimenting from becoming more regular users. The program began in 1984 and is based on an approach that helps motivate young people to avoid using drugs and teaches them the skills they need to understand and resist pro-drug social influences. These influences may come from family, peers, other adults, or the media. Originally, Project ALERT was organized into a three-month, eight-session curriculum taught during the seventh grade, followed by three "booster" sessions presented in the eighth grade that are designed to reinforce the lessons learned from earlier material. The program uses small-group activities, question-and-answer sessions, role-playing, and the practice of new skills to stimulate students' interest and participation in the ALERT curriculum. Subsequently, Project Alert was revised and strengthened. Parent involvement activities, material on alcohol misuse and a lesson to help smokers quit were added to the curriculum. Today, 18,000 trained Project ALERT classroom teachers present the revised 14-lesson curriculum in more than 3,500 school districts nationwide.

Program Participants

Project ALERT was first used and studied with seventh-grade students from 30 schools in northern California and southern Oregon. Participants were from urban, rural, and suburban areas and represented a variety of racial, ethnic, and socioeconomic backgrounds. Currently, the curriculum is used with students from varied backgrounds around the country. An evaluation of the revised curriculum was conducted with approximately 5,500 students in 48 school districts in South Dakota. Participants were drawn from schools in urban, small-town, and rural communities. This evaluation also included the ALERT Plus program for high school students, but the results have not been published yet.

Evaluation Methods

The initial Project ALERT evaluation of 6,527 students took place from 1984 to 1986 (Ellickson and Bell, 1990a). Thirty schools participated in the study and were assigned randomly to one of three groups after first being matched on school test scores, language spoken at home, drug use among eighth graders, and ethnicity and income level of students. Twenty schools received the Project ALERT

- 392 -

curriculum. Adult health educators taught the curriculum in ten of these schools and adult teachers assisted by teen peer leaders taught the curriculum in the other ten schools. Serving as a comparison group, ten additional schools did not receive the Project ALERT curriculum and continued to implement their school's usual drug education program. Students in both the program group and comparison group completed a self-report questionnaire before and after the presentation of the seventh-grade curriculum. These same students completed additional self-report questionnaires before and immediately after presentation of the eighth-grade booster sessions. Of the original 6,527 participants, 60 percent completed all three follow-up questionnaires and were included in the final analysis. (Attrition was similar across the experimental conditions, and analytic controls adjusted for the small differences across groups.) Questionnaires assessed alcohol, tobacco, and marijuana use and related behaviors and attitudes. In addition, student saliva samples were collected and analyzed for drug use. A later study (Ellickson, Bell, and McGuigan, 1993) surveyed this same group of students annually in the ninth, tenth, and twelfth grades to assess the long-term impact of Project ALERT. Students were classified according to their substance use at baseline. For alcohol use, students were non-users, experimenters, or users. The lowest-risk group (non-users) had never used alcohol at baseline survey, students in the moderate-risk group (experimenters) had used alcohol fewer than three times in the past year and not at all in the past month, and the high-risk group (users) was composed of students who had used alcohol three or more times in the past year or in the past month. The same classification was employed for tobacco use. For marijuana use, a student belonged to the low-risk group if he or she had used neither marijuana nor cigarettes by baseline, to the moderate-risk group if he or she had not used marijuana but had tried cigarettes, and to the high-risk group if he or she had used marijuana by the time of the first survey. A new sample of 5,412 students from South Dakota participated in the Revised Project ALERT curriculum from fall 1997 to spring 1999 (Ellickson et al., 2003). Forty-eight school clusters (high schools and their associated middle school feeders, grouped by geographic region and community size) participated in the study and were randomly assigned to either one of the two treatment groups or to a control group. In the first treatment group, students received the Revised Project ALERT curriculum in seventh and eighth grades; in the second, they also received booster lessons in ninth and tenth grades (ALERT Plus). Students from 34 middle schools received the Revised Project ALERT curriculum, while students from 21 schools were assigned to the control group. To test the effectiveness of the revised middle school curriculum, the two treatment groups were combined through eighth grade. Students in the treatment and comparison groups completed a self-report questionnaire in the fall of seventh grade, before administration of Project ALERT. They completed a follow-up questionnaire after the presentation of the lessons in the spring of eighth grade. Of the original 5,412 participants, 79 percent (4,275) completed the two questionnaires and were included in the final analysis. Students who did not complete both questionnaires included some whose parents refused consent and some who were absent from both the survey and makeup sessions or those who refused to participate. Attrition rates and lost students were similar across the experimental conditions. Similar to the first evaluation, the questionnaires assessed alcohol, tobacco, and marijuana use and related behaviors and attitudes. Also, student saliva samples were collected and analyzed for drug use.

Key Evaluation Findings

Findings are presented chronologically according to the date of the evaluation study. The results from the initial Project ALERT curriculum are presented first, and the results from the Revised Project ALERT curriculum follow. Initial Project ALERT The initial Ellickson and Bell Project ALERT study (1990a), which examined outcomes at grade 8, showed the following significant findings:

- 393 -

Marijuana and Cigarettes

Among all students who had not tried either cigarettes or marijuana at the beginning of seventh grade:

• Project ALERT participants were nearly 50 percent less likely than nonparticipants to have become current marijuana users by eighth grade; after delivery of the eighth-grade booster sessions, this figure increased to more than 60 percent.

• Project ALERT participants were 30 percent less likely than nonparticipants to have started using marijuana, both before and after delivery of the eighth-grade booster sessions.

Among all students identified as "experimenting" with cigarettes at the beginning of seventh grade:

• Project ALERT participants in the teacher-led group were 27 percent less likely than nonparticipants to be current smokers after delivery of the eighth-grade booster sessions and 33 percent less likely to be regular (weekly) smokers.

• Project ALERT participants in the schools using teen leaders were 50 percent less likely than nonparticipants to be regular smokers after delivery of the eighth-grade booster sessions and 55 percent less likely to be daily smokers.

Alcohol Project ALERT produced modest reductions in drinking among all participants immediately after delivery of the seventh-grade curriculum; however, these early gains had eroded by the time students entered the eighth grade. Other Project ALERT produced positive outcomes for participants from a variety of ethnic and economic backgrounds who were at both low risk and high risk for alcohol, tobacco, or marijuana use. It was equally effective with students from schools with low and high numbers of minority students and, where there were differences between the two groups of schools, the results tended to favor those schools with a high number of minority students. Overall, Project ALERT was equally effective when taught solely by classroom leaders and when teen leaders were included in classroom delivery. The Ellickson, Bell, and McGuigan (1993) study, which examined outcomes for 9th, 10th, and 12th graders, found: By the end of high school, Project ALERT no longer had a significant impact on drug use; earlier reductions in cigarette and marijuana use among participants had disappeared. Revised Project ALERT The Ellickson et al. (2003) study found that the Revised Project ALERT curriculum improved on the original, including the following significant results: Cigarettes: Among all students, compared with the control group:

• Fewer students from Revised Project ALERT initiated smoking (26 percent of Project ALERT participants were new smokers compared to 32 percent of the control group).

• Students in Revised Project ALERT reduced their current and regular smoking (20 percent versus 26 percent, and 13 percent versus 17 percent, respectively).

- 394 -

Among all students by risk group, compared with the control group,

• Experimenters were less likely to smoke cigarettes (past-month) in the spring of the eighth grade (29 percent of the program group versus 36 percent for the control group) or to have become regular (weekly) users (18 percent of the program group were regular smokers versus 23.5 percent of the control group).

• Users were less likely to continue smoking (57 percent of the program group were current smokers versus 71 percent of the control group) and to become regular smokers (45 percent of the program group were regular smokers versus 56 percent of the control group).

Project ALERT participants who had not tried cigarettes before exposure to the program were less likely to be regular smokers by the end of eighth grade than their counterparts in the control schools (4 percent of Program Participants were regular smokers versus 6.6 percent of non-participants).

Marijuana Among all students

• The Revised Project ALERT curriculum curbed marijuana initiation (12 percent of the program group initiated marijuana use versus 16 percent of non-participants).

Among all students by level of risk:

• The revised curriculum reduced initiation rates for non-users (5 percent of the program group initiated marijuana use versus 8 percent of the non-participants). Moderate-risk participants were less likely to initiate marijuana use than non-participants (27 percent versus 37 percent). There were no significant effects for the high-risk group.

Alcohol Among all students

• Overall misuse was less likely to occur for participants in Revised Project ALERT.

• Participants in Revised Project ALERT were also less likely than those in the control group to suffer alcohol-related consequences such as fighting and getting in trouble at home or school because of drinking.

Among all students by level of risk:

• High-risk participants in Revised Project ALERT curbed their misuse according to all three measures of alcohol use: overall misuse, alcohol-related consequences and high-risk use (such as binge drinking or use of alcohol with other substances).

A study by Ghosh-Dastidar et al. (2004) explored the impact of the revised curriculum on risk factors for drug use. Their results show that Project ALERT had a moderate effect on all the targeted risk factors associated with cigarette and marijuana use and more modest gains with the pro-alcohol risk factors.

Probable Implementers

Middle and junior high schools (grades 6 through 8)

Funding

The National Institute on Drug Abuse supported the evaluation of the revised middle school curriculum.

- 395 -

The Conrad N. Hilton Foundation provided grants in excess of $10 million to the RAND Corporation to develop and test Project ALERT over a ten-year period. Following the development of the program, grants of more than $40 million from the Conrad N. Hilton Foundation to the BEST Foundation have supported the dissemination of Project ALERT. This grant funding subsidizes the cost of training and program materials and makes it possible for the ALERT curriculum to be offered at a cost of $150 per teacher. This cost includes the required training as well as the curriculum package.

Implementation Detail

Program Design

• Program is age-appropriate and easy to use, and includes clear objectives, detailed lesson plans, preparation tips, and stimulating activities for participants.

• Curriculum contains content specifically geared to drug use; it motivates young people not to use drugs and deals with specific social pressures and societal norms that encourage the use of alcohol, tobacco, and other drugs.

• Program materials, both print and video, are updated on a regular basis. Regular updating is particularly important when using videos to teach adolescents about social influences and misperceptions that encourage alcohol consumption and other drug use.

Curriculum

The Project ALERT curriculum is based on the "health-belief model" (that one’s actions are dependent on one’s beliefs), social-learning theory (learning by exposure to the behaviors and attitudes of others), and the "self-efficacy theory" of behavioral change (belief in one's own competencies and self-reliance). Curriculum content includes lessons on learning the consequences of using alcohol, tobacco, or other drugs; identifying both internal and social pressures to use; practicing resistance skills; understanding that most people do not use drugs; recognizing the benefits of not using alcohol, tobacco, and other drugs; and developing positive alternatives to use. The current Project ALERT program has been strengthened by new sessions on inhalants, smoking cessation, and alcohol that were not part of the original curriculum sequence. It also includes home- learning opportunities designed to foster parental reinforcement of key curriculum messages.

Staffing

Classroom teachers administer the Project ALERT curriculum. However, schools do have the option of using student peer leaders to assist classroom teachers in presenting the ALERT lessons. A required one-day training (workshop or web-based) prepares the classroom teacher to implement the prevention program.

Issues to Consider

This program received a "proven" rating. The evaluations used an experimental design with more than 6,000 students and followed them from seventh grade through twelfth grade. The first study found that Project ALERT had sizeable positive effects on marijuana and tobacco use and modest effects on alcohol use into the eighth grade, although the program effects faded by the time students entered high school. The second evaluation for middle school students showed that the Revised Project ALERT curriculum replicated and improved on the original program’s effectiveness. Furthermore, the program was equally effective for different ethnic and economic groups. While the program developer was involved in all the major studies of Project ALERT, the research was conducted and reported according to high scientific standards.

- 396 -

Example Sites

Wise County Schools, Virginia; Cobb County Schools, Georgia; Round Rock Independent School District, Texas; Houston, Texas; Philadelphia, Pennsylvania; Rapid City Schools, South Dakota; Sioux Falls Schools, South Dakota

Contact Information

Project ALERT is administered by the BEST Foundation for a Drug-Free Tomorrow. For more information or for training, contact: BEST Foundation 725 S. Figueroa St., Suite 1615 Los Angeles, CA 90017 800-ALERT-10 Web: http://www.projectalert.best.org

Available Resources

A free introductory video on Project ALERT is available from the BEST Foundation. After completing the required training (workshop or web-based), teachers receive a curriculum package that includes an orientation video, teacher's manual, eight interactive student videos, and 12 full-color classroom posters. An optional teen-leader curriculum component is also available. Teachers continue to receive free print and video curriculum updates on a regular basis and technical assistance in the form of three newsletters per year, a toll-free teacher assistance phone line, and an on-line faculty chat room.

Bibliography

Bell, R., P. Ellickson, and E. Harrison, "Do Drug Prevention Effects Persist into High School? How Project ALERT Did With Ninth Graders," Preventive Medicine, Vol. 22, 1993, pp. 463-483. Ellickson, P., and R. Bell, Prospects for Preventing Drug Use Among Young Adolescents, Santa Monica, Calif.: RAND Corporation, R-3896-CHF, 1990b. Ellickson, P., and R. Bell, "Drug Prevention in Junior High: A Multi-Site Longitudinal Test," Science, Vol. 247, 1990a, pp. 1299-1305. Ellickson, P., D. F. McCaffrey, B. Ghosh-Dastidar, and D. L. Longshore, "New Inroads in Preventing Adolescent Drug Use: Results from a Large-Scale Trial of Project ALERT in Middle Schools," American Journal of Public Health, Vol. 93, No. 11, 2003, pp. 1830-1836. Ellickson, P., R. Bell, and E. Harrison, "Changing Adolescent Propensities to Use Drugs: Results from Project ALERT," Health Education Quarterly, Vol. 20, No. 2, 1993, pp. 227-242. Ellickson, P., R. Bell, and K. McGuigan, "Preventing Adolescent Drug Use: Long-Term Results of a Junior High Program," American Journal of Public Health, Vol. 83, 1993, pp. 856-861. Ghosh-Dastidar, B., D. L. Longshore, P. L. Ellickson, and D. F. McCaffrey, "Modifying Pro-Drug Risk Factors in Adolescents: Results From Project ALERT," Health Education and Behavior, Vol. 31, No. 3, 2004, pp. 318-334.

Last Reviewed

September 2004

- 397 -

http://www.projectalert.best.org/

Project Northland

Program Info

Outcome Areas Healthy and Safe Children

Indicators Youths not using alcohol, tobacco, or illegal drugs

Evidence Level Proven

Program Overview

Project Northland is a multi-year primary prevention program directed at young adolescents in Grades 6 through 8. The program’s goal is to delay the age when young people begin drinking and to reduce drinking among those who have already started. The project is based on an approach that helps young people understand and resist social pressures to drink alcohol or use other drugs. This involves addressing both individual behavioral change and environmental change. The University of Minnesota School of Public Health developed the program in 1991. Each year of the three-year program has a specific theme with activities tailored to the developmental level of the students. The goal during the first year is to establish communication between parents and students about alcohol use. During the second year, students are introduced to ways to resist and counteract influences to use alcohol. The goal during the third year is to introduce students to groups within the community that play a role in alcohol use and availability, as well as to teach community- action skills to students and parents. The intervention activities include parent involvement/education programs, behavioral curricula, peer participation, and community task force activities. As with many programs, the positive results tend to diminish over time. By the tenth grade there were no longer any significant differences between students who had been in the program and those who had not. To address this concern, Phase II of Project Northland developed a curriculum for Grades 10 through 12. A longitudinal study of that phase was scheduled to be completed in late 2001.

Program Participants

The program is designed for adolescents in Grades 6 through 8. The Phase I evaluation study followed a cohort of 2,351 students who began the sixth grade in 1998. The students were from mostly rural,

- 398 -

lower-middle to middle-class communities in northeast Minnesota; 94 percent of the students were white.

Evaluation Methods

The initial project evaluation took place from 1991 to 1994. Twenty-four school districts in northeast Minnesota participated in the study. The districts were matched for size and then assigned randomly to either Project Northland or to a comparison group. Schools in the comparison group continued to implement the district's usual alcohol and other drug education programs throughout the study period. The evaluation followed the cohort that entered sixth grade in 1991. Students were given a baseline survey in fall 1991, followed by annual surveys each spring in 1992, 1993, and 1994. Of the 2,351 sixth graders who were initially surveyed, 81 percent (1,901) completed the third spring survey. Among those lost to follow-up, there were no significant differences in baseline alcohol use between those in the intervention group and those in the control group. The majority (62 percent) of non-respondents moved out of the area. The surveys asked questions regarding program participation, personal and social influences, and alcohol and tobacco use. The researchers created a “Tendency to Use Alcohol Scale” by combining items about intentions to use alcohol and items concerning actual alcohol use. This scale has been tested for satisfactory psychometric properties. The evaluation assessed community-related factors through telephone surveys with parents and local merchants who sold alcohol, measures of alcohol purchase attempts by youth, and interviews with community leaders.

Key Evaluation Findings

The evaluation by Perry et al. (1996) found that by the end of the eighth grade:

• Differences in alcohol and drug use between the experimental and control groups were statistically significant in several measures.

o Monthly alcohol use was 20 percent lower and weekly drinking was 30 percent lower for all students in the intervention districts.

o Cigarette smoking was 37 percent lower and marijuana use was 50 percent lower among baseline non-users (adolescents who had not yet used alcohol before the program began).

o Combination alcohol and cigarette use in the previous month was lower among intervention students (14.3 percent) than among control students (19.6 percent).

• The program was most effective with baseline non-users.

• The program changed students perceptions about the ways in which many young people consume alcohol, increased students’ refusal skills, and increased parent-child communication about alcohol use.

Probable Implementers

Middle or junior high schools and their local communities.

Funding

The National Institute on Alcoholism and Alcohol Abuse of the U.S. National Institutes of Health funded Project Northland’s three-year demonstration study.

- 399 -

Implementation Detail

Program Design

• The program actively involved the students, starting in the sixth grade and continuing through eighth grade.

• The program used peer leaders at all three grade levels.

• Teachers in each grade received training to maintain consistency of implementation.

• Each year of the program had an overall theme that reflected the program’s components and that was tailored to the cohort’s developmental level.

Curriculum

6th Grade: In the "Slick Tracy Home Team" program, students completed four activity books with their parents over a four-week period. The fictional characters Slick Tracy and Breathtest Mahoney served as role models in the exercises. These exercises provided a forum for families to discuss alcohol-related issues and provided the groundwork for future activities. Parents were also provided with four issues of the newsletter Northland Notes for Parents, which contained information about adolescent alcohol use. A community task force was created. It included government officials, police, business representatives, parents, health professionals, clergy, and other concerned citizens. 7th Grade: The second year’s theme was "Amazing Alternatives!" The activities in this year were focused on the schools. Led by teachers and student leaders, the eight week curricula taught students to identify and resist pressures to use alcohol. Parents were provided with additional activity books and copies of Northland Notes for Parents. The “Exciting and Entertaining Northland Students” (TEENS) program was created as a peer participation program to create alternative alcohol-free activities. The community task force helped to get several ordinances passed during the year to prevent illegal alcohol sales to youth. 8th Grade: The "PowerLines" program included eight classroom sessions and a theater production. The TEENS program, the task force, and distribution of Northland Notes also continued in this year. The goals of the final year were to reinforce previous lessons and to teach community-action and citizen- participation skills.

Staffing

Regular classroom teachers and student peer leaders staffed the classroom portion of the program.

Issues to Consider

The program received a "proven" rating. The evaluation study used an experimental design and followed a cohort of 1,901 students over three years. Comparisons between the intervention and control groups, and between baseline users and nonusers, demonstrated significant effects on several measures of alcohol and drug use by the time students reached the end of eighth grade. The program was significantly more effective for those students who had not yet started using alcohol by the sixth grade. Baseline nonusers were strongly influenced not to initiate drinking by their parents and peers and were more able to resist offers to drink. In contrast, there were no statistically significant differences in alcohol use between baseline users in the intervention or control groups. The users in the intervention groups, however, did show several changes in perceived norms. For example, intervention users were significantly more likely to report that students their age do not drink, that their parents had communicated the consequences of alcohol use, that using alcohol had a negative effect on reputation, and that they were more likely to attend parties where no alcohol was present.

- 400 -

Although the program was effective in reducing alcohol and drug use among eighth graders, the benefits of the program did not persist to later years. By the tenth grade, students in the program and control groups had similar outcomes. A second phase of Project Northland that targets tenth through twelfth graders is currently being evaluated. The results of that second phase should be available by late 2002.

Example Sites

Minnesota

Contact Information

Kaylene McElfresh Special Projects and Training Manager Hazelden Foundation/Publishing Division 800.328.9000 ext 4324 kmcelfresh@hazelden.org Kris Vanhoof Executive Director Content Innovations Management Hazelden Foundation/Publishing Division 800.328.9000 ext 4331 Kvanhoof-haines@hazelden.org

Available Resources

Information regarding Project Northland’s results and implementation details are available from several sources, including:

• www.samhsa.gov

• Making the Grade: A Guide to School Drug Prevention Programs, Washington, D.C.: Drug Strategies, 1999.

In addition, Project Northland curricula, posters, audiocassettes, and other materials may be purchased directly through Hazelden Information and Educational Services.

Bibliography

Perry, C. L., C. L. Williams, S. Veblen-Mortenson, T. Toomey, et al., Project Northland: Outcomes of a Community-wide Alcohol Use Prevention Program During Early Adolescence American Journal of Public Health, Vol. 86, pp. 956-965, 1996. Perry, C. L., et al., Project Northland High School Interventions: Community Action to Reduce Adolescent Alcohol Use, Health and Education and Behavior, pp. 29-49, February, 2000.

Last Reviewed

November 2001

- 401 -

mailto:kmcelfresh@hazelden.org

mailto:Kvanhoof-haines@hazelden.org

http://www.samhsa.gov/centers/csap/modelprograms

Project TRUST

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children not experiencing physical, psychological or emotional abuse

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Elementary School Type of Service Youth Development Type of Outcome Addressed Child Abuse and Neglect

Evidence Level Promising

Program Overview

Project TRUST, which stands for Teaching Reaching Using Students and Theater, uses a series of plays to raise the issues of sexual abuse and violence prevention with students from elementary school through high school. The play Touch is specifically designed to introduce the concepts of sexual abuse and its prevention to elementary-school-age children. Trained high school students present the play, which generally lasts about 30 minutes. The play introduces the concepts of the touch continuum (nurturing, confusing, exploitive), the way to say no in uncomfortable situations, and the idea that perpetrators can be strangers or people the children know. An optional pre-play discussion led by the teachers covers terms used in the play, such as the names of various body parts, types of sexual abuse, and adults to whom students can turn for help. There is also a 15-minute question-and-answer session after the play, which is conducted by the facilitator and actors.

Program Participants

The play Touch is designed for elementary-grade children. The evaluation study consisted of 1,269 children drawn from four elementary schools in a Midwestern city during the 1994–1995 school year. The majority (86 percent) were Caucasian and 22 percent met the federal criteria for free or reduced- fee school lunches.

Evaluation Methods

The study (Oldfield et al., 1996) evaluated students after exposure to the program. Classrooms from the four schools were randomly assigned to either a control group (611 students) who didn’t see the play or a treatment group (658 students) who saw the play. Students in the control group saw the play after all the evaluation data were collected.

- 402 -

Several instruments were used to assess the impact of viewing the play. The first was the Children’s Knowledge of Abuse Questionnaire—Revised (CKAQ), which is a series of 33 true-false items. These items included questions regarding assertiveness, strangers, sexual abuse, secrets that should not be kept, and the possibility that familiar people may touch children in an uncomfortable way. This questionnaire can be administered to the youngest children by reading the items to them. Two other questionnaires assessed the children’s overall level of anxiety, one for children in grades 1–3 and one for those in grades 4–6. The children’s responses were assessed within two days of seeing the play. Retention of the play’s concepts was tested three months later by re-administering the CKAQ to a randomly selected subgroup (111) of the treatment students. The study assessed the effect of groups (treatment and control), their grade in school, and gender effects on knowledge acquired about maltreatment.

Key Evaluation Findings

According to the study by Oldfield et al. (1996):

• Students in the treatment group showed small but significant gains in the knowledge of abuse assessment. Their average score was 26.7 (out of 33) compared with 24.1 for the control group. Average knowledge scores increased with grade level, with treatment scores consistently higher than control scores.

• Earlier research indicated that some abuse prevention programs created fear and anxiety in students. This evaluation found no differences in anxiety between the control and treatment groups.

• The students in the treatment program showed retention of knowledge three months later. The average score for a subgroup of 111 students was 2.2 points higher than the first assessment.

• There were four reports of abuse in the treatment group (two previously confirmed and two first-time reports) and one in the control group (a first-time report), all of which were verified by Children Protective Services.

Probable Implementers

Public, private or parochial elementary schools, social service or public health educators, school theater departments, and youth organizations.

Funding

The evaluation was supported in part by a Prevention Block Grant through the Nebraska Department of Health. Lincoln Public Schools contribute to the funding of Project TRUST-Lincoln, Nebraska. Other programs are funded through local school systems.

Implementation Detail

Program Design

• Using high school students to present the play allows the elementary school students to identify with actors who serve as role models.

• The play intervention is time efficient, requiring about an hour of instructional time for the elementary school students.

Curriculum

There is a script for the play Touch. There is also a discussion guide to help with the question- response period.

- 403 -

Staffing

Facilitators are needed to implement the program. Their responsibilities include recruiting and sometimes training the high school student volunteers as well as coordinating the delivery of the play with the schools.

Issues to Consider

This program received a "promising" rating. There is only one evaluation, and it only assessed the elementary-school version of the program, the play Touch. While the independent evaluation of the play Touch found some positive results, the impact only indirectly affects the benchmark. The evaluation demonstrated that the program increased children’s knowledge about abuse, and clearly the intent is that informed children will be able to prevent instances of abuse. The evaluation did not show, however, that the gain in knowledge led to changes in behavior. Another issue is that the scores on the CKAQ indicate that children continue to have difficulty with concepts such as strangers and that abusers can be people they know and trust. The evaluation study recommended exploring ways to improve children’s understanding of these concepts, such as modifying the play, increasing audience participation, and perhaps adding a home component to reinforce the concepts. Although the evaluation did not directly demonstrate a reduction in abuse, the program does show evidence of being an effective way to communicate prevention concepts to elementary school children. The treatment group reported more incidents of first-time maltreatment after receiving the program, all of which were verified by Child Protective Services. Thus, there was no evidence that the intervention encourages misreporting of abuse. Finally, the program did not cause anxiety in the children, which may be a concern with raising children’s knowledge about this topic.

Example Sites

There are 40 licensed Project TRUST sites operating in the United States. The sites are located in Montana, Wyoming, Nebraska, Kansas, Missouri, Michigan, Ohio, Kentucky, New York, California, Minnesota, and Illinois.

Contact Information

Illusion Theater Karen Gundlach, Education Director & Coordinator 528 Hennepin Avenue, Suite 704 Minneapolis, MN 55403 Phone: (612) 339-4944, ext. 229 Fax: (612) 337-8042 Website: www.illusiontheater.org Susan Letheby, Project TRUST Director, Lincoln, Nebraska Phone: (402) 476-2424

Available Resources

Illusion Theater provides scripts of their other plays as well as a videotape of the play Touch. They also provide information packets on how to become a Project TRUST site.

- 404 -

http://www.illusiontheater.org/

Bibliography

Oldfield, D., B. J. Hays, and M. E. Megel, "Evaluation of the Effectiveness of Project Trust: An Elementary School-Based Victimization Prevention Strategy," Child Abuse & Neglect, Vol. 20, No. 9, pp. 821-832, 1996.

Last Reviewed

May 2009

Quantum Opportunity Program (QOP)

Program Info

Outcome Areas Healthy and Safe Children Children Succeeding in School

Indicators Students graduating from high school Youths abstaining from sexual activity or not engaging in risky sexual behavior

Topic Areas Age of Child Adolescence (13-18) Type of Setting High School Community-Based Service Provider Type of Service Case Management Instructional Support Mentoring Youth Development Type of Outcome Addressed Behavior Problems Cognitive Development/School Performance Teen Sex/Pregnancy

Evidence Level Proven/Promising

Program Overview

Developed by Opportunities Industrialization Centers of America, Inc. (OIC), the Quantum Opportunity Program (QOP) is a youth development program for socio-economically disadvantaged youth. Using a comprehensive case management approach, the program provides year-round services to youth throughout the four years of high school. The program's main goal is to improve academic deficiencies among high school aged youth with low grades at risk of dropping out of school. A secondary emphasis is to establish meaningful, long-term relationships between the student participants, who are called "associates," and program coordinators.

- 405 -

Finally, the program encourages involvement and commitment to school and community. Associates engage in 250 hours of activity in each of three areas every year: education, community service and development activities meant to reduce risky behavior, promote cultural awareness and/or promote recreation. Associates are provided financial incentives, through stipends and bonuses, for participating in QOP activities.

Program Participants

Youth with low grades entering high schools with high drop-out rates.

Evaluation Methods

In the early 1990's, Hahn, Leavitt, and Aaron (1994) analyzed survey data from students at four of five planned study sites: Saginaw, Philadelphia, Milwaukee, San Antonio and Oklahoma City. The program enrolled 25 students at each site. All the students chosen were entering the ninth grade and living in a family that was receiving welfare payments. At each demonstration program site, 50 students entering the ninth grade in 1989 were randomly chosen from lists of families receiving public assistance. Half of the students were then assigned to the program and recruited by the QOP directors. Directors were instructed to how many of the 25 youth assigned to the program could be persuaded to join and were not allowed to select additional youth if the original 25 did not join. Between 21 and 25 initial surveys were collected from each experimental group at the four sites. The remaining students became the control group. A questionnaire collected information on demographics, work experience, school experience, health knowledge, personal attitudes, and opinions. In addition, academic levels (vocabulary, reading comprehension, computation, concepts, mechanics, and expression) and functional skill levels (in knowledge of occupations, consumer economics, government, health, and knowledge of community resources) were assessed using the Test of Adult Basic Education and the Comprehensive Competencies Program (CCP) Tier Mastery Test, respectively. The tests were repeated in 1990, 1991, and in spring of 1993, as was distribution of the questionnaires. A follow-up study (Hahn, et al., 1994) of the original program is based on a questionnaire administered in late fall 1993, several months after participants' anticipated high school graduation dates. In the early 2000's, Schirm, et al (2003, 2004, 2005) conducted another study of QOP. For this study, the program enrolled 580 9th grade students across seven sites around the country: Cleveland, Fort Worth, Houston, Philadelphia, Memphis, Washington, D.C. and Yakima, Washington. Students were randomly selected for treatment from a group of 1,069 eligible youth. Students were eligible if they were starting 9th grade for the first time at a participating school at the start of the demonstration project and had an 8th grade GPA below the 67th percentile of their peer group. Some students with severe physical or learning disabilities were excluded from eligibility. Fifty-three percent of students enrolled in QOP were 14 years old at the start of the demonstration, 52 percent were male, 26 percent were Hispanic and 68 percent were African American. Out of 2,550 students meeting the eligibility criteria, 1,200 were randomly selected and contacted for study participation. The study was unable to locate some randomly selected students, but students who were located were recruited aggressively. Ninety-eight percent (1,069 students) of all located students agreed to participate. Of these, 580 were randomly assigned for enrollment in the QOP and 489 were assigned to a control group. Data were collected at four intervals: twice during the program and twice after the program ended. All available students in the study were contacted for in-person surveys during the spring of the fourth academic year of the program and were asked to take achievement tests in reading and mathematics. Achievement tests were developed through the National Education Longitudinal study and were scored by the Educational Testing Service. Schirm, et al (2004) and Schirm, et al (2006) collected follow-up data from the study participants. During the fifth academic year of the program, students were surveyed over the telephone and portions of students' school records were reviewed. Students were interviewed again by telephone two and four years after the program ended. Students at the Washington, D.C. site entered the program

- 406 -

one year after students at the other sites, so some data collection for these students was delayed by a year.

Key Evaluation Findings

Schirm et al. (2003) found significant differences between QOP and control students in the program's two primary objectives: high school graduation and postsecondary training enrollment. By the fifth year after students started 9th grade:

• Students enrolled in QOP were significantly more likely (46 percent) to graduate high school with a diploma than students in the control group (40 percent).

• Students enrolled in QOP were significantly more likely (32 percent) to attend postsecondary training or education after high school than students in the control group (26 percent).

However; Schirm et al (2006) found that ten years after students entered 9th grade, control students had attained high school graduation rates similar to those of Program Participants. The two groups were neither different in their rates of receiving a diploma nor in receiving either a diploma or GED. In addition, control group postsecondary enrollment rates similar to those of Program Participants. These findings suggest that QOP students, on average, complete high school and enroll in postsecondary training earlier than control students, but by the time students in both groups reach 23-25 years of age, differences in primary objective outcomes are insignificant. In an earlier evaluation, Hahn et al. (1994) found that, compared with the control group,

• There were no significant program effects after only one year.

• By the end of the program in 1993, average associate group scores were significantly higher in all 11 areas.

• Associates were significantly more likely to graduate from high school (of 84 total graduates, 52 were QOP students and 32 were control students).

• Associates were significantly more likely to be in postsecondary school (of 46 students in postsecondary school, 34 were QOP students and 12 were control students).

• Associates were significantly less likely to be high school dropouts (of 57 total dropouts, 19 were QOP students and 38 were control students).

• Associates were less likely to have children at the time of follow-up (of 49 students with children, 20 were QOP students and 29 were control students).

Probable Implementers

Nonprofit community-based organizations, secondary schools, and youth development professionals

Funding

The U.S. Department of Labor funded the program for five sites: Cleveland, Fort Worth, Houston, Memphis, and Washington, D.C. The Ford Foundation provided funding for the other two: Philadelphia and Yakima, Washington.

Implementation Detail

Program Details

• The program operates year-round and combines features of case management, mentoring, computer-assisted instruction, work experience, and financial incentives.

- 407 -

• The QOP program has three major components: 250 hours each year spent on education activities, community service and development activities meant to reduce risky behavior, promote cultural awareness and/or promote recreation.

• The activities are individually tailored using a case management approach, adjusting for short- term and long-term goals and advancement.

• The QOP motto is "Once in QOP, Always in QOP." Associates are never dropped from the program and may return at any point during the four years. Similarly, one of the program's goals is to have the same Coordinator stay with the group for four years.

• The program provides financial incentives to students for participating in the program. Associates receive a stipend for each hour spent on QOP activities, and a bonus of $100 after completing 100 hours of education, development, or service activities in a given year (for up to $300 total). The stipends and bonuses are placed in an interest bearing Quantum Opportunity Account and held for approved use, such as college or job training.

Curriculum

The Comprehensive Competencies Program was developed by Robert Taggart at the Remediation and Training Institute. The program consists of 96 courses, 48 academic and 48 functional, covering such topics as employment, health, and consumer economics. Each course consists of individually paced book lessons supplemented with audiocassettes, videos, and other multimedia CD materials. The educational activities take place in a computer-based learning lab located near the Associates' school. Location can be a factor in participation as transportation may be difficult for many of the Associates.

Staffing

Each site Coordinator is responsible for multiple duties. As a program manager, the Coordinator is responsible for budget and resource management, coordinating the program with schools and community agencies, and planning activities. As a case manager, the Coordinator develops an annual contract with each Associate, keeps monthly progress reports, arranges service activities, and works with Associates on a weekly basis. At larger sites, the case management tasks may be divided among individual counselors, while the Coordinator concentrates on the program management aspects.

Issues to Consider

Maxfield et al. (2003b) found that the QOP model may be difficult or impractical to replicate exactly as intended. The sites that were evaluated in this program implemented programs that were moderately to substantially different from the original QOP model. None of the sites fully incorporated the academic assistance component of the program. Sites faced limitations to the number of hours that staff could be available to enrollees and, on average, enrollees spent only 177 hours on program activities per year. This is 23 percent of the target goal of 750 hours per year. Of the 177 average hours spent participating in program activities, 76 were spent on educational activities, 77 on development activities and 24 on community service-related activities. In the first year of the program, nearly all (99 percent) of participants spent some time on program activities, but in the fourth year, about 26 percent of participants spent no time on program activities. While Schirm et al.'s (2003) study found that treatment students were significantly more likely (32 percent) than control students (26 percent) to enroll in postsecondary education after four years, the follow-up study (Schirm, et al., 2006) found that control students' high school completion and postsecondary enrollment rates were not significantly different from associates by the time most students reached 23-25 years of age. This suggests that, on average, QOP students started postsecondary training earlier than students in the comparison group. Earlier enrollment in postsecondary training could mean that QOP students obtain higher wage employment earlier than comparison group students. If this is true, students enrolled in the QOP likely experienced long-term financial benefits due to finishing high school and starting postsecondary training sooner than they would have without the program. The program did not show any impact on students' likelihood of completing their postsecondary training program.

- 408 -

Example Sites

The most recent demonstration project took place in Cleveland, Fort Worth, Houston, Philadelphia, Washington, D.C. and Yakima, Washington. The pilot demonstration was implemented in San Antonio, Philadelphia, Oklahoma City, and Saginaw, Michigan.

Contact Information

C. Benjamin Lattimore Director, Office of National Literacy Programs Opportunities Industrialization Centers of America, Inc. 1415 N. Broad Street Philadelphia, PA 19122 215-236-4500 fax: 215-236-7480 CBEL2@aol.com

Available Resources

There are several summaries of QOP, including: American Youth Policy Forum, Forum Brief: Promoting Youth Development in Urban Communities-- Unprecedented Success for the Quantum Opportunities Program, 1994. http://www.aypf.org/forumbriefs/1994/fb102894.htm U.S. Department of Health and Human Services, "A National Strategy to Prevent Teen Pregnancies-- Appendix III: Promising Strategies," no date. http://aspe.hhs.gov/hsp/teenp/examples.htm Creating and Evaluating Successful Teen Pregnancy Programs, Accord, N.Y.: Philliber Research Associates, 1998.

Bibliography

Hahn, A., Leavitt, Tom and Aaron, Paul, Evaluation of the Quantum Opportunities Program: Did the program work? A Report on the Post Secondary Outcomes and Cost-Effectiveness of the QOP Program (1989-1993), Heller Graduate School, Center for Human Resources, Brandeis University, Waltham, Mass., June 1994. Hahn, Andrew, "Extending the Time of Learning," in Douglas J. Besharov, ed., America's Disconnected Youth, Washington, D.C.: Child Welfare League of America, Inc., 1999. Lattimore, C. Benjamin, The Quantum Opportunities Program, Blueprints for Violence Prevention Series, Book 4, Center for the Study and Prevention of Violence, Institute of Behavioral Science, University of Colorado at Boulder, 1998. Maxfield, Myles, Laura Castner, Vida Maralani, and Mary Bencill, The Quantum Opportunity Program Demonstration: Implementation Findings, Princeton, NJ: Mathematica Policy Research, Inc., 2003. Schirm, Allen, Elizabeth Stuart, and Allison McKie, The Quantum Opportunity Program Demonstration: Final Impacts, Princeton, NJ: Mathematica Policy Research, Inc., 2006. Schirm, Allen, Nuria Rodriguez-Planas, The Quantum Opportunity Program Demonstration: Initial Post-Intervention Impacts, Princeton, NJ: Mathematica Policy Research, Inc., 2004. Schirm, Allen, Nuria Rodriguez-Planas, Myles Maxfield, and Christina Tuttle, The Quantum Opportunity

- 409 -

mailto:CBEL2@aol.com

http://www.aypf.org/forumbriefs/1994/fb102894.htm

http://aspe.hhs.gov/hsp/teenp/examples.htm

Program Demonstration: Short-Term Impacts, Princeton, NJ: Mathematica Policy Research, Inc., 2003.

Last Reviewed

November 2010

Reaching Educators, Children, and Parents (RECAP)

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children and youth not engaging in violent behavior or displaying serious conduct problems

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Child Care/Preschool Elementary School Type of Service Instructional Support Parent Education Youth Development Type of Outcome Addressed Behavior Problems Mental Health

Evidence Level Promising

Program Overview

The Reaching Educators, Children, and Parents (RECAP) program is a semi-structured, school-based skills training program that lasts for one academic year. It is a comprehensive program designed for children experiencing co-occurring internalizing problems (e.g., withdrawn, anxious, and depressed behavior) and externalizing problems (e.g., aggressive, oppositional, and impulsive behavior). The program consists of four components. These include (a) support for a structured classroom environment via teacher training and in-classroom consultation on implementation, (b) classroom lessons focusing on skills training, (c) individual and small-group sessions with students, and (d) parent training. The primary goal of the program is to reduce the level of children's psychological problems, as well as preventing the development of more serious problems among children who are not referred for formal mental health services. The pre-K RECAP program is a universal intervention program working toward the goals of increasing children's social skills and reducing their internalizing and externalizing problems. Pre-K RECAP

- 410 -

includes (a) a classroom-based curriculum and behavior management system, (b) weekly site-based teacher training and consultation on implementation, and (c) a biweekly parent group meeting.

Program Participants

Preschoolers and elementary school students in grade four

Evaluation Methods

Two studies have examined the impact of the RECAP program. The fourth grade program was evaluated by Weiss et al. (2003) in a sample of 93 children from three schools that served predominantly low-income students. Mental health screening data were obtained from assessments conducted at the end of grade three, and 113 children with elevated internalizing and externalizing problems, and overall psychopathology were found to be eligible for program enrollment. Ninety-three (72 percent) of the families enrolled in the program, and school principals randomly allocated students across two treatment classrooms (62 students) and one control classroom (31 students). Somewhat more of the children were male (63 percent), 56 percent were African American, and 38 percent were Caucasian. Student outcome assessments were conducted at baseline, twice mid-treatment during the fourth grade school year, at post-treatment at the end of the school year, and at follow-up approximately one year later. Eighty-nine students (96 percent) completed participation in the program. Five sources of data were used for the outcome assessment: reports of child emotional and behavioral problems from the child; his or her parents, teachers, and peers; and grade point average averaged across academic subjects. Treatment and control groups were compared on demographic variables and baseline outcome measures, and they were found to differ on one variable — family income — with treatment group parents reporting higher annual incomes than control group parents ($21,480 versus $14,720, respectively). Income was significantly related to treatment outcome for self-reports of externalizing problems, so income was included as a covariate in those analyses. The pre-kindergarten adaptation of RECAP was evaluated by Han et al. (2005). Participants were selected from 12 pre-K classrooms in six public schools that served children from low-income backgrounds. Of the six schools, three were randomly assigned to receive the intervention (six classrooms total), and three schools were randomly assigned to the control condition (also a total of six classrooms). Of the 220 children in the 12 classrooms, 166 (75 percent) children and their parents agreed to participate in the study and completed the pretest assessment. The final evaluation sample consisted of 149 children whose parents provided both pre- and posttest data (90 percent of those who agreed to participate). The average age of the children was 4.4 years, 56 percent of the sample was female, 89 percent was African American, 6 percent was Caucasian, and 5 percent was of another racial/ethnic background. The average annual family income was $16,200 in 2005 dollars. Assessments of treatment and control group students at baseline found that the groups differed significantly on annual household income, with parents in the comparison group reporting a significantly higher level of income than those in the treatment group. Additionally, treatment group children scored significantly higher than control group children for teacher-reports of total problems, internalizing problems, and externalizing problems, whereas treatment group children scored significantly lower than the control group in assertion skills. Comparisons between study dropouts and completers showed there were significantly more males and children living in two-parent households among dropouts than among completers. In addition, teachers rated dropouts as having higher levels of total problems and lower levels of total skills than completers. Post-treatment data were collected in the spring of the academic year, and they included both parent and teacher ratings of children's behavior problems and social skills.

Key Evaluation Findings

Weiss et al. (2003) found the following results for fourth-grade children's internalizing problems, measured across the five time points:

• Treatment group children had significantly better rates of improvement than control group children for teacher-, parent- and self-reports but not for peer-reports.

- 411 -

• There was an increasing treatment efficacy as assessed by parent-reported internalizing problems for those children with higher initial problem levels.

With regard to externalizing behaviors, Weiss et al. (2003) found:

• Treatment group children had significantly better rates of improvement than control group children for peer- and child self-reports of externalizing behaviors, and a marginally significant effect for teacher-report. The effect for parent-reports of externalizing problems was non- significant.

• For peer reports, higher initial levels of behavior problems were associated with lower levels of treatment efficacy.

Weiss and colleagues also found:

• Child outcomes were unrelated to parents’ symptoms of psychopathology.

• Of the eight tests of the effect of gender on outcomes for the RECAP program, only the test involving child self-reported externalizing problems was significant, with boys but not girls showing a significant positive treatment effect.

• No significant treatment effects were found for grade point average.

The study of pre-K RECAP by Han et al. (2005) found the following:

• Teacher ratings of children’s behavior problems showed that children in the treatment group had significantly greater improvement in both internalizing and externalizing behavior problems than children in the comparison group.

o Follow-up analyses also indicated significant benefits for the treatment group for decreases in emotional reactivity, withdrawn behavior, and attention problems.

o No significant differences were found between groups for anxious/depressive symptoms, physical complaints, or aggressive behavior.

• Teacher ratings for child social skills demonstrated some positive treatment effects, with treatment group children showing significantly greater improvement in skills related to cooperation and assertion (but not self-control) than children in the comparison group.

• No significant differences were found between groups for parent reports of children’s behavior problems or social skills.

Probable Implementers

Public and private preschools and elementary schools

Funding

Funding for the evaluations of RECAP has been provided in part by grants from the National Institute of Mental Health, the Vanderbilt Institute for Public Policy Studies, the Byrne Foundation, the Memorial Foundation, the Kroger Foundation, Centerstone Community Mental Health Centers, and the Tennessee Department of Mental Health & Developmental Disabilities.

Implementation Detail

Program Design

The RECAP program encompasses three broad treatment approaches: (a) coping skills training, (b) problem-solving skills training, and (c) parent training. Previous implementations of the RECAP program have taken measures to encourage participation of low-income parents in the weekly

- 412 -

meetings at the school by helping coordinate between families with and without cars, providing taxi service, offering child care, raffling gift certificates to attendees, etc.

Curriculum

Child components of the fourth-grade RECAP program focus on social skills (e.g., making friends and avoiding involvement with negative peer behavior), reattribution training for hostile attributions of others’ intentions as well as unrealistic self-appraisals, communication skills, self-monitoring and self- control improvement, recognition and expression of emotions, and relaxation. Parent and teacher components of the fourth grade program focus on using appropriate praise and punishment, improving adult-child communication, strengthening the adult-child relationship, and supporting children in their use of RECAP skills. Parent and child groups meet once per week, and classroom lessons are taught once or twice a week. The pre-K RECAP program emphasizes the remediation of children’s skills deficits at both the cognitive and behavioral levels, and it focuses on training in social skills, emotional regulation (e.g., awareness, labeling, and monitoring of emotions), and problem solving. The parent and teacher components of the program emphasize positive reinforcement, appropriate use of negative consequences, clear communication and expectations, and strengthening of adult-child relations. In the pre-K program, classroom lessons are taught by teachers two to three times per week and are reinforced daily by teachers using positive reinforcement tokens, modeling and mediation of problem- solving steps, and discussion of behavioral and emotional consequences of behavior choices. Program consultants spend one day per week in the classroom for ongoing training and consultation with teachers and teaching assistants, and observation of teachers' program implementation. The parent- training component is administered by the program consultant in a group format with 16 biweekly sessions at the school.

Staffing

In previous implementations of the RECAP program, master's level clinicians (e.g., psychologists, social workers, and psychiatric nurses) have served as program consultants to teachers and the group leaders for the parent group. Half-time graduate student assistants helped with the group and classroom sessions. Comprehensive training of consultants was provided by the program developers. Initial training involved two full-day sessions, which focused on discussing the conceptual and clinical background of RECAP, the importance of staying within the framework, forms of flexibility acceptable within the model, and how to handle clinical issues within the context of the model. Clinical personnel received 1.5 hours of group supervision per week, which focused on resolving clinical issues and maintaining treatment integrity. Additional individual supervision was provided as necessary.

Issues to Consider

This program received a "promising" rating. Significant treatment effects were found for reducing children's levels of both internalizing and externalizing problem behaviors, although findings were somewhat mixed across study outcomes. Issues to note include that the program developer served as an evaluator on both of the RECAP studies and that the program has been evaluated in only low- income schools in one Southern U.S. city, limiting the ability to generalize findings to other populations. It is important to consider that pre-K RECAP is a universal prevention program, and fourth-grade RECAP is designed for children with a range of severity of psychopathology. In other words, the samples evaluated were not limited to participants that met specific, clinical diagnostic criteria. Around half of the fourth grade sample scored in the clinical range for teacher-reported internalizing problems (48 percent), and teacher-reported externalizing problems (54 percent). In the pre-K study, in which students were not selected for elevated problem levels, 11 percent and about 20 percent of children at

- 413 -

pretest had behavior problem scores that were in the clinical range, based on parent and teacher reports, respectively. The pre-K RECAP evaluation (Han et al., 2005) found significant treatment effects for teacher reports of child behavior problems, but not for parent reports. The study authors' interpretation of these discrepant findings is that the program may be more effective at changing behavior at school than at home, perhaps because pre-K RECAP is a universal program targeting the whole classroom, which makes it efficient in creating behavioral norms to guide individuals' overt behavior. Furthermore, attendance at the parent meetings was very low, with less than 5 percent of parents in the treatment group attending any of the meetings. The lack of parent participation may have decreased program effectiveness at influencing child behavior at home.

Example Sites

Nashville, TN

Contact Information

Dr. Susan Han Vanderbilt Institute for Public Policy Studies 1207 18th Avenue South Nashville, TN 37212 phone: (615) 343-1671 fax: (615) 322-3839 email: Susan.Han@vanderbilt.edu

Available Resources

None at this time

Bibliography

Han, Susan S., Thomas Catron, Bahr Weiss, and Kristen K. Marciel, "A Teacher-Consultation Approach to Social Skills Training for Pre-Kindergarten Children: Treatment Model and Short-Term Outcome Effects," Journal of Abnormal Child Psychology, Vol. 33, No. 6, 2005, pp. 681-693. Weiss, Bahr, Vicki Harris, Thomas Catron, and Susan S. Han, "Efficacy of the RECAP Intervention Program for Children with Concurrent Internalizing and Externalizing Problems," Journal of Consulting and Clinical Psychology, Vol. 71, No. 2, 2003, pp. 364-374.

Last Reviewed

October 2006

Reading Recovery

Program Info

Outcome Areas Children Succeeding in School

- 414 -

mailto:Susan.Han@vanderbilt.edu

Indicators Students performing at grade level or meeting state curriculum standards

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Elementary School Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance

Evidence Level Proven

Program Overview

Reading Recovery is an early literacy intervention designed to help first-grade children who are having difficulty learning to read and write. Developed by Marie M. Clay in the 1970s in New Zealand and introduced in the United States in 1984, the goal of Reading Recovery is to prevent literacy failure. A diagnostic survey is used in conjunction with input from the classroom teacher to identify at-risk children. The lowest literacy achievers work individually with a specially trained teacher for 30 minutes daily for 12-20 weeks until the student's reading performance level reaches the average level of his or her class. Approximately 75 percent reach this level, at which point lessons are discontinued ("discontinued" status). For children who do not reach the average level of their class, a team of professionals offers positive recommendations for future action. In addition to the diagnostic survey and individual tutoring, Reading Recovery offers professional development to teachers. University trainers, site-based teacher leaders, and school-based teachers all engage in initial training for an academic year and ongoing professional development in subsequent years. Training involves working with children—analyzing reading and writing behaviors and relating those behaviors to theories of literacy learning. Professional learning also includes attention to the implementation of Reading Recovery in a school or district setting. Reading Recovery is also available in Spanish, known as Descubriendo la Lectura, for children whose classroom literacy instruction is in Spanish.

Program Participants

Reading Recovery is designed for first-graders who are having difficulty learning to read and write. The intervention is targeted at the lowest literacy achievers in regular first-grade classrooms.

Evaluation Methods

Schwartz (2005) examined the Reading Recovery intervention as implemented by 47 Reading Recovery teachers across 14 states. Teachers participating in the study were already implementing Reading Recovery. Participants volunteered for the study and were not necessarily representative of the national population of teachers implementing Reading Recovery. After obtaining parental consent for their students to participate in the study, each of the 47 teachers chose two low-performing readers to participate. A web program randomly assigned one of these students to the first round of Reading Recovery implementation and the other to the second round of Reading Recovery implementation.

- 415 -

Prior to the intervention and then again following the first round of Reading Recovery, teachers assessed the literacy skills of students from both groups. They used An Observation Survey of Early Literacy Achievement (the "Observation Survey," Clay, 2002, 2005), a diagnostic test developed by the Reading Recovery program developer (see the Issues to Consider section for more information). The Observation Survey comprises six subtests:

• Running Record of Text Reading (Text Level)

• Letter Identification

• Concepts About Print

• Word Reading (Ohio Word Test)

• Writing Vocabulary

• Hearing and Recording Sounds in Words.

The two groups were not different on any of the above measures of reading ability prior to the intervention. Additionally, teachers assessed students following the first round of intervention using the following standardized instruments:

• Phoneme Segmentation Test

• Deletion Task (syllable and phoneme deletion)

• Slosson Oral Reading Test

• Degrees of Reading Power Test (reading comprehension).

Both the first- and second-round students were provided with regular classroom literacy instruction throughout the school year. The first-round students participated in the Reading Recovery intervention during the first half of the school year. The intervention lasted up to 20 weeks, or until the student met the Reading Recovery "discontinuing" criteria, whichever came first. Discontinuing criteria indicate that lessons can be discontinued because the child has reached the average level of literacy performance in his or her classroom, has demonstrated use of reading strategies (based on an analysis of reading behaviors when reading continuous text), and is expected to continue to make literacy progress beyond the end of the intervention. First-round students who were not deemed to have successfully terminated the intervention were nevertheless included in the study sample. First- round students were compared at the end of their Reading Recovery intervention with the second- round students who had not yet received the intervention. While 47 teachers initially volunteered, only data from 37 teachers were included in the analyses. Some students were excluded from the analyses due to incomplete data resulting from study attrition or improper data collection on the part of the teacher. When one student moved or otherwise left the sample, both students for that Reading Recovery teacher were excluded from analysis to obviate issues arising from any teacher-specific effects on outcomes. There have been many additional evaluations of Reading Recovery conducted in the United States. However, only the Schwartz (2005) study meets the PPN evidence criteria.

- 416 -

Key Evaluation Findings

The group of students receiving the first round of Reading Recovery was shown to have improved significantly on all of the Observation Survey measures, compared with the group of second-round students who had not yet received the intervention. These included the following measures:

• Running Record of Text Reading (Text Level): improved by 19.24 times in the intervention group and 6.34 times in the control group.

• Concepts About Print: improved by 77 percent in the intervention group and 54 percent in the control group.

• Letter Identification: improved by 18 percent in the intervention group and 17 percent in the control group.

• Word Reading (Ohio Word Test): improved by 17.44 times in the intervention group and 9.69 times in the control group.

• Writing Vocabulary: improved by 13.75 times in the intervention group and 6.49 times in the control group.

• Hearing and Recording Sounds in Words: improved by 4.60 times in the intervention group and 2.91 times in the control group.

Additionally, students in the first round performed significantly better on the Slosson Oral Reading Test at the end of the first-round intervention compared with second-round students, achieving scores 68 percent higher than those in the control group (no baseline measurement was taken). There were no significant differences across groups on the following measures:

• Phoneme Segmentation Test

• Deletion Task (syllable and phoneme deletion)

• Degrees of Reading Power Test (reading comprehension).

Probable Implementers

School educators, administrators, and school district personnel.

Funding

Reading Recovery interventions may be funded federally as part of the Title I Reading program, from other Elementary and Secondary Education Act titles (professional development, migrant children, English language learners, and others), and from the budgets of state and local school systems. The U.S. Department of Education has funded the scale-up of Reading Recovery in the United States with a five-year (2010-2015) Investing in Innovation (i3) Grant: http://www.i3.readingrecovery.info/qualifying.html

Implementation Detail

Program Design

• Early intervention supplemental to regular reading and writing instruction in the first grade.

• One-to-one tutoring that adjusts to each child's individual needs.

• Lessons involving both reading and writing.

- 417 -

http://www.i3.readingrecovery.info/qualifying.html

• Ongoing training and teacher support.

• Data collected and reported for each child participating in Reading Recovery.

Curriculum

While lessons are individually designed and delivered for each child, tutorial sessions typically include six activities. First, the child reads some familiar books. Second, he or she reads the previous day's new text while the teacher takes a running record. Third, there is letter and word work. Fourth, the child composes and writes a message or story. Fifth, the child reconstructs his or her story that has been cut in parts by the teacher. Finally, the child reads a new text. Throughout the session, the teacher keeps a lesson record of the child's work and strategic behaviors, which is used in planning the next day's lesson.

Staffing

There are three levels of staffing in Reading Recovery: university trainers, site-based teacher leaders, and school-based teachers. Teacher leaders go through a full year of graduate-level training at a university center recognized by the North American Trainers Group. The courses cover Reading Recovery teaching procedures, theory, and implementation processes. Reading Recovery teachers also commit to an academic year of graduate-level training led by a registered teacher leader. The course uses clinical and peer-critiquing experiences to develop the skills necessary for Reading Recovery teaching. For example, during most of the weekly sessions the teachers take turns—one teaching a child while the rest observe on the other side of a one-way mirror and articulate child behaviors and teaching decisions—followed by an analysis of the lesson. Reading Recovery teachers work with children during the training year; courses are generally after the school day so that service to children is concurrent with training.

Issues to Consider

Reading Recovery received a "proven" rating, indicating that it used a rigorous research design to show that the observed impact on reading skills was likely due to the Reading Recovery intervention. It must be noted, however, that the main measure of reading success, An Observation Survey of Early Literacy Achievement (Clay, 2002, 2005), was designed by the individuals who developed Reading Recovery. There is some evidence that a relationship exists between the measures that were used and the intervention itself, suggesting that what is taught is what gets measured (Shanahan and Barr, 1995). Schwarz (2005) did use standardized tests in addition to the Clay Observation Survey. Three of the four of these standardized tools showed no significant difference between Reading Recovery children and the comparison group. It should be noted that in the Schwartz (2005) study, the participating teachers volunteered to participate in the study. These teachers might be different from the typical teacher implementing Reading Recovery, and likewise the classrooms might be different from the typical classrooms implementing Reading Recovery. Evaluation data on every child receiving the Reading Recovery intervention are available on the International Data Evaluation Center website: www.idecweb.us.

Example Sites

Reading Recovery was first implemented in the United States in several schools in Columbus, Ohio, and then expanded statewide. Since 1984, the intervention has been implemented in most of the states in the United States and provinces in Canada. Reading Recovery continues implementations in New Zealand, Australia, and the United Kingdom.

Contact Information

Jady Johnson Executive Director

- 418 -

http://www.idecweb.us/

Available Resources

Reading Recovery Council of North America A not-for-profit association, the Reading Recovery Council of North America (RRCNA) supports Reading Recovery and its implementation. The council's website is a rich resource of information about Reading Recovery. To contact the council:

Reading Recovery Council of North America 500 West Wilson Bridge Road, Suite 250 Worthington, OH 43085 614-310-READ (7323) Fax: 614-310-7345 Website: www.readingrecovery.org and www.readingrecoveryworks.org North American Trainers Group The North American Trainers Group supports the collaborative work of Reading Recovery trainers at recognized university training centers. These centers provide Reading Recovery training as well as technical support to teacher leaders, teachers, and school administrators. A list of university training centers and contacts is posted on the RRCNA website. International Data Evaluation Center The International Data Evaluation Center (IDEC) at The Ohio State University collects data on every Reading Recovery child, analyzes outcomes, and reports findings at multiple levels. A national report is available annually. The IDEC website provides information about evaluation of Reading Recovery. To contact IDEC: The Ohio State University International Data Evaluation Center 1100 Kinnear Road, Suite 126 Columbus, OH 43212

Bibliography

Baenen, Nancy, Alissa Bernhole, Chuck Dulaney, and Karen Banks, "Reading Recovery: Long-Term Progress After Three Cohorts," Journal of Education for Students Placed at Risk, Vol. 2, No. 2, 1997, p. 161. Burroughs-Lange, Sue, and Julia Douëtil, "Literacy Progress of Young Children from Poor Urban Settings: A Reading Recovery Comparison Study," Literacy Teaching and Learning, Vol. 12, No. 1, 2007, pp. 19-46. Center, Yola, Kevin Wheldall, Louella Freeman, Lynne Outhred, and Margaret McNaught, "An Evaluation of Reading Recovery," Reading Research Quarterly, Vol. 30, 1995, pp. 240-263. Chapman, James W., William E. Tunmer, and Jane E. Prochnow, "Does Success in the Reading Recovery Program Depend on Developing Proficiency in Phonological-Processing Skills? A Longitudinal

- 419 -

http://www.readingrecovery.org/

mailto:jjohnson@readingrecovery.org

http://www.readingrecovery.org/

Study in a Whole Language Instructional Context," Scientific Studies of Reading, Vol. 5, No. 1, pp. 141-176. Clay, Marie, An Observation Survey of Early Literacy Achievement, 2nd ed., Portsmouth, N.H.: Heinemann, 2002. Clay, Marie, An Observation Survey of Early Literacy Achievement, revised 2nd ed. with new U.S. norms, Portsmouth, N.H.: Heinemann, 2005. Iversen, Sandra, William E. Tunmer, and James W. Chapman, "The Effects of Varying Group Size on the Reading Recovery Approach to Preventive Early Intervention," Journal of Learning Disability, Vol., 38, September/October 2005, pp. 456-472. Iverson, Sandra, and William E. Tunmer, "Phonological Processing Skills and the Reading Recovery Program," Journal of Educational Psychology, Vol. 85, 1993, pp. 112-125. Pinnell, Gay Su, Carol A. Lyons, Diane E. DeFord, Anthony S. Bryk, and Michael Seltzer, "Comparing Instructional Models for the Literacy Education of High-Risk First Graders," Reading Research Quarterly, Vol. 29, No. 1, 1994, pp. 9-38. Pinnell, Gay Su, Diane E. DeFord, and Carol A. Lyons, Reading Recovery: Early Intervention for At- Risk First Graders (Educational Research Service Monograph). Arlington, Va.: Educational Research Service, 1988. Schwartz, Robert M., "Literacy Learning of At-Risk First-Grade Students in the Reading Recovery Early Intervention," Journal of Educational Psychology, Vol. 97, No. 2, 2005, pp. 257-267. Shanahan, Timothy, and Rebecca Barr, "Reading Recovery: An Independent Evaluation of the Effects of an Early Instructional Intervention for At-Risk Learners," Reading Research Quarterly, Vol. 30, No. 4, 1995, pp. 958-996.

Last Reviewed

March 2013

Reciprocal Teaching

Program Info

Outcome Areas Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards

Topic Areas Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School High School

- 420 -

Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance

Evidence Level Promising

Program Overview

The purpose of Reciprocal Teaching (RT) is to facilitate a group effort between teachers and students by bringing meaning to segments of text. In order to promote understanding, RT is an instructional activity that utilizes dialogue between teachers and students while segments of text are studied. The teacher and students take turns in assuming the role of instructor in leading the two-way dialogue. Four strategies are used to structure the discussion, including: (1) summarizing, which compels students to determine the most important information in the text; (2) question generating, which reinforces the summarizing strategy and increases comprehension; (3) clarifying, which is especially important for students with a history of difficulty with comprehension; and (4) predicting, which increases students’ depth of understanding by hypothesizing what the author will discuss next in the text. These strategies are modeled by the teacher and then practiced by the students in cooperative groups. While the students engage in discussions aimed at constructing meaning to the text, the teacher provides the support necessary for students to learn how to implement RT strategies.

Program Participants

Students in grades 3 through 12

Evaluation Methods

Miller, Miller, and Rosen (1988) studied the effects of an eight-week modified RT program among 64 seventh-grade students. The RT program was modified in that it added the identification of key words and phrases, and instruction was conducted biweekly as opposed to daily. Students were randomly assigned to one of three classes taught by a single social studies teacher. The three classes were in turn randomly assigned to one of three conditions: modified RT (26 students), Control Group I (20 students), or Control Group II (18 students). The two control groups differed in the type of assessments that were administered to students; participants in Control Group I completed the same ten-item comprehension tests and writing samples as the RT group, while participants in Control Group II did not complete any comprehension tests or writing samples. The three groups of students did not differ significantly from one another on their vocabulary scores from the Metropolitan Achievement Test, which was administered three weeks before the study began. Two outcome measures--a ten-question multiple-choice comprehension test and a three-minute writing sample (scored by number of words written)--were used for the RT group and Control Group I and administered during 11 of the 16 sessions. The third outcome measure--academic grades--was recorded for all three groups at the end of both the first and second quarters of the school year. The effects of a 13-session RT program were assessed in a sample of 36 fourth-grade and 36 seventh- grade students from London, Ontario (Lysynchuk, Pressley, and Vye, 1989). The fourth-grade students were enrolled in six schools, and the seventh-grade students were enrolled in two schools. All participants were nominated by their teachers as being adequate decoders of text but poor comprehenders (i.e., the students had satisfactory levels of word recognition but below-average levels of reading comprehension), and none of the students were classified as learning disabled or mentally challenged. All participating fourth-grade subjects had pretested below the 50th percentile on the reading comprehension subtest of the Metropolitan Achievement Test (MAT), while all seventh-grade participants had pretested below the 50th percentile on the reading comprehension subtest of the Gates-MacGinitie Reading Test (GMRT). Following the pretest, pairs of participants were identified at each grade level that had similar pretest scores, and one student in each pair was randomly assigned

- 421 -

to the RT condition and the other student to the control condition. Outcome measures included (1) MAT reading comprehension (fourth graders), (2) GMRT reading comprehension (seventh graders), (3) vocabulary subtest of the Canadian Tests of Basic Skills (fourth graders), and (4) GMRT vocabulary (seventh graders). Taylor and Frye (1992) studied a five-month RT program conducted with a sample of 150 students from four fifth-grade and four-sixth grade classrooms in two suburban elementary schools (the state was not specified). Two fifth-grade and two sixth-grade teachers from one school volunteered to have their average or above average reading students serve as the treatment groups. The control group consisted of two fifth-grade and two sixth-grade groups of students with similar reading abilities from a second elementary school. Average reading ability was determined by the teachers, as indicated by students' placement in the basal series, reading achievement scores, and daily class performance. Four two-group comparisons between the RT and control groups were assessed:

1. Group 1 (RT) consisted of 24 above-average sixth-grade readers, and Group 2 (control) consisted of 18 average sixth-grade readers.

2. Group 3 (RT) had 22 average sixth-grade readers and Group 4 (control) had 20 average sixth- grade readers.

3. Group 5 (RT) contained 15 above-average readers and Group 6 (control) contained 15 average fifth-grade readers.

4. Group 7 (RT) and Group 8 (control) had 20 and 16 average fifth-grade readers, respectively.

Students in Groups 5 and 6 were paired together because they were considered by their teachers to be of somewhat higher ability than students in Groups 7 and 8. Pretest data were collected during three assessments in December, and posttest data (on the same measures) were collected at the end of May. After reading a three- to four-page segment from their social studies book that had not yet been covered, students were asked to (1) summarize the passage in approximately ten sentences, (2) write six important questions on the material, and (3) respond to a set of short-answer questions on the material. Reciprocal Teaching was also assessed in a study of 46 eighth-grade students from seven classes in Auckland, New Zealand (Westera and Moore, 1995). Participating students were considered poor comprehenders but adequate decoders and were selected after scoring the lowest of 300 students on a standardized test of reading comprehension. Approximately half of the study participants were Maori or Pacific Islanders. Eight RT groups were conducted over five weeks, with four of the groups (20 students) exposed to between 12 and 16 RT sessions and the other four groups (15 students) receiving between six and eight sessions. The remaining 11 students served as a control group. No significant differences were found between the groups on pretest reading comprehension scores. Student outcomes were assessed through scores on the Progressive Achievement Test Reading Comprehension instrument. Lovett et al. (1996) studied 46 learning-disabled students in grades 7 and 8 in Toronto, Ontario. Unlike the other studies of RT that were conducted in regular classrooms during regular school hours, the researchers implemented RT in special laboratory classrooms at a pediatric teaching hospital and in satellite classrooms at two community schools. Students were randomly assigned to one of three 25-hour instructional programs. Two programs offered training in expository text comprehension -- RT (16 students) and the Text Content and Structure Program (TCS) (16 students) which required students to elaborate and further process new knowledge from the text being read, focusing on both specific informational text and knowledge of text structure conventions. The third program was the control-group program, which utilized the Classroom Survival Skills program (CSS) (14 students) that trained students in organizational strategies, academic problem solving, and self-help techniques but utilized no text-comprehension training. Subjects were randomly assigned to groups, and were also randomly assigned to instructional texts A (instructed or "taught-to" texts) and B (uninstructed or "not taught-to" texts). Three categories of outcome measures were used: (1) text comprehension strategies and operations, (2) comprehension of text content, and (3) ability to analyze text structure. In addition to measures in these three categories, a battery of standardized tests was also

- 422 -

administered to all students, including the Wide Range Achievement Test—Revised (WRAT-R) Spelling, Arithmetic, and Reading subtests; the Gilmore Oral Reading Test; the Test of Reading Comprehension (TORC) Paragraph Reading subtest; and the Woodcock Reading Mastery Test—Revised (WRMT-R) Word Identification, Word Comprehension, Word Attack, and Passage Comprehension subtests. A 20-day RT program was assessed in a sample of 75 freshman high-school students from two high schools in a middle-class, suburban school district who were enrolled in remedial reading classes (Alfassi, 1998). All students were considered to be poor comprehenders but adequate decoders. The RT group consisted of 53 students who were divided into five reading classes, while the control group included 22 students from a neighboring high school located in the same school district. Control group students were divided into three reading classes. The composition of the classes with respect to race and sex was similar across groups, and no significant differences were found between groups on reading assessments at baseline. Outcome measures assessed eight weeks after program completion included (1) Gates-MacGinitie Reading Test vocabulary and comprehension subtests; and (2) daily passage-comprehension assessments, which included assessing the answers to four text-explicit questions (questions based on information that was directly in the text), four text-implicit questions (questions that required students to locate the information and make links and references across information), and two script-implicit questions (questions that required use of the student’s own knowledge base for the answer). Finally, Johnson-Glenberg (2000) evaluated RT in a sample of 59 third-, fourth-, and fifth-graders from three schools. Students participated for ten weeks in the RT program (22 students) or the visualizing/verbalizing (V/V) program (23 students), or were part of the untreated control group (14 students). The V/V program involved training students to create mental images from the text being read and to discuss these images with other students. All participants were identified by their classroom teacher as having age-appropriate decoding skills but poor comprehension skills. Students were assigned to one of 12 small working groups (with two to five students each), and these groups were in turn assigned to an experimental condition. Ninety-five percent of the students were Caucasian, and less than 10 percent received free/reduced lunch. Outcome measures included (1) a test of oral reading skills, including mistakes, speed, predictions, question generation, recall, open- ended questions, self-reported strategy use, and listening recall; (2) the Detroit Test of Learning Aptitude; and (3) a task involving identification of associated word pairs.

Key Evaluation Findings

The Miller, Miller, and Rosen (1988) study of 64 students found the following:

• RT students scored significantly higher than control group students on the multiple-choice comprehension tests.

• Students in the RT group wrote a significantly higher average number of words (61.58 versus 58.59) per three-minute writing sample than did students in the control group.

• First-quarter grades were not significantly different among the three groups. However, an examination of second-quarter grades indicated that RT students performed significantly better than students in Control Group I. The mean grades for the second quarter, using a four- point grading scale (4.0 equaling an A), were 2.17 for Control Group I, 2.28 for Control Group II, and 2.80 for the RT group.

Lysynchuk, Pressley, and Vye’s (1989) study of 72 students in London, Ontario, reported the following:

• Overall reading-comprehension performance had greater improvement for the RT students than for the control students. The mean pretest-to-posttest gain of 9.97 percentile points in the RT condition was statistically significant, while the increase of 1.63 percentile points in the control condition was not.

• When each grade level was examined independently, the same results were found. The fourth- grade MAT pretest-to-posttest improvement in the RT condition was statistically significant;

- 423 -

likewise, the improvement of seventh-grade GMRT scores was statistically significant. In contrast, neither pretest-to-posttest change in the control condition approached statistically significance.

• No significant differences in vocabulary tests were found between groups at either grade level.

The study of 150 fourth- and fifth-grade students (Taylor and Frye, 1992) found the following:

• In three of the four comparisons of summarizing scores, RT students performed better than control subjects, including above-average sixth-graders (Groups 1 and 2), above-average fifth-graders (Groups 5 and 6), and average fifth-graders (Groups 7 and 8). No significant differences were found between groups for average sixth-graders (Groups 3 and 4).

• No significant differences were found among any of the groups for scores on student- generated questions.

• Results for short-answer scores were mixed. Average fifth-graders in Group 7 (RT) significantly outscored readers in Group 8 (control group). However, average sixth-graders in Group 4 (control group) scored significantly higher than students in Group 3 (RT). No other significant differences between groups were found.

Westera and Moore’s (1995) study of 46 students in New Zealand reported the following:

• Pretest-to-posttest reading comprehension scores differed among the three groups, with the extended RT group scoring significantly higher than the control group. Scores for the short RT group and scores for the control group did not differ significantly.

• These results indicate that the students who received between 12 and 16 RT sessions gained on average more than one age-equivalent year in tested reading comprehension over the five- week period. Ninety-five percent of the extended-RT students showed gains in reading comprehension, compared with 47 percent of students in the short-RT program and 45 percent of students in the control group.

Lovett et al.'s (1996) study of 46 learning-disabled students in Toronto, Ontario, found the following:

• Significant program effects were observed for the seven comprehension strategies/operations measures. The RT students’ score gains from pretest to posttest were significantly better than the score gains of both the TCS and control group students for the four strategies of predicting, summarizing, questioning, and clarifying. These advantages in outcomes were evidenced for both instructed ("taught-to") and uninstructed ("not taught-to") texts.

• For the remaining three comprehension strategies--generating titles, rating the importance of specific ideas, and detecting incongruities in text--RT students scored significantly better on instructed-text materials than the control group students, but not significantly better than the TCS group. For uninstructed texts, no differences were observed among the groups.

• Significant program effects were revealed for the two types of content-comprehension measures. The first measure included elaboration-comprehension questions on specific text content. Analyses indicated that both the RT and the TCS groups improved their scores by a significantly greater amount than did the control group on instructed texts (but not uninstructed texts).

• A significant program effect was also found for the second content comprehension measure, which involved semantic mapping exercises (in which students were asked to generate headings and subheadings that would help them remember the ideas from a text). Both the RT and TCS groups performed better than the control group on mapping content from instructed texts, while the TCS group scored higher than both the RT and control groups on mapping content from uninstructed texts.

• A significant program effect was revealed for the structural analysis measure, in which students attempted to recognize and use conventions of text linguistic structure to understand the author’s intended meaning. Comparisons of outcomes for this measure indicated that the

- 424 -

TCS group had a significant post-test advantage over both the RT and control groups for both instructed and uninstructed texts.

• No significant differences among groups were revealed for any of the standardized measures.

Alfassi's (1998) study of 75 freshman high-school students reported the following:

• A significant difference between the RT and control group for the measure of passage comprehension—the RT group demonstrated a significant improvement in scores from pretest to posttest, while the control group did not.

• No significant differences were found between groups for either the Gates-MacGinitie comprehension test or the vocabulary test.

Finally, Johnson-Glenberg’s (2000) study of 59 elementary school students reported the following:

• The RT and V/V groups on 11 of the measures made significant pretest-to-posttest gains. Significant gains were made by the control group on only one measure (DTLA-Following Directions).

• The experimental groups demonstrated statistically significant greater gains than the control group on four measures, including WRAT word recognition, question generation, answering explicit open-ended questions, and answering visual open-ended questions. Additionally, the RT group demonstrated a marginally significant gain over control group students on answering implicit open-ended questions.

• When the RT and V/V groups were compared, RT students significantly outperformed V/V students on answering explicit open-ended questions. Conversely, the V/V group performed marginally better than the RT group on DTLA—Following Directions.

Probable Implementers

Public and private elementary schools

Funding

No information available at this time

Implementation Detail

Program Design

RT is an instructional approach in which teachers and students engage in dialogue while segments of text are studied. Four strategies are used to structure the discussion, and each strategy attempts to help students construct meaning from text as well as to ensure that they are in fact understanding what they read.

1. Summarizing: In summarizing, students must identify and integrate the most important information in the text. Students may be asked to summarize across sentences, across paragraphs, or across the passage as a whole, and the difficulty level increases as the task includes a more comprehensive amount of text.

2. Question generating: To generate questions, students must first identify the kind of information that is significant enough and substantial enough to provide material for the answers as well as the questions. Students may be required to ask questions focusing on detailed information, or to go so far as to infer new information from the text.

3. Clarifying: When students are asked to clarify, they become aware that there are many reasons why text can be difficult to understand (e.g., new vocabulary, unclear reference words, and unfamiliar or difficult concepts). Students can be taught to recognize the effects of

- 425 -

such impediments to comprehension and to take measures to enhance meaning, for example, rereading the segment or asking for help.

4. Predicting: To successfully predict what the author will discuss next in the text, students must be cognizant of the relevant background knowledge that they already possess regarding the topic. The predicting strategy also makes use of text structure as students learn that headings, subheadings, and questions imbedded in the text are useful means of anticipating what might occur next.

Curriculum

No specific curriculum is required to implement Reciprocal Teaching, because it involves an instructional strategy rather than explicit content.

Staffing

Training for RT is available; however, any teacher who endorses the theoretical framework for RT can implement the program in his or her classroom.

Issues to Consider

This program received a "promising" rating. Four out of the seven studies of RT were conducted using rigorous experimental designs, and five out of the seven studies incorporated baseline measures in order to assess both pre- and post-test student achievement. While many of the evaluations of RT demonstrate that treatment-group students scored significantly better than control-group students on tests of reading comprehension and vocabulary, the results were often mixed. Overall, evaluation findings for this program should be interpreted cautiously due to small sample sizes and an inconsistent pattern of significant findings on outcome measures. For example, Lovett et al.’s (1996) study reported mixed effects favoring both the RT and the TCS group on non-standardized outcome measures, but no significant RT effects on any of the standardized achievement measures. Similarly, Alfassi (1998) reported positive findings for RT versus control group students on a non-standardized test of reading comprehension, but no significant differences were found between groups on standardized comprehension or vocabulary tests. Other studies also reported mixed findings (e.g., Taylor and Frye, 1992; and Johnson-Glenberg, 2000). Despite these mixed findings, RT has been successfully replicated in many locations both within and outside the United States (e.g., Canada and New Zealand). Additionally, all of the studies included in this review were conducted by independent evaluators. The study by Westera and Moore (1995) suggests that the length of RT implementation may be important. The authors found that the reading comprehension scores of extended-RT students (those who received 12 to 16 sessions) were significantly higher than the reading scores of students in a short version of the program. However, none of the evaluations reviewed conducted long-term follow- up of RT participants, e.g., for one year or more, therefore the sustainability of effects cannot be assessed.

Example Sites

Auckland, New Zealand London, Ontario Toronto, Ontario

Contact Information

Annemarie Sullivan Palincsar School of Education University of Michigan

- 426 -

610 East University Ave. Ann Arbor, MI 48109-1259 Tel (734) 647-0622 Fax (734) 936-1606 annemari@umich.edu

Available Resources

Web-based hypermedia to support teacher learning about RT is currently under development, and should be ready for dissemination in the summer of 2005. The materials include video and teacher interviews regarding RT lessons conducted with a group of fourth graders, which can serve as a model for teachers interested in implementing the program.

Bibliography

Alfassi, Miriam, "Reading for Meaning: The Efficacy of Reciprocal Teaching in Fostering Reading Comprehension in High School Students in Remedial Reading Classes," American Educational Research Journal, Vol. 35, No. 2, 1998, p. 309-332. Bell, Nancy, "Visualizing and Verbalizing for Language Comprehension and Thinking," 1986, Paso Robles, California: Academy of Reading Publications. Johnson-Glenberg, Mina C., "Training Reading Comprehension in Adequate Decoders/Poor Comprehenders: Verbal Versus Visual Strategies," Journal of Educational Psychology, Vol. 92, No. 4, 2000, p.772-782. Lovett, Maureen W., Susan L. Borden, Patricia M. Warren-Chaplin, Lea Lacerenza, Teresa DeLua, and Rosa Giovinazzo, "Text Comprehension Training for Disabled Readers: An Evaluation of Reciprocal Teaching and Text Analysis Training Programs," Brain & Language, Vol. 54, No. 3, 1996, p. 447-480. Lysynchuk, Linda, Michael Pressley, and Nancy J. Vye, Reciprocal Instruction Improves Standardized Reading Comprehension Performance in Poor Grade-School Comprehenders, paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA, March 27-31, 1989. Miller, C. Dean, Larry F. Miller, and Lee A. Rosen, "Modified Reciprocal Teaching in a Regular Classroom," Journal of Experimental Education, Vol. 56, No. 4, 1988, pp. 183-186. Taylor, Barbara M., and Barbara J. Frye, "Comprehension Strategy Instruction in the Intermediate Grades," Reading Research and Instruction, Vol. 32, No. 1, 1992, pp. 39-48. Westera, Julia, and Dennis W. Moore, "Reciprocal Teaching of Reading Comprehension in a New Zealand High School," Psychology in the Schools, Vol. 32, No. 3, 1995, p. 225-232.

Last Reviewed

March 2005

Reducing the Risk

Program Info

Outcome Areas Healthy and Safe Children

- 427 -

mailto:annemari@umich.edu

Indicators Youths abstaining from sexual activity or not engaging in risky sexual behavior

Evidence Level Promising

Program Overview

The Reducing the Risk (RTR) curriculum was developed in 1988 (revised in 2004) to help lower the rate of teenage pregnancy and exposure to sexually transmitted diseases (STDs). The RTR curriculum helps teens understand the personal responsibilities and consequences of sexual activity and develop and practice the decisionmaking, negotiating, and refusal skills needed to resist social pressures regarding sexual behavior. In addition, the program aims to strengthen parent-child communication about issues related to sexuality and sexual activity. The RTR curriculum is intended to supplement preexisting sexual education programming, and it is typically presented by specially trained school- system teachers over a three-week period as part of a family life or a general health education program. In addition to school-based settings, RTR has frequently been implemented in other community-based organizations.

Program Participants

Students in grades 7–12

Evaluation Methods

Kirby et al. (1991) studied the impact of RTR in a sample of students from 46 classrooms in 13 California high schools. Participating classrooms were required to offer a health education curriculum to students, and those classrooms with teachers who volunteered to attend RTR training were eligible for randomization and treatment group participation. Approximately half of the classes were randomly assigned to RTR or the comparison group with the same teacher teaching both types of classes, while the other half were assigned nonrandomly to treatment and control groups and were taught by separate teachers. Students in both treatment and control groups were assessed at the start of the program and at 6 and 18 months after program completion on their knowledge of contraceptives, sexual beliefs, and sexual behaviors. There were no statistically significant differences between treatment and control groups in terms of background characteristics or pretest scores, indicating that the two groups were fairly well matched. A total of 1,033 students completed the pretest assessment, 722 students completed the 6-month follow-up (a 30 percent attrition rate), and 758 students (429 RTR and 329 control) completed the 18-month follow-up (an attrition rate of 27 percent). The attrition rate was identical for both treatment and control groups.

- 428 -

A second study (Hubbard, Giese, and Rainey, 1998) evaluated RTR’s effectiveness with high school students in Arkansas. Ten school districts from both rural and urban areas were matched for geographic location, ethnicity, and average per-capita income. Five districts were assigned nonrandomly to participate in RTR, and five districts were assigned to the control group (in which students received their district’s regular health/sexual education curriculum). In each school district, RTR was part of a required, one-semester health education class, and teachers implementing the RTR curriculum were those who volunteered to attend a three-day training session. One classroom in each treatment school district and one classroom in each comparison school district were randomly selected for testing. Students were assessed using a 28-item survey measuring sexual behavior, and all participants were tested prior to program administration and 18 months after the completion. The RTR group was fairly well matched to the control group, as indicated by a lack of statistically significant differences in pretest data. However, in terms of demographic differences, the treatment group was more likely to be female and to attend religious services once a week and less likely to attend more than once per week. A total of 532 students were assessed at baseline (with 512 suitable for analysis), and 212 students were matched at the 18-month follow-up (106 RTR students and 106 control students). The overall attrition rate was 58 percent, with graduation accounting for a significant proportion of the loss in respondents because 23 percent of the sample at pretest were high school juniors and seniors. Students who were sexually active prior to the start of the program were excluded from analysis concerning the initiation of sexual activity (41 percent of the original sample), resulting in a final sample of 125 students. The study did not assess program impacts on teens who engaged in ongoing sexual activity.

Key Evaluation Findings

The research by Kirby et al. (1991) found the following:

• At both 6 and 18 months after program completion, the increase in knowledge regarding contraceptives and their proper usage was significantly greater among members of the treatment group than among comparison group members.

o Between pretest and 6-month follow-up, the average percentage of questions answered correctly by the treatment group rose from 57 percent to 75 percent (a gain of 18 percentage points), compared with an increase in the comparison group from 56 to 65 percent (a gain of nine percentage points).

o From pretest to the 18-month follow-up, the gain in the average score was 18 percentage points among members of the treatment group, and 11 percentage points among members of the comparison group.

• Among those students who were not sexually active at baseline, significantly fewer RTR students than control students became sexually active after initiation of the program.

o No significant differences were found between treatment and control groups in the percentage of students who had initiated intercourse after 6 months (12 percent and 14 percent, respectively).

o After 18 months, significantly fewer treatment than comparison group students had initiated intercourse (29 percent versus 38 percent, respectively).

• Participation in RTR did not significantly impact proportions of students who became pregnant or created a pregnancy.

• Among all students (including those both sexually active and nonsexually active at baseline), there were no significant differences between groups in the use of contraceptives at first intercourse, contraceptive use at most recent intercourse, or in the frequency and consistency of contraceptive use.

• At 18 months, among baseline virgins who initiated intercourse after the start of the program, RTR students were 44 percent less likely than control students to have had unprotected sex at most recent intercourse (9 percent versus 16 percent). RTR students were also 46 percent less

- 429 -

likely than control students to report unprotected sex "all or most of the time" (7 percent vs. 13 percent).

At the 18-month follow-up in a separate study, Hubbard, Giese, and Rainey (1998) found:

• Among baseline virgins (totaling 69 RTR students and 56 comparison students), significantly fewer treatment group students than comparison students initiated intercourse after the start of the program (27.5 percent versus 42.9 percent, respectively).

• Among baseline virgins who initiated intercourse after the start of the program, RTR youth were significantly more likely to use STD and pregnancy prevention (89 percent versus 46 percent, respectively).

Probable Implementers

High schools and community-based organizations

Funding

The evaluations of RTR were funded by the The Stuart Foundation, The William and Flora Hewlett Foundation, and the Arkansas State Department of Education, Comprehensive School Health Program.

Implementation Detail

Program Design

• The RTR curriculum emphasizes the development of decisionmaking and interpersonal negotiating skills.

• Discussion groups and role-playing components give students the opportunity to practice the skills they are being taught.

• Through out-of-class homework assignments, the curriculum encourages students to discuss program content and related issues with their parents.

Curriculum

The RTR curriculum is now in its 4th edition, and it involves 16 to 17 lessons implemented over a three-week period, normally in the school classroom during the regular school day. It is not a stand- alone program, but is intended to supplement preexisting health and sexual education curricula. Based upon three interrelated theoretical approaches—social learning theory, social inoculation theory, and cognitive behavior theory—the premise of RTR is that the likelihood of using condoms or other methods of birth control is determined by an understanding not only of what must be done to avoid STDs and pregnancy, but also on an individual’s belief in their ability to successfully use a given method of birth control/STD prevention. In addition, the theories suggest that youths need specific cognitive and behavioral skills to successfully recognize various forms of pressures (e.g., peer or internal pressures), resist pressures, and negotiate interpersonal encounters. Building upon this foundation, the curriculum offers three components: (a) activities to personalize information about sexuality, reproduction, and contraception; (b) training in decisionmaking, communication, and skills to resist peer pressure; and (c) practice in applying those skills. More specifically, the program provides information on abstinence, sexuality, contraceptives, STDs, and HIV/AIDS. Students are taught to avoid unprotected intercourse by not having sex or, for students who choose to have sex, by using condoms or other contraceptives. In addition to teacher-led information sessions, the program emphasizes interactive exercises including group discussions, role- playing, and homework assignments, such as talking with parents or other adults about the material presented in class.

- 430 -

Staffing

RTR is implemented by regular classroom teachers who have attended a one- to four-day training session. The training focuses on giving teachers an overview of the curriculum and an opportunity to practice the activities that will be utilized in implementing the program, including traditional lectures, discussion, role-playing, and out-of-class assignments.

Issues to Consider

This program received a "promising" rating. RTR was shown to significantly increase the level of knowledge regarding appropriate and correct use of contraceptives, significantly decrease the percentage of teens who initiated intercourse during the 18-month study period, and, for baseline virgins, significantly decreased rates of unprotected sex among those who initiated intercourse. The two evaluative studies each had significant methodological limitations. Both studies had rather high rates of program dropout or attrition (27 percent and 58 percent, respectively). In the Kirby et al. (1991) study, the students who dropped out of the study differed in several ways from those who remained in the study. Researchers conducted no analysis of how this group of dropouts may have impacted results, but they note that of the 758 students assessed at 18 months, no significant differences were found between RTR and control groups for any of the ten demographic characteristics assessed, including grade, gender, race/ethnicity, mother’s education, and religiosity. Further, the 58 percent attrition rate in the study by Hubbard, Giese, and Rainey (1998) suggests that the study’s findings may not necessarily apply to general populations of high school students. A further point of consideration relates to the relative homogeneity of the study populations in both studies. The Kirby et al. (1991) study population was more than 60 percent white, around 70 percent lived with both parents, and nearly 50 percent had mothers who had attended college. The Hubbard, Giese, and Rainey (1998) study population was 85 percent white and over half were moderately religious, attending religious services more than once per month. In addition to indicating homogeneity, these characteristics suggest that the populations of both studies were composed of individuals who may not be particularly high risk. This point is further illustrated by contrasting the 1991 study population with the population used in an evaluation of Self Center, a similar pregnancy prevention program that worked with a low-income, minority, inner-city population in Baltimore (Frost and Forrest, 1995). Thirty-seven percent of the Reducing the Risk population, consisting of 9th through 12th graders, was sexually active at pretest, compared with 92 percent of the boys and 79 percent of the girls in grades 10 through 12 who participated in the Self Center study. The issues of homogeneity and risk status may warrant consideration when weighing the appropriateness of the Reducing the Risk curriculum for a given population. In addition, aspects of the study designs may have biased the outcomes. First, in both studies, only teachers who volunteered to attend the program training were eligible to participate. It is possible that teachers willing to dedicate the time to training are more engaged and/or are better teachers than their counterparts; thus, teacher characteristics may have positively biased the results. Further, the study involved randomization at the classroom level rather than the school level. It is possible that communication and/or relationships (dating) between students of different treatment status influenced control outcomes.

Example Sites

California, Arkansas

Contact Information

John Henry Ledwith ETR Associates 4 Carbonero Way Scotts Valley, CA 95066-4200

- 431 -

phone: 1-800-321-4407 fax: 1-800-435-8433 email: jhl@etr.org

Available Resources

The RTR curriculum packet is available from ETR Associates in English and Spanish and includes background information, lecture notes, suggested program activities, and a student workbook. A supplementary activity kit also is available. Materials are available on the ETR Web site, http://www.etr.org/pub. Another RTR program package is available from the Sociometrics Corporation (http://www.socio.com/srch/summary/pasha/full/paspp04.htm). This package includes a user’s guide, teacher’s manual, student workbook, pamphlets, evaluation instruments, directory of evaluation consultants, and telephone technical support for one year.

Bibliography

Frost, Jennifer J., and Jacqueline Darroch Forrest, "Understanding the Impact of Effective Teenage Pregnancy Prevention Programs," Family Planning Perspectives, Vol. 27, No. 5, 1995, pp. 188-195. Hubbard, Betty M., Mark L. Giese, and Jacquie Rainey, "A Replication Study of Reducing the Risk, a Theory-Based Sexuality Curriculum for Adolescents," Journal of School Health, Vol. 68, No. 6, 1998, pp. 243-247. Kirby, Douglas, Richard P. Barth, Nancy Leland, and Joyce V. Fetro, "Reducing the Risk: Impact of a New Curriculum on Sexual Risk-Taking," Family Planning Perspectives, Vol. 23, No. 6, 1991, pp. 253- 263.

Last Reviewed

February 2007

Resolving Conflict Creatively Program (RCCP)

Program Info

Outcome Areas Healthy and Safe Children Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards Children and youth not engaging in violent behavior or displaying serious conduct problems

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Elementary School Middle School

- 432 -

mailto:jhl@etr.org

http://www.etr.org/pub

http://www.socio.com/srch/summary/pasha/full/paspp04.htm

Type of Service Youth Development Type of Outcome Addressed Behavior Problems Cognitive Development/School Performance Violent Behavior

Evidence Level Promising

Program Overview

The Resolving Conflict Creatively Program (RCCP) is a comprehensive, school-based, violence- prevention program designed for use with children in kindergarten through eighth grade. The 51- lesson program curriculum and accompanying activities are tailored to be developmentally appropriate for a given age group. Through promoting positive conflict resolution and understanding of different cultures, the program strives to create a more caring and peaceful school environment. RCCP now serves over 400 schools in 16 urban, suburban, and rural districts across the country. The program is based on the theory that aggressive and violent behaviors are learned and therefore can be affected through education. The primary goal of RCCP is to increase children’s levels of knowledge regarding ways in which to approach conflict situations, to develop children’s conflict resolution skills, and to promote children’s positive interpersonal and intergroup relations. In addition, RCCP attempts to combat prejudice and stereotypes, and the various types of conflict and violence that may arise as a result of these issues, by teaching children how to recognize and oppose prejudice and by increasing their respect for and understanding of diversity. Finally, RCCP aims to transform the overall school culture into one that exemplifies nonviolent conflict resolution and a respect for and openness to diversity. RCCP involves classroom instruction of children by trained teachers. In addition, the program includes the recruitment, training, and supervision of children to act as peer mediators. These mediators help facilitate the resolution of conflicts among children both in the classroom and elsewhere in school.

Program Participants

The program is designed for use with children in kindergarten through eighth grade. To date, however, the impact of the program has been evaluated on children of elementary school age only (grades one through six).

Evaluation Methods

The first study of RCCP was conducted by the National Center for Children in Poverty (Aber et al., 1998, and Aber, Brown, and Henrich, 1999). The evaluation employed a quasi-experimental research design, with a sample of 5,053 children in grades two through six and nearly 400 teachers in 15 public elementary schools across four major school districts in New York City. At the start of the study (fall 1994), the participating classrooms were divided into four groups in varying stages of intervention: non-intervention (control), the beginning stage of intervention, integration of some program components, and integration of all program components. The schools participating in the evaluation were drawn from among the 87 schools across New York City already involved with RCCP (with the exception of the year-one "no-intervention" group). Schools were selected equally from the four school districts and were chosen so that student race/ethnicity, poverty status, and school size were comparable. Participating children were evenly distributed across the grades. Overall, 48 percent of the sample was female; 43 percent was black, 36 percent was Latino, 16 percent was white, and 6 percent was of another race/ethnicity; and 82 percent received free lunch. Data were collected in fall 1994 and spring 1995. Outcomes, such as aggressive fantasies, attributional biases about aggression,

- 433 -

interpersonal negotiation strategies, psychological symptomatology (e.g., fighting, teasing), and depression, were assessed at the end of the first year of the study (spring 1995).

For analytic purposes, classrooms were observed to assess teacher level of RCCP implementation and then grouped accordingly. Three distinct profiles of implementation were identified. The High Lessons profile was characterized by teachers who received a moderate amount of training and taught many RCCP lessons (an average of 23 lessons), and had a classroom composition with relatively few peer mediators (children nominated for additional training in mediation skills; see Implementation Detail at the end of this program description). The Low Lessons profile was characterized by teachers who received the most training and coaching and taught few RCCP lessons (an average of two lessons) and had a classroom composition with the highest percentage of peer mediators. Finally, the No Lessons profile was characterized by teachers who received no RCCP training and taught no RCCP lessons. They may, however, have had some peer mediators included in their classroom. Teachers were not assigned to their implementation profile — in other words, they self-selected the amount of lessons they taught and the amount of training they received. Aber, Brown, and Jones (2003) reported a second study of the effects of RCCP in a sample of 11,160 first through sixth graders from the same 15 New York City public elementary schools. The study took place in the 1994-1995 and 1995-1996 academic years. As in the first study, the elementary schools were initially divided into four groups on the basis of stage of intervention: nonintervention, the beginning stage of intervention, integration of some program components, and integration of all program components. Among those in the sample, 48 percent were female, 41 percent were Hispanic, 40 percent were black, 14 percent were white, and 5 percent were of another race/ethnicity. Approximately 86 percent of the students were receiving free school lunches. Data were collected at four points in time in the fall and spring over the two academic years. Of the 11,160 children, approximately 9 percent participated at only one data-collection time point, 42 percent participated at two time points, 4 percent participated at three times points, and 45 percent participated at all four time points. Children who participated at fewer time points scored significantly lower in reading and math achievement in the spring of 1994 (Year 1) and had higher rates of absence during Years 1 and 2 than did children who participated more frequently. Multilevel growth curve modeling (a statistical technique that looks at children’s developmental trajectories, or rates of change, over time) was used to analyze program outcomes similar to those explored in Aber et al. (1998). A dissertation by Brown (2003) addressed a quasi-experimental study of RCCP in a sample of predominantly low-income, minority children in the second and third grades. The students were from 15 New York City public elementary schools in four school districts, and the study took place over three years in the mid-1990s, with data collected at three or four points in time, depending on the outcome studied. The schools were initially divided into four groups in varying stages of intervention of RCCP: non-intervention, beginning stage, integration of some program components, and integration of all program components. Groups of schools were chosen whose student race/ethnic background, poverty status, and school sizes were comparable both across the districts and across stages of RCCP implementation. Among the total sample, 49 percent were female, 40 percent were Hispanic, 39 percent were black, 16 percent were white, and 4 percent were of another race/ethnicity. A total of 2,543 children had California Achievement Test math scores at pretest in spring 1994, with 22.5 percent (n = 572) missing in spring 1995 and/or spring 1996. Compared with children who completed follow-up math tests, children with missing math scores had significantly lower math scores in spring 1994, were more likely to be Hispanic, and were more likely to receive free lunch. In addition to math achievement, other outcome measures included teachers’ perceptions of children’s aggressive behavior. Latent growth curve modeling was used to analyze program outcomes.

Key Evaluation Findings

Aber et al. (1998) and Aber, Brown, and Henrick (1999) found the following:

• Children in all three analytic profiles (High Lessons, Low Lessons, and No Lessons) experienced an increase over time in their hostile attributions (e.g., when presented with a hypothetical ambiguous vignette, the child was more likely to attribute hostile, rather than benign, intent to the actors in the scene). In addition, children in all three profiles experienced

- 434 -

http://www.promisingpractices.net/program.asp?programid=119%23implementation

an increase over time in their aggressive interpersonal negotiation strategies (i.e., by selecting from responses to hypothetical situations categorized on a four-point scale ranging from aggressive to nonagressive) and a decrease in their positive, proactive interpersonal negotiation strategies.

o The rate of negative changes was significantly slowed for those children in the High Lessons profile as compared with children in the other two groups.

o The High-Lessons profile children did not experience a significant decrease in their ability to positively and proactively problem-solve and negotiate, whereas for both the Low Lessons and No Lessons (control) group, the decrease was significant.

o The largest negative increases were found in the Low-Lessons profile children. Their rate of increase in negative behaviors exceeded that of the No Lessons population.

• Children in all three analytic profiles experienced an increase over time in their aggressive fantasies (e.g., responses to questions such as, Do you sometimes have daydreams about hitting or hurting someone you don’t like?) and conduct problems.

o Among children in the High Lessons profile, the increase in aggressive fantasies was not statistically significant, although children experienced a significant increase in conduct problems.

o The size of the increase in both aggressive fantasies and conduct problems was significant for both the Low Lessons and the No Lessons profiles. The average level of increase of aggressive fantasies and conduct problems at the end of the year in children in the Low Lessons profile was significantly larger than the average level of increase in both the High Lessons and Low Lessons groups.

• Program impact was different for boys than it was for girls. While boys in the High Lessons profile did not have a significant decrease over time in their positive negotiation strategies and behaviors as compared with the girls in the High Lessons profile, boys in the other two profiles had a significantly larger decrease than did girls in those groups.

• Program impact was found to be slightly less effective with older children and children in high- risk classrooms and neighborhoods. This finding was consistent across the analytic profiles.

Aber, Brown, and Jones (2003) reported the following:

• Higher levels of classroom instruction in RCCP were associated with lower levels of hostile attribution bias, aggressive strategies, depression, and conduct problems, and with higher levels of competent interpersonal strategies.

• In contrast, higher levels of teacher training and coaching were significantly associated with an increase in hostile attribution bias, aggressive strategies, depression, and conduct problems, as well as with a decline in competent interpersonal strategies.

• Children receiving higher levels of classroom instruction relative to teacher training and coaching ("high lessons") had a slower rate of increase in aggressive fantasies than did children receiving higher levels of teacher training and coaching relative to classroom instruction ("high training and coaching").

• Children in the High Lessons group were reported by teachers as being generally consistent in their levels of aggressive behavior over time compared with the "high training and coaching" children, whose aggressive behavior was reported as steadily increasing over time.

• Similarly, the High Lessons children’s teacher-reported prosocial behavior steadily increased over time, whereas such behavior in "high training and coaching" children declined slightly during the same period.

• Intervention effects were for the most part consistent across different demographic groups, such as race/ethnicity, gender, and economic resources (as defined by school lunch eligibility).

- 435 -

The study by Brown (2003) found the following:

• A higher level of exposure to RCCP lessons (analyzed on a continuum) predicted significant growth in math achievement, as well as significant decreases in teacher perceptions of negative behavior. In addition, a significant, negative relationship was found between teacher perceptions of negative behavior and growth in math achievement.

• The program was effective for both boys and girls, and for both students receiving free/reduced-price lunch and those receiving full-priced lunch.

• Differences between groups based on race/ethnicity included the following:

o The direct effect of RCCP lessons on math achievement was significant for black and Hispanic children but not for white children.

o The relationship between RCCP lessons and teachers’ perceptions was not significant for black children, nor was the relationship between teachers’ perceptions and math achievement. These findings suggest that the indirect effects of lessons on math achievement through reductions in teachers’ perceptions of negative behavior are evident only for white and Hispanic children.

Probable Implementers

Public or private elementary and middle schools

Funding

The research on RCCP was funded by the Centers for Disease Control and Prevention and the W.T. Grant Foundation. Additional funding for program research was provided by the Pinkerton Foundation, the Surdna Foundation, and the Kellogg Foundation.

Implementation Detail

Program Design

• RCCP is introduced into a school gradually over the course of several years. Initially, a few motivated teachers are recruited to participate, and a relatively low number of RCCP lessons is offered. This marks the "beginning" phase of implementation. From that phase, a school moves to the "consolidation" phase, in which additional teachers are added and the administrator training and peer mediation components are introduced. Next comes the "saturation" phase, in which even more classrooms are added to the program. Finally, a school reaches the "full model" of intervention, in which the program has been implemented schoolwide and a targeted intervention for high-risk youth has been added.

• The peer mediation component of the program is designed to give children opportunities to practice skills they have learned. Children in grades four through six are nominated by their classmates or appointed by teachers. Nominated children receive a three-day training session and are then identified as peer mediators who may intervene in conflicts during non-classroom time. Mediators wear special T-shirts and work in pairs. They do not intervene in physical fights.

• RCCP offers instruction to school administrators to educate them about the program and to provide implementation support.

Curriculum

RCCP has a set curriculum. The number of lessons included in the curriculum, however, varies with the individual teacher and level of implementation. The number of lessons given over the course of the years can range from zero to 55, with an average of 13 lessons. The number of lessons included is determined by the level of implementation in the school. As the school moves through the four levels

- 436 -

of implementation (beginning, consolidation, saturation, and full model), additional lessons are added to participating classrooms. The RCCP curriculum is designed around several core skill areas: building communication, learning to effectively recognize and express feelings, dealing with anger, resolving conflicts, fostering cooperation, respecting and appreciating diversity, and countering prejudice. The lessons are organized into units based on these skills. Each unit is designed to take from one-half to one full hour. Lessons are presented in a "workshop" format, that is, the teacher’s role in the lesson is not to lecture and impart specific knowledge, but rather to facilitate student-directed discussions and learning. Skills are taught through role-playing, interviewing, brainstorming, and small-group discussions. Different versions of the curriculum have been developed for children in the lower and upper elementary school grades and for students in junior high and high school.

Staffing

The program is implemented by classroom teachers. In addition to an 18- to 24-hour introductory course in which teachers learn about and practice the skills they will be teaching in the RCCP, curriculum teachers receive follow-up support and development from RCCP program staff.

Issues to Consider

This program received a "promising" rating. The research was conducted using a well-implemented, semi-randomized controlled experiment and involved a large sample size. Despite some positive findings, there are some methodological limitations to the evaluations, and there were conflicting outcomes. The first study suggests that while there were positive findings, those outcomes were limited to one intervention group—the High-Lessons students. Further, the findings indicated that the program may not be as effective with higher-risk populations that are likely to be more prone to violent and aggressive behaviors. In addition, the individual program components were not analyzed to assess their relative contributions to outcomes (positive or negative) or level of implementation (High or Low). As such, it may be difficult to determine what aspects of the program are producing the positive or negative outcomes. While children’s assignment to classrooms was random, teacher participation was not. Only teachers who agreed to participate in RCCP training, or who had completed RCCP training, participated in the evaluation. The program’s materials describe the teachers who are recruited to participate in the program as "highly motivated." It is possible that the characteristics of teachers who are willing and interested in participating in the RCCP program differ from the characteristics of other teachers, and that it is these differences among teachers, rather than the program curriculum, that are producing the observed outcomes. An issue of considerable concern from the first study (Aber et al., 1998) is in regard to the findings that indicate that individuals who received low numbers of lessons had poorer outcomes relative to those in the other two groups. It is worth noting that if low levels of lessons are detrimental or in fact produce worse outcomes than no exposure to the program whatsoever, then the issues of fidelity of implementation and teacher motivation become very important to the success of the program. A finding of interest from Aber, Brown, and Jones (2003) is that higher levels of classroom instruction and lower levels of teacher training and coaching were related to significant reductions in the risk of a child having poorer outcomes. The authors suggest that this somewhat counterintuitive finding may be due to the positive relationship between classroom instruction and teacher training. In other words, if RCCP program staff perceived that a teacher needed additional help, in part because they believed the teachers were not delivering the intended amount of RCCP classroom instruction, they focused more intently on training those teachers. Therefore, those teachers who received lower rates of training and coaching may be those who were successfully implementing the classroom instruction to begin with, which in turn led to the children’s observed positive outcomes. Finally, it is important to note that positive program effects may not be consistent across all populations. The findings from Aber et al. (1998) and Aber, Brown, and Henrich (1999) indicate that the program is less effective on older children. At the same time, the baseline measures indicate that the behaviors and attitudes that the program targets increase with age. The program has been

- 437 -

evaluated to date with children of elementary school age only. Additionally, the results from Brown (2003) suggest that the RCCP lessons’ direct effects on math achievement extend only to Hispanic and black children and not to white children, while the effects of RCCP on reductions in teachers’ perceptions of negative behavior are evident only for Hispanic and white children.

Example Sites

Anchorage, Alaska; Atlanta, Georgia; Boston, Massachusetts; Lawrence, New York; Lincoln County, Oregon; New Orleans, Louisiana; New York City Public Schools; Newark, West Orange, and South Orange-Maplewood, New Jersey; Roosevelt School District, Arizona; and Vista Unified School District, West Contra Costa, and Modesto City Schools, California.

Contact Information

Lisia Morales Program Director RCCP/ESR 23 Garden Street Cambridge, MA 02131 phone: 617-492-1764 x31 fax: 617-864-5164 email: lmorales@esrnational.org

Available Resources

The Educators for Social Responsibility (ESR) Web site provides assistance with RCCP training and implementation online at http://www.esrnational.org/es/rccp.htm; assistance with the online training and implementation may be obtained by calling 1-800-370-2515. ESR’s services include a locals needs and resource assessment to guide the development of the program and provide baseline data for future tracking of RCCP’s effectiveness, professional development for teachers, on-site classroom support, leadership training for administrators, support staff workshops, parent workshops, advanced training workshops for experienced RCCP classroom teachers, peer mediation training for selected students, and district capacity building. Costs of services include a four-day on-site workshop ($4,800), on-site follow-up services ($1,200 per day), and program materials ($55 for every 25 students).

Bibliography

Aber, J. Lawrence, Joshua L. Brown, and Christopher C. Henrich, Teaching Conflict Resolution, An Effective School-Based Approach to Violence Prevention, New York: National Center for Children in Poverty, 1999. Aber, J. Lawrence, Joshua L. Brown, and Stephanie M. Jones, "Developmental Trajectories Toward Violence in Middle Childhood: Course, Demographic Differences, and Response to School-Based Intervention," Developmental Psychology, Vol. 39, No. 2, 2003, pp. 324-348. Aber, J. Lawrence, Stephanie M. Jones, Joshua L. Brown, Nina Chaudry, and Faith Samples, "Resolving Conflict Creatively: Evaluating the Developmental Effects of a School-Based Violence Prevention Program in the Neighborhood and Classroom Context," Development and Psychopathology, Vol. 10, 1998, pp. 187-213. Brown, Joshua L., "The Direct and Indirect Effects of a School-Based Social-Emotional Learning Program on Trajectories of Children's Academic Achievement," dissertation, Columbia University, 2003.

- 438 -

mailto:lmorales@esrnational.org

http://www.esrnational.org/es/rccp.htm

Last Reviewed

January 2006

SafeCare

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children not experiencing physical, psychological or emotional abuse

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Home Visiting Type of Service Family Support Parent Education Type of Outcome Addressed Child Abuse and Neglect

Evidence Level Promising

Program Overview

SafeCare® is a home visiting program for parents of children ages 0-5 years who are at risk for child maltreatment or have been reported to Child Protective Services (CPS) for child maltreatment. The program aims to reduce subsequent child maltreatment by educating parents on home safety and organization skills, child health and nutrition management, and parent-child interaction skills. SafeCare uses trained home visitors to educate parents on these components such that their skills are generalizable across settings, time, and behaviors (Lutzker and Bigelow, 2002).

Program Participants

SafeCare was designed for use among parents of children ages 0-5 years who are at risk for child maltreatment or parents who have been reported for child maltreatment.

Evaluation Methods

Several studies have evaluated the effectiveness of SafeCare. Only one study assessed the effect of SafeCare using a randomized controlled design where intervention participants were compared with control participants (Chaffin et al., 2012). Other studies assessed the impact of SafeCare without a control comparison group or without randomization of participants to intervention and control groups. The results of the study by Chaffin et al. (2012) that meets Promising Practices Network (PPN) criteria are presented here.

- 439 -

http://www.promisingpractices.net/criteria.asp

Chaffin et al. (2012) assessed the impact of SafeCare among 2,175 parents referred for home-based services by CPS over a three-year period between September 30, 2003, and October 1, 2006, after being reported for child maltreatment in two urban and four rural CPS regions within one state. Families were eligible for participation if all children were 12 years of age or younger, at least one parent had been reported for maltreatment, and no parents had been reported for sexual abuse. The 2,175 parents were assigned to one of 219 home visitors. Home visitors were randomly assigned at the region level to provide intervention or control services to the parents they served. There were 1,153 parents in the SafeCare intervention group and 1,022 parents in the control group. Control parents received the standard home visiting program provided by CPS, which included six months of unstructured services provided at a minimum of once a week. Parents in the intervention group received weekly visits for six months from home visitors who had received SafeCare training. The intervention group's home visits were structured following the SafeCare model, which focuses on behavior skills, training to address parent-child interactions, basic caregiving, parenting routines, home safety, and child health. Parents' CPS reports were collected from a statewide CPS database for an average of six years after study enrollment. Parents were determined to have had a subsequent maltreatment event if there was any report in the CPS system after study enrollment. There were significant baseline differences between the intervention and control participants on several characteristics. Significantly more intervention parents than control parents were American Indian (19% vs. 14%); had preschool-aged children (79% vs. 72%); lived in a community with a population greater than 75,000 people (29% vs. 26%); had a higher baseline risk estimate for child maltreatment based on a number of factors, including demographics, previous CPS referrals, score on the Family Resources Scale, substance use screen score, and depression screener scale; and had a higher per capita child maltreatment report rate in the county of residence (45 vs. 41 per 1,000 children). Significantly fewer intervention parents than control parents were non-Hispanic white (64% vs. 70%). Intervention parents also had fewer prior CPS reports on average (4.41 vs. 5.07), were more likely to complete the home visiting program (90% vs. 86%), and had a shorter average follow- up time (6.15 years vs. 6.25 years). Among the home visitors, there were significant differences between the intervention and control groups on gender (85% female vs. 91% female), average age (40.4 years vs. 36.3 years), years on the job (7.1 years vs. 4.1 years), percentage American Indian (18% vs. 10%), percentage Hispanic (2% vs. 7%), percentage non-Hispanic white (60% vs. 66%), licensure for home visiting (15% vs. 20%), attainment of any graduate education (23% vs. 42%), and average number of study cases (16.8 vs. 18.5). These differences across both parents and home visitors were accounted for in the analysis by a technique called propensity score weighting.

Key Evaluation Findings

The study assessed time from study enrollment until a subsequent maltreatment event as determined by a report in the CPS system. There was a significantly lower risk of a subsequent CPS report among families in the intervention group compared with families in the control group. Specifically, for every 1,000 cases treated, SafeCare would prevent between 64 and 104 recurrences of reported maltreatment in the first year compared to the standard CPS intervention.

Probable Implementers

Child Protective Services and home visiting agencies

Funding

Parents often receive SafeCare services through a referral from their Child Protective Services worker. However, SafeCare has been implemented in public health organizations, as well as a prevention model by several organizations.

- 440 -

Implementation Detail

Program Design

SafeCare is a home visiting intervention through which parents of children ages 0-5 years are educated on home safety and organization skills, child health and nutrition management, and parent- child interaction skills during weekly visits (Chaffin et al., 2012).

Staffing

SafeCare home visitors participate in a one-week workshop training followed by fieldwork support for certification determined by demonstrating fidelity of 85 percent or greater across several SafeCare sessions. After certification, they receive monthly fidelity monitoring and coaching.

Curriculum

Home visitors trained on the SafeCare program provide parenting education on health, home safety, parent-child interactions, and problem solving. Home visitors provide 18 to 20 weeks of trainings to parents via weekly or biweekly home visits, lasting one or two hours. SafeCare aims to train parents in such a way that their skills are generalizable across settings, time, and behaviors. More information is available about the SafeCare curriculum through the program's website: http://www.safecare.org

Issues to Consider

This program was given a "promising" rating. The randomized controlled evaluation of SafeCare conducted by Chaffin et al. (2012) took place within six CPS regions of one state. There were significant baseline differences between intervention and control parents and between intervention and control home visitors. These differences were accounted for in the analysis using propensity score matching. As implemented, this technique meets Promising Practices Network criteria for a "promising" program. Several other studies of SafeCare have been conducted (Damashek, Bard, and Hecht, 2012; Gershater-Molko, Lutzer, and Wesch, 2002; and Gershater-Molko, Lutzer, and Wesch, 2003). Only the study presented here, Chaffin et al. (2012), meets Promising Practices Network evidence criteria, including study design, effect size, and statistical significance.

Example Sites

Oklahoma Child Protective Services Georgia Department of Human Services Parent Aid Arizona Riverside County Public Health Safe Kids California Project

Contact Information

John Lutzker, Ph.D. Director Center for Healthy Development Georgia State University P.O. Box 3995 Atlanta, Georgia 30302 jlutzker@gsu.edu Daniel Whitaker, Ph.D. Director

- 441 -

http://www.safecare.org/

http://www.promisingpractices.net/criteria.asp

mailto:jlutzker@gsu.edu

National SafeCare Training and Research Center Center for Healthy Development Georgia State University P.O. Box 3995 Atlanta, Georgia 30302 dwhitaker@gsu.edu

Available Resources

More information about SafeCare is available through the SafeCare website: http://www.safecare.org

Bibliography

Chaffin, Mark, Debra Hecht, David Bard, Jane F. Silovsky, and William Howard Beaseley, "A Statewide Trial of the SafeCare Home-Based Services Model with Parents in Child Protective Services," Pediatrics, Vol. 129, No. 3, 2012, pp. 509-515. Gershater-Molko, Ronit M., John R. Lutzker, and David Wesch, "Project SafeCare: Improving Health, Safety, and Parenting Skills in Families Reported for, and At-Risk for Child Maltreatment," Journal of Family Violence, Vol. 18, No. 6, December 2003, pp. 377-386. Gershater-Molko, Ronit M., John R. Lutzker, and David Wesch, "Using Recidivism Data to Evaluate Project SafeCare: Teaching Bonding, Safety, and Health Care Skills to Parents," Child Maltreatment, Vol. 7, No. 3, August 2002, pp. 277-285. John R. Lutzker and Kathryn M. Bigelow, Reducing Child Maltreatment: A Guidebook for Parent Services, New York: Guilford Press, 2002. Programs to Treat Child Abuse and Neglect," Child Maltreatment, Vol. 17, No. 1, February 2012, pp. 56-66. Silovsky, Jane F., David Bard, Mark Chaffin, Debra Hecht, Lorena Burris, Arthur Owora, Lana Beasley, Debbie Doughty, and John Lutzker, "Prevention of Child Maltreatment in High-Risk Rural Families: A Randomized Clinical Trial with Child Welfare Outcomes," Children and Youth Services Review, Vol. 33, 2011, pp. 1435-1444.

Last Reviewed

June 2013

Safe Child Program

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children not experiencing physical, psychological or emotional abuse

- 442 -

mailto:dwhitaker@gsu.edu

http://www.safecare.org/

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Child Care/Preschool Elementary School Type of Service Youth Development Type of Outcome Addressed Child Abuse and Neglect

Evidence Level Promising

Program Overview

Developed in 1981 and updated in 1994, the Safe Child Program provides children ages three to ten with a set of skills to help them prevent sexual, emotional, and physical abuse. The main idea guiding the program is that there are times when children can and must be responsible for their own well being. At the same time, however, the program emphasizes that a child’s security can be enhanced without creating misunderstanding, fear, or anxiety or reducing a child’s sense of trust. The program is designed to be used in a group classroom setting. By building on everyday experiences, the program teaches children how to speak up for themselves, how to assess and handle various situations, and to know where to get help. Children actively engage in discussions, games, and role-playing in ten separate sessions, giving them time to practice these skills. The program has two main focuses. The first is prevention of sexual, emotional, or physical abuse by people known to the child. This is taught in the context of life skills, including communication, choices, asking for help, and decision-making. These skills are developed not as ideas, but as skills through role-play. The other focus of the program is safety around strangers. The classroom session discusses misconceptions the children may have, defines who a stranger is, and then provides four rules for dealing with strangers. These rules are these: keep an arm’s length away, don’t talk to strangers, don’t take anything from strangers, and don’t go anywhere with a stranger.

Program Participants

The program is designed for children in preschool through third grade. Children in the evaluations ranged in age from 3 to 10, attended a variety of rural, suburban, and urban schools, and came from diverse backgrounds.

Evaluation Methods

The program has been evaluated in two separate studies. An evaluation in Denver (Fryer, Kraizer, and Miyoshi, 1987a) involved 48 children in kindergarten through second grade, half randomly assigned to the treatment group and half to a control group. The program was administered over eight consecutive classes in 20-minute sessions. All the children were assessed in two simulations—a pretest simulation and a posttest simulation that was given after the program had been administered to the treatment group. In the simulation, the children individually encountered a researcher who played the role of a stranger. The researcher would meet the child in the hallway and ask the child to help bring in birthday treats from his car. Children “failed” if they agreed to help the stranger. If they agreed to help, the researcher then told the child he would need his or her help later. Someone from the school later informed the child that the stranger no longer needed assistance. After the simulation,

- 443 -

each child had a 30-minute meeting with a member of the research team to make sure they were not experiencing any fear or anxiety from the encounter.

The children were also assessed with three other standardized tests to complement the simulation. The Peabody Picture Vocabulary test measured cognitive ability, the Harter Perceived Competence Scale measured self-esteem, and the Children Need to Know Knowledge Attitude Test provided a measure of a child’s awareness and understanding of risk issues. Six months later the program was given to the control children and to the children in the treatment group who had failed the posttest simulation. The follow-up study (Fryer, Kraizer, and Miyoshi, 1987b) administered a third simulation to the 30 children who were still enrolled and present on the day of testing. A second evaluation (Kraizer Witte, and Fryer, 1989) examined 669 (494 in the treatment group and 175 in a control group of students with similar profiles) students from three states, ranging in age from 3 to 10 years old. The researchers used a scripted role-play to measure behavioral change. In the simulation, each child was asked to help the interviewer by responding to a series of “what if’ stories. The simulation then measured the child’s willingness to speak up about unwanted touching; assessed the child’s behavioral consistency in the face of emotional pressure; evaluated his or her willingness to tell others about the incident; [about the experience? about the touching incident?]; and willingness to keep a secret. Scoring was based on verbal response and body language. Children received one point every time they refused to go along with the examiner, for a total of 14 possible points.

Key Evaluation Findings

In the Denver evaluation, Fryer, Kraizer, and Miyoshi (1987a) found that:

• In the posttest simulation, 78.3 percent of the children in the treatment group passed compared with 52.4 percent of the children in the control group.

• Children who had initially higher self-esteem scores were more successful in the second simulation.

In the follow-up study, Fryer, Kraizer, and Miyoshi (1987b) found that:

• All the children in the treatment group who passed the posttest simulation also passed the third simulation six months later, showing retention of skills over time.

• A small percentage of children did not benefit from even repeated exposure to the program.

In the second evaluation, Kraizer, Witte, and Fryer (1989) found that:

• Prior exposure to other prevention materials made no significant difference in the children’s knowledge or attitudes or in the role-playing score on the pretest.

• Children in the treatment group showed significant gains after the program. Out of a possible score of 14, the three treatment groups saw average gains of 3.7, 3.9, and 4.3 points, compared with 0.5 and 0.3 for the control groups.

• The role-playing aspect appears to have achieved the desired effect of changing children’s actual behaviors, rather than a child’s self-esteem or knowledge of and attitudes toward risk and safety.

• Young children (kindergarten, first grade, and preschool) showed the greatest gains from the program.

• 95.5 percent of the children reported having no fear or anxiety after the completion of the program.

- 444 -

Probable Implementers

Elementary and preschools

Funding

The original evaluation was funded through a grant from the National Center on Child Abuse and Neglect.

Implementation Detail

Program Details

• A key component of the program is the use of role-playing, which allows children to practice the skills the program is trying to teach.

• The program’s goal is to provide only the information that is absolutely necessary to teach the prevention skills. Thus, the program does not provide explicit information about child abuse, does not define touching--as good or bad, and does not suggest that the adults that children know may abuse them.

• The program can effectively be used with young (preschool-aged) children.

Curriculum

The Safe Child Program is a videotaped curriculum delivered in ten different segments. The videotapes are used to teach the basic concepts and role-playing techniques. The videotapes are then followed up with actual role-playing, discussion, and other classroom activities.

Staffing

Classroom teachers typically present the program, although school counselors and staff from local child abuse prevention agencies may also teach the sessions. The professional training component of the program consists of an introduction to the program, an overview of the problems of child abuse, techniques for teaching the classroom program, instruction on how to recognize and report abuse, and information for implementation of the program.

Issues to Consider

This program received a “promising” rating. Neither the Fryer, Kraizer, and Miyoshi study nor the Kraizer, Witte, and Fryer study actually demonstrates a decrease in abuse. Rather, the evaluations show that this program is effective in changing children’s behavior in ways that should reduce the risk of child abuse from strangers. While the Denver evaluation used an experimental design and showed significant and positive effects, the sample was very small. The evaluation by Kraizer, Witte, and Fryer (1989) also used an experimental design with a much larger sample. Both evaluations emphasized that the positive results are due to the role-playing aspect of the program. The evaluations found that correct answers on the paper tests were not predictive of success in the simulations, nor was previous exposure to educational materials. The researchers concluded that knowledge of facts and even conceptual understanding are not enough to translate into behavioral changes. Rather, children need to practice the skills.

Example Sites

Denver, Colorado; Nashua, New Hampshire; Tulsa, Oklahoma; Pearl River, New York; Dade County, Florida; and sites in Canada, Europe, and Australia.

- 445 -

Contact Information

Sherryll Kraizer, Ph.D. Coalition for Children P.O. Box 6304 Denver, CO 80206 Fax: (303) 320-6328 Email: kraizer@safechild.org

Available Resources

The Safe Child Program package is available for purchase in English, Spanish, French, and Creole and includes a teacher’s guide, implementation guide, resource manual, and parents’ videotape. Also included are separate videotapes and teacher manuals for each grade level. A description of the program can be found at http://www.safechild.org/

Bibliography

Fryer, G. E., S. K. Kraizer, and T. Miyoshi, "Measuring Actual Reduction of Risk to Child Abuse: A New Approach," Child Abuse and Neglect Vol. 11, pp. 173-179, 1987a. Fryer, G. E., S. K. Kraizer, T. Miyoshi, "Measuring Children's Retention of Skills to Resis Stranger Abduction: Use of the Simulation," Child Abuse & Neglect Vol. 11, pp. 181-185, 1987b. Kraizer, S., S. S. Witte, and G. E. Fryer, "Child Sexual Abuse Prevention Programs: What Makes Them Effective in Protecting Children?," Children Today September-October 1989, pp. 23-27, 1989.

Last Reviewed

May 2009

Seattle Social Development Project

Program Info

Outcome Areas Healthy and Safe Children Children Succeeding in School

Indicators Youths not using alcohol, tobacco, or illegal drugs Students graduating from high school Youths abstaining from sexual activity or not engaging in risky sexual behavior Children and youth not engaging in violent behavior or displaying serious conduct problems Children not experiencing anxiety or mood disorders, such as depression

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12)

- 446 -

mailto:

http://www.safechild.org/

Type of Setting Elementary School Middle School Type of Service Health Education Instructional Support Parent Education Youth Development Type of Outcome Addressed Behavior Problems Cognitive Development/School Performance Juvenile Justice Mental Health Physical Health Substance Use and Dependence Teen Sex/Pregnancy Violent Behavior

Evidence Level Promising

Program Overview

The Seattle Social Development Project (SSDP) was a multi-year, school-based intervention that used a risk-reduction and skill-development strategy to improve outcomes for participating children and youths. The program was guided theoretically by the social development model, which hypothesizes that youths who are provided with opportunities for greater involvement with their schools and families, who develop the competency or skills they need for fuller participation with their schools and families, and for whom skillful participation is constantly reinforced, ultimately develop strong bonds with their families and schools. Further, the model proposes that these strong bonds set children on a positive developmental trajectory, resulting in more positive outcomes and fewer health-risk behaviors later in life. The SSDP was first implemented in 1981. It combined teacher, child, and parent components with the goal of enhancing children’s bonding with their families and schools. Teachers were trained in proactive classroom management, interactive teaching, and cooperative learning, while the students themselves were provided with direct instruction in interpersonal problem-solving skills and refusal skills to avoid problem behaviors. Parents were offered courses in child behavior management skills, academic support skills, and skills to reduce their children’s risk of drug use.

Program Participants

Program Participants initially included students in grades one through six in eight Seattle, Washington, public schools. Students were from mixed socioeconomic and ethnic backgrounds, and males and females were equally represented. The study was subsequently expanded to include fifth- and sixth- grade students in ten additional Seattle public schools.

Evaluation Methods

The SSDP has been implemented and evaluated in Seattle. Several evaluations of the program have been conducted that involve an ongoing longitudinal follow-up study of the original participants. The program was first implemented with a group of multiethnic urban students who entered the first grade in eight Seattle public schools in fall 1981. At that time, one whole school was assigned to be a control site, and one whole school was assigned to be an experimental site. In the remaining six schools, students entering the first grade and their teachers were randomly assigned to intervention or to control classrooms. Over the next three school years, in grades one through four, newly entering

- 447 -

students were randomly assigned to intervention or to control classrooms and were added to the study panel. Of the 520 students remaining in the eight schools at the end of second grade, 458 completed the post-test assessments (285 from the treatment group and 173 from the control group). In 1985, when the original subjects entered the fifth grade, the evaluation study was expanded to include all fifth-grade students in ten additional Seattle elementary schools. Schools added in the expansion group were matched to the original intervention schools with respect to the grades served and the percentage of students drawn from high-crime neighborhoods. Schools were then assigned to a treatment status nonrandomly. Beginning in fifth grade, the treatment groups participated in SSDP for two years. Therefore, the additions to the program resulted in three possible groups for analysis:

1. students participating in SSDP from the first to sixth grades (full intervention),

2. students participating in SSDP in the fifth and sixth grades only (late intervention),

3. students in the control group, which was a combination of the control group participants from 1981 and those that were added to the study in 1985.

Parents of 643 fifth-grade students consented to participate in the longitudinal study. Of those students, 156 were in the full-intervention group, 267 were in the late-intervention group, and 220 were in the control group. (A number of students were also assigned to a parent-training-only group; however, results for this subgroup have not been assessed and are not reported here). Both treatment-group and control-group SSDP teachers were observed periodically throughout the year to evaluate classroom performance and to determine the level and integrity of SSDP implementation. An early study of SSDP by Hawkins, von Cleve, and Catalano (1991) assessed students in the original eight participating schools in the spring of the second grade. The sample included 285 treatment and 173 control students in 21 classes, representing an 88 percent participation rate. Parents of 122 (43 percent) of the 285 treatment-group subjects attended one or more parent-training classes. Comparisons between treatment and control groups indicated that there were no significant differences between the groups on demographic variables. However, pretest data indicated that control-group subjects were less at risk than were treatment-group subjects at baseline on eight of 28 self-reported items. Significant baseline differences favoring the control group were found on measures of students’ attachment to school, family communication, family supervision, and antisocial orientations. Treatment-group and control-group outcomes were derived from a teacher-reported assessment of the Achenbach Child Behavior Checklist (CBCL). The checklist included measures on behavior problems, ten measures for boys and ten for girls, including external antisocial measures (e.g., levels of aggression and inattentiveness) and internal antisocial measures (e.g., anxiety levels and extent of social withdrawal). Hawkins et al. (1992) reported on data collected from 919 students during fall 1985 when normally progressing students entered the fifth grade. The intervention group consisted of students exposed to at least one semester of the full intervention in grades one through four (199 students). The control group consisted of students enrolled in control classrooms during grades one through four plus unexposed students who were added to the project in the fall of fifth grade when the study was expanded (709 students). Fifty-two percent of the student sample was male, 47 percent of the sample was Caucasian, 24 percent was African-American, 21 percent was Asian American, and 8 percent was of another racial/ethnic background. Thirty-eight percent of the students qualified for the National School Lunch Program in the fall of fifth grade. No significant differences were found between intervention and control students on these demographic variables. In addition, an analysis compared the initial control group with the additional available control students on each of the descriptive and outcome variables and found no significant overall differences. The outcomes assessed included family involvement and interaction, academic outcomes, alcohol use, and delinquency. Another early study of SSDP was conducted by O’Donnell et al. (1995). It examined the impact of the full intervention (grades one through six) on a high-risk, low-income subsample of SSDP students. Students were identified as high risk based on their participation in the National School Lunch/School

- 448 -

Breakfast Program during the fall of fifth grade. Intervention and control groups were matched on ethnicity, resulting in samples of 102 students in the intervention group and 75 students in the control group. Data were collected from students during fall 1985, when normally progressing students entered the fifth grade, and in spring 1987, when they completed the sixth grade. Students were administered a self-report survey, which measured peer interactions and risk-related behaviors. In addition, teachers completed the CBCL for each student, and student grades and outcomes on the California Achievement Test (CAT), a standardized test in math, reading, and language arts, were collected. At the completion of sixth grade, 43 percent of the intervention group (44 students) and 83 percent of the control group (62 students) completed the evaluation measures. Fifty-four percent of the sample was female, 42 percent was African-American, 24 percent was Caucasian, 25 percent was Asian American, and 9 percent was of another race/ethnicity. The intervention group had a higher percentage of female students, but there were no significant differences between groups in terms of race/ethnicity. A study by Abbott et al. (1998) involved all consenting fifth-grade students in the 18 schools (808 students total). Fifty-one percent of the sample was male, 46 percent was Caucasian, 24 percent was African-American, 21 percent was Asian-American, and 9 percent was of another ethnicity. Fifty-two percent of the students were classified as low income on the basis of eligibility for the National School Lunch/School Breakfast Program in grades five, six, or seven, and 42 percent of the students lived in single-parent households. Analyses investigated the effects of the combined intervention groups (both full and late) on academic achievement and school bonding at the end of sixth grade. The next evaluation of the effects of SSDP was published in 1999 by Hawkins and colleagues. Ninety- three percent of the original 643 students who consented to participate in the longitudinal study were interviewed at age 18. The study assessed the impact of the full intervention (156 students) and the late intervention (267 students) compared with the no-treatment control group (220 students). Rates of dropping out of the study were approximately the same across all three treatment conditions. The sample included approximately equal proportions of males and females, and more than 56 percent of the students had participated in their schools’ free-lunch program in grades five, six, and/or seven. Forty-five percent of the sample was Caucasian, 26 percent was African-American, 22 percent was Asian American, and 8 percent was of another race/ethnicity. There were no significant racial/ethnic, gender, or free-lunch eligibility differences among the full-intervention, late-intervention, and control populations. In 1993, participants were interviewed to gather information on a variety of school- related variables and health-risk behaviors. Standardized achievement test scores, grade-point averages, and school disciplinary records through age 17 were collected, as were delinquency charges from juvenile court. An age-21 follow-up study of sexual behavior was conducted by Lonczak et al. (2002). Comparisons between the full-intervention group (144 students) and the control group (205 students), representing 93 percent of the available sample, were assessed. Outcomes for the late-intervention group and the parent-training-only group were not assessed in this study. Fifty-one percent of the sample was male, 47 percent was Caucasian, 26 percent was African-American, 21 percent was Asian-American, and 7 percent was of another race/ethnicity. Fifty-five percent of the sample had been eligible for the free lunch program between grades five and seven. In regard to marital status, 75 percent of respondents were single, 9 percent were married, 15 percent were living with a partner, and 2 percent were separated or divorced. Analyses of possible differences between students who dropped out of the program and those who remained in the program found no significant differences with regard to any of the demographic variables. Outcome measures included sexual activity, condom use, age of sexual onset, number of sexual partners, history of sexually transmitted diseases (STDs) other than HIV, history of pregnancy, and history of having a baby. Finally, Hawkins et al. (2005) analyzed additional outcomes at age 21 for a sample of 605 respondents (94 percent of the longitudinal study population of 643). The authors compared the full-intervention (144 students), the late-intervention (256 students), and the no-treatment (205 students) groups. Fifty percent of the sample was female, 45 percent was Caucasian, 25 percent was African-American, 22 percent was Asian American, 6 percent was Native-American, and 3 percent was of another race/ethnicity. As children, 56 percent of participants were eligible for the school free lunch program

- 449 -

at some point in grade five, six, or seven. Outcomes assessed included high school graduation rates, generalized anxiety, depression, suicidal thoughts, self-reported crime, arrests, drug selling, court charges, and substance use.

Key Evaluation Findings

At the end of the second grade, the study by Hawkins, von Cleve, and Catalano (1991) found the following for male students:

• The control-group students exhibited more externalizing antisocial behaviors, particularly aggressiveness, than did the SSDP group.

• No significant differences were found between groups for anxious, social withdrawal, unpopular, self-destructive, obsessive-compulsive, inattentive, nervous-overactive, or internalizing antisocial behavior measures.

• When outcomes for African-American boys were examined separately, no significant differences were found for them on any of the behavior measures. However, when outcomes for Caucasian boys (including 59 students in the treatment group and 40 in the control group) were examined separately, control subjects scored significantly higher (more poorly) on the aggressive and externalizing antisocial behavior measures than did the treatment-group students.

For female students, the authors found the following:

• Control-group students were rated as significantly more self-destructive than treatment-group students.

• No significant differences were found between groups on the anxious, social withdrawal, unpopular, depressive, inattentive, nervous-overactive, aggressive, externalizing, or internalizing behavior measures.

• Similar to the findings for males, when outcomes for African-American girls were examined separately, no significant differences were found between the treatment and control groups. For Caucasian girls (including 66 treatment and 41 control students), control students scored significantly higher on the self-destructive behavior measure than did treatment subjects. In addition, marginally significant outcomes were found for the depressive and nervous- overactive behavior measures.

Overall, the racial differences for both male and female students suggest that the intervention was effective in improving some outcomes for Caucasians but not for African-Americans. After four years of intervention, Hawkins et al. (1992) found the following:

• Intervention students reported significantly lower rates of alcohol initiation than did control students (21 percent versus 27 percent).

• Intervention students reported significantly less delinquency initiation than did control-group students (46 percent versus 52 percent).

• Female control students scored significantly higher than intervention students on the CAT, but this difference faded by sixth grade.

The study by O’Donnell et al. (1995) of low-income students reported the following findings at the completion of sixth grade:

• Compared with control-group boys, treatment-group boys

o were rated more socially competent by teachers.

o had significantly higher grades and scored significantly higher on the CAT.

- 450 -

o were not rated any differently on peer reports of antisocial behavior, were no less likely to run away from home, and were no less likely to have tried alcohol, cigarettes, or marijuana.

• Compared with their control counterparts, treatment-group girls

o were significantly less likely to have smoked cigarettes (7 percent versus 36 percent).

o were marginally significantly less likely to have tried alcohol (19 percent versus 39 percent) or marijuana (4 percent versus 17 percent).

o did not have significantly different scores on the CAT, did not show differences in their academic grades, were not rated any differently on peer or teacher reports of antisocial behavior, and were no less likely to run away from home.

The study by Abbott et al. (1998) found the following at the end of sixth grade:

• Students in the intervention classrooms had significantly higher CAT scores than did students in control classrooms.

Research by Hawkins, et al. (1999) found the following at the age-18 follow-up:

• Compared with control-group participants, full-intervention (grades one through six) participants

o scored marginally higher in overall grade point average (2.42 versus 2.18).

o were significantly less likely to have repeated a grade (14 percent versus 23 percent).

o were marginally less likely to have an official high school disciplinary action report (46 percent versus 58 percent).

o were significantly less involved in school misbehavior, such as skipping class, cheating, or being removed from the classroom (3 percent versus 5 percent).

o were significantly less likely to have committed violent delinquent acts (48 percent versus 60 percent).

o were significantly less likely to have engaged in drinking ten or more times in the year prior to assessment (15 percent versus 25 percent).

o were significantly less likely to have engaged in sexual intercourse (72 percent versus 83 percent) or to have had multiple sexual partners (50 percent versus 62 percent).

o were significantly less likely to have been pregnant or to have caused a pregnancy (17 percent versus 26 percent).

• There were no significant differences between groups in their scores on the California Achievement Test, school dropout rates, or levels of suspension or expulsion.

• There were no significant differences between groups for lifetime rates of nonviolent delinquency, arrests, court charges, use of cigarettes, use of alcohol, use of marijuana, use of other drugs, or drinking and driving.

• Exposure to the late intervention (fifth and sixth grades only) resulted in only two impacts on long-term outcomes. Compared with control-group participants, late-intervention participants were

o significantly less likely to have engaged in school misbehavior (i.e., cheating, truancy, being removed from class for misbehavior).

o less likely to report sexual activity, although this was only marginally significant.

Lonczak et al.’s (2002) age-21 follow-up study of sexual behavior in the full-intervention group reported the following:

- 451 -

• A small but statistically significant effect was found for the average age at which those in the intervention group reported having had their first sexual experience versus the average age of those in the control group (16.3 years versus 15.8 years). A statistical analysis found that the program had a marginally significant effect in reducing the overall relative risk of engaging in sexual intercourse for the first time before age 21.

• No significant effects were found for the full sample for frequency of past-year condom use among single individuals.

• On average, young adults who had participated in the intervention group reported significantly fewer sexual partners in their lifetimes than did those in the control group. The difference between the intervention and control groups was especially pronounced for those reporting six or more partners (32 percent versus 43 percent).

• Participants in the full-intervention group were significantly more likely to report condom use during the most recent sexual intercourse than were those in the control group (60 percent versus 44 percent). There was no significant effect from the intervention on condom use during the first sexual intercourse.

• There was no significant overall treatment effect for diagnosis of STDs.

• Females in the intervention group were significantly less likely to become pregnant (38 percent versus 56 percent) and were significantly less likely to have had a baby (23 percent versus 40 percent) by age 21 than were females in the control group. The proportion of males in each group who reported causing a pregnancy or birth did not differ significantly.

• After controlling for the effects of poverty, statistically significant program effects were found for African-Americans.

o The difference in condom-use frequency between the intervention group and the control group was significantly greater for single African-Americans. Fifty percent of those in the intervention group reported always using a condom, compared with 12 percent of those in the control group.

o The difference in most-recent condom use between the intervention and control group was significantly greater for single African-Americans, with 79 percent of intervention-group participants and 36 percent of the control-group participants reporting condom use during the last intercourse.

o Among African-Americans, 7 percent of intervention-group participants and 34 percent of control-group participants reported being diagnosed with an STD over their lifetimes.

Finally, the age-21 follow-up by Hawkins et al. (2005) reported the following:

• Full-intervention participants were significantly more likely to have graduated from high school than were control-group participants (91 percent versus 81 percent). No significant differences were found between the late-intervention group and the control group on high school graduation rates.

• Both the full-intervention and late-intervention participants reported significantly fewer thoughts of suicide than did the control-group participants.

• Effects of the full intervention on reducing symptoms of depression were marginally significant when comparing full-intervention with control-group participants. No significant differences in symptoms of depression were found between the late-intervention group and the control group.

• Full-intervention participants reported significantly fewer symptoms of social phobia in the past year, but no significant differences were found between groups for anxiety.

• Both full-intervention participants and late-intervention participants were significantly less likely than those in the control group to have sold illegal drugs in the past year.

- 452 -

• Full-intervention participants were significantly less likely than those in the control group to have had a court charge in their lifetimes. No significant differences were found between the late-intervention group and the control group in the likelihood of having had a court charge.

• Full-intervention participants were marginally less likely than were control participants to have used alcohol, tobacco, or illicit drugs in the past month or year. No significant differences were found between the late-intervention group and the control group in the likelihood of using these substances in the past month or year.

• No significant effects were found among any of the groups for past-year crime, past-year arrests, or past-year court charges.

A few gender differences in program effects were also found at age 21, with additional effects found for women but not for men.

• Women who had participated in the full intervention had significantly fewer generalized anxiety symptoms than did women in the control group, while women in the late-intervention group reported marginally fewer generalized anxiety symptoms than women in the control group.

• Although the prevalence of having been arrested was low in both groups, women in the full- intervention group were significantly more likely to have been arrested in the past year than were women in the control group (7 percent versus 1 percent).

Probable Implementers

Public or private elementary schools

Funding

The SSDP was funded by the National Institute on Drug Abuse (Prevention Research Branch), the Office of Juvenile Justice and Delinquency Prevention, the Robert Wood Johnson Foundation, and the Burlington Northern Foundation.

Implementation Detail

Program Design/Curriculum

Classroom teachers were trained in instructional methods with three major components: proactive classroom management, interactive teaching, and cooperative learning.

• Proactive classroom management is a technique through which teachers engage in positive management practices before problem behaviors, such as academic failure and conduct disorders, begin to develop. Teachers were trained to establish classroom routines at the beginning of the year to create a consistent pattern of expectations for students.

• SSDP teachers also used interactive teaching methods, which require mastery of specified learning objectives before proceeding on to new material, grading based on improvement over past performance, and frequent monitoring of students to assess progress.

• Finally, teachers used cooperative learning methods, which involve having small groups of students with differing abilities and backgrounds work together on curriculum material.

In addition, students received specific cognitive- and social-skills instruction during the first and sixth grades.

• During the first grade, students participated in the "Interpersonal Problem-Solving" curriculum, a program teaching communication skills, decision-making, negotiation, and conflict resolution.

- 453 -

• Sixth-grade students received a four-hour refusal-skills training program. The program’s goals were to help students recognize and resist social influences that may encourage problem behaviors (e.g., drug use or sex) and to develop positive alternatives to engaging in those negative behaviors.

Parents of children in SSDP were encouraged to participate in educational workshops provided by SSDP staff. Three different parent classes, offered during the first, second, third, fifth, and sixth grades, corresponded to the developmental level of the child. Workshops were designed to increase the level of a child’s bonding to his or her family by targeting risk factors, such as poor and inconsistent family management, family conflict, low commitment to school, norms favorable to substance use, and association with antisocial peers.

• Parents of first- and second-grade students were able to participate in the seven-session "Catch ‘Em Being Good" curriculum. The curriculum teaches parents to effectively convey expectations for behavior, to identify and reinforce desirable behavior, and to address what would be appropriate responses to negative behavior.

• During the students’ third-grade year, parents could enroll in "How to Help Your Child Succeed in School," a four-session curriculum designed to improve parent-child communication and increase parental involvement in a child’s academic activities.

• The final component of parent education was offered during the fifth and sixth grades. "Preparing for the Drug (Free) Years" is a five-session, risk-focused, skills-training curriculum to help parents establish family policies, enhance family communication, and teach their children resistance skills.

Staffing

The classroom program was implemented by regular classroom teachers. Five days of teacher training were sequenced throughout the school year, and teachers were regularly observed by SSDP staff and the school principal, who provided feedback on the use of the program’s techniques. In addition, teachers of first- and sixth-grade students received training in the specific curricula provided to students in those grades.

Issues to Consider

This program received a "promising" rating. The initial phase of the research involved random assignment of teachers and students in six schools to treatment or control groups and non-random assignment of two schools (at the whole-school level) to treatment or control conditions. In the second phase of the study, a quasi-experimental design was used, wherein ten additional schools were assigned to treatment or control groups non-randomly. The impact of this program was evaluated at multiple time periods, including evaluations while the intervention was ongoing (at ages 7, 8, and 10), soon after the end of the intervention (age 13), and at six- and nine-year follow-ups (ages 18 and 21). Each evaluation period yielded significantly positive results, although the findings were not always consistent among time periods, nor were they significant for all measures. Findings sometimes differed by gender or race. There is suggestive evidence that the "full intervention," which treated children from first through sixth grade, may be more effective than the "late intervention," which takes place only in fifth and sixth grade. The majority of research on this program is based on the quasi-experimental portion of the study. Several evaluation reports have examined whether there is evidence of bias due to the non-random group assignment of schools. These found similar observable characteristics of the treatment and control groups (gender, ethnicity, childhood poverty, residential stability, etc.). For a discussion of potential threats to validity due to the quasi-experimental design, see Hawkins and Catalano (1995). Some consideration should be given to the impact of attrition (dropouts from the study). There was a high rate of attrition in the O’Donnell et al. (1995) study, with only 60 percent of the original sample included in the evaluation at the completion of the sixth grade. In addition, in the O’Donnell analysis of sixth-grade data, the intervention group had nearly twice the attrition level of the control population. Attrition analysis indicated that students for whom sixth-grade data were unavailable were

- 454 -

less attached to their peers, had more suspensions from school, and were less committed to school. However, with one exception (girls’ opportunities for antisocial behaviors, such as delinquency and drug use), there was no evidence based on observable characteristics of differential attrition among the study groups. In other words, those students who dropped out of the intervention and control groups differed little on baseline characteristics. Another issue with regard to attrition is the response rate for the age-21 follow-up studies, which analyzed outcomes for 144 full-intervention youths. There has been debate in the literature about the appropriate baseline denominators to use for attrition analysis. A 93 percent follow-up rate was calculated using as a baseline the 156 fifth-graders who had parental consent to participate in the follow-up study. However, Gorman (2002) suggests that because involvement of these subjects in the study began four years earlier, in 1981, the correct follow-up rate for the full-intervention group at age 21 years is no more than 50 percent (using a baseline of 285 second grade respondents). Hawkins and Catalano (2005), in reply to Gorman, argue that the sample for the longitudinal quasi- experimental study was constituted in grade five and takes advantage of including those students from the earlier experimental study who remained in project schools. Two studies assessed the effects of the intervention on "school bonding," that is, the development of a positive emotional link and personal commitment to school (Abbott et al., 1998; Hawkins et al., 2001). Previous research has shown that school bonding can be a protective factor against such negative outcomes as school dropout, delinquency, violence, and drug abuse and can be a mediating factor in improving such outcomes as academic achievement (e.g., see Hirschi, 1969; Catalano and Hawkins, 1996). While Abbott and colleagues did not find any significant differences among the groups, Hawkins et al. (2001) found that by age 18, the level of bonding to school in the full- intervention group was significantly higher than in the control group. This result remained significant after controlling for gender, ethnicity, poverty, and earlier academic achievement. No significant differences were found between the late-intervention group and the control group, suggesting that positive impacts on school bonding are attained only by program implementation prior to fifth grade. Although the parental component was considered to be an integral and vital part of the program and one of the foundations on which the theoretical model was built, parent participation in the program was not a mandatory component. Research by Hawkins et al. (1999) showed that a relatively low percentage (43 percent) of parents attended the parenting workshops. No measure was in place to determine how well skills and information were disseminated in the parental workshops, or if those skills and information were used in the home. However, Hawkins et al. (1992) found that children in the full-intervention condition reported significantly better family management and family communication, as well as more family involvement and attachment to family, than did control-group children. While this finding suggests that the parenting component may have a positive impact, the evaluations do not indicate the extent to which the parent component may have played a part in generating the impact that was observed. Finally, when weighing the program evaluation outcomes, it is important to keep in mind that the program has been studied in only one urban metropolitan area, thus limiting the applicability of the findings to other populations. In addition, all evaluations of the program to date have been conducted by the program’s designers rather than outside evaluators.

Example Sites

Seattle and Edmonds, Washington

Contact Information

For information on the Seattle Social Development Project, contact: Karl G. Hill, PhD Project Director, Seattle Social Development Project Social Development Research Group

- 455 -

University of Washington 9725 3rd Avenue NE, Suite 401 Seattle, WA 98115-2024 Phone: (206) 685-3859 Fax: (206) 543-4506 E-mail: khill@u.washington.edu Web site: www.depts.washington.edu/ssdp The Social Development Research Group (SDRG) provides guides for the teacher training components of the SSDP intervention as well as trainers for these components. Two of the parent programs included in SSDP, now called Preparing for School Success (grades one through three) and Guiding Good Choices (grades 5 and 6), are available from the Channing Bete Company, www.channing- bete.com. Contacts for trainers for these components are provided by SDRG.

Available Resources

The Social Development Research Group summarizes the Seattle Social Development Project at http://depts.washington.edu/ssdp/.

Bibliography

Abbott, Robert D., Julie O'Donnell, J. David Hawkins, Karl G. Hill, Richard Kosterman, and Richard F. Catalano, "Changing Teaching Practices to Promote Achievement and Bonding to School," American Journal of Orthopsychiatry, Vol. 68, No. 4, 1998, pp. 542-552. Catalano, Richard F., and J. David Hawkins, "The Social Development Model: A Theory of Antisocial Behavior," in J. David Hawkins, ed., Delinquency and Crime: Current Theories, Cambridge, England: University Press, 1996. Gorman, Dennis M., "Overstating the Behavioral Effects of the Seattle Social Development Project," Archives of Pediatric and Adolescent Medicine, Vol. 156, 2002, pp. 155-156. Hawkins, J. David, and Richard F. Catalano, "Doing Prevention Science: A Response to Dennis M. Gorman and a Brief History of the Quasi-Experimental Study Nested Within the Seattle Social Development Project," Journal of Experimental Criminology, Vol. 1, 2005, pp. 79-86. Hawkins, J. David, Elizabeth von Cleve, and Richard F. Catalano, "Reducing Early Childhood Aggression: Results of a Primary Prevention Program," Journal of the American Academy of Child & Adolescent Psychiatry, Vol. 30, No. 2, 1991, pp. 208-217. Hawkins, J. David, Jie Guo, Karl G. Hill, Sara Battin-Pearson, and Robert D. Abbott, "Long-Term Effects of the Seattle Social Development Intervention on School Bonding Trajectories," Applied Developmental Science: Special Issue: Prevention as Altering the Course of Development, Vol. 5, No. 4, 2001, pp. 225-236. Hawkins, J. David, Richard F. Catalano, Dianne M. Morrison, Julie O'Donnell, Robert D. Abbott, and L. Edward Day, "The Seattle Social Development Project: Effects of the First Four Years on Protective Factors and Problem Behaviors," in Joan McCord and Richard E. Tremblay, eds., Preventing Antisocial Behavior: Interventions from Birth Through Adolescence, New York, N.Y.: Guilford Press, 1992, pp. 139-161. Hawkins, J. David, Richard F. Catalano, Rick Kosterman, Robert Abbott, and Karl G. Hill, "Preventing Adolescent Health-Risk Behaviors by Strengthening Protection During Childhood," Archives of Pediatric Medicine, Vol. 153, 1999, pp. 226-234. Hawkins, J. David, Rick Kosterman, Richard F. Catalano, Karl G. Hill, and Robert D. Abbott, "Promoting Positive Adult Functioning Through Social Development Intervention in Childhood: Long-Term Effects

- 456 -

mailto:khill@u.washington.edu

http://www.depts.washington.edu/ssdp

http://www.channing-bete.com/

http://depts.washington.edu/ssdp/

from the Seattle Social Development Project," Archives of Pediatrics & Adolescent Medicine, Vol. 159, No. 1, 2005, pp. 25-31.

Hirschi, Travis, Causes of Delinquency, Berkeley, Calif.: University of California Press, 1969. Lonczak, Heather S., Robert D. Abbott, J. David Hawkins, Rick Kosterman, and Richard F. Catalano, "Effects of the Seattle Social Development Project on Sexual Behavior, Pregnancy, Birth, and Sexually Transmitted Disease Outcomes by Age 21 Years," Archives of Pediatrics & Adolescent Medicine, Vol. 156, No. 5, 2002, pp. 438-447. O'Donnell, Julie, J. David Hawkins, Richard F. Catalano, Robert D. Abbott, and Edward Day, "Preventing School Failure, Drug Use, and Delinquency Among Low-Income Children: Long-Term Intervention in Elementary Schools," American Journal of Orthopsychiatry, Vol. 65, No. 1, 1995, pp. 87-100.

Last Reviewed

August 2006

Second Step Violence Prevention

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children and youth not engaging in violent behavior or displaying serious conduct problems

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Child Care/Preschool Elementary School Middle School Type of Service Parent Education Youth Development Type of Outcome Addressed Behavior Problems Violent Behavior

Evidence Level Promising

Program Overview

Second Step: A Violence Prevention Program is a classroom-based social skills curriculum for students from preschool through middle school. The curriculum aims to reduce impulsive and aggressive

- 457 -

behaviors and increase protective factors and social-emotional competence. Organized by grade level, the program teaches children empathy, problem-solving skills, risk assessment, decision-making, and goal-setting skills. The Second Step program is classified as a universal intervention, meaning that it is appropriate for whole classrooms of children and not just those at risk. Second Step lessons are organized into three skill-building units that focus on empathy, impulse control and problem solving, and anger management. Lessons are sequential, developmentally appropriate, and provide opportunities for modeling, practice, and skills reinforcement. The curriculum includes discussion, teacher modeling, coaching skills, and role-plays. Stories are used to demonstrate important peer-relations skills and to teach affective (emotional), cognitive, and behavioral social skills. Lessons can be incorporated into health, science, math, social studies, and language arts. The Second Step Family Guide is available as a supplement to the program for preschool-grade 5. The six-session, facilitator-led program helps parents learn about the curriculum and assists them in reinforcing children’s skills in communicating feelings, solving problems, controlling anger, and dealing with conflict. Segundo Paso, a Spanish translation supplement of the Second Step curriculum, is available for preschool through grade five classroom kits. A Spanish translation of the Second Step Family Guide is also available.

Program Participants

Children in preschool through middle school (4–14 years of age)

Evaluation Methods

Orpinas et al. (1995) evaluated the effects of Second Step in a sample of 223 sixth graders from ten classes in four middle schools in a large urban school district in Texas. The school principals were asked to choose "good and interested" teachers to teach the curriculum, as well as a control class that was similar to the intervention classes. Four of the classrooms implemented the Second Step program two to three times a week, and six of the classrooms served as the control group. In two of the four intervention schools, three classes were assigned nonrandomly to one of the following conditions: (1) Second Step administered by the teacher, (2) Second Step administered by the teacher with assistance from trained peer leaders (leaders were from the same classroom and were nominated by peers), or (3) control group. The other two intervention schools each nonrandomly assigned one class to teacher-implemented Second Step and one class to the control group. Outcomes were measured one week before and one week after implementation of the curriculum, as well as three months after its completion. Of the 223 respondents who completed all three surveys, 64 percent were Hispanic, 17 percent were African American, and 18 percent were Caucasian. Fifty percent of the students were eligible for free or reduced-price lunch. At baseline, no significant differences were found between intervention and control groups on the researcher-developed Aggression Scale, a measure of common aggressive behaviors, during the week prior to the survey. Grossman and colleagues (1997) conducted a randomized controlled trial involving 1,100 students in six pairs of matched schools. The study was conducted in a total of 49 second- and third-grade classrooms in 12 elementary schools from four school districts in King County, Washington. Participating schools were matched based on their school district, the percentage of students receiving free or reduced-cost school lunches, and the proportion of minority students. After matching, schools in each pair were randomly assigned to control or treatment status. A total of 418 intervention and 372 control students (66 percent of the 1,100 eligible students) obtained parental consent and completed all parent, teacher, and observer evaluation measures. These students were mostly similar at baseline; however, a somewhat larger proportion of control group students were identified as receiving special education services and were African American, while a higher proportion of intervention group students were Asian American. For the purposes of the evaluation, the Second Step program was implemented by classroom teachers over a 16- to 20-week period. Twice during the intervention period, classrooms were observed to determine the quality and fidelity of program implementation. Outcome data were collected before the start of the curriculum, two weeks following

- 458 -

the conclusion of the program, and at follow-up six months after completion of the program. Outcomes included parent reports of child behavior problems and positive and negative social adjustment; teacher reports of child behavior problems and social competence and aggressive/antisocial behavior; and classroom and playground observations of social interactions between the child and other students or the teacher. In addition, 12 children from each study classroom (for a total of 588 students) were randomly selected to be part of an "intense observation" subsample. Children included in the observation subsample were observed in classroom, cafeteria, and playground settings for approximately one hour each at baseline, at two weeks after program completion, and at six months following program completion. A dissertation by Lillenstein (2001) assessed Second Step in a sample of 285 kindergarten, first, and second grade students from 15 classrooms from four elementary schools in York County, Pennsylvania. The majority of students in the participating schools were white, Catholic, middle to upper class, and from two-parent households. A total of 184 students were nonrandomly assigned by the school principal to the Second Step program (nine classrooms) and 101 students to the waiting list control group (six classrooms). Second Step lessons lasted 15-30 minutes and were implemented once per week for six months. Of the 391 students in the participating classrooms, 367 obtained initial parental consent, and 285 parents (83 percent) completed both pretest and posttest outcome measures. Outcomes were assessed one week prior to and one week following implementation of the Second Step curriculum. Child outcomes were assessed via parent and teacher ratings of social skills and problem behavior, as well as classroom and lunchroom observations of aggressive behaviors and pro-social behaviors. A second dissertation assessed the effects of Second Step in a sample of 109 third grade students from six classes in two public schools in Corpus Christi, Texas (Nicolet, 2004). The study evaluated the effectiveness of a brief form of the Second Step curriculum administered by the school counselor, which included three of the five lessons from each of the standard topics. The two schools were randomly assigned to participate, with a total of 54 students in the Second Step group and 55 students in the control group. Attrition throughout the study was very minimal. The schools included low- to upper-middle-class students, with slightly less than half of the students eligible for free or reduced-price lunch. The groups were pretested one week prior to implementation of the curriculum, and again the week following program implementation. Outcome measures included teacher rating of student aggressive behavior and pro-social behavior, and student self-reported anger management. Frey et al. (2005) studied Second Step in a sample that involved a total of 1,989 children in grades two and four from 15 elementary schools in western Washington. The schools were located in urban districts of two moderate-sized cities, two proximal suburban districts, and one small city adjacent to a naval base. In the first year of the study, 11 schools were randomly assigned to the intervention (two- thirds of the sample) or to the control group (one-third of the sample). The other four schools were recruited the following year and were all assigned to the control group. None of the schools differed with respect to ethnic makeup or the proportion of students receiving free or reduced-price lunch. All participating schools, regardless of group assignment, received program materials, teacher training, and substitute teachers to fill in for teachers during regular teacher training; control schools received these benefits for classrooms that did not participate in the study. Teachers in the treatment group taught one or two program lessons per week. Of the 1,253 children with parental consent (63 percent of the initial sample), 462 (74 percent) in the treatment group and 436 (71 percent) in the control group completed all outcome measures over two years. No significant group differences in attrition were found. Outcomes were assessed via teacher ratings of students’ behavior at pretest and in April of each school year, and at baseline the intervention groups scored significantly higher on antisocial behavior and marginally lower on social competence than did control students.

Key Evaluation Findings

The study by Orpinas et al. (1995) found:

• When the full intervention and control groups were evaluated across the three measurement periods, no significant intervention effect was found on the Aggression Scale scores.

- 459 -

• Race was significantly associated with aggression scores among boys of only one school. Since almost 90 percent of the students at the school were Hispanic, the analysis was repeated only for Hispanic boys in this school.

o At posttest, results showed that boys from the six intervention classes had reduced their aggressive behavior in rates ranging from 4 to 51 percent more than the control group. This reduction was marginally significant in only one class with a sufficient sample size to meet Promising Practices Network criteria, with boys from the teacher-only group reducing their aggressive behavior by 23 percent.

• No statistically significant differences between the intervention and control groups were found for girls.

Grossman et al. (1997) found the following:

• No significant differences were found between intervention and control groups for any of the teacher or parent reports.

• Observer ratings found:

o At posttest, there was a marginally significant difference between the intervention and control groups in physical negative and overall negative behaviors in the classroom. Rates of negative behavior decreased from baseline to posttest in the intervention group but increased in the control group.

o At posttest, there was a significant difference between groups in observed instances of negative physical behavior in the cafeteria or playground, with intervention group rates decreasing from 2.2 to 1.6 episodes per child-observation hour and control group rates increasing from 1.8 to 2.6 episodes per child-observation hour. At posttest, there were no significant differences between the intervention and control groups in instances of negative verbal behavior in the cafeteria or playground setting.

o At posttest, the rate of observed neutral or prosocial behavior in the cafeteria or playground setting increased by 17.1 more episodes per child-observation hour in the intervention group than in the control group.

o At the six-month follow-up, most of the significant differences between the intervention and control groups had dissipated because of a decline in negative behavior in the control group. The exception was physical aggression in the classroom setting, which was significantly lower in the intervention schools than in the control schools.

The study by Lillenstein (2001) reported the following:

• A marginally significant difference was found between groups for the teacher-rated total problem behavior score, with the control group exhibiting significantly more problem behaviors than the intervention group.

• No significant differences were found between groups for either the parent-rated or teacher- rated social skills scores, parent ratings of problem behaviors, or classroom observations of pro-social behaviors or aggressive behaviors.

Nicolet’s (2004) study found:

• No significant differences between groups for aggressive behavior or anger management scores.

• No significant differences between groups for pro-social scores; however, results indicated that the intervention group’s scores increased significantly more from pretest to posttest than did the control group’s scores.

The study by Frey et al. (2005) reported:

- 460 -

• In the first year, significant group differences were found in teacher-reported social behavior.

o Among those children with high baseline ratings in antisocial behavior, the intervention group showed greater declines in antisocial behavior than the control group.

o Significant but smaller differences also favored the intervention group among students with low baseline scores. Intervention students with low baseline scores showed no change, whereas control students increased their rates of antisocial behavior.

• Positive effects faded by the second year, with no remaining group differences in antisocial behavior.

Probable Implementers

Public and private preschools, elementary schools, middle schools, after-school programs, and community-based child and youth organizations.

Funding

In the past, sites have funded the program through the Federal Safe and Drug Free Schools Program and through state violence prevention initiatives. Committee for Children maintains a list of possible funding sources on their website at www.cfchildren.org.

Implementation Detail

Program Design

The Second Step program is designed for use by classroom teachers and requires minimal teacher- preparation time. A program kit contains all needed materials including an Administrator’s Guide, a Teacher’s Guide, scripted lessons, and a classroom video. The preschool level includes puppets and a sing-along audio CD. The multiple levels of lessons allow students in Second Step schools to receive multiyear training in pro-social skills designed at a grade-appropriate level. Preschool and Elementary school program:

• Program materials include photo-lesson cards and videos depicting children expressing emotions in real-life situations. (Video is not used in the preschool kit.)

• Teachers follow the scripted lessons written on the back of each photo card to help students connect their own emotional experiences to those depicted on the cards and in the videos.

• Lesson cards include discussion, role-playing, and other activities, as well as suggestions for how teachers can model the skills taught in the lesson throughout the week.

• Younger students are encouraged to interact with the Impulsive Puppy and Slow-Down Snail puppets and the Be-Calm Bunny plush toy.

Middle school program:

• Lessons place a stronger emphasis on student attitudes and beliefs about aggression and provide skills for conflict resolution.

• Helps students understand their own social and emotional experiences; students are encouraged to relate them to those of a book character.

• The curriculum includes videos that stimulate discussion and role-playing lessons that are relevant to students’ lives.

- 461 -

http://www.cfchildren.org/

Family guide:

• The video-based Family Guide encourages family support of the Second Step program.

• Skills in empathy, impulse control, problem solving, and anger management are taught to help families practice and reinforce these skills with their children at home.

Curriculum

Lessons vary in length from about 20 minutes at the preschool level to 50 minutes in middle school/junior high. There are between 15-25 lessons in preschool-grade 5. For middle school/junior high, there are 15 lessons in the first year and 8 lessons each in the second and third years. Each lesson uses a social scenario that forms the basis for discussion, role-playing, and other applied activities. The same topics are covered in each grade level, and content and activities are tailored and targeted to each grade level. Lessons build sequentially and are designed to be taught in order. Second Step lessons are divided into the following areas of focus: empathy, problem solving, impulse control and anger management.

• Empathy: Teaches empathy as a series of skill steps, which include recognizing feelings in oneself and others, learning to understand others’ perspectives during discussions and role- playing, reducing labeling and stereotyping, and practicing respectful methods of communication and listening.

• Problem Solving: Provides children with problem-solving procedures. Children are taught to (1) identify problems, (2) brainstorm for solutions, (3) evaluate the possible effects and outcomes of solutions, (4) appropriately implement their solution, and (5) evaluate the outcome of their solution. In addition, this unit focuses on behavioral-skills training, providing students with the opportunity to practice interpersonal skills.

• Impulse Control/ Anger Management: Combines behavioral-skills training with problem-solving steps to manage anger. Lessons focus on recognizing anger cues and triggers and using positive self-talk and relaxation strategies to prevent escalation of anger. The unit focuses particularly on situations that typically generate stress for children, including feeling left out in social situations or activities, being criticized, and conflicts with siblings, peers, or parents.

Staffing

The classroom program is implemented by regular classroom teachers. The Family Guide sessions are led by trained facilitators who are comfortable with leading parent groups.

Issues to Consider

This program received a "promising" rating. Two large studies using random assignment and large sample sizes, Grossman et al. (1997) and Frey et al. (2005), found that Second Step participants produced more positive scores in observer ratings of negative behaviors in the classroom, lunchroom, and playground, as well as reductions in teacher-reported antisocial behavior. Findings across other outcomes were not as positive; Grossman et al. found that reports of student behavior by teachers and parents were not significantly affected by program participation. Furthermore, quasi-experimental studies by Orpinas et al. (1995), Lillenstein (2001), and Nicolet (2004) found few to no significant program effects. While the Grossman et al. (1997) study involved randomized assignment of treatment status at the school level, a school’s selection for inclusion in the study as a whole was not random. Schools were selected for inclusion based on their perceived willingness and ability to deliver the curriculum. In addition, participation was contingent on the teachers’ willingness to attend the program training sessions. Therefore, it is possible that these selection criteria for participation may have resulted in an atypical set of schools and teachers. Further, only 66 percent of the total number of eligible students

- 462 -

obtained consent, had all evaluation measures completed, and were included in the evaluation. This nonrandom participation in the evaluation may have influenced outcomes.

When considering the fade-out of results that emerged from the Grossman et al. (1997) study, it is important to note that for the purposes of the Grossman team’s study and evaluation, the implementation of the program was altered from the regular design and procedure. Under normal program conditions, the program is implemented evenly and consistently throughout the school year and throughout the entire school. In contrast, the Grossman et al. implementation took place over a 16- to 20-week period. Further, not all second- and third-grade classrooms at the participating schools took part in the program and evaluation. Under regular program design and implementation, students at six months would have progressed into another Second Step unit for continued program exposure, as opposed to the study design in which participants were exposed to one single Second Step unit for the length of the study and then exposure ceased. In addition, attenuation may have been further compounded by the fact that at the six-month follow-up students had progressed into a new grade level. The experimental and control groups did not remain within intact cohorts, but were mixed. It may be that the experiences and behaviors of each group affected each other at the six-month follow- up. These data do, however, call into question whether, even under full program design and implementation, positive outcomes would persist after students were no longer exposed to the program and were actively interacting with individuals that had not received the program. There are several limitations of the study by Frey et al. (2005), including the possibility that the manner in which the program was implemented may have led to spillover effects in the control schools, thus contaminating the control student population. The authors note that schools in the study were reluctant to agree to a waiting-list control design, and an agreement was reached so that those schools assigned to the control group were trained and allowed to implement the Second Step program in grades other than those being studied. There is a possibility that students and teachers who were not in the intervention Second Step classrooms may have shared information with students and teachers from the control classrooms, thus obscuring the true outcomes of the control groups in the absence of any Second Step implementation at the schools. Another study design limitation is that random assignment occurred only with the 11 schools recruited in the first year; in the second year, the remaining four schools were assigned nonrandomly to the control group. Thus, the overall design cannot be considered a true randomized controlled trial. In addition, similar to Grossman et al. (1997), positive treatment effect findings for reductions in antisocial behavior that were present in the first year of the study faded by the second year, suggesting that positive effects of the program may be limited in duration. Finally, although the program has curricula available for preschool through middle school, the program has been evaluated only among children of elementary-school age, i.e., kindergarten through sixth grade. To date, there has been no rigorous research conducted and published on the program’s effectiveness and outcomes with preschoolers or middle school children.

Example Sites

Prince Georges County, Maryland Public Schools, Chicago Catholic Charities Samaritan House Homeless Shelter, Denver York County, Pennsylvania Corpus Christi, Texas San Antonio, Texas King County, Washington

Contact Information

Client Support Services Committee for Children 568 First Avenue South, Suite 600 Seattle, WA 98104-2804.

- 463 -

phone: (800) 634-4449 ext. 200 fax: (206) 343-1445 email: clientsupport@cfchildren.org http://www.cfchildren.org

Available Resources

Program training, support, materials, and information are available from Committee for Children, a nonprofit organization that develops classroom programs focusing on youth violence, bullying, personal safety/child abuse prevention, and emergent literacy. Committee for Children offers both regional and on-site training for Second Step staff, trainers, and Family Guide facilitators; the schedule is available at http://www.cfchildren.org. Program materials for staff, students, and families can be ordered online, and are available English. A Spanish language supplement is also available for an additional purchase.

Bibliography

Frey, Karin S., Susan Bobbitt Nolen, Leihua Van Schoiack Edstrom, and Miriam K. Hirschstein, "Effects of a School-Based Social-Emotional Competence Program: Linking Children’s Goals, Attributions, and Behavior," Journal of Applied Developmental Psychology, Vol. 26, No. 2, 2005, pp. 171-200. Grossman, David C., Holly J. Neckerman, Thomas D. Koepsell, Ping-Yu Liu, Kenneth N. Asher, Kathy Beland, Karin Frey, and Frederick P. Rivara, "Effectiveness of a Violence Prevention Curriculum Among Children in Elementary School: A Randomized Controlled Trial," Journal of the American Medical Association, Vol. 277, No. 20, May 28, 1997, pp. 1605-1611. Lillenstein, Jennifer A., "Efficacy of a Social Skills Training Curriculum with Early Elementary Students in Four Parochial Schools," dissertation, Indiana, Pa.: Indiana University of Pennsylvania, 2001. Nicolet, Irene A., "The Second Step Violence Prevention Program: Effectiveness of a Brief Social Skills Curriculum with Elementary-Age Children," dissertation, Minneapolis, Minn.: Capella University, 2004. Orpinas, Pamela, Guy S. Parcel, Alfred McAlister, and Ralph Frankowski, "Violence Prevention in Middle Schools: A Pilot Evaluation," Journal of Adolescent Health, Vol. 17, No. 6, December 1995, pp. 360- 371.

Last Reviewed

August 2006

Self Center (School-Linked Reproductive Health Services)

Program Info

Outcome Areas Healthy and Safe Children

Indicators Youths abstaining from sexual activity or not engaging in risky sexual behavior

- 464 -

mailto:clientsupport@cfchildren.org

http://www.cfchildren.org/

Topic Areas Age of Child Adolescence (13-18) Type of Setting Middle School High School Health Care Provider Type of Service Health Care Services Health Education Youth Development Type of Outcome Addressed Physical Health Teen Sex/Pregnancy

Evidence Level Promising

Program Overview

The Self Center (School-Linked Reproductive Health Services) program was established to provide year-round contraceptive and reproductive health services and education to middle and high school students in Baltimore, Maryland. Funded and evaluated as a three-year program, it began in November 1981 and provided services through June 1984. In the program model, a social worker/nurse team provided students with in-school support and education as well as helped them to obtain free reproductive and contraceptive health services and counseling at a health clinic adjacent to the schools. Within-clinic services included pregnancy testing, contraceptives, treatment for sexually transmitted diseases (STDs), and ongoing education and counseling services. The school-based component included counseling and education services, with teams conducting homeroom and classroom presentations at least twice yearly, and a presence on- site in school health suites for several hours daily to facilitate small group discussions and informal individual counseling. The curriculum emphasized behavioral skills development, sexual abstinence, and contraceptive and sexuality/STD/HIV/AIDS education. A student resource team assisted staff with outreach.

Program Participants

The program study included all seventh through twelfth grade students at one middle school and one high school in Baltimore, Maryland. Students came from low-income, inner-city, African-American neighborhoods with high rates of sexual activity and teen pregnancy.

Evaluation Methods

The research study compared 1,700 students from the two schools participating in the program over a three-year period (1981 to 1984) with 1,950 students from similar backgrounds in schools not participating in the program. Students in the comparison group participated in their schools' standard sex/health education curricula. The evaluation design included both pre/post measurements and intervention/control groups. Student questionnaires were used to evaluate the following indicators: changes in knowledge, attitudes, and behaviors, including the percentage of students having sexual intercourse; the percentage of students using contraceptives; and the percentage of students becoming pregnant or causing a pregnancy. Student knowledge and behaviors were assessed before the program began and annually for three years.

- 465 -

In addition, an evaluation was done assessing the distribution of utilization of the various curriculum components, for example, the percentage of students accessing in-school services only, the percentage of students accessing both in-school and clinic services, and the percentage of students accessing small group versus individual counseling services.

Key Evaluation Findings

The study by Zabin et al. (1986) found that, compared with students in the nonparticipating schools,

• Significantly more girls in the program schools were likely to delay the start of sexual intercourse; the median delay was seven months.

• Both boys and girls showed a significant increase in contraceptive use at last intercourse. This effect was greatest among the younger, sexually active girls and boys whose use of contraception was minimal at the start of the program.

• Significantly more students in the program schools attended a health clinic before becoming sexually active and during the first months of sexual activity.

• By the program's third year, the pregnancy rate dropped 30 percent among the high school girls in the program schools, while it rose 58 percent among students who were not in the program schools.

• Pregnancy rates among younger teens in the program schools decreased slightly, while the pregnancy rates for younger teens that were not in the program schools increased dramatically.

Probable Implementers

This program could be implemented in partnership among public schools, social services, and health care professionals.

Funding

Program services were funded by the Educational Foundation of America. The Ford Foundation and the W. T. Grant Foundation funded the evaluation.

Implementation Detail

Program Design

• The program linked contraceptive and reproductive health services to school-based educational services.

• Clinic services were located in proximity to the participating schools in a place that was easily accessible to teens.

• Students had access to year-round, free, high-quality clinic services.

• The program was facilitated through commitments from the school superintendent, school principals, the health committee, and the health department.

• Clinic and school-based services were provided by the same staff in order to facilitate and encourage clinic attendance.

• Program staff had considerable expertise in education and/or health education.

Staffing

Self Center staff provided full-time, year-round support to teens through a combination of in-school and adjacent clinic-based services. Each of the program schools were assigned a Self Center team,

- 466 -

which was made up of a social worker or nurse midwife, who were particularly interested in adolescent health. An educator experienced in sex education was made available intermittently. For the in-school component, Self Center staff made at least one presentation a semester to every homeroom to introduce the program and to discuss student value definition, decision-making, and reproductive health. In addition, Self Center staff members were available on-site in the school health suite several hours daily to facilitate informal group discussions and individual counseling. These sessions allowed students to access information on topics such as relationships, physical development, drug use, and parenting. Furthermore, 12 students in each school served as spokespeople for the program in its last two years, creating visual aides for use around the schools. Parents were also informed about the program prior to its initiation, and a parental advisory committee was formed for the duration of the program. The same Self Center staff team provided year-round contraceptive, reproductive health, and other services at a clinic adjacent to the schools. Gynecologists from Johns Hopkins University provided needed medical care at the clinic. Teens requiring additional medical or psychological care not available through the clinic were given appropriate referrals. Discussion groups, educational videos, and pamphlets were also available to students waiting for clinic appointments. Individual and group counseling were offered through the clinic as well. Clinic services were available to teens on weekday afternoons and during school vacations. Additional details on program implementation, as well as on the day-to-day work of the social workers and nurses, are available in Zabin et al. (1988a).

Curriculum

The Self Center program does not have a prescribed curriculum.

Issues to Consider

This program received a "promising practice" rating. The research, conducted over three years, was implemented according to rigorous standards and included an experimental group of 1,700 teenagers and a comparison group of 1,950 teens. One possibly significant variable not clearly included in the program evaluation was the level and importance of involvement of Johns Hopkins University. The evaluation indicates that professionals from Johns Hopkins Medical Center offered their services in cooperation with the program and that resources (facilities, supplies, support, and such) were shared; however, there is no elaboration as to the degree of collaboration or analysis of what impact the involvement (such as saved or added costs) had on the function and viability of the program. The possibility exists that the cooperative partnership with a major university or medical center was an important contributor to the success of the Self Center program. The cost of the Self Center program was evaluated in Zabin et al. (1988b). The average cost of the program per student was $122, but this cost was not equally distributed; females were on average four times more costly to serve than males. Forty percent of the program budget was used on the school-based components, while the remaining budget was used in the medical clinic. A frequent objection voiced in response to aggressive and direct sex-education programming in schools involves the opinion that such programming, and the ready availability of birth control, will result in increased and earlier sexual activity among teens. Findings from the Self Center evaluation indicating a delay of first sexual intercourse as a result of the program counter this concern. For example, results indicate that approximately two-thirds more of the girls had become sexually active by age 14 before the program began as compared with after three years of exposure. This is a significant reduction in sexual initiation and supports the notion that rather than fostering new sexual activity among teens, exposure to the program is in fact reducing such activity. Because sexual exposure during early teens is associated with the highest risk of unintended conception, this created delay is of real importance.

- 467 -

In addition to reducing the number of teenagers who become pregnant or create a pregnancy, the program also affects the emotional and psychological well-being of participants through its focus on group and individual counseling and a focus on judgment and decision-making skills.

Example Sites

Baltimore, Maryland

Contact Information

Dr. Laurie Schwab Zabin School of Hygiene and Public Health Johns Hopkins University 615 N. Wolfe Street, Room W4503 Baltimore, MD 21205 Tel.: (410) 955-5753 Fax: (410) 955-0792

Available Resources

Detailed information and materials needed to establish, implement, and evaluate the program, as well as to obtain parental consent, have been published in Zabin et al. (1987). In addition, a Self Center program package developed by the Sociometrics Corporation includes a user's guide, program manual, educational pamphlets, evaluation materials, parental notification form, directory of evaluation consultants, and telephone technical support for a year (1-800-846-DISK).

Bibliography

Frost, Jennifer J., and Jacqueline Darroch Forrest, "Understanding the Impact of Effective Teenage Pregnancy Prevention Programs," Family Planning Perspectives, Vol. 27, 1995, pp. 188-195. Hardy, Janet B., and Laurie Schwab Zabin, Adolescent Pregnancy in an Urban Environment, Baltimore, Md., and Washington, D.C.: Urban & Schwarzenberg and The Urban Institute, 1991. Zabin, Laurie Schwab, and Marilyn B. Hirsch, Evaluation of Pregnancy Prevention Programs in the School Context, Lexington, Mass.: Lexington Books, 1987. (Note: This book is available from Dr. Laurie Schwab Zabin. See Contact Information above.) Zabin, Laurie Schwab, Marilyn B. Hirsch, Edward A. Smith, Rosalie Streett, and Janet B. Hardy, "Evaluation of a Pregnancy Prevention Program for Urban Teenagers," Family Planning Perspectives, Vol. 18, 1986, pp. 119-126. Zabin, Laurie Schwab, Marilyn B. Hirsch, Rosalie Streett, Mark R. Emerson, Morna Smith, Janet B. Hardy, and Theodore M. King, "The Baltimore Pregnancy Prevention Program for Urban Teenagers: I. How Did It Work?" Family Planning Perspectives, Vol. 20, 1988a, pp. 182-187. Zabin, Laurie Schwab, Marilyn B. Hirsch, Rosalie Streett, Mark R. Emerson, Morna Smith, Janet B. Hardy, and Theodore M. King, "The Baltimore Pregnancy Prevention Program for Urban Teenagers: II. What Did It Cost?" Family Planning Perspectives, Vol. 20, 1988b, pp. 188-192.

Last Reviewed

February 2009

- 468 -

Smart Start

Program Info

Outcome Areas Children Ready for School

Indicators Children ages 0 to 5 exhibiting age-appropriate mental and physical development

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Community-Based Service Provider Health Care Provider Type of Service Family Support Health Care Services Health Education Parent Education Type of Outcome Addressed Cognitive Development/School Performance

Evidence Level Promising

Program Overview

Smart Start is a comprehensive public-private community-based initiative to help all North Carolina children enter school ready to succeed. Created in 1993, the primary focus of Smart Start is to provide families with access to high-quality childcare. The program is predicated on the notions that (1) the first six years of life are the most critical; (2) better quality childcare programs can increase a child's ability at school entry; and (3) a child's ability at school entry can often predict later academic success. Currently, 81 local partnerships, encompassing all of North Carolina's 100 counties, have begun implementation of Smart Start to assure that children in their communities begin school healthy and ready to succeed. Smart Start's approach allows communities to make decisions and plans that are specific to the needs of their young children and families. All Smart Start programs are based on three core areas: (1) child care and education; (2) health care and education; and (3) family support and education. The individual services provided by each site under these core areas are tailored to each community depending on its specific needs, goals, and priorities. As such, the full range of services is not likely to be available at all sites.

Program Participants

All children with geographic access to a Smart Start program are eligible to participate. Although not specifically aimed toward children of low income, the program does attempt to reach those who would not otherwise necessarily have access to high-quality services such as childcare. In two comprehensive evaluative studies of the Smart Start program (FPG-UNC Smart Start Evaluation Team, 1997, and Maxwell, Bryant, and Bernier, 1998a), 24 percent and 29 percent of the participants,

- 469 -

respectively, were categorized as "poverty" level. These percentages are fairly consistent with the number of children under six years of age nationwide who were living in poverty in 1997, estimated at 22 percent by the National Center for Children in Poverty at Columbia University, using data from the Current Population Survey. Because of the multiple Smart Start-related services that may be embedded within the community, it is difficult to definitively or accurately categorize the Smart Start population in terms of characteristics, demographics, or numbers.

Evaluation Methods

Two multi-site comprehensive evaluations of Smart Start were conducted by the University of North Carolina’s Frank Porter Graham Center (FPG-UNC) Smart Start Evaluation Team. In the first evaluation, conducted in 1997, Smart Start families who had children entering kindergarten were invited to be part of a pilot study measuring kindergarten "readiness". Two years of previous attendance at the Smart Start center was required for participation. The sample, drawn from a single county, was composed of 39 Smart Start children and a comparison group of 272 children. Both groups were assessed upon initial entry to kindergarten using the Kindergarten Teacher Checklist (KTC), which asks teachers to rate the child’s cognitive, language, social, and motor skills on a scale of 1 to 5 with a higher score indicating greater skill. Outcomes analysis compared Smart Start children to those who had attended some type of other childcare prior to kindergarten entry, and to those who had no previous child care experience. A six-county study was conducted in 1998 that replicated the original single-county study with a larger, more geographically diverse population. The sample for this later study included 142 children in the Smart Start group and 294 children in the control group. This study (FPG-UNC Smart Start Evaluation Team, 1998) performed two levels of comparison, including: (1) an assessment of children who had attended a center that engaged in any Smart Start practices versus children who had attended non-Smart Start child care, and (2) a comparison of children who had attended Smart Start centers that engaged in specific activities related to improving childcare quality (such as enhanced subsidies for higher childcare quality, higher teacher education, license upgrades, on-site technical assistance, quality improvement, and facility grants) versus children who had attended non-Smart Start child care.

Key Evaluation Findings

The research studies (Maxwell, et al., 1998a; FPG-UNC Smart Start Evaluation Team, 1999) found the following:

• The Smart Start program has been effective at improving the quality of childcare at participating centers. This improvement is evidenced through a comparison of Smart Start programs to themselves (for example, increased accreditation by the National Association for the Education of Young Children, governmental licensing ratings, or increase in teacher credentials) and to non-Smart Start facilities. Programs that have increased their quality of childcare have produced more and a higher level of positive results than all other programs. This finding suggests that the improved quality of childcare has been an important contributor to positive outcomes.

• Children from low-income families who attended Smart Start programs were rated significantly higher in readiness for school than were children from low-income families who attended other centers.

• For non-poverty children, the mean KTC score was not significantly different for those who attended Smart Start compared with those who attended another childcare center.

• When compared with children who had not received previous childcare, children who had attended Smart Start scored significantly higher on the KTC, indicating an increased readiness for kindergarten.

- 470 -

• When compared with children who had attended non-Smart Start programs, children who attended Smart Start programs that focused specifically on childcare quality improvements performed significantly better on the KTC.

Probable Implementers

Agencies within the community who are able to collaborate, including preschools, health clinics, dental clinics, teacher education programs, parent education programs, family resource centers, and other such agencies.

Funding

Smart Start is funded by the North Carolina Department of Health and Human Services and by private donations. An extensive list of the private donors can be found on the Smart Start Web page at http://www.smartstart-nc.org/overview/donors.htm. Smart Start funds are administered at the local level through local nonprofit organizations. The North Carolina Partnership for Children is the statewide nonprofit organization that provides oversight and technical assistance for the 82 local partnerships established throughout the state. Services at the local level range depending on local needs. In 2004, funding for Smart Start was approximately $192 million in state funds. Since 1995, Smart Start has also raised more than $200 million in donations from businesses, foundations, and individuals.

Implementation Detail

Program Design

At each site the programs are designed and tailored to meet the needs of the specific communities. Smart Start's approach requires local communities to plan how best to meet their needs. Individual sites may offer a wide range of services including childcare, health screening, parent education, and family support services. All sites must adhere to the Smart Start goal of working to improve the quality of early childhood education and center-based care. Not all services are available at all sites.

Curriculum

The Smart Start project does not have a prescribed or "set" curriculum.

Staffing

Staffing decisions are made by each site and depend on the communities’ specific needs as far as education, childcare, training and technical assistance, and health care access.

Issues to Consider

This program received a "promising" rating. Although two multi-site evaluations were done, one with a large sample size, the studies had some methodological weaknesses. There were some problems with the way in which participants were categorized into the two observation groups. The length of time students in the test group had participated in Smart Start varied greatly, from 8 to 60 months. No analysis or consideration is given to the impact of this time range on findings or the possible influence that students with varying experiences may exert on each other. In addition, the researchers acknowledge that they had no way of being certain that children included in the control group had not attended child care centers that had benefited from some Smart Start services. Also, the utilization and impact of family support services offered by the Smart Start program, such as health screening and parent education, have not been evaluated in any way. The Frank Porter Graham Child Development Center at the University of North Carolina is engaged in

- 471 -

http://www.smartstart-nc.org/overview/donors.htm

ongoing research and evaluation of the Smart Start program. As further reports presenting additional findings and measuring longitudinal outcomes become available they will be reviewed and incorporated, and the PPN program rating will be revised as appropriate. In addition to identified benchmarks, research on Smart Start indicates that participation in the program may impact on several other outcomes. The program impacts student success in school, as shown through evaluations indicating that children who have benefited from the program perform at a higher level in kindergarten than children who have not had prior child care. Research indicates that programs offering a more diverse range of services are not necessarily more successful. This suggests that new Smart Start programs may wish to focus on implementing services such as improving the quality of childcare, which have been demonstrated to yield positive outcomes.

Example Sites

The complete list can be found at http://www.ncsmartstart.org/services/main.htm.

Contact Information

The NC Partnership for Children 1100 Wake Forest Road Suite 300 Raleigh, NC 27604 Tel (919) 821-7999 Fax (919) 821-8050 Smart Start Evaluation Team Marie Butts, Administrative Assistant Frank Porter Graham Child Development Center 105 Smith Level Road CB#8180 UNC Chapel Hill Chapel Hill, NC 27599-8180 Tel (919) 966-4295 smartstart@unc.edu

Available Resources

Smart Start resources are available on the program home page: http://www.smartstart- nc.org/index.htm. Available resources include, for example, an annual report, detailed program descriptions in both written and video format, a parent resource guide (in English and in Spanish), a toolkit to inform decisions on establishing high quality programs, and brochures regarding effective practices for early childhood educators and parents. In 2001, a National Technical Assistance Center (NATC) was established to assist other states with the development of an early education initiative (http://www.smartstart-nc.org/national/main.htm). NATC services include the provision of speakers on a variety of subjects related to Smart Start, site visits to state and local Smart Start partnerships, intensive technical assistance, and a one-year mentoring relationship with a Smart Start local partnership. In addition, NATC established National Smart.Net as a membership-based service to allow individuals and organizations to access resources such as information that is targeted to other states, more details regarding key components of Smart Start, and help in utilizing Smart Start materials. Through funding from the David and Lucile Packard Foundation, a technical assistance grant is available to up to ten communities and states to fund participation in the intensive technical assistance program.

- 472 -

http://www.ncsmartstart.org/services/main.htm

mailto:smartstart@unc.edu

http://www.smartstart-nc.org/index.htm

http://www.smartstart-nc.org/national/main.htm

Bibliography

FPG-UNC Smart Start Evaluation Team, North Carolina’s Smart Start Initiative: 1994-1995 Annual Evaluation Report, Frank Porter Graham Child Development Center at the University of North Carolina, Chapel Hill, 1995. FPG-UNC Smart Start Evaluation Team, North Carolina’s Smart Start Initiative: 1996-1997 Annual Evaluation Report, Frank Porter Graham Child Development Center at the University of North Carolina, Chapel Hill, 1997. FPG-UNC Smart Start Evaluation Team, A Six-County Study of the Effects of Smart Start Child Care on Kindergarten Entry Skills, Frank Porter Graham Child Development Center at the University of North Carolina, Chapel Hill, 1999. Maxwell, Kelly, Donna Bryant, and Kathleen Bernier, Child Care in the Pioneer Partnerships: 1994 and 1996, Frank Porter Graham Child Development Center at the University of North Carolina, Chapel Hill, 1997. Maxwell, Kelly, Donna Bryant, and Kathleen Bernier, The Effects of Smart Start Child Care on Kindergarten Entry Skills, Frank Porter Graham Child Development Center at the University of North Carolina, Chapel Hill, 1998a. Maxwell, Kelly, Donna Bryant, and Kathleen Bernier, Smart Start Client Information System Feasibility Study, Frank Porter Graham Child Development Center at the University of North Carolina, Chapel Hill, 1998b.

Last Reviewed

April 2005

Social Decision Making/Problem Solving

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children and youth not engaging in violent behavior or displaying serious conduct problems Children not experiencing anxiety or mood disorders, such as depression

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School Community-Based Service Provider Type of Service Youth Development

- 473 -

Type of Outcome Addressed Behavior Problems Mental Health Physical Health Substance Use and Dependence

Evidence Level Promising

Program Overview

The Social Decision Making/Problem Solving (SDM/PS) program was originally developed in 1979 as the Improving Social Awareness—Social Problem Solving Program. SDM/PS is a universal program, meaning it can be provided to any students, rather than targeting those with special characteristics. The program aims to help students acquire social and decision-making skills and to develop their ability to effectively use those skills in real-life and academic situations. More specifically, the program seeks to develop children's self-esteem, self-control, and social awareness skills, including identifying, monitoring, and regulating stress and emotions; increasing healthy lifestyle choices; avoiding social problems such as substance abuse, violence, and school failure; improving group cooperation skills; and enhancing the ability to develop positive peer relationships.

Program Participants

Students in grades K-8

Evaluation Methods

The first evaluation of SDM/PS (Elias, 1983) used a quasi-experimental design to study the program's impact on a sample of boys in special education classes at a residential treatment center for emotionally and educationally handicapped children. Most of the children were referred from homes in which there were absent, neglectful, abusive, or physically disabled parents. In their communities, these boys were typically considered difficult to manage at home and showed academic and behavior problems in school. The 109 participants ranged in age from 7 to 15 years, and the sample was 50 percent African American, 30 percent Caucasian, and 20 percent Hispanic. Fifty-two boys participated in the SDM/PS treatment group, and they were not selected randomly. The treatment involved showing ten videotapes that depicted children working through problem situations, and two different tapes were shown each week for five weeks. The videotapes were followed by discussions led by teachers and aides. (This early version of the SDM/PS program did not involve the "readiness" phase that is a component of the current program; see Implementation Detail). Fifty-seven boys made up the comparison group, and were drawn at random from the same or similar living units as the treatment group participants. Self-report data and measures of student behavior as assessed by teachers and counselors were taken three months prior to program implementation, at pretest within two weeks before program implementation, at posttest within two weeks of program completion, and at follow-up two months after program completion. Teachers were aware of which students had participated in the SDM/PS treatment group. A second study of SDM/PS was reported by Elias et al. (1986). The quasi-experimental design used a sample of 158 fifth-grade students from all four elementary schools in a predominantly white community of 15,000 residents in central New Jersey. Ninety-eight percent of the total possible sample participated, including 80 boys and 78 girls, who at baseline scored an average of one year above grade level on standardized academic tests. The introduction of SDM/PS was staggered, with two of the schools implementing the program before the other two schools. A control group was also used, consisting of children entering middle school (sixth grade) during the year preceding the study. Thus, comparisons were made among three groups: (1) No training (1978-1979); (2) Full training (instructional phase Oct.-Dec. 1979, application phase Jan.-May 1980); and (3) Partial training (instructional phase only, Jan.-May, 1980). The instructional phase of SDM/PS consisted of 20 lessons

- 474 -

averaging 40 minutes each, conducted twice per week. The application phase of SDM/PS consisted of teachers implementing specific problem-solving activities in the classroom, as well as teachers' use of "life space intervention," in which they attempted to mediate conflicts between students by facilitating children's problem-solving thinking rather than intervening and imposing their own solutions to the conflict. Student outcomes were assessed at the beginning of sixth grade on the self-reported Survey of Middle School Stressors, which included questions about adjustment to middle school and frequency and intensity of responses to 28 stressors (e.g., having many different teachers, having more homework, being approached to smoke or drink, etc.).

Key Evaluation Findings

The study by Elias (1983) reported:

• Immediately after completion of the program, treatment group children were rated by their teachers as more clearly improved in their ability to exercise emotional and behavioral self- control than were comparison group children, as shown by significant differences during instances of temper outbursts and being visibly upset.

• In addition, comparison group children showed a significant increase in loss of control under pressures of testing or other distressing conditions, while treatment group children showed no such tendency.

• Two-month follow-up scores on counselor ratings found significant effects on four of the nine child behavior scores. In relation to the comparison group, treatment group children were rated as significantly less detached and less socially isolated, and they showed a decrease in overall personality problems, as well as a marginal improvement in their ability to delay gratification. In the classroom, treatment group children showed a marginally significant increase in self-reliant learning when contrasted with comparison group children.

Elias et al.'s (1986) study of 158 students reported the following results:

• For 11 school stressors on the list of 28, the children receiving full SDM/PS training reported the stressor in question to be significantly less of a problem than did children receiving partial training. These included stressors such as the logistics of adjusting to middle school, coping with peer pressure, and adjusting to academic requirements.

• For summary scores of problem frequency and problem intensity, significant differences were found among the three groups on 14 individual stressor items. The overall pattern of differences indicates that children in the full training group experienced significantly fewer stressors than children in the partial training group, and that both treatment groups reported significantly fewer stressors than did comparison group students.

Probable Implementers

Public and private elementary schools

Funding

The research, development, and expansion of the SDM/PS program and evaluation were funded by the William T. Grant Foundation, the National Institute of Mental Health, the Middlesex County (NJ) Board of Freeholders, and in-kind contributions from the University of Medicine and Dentistry of New Jersey (UMDNJ) and Rutgers University. The UMDNJ provides implementation assistance and professional development for the SDM/PS program. Professional development services include one-day on-site workshops ($140 per teacher including curriculum materials or $1,500 per group per day plus trainer expenses); on-site follow-up services ($1,500 per day plus expenses) including optional classroom observation and feedback and consultation on program planning, implementation, evaluation, and coordination with other curricula;

- 475 -

advanced training; and training for teams of administrators and supervisors. Costs of program materials are $80 per 25 students and include teacher manuals, parent materials, and lesson sets for each grade.

Implementation Detail

Program Design

The SDM/PS lessons are believed by program developers to be most effectively taught in at least one dedicated classroom session per week (two for special education students), but since prosocial, critical thinking and life learning skills are relevant to many subject areas, the approach can be incorporated into regular classroom lessons if desired. Parent participation and support in the SDM/PS program is considered a helpful and important factor for program success, and a wide variety of outreach activities and materials are available from the UMDNJ, such as local cable video programs, books, and "refrigerator notes." Other recommended school-based parent outreach activities include evening sessions that provide dinner and child care, bagel breakfasts during which parents join a classroom session in which children share what they are learning, present student work through role play and videos, or participate in a "back-to-school" night to introduce parents to the skills their children will be learning.

Curriculum

SDM/PS is a curriculum-based program that is provided in three developmental phases.

(a) The first phase is the "readiness" phase, which targets self-control and social awareness skills. Lessons and activities target skills such as listening, following directions, resisting provocation, avoiding provoking others, self-monitoring stress and emotions, and group/social awareness skills such as how to select friends and show a caring attitude toward others. These skills are a primary focus when the program is first introduced into a school. (b) The "instructional" phase teaches students an eight-step social decision-making strategy to help students maintain clear thinking in social problem situations. (c) The "application" phase helps children to use their newly acquired skills in real-life interpersonal and academic situations, through guided practice, role-playing, skill modeling, and the use of hypothetical social problem situations. Teacher-facilitated questioning and discussion, as well as cooperative group projects and writing assignments, further integrate the techniques. Staffing

SDM/PS is taught and implemented in classrooms by regular classroom teachers. The UMDNJ offers training opportunities to individual schools and/or school districts. At the school district level, training can be tailored to suit each district's local needs. The SDM/PS program staff provides a two to three day in-service training for a team of up to 30 teachers, administrators, and support personnel. Participants are provided with all of the curriculum materials, classroom posters, and worksheets needed to implement the program immediately following training. To help ensure that the program becomes an integrated part of the school's curricula, an on-site SDM/PS leadership team is formed to plan and guide the program toward institutionalization. The leadership team consists of a small group of representative teachers, the school principal, and other key resource staff such as a guidance counselor. A half- or full-day of leadership and management training for the leadership team upon the conclusion of the regular training workshop is strongly recommended. Training for individual teachers or counselors is also available if the number of people to be trained is too small to warrant district-level training. Training sessions cosponsored by the UMDNJ and Rutgers

- 476 -

University are held several times per year, and information can be obtained by contacting Linda Bruene at (732)-235-9280.

Issues to Consider

This program received a "promising" rating. The studies of the SDM/PS program were quasi- experimental in design, but they used reasonably well-matched comparison groups and statistical methodology to assess program impacts. When compared with children in control groups, SDM/PS Program Participants were found to experience significant decreases in depression (boys only), increases in emotional and behavioral control, and decreases in violent behavior and conduct problems. While the SDM/PS evaluations have demonstrated the program's effectiveness across several realms, it should be noted that the studies have been limited to New Jersey, that the program as evaluated was implemented in the early 1980s, and that the program developer served as an author on both of the studies.

Example Sites

New Jersey (Cape May Special Services School District, Jersey City, Highland Park, Westwood, Montvale, Bloomfield, Orange, and The Children's Institute in Verona). Illinois (St. Charles).

Contact Information

Linda Bruene University of Medicine and Dentistry of New Jersey University Behavioral HealthCare Behavioral Research and Training Institute SDM/PS Program 151 Centennial Ave. Suite 1140 Piscataway, NJ 08854 phone: (732) 235-9280 fax: (732) 235-9277 email: spsweb@umdnj.edu

Available Resources

The SDM/PS program is housed at the University of Medicine and Dentistry of New Jersey (UMDNJ). More information can be obtained at the program's Web site: http://www.umdnj.edu/iqrtweb/services/. In addition to training, other UMDNJ services include on-site consultation, support, technical assistance, program manuals and toll-free phone consultation. Additional SDM/PS program manuals are published by Research Press, and are available at http://www.researchpress.com.

Bibliography

Elias, Maurice J., "Improving Coping Skills of Emotionally Disturbed Boys Through Television-Based Social Problem Solving," American Journal of Orthopsychiatry, Vol. 53, No. 1, 1983, pp. 61-72. Elias, Maurice J., Michael Gara, Michael Ubriaco, Peggy A. Rothbaum, John F. Clabby, and Thomas Schuyler, "Impact of a Preventive Social Problem Solving Intervention on Children's Coping with

- 477 -

mailto:spsweb@umdnj.edu

http://www.umdnj.edu/iqrtweb/services/

http://www.researchpress.com/

Middle-School Stressors," American Journal of Community Psychology, Vol. 14, No. 3, 1986, pp. 259- 275.

Last Reviewed

January 2007

SPORT

Program Info

Outcome Areas Healthy and Safe Children

Indicators Youths not using alcohol, tobacco, or illegal drugs Children experiencing good physical health

Topic Areas Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Middle School High School Out of School Time Type of Service Health Education Type of Outcome Addressed Physical Health Substance Use and Dependence

Evidence Level Promising

Program Overview

SPORT aims to integrate physical activity and other health-enhancing habits with substance abuse prevention topics in order to appeal to adolescents by linking alcohol avoidance with attractive images of active and healthy adolescents. SPORT includes a health behavior screen, a health consultation, and a take-home fitness prescription, all of which integrate alcohol avoidance and other topics. The program design draws on several theories. The predominant one is the Integrative Behavior-Image Model (BIM) (Werch, 2007), which emphasizes normative youth development, promoting positive goals, and healthy behavior along with awareness of health risks of negative behaviors. SPORT applies this concept by linking images of physical activity and other health habits, such as healthy eating and sleep habits, to alcohol prevention. For example, one message emphasized through SPORT is that alcohol use does not correspond to an active lifestyle (Werch et al., 2005). The health behavior screen assesses six different types of health behaviors, including sport and physical activity, exercise, physical activity norms, breakfast and nutrition, sleep and rest, and alcohol initiation and use. The health consultation includes scripted messages about normative behaviors and

- 478 -

risks associated with risky behaviors, tailored to the individual’s circumstances. The take-home fitness prescription includes goals for youth to improve their health-related behaviors based on their individual circumstances (Werch et al., 2005).

Program Participants

SPORT was designed for use among adolescents.

Evaluation Methods

There is a large body of research on the effectiveness of SPORT. One study utilized an experimental design that compared outcomes for intervention group participants with those of control group participants (Werch et al., 2005). Other studies compared SPORT to modified versions of the intervention without a control group or were conducted as process evaluations. Only the results of the study that meets Promising Practices Network (PPN) criteria and uses an experimental design — Werch et al., 2005 — are presented here. Werch et al. (2005) conducted an evaluation of SPORT among students in ninth and eleventh grades at a suburban high school in northeast Florida between fall 2002 and fall 2003. Students were recruited through classroom presentations, announcements from teachers and staff, and schoolwide announcements. A total of 604 students participated: 335 ninth grade students and 269 eleventh grade students. Students were randomly assigned within grade levels to receive SPORT (intervention group) or to be in the control group. Students in the control group were seated in a quiet place and given pamphlets about alcohol prevention and health promotion to read. One week after this reading session, the control group students were mailed a pamphlet on health and fitness. For both intervention and control groups, the procedures were scheduled so as not to last longer than one class period. Despite random assignment, participants in the control group were significantly more likely than intervention group participants to have a family drug or alcohol problem (42.7 percent versus 34.9 percent). There were no other significant differences between the two groups at baseline, including no differences in alcohol or drug consumption or physical activity. Baseline assessments were conducted for all participants in fall 2002 prior to the intervention. Post-intervention assessments were conducted at three and 12 months after baseline. At the three time points, participants completed the Youth Alcohol and Health Survey (Werch et al., 2005), which assesses alcohol and drug use, risk factors and protective factors associated with alcohol and drug use, and exercise habits. The survey asked about alcohol, marijuana, and cigarette consumption (self-reported 30 day frequency and quantity), stage of initiation of alcohol, marijuana, and cigarette use (ranging from will never try substance to using substance for longer than six months), episodes of moderate physical activity in a seven-day period (self-report of at least 30 minutes of activity with no sweating or breathing hard), and episodes of vigorous physical activity in a seven-day period (at least 20 minutes of activity with sweating and breathing hard). At the 12-month follow up, 85 percent of the original sample was retained, and there were approximately equal frequencies of attrition across both the intervention and control groups. More participants who dropped out of the control group had mothers who drank a minimum of a few times a year than participants who dropped out of the intervention group. There were no other significant differences between the participants who dropped out of the control and intervention groups based on baseline data.

Key Evaluation Findings

At the three-month post-intervention assessment conducted by Werch et al. (2005), the following results were found:

• Alcohol behaviors improved significantly more for intervention students than for control students, as measured by 30-day alcohol frequency, 30-day alcohol quantity, 30-day heavy use of alcohol, and alcohol problems.

- 479 -

• Students in the intervention group were significantly less likely to initiate alcohol use than students in the control group, as measured by stage of alcohol initiation and length of alcohol use.

• A significantly higher frequency of alcohol protective factors was seen among intervention students than among control group students, as measured by negative expectancy beliefs, behavioral capability, resistance self-efficacy, self-control, value incompatibility, perceived susceptibility, parental monitoring, parent-child communication, and positive parent-child relationship.

• Students in the intervention group were at less risk for alcohol use than control group students, as measured by positive expectancy beliefs, subjective norms, intentions to drink the in the future, alcohol attitudes, perceived peer prevalence, and ability to be influenced by peers.

• Intervention group students were more likely to engage in physical activity than control group students, as measured by moderate and vigorous physical activity.

• Students in the intervention group used cigarettes and marijuana less frequently than students in the control group, as measured by 30-day cigarette frequency and 30-day marijuana frequency.

• There were no significant differences in drug initiation, as measured by stage of cigarette initiation and stage of marijuana initiation.

At the 12-month post-intervention assessment conducted by Werch et al. (2005), the following results were found:

• The three-month finding for alcohol protective measures was maintained at 12 months. A significantly higher frequency of alcohol protective factors was seen among intervention students than among control group students, as measured by negative expectancy beliefs, behavioral capability, resistance self-efficacy, self-control, value incompatibility, perceived susceptibility, parental monitoring, parent-child communication, and positive parent-child relationship.

• The three-month finding for alcohol risk factor measures was also maintained. Students in the intervention group were at less risk for alcohol use than control group students, as measured by positive expectancy beliefs, subjective norms, intentions to drink the in the future, alcohol attitudes, perceived peer prevalence, and ability to be influenced.

• Likewise, the three-month finding for drug use was maintained at 12 months. Students in the intervention group used cigarettes and marijuana less than students in the control group, as measured by 30-day cigarette frequency and 30-day marijuana frequency.

• While there was no significant difference in drug initiation at three months, intervention group students were significantly less likely to initiate drug use at 12 months, as measured by stage of cigarette initiation and stage of marijuana initiation.

• The significant differences in alcohol behaviors, alcohol initiation, and engagement in physical activity found at three months were not maintained at 12 months.

Werch et al. (2005) also assessed outcomes trends over baseline, three months, and at the 12-month follow-up and found the following results:

• There was a significant positive trend over time for the intervention group participants compared with control group participants for alcohol behaviors, as measured by 30-day alcohol frequency, 30-day alcohol quantity, 30-day heavy use of alcohol, and alcohol problems.

• At the three- and 12-month follow-up there were significant positive trends for intervention group participants compared with control group participants for alcohol initiation, as measured by length of alcohol use and stage of alcohol initiation.

- 480 -

• There were also significant positive trends over time for intervention group participants compared with control group participants for alcohol protective factors, as measured by negative expectancy beliefs, behavioral capability, resistance self-efficacy, self-control, value incompatibility, perceived susceptibility, parental monitoring, parent-child communication, and positive parent-child relationship.

• Likewise, there were significant positive trends over time for intervention group participants compared with control group participants for alcohol risk factors, as measured by positive expectancy beliefs, subjective norms, intentions to drink the in the future, alcohol attitudes, perceived peer prevalence, and ability to be influenced.

• Intervention group participants demonstrated significant positive trends over time compared with control group participants for drug initiation, as measured by stage of cigarette initiation and stage of marijuana initiation.

• There were no significant trends over time in drug behaviors or engagement in physical activity.

This evaluation (Werch et al., 2005) also assessed the interaction between self-reported cigarette and/or marijuana use and intervention exposure. Where there is a significant positive interaction effect between self-reported drug use and intervention exposure, this indicates that the effect of the intervention was stronger for drug users than for non-drug users. At baseline, there were 53 drug users in the intervention group and 65 drug users in the control group. Based on this analysis, the following results were found:

• At three months, there was a significant positive interaction effect between drug use and intervention exposure for alcohol behaviors. This finding was not maintained at 12 months.

• At three months, there was also a significant positive interaction effect between drug use and intervention exposure for drug behaviors. This finding was maintained at 12 months.

• At three months, there was a significant positive interaction effect between drug users and intervention exposure for drug initiation. This finding was not maintained at 12 months.

• There were no differences in alcohol initiation at three months or 12 months.

• At three months, there were no significant differences in participation in physical activity. At 12 months, however, there was a significant positive interaction effect between drug user and intervention exposure for engagement in physical activity.

Probable Implementers

High school and middle school health and physical education teachers, school counselors and nurses, after school teachers, community prevention and behavioral mental health providers, youth organizations, faith-based organizations, sporting groups and agencies, primary health care and health clinic providers, and others working with adolescents.

Funding

Possible sources of funding to support SPORT implementation include the Substance Abuse and Mental Health Services Administration (SAMHSA) and the Drug Free Communities Program. In addition, implementation of SPORT may be eligible for reimbursement by Medicaid and commercial health insurance in clinical and primary care settings as a Screening and Brief Intervention (SBI). More information regarding funding can be obtained through the Brief Programs for Health website: http://www.briefhealthprograms.com/index.php/programs-training/funding-resources

- 481 -

http://www.briefhealthprograms.com/index.php/programs-training/funding-resources

Implementation Detail

Program Design

Adolescents participating in SPORT receive an in-person health behavior screen and a one-on-one consultation with a fitness specialist to promote an active lifestyle and to discuss the risks of alcohol consumption and the impact of alcohol on an active lifestyle. The adolescents then receive a fitness prescription, including personalized goals for sleep, nutrition, physical activity, and alcohol use. One week after the fitness specialist consultation, the participants are mailed a flyer reiterating the messages conveyed during the consultation.

Staffing

A SPORT fitness consultant administers the fitness consultation. Fitness specialists are health care professionals, including nurses and certified health education specialists. All fitness specialists need to undergo a training programing prior to administering consultations (Werch et al., 2005).

Curriculum

Werch (2007) explains the Behavior-Image Model (BIM) as a framework that links a positive health image to a health risk behavior, with the aim of increasing the positive health behavior and decreasing the health risk behavior through one intervention. The underlying theory to the BIM is that two positive health behaviors can be coupled to reinforce each other, whereas a positive health behavior and a negative health behavior can be coupled to oppose each other. This opposition occurs by educating individuals to the negative correlation between a positive health state and a negative health behavior. In the context of SPORT, the intervention aims to create images of physical health for participants and to demonstrate that substance use does not fit with this image (Werch et al., 2005).

Issues to Consider

This program has been given a "promising" rating. The evaluation conducted by Werch et al. (2005) was a randomized evaluation of SPORT that included participants from one high school in a suburban neighborhood of northeast Florida. Also, participants in the control group were significantly more likely than intervention group participants to have a family drug or alcohol problem, and this difference was not controlled for in the analysis. There were no other significant differences between intervention and control participants at baseline. Note also that Dr. Werch, lead author on the evaluations cited, is the program developer. Werch et al. (2003) conducted an evaluation of SPORT compared with SPORT plus an alcohol consultation (SPORT Plus) and compared with SPORT plus an alcohol consultation and materials on alcohol prevention mailed to parents (SPORT Plus Parent) among 465 eighth grade students in three northeast Florida schools. This study did not use a control group that did not receive SPORT and therefore did not meet PPN evidence criteria. Nonetheless, the findings of this study are of interest because students in all programs showed reductions in alcohol consumption and problems, and increases in physical activity. SPORT had the larger positive impact among those participants who were substance users at baseline. SPORT Plus Parent was effective in increasing parent-child alcohol communication and in increasing alcohol use self-control.

Example Sites

SPORT was implemented in one high school in a suburban neighborhood of northeast Florida.

Contact Information

Chudley Edward Werch, PhD PreventionPLUSWellness 3595 Forest Bend Terrace

- 482 -

Jacksonville, FL 32224 (904) 472-5022 cwerch@preventionpluswellness.com http://preventionpluswellness.com

Available Resources

Webinar or workshop training options are available to certify SPORT implementers. Training includes all reproducible program materials needed to implement and evaluate SPORT, as well as re-implement the program as a booster or follow-up. 30-day program support and technical assistance by phone and email are also included with purchase. www.briefhealthprograms.com

Bibliography

Werch, Chudley (Chad), "The Behavior-Image Model: A Paradigm for Integrating Prevention and Health Promotion in Brief Interventions," Health Education Research, Vol. 22, No. 5, 2007, pp. 677- 690. Werch, Chudley (Chad), Michele J. Moore, Carlo C. DiClemente, Rhonda Bledsore, and Edessa Jobli, "A Multihealth Behavior Intervention Integrating Physical Activity and Substance Use Prevention for Adolescents," Prevention Science, Vol. 6, No. 3, 2005, pp. 213-226. Werch, Chudley (Chad), Michele Moore, Carlo C. DiClemente, Deborah M. Owen, Edessa Jobli, and Rhonda Bledsoe, Journal of School Health, Vol. 73, No. 10, 2003, pp. 380-388.

Last Reviewed

September 2012

Student Achievement Guarantee in Education (SAGE)

Program Info

Outcome Areas Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Elementary School Out of School Time Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance

- 483 -

mailto:cwerch@preventionpluswellness.com

http://preventionpluswellness.com/

http://www.briefhealthprograms.com/

Evidence Level Promising

Program Overview

The Student Achievement Guarantee in Education (SAGE) program is a statewide effort in Wisconsin to increase the academic achievement of children living in poverty. The key mechanism used to achieve this goal is a reduction of the student-teacher ratio in kindergarten through third grade to 15 to 1. In addition to class size reduction, schools participating in the program are expected to implement curricula with a rigorous academic focus, engage in professional development and accountability plans, and develop "lighted schoolhouse" before- and after-school programming with activities for both students and community members. SAGE was created as a five-year pilot program in 1995. The program was phased in over a three-year period beginning with kindergarten and first grade in 21 school districts during the 1996-1997 school year, with the addition of second grade in the second year, and finally third grade in the third year. SAGE schools changed class sizes in a number of ways in order to meet program requirements, including

• a single classroom in which the student-teacher ratio is 15 to 1

• a shared-space classroom in which a classroom is fitted with a temporary wall, thereby generating two classroom spaces, each with one teacher and approximately 15 students

• a dual-teacher classroom in which two teachers work collaboratively with approximately 30 students

• a floating-teacher classroom in which the room ordinarily has between 16 and 20 students, except during reading, language arts, and mathematics instruction when another teacher joins the class to reduce the ratio to 15 to 1.

Program Participants

When SAGE began in 1996 the program was limited to districts with high poverty schools. In 2000 the law was changed to allow any Wisconsin school to apply. Approximately 500 schools participated in the 2005-06 school year.

Evaluation Methods

A five-year evaluation of the SAGE demonstration project was conducted by the School of Education at the University of Wisconsin-Milwaukee (UWM). The evaluation involved 30 schools from 21 school districts across Wisconsin. SAGE was first implemented with 80 kindergarten, 96 first-grade, and 5 mixed kindergarten and first-grade classrooms at the start of the 1996-1997 school year. An additional 113 second-grade classrooms were added in 1997, and 88 third-grade classrooms in 1998. The UWM evaluation compared students in SAGE schools with a group of 14 to 17 non-SAGE comparison schools located in SAGE school districts. Comparison schools were not matched to SAGE schools, but resembled SAGE schools in terms of family income, achievement in reading, kindergarten through third-grade enrollment, and racial composition. In the fall of 1996, 30 schools (seven of which were in Milwaukee) implemented the SAGE program. Over the course of the year, the study involved 3,614 students and 220 teachers. Across all years of the evaluation (1996-1997 through 2000-2001), SAGE and comparison schools were composed of approximately 25 percent African American students, 45 percent Caucasian students, 10 percent Native American students, and 20 percent students of other ethnicities (comparison schools tended to have a slightly higher percentage of Caucasian students). An average of approximately 55 percent of SAGE students qualified for free lunch, as compared with approximately 45 percent of comparison

- 484 -

students, and a slightly higher percentage of SAGE students had repeated a grade, spoke English as a second language, or were classified as having special education needs.

The primary method of evaluation to determine the impact of class-size reduction was the annual administration of the Comprehensive Test of Basic Skills (CTBS) to both the SAGE and comparison schools. The complete CTBS battery includes subtests in reading, language arts, and mathematics. Although SAGE was implemented in kindergarten classrooms, students in kindergarten were not tested until the following year when they were first-grade students. Analyses of findings were conducted to assess the impact of SAGE participation on all students, as well as to compare performance of African American SAGE students to white SAGE students, and African American SAGE students to African American comparison students. Only those students who participated in the entire program, that is, who were present in the 1997-1998 SAGE and comparison first-grade classrooms, the 1998-1999 SAGE and comparison second-grade classrooms, and the 1999-2000 third-grade classrooms are included in the presented results. This group consisted of a total of 1,212 students, including 779 SAGE students and 433 comparison students. It should be noted that not all students completed all CTBS testing each year. A further study of SAGE was conducted by Webb et al. (2004), and it assessed program effects on state assessments of students in grades three and four. The original 30 SAGE schools (approximately 4,900 students) and 10 to 17 comparison schools in each grade (approximately 7,800 students) took part in the study. Data used in the analysis included the CTBS scores in reading, language arts, and mathematics collected by UWM researchers, the grade three Wisconsin Reading Comprehension Test (WRCT), and the grade four Wisconsin Knowledge and Concepts Examination (WKCE).

Key Evaluation Findings

SAGE was evaluated over numerous years at different grade levels from first through fourth grades. Overall, findings suggest that SAGE participants performed significantly better than comparison children on the majority of reading, language arts, and mathematics achievement tests at all grade levels. There is evidence of a cumulative beneficial effect of the intervention over multiple years. Specifically, in reports published in each year of the evaluation (1997 through 2001), the UWM evaluation team found the following: Achievement Outcome Findings, 1997 (Implementation Year One)

• After one year, when controlling for pretest scores, family income, school attendance, and race/ethnicity, SAGE first-grade students scored significantly higher on the total score and on reading, language arts, and mathematics subtests of the CTBS than did comparison students.

• African American SAGE students scored lower on the CTBS pretest total score than did African American comparison students, but they scored significantly higher than comparison students on the posttest.

• African American male SAGE students attained comparable or higher change scores from pre- to posttest than did African American female SAGE students. Conversely, in comparison schools, change scores for females exceeded change scores for males on every subtest as well as on the total score.

Achievement Outcome Findings, 1998 (Implementation Year Two)

• Second-grade classrooms were included in 1997-1998, and for all analyses, SAGE classrooms scored significantly higher than comparison classrooms. The results suggest that the positive first-grade effects of the SAGE program were maintained but did not increase in second grade.

• For first-grade classrooms, African American SAGE students scored significantly higher than African American comparison students on every posttest and on the total CTBS score. They also made significantly larger gains than comparison students on every pre- to posttest change score.

- 485 -

Achievement Outcome Findings, 1999 (Implementation Year Three)

• For first-, second-, and third-grade classrooms in 1998-1999, SAGE students significantly outscored comparison students on all posttests, with the exception of reading scores in second grade.

o African American SAGE students made significantly larger gains than African American comparison students from pre- to posttest and significantly outscored comparison students on all posttest scores.

Achievement Outcome Findings, 2000 (Implementation Year Four)

• First-grade students were not tested in 1999-2000; results were limited to second- and third- grade students.

• Second-grade SAGE students showed a significant achievement advantage over comparison group students for all subtests (when the first-grade pretest was used to adjust for achievement differences).

o African American SAGE students made significantly larger gains than African American comparison students from pre- to posttest and surpassed comparison students on all posttests.

o While African American students, as a group, scored significantly lower than Caucasian students in both SAGE and comparison schools, the gap between African American and Caucasian students was larger in comparison schools.

• SAGE third-grade students showed significant improvement over their comparison school counterparts from the beginning of first grade to the end of third grade across all academic areas.

o Using the first-grade pretest as the baseline score, African American SAGE students scored significantly higher than African American comparison students on the total score and on every CTBS subtest except for language arts.

o Gains made by African American versus Caucasian students were significantly better in SAGE schools from the beginning of first grade to the end of third grade. The opposite pattern was observed in comparison schools.

Achievement Outcome Findings, 2001 (Implementation Year Five)

• First- and second-grade students were not assessed in 2000-2001, limiting results to third graders (who had participated since kindergarten).

• Third-grade SAGE students scored significantly higher than comparison students on the total CTBS score and on all subtests.

o For African American students, statistically significant change scores from first grade to third grade were found on all CTBS tests, with SAGE students outperforming comparison students. The only exception was language arts, for which no statistically significant effects were found.

o African American students continued to score significantly lower than Caucasian students on total scale score and on all subtests, regardless of whether they were SAGE or comparison school students. No significant differences in the gains made by African American students versus white students were observed for this group of students.

The study by Webb et al. (2004) reported:

• By reanalyzing the data collected by the University of Milwaukee research team, Webb et al. (2004) replicated the findings that the SAGE program had a significant cumulative effect from the beginning of first grade through third grade in all three CTBS content areas (reading, language arts, and mathematics). The greatest cumulative effect was in mathematics, with a

- 486 -

more than 13-point difference in favor of the SAGE students. These data indicate that there is a benefit to students for being in SAGE classrooms over multiple years.

• These effects were not sustained into fourth grade on the WKCE in reading, language arts, or mathematics tests, with SAGE students scoring statistically equivalent to comparison students. Similarly, no cumulative effect for the SAGE program was found for the grade 3 WRCT. The results on the grade 3 WRCT and the grade 3 CTBS reading test produced conflicting results for the effectiveness of SAGE; WRCT results indicated no significant effect, whereas the CTBS results indicated significant cumulative effects.

Probable Implementers

Public or private elementary schools

Funding

SAGE is funded by the state of Wisconsin and administered by the Department of Public Instruction. Wisconsin state law limits SAGE funding to $2,000 per low-income student, an amount that has been unchanged since the program’s inception. Since 2000-2001, thirty-three schools have dropped out of the program, many citing financial issues. If the $2,000 per student had been increased by the same Consumer Price Index as district revenue limits, the aid amount would become $2,405 per student for 2005 and $2,529 for 2007.

Implementation Detail

Program Design

The SAGE program consists of four components: class-size reduction (maximum 15 to 1 student-to- teacher ratio), rigorous curriculum, professional development, and "lighted schoolhouse."

• Only the class-size-reduction portion of the program has been uniformly implemented and evaluated among SAGE program sites.

• A "rigorous curriculum" is defined by SAGE as employing a curriculum that meets state standards. Schools are permitted to determine the specific curriculum that best fits their needs.

• Each SAGE school must create a staff professional development and accountability program.

• Every SAGE school is required to implement a "lighted schoolhouse" program, providing before- and after-school extended-hours programming. Schools are expected to tailor this program to meet their resources and needs.

Staffing

The class size reduction component of SAGE has no special staffing needs, other than the specific ratio of 15 students per teacher. Classrooms are taught by the regular classroom teachers.

Issues to Consider

This program was given a "promising" rating. While the evaluations found positive program effects on outcomes of reading, language arts, and mathematics, SAGE and comparison schools were not particularly well-matched, and data on group characteristics indicate that there were meaningful discrepancies between the two populations of students. As such, it is possible that observed outcomes may have been influenced by these preexisting differences. Further, observed changes after the start of the program show a consistently unstable level of enrollment in both the SAGE and comparison schools. For example, during the second year of the program, only 42.3 percent and 44.3 percent of the student population at study and comparison schools, respectively, were ongoing. Of the students

- 487 -

at study and comparison schools, 18.8 percent and 18.6 percent, respectively, withdrew, and 39.0 percent and 38.0 percent of students, respectively, were new enrollees. Researchers did not conduct an analysis to determine what impact this might have had on test scores. It is also important to consider that there is a possible selection bias regarding the way in which schools were designated as SAGE participants. Participating districts were allowed to select the SAGE school from among all of the schools in the district that met SAGE inclusion criteria. No information is given on how districts made this selection. It is possible that districts selected certain schools based on their anticipated success and higher level of program participation or higher level of school functioning (as could possibly be inferred by the baseline discrepancies between SAGE and comparison schools). Without any further analysis of this factor, it cannot be ruled out as a potential contributor to the demonstrated outcomes. Overall, analyses of test results at the classroom level suggest that students in smaller classrooms tended to score significantly higher in language arts, mathematics, and reading as well as the total CTBS score after adjusting for individual pretest results, socioeconomic status, and attendance. In other words, classrooms with fewer students were more likely to have higher class average achievement scores and were more likely to contribute to closing the achievement gap between African American and white students than were classrooms with a higher number of students. Only the class-size reduction component of SAGE has been explicitly evaluated, as participating classrooms closely adhered to the 15 to 1 student/teacher ratio. The remaining three initiatives— rigorous curriculum, professional development, and lighted schoolhouse—were not directly examined in the UWM evaluation. The evaluation team felt that because these three initiatives were loosely executed and not uniformly implemented across sites, a single standard of measure was not possible. The UWM authors indicated that they are fairly comfortable in concluding that the impact of reduction in class size is most likely the primary cause of observed student outcomes. However, as these additional initiatives were implemented to some extent in some of the SAGE schools, they cannot be entirely ruled out as contributing to the findings. As such, it is not possible to ascertain whether class- size reduction alone would generate the test-score gains observed, or whether all four program components are required to produce these gains. In addition, the results indicate that although SAGE students outperform comparison students in terms of overall test scores throughout the duration of the program, after the first grade, comparison students begin to narrow the achievement gap. This calls into question whether gains initially made by program participation will translate into a continued long-term advantage and enhanced performance on the part of SAGE students. This analysis also did not take into consideration the different approaches that schools and teachers used to reduce class size. It may be that some approaches are more effective than others for increasing performance on the state assessments. It is also feasible that when the SAGE program is administered in specific ways, students in primary grades are given an advantage that is not as meaningful for students in later grades. Study results raise the possibility that SAGE has a more positive effect on African American students than on white students. The same finding was not evident when students from low-income families were compared with students from high-income families.

Example Sites

Various school districts throughout Wisconsin are currently implementing SAGE. A current site list is available at http://dpi.wi.gov/sage/schools.html.

Contact Information

Janice S. Zmrazek SAGE Program Coordinator Wisconsin Department of Public Instruction

- 488 -

http://dpi.wi.gov/sage/schools.html

P.O. Box 7841 Madison, WI 53707-7841 phone: (608) 266-2489 email: janice.zmrazek@dpi.state.wi.us

Available Resources

The SAGE Web site, http://dpi.wi.gov/sage/index.html, provides detailed information about SAGE guidelines, regulations, budget, and history.

Bibliography

Maier, Peter, Alex Molnar, Stephen Percy, Phillip Smith, John Zahorik, Greg Giglio, Sally Hochstein, Lisa Radtke, Laura Roskos, Mark Schill, and Kathy Shields, First Year Results of the Student Achievement Guarantee in Education Program, Milwaukee, Wisc.: Center for Urban Initiatives and Research, University of Wisconsin-Milwaukee, December 1997. Molnar, Alex, Philip Smith, John Zahorik, Anke Halbach, Karen Ehrle, Lawrence M. Hoffman, and Beverly Cross, 2000-2001 Evaluation Results of the Student Achievement Guarantee in Education (SAGE) Program, Milwaukee, Wisc.: School of Education, University of Wisconsin-Milwaukee, December 2001. Molnar, Alex, Philip Smith, John Zahorik, Karen Ehrle, Anke Halbach, and Barbara Kuehl, 1999-2000 Results of the Student Achievement Guarantee in Education (SAGE) Program Evaluation, Milwaukee, Wisc.: School of Education, University of Wisconsin-Milwaukee, December 2000. Molnar, Alex, Philip Smith, John Zahorik, Lee Breese, Karen Ehrle, Anke Halbach, Amanda Palmer, Alan Silverman, and William Harvey, 1997-1998 Evaluation Results of the Student Achievement Guarantee in Education (SAGE) Program, Milwaukee, Wisc.: School of Education, University of Wisconsin-Milwaukee, December 1998. Molnar, Alex, Philip Smith, John Zahorik, Lee Breese, Karen Ehrle, Anke Halbach, Amanda Palmer, and Betsy Schoeller, 1998-1999 Evaluation Results of the Student Achievement Guarantee in Education (SAGE) Program, Milwaukee, Wisc.: School of Education, University of Wisconsin-Milwaukee, December 1999. Webb, Norman L., Robert H. Meyer, Adam Gamoran, and Jianbin Fu, Participation in the Student Achievement Guarantee in Education (SAGE) Program and Performance on State Assessments at Grade 3 and Grade 4 for Three Cohorts of Students-Grade 1 Students in 1996-97, 1997-98, and 1998- 99, Milwaukee, Wisc.: Wisconsin Center for Education Research, February 9, 2004.

Last Reviewed

September 2006

Syracuse Family Development Research Program

Program Info

Outcome Areas Healthy and Safe Children Children Succeeding in School

- 489 -

mailto:janice.zmrazek@dpi.state.wi.us

http://dpi.wi.gov/sage/index.html

Indicators Students performing at grade level or meeting state curriculum standards Children and youth not engaging in violent behavior or displaying serious conduct problems

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Community-Based Service Provider Home Visiting Type of Service Case Management Family Support Health Education Instructional Support Parent Education Type of Outcome Addressed Behavior Problems Cognitive Development/School Performance Juvenile Justice Mental Health Violent Behavior

Evidence Level Promising

Program Overview

The Syracuse Family Development Research Program (FDRP) was a comprehensive early childhood program designed to improve child and family functioning through home visitation, parent training, and individualized day care. The program operated in a single site in Syracuse, New York, between 1969 and 1976, and has undergone long-term assessments of its effects on participants. The FDRP provided a full range of education, nutrition, health and safety, and human services resources to participating families beginning prenatally and lasting until children reached elementary school age. The program targeted economically disadvantaged families in order to improve children’s cognitive and emotional functioning, create a positive outlook among the children, and decrease juvenile delinquency. Child Development Trainers (CDTs) visited each family weekly and focused on increasing family interaction, cohesiveness, and nurturing. In the Children’s Center (for day care), infants were assigned to a caregiver for attention, cognitive and social games, sensorimotor activities, and language stimulation. The preschool program supported child-chosen opportunities for learning and peer interaction in a space-oriented framework, i.e., specific areas of the Center were designated for specific types of activities.

Program Participants

The FDRP targeted young, African-American, single-parent, low-income families. Mothers were in the last trimester of their first or second pregnancy.

- 490 -

Evaluation Methods

Between 1969 and 1971, the FDRP recruited 108 families with incomes of less than $24,000 per year (in 2003 dollars) into the program in the last trimester of each mother’s pregnancy. Mothers had less than a high school education and a history of either no paid work or semiskilled work. Their average age was 18 years, and more than 85 percent were heads of single-parent households. The majority of families served were African-American. In 1982, Honig, Lally, and Mathieson assessed 37 FDRP students in kindergarten and 20 FDRP students in the first grade. Comparison group children were selected from each of the 15 schools in the city where FDRP students were enrolled. A comparison child was chosen for each FDRP child and was matched for age, sex, race, and socioeconomic status of the family, classroom, and teacher. Child social interactions in the classroom were assessed using the Emmerich Observer Ratings of Personal- Social Behaviors (during which the child was observed for a 20-minute period). In addition, children were assessed using Stanford-Binet IQ scores. A 13-year longitudinal study by Lally, Mangione, and Honig (1988) followed the FDRP students into their teen years to assess the program’s effects on school performance and juvenile delinquency. The sample consisted of 65 FDRP and 54 comparison group children. Data sources included school records, court records, and probation department records. School performance was assessed in each group by the number of failing grade-point averages, the number of students performing at a C-average grade or better, and the number of school absences. Results were examined separately for male and female students. Juvenile delinquency was determined by the number of children processed as probation cases by the County Probation Department and the severity of delinquent acts committed by offenders. In addition, the results from the juvenile delinquency analysis were expanded by Aos, Barnoski, and Lieb (1998) and by Aos, Phipps, Barnoski, and Lieb (2001). These studies computed the prevalence rate (number of juveniles who offended) and incidence rate (number of offenses committed by those youths who offended at all) of crimes, and conducted a cost-benefit analysis of the FDRP.

Key Evaluation Findings

The kindergarten portion of the study by Honig, Lally, and Mathieson (1982) found that:

• FDRP children exhibited social-emotional functioning significantly superior to that of control children. As measured by the Emmerich Observer Ratings:

o FDRP children were more involved, relaxed, dominant, energetic, social, independent, purposeful, flexible, and affectionate to others than control children.

o Control group children were more restrained, self-centered, passive, unstable, timid, destructive, socially insecure, and unhappy than treatment children.

• Significantly more of the FDRP children than the comparison children attained an IQ score above 89.

The first-grade portion of the study by Honig, Lally, and Mathieson (1982) found that:

• FDRP children continued to behave positively toward other children, but their behavior toward teachers had changed. FDRP children displayed significantly more positive and negative behavior toward adults than did comparison children, including:

o Positive and negative bids for attention, bossiness, physical aggressiveness toward adults and towards property, information-seeking behavior, and defiance in response to frustration or threats.

• Compared with FDRP children, the comparison group children more frequently expressed criticism of adults and other children, smiled and/or laughed, and made aggressive threats towards other children.

- 491 -

In 1988, Lally, Mangione, and Honig assessed school performance and found that:

• By the eighth grade, 0 percent of the FDRP females had a failing average in school compared with 16 percent of comparison group females.

• By the eighth grade, 76 percent of the FDRP females versus 47 percent of comparison group females were performing at a C average or better.

• School attendance data over four years showed no significant differences between treatment and control group females for the first two years. By year three, 14 percent of the FDRP females versus 50 percent of the comparison group females had more than 20 school absences during the previous year, and by year four, 0 percent of the FDRP females versus 31 percent of the comparison group females had more than 20 school absences.

• No significant differences appeared between males in the FDRP and comparison groups.

Lally, Mangione, and Honig (1988) also assessed delinquency outcomes and found that:

• Significantly more comparison children (three) than FDRP children (zero) committed violent crimes.

• Significantly fewer FDRP children (6 percent) as compared with control children (22 percent) were processed as probation cases.

o Of the four FDRP children with probation records, three were charged with simple unruliness, and one was charged with one-time juvenile delinquency.

o Of the 12 comparison group children with probation records, five were chronic offenders. Charges included acts of unruliness, delinquency, mischief, larceny, burglary, robbery, physical assault, and sexual assault.

• The total criminal justice system costs for the FDRP delinquency cases was $12,111 versus $107,192 for the comparison group children. In other words, the total cost of crimes per FDRP child (n = 65) was $186, and the total cost of crimes per comparison group child (n = 54) was $1,985.

Aos, Barnoski, and Lieb’s (2001) analysis of the same data calculated that:

• The FDRP children had a significantly lower prevalence rate of crimes than the comparison group children, i.e., the number of youths who committed any crimes was lower in the FDRP group than in the comparison group.

• No significant differences were found between the groups for the incidence rate of crimes, i.e., there was no difference in the number of offenses committed by the youths who committed any crimes at all.

Probable Implementers

Public and private child care programs and elementary schools

Funding

The project was federally funded through the Office of Child Development under a U.S. Department of Health, Education, and Welfare grant.

Implementation Detail

Program Design

During weekly home visits, CDTs taught parents ways to nurture child development and play games during daily routines. CDTs offered positive support to mothers, and helped families to deal with

- 492 -

emotional, social, financial, and nutritional problems. The CDTs served as liaisons between the family and community support services, and helped families identify and use such services on their own. In addition, a library of toys and books was created and shared with families. Each CDT visited 15 families each week. The Children's Center served children from all parts of the city, and children were picked up by a bus driver each morning and delivered home in the afternoon. Infants six months to 15 months of age were cared for in an "Infant-Fold" area in the Children's Center five half-days per week. Four infants were assigned to each caregiver, and caregivers worked in pairs, with a group size limited to eight infants. Babies from 15 to 18 months of age were placed in a special transition group with full-day care five days a week. Self-feeding was encouraged, and larger play spaces with sliding cabinets encouraged the toddlers' autonomy and freer choice of materials. Caregivers continued to provide these babies with comforting and emotional support. Children from 18 to 60 months of age spent their days in a family-style environment, with freedom of choice and access to the four major areas of the Children's Center: "large muscle," "small muscle," "sense perceptions," and "creative expression/snack." Additionally, the children had a large variety of wheeled toys and equipment in a large gymnasium, and use of an enclosed outdoor play area.

Curriculum

The mother, rather than the child, was the focus of the CDT's attention and teaching during home visits. The CDTs taught families sensorimotor games, language interactions, and learning tasks appropriate to each child's developmental level. They also provided nutrition information, helped parents learn how to engage their children in cognitive and language activities, and helped mothers observe their child's development and adapt games to the appropriate level. In addition, CDTs offered positive support and encouragement to mothers, actively helped the parents to fulfill their personal aspirations, and encouraged parents to take an active role in their children's classroom at school. Caregivers in the Children's Center employed creative methods to incorporate sensorimotor games, fine and gross motor activities, sensory stimulation and activities, and language and book experiences in children' daily activities. Activities were modified to appropriately match each child's need. The caregivers worked to maintain positive relationships with parents. Parents were made to feel welcome when they visited their children at play or came to share lunch with a preschool group. Caregivers prepared a daily "Memo to Mommy" that was safety-pinned to each child's clothing and contained messages highlighting the child's newly formed skills, friendships the child had made, and other such positive developments.

Staffing

A two-week intensive training session was provided every fall for all staff, including caregivers, child development trainers, researchers, testers, secretarial staff, the cook, bus drivers, and driver aides. These sessions were used for staff motivation, for refining the staff's child-observation skills, and to teach staff about child development processes such as sensorimotor and preoperational activities. Throughout the year, weekly staff meetings were held to discuss the progress, problems, and strengths of a particular child. In addition, daily learning sessions for staff were scheduled during children's naptime. These short sessions at naptime permitted staff to generate or to share their own innovative curricular ideas or to construct learning games based on ideas suggested during training sessions. Weekly case conferences were held among caregivers, child development trainers, and supervisory staff to report on and discuss work with participating families. Goals for the children were reviewed, and discussions were conducted on how to best achieve these goals.

- 493 -

Issues to Consider

This program received a "promising" rating. While results indicated superior school performance for FDRP female participants, male students did not show any lasting program effects. Significantly fewer FDRP youths than comparison group youths committed acts of juvenile delinquency, but the FDRP reduced the number of violent crimes committed by participants by an amount that was only marginally significant. Another key issue to consider is the result of a cost analysis of the program. Aos, Barnoski, and Lieb’s (1998) cost-benefit analysis of the FDRP calculated that in 1997 dollars:

• The cost of the program per participant was $18,037.

• The total criminal justice costs avoided per participant was $3,953.

• The total crime victim costs avoided was $3,842.

• The total net cost to taxpayers per FDRP participant was ($18,037 - $3,953 - $3,842) = $10,242.

These findings demonstrate that the program’s savings to government do not outweigh or break even with the program’s costs.

Example Sites

Syracuse, New York

Contact Information

Dr. Alice S. Honig Professor Emerita of Child Development Syracuse Family Development Research Program (FDRP) 202 Slocum Hall Syracuse University Syracuse, NY 13244 Ph: (315) 443-4296 Fax: (315) 443-9402 Email: ahonig@mailbox.syr.edu

Available Resources

For training, technical assistance, and materials, contact Dr. Alice Honig at FDRP.

Bibliography

Aos, Steve, P. Phipps, Robert Barnoski, and Roxanne Lieb, The Comparative Costs and Benefits of Programs to Reduce Crime: Version 4.0, Olympia, Washington: Washington State Institute for Public Policy, 2001. Aos, Steve, Robert Barnoski, and Roxanne Lieb, Watching the Bottom Line: Cost-Effective Interventions for Reducing Crime in Washington, Olympia, Washington: Washington State Institute for Public Policy, 1998. Honig, Alice S., J. Ronald Lally, and Deborah H. Mathieson, "Personal and Social Adjustment of School Children after Five Years in the Family Development Research Program," Child Care Quarterly, Vol. 11, No. 2, 1982, pp. 136-146.

- 494 -

mailto:ahonig@mailbox.syr.edu

Lally, J. Ronald, Peter L. Mangione, and Alice S. Honig, "The Syracuse University Family Development Research Program: Long-Range Impact on an Early Intervention with Low-Income Children and Their Families," in D. R. Powell and I. E. Sigel (eds.), Parent Education as Early Childhood Intervention: Emerging Direction in Theory, Research, and Practice. Annual Advances in Applied Developmental Psychology, Volume 3, Norwood, N.J.: Ablex Publishing Corporation, 1988.

Last Reviewed

January 2013

Talent Development Secondary

Program Info

Outcome Areas Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards

Topic Areas Age of Child Adolescence (13-18) Type of Setting Middle School High School Type of Service Instructional Support Youth Development Type of Outcome Addressed Cognitive Development/School Performance

Evidence Level Promising

Program Overview

The Talent Development Secondary model seeks to enhance student achievement by raising teacher expectations and youth's own expectations for themselves. The goal of the model is to change the school climate by reorganizing the school into smaller learning communities. In these learning communities, students share a common set of peers and teachers across their four years of high school, and course curricula are designed around a common, career-related theme. Teachers are expected to share common planning time and are offered professional development opportunities. Talent Development Secondary makes assistance available to students who need extra help through after-hours "twilight school" and replacement of elective courses with supplemental math and reading courses. For high schools, the Talent Development Secondary model also includes a "Ninth Grade Success Academy" with a freshman seminar that incorporates discussions on peer relations, goal setting and study skills, two additional courses intended to prepare students for high school, and a double-dose of math and English courses.

- 495 -

Program Participants

Talent Development Secondary offers a model for middle schools and high schools.

Evaluation Methods

Four studies of Talent Development Secondary schools have met PPN criteria for evidence of a promising program. Balfanz, Legters, and Jordan (2004) studied the impacts of the Talent Development Secondary 9th grade instructional program on student math and reading test performance. The study compared three Talent Development Secondary intervention high schools with three comparison schools. The researchers chose comparison schools that were statistically equivalent on demographic and prior achievement variables. All schools were located in Baltimore. The 9th grade instructional program is part of the Talent Development Secondary Ninth Grade Success Academy and includes a double-dose of math and English courses for students for the entire 9th grade year, three courses aimed at helping students overcome poor preparation for high school (Strategic Reading, Transition to Advanced Mathematics, and Freshman Seminar), and 25 to 30 hours of course-specific professional development for teachers. Twenty classes across the three schools participated in the Strategic Reading course, and 16 classes participated in the Transition to Advanced Mathematics course. The comparison schools also provided their 9th grade students with a double-dose of math and English for the entire school year and used a small learning community format for their 9th grade class. The effective contrast, then, between Talent Development Secondary and comparison schools was the continuation of small learning format into grades 10-12, the three courses intended to prepare 9th graders for high school, and Talent Development Secondary-specific professional development for teachers. Students in the study were tested on math and reading achievement, using the Comprehensive Test of Basic Skills (CTBS)-5 Terra Nova achievement test in February and May of the school year when students were in 9th grade. These scores were compared against the students' performance on the same test administered in 8th grade. One intervention school was dropped from the math score analysis because too few teachers participated in the program. Kemple and Herlihy (2004) studied the impacts of the Talent Development Secondary model over the first three years of implementation at five high schools in a large urban school district. The study looked at impacts on student attendance, student course credits earned, and student promotion from 9th to 10th grade and measured the change in these outcome variables between the historical averages three years before the Talent Development Secondary model was introduced and the average outcome in the three years following the model's introduction. These changes were compared with changes in seven matched comparison schools over the same time period. Outcomes were measured in the spring of students' 9th grade year. The comparison students were statistically equivalent to the intervention schools in ethnicity and the study outcome variables prior to the start of the intervention. On average, students in the Talent Development Secondary intervention schools were 82 percent black and 16.8 percent Latino, with a 67 percent attendance rate and 52.3 percent promotion to 9th grade. The average number of students in 9th grade at the study schools was 667. Kemple et al. (2005) used the same methods as the above study to examine the impact of the Talent Development Secondary model among students attending one of five high schools in Philadelphia, using six same-district comparison schools. Students in the Talent Development Secondary schools and the matched comparison schools were statistically equivalent in ethnicity, SAT-9 test scores, attendance rates, credits earned in the prior year, and rates of promotion to 10th grade. As in the above study, for students in the intervention and matched schools, the researchers compared the difference in three-year pre-intervention average just before students began the intervention and the three-year average just after they started the intervention for the outcomes of attendance, course credits, and rates of promotion to 10th grades. Outcomes are reported for first-time 9th grade students and follow students through their first three years in high school. Thus, the three-year average is calculated for three cohorts of students, and the study period allowed each cohort to attend three years of high school.

- 496 -

Herlihy and Kemple (2004) examined the impact of the Talent Development Secondary model in 11 middle schools in a northeastern urban school district. The middle school model includes the smaller learning community approach with climate change and teacher support elements, but does not include the Talent Development Secondary Ninth Grade Success Academy or twilight school. This analysis compared the average of outcomes of the three years just before intervention began to outcomes at the first, second, and third years following the start of the intervention. Herlihy and Kemple report outcome for six of the intervention schools, because the other five intervention schools had not been implementing Talent Development Secondary long enough to calculate the three-year post period average. The difference between the pre- and post-scores at the intervention schools are compared with the difference in 18 matched comparison schools. Students in the intervention schools were 81.5 percent black, 11.2 percent Latino, and 4.4 percent white, with an 84.3 percent attendance rate and with 97.5 percent promoted to 9th grade. Matched schools were similar to intervention schools in ethnicity and test achievement scores prior to the intervention. Students were compared on test scores, attendance, and promotion to 9th grade.

Key Evaluation Findings

Balfanz, Legters, and Jordan (2004) found that 9th grade students who attended Talent Development Secondary intervention schools showed significantly greater improvement in their math (18 percent of one standard deviation higher) and reading test scores (28 percent of one standard deviation higher) than the comparison students. Kemple and Herlihy (2004) found that students who attended schools that implemented the Talent Development Secondary model showed significant improvement, compared with matched students, in:

• Attendance: Intervention students' attendance increased by 2.8 percent, while matched comparison students' attendance rate increased by 0.2 percent.

• Chronic absenteeism: There was a significant reduction (-5.2 percent) in the percent of intervention students who had an attendance rate of 80 percent or lower, compared with the matched students (-1.7 percent).

• Regular attendance: There was a significant increase (5.4 percent) in the percent of intervention students who had an attendance rate of 90 percent or higher, compared with the matched students (0.2 percent).

• Course credits in English, Algebra and math: Students in the intervention group showed a significant increase in English, Algebra, and math course credits earned, compared with the matched students.

• Earning four or more credits: There was a significant increase (7.9 percent) in the percentage of intervention students who earned four credits or more in one academic year, compared with the matched students (3 percent).

• Core academic credits: There was a significant increase (12.8 percent) in the percentage of intervention students who passed a core academic curriculum, compared with the matched students (4.2 percent).

Kemple et al. (2005) found that, after students' first year in high school, those who attended schools that implemented the Talent Development Secondary model showed significant improvement relative to matched students in:

• Attendance: Students in the intervention group had an average increase in attendance rate 5.1 percentage points above that of matched school students.

• Chronic absenteeism (attendance rate of 80 percent or lower): Students in the intervention group decreased chronic absenteeism by 11 percentage points more than matched school students.

- 497 -

• Regular attendance (attendance rate of 90 percent or higher): Students in the intervention group increased regular attendance by 7.6 percentage points more than matched school students.

• Total course credits earned: Students in the intervention increased their average total number of course credits earned by 0.67 more credits than matched school students.

• Earning 5 or more credits: There was a significantly greater increase in the percentage of students at intervention schools who earned at least 5 academic credits (difference of 7 percent over matched school students).

• Core academic credits: There was a significantly greater increase in the percentage of students at intervention schools who passed a core academic curriculum (difference of 8.2 percent over matched school students).

• Course credits in English: There was a significantly greater increase in the percentage of students at intervention schools who earned at least one credit in English (difference of 8.6 percent over matched school students).

• Course credits in math: There was a significantly greater increase in the percent of students at intervention schools who earned at least one credit in math (difference of 11.6 percent over matched school students).

• Course credits in Algebra: There was a significantly greater increase in the percentage of students at intervention schools who earned at least one credit in Algebra (difference of 24.5 percent over matched school students).

Kemple et al. (2005) also found that after students' second year in high school, those who attended schools that implemented the Talent Development Secondary model showed significant improvement relative to matched students in:

• 10th grade enrollment: There was a significantly greater increase in students in the intervention schools who were enrolled in 10th grade (difference of 8 percent over matched school students).

• 9th grade enrollment: There was a significantly greater reduction in students in the intervention schools who were still enrolled in 9th grade their second high school year (difference of 8.7 percent from matched school students).

• Total course credits earned: Students in the intervention increased their average total number of course credits earned by 0.85 credits more than matched school students.

• Course credits in English: There was a significantly greater increase in the percentage of students at intervention schools who earned at least two credits in English (difference of 13.7 percent over matched school students).

After the students' third year in high school, Kemple et al. (2005) found that students who attended schools that implemented the Talent Development Secondary model showed significant improvement relative to matched students in:

Herlihy and Kemple (2004) found that middle school students who attended Talent Development Secondary schools showed significant improvement in the third year after the intervention began compared with matched students in:

• Promotion to 9th grade: Talent Development Secondary schools had significantly greater increase in students who were promoted to 9th grade than did matched schools.

- 498 -

• Regular attendance: There was a significant increase in the percentage of intervention students who had an attendance rate of 90 percent or higher, compared with matched students.

Probable Implementers

Middle and high school administration across the United States wishing to reform or reorganize their schools to improve school climate and student performance.

Funding

Interested schools may contact a Talent Development Secondary regional manager to learn about costs associated with working with the Talent Development Secondary team and to set up the model in their school.

Implementation Detail

Program Design

The Talent Development Secondary model prescribes comprehensive reform and school-wide reorganization to improve student performance in school. Central to the model are small learning communities in which students in a school are grouped together and take classes together with a set of teachers. Teachers in each small learning community share a common planning period and receive professional development. The Talent Development Secondary model also emphasizes extra assistance to help students catch up to their expected grade level performance.

Staffing

Talent Development Secondary staff work together with school staff to reform the school and offer ongoing implementation assistance.

Curriculum

The high school Talent Development Secondary model includes a Ninth Grade Success Academy, which is a small learning community comprising only 9th grade students that includes a freshman seminar and catch-up courses in math and English. Beyond 9th grade, schools are organized into small learning communities centered on a common career theme. Teachers in small learning communities receive professional development and share a common planning period, allowing for interdisciplinary teacher teams. The model emphasizes growth in literacy and math skills. The middle school Talent Development Secondary model also uses a small learning community design, with teachers across disciplines sharing a common planning period. The model emphasizes growth in math, literacy, science, and history skills and offers opportunities for middle school youth to explore career options.

Issues to Consider

The above studies found evidence that the Talent Development Secondary model impacts student behavior (attendance) and performance (credits earned, test scores, and promotion rates). One issue to note is that these studies measure multiple outcomes, which increases the possibility that a positive outcome is found by chance. Additionally, these studies relied on matched schools for comparison, which could potentially bias the positive findings, if matched school students are systematically different in unknown ways that affect the study outcomes. The studies found no differences between intervention students and matched students on variables researchers were able to observe, but differences might exist in student characteristics or school characteristics, such as teacher motivation or school climate, that researchers did not measure or were not able to observe. Some of the studies attempted to strengthen the validity of their findings by using the averages across three years of

- 499 -

students to correct for year-to-year variation in student outcomes. However, Herlihy and Kemple (2004) compared the three-year "pre-intervention" average to single-year intervention cohorts; this study found only two significant results in one cohort year out of multiple outcomes tested across multiple years.

Example Sites

The Talent Development Secondary model has been implemented in high schools and middle schools in 15 states.

Contact Information

Maxine Wood Chief Operating Officer Talent Development Secondary Phone: 410-516-6423 Email: mwood@csos.jhu.edu Kathy Nelson Director of Implementation Talent Development Secondary Middle Grades Phone: 410-516-6431 Email: knelson@csos.jhu.edu

Available Resources

Information about Talent Development Secondary can be found on its website: http://www.talentdevelopmentsecondary.com. The website includes an overview of the high school and middle school programs, as well as information for contacting a regional manager to learn more about partnering with Talent Development Secondary.

Bibliography

Balfanz, Robert, Nettie Legters, and Will Jordan, "Catching Up: Effect of Talent Development Ninth Grade Instructional Interventions in Reading and Mathematics in High-Poverty Schools," NASSP Bulletin, Vol. 88, No. 641, 2004. Herlihy, Corrine M., and James J. Kemple, The Talent Development Middle School Model: Context, Components, and Initial Impacts on Students' Performance and Attendance, New York: MDRC, December 2004. Kemple, James J., and Corinne M. Herlihy, The Talent Development High School Model: Context, Components, and Initial Impacts on Ninth-Grade Students' Engagement and Performance, New York: MDRC, June 2004. Kemple, James J., Corinne M. Herlihy, and Thomas J. Smith, Making Progress Toward Graduation: Evidence from the Talent Development High School Model, New York: MDRC, May 2005.

Last Reviewed

June 2012

- 500 -

mailto:mwood@csos.jhu.edu

mailto:knelson@csos.jhu.edu

http://www.talentdevelopmentsecondary.com/

Targeted Reading Intervention (TRI)

Program Info

Outcome Areas Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards

Topic Areas Age of Child Early Childhood (0-8) Type of Setting Elementary School Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance

Evidence Level Promising

Program Overview

Targeted Reading Intervention (TRI) is a classroom reading intervention. The program aims to improve reading skills among kindergarten and first-grade students who are struggling in this area through one-on-one instruction. In the classroom setting, teachers focus on one struggling student at a time, providing one-on-one instruction to the target for several weeks before moving on to work with another struggling student. The main goal of TRI is to improve reading comprehension while also working on word identification, fluency, and vocabulary (Amendum, Vernon-Feagans, and Ginsberg, 2011).

Program Participants

Seven schools in five school districts in the southwestern United States participated in the evaluation of TRI. Across the seven schools, 264 students in 43 kindergarten and first-grade classrooms participated in the evaluation.

Evaluation Methods

Amendum, Vernon-Feagans , and Ginsberg (2011) evaluated the impact of TRI with a distance technology component in rural school districts in the southwestern United States. Schools within each district were paired based on school size, racial and ethnic composition, participation in Reading First (another program aimed at improving reading among struggling students), and percentage of students eligible for reduced-price lunch. Then, the schools within each pairing were randomized to the TRI group or the control (nonintervention) group. One school randomized to TRI dropped out of the evaluation due to technology issues, leaving seven schools. Students were eligible to participate in the evaluation if they were not diagnosed with a severe disability and if they spoke some conversational English. Teachers along with TRI literacy coaches determined further eligibility as follows. First, teachers assessed reading skills using all state- and district-mandated assessments of emergent reading skills. Then, TRI coaches guided teachers in using a TRI screening instrument to rank students' progress. The TRI screening instrument asked teachers to rank students from low to

- 501 -

high performance on state- or district-mandated assessments of reading skills. The teacher also rated whether the student was benefiting from regular classroom instruction and whether the student was below, at, or above grade level. Using the rankings, teachers formed two groups of students — focal (below grade level and struggling to learn to read) and nonfocal (at or above grade level and benefiting from regular classroom instruction). From among these students, five focal and five nonfocal students were randomly selected to participate in the evaluation. Focal students in the intervention schools received TRI. Nonfocal students in the intervention schools, and focal and nonfocal students in the control schools, received regular classroom instruction. There were 112 students in the intervention focal group and 63 students in the control focal group. Overall, there were 237 students in the intervention schools and 127 students in the control schools. Children were assessed in the fall and spring on four subtests of the Woodcock-Johnson Diagnostic Reading Battery, III. The subtests included Word Attack (a measure of phonetic skills), Letter/Word Identification (a measure of word identification skills), Passage Comprehension (a measure of symbolic understanding and passage comprehension), and Spelling of Sounds (a measure of spelling ability). Performance by focal children at intervention and control schools was compared using a hierarchical model that accounted for nesting of students within classrooms. There was a greater proportion of white students in the intervention focal group than in the control focal group of students (39.3 percent and 17.5 percent, respectively). There were no differences in pretest scores among students in the intervention focal group and control focal group. The analysis also controlled for grade level (kindergarten versus first grade), student race, student gender, and mother's education level.

Key Evaluation Findings

On all four subtests of the Woodcock-Johnson Diagnostic Reading Battery, III — Word Attack, Letter/Word Identification, Spelling of Sounds, and Passage Comprehension — focal students in the intervention group performed significantly better in the spring than focal students in the control group.

Probable Implementers

Kindergarten and first-grade teachers

Funding

Funding for the evaluation was provided via IES Grant for the National Research Center for Rural Education Support. The distance technology module portion of the TRI intervention costs approximately $1,300 per classroom.

Implementation Detail

Program Design

The TRI model was developed to assist teachers in improving reading skills among struggling students in kindergarten and first-grade classrooms. The intervention was designed with a focus on rural, low- income communities. Teachers implementing TRI work one-on-one with students struggling with reading while the other students in the class work on reading independently or with a teaching assistant. During the 15- to 20-minute one-on-one sessions, the teacher and student work through lessons that focus on three components of TRI: Re-Reading for Fluency, Word Work, and Guided Oral reading. Teachers' lessons are informed by daily diagnostic feedback from the day's lesson, such that the lessons are tailored to the student's specific needs. Teachers also work with TRI literacy coaches via web videoconferences. TRI literacy coaches observe teachers working with students struggling with reading and provide real-time feedback on implementing the TRI strategies (Amendum, Vernon- Feagans, and Ginsberg, 2011).

- 502 -

Staffing

TRI is implemented by kindergarten and first-grade teachers. The one-on-one sessions may require a teaching assistant to work with the other students in the classroom.

Curriculum

TRI focuses on three components during the one-on-one sessions. As described by Amendum, Vernon- Feagans, and Ginsberg (2011), these components are:

• Re-Reading for Fluency: Students read a passage that they have read previously in order to develop word identification automaticity and reading fluency. The teacher may also read aloud parts of the text to demonstrate fluency.

• Word Work: The teacher works with students to manipulate and write words in order to develop phonological decoding and sight word recognition skills.

• Guided Oral Reading: The teacher assists the student in reading to improve passage comprehension and word identification.

For more information regarding the TRI curriculum, see http://www.targetedreadingintervention.org/tri-program.

Issues to Consider

This program received a "promising" rating. The 2011 evaluation of TRI had significant differences in the racial composition of students in the intervention focal and control focal groups. These differences were controlled for in the analysis. Additional evaluations of TRI have been conducted, including an evaluation of TRI with on-site literacy coach consulting. This evaluation found positive effects of the TRI program for struggling students, however, the evaluation did not meet the Promising Practices Network (PPN) evidence criteria due to a small sample size. This evaluation included two control schools and one treatment school. The randomized evaluation using distance technology presented here is the only evaluation which meets the PPN evidence criteria, including study design, effect size, and statistical significance. Note also that this evaluation was conducted by the program developer.

Example Sites

Kindergarten and first-grade classrooms in rural regions of the southwestern United States.

Contact Information

Lynne Vernon-Feagans The William C. Friday Distinguished Professor 301K Peabody Hall #3500 The University of North Carolina Chapel Hill, North Carolina 27599 Phone: 919-843-5623 Phone: 919-966-5484 Fax: 919-962-1533 Email: lynnevf@email.unc.edu

- 503 -

http://www.targetedreadingintervention.org/tri-program

mailto:lynnevf@email.unc.edu

Available Resources

More information on TRI can be found at http://www.targetedreadingintervention.org/

Bibliography

Amendum, S. J., L. Vernon-Feagans, and M. C. Ginsberg, "The Effectiveness of a Technology Facilitated Classroom-Based Early Reading Intervention," The Elementary School Journal, Vol. 112, 2011, pp. 107-131. National Institute of Child Health and Development, Reporting of the National Reading Panel. Teaching Children to Read: An Evidence-Based Assessment of the Scientific Research Literature of Reading and Its Implications for Reading Instruction: Reports of the Subgroups. Washington, D.C.: U.S. Government Printing Office, 2000. Snow, C. E., M. S. Burns, and P. Griffin, eds., Preventing Reading Difficulties in Young Children: A Report of the National Research Council, Washington, D.C.: National Academy Press, 1998.

Last Reviewed

December 2012

Team Accelerated Instruction: Math

Program Info

Outcome Areas Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards

Evidence Level Promising

Program Overview

The Team-Accelerated Instruction: Math program (TAI) incorporates cooperative learning and individualized instruction in math education. TAI allows children to progress on an individual basis, and

- 504 -

http://www.targetedreadingintervention.org/

it teaches cooperation by forming teams of students who can help each other to achieve team goals. TAI combines interactive instruction by teachers with cooperative team learning to accelerate the achievement of all students, maximize teaching and learning time, enhance student motivation and positive attitudes toward math, and improve students' social interactions.

Program Participants

Students in grades 3 through 6, or older students who are not ready for algebra.

Evaluation Methods

Slavin and Karweit (1984) studied TAI in a yearlong randomized controlled experiment involving 588 ninth-grade students in 16 urban schools in Philadelphia. Teachers were randomly assigned to one of four groups: (1) Mastery (involving teaching, individual worksheet-based study, a formative quiz (to assess how student learning is proceeding), corrective instruction, and a summative quiz (to assess knowledge learned at the end of a unit); (2) Teams (involving teaching, worksheet-based study within heterogeneous four-member teams, and a quiz); (3) Teams + Mastery (involving teaching, team study, a formative quiz, corrective instruction within teams, and a summative quiz); and (4) the control group, which used Focused Instruction (involving teaching, individual worksheet-based study, and a quiz). All four groups used the same curriculum materials and schedule of instruction, i.e., only the style of the delivery of instruction differed. An analysis of baseline differences among the four groups at the outset of the study did not indicate any significant differences. From an initial sample of 1,092 students, both pretest and posttest achievement data were available for 588 of those students. Analyses of pretest scores of students who did not complete posttests revealed no significant differences in achievement among the four groups, suggesting that the 504 students who dropped out of the study were evenly distributed among the four groups. Among the sample of students, 76 percent were African-American, 19 percent were white, 6 percent were Hispanic, and 1 percent was Asian. Outcome measures included a shortened version of the Mathematics Computation and Concepts & Applications subscales of the Comprehensive Test of Basic Skills (CTBS). TAI was also assessed in a quasi-experiment of TAI involving 1,371 students in 59 third-, fourth-, and fifth-grade math classes in a suburban Maryland school district (Slavin et al., 1984a); 719 students in 31 classes from five schools were assigned to TAI, and 652 students in 28 classes from three schools were assigned to the control group. Participation in the 24-week TAI program was voluntary, and treatment and control group students were matched on California Achievement Test average pre-test scores. Sixty-three students in the TAI classes and 50 in the control classes were receiving special education services and were designated as being academically handicapped. The outcomes studied were the Mathematics Computation and Concepts & Applications subscales of the CTBS. Slavin et al. (1984b) conducted two randomized controlled experiments on TAI in a middle-class suburban Maryland school district. The first study involved 504 students in grades 3, 4, and 5 from 18 classes in six schools. Eighty percent of the students in the sample were white, 15 percent were African-American, and 5 percent were Asian. Six percent of the students were receiving special education services for a serious learning problem at least one hour per day, and an additional 17 percent were receiving other education services (such as special reading or speech instruction). The schools were randomly assigned for an eight-week period to one of three groups: 1) the TAI group; 2) an individualized instruction group (in which students worked individually, not in teams); or 3) the control group. Baseline analyses did not indicate any significant differences among the three groups on CTBS test scores. The second study involved 375 students in grades 4, 5, and 6 from 16 classes in four schools (two TAI schools and two control schools) in another suburban Maryland school district. Classes were randomly assigned for a ten-week period to TAI or a control group, and no pretest differences were found between the groups on CTBS scores. Fifty-five percent of the students were 55 white, 43 percent were black, and 2 percent were Asian. Four percent of students were receiving special education services for a serious learning problem at least one hour per day, and an additional 23 percent were receiving other education services. Students were compared on the mathematics computation subtest of the CTBS.

- 505 -

Slavin and Karweit (1985) also conducted two randomized controlled experiments of TAI. The first study was set in Wilmington, Delaware, and involved 345 students in 15 fourth-, fifth-, and sixth- grade classes. Approximately 71 percent of the students were white, 26 percent were African- American, and 3 percent were Asian. Classes and teachers were randomly assigned to one of three 18-week treatment groups: (1) Missouri Mathematics Program (MMP), a whole class, group-paced math program; (2) Ability Grouped Active Teaching (AGAT), an instructional method using within-class ability grouping; or (3) Team Assisted Individualization (TAI), which consisted of heterogeneous self- instructed learning teams. Baseline analyses found no significant differences among groups on CTBS scales. The second study involved 480 students in 22 third-, fourth- and fifth-grade classrooms in Hagerstown, Maryland. Ninety-one percent of the students were white, 7 percent were African- American, and 2 percent were Asian. Classes and teachers were randomly assigned to one of four experimental groups: (1) MMP, (2) AGAT, (3) TAI, or (4) an untreated control group in which teachers used traditional whole-class instructional methods. Pretest analyses showed no statistically significant differences among groups on CTBS Computations, but there were significant differences among the groups on the CTBS Concepts & Applications scale due to high scores in the AGAT classes and low scores in the control classes. Outcomes for the four groups were compared on the Mathematics Computations and Concepts & Applications subscales of the CTBS. Finally, Karper and Melnick (1993) conducted a randomized controlled experiment of 12 classrooms of third-, fourth-, and fifth-grade students in Derry Township School District in Hershey, Pennsylvania. Derry Township is a small, very affluent district with approximately 2,500 students. For each grade level, two classrooms were taught using TAI, and two classrooms (the control group) were taught using traditional mathematics instruction. The grade 3 sample consisted of 34 TAI and 35 control students; the grade 4 sample included 44 TAI and 44 control students; and the grade 5 sample included 46 TAI and 44 control students. With a few exceptions due to necessary placements for gifted students, students were randomly assigned to each classroom. No pretest differences were found between the TAI and control classes in grades 4 and 5; and no pretest analyses were conducted for grade 3 classes. Outcome measures included student achievement scores from the district’s standardized testing program, including scores on tests for total math aptitude and math concepts, and scores on math computation subtests.

Key Evaluation Findings

Results from Slavin and Karweit (1984) indicated the following:

• The Teams group achieved significantly higher Mathematics Computation and Concepts & Applications scores than the control group or other two treatment groups.

• There were no significant differences found for the Mastery group or the Mastery + Teams group when compared with the control group.

Slavin et al. (1984a) reported the following:

• The district-level analysis showed that the TAI group scored higher than the control group for Mathematics Computations. No significant differences were found between the two groups’ Concepts & Applications scores.

• When analyzed at the individual-student level, TAI students scored higher on the Concepts & Applications scale and the Mathematics Computations scale.

In the experiment involving two studies of Maryland students, Slavin et al. (1984b) reported the following:

• In the first study, the CTBS results indicated a marginally significant overall positive effect for the TAI and Individualized Instruction groups when compared with the control group. The TAI group scored significantly higher than the control group in Mathematics Computation achievement, but no significant differences were found between the TAI and Individualized Instruction groups.

- 506 -

• In the second study, it was found that TAI students scored significantly higher than control students in Mathematics Computation.

Slavin and Karweit (1985), in their first study, reported the following:

• The overall analysis showed a statistically significant difference among treatment groups in Mathematics Computation, but there were no differences among the groups on the Concepts & Applications scale.

• Treatment group comparisons on Computations showed that the average TAI and AGAT scores were nearly identical, but both average scores were significantly higher than the average score for the MMP group. Both the TAI and AGAT classes exceeded the MMP classes on Computations by slightly more than a full grade equivalent.

Slavin and Karweit (1985), in their second study, reported the following:

• The overall analysis showed marginally significant treatment effects on Computations. As in the first study, the AGAT and TAI students did not differ from one another in Computations scores, but both groups were superior to MMP students. All three treatment groups significantly outperformed the control group in Computations.

• The overall analysis for Concepts & Applications did not reveal any significant differences among treatment groups.

Finally, Karper and Melnick (1993) found no significant differences between treatment and control groups at any grade level in the groups’ scores on math aptitude, Concepts & Applications, or Computations.

Probable Implementers

Public and private elementary schools

Funding

There are a number of possible funding sources, including:

• Title I Comprehensive School Reform: A major funding opportunity for schools to adopt proven, whole school reform models such as Success for All.

• Title I School Improvement: Grants are awarded to schools that have failed to meet adequate yearly progress goals for two consecutive years. Schools may use the funding to implement scientifically based programs to increase student achievement.

• Title II Improving Teacher Quality: Title II funds are available to every school district. These funds can be used to pay for professional development, including the salary of a Success for All facilitator.

Implementation Detail

Program Design

TAI is based on the idea that four modifiable elements of classroom organization must be satisfactory if instruction is to be effective: (a) quality of instruction; (b) appropriate levels of instruction; (c) incentives for students to learn; and (d) time spent learning (Slavin, 1984). The TAI program was designed in an attempt to address these four elements of instruction. In TAI, students work in four- or five-member learning teams on individualized materials. To assure appropriate levels of instruction, students are placed (according to their scores on an initial test) in an individualized instruction sequence and allowed to proceed at their own rates of learning. Students within teams also assist one another with problems and take responsibility for checking each other’s work and for other routine

- 507 -

management of class work. This sort of instructional design improves the quality of instruction and maximizes the time that students spend learning by making it possible for teachers to provide direct instruction to small groups of students who are performing at similar levels. Finally, incentives to learn are guaranteed through the use of team recognition, in which students receive weekly awards based on the average performance of their teams.

Issues to Consider

The TAI program received a "promising" rating. Although the results of the program have been mixed, program evaluations indicate that TAI students have experienced some improvements in their math skills when compared with control group students. Results from Slavin et al. (1984) and Slavin and Karweit (1985) suggest that TAI may be more effective at improving students’ scores in Mathematics Computation than in Concepts & Applications. It should be noted that one or more of the TAI program developers are authors on four of the six studies cited in this program description. The TAI program was adapted several years ago to form one component of a more comprehensive mathematics program, MathWings. MathWings is one element of the Roots & Wings whole school reform model, which consists of early learning programs, reading and language arts instruction, tutoring, family support and integrated services, social studies instruction, science instruction, and mathematics instruction. The MathWings program involves both a primary-level program for grades 1 and 2, and an intermediate program (called PowerMath) for grades 3 through 5. TAI has been incorporated into one portion of the PowerMath program. Existing evaluations of MathWings do not meet PPN criteria, so we are unable to determine whether the entire MathWings program is also promising. The results presented in this program description reflect only the TAI component.

Example Sites

Hagerstown, Maryland Hershey, Pennsylvania Philadelphia, Pennsylvania Wilmington, Delaware

Contact Information

Brent Farmer, President Charlesbridge Publishing bfarmer@charlesbridge.com Phone: 800-225-3214 Fax: 617-926-5720

Available Resources

TAI is one component of the MathWings program, which has not been rigorously evaluated. Inquire with the Success for All Foundation to learn more about the MathWings program, at http://www.successforall.com/Curriculum/MathWings.cfm.

Bibliography

Karper, Jane, and Steven A. Melnick, "The Effectiveness of Team Accelerated Instruction on High Achievers in Mathematics," Journal of Instructional Psychology, Vol. 20, No. 1, 1993, pp. 49-54.

- 508 -

mailto:bfarmer@charlesbridge.com

http://www.successforall.com/Curriculum/MathWings.cfm

Slavin, Robert E., "Component Building: A Strategy for Research-Based Instructional Improvement," Elementary School Journal, Vol. 84, 1984, pp. 255-269. Slavin, Robert E., and Nancy L. Karweit, "Effects of Whole Class, Ability Grouped, and Individualized Instruction on Mathematics Achievement," American Educational Research Journal, Vol. 22, No. 3, 1985, pp. 351-367. Slavin, Robert E., and Nancy L. Karweit, "Mastery Learning and Student Teams: A Factorial Experiment in Urban General Mathematics Classes," American Educational Research Journal, Vol. 21, No. 4, 1984, pp. 725-736. Slavin, Robert E., Marshall B. Leavey, and Nancy A. Madden, "Combining Cooperative Learning and Individualized Instruction: Effects on Student Mathematics Achievement, Attitudes, and Behaviors," The Elementary School Journal, Vol. 84, No. 4, 1984b, pp. 408-422. Slavin, Robert E., Nancy A. Madden, and Marshall Leavey, "Effects of Team Assisted Individualization on the Mathematics Achievement of Academically Handicapped and Nonhandicapped Students," Journal of Educational Psychology, Vol. 76, No. 5, 1984a, pp. 813-819.

Last Reviewed

November 2004

Teen Talk

Program Info

Outcome Areas Healthy and Safe Children

Indicators Youths abstaining from sexual activity or not engaging in risky sexual behavior

Topic Areas Age of Child Adolescence (13-18) Type of Setting Middle School High School Out of School Time Community-Based Service Provider Health Care Provider Type of Service Health Education Youth Development Type of Outcome Addressed Physical Health Teen Sex/Pregnancy

Evidence Level Promising

- 509 -

Program Overview

The Teen Talk program was developed in 1983 to 1984 to help adolescents become aware of the serious negative consequences of teenage pregnancy, the likelihood that they can become pregnant, and the benefits of and barriers to abstinence and contraceptive use. The program is based on two theories of human behavior: the Health Belief Model (viewing actions as dependent on beliefs) and Social Learning Theory (emphasizing actions as proceeding from the behaviors, thoughts, and feelings of others). It is thought that this behavioral model foundation enables Teen Talk to address and measure not only basic knowledge, but also participants' perceptions relating to pregnancy and sexual activity, issues of self-efficacy, and behavioral components. The program is predicated on the assumption that a change in participants' beliefs will translate into real-world behaviors and decision- making. During its pilot phase, the program was implemented in family planning (such as Planned Parenthood) and health-specific organizations for youth populations. In addition, the curriculum was tested within a single small independent school system consisting of one high school and one middle school. The curriculum combines large group presentations on reproductive health, physiology, and contraception with small group discussion sessions during which participants engage in role-playing and role reversal, and practice decision-making and refusal skills. This combination of approaches is designed to help the students comprehend and personalize the risks and consequences of becoming pregnant as teenagers, develop the skills needed to remain abstinent, and learn and understand more about contraception.

Program Participants

The pilot project served and studied youths aged 13 through 19 from organizations and schools in rural and urban California and Texas. Approximately 80% were low income or from inner cities. In terms of age, 29% of the students were 13 through 14, 67% were 15 through 17, and 4% were 18 through 19. The students came from varied ethnic backgrounds: 53% were Latino, 24% were African- American, 15% were white, and 8% had other ethnic backgrounds. Based on the diversity of the pilot populations, this program should be applicable to diverse and varied communities nationwide.

Evaluation Methods

A controlled study involving 1,444 teenage males and females 13 through 19 years of age was performed to compare the Teen Talk program with other sexual and pregnancy prevention programs already in place. School classrooms or individual students from community organizations were assigned randomly to classes that used either the Teen Talk curriculum or other school or community- based curricula on the same topics. All students were assessed via questionnaires at the start of the program, and 92% of the sample participated in all or part of an intervention and completed the immediate follow-up evaluation. Participants were assessed again via interview one year after program completion; 67% of the immediate follow-up sample participated in the one-year follow-up interview. Analysis of collected data was conducted along a fourfold framework, looking separately at program impact on participants who had or had not engaged in intercourse at baseline and at males and females.

- 510 -

Key Evaluation Findings

The research studies by Eisen, Zellman, and McAlister (1990 and 1992) and Eisen and Zellman (1992) found the following:

• The program was more effective for boys than for girls. Researchers hypothesize that this finding may be a direct result of the role-playing and role reversal tactics utilized in the curriculum design.

• Although both experimental and control groups showed an increase in levels of sexual and contraceptive knowledge and health beliefs following the completion of their respective programs, this improvement did not translate into changes in individuals' sexual experience and behavior at the one-year follow-up. In addition, there was no significant difference in the change of beliefs between control and experimental groups. These findings contradict expectations derived from the behavioral model foundation on which the program is based.

• The impact of the curriculum varied significantly according to various demographic characteristics of participants. For example, the results indicate that the program was least effective on Hispanics. This finding highlights the importance of tailoring the program to the specific needs and characteristics of a given population and indicates that the curriculum may not be as effective in dealing with a heterogeneous, diverse population.

One year after the program ended, of the boys who had no previous intercourse, Program Participants were significantly more likely to stay abstinent than boys who were not in the program.

• For boys who were sexually active before the program, those participating in the program were significantly more likely to have used contraception at the one-year follow-up than boys not in the program. No difference in contraceptive use was noted for boys who were sexually inexperienced before the program who later decided to become sexually active.

• Of the girls who were sexually active, those not in the program were more likely to have used contraceptives during their last sexual encounter.

• Prior exposure to sex education was linked to increased contraceptive use for all groups in the study at the one-year follow up.

Probable Implementers

Schools and/or community groups

Funding

The original research was funded with assistance from the states of Texas and California.

Implementation Detail

Program Design

• Group leaders were thoroughly trained on the program curriculum and on how to conduct effective small group discussions.

• The small group facet was utilized for activities such as role-playing and role reversal activities and discussion. This component was thought to give participants a safe forum in which to practice developing skills and discover how they may react in real-life situations. As indicated in the findings, the practice afforded by the small group sessions is a likely reason why the program proved so effective for boys.

- 511 -

Curriculum

The Teen Talk program consists of a six-session curriculum offered over a two- to three-week period for a total of 12 to 15 hours. Components include a minimum of two large group lectures addressing reproductive health, physiology, and contraception. In addition, adult facilitators lead small group sessions of six to eight students focusing on the factual realities of teenage pregnancy and parenthood, games, discussions, script writing, videos, role-playing, and role reversals. The small groups must meet for a minimum of eight hours and provide the students a forum in which to practice decision-making and refusal skills as well as give them an opportunity to explore their beliefs, feelings, and values. The sessions encourage active participation and discussion.

Staffing

A lecturer and small group discussion leaders are needed to staff the program. Family planning agency educators and school staff led the sessions for the pilot project. An intensive two-day training session trains potential staff to be group leaders and facilitators.

Issues to Consider

This program received a "promising" rating. Evaluations indicate that the program produced some positive results, however the results are somewhat inconsistent and fluctuate greatly within and between test groups. Analysis of program data was conducted along a fourfold framework, looking separately at participants who had or had not engaged in intercourse at baseline and at males and females. The results indicated that effectiveness varied widely among these four groups (female baseline virgins and nonvirgins and male baseline virgins and nonvirgins). Effectiveness was shown to vary according to other demographic characteristics as well, such as ethnicity and risk-status (for example, lower income). This would suggest that the curriculum should be tailored to the unique and specific characteristics of the given population in order to maximize effectiveness. Although in most cases the Teen Talk curriculum was shown to have a positive effect, in some instances at least as great if not greater results were found among participants in the comparison group. For example, use of an effective method of contraception at last sexual encounter among girls who had initiated intercourse after the start of the study period was 62% in the comparison group versus 38% in the experimental group (Eisen, Zellman, and McAlister, 1992). It is possible that the proven effectiveness of the comparison programs may be resulting from the fact that in many instances the comparison programs were modified from their usual structure to meet the length of the experiment curriculum. As such, the comparison programs were likely to be as in-depth and comprehensive as the experiment, although different methodologies of implementation were employed. An additional possible explanation is given in Frost and Forrest (1995), suggesting that the alternative curricula were themselves often newly instituted and innovative programs that included small group discussions and addressed decision-making issues. In addition to identified benchmarks, research (Eisen and Zellman, 1992) on Teen Talk indicates that the program generates other significant results. Through the encouragement of safe sex practices and education on sexuality, reproductive health, and the use of appropriate forms of birth control, the program impacts students' knowledge regarding sexually transmitted diseases and, in encouraging the use of condoms, reduces the number of teens exposed to such diseases. The program may also affect the emotional and psychological well-being of participants through its focus on judgment and decision- making skills.

Example Sites

Texas and California.

- 512 -

Contact Information

Shobana Ragupathy, Ph.D. Senior Research Associate Sociometrics Corporation Program Archive on Sexuality, Health and Adolescence (practitioner support) 170 State Street, Suite 260 Los Altos, CA 94022-2812 Phone: (650) 949-3282, x209 Toll free: 1 (800) 846-DISK fax: 650-949-3299 e-mail: pasha@socio.com

Available Resources

A Teen Talk program package from the Sociometrics Corporation includes a user's guide, curricula guides, handouts/overheads, sexually transmitted disease (STD) facts pamphlets, staff training and evaluation materials, videos, directory of evaluation consultants, and phone technical support for a year. Program training sessions also are available.

Bibliography

Eisen, M., G. Zellman, "A Health Beliefs Field Experiment: Teen Talk," in Miller, B., et al., eds., Preventing Adolescent Pregnancy, Newbury Park, Calif.: Sage Publications, 1992, pp. 220-264. Eisen, M., G. Zellman, and A. L. McAlister, "A Health Beliefs Model-Social Learning Theory Approach to Adolescents' Fertility Control: Findings from a Controlled Field Trial," Health Education Quarterly, Vol. 19, No. 2, 1992, pp. 229-262. Eisen, M., G. Zellman, and A. L. McAlister, "Evaluating the Impact of a Theory-Based Sexuality and Contraceptive Education Program," Family Planning Perspectives, Vol. 22, 1990, pp. 261-271. Frost, J.J., and J. Darroch Forrest, Understanding the Impact of Effective Teenage Pregnancy Prevention Programs, Family Planning Perspectives, 1995. Vol. 27, No. 5, pp. 188-195. National Academy of Sciences, The Best Intentions: The Causes, Consequences and Prevention of Unintended Pregnancy, Washington, D.C.: National Academy Press, 1995.

Last Reviewed

January 2009

Text4Health-Adolescents

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children experiencing good physical health

- 513 -

mailto:pasha@socio.com

Topic Areas Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Health Care Provider Type of Service Health Care Services Type of Outcome Addressed Physical Health

Evidence Level Promising

Program Overview

Text4Health-Adolescents is a text-messaging platform which sent automated vaccination reminders to parents of adolescents in New York City. Utilizing electronic medical records and a hospital immunization registry that receives data from the New York Citywide Immunization Registry, Text4Health delivered a series of automatic messages to parents of adolescents ages 11-18 who had not yet received standard vaccinations reminding them of their child's need for meningococcal (MCV4) and/or tetanus-diphtheria-acellular pertussis (Tdap) immunizations.

Program Participants

Parents of adolescents (ages 11-18) in a network of community-based clinics affiliated within an academic medical center in New York City, primarily serving a low-income, minority population.

Evaluation Methods

Stockwell et al. (2011) assessed the effect of Text4Health-Adolescents on receipt of one or both of two routinely recommended adolescent vaccines: meningococcal (MCV4) and tetanus-diphtheria-acellular pertussis (Tdap). Study authors utilized electronic medical records from a network of community- based pediatric clinics and a hospital immunization registry which receives data from the city of New York's Citywide Immunization Registry to determine each child's immunization status. All six pediatric clinics associated with an ambulatory care network affiliated with an academic institution in New York City were included in the study; 4 sites were designated as intervention sites and 2 were designated as control sites. Intervention and control sites were selected in order to be able to have comparably sized adolescent populations to choose from at the study baseline. At baseline, the intervention and control sites had comparable MCV4 and Tdap vaccination coverage rates. All sites served primarily minority patients receiving Medicaid and free Vaccines for Children provided vaccines. Authors performed tests in order to determine whether there were differences across the sites that may have led to the differences in observed outcomes, and found no such effect (Stockwell et al. 2011). Parents were eligible to participate in the Text4Health-Adolescents study if they had an 11- to 18- year-old child with any visit (including sick visits) at a study site within the previous 12 months who was in need of MCV4 and/or Tdap vaccine. Additionally, eligible parents needed to have a cell phone number recorded in the clinic registration system. . Over the course of four months from January 2009 to April 2009, a random sample of parents was drawn weekly from intervention sites who met the above-specified eligibility criteria. Intervention patients were then matched by gender and age to randomly selected eligible patients from control sites. Throughout the study, the group of eligible parents was updated based on current immunization status information.

- 514 -

Over the course of the 4 months of the study, there were 1,656 adolescents who needed Tdap and/or MCV vaccination at the intervention sites and 1,460 at control sites. Of these, only twenty percent of parents (625) had a cell phone number in the clinic registration system, as cell phone numbers were not routinely collected during the registration process until 2008. Authors evaluated the receipt of MCV and/or Tdap vaccination specifically, as well as the receipt of any vaccine at three time points: 4, 12, and 24 weeks (Stockwell et al. 2011).

Key Evaluation Findings

Stockwell et al. (2011) found that receipt of any vaccine was significantly higher for the Text4Health- Adolescents participants than for those in the control group at the three time points assessed (4, 12 and 24 weeks following randomization). Additionally, specific receipt of MCV4 and/or Tdap vaccine was also significantly higher among Text4Health-Adolescents participants compared with those in the control group at all three time points:

• At 4 weeks after randomization, 15.4 percent of Text4Health-Adolescents participants had received one or both of MCV4 or Tdap, compared with 4.2 percent of controls

• At 12 weeks, 26.7% versus 13.9% received one or both of MCV4 or Tdap

• At 24 weeks, 36.4% versus 18.1% received one or both of MCV4 or Tdap

Probable Implementers

Regional and local care networks; health care clinics

Funding

The Stockwell et al. 2011 study of Text4Health-Adolescents was supported by the Maternal and Child Health Bureau (Title V, Social Security Act), Health Resources and Services Administration, Department of Health and Human Services.

Implementation Detail

Text4Health-Adolescents participants' parents were sent text messages that were personalized to include the adolescent's first name, their regular clinic name, and a schedule of times when immunizations were administered at that clinic. Messages were sent in English or Spanish based upon the preferred language listed in the electronic medical record. Families were also given information on how to opt-out of future messages.

Issues to Consider

Text4Health-Adolescents received a "promising" rating. Although sites were not randomly assigned, authors demonstrated that the sites were similar on baseline demographic characteristics as well as baseline vaccine coverage rates. An additional evaluation of a Text4Health-Peds intervention was conducted among a pediatric population. Text4Health-Peds attempted to galvanize parents to attend special Haemophilus influenzae type B (HiB) vaccination sessions for children overdue for their HiB vaccination due to a national shortage that occurred following a voluntary recall of HiB vaccine from the market. While this intervention did have a significant impact on parental attendance at the special sessions, it did not impact the rates of vaccination in the total population studied, so it does not qualify for inclusion on PPN.

- 515 -

Example Sites

All six pediatric clinics associated with the New York-Presbyterian Ambulatory Care network (affiliated with a New York City academic medical center) were involved in the 2011 study.

Contact Information

Dr. Melissa Stockwell mss2112@columbia.edu phone: 212-342-5732 fax: 212-305-8819 mailing: 622 West 168th Street, VC402 NY, NY 10032

Available Resources

Additional information can be found at the study website: http://www.columbia.edu/cu/text4health/

Bibliography

Stockwell, Melissa S., Elyse Olshen Kharbanda, Raquel Andres Martinez, Marcos Lara, David Vawdrey, Karthik Natarajan, and Vaughn I. Rickert, "Text4Health: Impact of Text Message Reminder-Recalls for Pediatric and Adolescent Immunizations," American Journal of Public Health, 2012, 102:2, e15-e21.

Last Reviewed

April 2012

Triple P Positive Parenting Program

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children not experiencing physical, psychological or emotional abuse

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Community-Based Service Provider Home Visiting Type of Service Family Support Parent Education

- 516 -

mailto:mss2112@columbia.edu

http://www.columbia.edu/cu/text4health/

Type of Outcome Addressed Child Abuse and Neglect Physical Health

Evidence Level Promising

Program Overview

The Triple P Positive Parenting Program is a multilevel system of family intervention that aims to prevent severe emotional and behavioral disturbances in children by promoting positive and nurturing relationships between parent and child. According to the Triple P developers, apart from improving parenting skills, "the program aims to increase parents' sense of competence in their parenting abilities, improve couples' communication about parenting, and reduce parenting stress. The acquisition of specific parenting competencies results in improved family communication and reduced conflict that in turn reduces the risk that children will develop a variety of behavioral and emotional problems" (Sanders, Turner, et al., 2002). The program has five intervention levels of increasing intensity:

• Level 1: The first level consists of a universal media information campaign that targets all parents in a community and involves social marketing and health promotion.

• Level 2: The second level involves primary care providers offering advice and discussion to parents on children's developmental and behavioral issues.

• Level 3: Also a brief health care intervention conducted in primary care, Level 3 targets children with mild to moderate behavior difficulties and includes active skills training for parents. Moderate behavior difficulties include such problems as tantrums, whining, and fighting with siblings.

• Level 4: The fourth and penultimate level is an intensive 10-session individual or 8-session group parent training program for children with more severe behavioral difficulties, such as oppositional defiant disorder, conduct disorder, and learning difficulties.

• Level 5: The fifth level, "Enhanced Triple P," is offered to families that complete a level four Triple P intervention and who either request or are identified as eligible for further services. This level includes behavioral interventions for parents, home-based skills training, and training in other coping skills. This additional four-session intervention is available to families who are identified as at-risk for child maltreatment. These are generally families with co- occurring child behavioral issues, such as conduct disorder or learning disabilities, and parental problems such as stress and/or depression (Sanders, Turner, et al., 2002).

The program aims to engage the participating parent in the minimally sufficient intervention required in order to identify and improve parenting skills. The program allows parents to choose from a range of delivery contexts, including individual face-to-face, group, telephone-assisted, and self-directed programs (Sanders, 1999).

Program Participants

The Triple P Positive Parenting Program is suitable for parents of all children and adolescents. The evaluation cited in this program summary applies to Triple P when provided to parents of children 8 years old and younger.

Evaluation Methods

Prinz, Sanders, et al. (2009) conducted the first population-based trial of the entire Triple P system, randomly assigning entire communities to the Triple P intervention to assess the effects of Triple P on

- 517 -

substantiated cases of child abuse and neglect, among other outcomes. The study was conducted on families with children below 8 years of age, and it was the first trial of Triple P to have taken place in the United States. The authors randomly assigned 18 counties in a southeastern state, with population sizes between 50,000 and 175,000 and ranging from rural to semi-urban, to treatment or control conditions. The authors examined the following population-level outcomes, all three of which were derived from independent data-collection systems deposited within a state-run statistical division:

• substantiated cases of child maltreatment, recorded by Child Protective Services staff

• child out-of-home placements, recorded through the foster care system

• child hospitalizations and emergency room visits due to injuries related to child maltreatment, recorded by hospital staff in compliance with mandatory reporting requirements.

Before the Triple P intervention took place, treatment and control counties were not significantly different with respect to population size, the percentage of children in poverty, or racial composition. Additionally, the authors examined five-year averages of the three outcomes listed above prior to the intervention and found no significant differences in those multiyear averages across treatment and control conditions.

Key Evaluation Findings

Prinz, Sanders, et al. (2009) examined the difference in the pre-post change across communities and found statistically significant effects of Triple P on the change of all three of the measured population- level outcomes from pre- to posttest. Specifically:

• Substantiated rates of child maltreatment grew in the control counties during the implementation period, from 11.12 cases per 1,000 children to 15.06 cases per 1,000. In the treatment communities, substantiated cases of child abuse and neglect did not change significantly over the course of the intervention.

• Out-of-home placements in the treatment counties fell from 4.27 to 3.75 per 1,000 children, compared with an increase in the control counties from 3.10 to 4.46 per thousand.

• Rates of child hospitalizations and emergency room visits resulting from child maltreatment fell from 1.73 to 1.41 cases per 1,000 in the treatment communities, compared with an increase in the control communities from 1.41 to 1.69 per 1,000.

Probable Implementers

Community-based service providers, public health and social service agencies, and government agencies.

Funding

No information at this time

Implementation Detail

Program Design

• The Triple P program implemented by Prinz, Sanders, et al. (2009) was a population-based program that implemented all five levels of Triple P intervention, based on family need, in each county, utilizing the existing workforce. Counties were randomly selected for the study based on their population size and location, not for their "readiness" to adopt Triple P.

- 518 -

• The intensity of the Triple P intervention varies. The intervention ranges from, at Level 1, a media campaign related to positive parenting and the provision of brief information resources, such as tip sheets and videos, to, at Levels 4 and 5, more-intensive parent training programs that target broader family issues, such as relationship conflict and parental depression, anger, and stress.

• The Triple P intervention can be ongoing; however, in the case of the evaluation presented above, it was conducted over the course of two years.

• Service providers involved in delivering the intervention included family support services, social services, preschool and child care settings, elementary schools, nongovernmental organizations, health centers, private-sector practitioners, and other community entities that were engaged, trained, and supported in the implementation of Triple P. This involved engaging 649 service providers across nine treatment counties.

Curriculum

All five levels of Triple P have intervention manuals, systematic training regimens for providers/ practitioners, and coordinated resource materials for parents (videos, workbooks, and tip sheets). Cost

The cost of the infrastructure necessary to implement all levels of Triple P is estimated at less than $12 per child for a community of 100,000 families with children ages 0-8 (Foster, Prinz, et al., 2008). This includes the costs of training service providers and the universal education and media component.

Issues to Consider

The trial conducted by Prinz, Sanders, et al. (2009) offers the first direct empirical evidence of Triple P's effectiveness at preventing and reducing child abuse and neglect. While there is substantial evidence of Triple P's effectiveness from more than 50 well-designed trials, prior to the trial conducted by Prinz, Sanders, et al. (2009) none of the evaluations measured the program's impact on child abuse and neglect, none were conducted in the United States, and many did not evaluate the program as a whole, but rather one or two of its constituent parts (Connell, Sanders, et al., 1997; Sanders, Markie-Dadds, et al., 2000; Bor, Sanders, et al., 2002; Hoath and Sanders, 2002; Leung, Sanders, et al., 2003; Markie-Dadds and Sanders, 2006; Morawska and Sanders, 2006; Roberts, Mazzucchelli, et al., 2006; Turner and Sanders, 2006; Plant and Sanders, 2007; Sanders, Bor, et al., 2007; Bodenmann, Cina, et al., 2008; Hahlweg, Heinrichs, et al., 2008; Sanders, Prior, et al., 2009; Whittingham, Sofronoff, et al., 2009). The Triple P program received a "promising" rating because the study design included only nine counties that were randomized to treatment or control groups. Furthermore, prior to the intervention, the five-year average of each of the measures of child abuse and neglect were used to ensure the comparability of counties that did and did not receive the Triple P intervention to one another. Based on these five-year averages, it was determined that the intervention and control counties were statistically the same. It appears, however, that the counties may have been significantly different on these measures in the one year prior to the intervention. The authors attempt to account for this difference by examining the difference in the change in the scores; that is, they examine the difference between pretest and posttest scores for the counties that did not receive the treatment, and compare that with the difference between pretest and posttest scores for the counties that did receive the treatment.

Example Sites

Nine counties in a southeastern state of the United States

- 519 -

Contact Information

Triple P America PO Box 12755 Columbia, SC 29211 (803) 451.2278 Email: contact.us@triplep.net

Available Resources

Triple P has developed a wide range of resources and curricular materials for organizations and practitioners. Those can be found at the Triple P website: http://www.triplep- america.com/pages/resources/description.html

Bibliography

Bodenmann, G., A. Cina, et al., "The Efficacy of the Triple P-Positive Parenting Program in Improving Parenting and Child Behavior: A Comparison with Two Other Treatment Conditions," Behavior Research and Therapy, Vol. 46, No. 4, 2008, pp. 411-427. Bor, W., M. R. Sanders, et al., "The Effects of the Triple P-Positive Parenting Program on Preschool Children with Co-Occurring Disruptive Behavior and Attentional/Hyperactive Difficulties," Journal of Abnormal Child Psychology, Vol. 30, No. 6, 2002, pp. 571-587. Connell, S., M. R. Sanders, et al., "Self-Directed Behavioral Family Intervention for Parents of Oppositional Children in Rural and Remote Areas," Behavior Modification, Vol. 21, No. 4, 1997, pp. 379-408. Foster, E. M., R. J. Prinz, et al., "The Costs of a Public Health Infrastructure for Delivering Parenting and Family Support," Children and Youth Services Review, Vol. 30, No. 5, 2008, pp. 493-501. Hahlweg, K., N. Heinrichs, et al., "Therapist-Assisted, Self-Administered Bibliotherapy to Enhance Parental Competence: Short- and Long-Term Effects," Behavior Modification, Vol. 32, No. 5, 2008, pp. 659-681. Hoath, F. E., and M. R. Sanders, "A Feasibility Study of Enhanced Group Triple P-Positive Parenting Program for Parents of Children with Attention-Deficit/Hyperactivity Disorder," Behaviour Change, Vol. 19, No. 4, 2002, pp. 191-206. Leung, C., M. R. Sanders, et al., "An Outcome Evaluation of the Implementation of the Triple P- Positive Parenting Program in Hong Kong," Family Process, Vol. 42, No. 4, 2003, pp. 531-544. Markie-Dadds, C., and M. R. Sanders, "Self-Directed Triple P (Positive Parenting Program) for Mothers with Children at-Risk of Developing Conduct Problems," Behavioural and Cognitive Psychotherapy, Vol. 34, No. 3, 2006, pp. 259-275. Morawska, A., and M. R. Sanders, "Self-Administered Behavioural Family Intervention for Parents of Toddlers: Effectiveness and Dissemination," Behavior and Research Therapy, Vol. 44, No. 12, 2006, pp. 1839-1848. Plant, K. M., and M. R. Sanders, "Reducing Problem Behavior During Care-Giving in Families of Preschool-Aged Children with Developmental Disabilities," Research in Developmental Disabilities, Vol. 28, No. 4, 2007, pp. 362-385. Prinz, R. J., M. R. Sanders, et al., "Population-Based Prevention of Child Maltreatment: The U.S. Triple P System Population Trial," Prevention Science, 10, No. 1, 2009, pp. 1-12.

- 520 -

mailto:contact.us@triplep.net

http://www.triplep-america.com/pages/resources/description.html

Roberts, C., T. Mazzucchelli, et al., "Behavioral Family Intervention for Children with Developmental Disabilities and Behavioral Problems," Journal of Clinical Child Adolescent Psychology, Vol. 35, No. 2, 2006, pp. 180-193. Sanders, M. R., "Triple P-Positive Parenting Program: Towards an Empirically Validated Multilevel Parenting and Family Support Strategy for the Prevention of Behavior and Emotional Problems in Children," Clinical Child and Family Psychology Review, Vol. 2, No. 2, 1999, pp. 71-90. Sanders, M. R., C. Markie-Dadds, et al., "The Triple P-Positive Parenting Program: A Comparison of Enhanced, Standard, and Self-Directed Behavioral Family Intervention for Parents of Children with Early Onset Conduct Problems," Journal of Consulting and Clinical Psychology, Vol. 68, No. 4, 2000, pp. 624-640. Sanders, M. R., K. M. Turner, et al., "The Development and Dissemination of the Triple P-Positive Parenting Program: A Multilevel, Evidence-Based System of Parenting and Family Support," Prevention Science, Vol. 3, No. 3, 2002, pp. 173-189. Sanders, M. R., W. Bor, et al., "Maintenance of Treatment Gains: A Comparison Of Enhanced, Standard, and Self-Directed Triple P-Positive Parenting Program," Journal of Abnormal Child Psychology, Vol. 35, No. 6, 2007, pp. 983-998. Sanders, M., J. Prior, et al., "An Evaluation of a Brief Universal Seminar Series on Positive Parenting: A Feasibility Study," Journal of Children's Services, Vol. 4, No. 1, 2009, pp. 4-20. Turner, K. M. and M. R. Sanders, "Help When It's Needed First: A Controlled Evaluation of Brief, Preventive Behavioral Family Intervention in a Primary Care Setting," Behavior Therapy, Vol. 37, No. 2, 2006, pp. 131-142. Whittingham, K., K. Sofronoff, et al., "Stepping Stones Triple P: An RCT of a Parenting Program with Parents of a Child Diagnosed with an Autism Spectrum Disorder," Journal of Abnormal Child Psychology, Vol. 37, No. 4, 2009, pp. 469-480.

Last Reviewed

January 2011

Triple Play

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children experiencing good physical health

Topic Areas Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Out of School Time Community-Based Service Provider

- 521 -

Type of Service Health Education Youth Development Type of Outcome Addressed Physical Health

Evidence Level Promising

Program Overview

Launched in 2005, "Triple Play: A Game Plan for the Mind, Body & Soul" is a Boys and Girls Clubs of America (BGCA) program that promotes healthy living among all youth ages 6-18 who attend participating Boys and Girls Clubs. The program is designed to be incorporated into multiple aspects of youth experiences at participating Boys and Girls Clubs. It is called "Triple Play" in reference to its three-pronged strategy of targeting youth's minds, bodies, and souls. To engage youth's "minds," youth attending these clubs are exposed to a 10-lesson Healthy Habits curriculum, which teaches nutrition and healthy living to three different age groups: 6-8, 9-12, and teens. The curriculum teaches such concepts as limiting portion size, setting personal goals, and making smart snack choices. To target the "body," youth attending participating Boys and Girls Clubs are encouraged to partake in non-competitive daily fitness challenges, which allow youth to challenge their own limits in physical activities. Triple Play clubs may also arrange inter-club competitions, which allow youth to compete against others in team sports and other games. Additionally, Triple Play club teens are engaged in club and community service efforts to promote health and physical activity. To target the "soul," Triple Play teaches club professionals how to design a program that encourages youth's personal development. These social recreation activities typically take place in the club's gamesroom, which is considered by many to be the hub of a club. Members strengthen their character, increase confidence, and enhance their ability to relate well to others through a well-run gamesroom. The "soul" aspect of Triple Play clubs is intended to promote feelings of belonging, influence, usefulness, and competence in youth.

Program Participants

BGCA youth participants, ages 6-18, who attend BGCA Clubs that offer the Triple Play program.

Evaluation Methods

The purpose of the study was to examine whether BGCA's Triple Play program had an impact on youth outcomes in the areas of (1) healthy nutrition knowledge and behavior, (2) physical activity and increased exercise levels, and (3) social relationship skills.

Gambone et al. (2009) evaluated the effectiveness of Triple Play in improving health outcomes among 2,242 youth attending 30 Boys and Girls Club locations throughout the United States. The 30 Boys and Girls Clubs were selected to ensure adequate representation within five regions. Within each region, clubs that applied to receive grant funding to implement Triple Play were selected for the study if they scored well on the criteria that BGCA uses to award these grants. Clubs that agreed to participate were randomized to either implement Triple Play or to serve as a control group that would not receive funding to implement Triple Play until after the study ended. Randomized clubs were then contacted and invited to participate in the study. If a club refused participation, replacement clubs were contacted until four treatment and two control clubs were obtained per region. This resulted in a total of 20 treatment clubs (1,476 youth) and 10 control clubs (766 youth) that participated in the study. Youth were surveyed and asked to complete activity diaries to assess nutritional knowledge, physical activity, and developmental outcomes and experiences. Assessments were completed at baseline, in March 2006; at midpoint of the study, in December 2006; and at post-study, in December 2007. The study examined differences between treatment and control club youth over time in the

- 522 -

outcome areas of interest, using statistical methods to adjust for similarities that may exist between youth that attended the same club.

Key Evaluation Findings

Gambone et al. (2009) found the following:

• Youth in Triple Play clubs increased their total knowledge of healthy eating significantly more than youth in control clubs (11 percent increase versus 2 percent increase).

• Youth in Triple Play clubs increased their knowledge of portion sizes significantly more than youth in control clubs (8 percent increase versus 1 percent decrease).

• Youth in Triple Play clubs ate significantly more healthy foods in a day than those in treatment clubs (7 versus 5.7). For Triple Play youth, the trend showed a drop in the number of healthy foods eaten the day before from baseline to the midpoint survey but an increase between midpoint and final survey. For the control youth, this number continued to drop over the study period.

• Youth in Triple Play clubs ate significantly more healthy fruits and vegetables than control club youth (0.32 increase in the number of fruits and vegetables eaten versus 0.55 decrease in fruits and vegetables eaten).

• The frequency of breakfast eating for youth in Triple Play clubs and control clubs both decreased. However, the number of days dropped at a slower rate for Triple Play youth than for youth a control club youth (0.6 versus 1.1).

• Youth in Triple Play clubs significantly increased their number of minutes spent exercising per day, compared with control club youth (6-minute increases versus a 4-minute decrease).

• Youth in Triple Play clubs showed a significant increase in the number of days spent exercising more than 60 minutes, compared with control club youth (0.8 increase in days versus 0.2 decrease in days).

• Youth in Triple Play clubs showed a significant decrease in the number of days spent exercising less than 30 minutes, compared to control club youth (.93 decrease in days compared to .02 increase in days).

• High-quality peer interactions, along the dimensions of communication, conflict, instrumental support and emotional support by peers, increased more for Triple Play club youth than control club youth (5 percent increase in youth reporting high-quality interactions versus a 4 percent decrease).

• Low-quality peer interactions, along the dimensions of communication, conflict, instrumental support and emotional support by peers, decreased more for Triple Play club youth than control club youth (10 percent decrease in youth reporting low-quality interactions verses 1 percent decrease).

• Triple Play club youth showed a significantly greater improvement in sense of mastery and control than control club youth (4 percent increase in youth reporting high sense of mastery and control versus a 5 percent decrease).

Probable Implementers

Most Boys and Girls Clubs are Probable Implementers. Some clubs receive funding to serve as sustainable models of effective implementation of the Triple Play initiative for young people ages 6-18.

Funding

Triple Play is funded with the assistance of the Coca-Cola Company and the WellPoint Foundation.

- 523 -

Implementation Detail

Program Design

Triple Play is implemented as part of the Boys and Girls Club daily operations and is available to all youth who attend participating clubs.

Staffing

Triple Play is implemented by Boys and Girls Clubs professionals on site at some 4,000 local club locations across the country and on military installations abroad.

Curriculum

Triple Play includes the 10 lesson Healthy Habits curricula targeted at three age groups: 6-8 years, 9- 12 years, and teens. These curricula teach concepts around healthy living and nutrition, including limiting portion sizes, setting personal goals, and making smart snack choices. In addition, a BGCA Leadership University offers several one-day workshops to teach BGCA staff how to develop physical education and nutrition programs, organize youth sports leagues, and develop social recreation programs. Triple Play offers The Smart Guide to Social Recreation: Effective Gamesroom Management and Leadership, to teach BGCA staff how to build a youth development- oriented gamesroom and a Back Pocket Program Hints guide for creating social recreation activities.

Issues to Consider

Triple Play has been listed as a "Promising" program. In the Gambone et al. (2009) study, 36 percent of youth who completed the baseline measurements did not complete the midpoint measurements, and an additional 19 percent of youth did not complete the post-study measurements. Youth who did not complete these follow-up measurements were more likely to be white, less likely to be black, less likely to have friends who encouraged them to participate in physical activity, and more likely to report high levels of peer conflict. In addition, more black youth in the treatment group did not provide follow-up measurements, and more white youth in the control group did not provide follow-up measurements. It is possible that this difference between types of youth who failed to complete the study could cause the control group and treatment group to differ in a way that could affect the study findings. Alternatively, if certain types of youth were less likely to complete the study, this could mean that any detected Triple Play effects might not apply to all youth who attend Boys and Girls Clubs. However, there were no differences in the measurement outcomes of interest between youth in the treatment and control groups who did not complete the study, suggesting that the study's findings were not affected by the differences between types of youth who completed the study and those who did not. It should also be noted that the outcomes measured in the Gambone et al. (2009) study are intermediate outcomes. The study focused on measuring changes in youth behaviors that are associated with positive health outcomes, such as exercising and healthy eating, rather than direct measures of physical health, such as pulse rate or body mass index.

Example Sites

The Triple Play program was evaluated in 30 clubs across the United States.

Contact Information

Wayne B. Moss, Senior Director, Sports, Fitness and Recreation BGCA 1275 Peachtree Street NE Atlanta, GA 30309

- 524 -

Phone: 404.487.5761 Email: wmoss@bgca.org

Available Resources

The Triple Play website contains an overview of the program and links to a parent guidebook: http://bgca.org/whatwedo/SportsFitnessRecreation/Pages/TriplePlayDetail.aspx

Bibliography

Gambone, Michelle Alberti, Theresa M. Akey, Kathryn Furano, and Lisa Osterman, Promoting Healthy Lifestyles: The Impact of Boys & Girls Clubs of America's Triple Play Program on Healthy Eating, Exercise Patterns, and Developmental Outcomes: Final Evaluation Report, Philadelphia, Pa.: Youth Development Strategies, August 2009.

Last Reviewed

October 2011

Who Do You Tell?

Program Info

Outcome Areas Healthy and Safe Children

Indicators Children not experiencing physical, psychological or emotional abuse

Topic Areas Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Elementary School Type of Service Parent Education Youth Development Type of Outcome Addressed Child Abuse and Neglect

Evidence Level Promising

Program Overview

The "Who Do You Tell?" Child Sexual Abuse Education Program was designed to teach elementary school aged children basic concepts to help them prevent and protect themselves from sexual abuse. The program aims to help students to learn to distinguish between inappropriate and appropriate

- 525 -

mailto:wmoss@bgca.org

http://bgca.org/whatwedo/SportsFitnessRecreation/Pages/TriplePlayDetail.aspx

types of touching. In addition, students are also instructed on methods to communicate instances of abuse to trusted adults.

Originally developed by the Calgary Sexual Assault Centre in 1983, the program was taken over and revised by the Calgary Communities Against Sexual Assault (CCASA) in 1995. Two educators from CCASA present the program to groups of 15 to 30 elementary school aged children in their regular school classrooms during school hours. The program is conducted in two 45-minute sessions on consecutive days. In addition, parents are invited to attend an informational session that addresses program content and offers advice on how to speak with children about the covered topics. Teachers in participating schools also receive training on how to proceed should abuse be disclosed to them. The program’s emphasis is on defining child sexual abuse in a way that is meaningful to children and how to say "no" to activities or instances of physical touching that are inappropriate and make children feel uncomfortable. The program also addresses what constitutes appropriate physical touching so that the children will not become confused or anxious. A variety of modalities are employed in the instruction, including discussions, visuals, videos and role-playing. The content of the tools are adapted to be age appropriate. The role-playing dynamic of the program allows children to practice and reinforce the skills they are being taught, and students are encouraged to ask questions throughout the program. Following completion of the program, children are given the opportunity to meet one-on-one with program educators to discuss any issues or concerns participation in the program may have raised.

Program Participants

The program is designed for use with elementary school-aged children. Three versions of the program for grades K-2, grades 3 and 4, and grades 5 and 6, each with specific age-appropriate content and materials allow the program to be appropriate for use with a wide age group. Elementary school aged children at two Catholic schools in Calgary, Canada took part in an evaluation of the program. Eighty-eight percent of the participants were white. Approximately half came from two-parent families. The sample included children in grades 1 through 6.

Evaluation Methods

An evaluation took place in 1996, with 231 students from two Catholic elementary schools. The grade level of the children ranged from first through sixth. Students within each school were randomly assigned to participate in the program (117 of the students), or not participate (114 of the students). The experimental and control groups were matched based on age. Students were assessed prior to the start of the program using the 33-item Children’s Knowledge of Abuse Questionnaire-Revised (CKAQ-R). The test asked questions to determine students’ level of knowledge of concepts to prevent sexual abuse; and, in particular, their understanding of inappropriate and appropriate touching was evaluated. The two groups were fairly well matched on pre-intervention test scores, with the control group in grades 2 and 4 slightly outscoring the test group on the inappropriate-touch subscale, and the control group in grades 2 and 3 slightly outscoring the test group on the appropriate-touch subscale. Students were retested within two weeks of program completion. Students in grades 1 and 2 were assessed on an individual basis, whereas the test was administered to higher grades as a group. Scores are given for each grade group. Results for inappropriate and appropriate touching knowledge are presented separately. In addition, a survey was administered to participants’ parents to determine their perceptions of their child’s reaction to the program and measure for any negative effects from the program (such as increased nightmares, recoiling from or suspicion of all touching from adults, and other such behaviors). Caution should be exercised in drawing conclusions based on the parent survey, however, due to the low level of parent participation and response. Only 27 parents (21.4%) attended the program’s parent information session, and only 126 parents returned surveys.

- 526 -

Key Evaluation Findings

The research by Leslie M. Tutty (1997) found the following:

• It is difficult to determine the statistical significance of changes in test scores because of the small sizes of the individual grade subgroups. Subgroups ranged from 17 to 24 students.

• Nearly all students in both the control and test populations experienced gains in their appropriate-touch subscale scores from pre- to posttesting, regardless of whether they received the intervention.

o The grade 1 control group is an exception to this finding – the grade 1 control group experienced a decrease in scores on the appropriate touch subscale.

• Participation in the program significantly increased students’ level of knowledge of inappropriate touching and slightly increased their knowledge of appropriate touching.

• Younger children experienced a larger gain in test scores on the inappropriate-touch subscale, with the largest gains seen in grades 1 and 2.

• At both pre- and post-testing, all children (in both the control and experiment groups) in higher grades had a significantly greater level of knowledge and understanding about inappropriate touching and a slightly greater level of knowledge and understanding about appropriate touching than did children in the lower grades.

• Parents of participating children reported no negative reactions to the program on the part their children, such as an increase in the children’s nightmares or discomfort with affection from family members.

Probable Implementers

Public or private elementary schools, social service or health educators, state and local child abuse prevention agencies, and sexual assault agencies.

Funding

The program has been funded by Family and Community Social Services, a municipal preventative social service agency.

Implementation Detail

Program Design

• The curriculum and materials are tailored to the appropriate developmental level of the participants. Three versions of the program are available for different age groups.

• The interactive component of the curriculum allows children to practice the skills on which they are being instructed.

Curriculum

The "Who Do You Tell" curriculum is composed of two 45-minute sessions, presented on two consecutive days. Three versions of the program tailor content to the appropriate developmental level of the recipient group according to age. Prior to the program, teachers at participating schools are given training that addresses appropriate response and action should abuse be disclosed. In addition, parents are invited to attend an evening information session that provides an overview of the program and information on child sexual abuse in general, how to speak with a child about abuse, and available resources should abuse occur.

- 527 -

The curriculum is administered by educators from the Calgary Communities Against Sexual Assault. The first day of the program involves an exploration of subject related definitions, such as what is "sexual abuse," which are the "private parts" of the body, and what constitutes "appropriate" and "inappropriate" touching. Educators lead a discussion on how and when children should say "no" and also inform students about what resources are available to them and the appropriate and available paths of action to take should abuse occur. The day is concluded with an interactive session, allowing children to ask questions, discuss topics, and practice the skills covered in the course. After a review of material covered during the first day, the second day’s session addresses the contextual issues of child sexual abuse – when and where abuse can occur, and possible perpetrators. Children are again given the opportunity to ask questions and practice skills through skits and role- playing.

Staffing

The program is implemented by educators from the Calgary Communities Against Sexual Assault. All program staff have social service backgrounds, with knowledge of and/or experience working with children and families affected by sexual and domestic violence.

Financing

The program is offered to area schools at no cost; however, donations are accepted.

Issues to Consider

This program received a "promising" rating. Although the research yielded positive outcomes, the evaluation was based on a relatively small and homogenous sample, and the outcomes analysis was based on even smaller subgroups of the population. The small sample size makes it difficult to draw strong and reliable conclusions from the data. In addition, follow-up was conducted only two weeks after the completion of the program. This is particularly significant given the young age of the participants and the extremely short duration of the curriculum (a total of 90 minutes over two days). The study by Tutty (1997) cites prior research that supports the notion that longer exposure to prevention programs yields more positive and longer-lasting results. Given that, it is reasonable to wonder whether demonstrated increases in knowledge would persist over time. Without the availability of longitudinal data, this question cannot be satisfactorily answered. Of note, whereas the program has an impact on levels of knowledge regarding sexual abuse, it does not reduce the number of children exposed to or experiencing sexual abuse. Although children may use the knowledge they gain to assist themselves in reducing their individual risk of experiencing sexual abuse, the program does not in any way have an impact on the occurrences of abuse itself, nor does it measure whether the evidenced reduction in risk translates into a reduction of actual occurrences of abuse.

Example Sites

Calgary, Canada

Contact Information

Annemarie Koszegi, Program Manager "Who Do You Tell?"TM Program Calgary Communities Against Sexual Abuse YWCA Mary Dover Building 7th Floor, 320 - 5th Avenue SE Calgary, Alberta T2G 0E5

- 528 -

TTY: (403) 508-7888 Phone: (403) 237-6905 Fax:(403) 264-8355 info@calgarycasa.com www.calgarycasa.com

Available Resources

Program plans and a training package are available from Calgary Communities Against Sexual Assault.

Bibliography

Tutty, L. M., "Child Sexual Abuse Prevention Programs: Evaluating Who Do You Tell," Child Abuse & Neglect Vol. 21, No. 9, pp. 869-881, 1997.

Last Reviewed

May 2009

Wyman's Teen Outreach Program

Program Info

Outcome Areas Healthy and Safe Children Children Succeeding in School

Indicators Students performing at grade level or meeting state curriculum standards Youths abstaining from sexual activity or not engaging in risky sexual behavior

- 529 -

mailto:info@calgarycasa.com

http://www.calgarycasa.com/

Evidence Level Promising

Program Overview

Wyman's Teen Outreach Program® (TOP) is a national youth development program designed to prevent adolescent problem behaviors by helping adolescents ages 12 through 18 to develop healthy behaviors, life skills, and a sense of purpose. The Wyman Center is a nonprofit and the developer of TOP. The nine-month TOP curriculum combines community service learning, adult support and guidance, and curriculum-based group activities. The curriculum has four levels appropriate for a range of grades and ages. Participants at all levels engage in a minimum of 20 hours of community service learning per academic year. TOP staff guide the youth in choosing, planning, implementing, reflecting on, and celebrating their service learning project. Service projects may include direct service, indirect service, or civic actions. TOP groups also meet at least once a week throughout the school year to discuss topics from the curriculum, including communication skills/assertiveness, understanding and clarifying values, relationships, goal-setting, influences, decisionmaking, and adolescent health and sexual development.

Program Participants

The program is designed for use with youths aged 12 through 17. Program impact has been rigorously evaluated for high-school-age youths only.

Evaluation Methods

Allen et al. (1997) conducted a study between 1991 and 1995 involving 25 TOP sites nationwide. All active TOP sites were invited to participate in the evaluation. The 25 participating sites represented roughly 10 percent of all the TOPs. Teens at each site were randomly assigned to treatment or control status, usually at the student level and less frequently at the classroom level. Analysis of demographic characteristics of the TOP and control groups found that the groups were similar at program entry. More than 80 percent of the participants were female, approximately 67 percent were black, and approximately 46 percent resided in two-parent households. Nearly 70 percent of the study population was in grades 9 and 10, approximately 20 percent was in grade 11, and approximately 11 percent was in the 12th grade. A total of 695 students were randomized at the start of the study, with 342 students in the treatment group and 353 in the control group. All participants were assessed via questionnaire at entry (one to two weeks into the course) and upon completion of the program (at the end of the school year) on their background characteristics and histories of problem behaviors. Despite random assignment procedures, at baseline the control group was found to have significantly higher levels of prior course failure, school suspension, and teen pregnancy than the treatment group. A site-by-site inspection of the 25 sites found that one site had a control group with a significantly higher prevalence of risk factors and prior problem behaviors, and two sites had failed in efforts to track control students. As a result, these three sites were excluded from the study, resulting in a total of 22 program sites included in the analyses. Study dropout rates were 5 percent in the treatment group and 8 percent in the control group. Analyses of this population indicated that students who dropped out of the study did not differ significantly at program entry from those who completed the program in terms of ethnicity, parents' educational attainment, or household composition (single-parent versus two-parent household). Dropout students, however, were found to be significantly younger and more frequently male than were completing students, and they were more likely to have had or have caused a pregnancy, and to have been suspended from school.

- 530 -

In addition to analysis to determine program impact, analysis was also conducted to determine whether there was a "dosage effect" (that is, whether variations in program intensity, particularly more or less volunteer hours, were related to outcomes).

Key Evaluation Findings

The study by Allen et al. (1997) found the following:

• The TOP participants' school suspension rate decreased by 24 percent over the course of the study (from 17 percent at baseline to 13 percent at posttest), while the control group experienced a 21 percent increase in suspension rate (from 24 percent at baseline to 29 percent at posttest).

• The TOP group's course failure rate decreased by 12 percent after the study (from 30.3 percent to 26.6 percent), whereas the control group experienced a 24 percent increase in failure rate (rising from 38 percent to 47 percent).

• Program participation had a significant impact on the pregnancy rate among female participants, with the TOP pregnancy rate decreasing 31 percent (from 6 percent at baseline to 4 percent at posttest), and the control group's pregnancy rate decreasing only 2 percent (10 percent at baseline to 9.8 percent at follow-up).

• There was no significant relationship between program dosage and pregnancy rates or program dosage and suspension.

Probable Implementers

High schools, middle schools, after-school and community-based youth organizations, teen pregnancy prevention coalitions, local health departments, and social service agencies

Funding

The program is funded primarily through local funding sources. The Charles Stewart Mott Foundation and the Lila Wallace Reader's Digest Fund supported evaluation of the program. Several current replications of Wyman's Teen Outreach Program are funded by the U.S. Department of Health and Human Services through the Office of Adolescent Health and the Administration for Children and Families. According to Allen et al. (1997), the cost of implementing TOP for a full academic year to a class of 18 to 25 students ranged from approximately $500 to $700 per student (in 1997 dollars). These figures include costs for facilitator and site-coordinator time. When facilitator/coordinator time is provided as an in-kind contribution by schools and community volunteer service organizations, the direct program costs drop to approximately $100 per student.

Implementation Detail

Program Design

• Group discussions allow students to share and learn from others' volunteer experiences.

• The role-playing component of the curriculum helps students to practice and prepare for their volunteer service experience.

• Participants take part in a minimum of 20 hours of community service during the program year.

- 531 -

• A wide range of volunteer activities is available, including work as hospital and nursing home aides, peer tutoring, and fund-raising activities.

Curriculum

TOP's curriculum, "Changing Scenes," uses a variety of experiential methods to engage youth, including small-group discussions and role-playing. A community service learning guide aids discussions about volunteer experiences, tying together the classroom and community service learning aspects of the program and allowing youth to process and reflect on their service activities. The TOP curriculum has four levels. Each level contains material that is developmentally appropriate for the age group involved (Level I: 12- to 13-year-olds. Level II: 14-year-olds. Level III: 15- to 16- year-olds. Level IV: 17- to 18-year-olds.) Curriculum theme areas include the following:

• communication skills/assertiveness

• understanding and clarifying values

• relationships

• goal-setting

• influences

• decisionmaking

• human development and sexuality.

Facilitators choose which lessons to implement with their TOP group, allowing the group's weekly meetings to be adapted to the needs of the group. For each subject area, the curriculum contains a variety of activities and materials for discussion, so that implementers may select those that seem to be the most relevant and helpful for their particular youth population.

Staffing

The program is most commonly implemented by youth workers, classroom teachers, or guidance personnel who have been trained to facilitate the discussions outlined in the curriculum.

Issues to Consider

This program received a "promising" rating. Despite positive outcomes for both academic achievement and teen pregnancy, the program evaluation had a number of methodological limitations regarding sample selection and comparability of treatment groups. When considering the results of this study, there is a significant issue of self-selection bias. Only 10 percent of the TOP sites chose to participate in the study. Allen and colleagues provided no information on how these sites and their populations compare with TOP sites in general. It is possible that the sites that chose to participate had a history of greater program success and/or served higher- functioning populations. If so, the results found may not be indicative of implementation of TOP in all settings.

A point of concern is the fact that despite random assignment to treatment status and similar sociodemographic profiles, the treatment and control groups differed significantly at entry on all measures of problem behaviors. At initial data collection, the control group showed higher levels of prior course failure (38 percent versus 30 percent), suspension (24 percent versus 17 percent), and pregnancy (10 percent versus 6 percent) than the treatment group. Although an attempt was made to control for these differences in the statistical analyses, these discrepancies could suggest that the TOP

- 532 -

group was "better off" from the start and may have been predisposed toward more favorable outcomes.

Consideration must also be given to the gender composition of the participant population. Although intended to work with all teens, this program seems to favor participation by female students. In the 1997 study, the sample had large numbers of females, with the treatment and control groups being 86 percent and 83 percent female, respectively. This makes it difficult to conclude that the program would work as effectively with a male population. Further, as the classroom component is based on discussion and sharing and, to some extent, is shaped around the experiences, interests, and needs of the participants, there is a question of just how effective it would be (both in terms of subject matter covered and the open participation of students) with a more equally balanced gender mix. It is possible that some of the program's exhibited success results from the ability to target the common needs of the female population. Thus, it may not be appropriate to conclude that the program works equally well with male and female teen populations. Another issue to consider is the lack of longitudinal data. Conclusions are based on data collected upon immediate conclusion of the program. It may be easier for a teen to "stay on track" while he or she is the direct recipient of focused attention, such as the kind of attention provided by the program. There is no measure of whether participation in the program carries into continued improved performance and reduced risk of pregnancy beyond the boundaries of direct program involvement. Allen and Philliber (2001), using a somewhat less rigorous design with a sample that included the same students assessed in Allen et al. (1997) as well as additional students, suggests that the program's effects are particularly robust for youths at higher risk of the specific types of problem behaviors being measured. For example, the authors found that the program had a larger effect in reducing pregnancies among youths who had already given birth to a child (compared with those who had never given birth). For this group of study participants, the likelihood of an additional pregnancy was less than one-fifth as large in the intervention group as in the comparison group. Similarly for the outcome of academic failure, TOP had a larger impact for youths who had been previously suspended than for those who had not. The program also displayed greater effectiveness for members of racial ethnic minority groups than for Caucasian students.

Example Sites

In operation since 1976, TOP serves more than 30,000 teens in 1,500 clubs, ranging across 32 states plus Washington, D.C. It is used in rural, urban, and suburban settings. The program is typically integrated into the curriculum of schools but is also implemented as an out-of-school program through community-based youth organizations, local health departments, and social service agencies.

Contact Information

Felice McClendon Wyman's National Network Partner Development Manager 600 Kiwanis Drive Eureka, MO 63025 phone: (636) 636-549-1238 fax: (636) 938-5289 Felice.mcclendon@wymancenter.org www.wymancenter.org

Available Resources

Wyman offers a dedicated website for those interested in Wyman's Teen Outreach Program®: www.wymancenter.org/TOP. Resources to support the development and implementation of local TOP programs, including curriculum, implementation training, training of trainers, technical assistance, and self-assessment

- 533 -

mailto:Felice.mcclendon@wymancenter.org

http://www.wymancenter.org/

http://www.wymancenter.org/TOP

program evaluation materials, are available from the Wyman Center and its certified replication partners exclusively.

Bibliography

Allen, Joseph, and Susan Philliber, "Who Benefits Most from a Broadly Targeted Prevention Program? Differential Efficacy Across Populations in the Teen Outreach Program," Journal of Community Psychology, Vol. 29, No. 6, 2001, pp. 637-655. Allen, Joseph, Susan Philliber, Scott Herrling, and Gabriel P. Kuperminc, "Preventing Teen Pregnancy and Academic Failure: Experimental Evaluation of a Developmentally Based Approach," Child Development, Vol. 68, No. 4, 1997, pp.729-742.

Last Reviewed

July 2013

- 534 -

Preface
Table of Contents
List of Tables
Acknowledgements
Introduction

Table 1. Primary PPN Evidence Criteria

Types of Programs

Table 2. PPN Programs, by Age of Child
Table 3. PPN Programs, by Type of Setting
Table 4. PPN Programs, by Type of Service
Table 5. PPN Programs, by Type of Outcome Addressed

Programs That Work
The Abecedarian Project

Program Info

Healthy and Safe Children Children Ready for School Children Succeeding in School
Youths not using alcohol, tobacco, or illegal drugs Students performing at grade level or meeting state curriculum standards Children ages 0 to 5 exhibiting age-appropriate mental and physical development Children experiencing good physical health
Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Home Visiting Type of Service Family Support Health Care Services Instructional Support Type of Outcome Addressed Cognitive Development/Sc...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings

Early Childhood Outcomes
Primary School Findings
Middle School- and High School-Age Outcomes
Adult Outcomes

Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum: Preschool
Curriculum: Primary School
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Accelerated Academics Academy (AAA)

Program Info

Children Succeeding in School
Students graduating from high school
Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Middle School Type of Service Instructional Support Youth Development Type of Outcome Addressed Cognitive Development/School Performance
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing
Curriculum

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Accelerated Reader

Program Info

Children Succeeding in School
Students performing at grade level or meeting state curriculum standards
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School High School Type of Service Instructional Support Type of Outcome Addressed Cog...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Adolescent Coping with Depression Course

Program Info

Healthy and Safe Children
Children not experiencing anxiety or mood disorders, such as depression
Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Middle School High School Community-Based Service Provider Health Care Provider Type of Service Health Care Services Health Education ...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Athletes Training and Learning to Avoid Steroids (ATLAS)

Program Info

Healthy and Safe Children
Youths not using alcohol, tobacco, or illegal drugs
Age of Child Adolescence (13-18) Type of Setting High School Type of Service Health Education Youth Development Type of Outcome Addressed Mental Health Physical Health Substance Use and Dependence
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Attachment-Based Family Therapy

Program Info

Healthy and Safe Children
Children not experiencing anxiety or mood disorders, such as depression
Age of Child Adolescence (13-18) Type of Setting
Health Care Provider Type of Service Family Support Health Care Services Youth Development Type of Outcome Addressed Mental Health
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing
Curriculum

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Be Proud! Be Responsible!

Program Info

Healthy and Safe Children
Youths abstaining from sexual activity or not engaging in risky sexual behavior
Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Middle School High School Community-Based Service Provider Health Care Provider Type of Service Health Education Youth Development Ty...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Big Brothers Big Sisters of America

Program Info

Healthy and Safe Children Children Succeeding in School
Youths not using alcohol, tobacco, or illegal drugs Students performing at grade level or meeting state curriculum standards Children and youth not engaging in violent behavior or displaying serious conduct problems
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Community-Based Service Provider Type of Service Mentoring Youth Development Type of Outcome Addressed Behavior Pro...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Child Development Project

Program Info

Healthy and Safe Children Children Succeeding in School
Youths not using alcohol, tobacco, or illegal drugs Students performing at grade level or meeting state curriculum standards
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Elementary School Type of Service Instructional Support Type of Outcome Addressed Behavior Problems Cognitive Development/School Performanc...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Child Sexual Abuse Prevention: Teacher Training Workshop

Program Info

Healthy and Safe Children
Children not experiencing physical, psychological or emotional abuse
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School High School Type of Service Health Education Type of Outcome Addressed Child Ab...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Child-Parent Centers

Program Info

Healthy and Safe Children Children Ready for School Children Succeeding in School Strong Families
Students performing at grade level or meeting state curriculum standards Students graduating from high school Children not experiencing physical, psychological or emotional abuse Families increasing economic self-sufficiency Children ages 0 to 5 exhib...
Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Type of Service Family Support Health Care Services Instructional Support Parent Education Type of Outcome Addressed Behavior Problems ...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Class Wide Peer Tutoring Program

Program Info

Children Succeeding in School
Students performing at grade level or meeting state curriculum standards
Age of Child Middle Childhood (9-12) Type of Setting Elementary School Middle School Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Cognitive Behavioral Intervention for Trauma in Schools (CBITS)

Program Info

Healthy and Safe Children Children Succeeding in School
Students performing at grade level or meeting state curriculum standards Children not experiencing anxiety or mood disorders, such as depression
Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School Type of Service Health Care Services Parent Education Youth Development Type of Outcome Addressed Beh...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Cognitive Relaxation Coping Skills

Program Info

Healthy and Safe Children
Children and youth not engaging in violent behavior or displaying serious conduct problems Children not experiencing anxiety or mood disorders, such as depression
Age of Child Middle Childhood (9-12) Type of Setting Elementary School Middle School Type of Service Health Care Services Youth Development Type of Outcome Addressed Behavior Problems Mental Health Violent ...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Cognitively Guided Instruction (CGI)

Program Info

Children Succeeding in School
Students performing at grade level or meeting state curriculum standards
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Elementary School Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Communities In Schools

Program Info

Children Succeeding in School
Students graduating from high school
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School High School Community-Based Service Provider Health Care Provider Type of Service ...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Cooperative Integrated Reading and Composition

Program Info

Children Succeeding in School
Students performing at grade level or meeting state curriculum standards
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Elementary School Middle School Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Coping Cat

Program Info

Healthy and Safe Children
Children not experiencing anxiety or mood disorders, such as depression
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School High School Community-Based Service Provider Health Care Provider Type of Service ...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Coping with Stress Course

Program Info

Healthy and Safe Children
Children not experiencing anxiety or mood disorders, such as depression
Age of Child Adolescence (13-18) Type of Setting Middle School High School Community-Based Service Provider Health Care Provider Type of Service Health Education Youth Development Type of Outcome Addressed ...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Core Knowledge

Program Info

Children Succeeding in School
Students performing at grade level or meeting state curriculum standards
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School Type of Service Instructional Support Type of Outcome Addressed Cognitive Developme...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Counselors Care (CARE)

Program Info

Healthy and Safe Children
Children not experiencing anxiety or mood disorders, such as depression
Age of Child Adolescence (13-18) Type of Setting High School Type of Service Health Care Services Youth Development Type of Outcome Addressed Mental Health Physical Health Substance Use and Dependence
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Creating Lasting Family Connections

Program Info

Healthy and Safe Children
Youths not using alcohol, tobacco, or illegal drugs
Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School High School Community-Based Service Provider Health Care Provider Type of Service Family Support Hea...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider

Awards Received

Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

DARE to be You

Program Info

Children Ready for School
Children ages 0 to 5 exhibiting age-appropriate mental and physical development
Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Community-Based Service Provider Type of Service Family Support Health Education Parent Education Type of Outcome Addressed Behavior Probl...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Direct Instruction

Program Info

Children Succeeding in School
Students performing at grade level or meeting state curriculum standards Students graduating from high school
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School High School Type of Service Instructional Support Type of Outcome Addressed Cog...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Materials
Staffing
Project Costs

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Early Childhood Education and Assistance Program (ECEAP)

Program Info

Strong Families
Families increasing economic self-sufficiency
Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Community-Based Service Provider Health Care Provider Home Visiting Type of Service Family Support Health Care Services Instructional Sup...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Early Head Start

Program Info

Children Ready for School
Children ages 0 to 5 exhibiting age-appropriate mental and physical development
Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Home Visiting Type of Service Family Support Health Care Services Instructional Support Parent Education Type of Outcome Addressed Beh...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Training and Technical Assistance

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Early Intervention in Reading

Program Info

Children Succeeding in School
Students performing at grade level or meeting state curriculum standards
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Elementary School Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

The Effective Learning Program

Program Info

Children Succeeding in School
Students graduating from high school
Age of Child Adolescence (13-18) Type of Setting High School Type of Service Instructional Support Youth Development Type of Outcome Addressed Cognitive Development/School Performance
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing
Curriculum

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Family Foundations

Program Info

Healthy and Safe Children Children Ready for School Strong Families
Children not experiencing physical, psychological or emotional abuse Fathers maintaining regular involvement with their children Children ages 0 to 5 exhibiting age-appropriate mental and physical development Children and youth not engaging in violent...
Age of Child Early Childhood (0-8) Type of Setting Community-Based Service Provider Health Care Provider Type of Service Family Support Parent Education Type of Outcome Addressed Behavior Problems Mental Health
Proven/Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing
Curriculum

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Family Support and Parenting Education in the Home

Program Info

Healthy and Safe Children
Children not experiencing physical, psychological or emotional abuse Children experiencing good physical health
Age of Child Early Childhood (0-8) Type of Setting Community-Based Service Provider Health Care Provider Home Visiting Type of Service Family Support Health Education Parent Education Type of Outcome Addressed ...
Proven/Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing
Curriculum

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Family Thriving Program

Program Info

Healthy and Safe Children
Children not experiencing physical, psychological or emotional abuse Children not experiencing anxiety or mood disorders, such as depression Children experiencing good physical health
Age of Child Early Childhood (0-8) Type of Setting Home Visiting Type of Service Family Support Parent Education Type of Outcome Addressed Child Abuse and Neglect Mental Health
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Father/Male Involvement Preschool Teacher Education Program

Program Info

Strong Families
Fathers maintaining regular involvement with their children
Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Type of Service Parent Education
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

FluText

Program Info

Healthy and Safe Children
Children experiencing good physical health
Age of Child Early Childhood (0-8) Type of Setting Health Care Provider Type of Service Health Care Services Health Education Type of Outcome Addressed Physical Health
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Gang Resistance Education and Training (G.R.E.A.T.)

Program Info

Healthy and Safe Children
Children and youth not engaging in violent behavior or displaying serious conduct problems
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Elementary School Middle School Out of School Time Type of Service Family Support Mentoring Parent Education Youth Development ...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Get Real About AIDS

Program Info

Healthy and Safe Children
Youths abstaining from sexual activity or not engaging in risky sexual behavior
Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School High School Community-Based Service Provider Type of Service Health Education Youth Development Type ...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Go Grrrls

Program Info

Healthy and Safe Children
Children not experiencing anxiety or mood disorders, such as depression Children experiencing good physical health
Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Out of School Time Type of Service Youth Development Type of Outcome Addressed Mental Health
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing
Curriculum

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Guiding Good Choices

Program Info

Healthy and Safe Children
Youths not using alcohol, tobacco, or illegal drugs
Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School High School Community-Based Service Provider Health Care Provider Type of Service Family Support Hea...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Head Start

Program Info

Healthy and Safe Children Children Ready for School
Children ages 0 to 5 exhibiting age-appropriate mental and physical development Children experiencing good physical health
Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Type of Service Family Support Instructional Support Parent Education Type of Outcome Addressed Behavior Problems Cognitive Development/Sc...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing
Cost

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Healthy Families New York (HFNY)

Program Info

Healthy and Safe Children
Children not experiencing physical, psychological or emotional abuse Babies born weighing more than 5.5 pounds and improving outcomes for low birth weight babies
Age of Child Early Childhood (0-8) Type of Setting Home Visiting Type of Service Family Support Health Care Services Health Education Parent Education Type of Outcome Addressed Child Abuse and Neglect Physi...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Healthy Start

Program Info

Healthy and Safe Children
Babies born weighing more than 5.5 pounds and improving outcomes for low birth weight babies
Age of Child Early Childhood (0-8) Type of Setting Community-Based Service Provider Health Care Provider Type of Service Family Support Health Care Services Health Education Parent Education Type of Outcome Add...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing
Curriculum

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Healthy Steps for Young Children

Program Info

Healthy and Safe Children Children Ready for School
Children ages 0 to 5 exhibiting age-appropriate mental and physical development Children experiencing good physical health
Age of Child Early Childhood (0-8) Type of Setting Health Care Provider Home Visiting Type of Service Health Care Services Health Education Parent Education Type of Outcome Addressed Physical Health
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

HighScope Perry Preschool Program

Program Info

Healthy and Safe Children Children Ready for School Children Succeeding in School
Students performing at grade level or meeting state curriculum standards Students graduating from high school Children ages 0 to 5 exhibiting age-appropriate mental and physical development Children and youth not engaging in violent behavior or displa...
Age of Child Early Childhood (0-8)
Type of Setting Child Care/Preschool Home Visiting Type of Service Instructional Support Parent Education Type of Outcome Addressed Cognitive Development/School Performance Juvenile Justice Poverty/Welfare Viol...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Hip-Hop to Health Jr.

Program Info

Healthy and Safe Children
Children experiencing good physical health
Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Type of Service Health Education Type of Outcome Addressed Physical Health
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail
Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Incredible Years

Program Info

Healthy and Safe Children
Children and youth not engaging in violent behavior or displaying serious conduct problems
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Child Care/Preschool Elementary School Out of School Time Community-Based Service Provider Type of Service Family Support Instructiona...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing
Curriculum

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Infant Health and Development Program

Program Info

Healthy and Safe Children Children Ready for School Children Succeeding in School
Students performing at grade level or meeting state curriculum standards Children ages 0 to 5 exhibiting age-appropriate mental and physical development Babies born weighing more than 5.5 pounds and improving outcomes for low birth weight babies
Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Health Care Provider Home Visiting Type of Service Family Support Health Care Services Health Education Parent Education Type of Outco...
Proven/Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Know Your Body

Program Info

Healthy and Safe Children
Youths not using alcohol, tobacco, or illegal drugs
Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School Type of Service Health Education Youth Development Type of Outcome Addressed Physical Health Substanc...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

LifeSkills Training

Program Info

Healthy and Safe Children
Youths not using alcohol, tobacco, or illegal drugs Youths abstaining from sexual activity or not engaging in risky sexual behavior Children and youth not engaging in violent behavior or displaying serious conduct problems
Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School High School Type of Service Health Education Youth Development Type of Outcome Addressed Physical Hea...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Making Proud Choices!

Program Info

Healthy and Safe Children
Youths abstaining from sexual activity or not engaging in risky sexual behavior
Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Middle School High School Out of School Time Community-Based Service Provider Health Care Provider Type of Service Health Education ...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Midwestern Prevention Project/Project STAR

Program Info

Healthy and Safe Children
Youths not using alcohol, tobacco, or illegal drugs
Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Middle School Type of Service Parent Education Youth Development Type of Outcome Addressed Physical Health Substance Use and Dependence
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Components
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Multisystemic Therapy (MST)

Program Info

Healthy and Safe Children Strong Families
Youths not using alcohol, tobacco, or illegal drugs Children living in a permanent home Children and youth not engaging in violent behavior or displaying serious conduct problems
Age of Child Adolescence (13-18) Type of Setting High School Community-Based Service Provider Health Care Provider Type of Service Family Support Parent Education Youth Development Type of Outcome Addressed ...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing and Training

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

National Guard Youth ChalleNGe Program

Program Info

Healthy and Safe Children Children Succeeding in School
Students graduating from high school Children and youth not engaging in violent behavior or displaying serious conduct problems Children experiencing good physical health
Age of Child Adolescence (13-18) Type of Setting High School Out of School Time Type of Service Instructional Support Mentoring Youth Development Type of Outcome Addressed Cognitive Development/School Performan...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing
Curriculum

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

New Hope Project

Program Info

Healthy and Safe Children Children Succeeding in School Strong Families
Students performing at grade level or meeting state curriculum standards Families increasing economic self-sufficiency Children and youth not engaging in violent behavior or displaying serious conduct problems
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Child Care/Preschool Community-Based Service Provider Health Care Provider Type of Service Family Support Type of O...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

New York City's Small Schools of Choice

Program Info

Children Succeeding in School
Students performing at grade level or meeting state curriculum standards Students graduating from high school
Age of Child Adolescence (13-18) Type of Setting High School Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail
Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Newborn Individualized Developmental Care and Assessment Program (NIDCAP)

Program Info

Healthy and Safe Children Children Ready for School
Children ages 0 to 5 exhibiting age-appropriate mental and physical development Babies born weighing more than 5.5 pounds and improving outcomes for low birth weight babies Children experiencing good physical health
Age of Child Early Childhood (0-8) Type of Setting Health Care Provider Type of Service Case Management Family Support Health Care Services Health Education Type of Outcome Addressed Cognitive Development/Schoo...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing
Cost

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Nurse Family Partnership

Program Info

Healthy and Safe Children Children Succeeding in School Strong Families
Youths not using alcohol, tobacco, or illegal drugs Students performing at grade level or meeting state curriculum standards Children not experiencing physical, psychological or emotional abuse Families increasing economic self-sufficiency Babies born...
Age of Child Early Childhood (0-8) Type of Setting Health Care Provider Home Visiting Type of Service Family Support Health Care Services Health Education Parent Education Type of Outcome Addressed Behavior...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Parent-Child Home Program

Program Info

Children Ready for School
Children ages 0 to 5 exhibiting age-appropriate mental and physical development
Age of Child Early Childhood (0-8) Type of Setting Home Visiting Type of Service Parent Education Type of Outcome Addressed Cognitive Development/School Performance
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing
Curriculum

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Parents as Teachers

Program Info

Healthy and Safe Children Children Ready for School Children Succeeding in School
Students performing at grade level or meeting state curriculum standards Children ages 0 to 5 exhibiting age-appropriate mental and physical development Babies born weighing more than 5.5 pounds and improving outcomes for low birth weight babies
Age of Child Early Childhood (0-8) Type of Setting Community-Based Service Provider Home Visiting Type of Service Family Support Health Care Services Parent Education Type of Outcome Addressed Behavior Problems...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Parents' Fair Share

Program Info

Strong Families
Fathers maintaining regular involvement with their children
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Community-Based Service Provider Type of Service Family Support Parent Education Type of Outcome Addressed Poverty/...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Partners in Reading

Program Info

Children Succeeding in School
Students performing at grade level or meeting state curriculum standards
Age of Child Early Childhood (0-8) Type of Setting Elementary School Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Peer-Assisted Learning Strategies (PALS)

Program Info

Children Succeeding in School
Students performing at grade level or meeting state curriculum standards
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School High School Type of Service Instructional Support Type of Outcome Addressed Cog...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Postponing Sexual Involvement/Human Sexuality Educational Series

Program Info

Healthy and Safe Children
Youths abstaining from sexual activity or not engaging in risky sexual behavior
Age of Child Adolescence (13-18) Type of Setting Middle School Health Care Provider Type of Service Health Education Youth Development Type of Outcome Addressed Physical Health Teen Sex/Pregnancy
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Primary and Secondary Control Enhancement Training

Program Info

Healthy and Safe Children
Children not experiencing anxiety or mood disorders, such as depression
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Elementary School Middle School Type of Service Health Care Services Youth Development Type of Outcome Addressed Mental Health
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Project ALERT

Program Info

Healthy and Safe Children
Youths not using alcohol, tobacco, or illegal drugs
Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Middle School Type of Service Health Education Parent Education Youth Development Type of Outcome Addressed Physical Health Substance...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Project Northland

Program Info

Healthy and Safe Children
Youths not using alcohol, tobacco, or illegal drugs
Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Middle School Type of Service Health Education Parent Education Youth Development Type of Outcome Addressed Physical Health Substance...
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Project TRUST

Program Info

Healthy and Safe Children
Children not experiencing physical, psychological or emotional abuse
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Elementary School Type of Service Youth Development Type of Outcome Addressed Child Abuse and Neglect
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Quantum Opportunity Program (QOP)

Program Info

Healthy and Safe Children Children Succeeding in School
Students graduating from high school Youths abstaining from sexual activity or not engaging in risky sexual behavior
Age of Child Adolescence (13-18) Type of Setting High School Community-Based Service Provider Type of Service Case Management Instructional Support Mentoring Youth Development Type of Outcome Addressed Beha...
Proven/Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Details
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Reaching Educators, Children, and Parents (RECAP)

Program Info

Healthy and Safe Children
Children and youth not engaging in violent behavior or displaying serious conduct problems
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Child Care/Preschool Elementary School Type of Service Instructional Support Parent Education Youth Development Type of Outcome Address...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Reading Recovery

Program Info

Children Succeeding in School
Students performing at grade level or meeting state curriculum standards
Age of Child Early Childhood (0-8) Type of Setting Elementary School Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance
Proven

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Reciprocal Teaching

Program Info

Children Succeeding in School
Students performing at grade level or meeting state curriculum standards
Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School High School Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School ...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Reducing the Risk

Program Info

Healthy and Safe Children
Youths abstaining from sexual activity or not engaging in risky sexual behavior
Age of Child Adolescence (13-18) Type of Setting Middle School High School Community-Based Service Provider Health Care Provider Type of Service Health Education Youth Development Type of Outcome Addressed ...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Resolving Conflict Creatively Program (RCCP)

Program Info

Healthy and Safe Children Children Succeeding in School
Students performing at grade level or meeting state curriculum standards Children and youth not engaging in violent behavior or displaying serious conduct problems
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Elementary School Middle School Type of Service Youth Development Type of Outcome Addressed Behavior Problems Cognitive Development/Sch...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

SafeCare

Program Info

Healthy and Safe Children
Children not experiencing physical, psychological or emotional abuse
Age of Child Early Childhood (0-8) Type of Setting Home Visiting Type of Service Family Support Parent Education Type of Outcome Addressed Child Abuse and Neglect
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing
Curriculum

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Safe Child Program

Program Info

Healthy and Safe Children
Children not experiencing physical, psychological or emotional abuse
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Child Care/Preschool Elementary School Type of Service Youth Development Type of Outcome Addressed Child Abuse and Neglect
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Details
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Seattle Social Development Project

Program Info

Healthy and Safe Children Children Succeeding in School
Youths not using alcohol, tobacco, or illegal drugs Students graduating from high school Youths abstaining from sexual activity or not engaging in risky sexual behavior Children and youth not engaging in violent behavior or displaying serious conduct ...
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Elementary School Middle School Type of Service Health Education Instructional Support Parent Education Youth Development Type of O...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design/Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Second Step Violence Prevention

Program Info

Healthy and Safe Children
Children and youth not engaging in violent behavior or displaying serious conduct problems
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Child Care/Preschool Elementary School Middle School Type of Service Parent Education Youth Development Type of Outcome Addressed B...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Self Center (School-Linked Reproductive Health Services)

Program Info

Healthy and Safe Children
Youths abstaining from sexual activity or not engaging in risky sexual behavior
Age of Child Adolescence (13-18) Type of Setting Middle School High School Health Care Provider Type of Service Health Care Services Health Education Youth Development Type of Outcome Addressed Physical Hea...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing
Curriculum

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Smart Start

Program Info

Children Ready for School
Children ages 0 to 5 exhibiting age-appropriate mental and physical development
Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Community-Based Service Provider Health Care Provider Type of Service Family Support Health Care Services Health Education Parent Educati...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Social Decision Making/Problem Solving

Program Info

Healthy and Safe Children
Children and youth not engaging in violent behavior or displaying serious conduct problems Children not experiencing anxiety or mood disorders, such as depression
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Elementary School Middle School Community-Based Service Provider Type of Service Youth Development Type of Outcome ...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

SPORT

Program Info

Healthy and Safe Children
Youths not using alcohol, tobacco, or illegal drugs Children experiencing good physical health
Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Middle School High School Out of School Time Type of Service Health Education Type of Outcome Addressed Physical Health Substance Use...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing
Curriculum

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Student Achievement Guarantee in Education (SAGE)

Program Info

Children Succeeding in School
Students performing at grade level or meeting state curriculum standards
Age of Child Early Childhood (0-8) Type of Setting Elementary School Out of School Time Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Syracuse Family Development Research Program

Program Info

Healthy and Safe Children Children Succeeding in School
Students performing at grade level or meeting state curriculum standards Children and youth not engaging in violent behavior or displaying serious conduct problems
Age of Child Early Childhood (0-8) Type of Setting Child Care/Preschool Community-Based Service Provider Home Visiting Type of Service Case Management Family Support Health Education Instructional Support ...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Talent Development Secondary

Program Info

Children Succeeding in School
Students performing at grade level or meeting state curriculum standards
Age of Child Adolescence (13-18) Type of Setting Middle School High School Type of Service Instructional Support Youth Development Type of Outcome Addressed Cognitive Development/School Performance
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing
Curriculum

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Targeted Reading Intervention (TRI)

Program Info

Children Succeeding in School
Students performing at grade level or meeting state curriculum standards
Age of Child Early Childhood (0-8) Type of Setting Elementary School Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing
Curriculum

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Team Accelerated Instruction: Math

Program Info

Children Succeeding in School
Students performing at grade level or meeting state curriculum standards
Age of Child Middle Childhood (9-12) Type of Setting Elementary School Middle School Type of Service Instructional Support Type of Outcome Addressed Cognitive Development/School Performance
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Teen Talk

Program Info

Healthy and Safe Children
Youths abstaining from sexual activity or not engaging in risky sexual behavior
Age of Child Adolescence (13-18) Type of Setting Middle School High School Out of School Time Community-Based Service Provider Health Care Provider Type of Service Health Education Youth Development Type of...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Text4Health-Adolescents

Program Info

Healthy and Safe Children
Children experiencing good physical health
Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Health Care Provider Type of Service Health Care Services Type of Outcome Addressed Physical Health
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail
Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Triple P Positive Parenting Program

Program Info

Healthy and Safe Children
Children not experiencing physical, psychological or emotional abuse
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Adolescence (13-18) Type of Setting Community-Based Service Provider Home Visiting Type of Service Family Support Parent Education
Type of Outcome Addressed Child Abuse and Neglect Physical Health
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Cost

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Triple Play

Program Info

Healthy and Safe Children
Children experiencing good physical health
Age of Child Middle Childhood (9-12) Adolescence (13-18) Type of Setting Out of School Time Community-Based Service Provider Type of Service Health Education Youth Development Type of Outcome Addressed Physical...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Staffing
Curriculum

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Who Do You Tell?

Program Info

Healthy and Safe Children
Children not experiencing physical, psychological or emotional abuse
Age of Child Early Childhood (0-8) Middle Childhood (9-12) Type of Setting Elementary School Type of Service Parent Education Youth Development Type of Outcome Addressed Child Abuse and Neglect
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing
Financing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed

Wyman's Teen Outreach Program

Program Info

Healthy and Safe Children Children Succeeding in School
Students performing at grade level or meeting state curriculum standards Youths abstaining from sexual activity or not engaging in risky sexual behavior
Age of Child Adolescence (13-18) Type of Setting Middle School High School Out of School Time Community-Based Service Provider Health Care Provider Type of Service Health Education Youth Development Type of...
Promising

Program Overview
Program Participants
Evaluation Methods
Key Evaluation Findings
Probable Implementers
Funding
Implementation Detail

Program Design
Curriculum
Staffing

Issues to Consider
Example Sites
Contact Information
Available Resources
Bibliography
Last Reviewed