Analyzing

What Works in Prevention Principles of Effective Prevention Programs

Maury Nation Vanderbilt University Cindy Crusto Yale University School of Medicine

Abraham Wandersman University of South Carolina Karol L. Kumpfer University of Utah

Diana Seybolt University of Maryland, Baltimore Erin Morrissey-Kane and Katrina Davino University of South Carolina

The high prevalence of drug abuse, delinquency, youth violence, and other youth problems creates a need to identify and disseminate effective prevention strategies. General principles gleaned from effective interventions may help prevention practitioners select, modify, or create more effective programs. Using a review-of-reviews ap- proach across 4 areas (substance abuse, risky sexual be- havior, school failure, and juvenile delinquency and vio- lence), the authors identified 9 characteristics that were consistently associated with effective prevention programs: Programs were comprehensive, included varied teaching methods, provided sufficient dosage, were theory driven, provided opportunities for positive relationships, were ap- propriately timed, were socioculturally relevant, included outcome evaluation, and involved well-trained staff. This synthesis can inform the planning and implementation of problem-specific prevention interventions, provide a ratio- nale for multiproblem prevention programs, and serve as a basis for further research.

Recent analyses concerning the status of Americanyouth and families have concluded that the UnitedStates is a nation at risk in regard to many social indicators such as substance abuse, adolescent pregnancy, youth violence, and school dropouts (Bronfenbrenner, Mc- Clelland, Wethington, Moen, & Ceci, 1996; Weissberg, Walberg, O’Brien, & Kuster, 2003). In addition to the obvious public health concerns, the cost of the social, therapeutic, and rehabilitative services needed to address these problems has made the search for effective preven- tion programs essential.

Our work with community coalitions indicates that community-based preventionists and mental health practi- tioners are seeking to provide effective prevention pro- grams. The science-based research and evaluation literature has consistently shown that a number of prevention pro- grams are beneficial in helping youth to avoid numerous problems (Albee & Gullotta, 1997; Durlak & Wells, 1997; Price, Cowen, Lorion, & Ramos-McKay, 1989; Weissberg & Greenberg, 1998). However, the difficulty in replicating expensive, science-based prevention models or proprietary commercial products has resulted in many local agencies

creating or adapting their own prevention programs with marginal effects. Consequently, there is a gap between the science-based prevention programs and what is provided by practitioners to families and children in the United States (Morrissey et al., 1997). As private and public funders require greater accountability, practitioners are asking the question: What practical information does pre- vention research have to offer to improve the effectiveness of prevention practice? Furthermore, granting agencies and practitioners ask questions such as, What are the evidence- based programs that work? What is the essence of good prevention programs? and Whom should these programs target?

Reviews of prevention programs have provided some answers to these questions. Some reviews provide case studies of effective programs (Albee & Gullotta, 1997) or summarize the research within a particular content area such as substance abuse (Center for Substance Abuse Pre- vention [CSAP], 2001; Tobler & Stratton, 1997), teen pregnancy (Kirby, 1997), and HIV/AIDS (Choi & Coates, 1994). An advantage of these reviews is that they can bring prevention theory within a content area to bear in drawing conclusions about the effectiveness of programs. These reviews suggest that there are some principles that tran-

Editor’s note. This special issue was developed by Roger P. Weissberg and Karol L. Kumpfer as a result of the work of the APA Task Force on Prevention: Promoting Strength, Resilience, and Health in Young People.

Author’s note. Maury Nation, Department of Human and Organizational Development, Vanderbilt University; Cindy Crusto, Department of Psy- chiatry, Yale University School of Medicine; Abraham Wandersman, Erin Morrissey-Kane, and Katrina Davino, Department of Psychology, Uni- versity of South Carolina; Karol L. Kumpfer, Department of Health Promotion and Education, University of Utah; Diana Seybolt, School of Medicine, University of Maryland, Baltimore.

Some of the supporting documentation and references were omitted because of space limitations.

We thank the following people for recommending reviews and providing feedback on earlier drafts of this article: Joy Dryfoos, Joseph Durlak, Anthony Biglan, and Maurice Elias.

Correspondence concerning this article should be addressed to Maury Nation, Department of Human and Organizational Development, Vanderbilt University, Peabody College, #90, Nashville, TN 37203. E-mail: [email protected]

449June/July 2003● American Psychologist Copyright 2003 by the American Psychological Association, Inc. 0003-066X/03/$12.00 Vol. 58, No. 6/7, 449 – 456 DOI: 10.1037/0003-066X.58.6-7.449

scend the individual programs or content areas. Prevention now has a sufficient knowledge base to begin a meta- assessment of the characteristics of effective prevention programming and thereby assist practitioners in selecting programs that are likely to produce positive outcomes.

The first step of this process has been accomplished by reviews of prevention programs specific to one outcome area. By comparing findings across problem outcome areas, the usefulness of these studies can be strengthened. Dry- foos (1990) made a vital contribution toward this goal by reviewing over 100 prevention programs related to sub- stance abuse, teen pregnancy, school dropout, and juvenile delinquency. Her review yielded several key characteristics associated with successful programs, such as the provision of intense individualized attention, intervention in several domains of the child’ s life, early identification of and intervention in the development of problem behaviors, training in social skills, and engagement of peers and parents in the intervention. Similar reviews conducted on school-based curricula (Elias, Gager, & Leon, 1997) and programs focused on children and adolescents (Weissberg & Greenberg, 1998) continue to identify the types of in- terventions that work and to suggest general principles of effective prevention.

To complement earlier reviews, we used a review-of- reviews approach to identify general principles of effective prevention programs that might transcend specific content areas. At the start of this process, we placed some limits on the scope of the reviews. First, we limited our review to four content areas: (a) substance abuse—prevention of use/abuse of alcohol, tobacco, or other drugs; (b) risky sexual behavior—prevention of unwanted pregnancies and HIV/AIDS; (c) school failure—prevention of general aca- demic problems and high school dropout; and (d) juvenile delinquency and violence—prevention of aggressive or an- tisocial behavior. Although this is not a comprehensive list of issues affecting young people, they are critical public health issues, and our work with community-based practi- tioners indicates these are priority areas.

A second important limit involved the types of pre- vention programs that would be included in our review of reviews. The Institute of Medicine (Mrazek & Haggerty, 1994) identified three categories of prevention that are appropriate for participants with different levels of risk factors: universal, selective, and indicated. A complete survey of all three types of preventive interventions was not attempted because the theory, goals, and structure of indi- cated interventions are significantly different from those of universal and selective interventions and therefore may limit the applicability of the results to any of the categories. Consequently, we limited the review for this special issue on primary prevention to reviews of universal and selective prevention programs.

Search Method We conducted a literature search through PsycLIT and Criminal Justice Abstracts from 1990 to 1999 and con- sulted key informants (see acknowledgments) for journal articles and book chapters that reviewed the efficacy of

prevention programs. Multiple selection criteria were re- quired for inclusion of an article. First, the article had to be a narrative literature review that summarized the results of prevention research in one of our selected content areas. Second, the article had to go beyond lists of best practices or describing the status of research to provide an explicit discussion of the common features of effective program or recommendations (based on the review) for new content area–specific prevention programs. Articles that only dis- cussed theoretical issues (e.g., reviews of etiological theo- ries and implications for prevention) were excluded. Fi- nally, the reviews were limited to one review per first author (unless it was clear the reviews used different data) to avoid including multiple reviews based on the same data. We identified 35 journal articles, books, or book chapters that fit our criteria (see Table 1). Although our search efforts may not have resulted in an exhaustive coverage of the field, there is a sufficient critical mass to provide practitioners and future reviews principles that merit con- sideration when designing and implementing effective pre- vention programming.

We reviewed the articles meeting the inclusion criteria to identify the characteristics of effective programs. Be- cause all of the articles included in this review provided explicit lists or clear sections in which characteristics were discussed, this step involved merely writing down those characteristics identified in the list. This resulted in a listing of 252 characteristics from 35 articles. Next, two members of the research team independently coded the characteris- tics (based on their similarity) into categories, with 84% agreement. From this process, we constructed a list of characteristics that were important in addressing each prob- lem behavior. From those characteristics, we looked for patterns that might indicate that certain characteristics were generalizable. Principles were chosen based on the percent- age of reviews endorsing a characteristic. Support for the nine identified principles discussed in our review ranged from strong (80% of reviews indicating it was an important characteristic) to moderate (31% endorsement). Then there was a sharp drop off in endorsement of principles and therefore they were not further identified in this article. Table 2 provides the percentage of reviews endorsing each principle.

Our analysis yielded nine principles associated with effective prevention programs that were related to three broad areas of prevention programming: program charac- teristics, matching programs to target population, and im- plementing and evaluating prevention programs. There were five principles associated with program characteris- tics: Programs (a) were comprehensive, (b) included varied teaching methods, (c) provided sufficient dosage, (d) were theory driven, and (e) provided opportunities for positive relationships. Two principles were specifically related to matching programs to the target group: Programs (a) were appropriately timed and (b) were socioculturally relevant. Finally, there were two principles related to program im- plementation and evaluation: Programs (a) included out- come evaluation and (b) involved well-trained staff. Table

450 June/July 2003 ● American Psychologist

3 provides definitions of the principles. The remainder of this article describes these principles in more detail.

Principles Related to Program Characteristics The review identified five program characteristics associ- ated with effective prevention programs. These effective

qualities of the interventions or curricula are presented in order of the strength of support for the principle.

Comprehensive We define comprehensive as providing an array of inter- ventions to address the salient precursors or mediators of the target problem. There are two important dimensions to consider for comprehensive programming: multiple inter- ventions and multiple settings. Multiple interventions refers to the importance of having several interventions address- ing the problem behavior. In the prevention of unwanted pregnancies, reviews indicated that successful programs incorporate a combination of interventions focused on in- creasing information and awareness, promoting skill devel- opment, and providing reproductive health services (Miller & Paikoff, 1992). Substance abuse prevention reviews also indicated that multimodal interventions that increased awareness and encouraged the development of specific skills were associated with positive outcomes.

Multiple settings refers to the need to engage the systems that have an impact on the development of the problem behavior. Several reviews indicated the need to address community or school norms related to the problem behaviors (Center for Substance Abuse Prevention, 1996; Janz, Zimmerman, Wren, & Israel, 1996). Other reviews suggested that combined parent, peer, and school interven- tions support positive outcomes (e.g., Sagrestano & Paikoff, 1997). Initial assessments that identify important risk and protective factors provide guidance about which systems to include in the program. For instance, Hawkins and Catalano (1992) argued that drug prevention programs should address risk and protective factors across domains or settings (e.g., community, family, school, peer group) that have primary influence on the participants (also see Kumpfer, 1997).

Varied Teaching Methods The majority of characteristics coded under this principle emphasized the need for some type of active, skills-based component in preventive interventions. Effective preven- tion programs involve interactive instruction (Tobler & Stratton, 1997) and provide active, hands-on experiences that increase the participants’ skills (Dusenbury & Falco, 1995). Although there appeared to be consensus that skill development is important, the nature of the skills varied depending on the target behavior. The National Institute on Drug Abuse (NIDA, 1997) concluded that programs that prevent alcohol and drug use help participants develop resistance skills, including the ability to be assertive and effectively communicate around issues related to drug use. Similar recommendations were suggested for preventing problematic sexual behavior. Kirby (1997) reported that effective programs provided verbal or written practice in negotiating situations that might lead to sexual intercourse. However, for school failure, the skill focus was distinctly different. Slavin and colleagues (Slavin, Karweit, & Wasik, 1992/1993) emphasized the importance of facilitating the development of cognitive, language, and social skills as a way of ensuring success in school. Despite the differences

Table 1 Articles Included in the Review of Reviews

Topic and authors No. of studies reviewed

Substance abuse Center for Substance Abuse

Prevention (1996) 12 Center for Substance Abuse

Prevention (1997) More than 309 Durlak (1997) Unspecified Dusenbury and Falco (1995) Unspecified General Accounting Office (1992) 10 Hansen (1992) 12 Kumpfer and Alvarado (1995) 25 Lewis, Battistich, and Schaps (1990) Unspecified May and Moran (1995) 29 National Institute on Drug Abuse

(1997) 10 Norman and Turner (1993) 30 Paglia and Room (1999) Unspecified

Risky sexual behavior Choi and Coates (1994) 30 Fisher and Fisher (1992) 48 Frost and Forrest (1995) 5 Holtgrave et al. (1995) 23 Janz, Zimmerman, Wren, and Israel

(1996) 37 Kirby (1997) 50 Miller and Paikoff (1992) 9 Nitz (1999) Unspecified Ogletree, Riezno, Drolet, and Fetro

(1995) 5 Sagrestano and Paikoff (1997) 4 White and White (1991) 24

School failure/dropout Carnahan (1994) Unspecified Carlton and Winsler (1999) Unspecified Durlak (1997) Unspecified Ramey and Ramey (1992) Unspecified Slavin, Karweit, and Wasik

(1992/1993) Unspecified Delinquency and violence

Catalano, Arthur, Hawkins, Berglund, and Olson (1998) Unspecified

Elliot (1998) 10 Mulvey, Arthur, and Reppucci (1993) Unspecified Tolan and Guerra (1994) Unspecified U.S. Department of Justice (1995a) Unspecified U.S. Department of Justice (1995b) 50 Zigler, Taussig, and Black (1992) 5

451June/July 2003 ● American Psychologist

in the types of skills emphasized, there is general agree- ment that programs should be careful not to rely too much on knowledge, information, or group discussions as the major change mechanism (Durlak, 1997).

Sufficient Dosage This principle refers to the need for participants to be exposed to enough of the intervention for it to have an effect. Dosage, or program intensity, may be measured in quantity and quality of contact hours. Aspects of dosage include the session length, number of sessions, spacing of sessions, and the duration of the total program. Whereas many reviews refer to this as a general principle with comments like “ continue (intervention) over a long period of time,” other reviews make specific recommendations

regarding the number and frequency of contacts (U.S. Department of Justice, 1995a). Also, some reviews indi- cated that intensity needs to be gauged to the risk faced by the individual: The greater the needs or deficits of the participants, the greater the dosage or intensity of the intervention (Carnahan, 1994).

In addition to initial exposure to the intervention, effective interventions generally include some type of fol- low-up or booster sessions to support durability of impact. In many cases, the long-term effects of preventive inter- ventions remain unknown because of a lack of research data on long-term results (e.g., Frost & Forrest, 1995). In a meta-analytic study of school-based prevention programs with controlled outcome studies, Durlak and Wells (1997) found that few studies included follow-up assessments.

Table 2 Percentage of Reviews Endorsing Prevention Principles by Content Area

Principle Substance abuse

(n � 12) Risky sexual behavior

(n � 11) Delinquency and violence

(n � 7) School failure

(n � 5) Total

(N � 35)

Comprehensive 83 73 100 60 80 Varied teaching methods 83 82 57 20 69 Sufficient dosage 42 55 86 60 57 Theory driven 58 73 29 0 49 Positive relationships 33 0 57 80 34 Appropriately timed 67 73 57 60 66 Socioculturally relevant 67 82 29 0 54 Outcome evaluation 42 64 29 0 40 Well-trained staff 33 27 29 40 31

Table 3 Definitions of the Principles of Effective Programs

Principle Definition

Comprehensive Multicomponent interventions that address critical domains (e.g., family, peers, community) that influence the development and perpetuation of the behaviors to be prevented

Varied teaching methods Programs involve diverse teaching methods that focus on increasing awareness and understanding of the problem behaviors and on acquiring or enhancing skills

Sufficient dosage Programs provide enough intervention to produce the desired effects and provide follow-up as necessary to maintain effects

Theory driven Programs have a theoretical justification, are based on accurate information, and are supported by empirical research

Positive relationships Programs provide exposure to adults and peers in a way that promotes strong relationships and supports positive outcomes

Appropriately timed Programs are initiated early enough to have an impact on the development of the problem behavior and are sensitive to the developmental needs of participants

Socioculturally relevant Programs are tailored to the community and cultural norms of the participants and make efforts to include the target group in program planning and implementation

Outcome evaluation Programs have clear goals and objectives and make an effort to systematically document their results relative to the goals

Well-trained staff Program staff support the program and are provided with training regarding the implementation of the intervention

452 June/July 2003 ● American Psychologist

When follow-up assessments were conducted, the studies indicated that the effects of many preventive interventions tend to gradually decay over time (Zigler, Taussig, & Black, 1992). This suggests that booster sessions focusing on prior skills learned or on new developmentally appro- priate skills are needed to maintain positive outcomes.

Theory Driven

This principle refers to the need for scientific justification of a preventive intervention. Although this principle may seem basic, an examination of actual prevention programs used in many communities and schools indicates that it is sometimes overlooked. In the areas of risky sexual behav- ior, one study indicated that the majority of preventive interventions were based on a blend of logic and past experiences (Fisher & Fisher, 1992). Across multiple con- tent areas, there was consistent emphasis on the importance of theory-based interventions that have a basis in research. Two types of theories that play a role in prevention pro- gramming are etiological theories and intervention theo- ries. Etiological theories focus on the causes (e.g., risk or protective factors and processes) of the targeted problem (Kumpfer, 1997). Intervention theories are focused on the best methods for changing these etiological risks. Once the causes are identified, effective prevention programs are then based on empirically tested intervention theories shown to produce the desired changes in the causes and ultimately in the behavior being prevented.

Positive Relationships

Providing opportunities for children to develop strong, positive relationships was consistently associated with pos- itive outcomes. Some reviews emphasized the importance of improving parent–child relationships and parenting skills (Kumpfer & Alvarado, 1995), whereas others fo- cused on leveraging peer influences on preventing problem behavior (Mulvey, Arthur, & Reppucci, 1993). Reviews of substance abuse prevention emphasized the necessity to have strong connections between children and significant others (including peers, teachers, community members) as a way of preventing drug use (National Institute on Drug Abuse, 1997). The reviews supported the idea that it is critical for children to have a strong relationship with at least one adult. This was most evident in the areas of school failure and dropout, in which almost all the reviews high- lighted the need for strong relationships with positive adult models. Several reviews endorsed the use of adult mentors to encourage the development of these relationships. Eval- uation of quality mentoring programs like Big Brothers and Big Sisters suggests that it is a promising strategy (Gross- man & Tierney, 1998).

However, individualized intervention was not uni- formly endorsed as being essential. Tolan and Guerra (1994) suggested that intense individual intervention (e.g., therapy or case management) is not sufficient to prevent delinquency. In light of their findings, it seems important to study the source of the individualized attention. The impact of attention from a mentor or teacher may be qualitatively

different from that of a probation officer or therapist, even if it occurs in equal amounts.

Principles Related to Matching the Program With a Target Population A subset of the principles seemed specifically focused on selecting programs that are appropriate for the identified population. The reviews indicate that even programs that are of good quality need to be matched to the needs of the participants to maximize effectiveness.

Appropriately Timed

Interventions should be timed to occur in a child’ s life when they will have maximal impact. Unfortunately, many programs tend to be implemented when children are al- ready exhibiting the unwanted behavior or when the pro- grams are developmentally less relevant to the participants. This led the Institute of Medicine to warn that “ if the preventive intervention occurs too early, its positive effects may be washed out before onset; if it occurs too late, the disorder may have already had its onset” (Mrazek & Hag- gerty, 1994, p. 14).

Prevention programs should be timed to focus on changeable precursor behaviors prior to the full-blown problem behavior being prevented (Dryfoos, 1990). The importance of early intervention was demonstrated in the “ Reducing the Risk” HIV/AIDS prevention program. The program was effective in reducing risky behavior among all adolescents except those who were sexually active prior to beginning the program (Kirby, Barth, Leland, & Fetro, 1991). Early intervention allows programs to have a chance to affect the developmental trajectory of the problem be- havior (National Institute on Drug Abuse, 1997). This suggests that the elementary school to middle school tran- sition may be an important window for intervention.

Also coded under this principle were several charac- teristics related to the developmental appropriateness of the intervention. Several reviews indicated that programs needed to have materials that were tailored to the intellec- tual, cognitive, and social development of the participants (Zigler et al., 1992). The importance of the factor was most clearly indicated in studies of adolescent sexual behavior, in which changing the message of the intervention accord- ing to the developmental stage of the participants was associated with positive outcomes (Miller & Paikoff, 1992).

Socioculturally Relevant

The relevance of prevention programs to the participants appears to be a primary concern in producing positive outcomes. The concept of relevance spanned a variety of dimensions, including local community norms and cultural beliefs and practices (e.g., Ramey & Ramey, 1992). De- signing a prevention program to be culturally appropriate is one recommendation for increasing relevance. Culturally tailoring prevention programs goes beyond surface struc- ture language translation to deep structure modifications sensitive to cultural factors that influence development and

453June/July 2003 ● American Psychologist

receptiveness to the intervention (Resnicow, Solar, Braith- waite, Ahluwalia, & Butler, 2000). Cultural attitudes have been shown to be important factors in sexual behavior and aggressive/delinquent behavior. Kirby (1997), for example, reported that effective pregnancy prevention programs tai- lored statistics and example situations to the culture of the participants.

In addition, programs must address the individual needs of participants. Carnahan (1994) documented the importance of tailoring the intervention for preventing stu- dents from dropping out of school, citing that one-size- fits-all programs appear to work best for those who least need the intervention and may exacerbate the problem faced by those most in need. When a prevention program is not relevant, programs may have difficulty in recruiting and retaining high-risk participants (Kumpfer & Alvarado, 1995). To improve program and evaluation relevance, Dry- foos (1990) and Janz et al. (1996) suggested that the participants in the intervention be included in the program planning and implementation to ensure that their needs are recognized.

Principles Related to Implementation and Evaluation of Prevention Programs In addition to the intervention or curriculum itself, we found that characteristics related to implementation and evaluation also associate with effective prevention programming.

Outcome Evaluation

The evaluation of prevention programs is necessary to determine program effectiveness. Otherwise, practitioners may assume that a program is effective on the basis of anecdotal or case study evidence. As evaluation has be- come more common, the results indicate that many pro- grams that are anecdotally believed to be successful may actually not be effective. For example, the most widely disseminated and commercially marketed drug prevention programs are not as effective as many of the research-based programs listed in NIDA’ s review (National Institute on Drug Abuse, 1997) or in the CSAP National Registry of Effective Prevention Programs (see www.samhsa.gov/ csap/modelprograms). Evaluation strategies that emphasize continuous quality improvement can be useful in feeding back information at several stages of the intervention (Wandersman et al., 1998). Reviews of HIV/AIDS and alcohol and drugs prevention programs indicate that this type of information may be very important for achieving positive outcomes for prevention programs (Dusenbury & Falco, 1995; Hansen, 2002).

Well-Trained Staff

A high-quality, research-based program can produce dis- appointing results in dissemination field trials if the pro- gram providers are poorly selected, trained, or supervised. The implementation of prevention programs is enhanced when the staff members are sensitive, are competent, and

have received sufficient training, support, and supervision (Lewis, Battistich, & Schaps, 1990). Kirby’ s (1997) review indicated that formalized training for effective unwanted pregnancy prevention programs ranged from six hours to three days. These training programs give teachers or staff practice in implementation and an opportunity to have their questions answered. Evaluations of alcohol and drug pre- vention programs also indicate that training of teachers enhanced the impact of school-based programs (Dusenbury & Falco, 1995). Even when staff members are sufficiently competent, their effectiveness can be limited by high rates of turnover, low morale, or a lack of “ buy-in” (U.S. De- partment of Justice, 1995b).

Discussion and Conclusions Before drawing conclusions based on this review, we want to acknowledge some of its weaknesses. In addition to the fact that this is not an exhaustive list of reviews, the reviews that were included in this process clearly varied in rigor. Some provided documentation of all studies included in the review and only included studies that had published evaluations (e.g., Kirby, 1997). Other reviews appeared to have less documentation and included some community- based programs whose effectiveness were verified by eval- uation reports (e.g., General Accounting Office, 1992). As illustrated in Table 1, for example, some reviews docu- mented the number of programs included in the review and others did not. Also, the apparent lack of reviews of drop- out and school failure prevention programs was troubling. This was likely the result of many of the review efforts related to school issues being focused on promoting aca- demic success rather than preventing school failure (Ross, Powell, & Elias, 2002). The studies that do exist focus more on early intervention with children “ at risk” rather than universal prevention.

Another issue affecting this review was the lack of uniform standards for determining effectiveness. Some re- views were explicit, whereas others provided minimal jus- tification for their definition of effectiveness. In prevention of risky sexual behavior, for example, successful outcomes included appropriate use of contraception, delays in the initiation of sexual activity, and lower rates of pregnancy. Other areas were less explicit and in some cases included changes in attitudes and intentions in addition to behavior change.

Finally, there were some areas in which our review did not overlap with previous multiple content area re- views. For example, Dryfoos (1990) noted the importance of programs being connected to the work world or other opportunity structures. Likewise, we found some factors to be important (e.g., emphasis on the theoretical substrates of prevention programs) that were not cited in some of the other reviews. The differences in our conclusions may be an artifact of the differences in our methods. Dryfoos gathered her data through direct observation and reviews of primary documents of programs in four content areas. Durlak and Wells (1997) focused on examination of the published evaluations of prevention programs and included content areas not included in this review or in the Dryfoos

454 June/July 2003 ● American Psychologist

study. It is likely that closer observation of individual programs made some meta-reviews more sensitive to the nuances of service delivery in prevention programs. In contrast, the review-of-reviews approach benefits from pre- vious authors’ synthesis of content-specific prevention ef- forts. However, this approach is potentially vulnerable to the interpretive and conceptual biases of the previous and current reviewers.

Despite the limitations of this method, there are sev- eral general conclusions that can be drawn. First, this review adds to the mounting evidence that prevention pro- grams that are carefully designed and implemented can be effective in preventing many of the problems facing chil- dren and adolescents. Programs that engage children and their environmental context are most likely to produce change. Similar to Dryfoos (1998), our analysis suggests there is substantial overlap in the principles of effective programs across prevention domains that allow us to iden- tify general principles of effectiveness. In particular, pro- gram characteristics like comprehensiveness, sufficient dosage, skill development focus, and the importance of timing were identified as being important in both reviews. Given this convergent evidence, these common character- istics offer a benchmark for scientists and practitioners involved in designing and implementing problem-specific preventive interventions.

As we reflect on the gap between the science and practice of prevention, our review suggests the results may have several implications for the future of prevention re- search and practice. First, practitioners may not be getting up-to-date information on what works in prevention. These principles could serve as a guide to encourage practitioners to search deeper for prevention programs that reflect these principles. Second, many practitioners cannot afford to implement research-based programs that were developed on well-funded, university-based research grants. Efficacy trials offer a sharp contrast to most prevention programs, which are frequently conducted with small budgets and small staffs (Morrissey et al., 1997). These principles might assist researchers and practitioners in identifying cost-ef- fective ways of implementing the essential elements of programs. Third, there is a call for more systematic pre- vention science research. While content areas differentially endorsed principles, this does not necessarily represent a differential in the importance of the principles. This review could serve as call for researchers to examine the relation- ships between previously ignored issues (e.g., staff train- ing) and program outcomes.

Finally, this review offers a rationale for multiple- problem prevention programs because at-risk children tend to be at risk for multiple negative outcomes as a result of dysfunctional families, neighborhoods, schools, and peer relationships (Donovan, Jessor, & Costa, 1988). In fact, the most frequently endorsed principles (comprehensive, var- ied teaching methods, appropriately timed) support the call for policies that encourage multicomponent, coordinated preventive interventions (Elias, 1995) such as those advo- cated in the Safe Schools/Healthy Students initiative (see http://ojjdp.ncjrs.org/grants/safeschools.html). The grow-

ing consensus among meta-reviews suggests that these common characteristics of effective programs may offer guidelines for conceptualizing and developing these policies.

REFERENCES

*References marked with an asterisk indicate studies included in the review of reviews.

Albee, G., & Gullotta, T. (Eds.). (1997). Primary prevention works. Thousand Oaks, CA: Sage.

Bronfenbrenner, U., McClelland, P., Wethington, E., Moen, P., & Ceci, S. (1996). The state of Americans: This generation and the next. New York: Free Press.

*Carlton, M., & Winsler, A. (1999). School readiness: The need for a paradigm shift. School Psychology Review, 27, 398 –406.

*Carnahan, S. (1994). Preventing school failure and dropout. In R. J. Simeonsson (Ed.), Risk resilience and prevention: Promoting the well- being of all children (pp. 103–123). Baltimore: Paul H. Brooks.

*Catalano, R. F., Arthur, M. W., Hawkins, J. D., Berglund, L., & Olson, J. J. (1998). Comprehensive community- and school-based interven- tions to prevent antisocial behavior. In R. Loeber & D. Farrington (Eds.), Serious and violent juvenile offenders: Risk factors and success- ful interventions (pp. 248 –283). Thousand Oaks, CA: Sage.

*Center for Substance Abuse Prevention. (1996). A review of alternative activities and alternatives programs in youth-oriented prevention. Washington, DC: Author.

*Center for Substance Abuse Prevention. (1997). Selected findings in prevention: A decade of results from the Center for Substance Abuse Prevention (CSAP). Washington, DC: Author.

Center for Substance Abuse Prevention. (2001). Principles of substance abuse prevention (DHHS Publication No. SMA 01-3507). Rockville, MD: National Clearinghouse for Alcohol and Drug Information.

*Choi, K. H., & Coates, T. J. (1994). Prevention of HIV infection. AIDS, 8, 1371–1389.

Donovan, J., Jessor, R., & Costa, F. (1988). Syndrome of problem be- havior in adolescence: A replication. Journal of Consulting and Clin- ical Psychology, 56, 762–765.

Dryfoos, J. G. (1990). Adolescents at risk: Prevalence and prevention. New York: Oxford University Press.

Dryfoos, J. G. (1998). Safe passage: Making it through adolescence in a risky society. New York: Oxford University Press.

*Durlak, J. A. (1997). Successful prevention programs for children and adolescents. New York: Plenum.

Durlak, J. A., & Wells, A. M. (1997). Primary prevention mental health programs for children and adolescents: A meta-analytic review. Amer- ican Journal of Community Psychology, 25, 207–214.

*Dusenbury, L., & Falco, M. (1995). Eleven components of effective drug abuse prevention curricula. Journal of School Health, 65, 420 –425.

Elias, M. J. (1995). Primary prevention as health and social competence promotion. Journal of Primary Prevention, 16, 5–24.

Elias, M. J., Gager, P., & Leon, S. (1997). Spreading a warm blanket of prevention over all children: Guidelines for selecting substance abuse and related prevention curricula for use in the schools. Journal of Primary Prevention, 18, 41–69.

*Elliot, D. S. (1998). Prevention programs that work for youth: Violence prevention. Boulder, CO: Center for the Study and Prevention of Violence.

*Fisher, J. D., & Fisher, W. A. (1992). Changing AIDS-risk behavior. Psychological Bulletin, 111, 455–474.

*Frost, J. J., & Forrest, J. D. (1995). Understanding the impact of effective teenage pregnancy prevention programs. Planning Perspectives, 27, 188 –195.

*General Accounting Office. (1992). Adolescent drug use prevention: Common features of promising community programs. Washington, DC: Author.

Grossman, J. B., & Tierney, J. P. (1998). Does mentoring work? An impact study of the Big Brothers Big Sisters program. Evaluation Review, 22, 403–426.

*Hansen, W. B. (1992). School-based substance abuse prevention: A

455June/July 2003 ● American Psychologist

review of the state of the art in curriculum, 1980 –1990. Health Edu- cation Research, 7, 403–430.

*Hansen, W. B. (2002). Program evaluation strategies for substance abuse prevention. Journal of Primary Prevention, 22, 409 –436.

Hawkins, J. D., & Catalano, R. F., Jr. (1992). Preparing for the drug-free years. San Francisco: Jossey-Bass.

*Holtgrave, D. R., Qualls, N. L., Curran, J. W., Valdiserri, R. O., Guinan, M. E., & Parra, W. C. (1995). An overview of the effectiveness and efficiency of HIV prevention programs. Public Health Reports, 110, 134 –146.

*Janz, N. K., Zimmerman, M. A., Wren, P. A., & Israel, B. A. (1996). Evaluation of 37 AIDS prevention projects: Successful approaches and barriers to program effectiveness. Health Education Quarterly, 23, 80 –97.

*Kirby, D. (1997). No easy answers: Research findings on programs to reduce teen pregnancy. Washington, DC: National Campaign to Pre- vent Teen Pregnancy.

Kirby, D., Barth, R., Leland, N., & Fetro, J. (1991). Reducing the risk: A new curriculum to prevent sexual risk-taking. Family Planning Per- spectives, 23, 253–263.

Kumpfer, K. L. (1997). What works in the prevention of drug abuse: Individual, school and family approaches. Secretary’s youth substance abuse prevention initiative: Resource paper (pp. 69 –105). Washington, DC: Center for Substance Abuse Prevention.

*Kumpfer, K. L., & Alvarado, R. (1995). Strengthening families to prevent drug use in multi-ethnic youth. In G. J. Botvin, S. Schinke, & M. A. Orlandi (Eds.), Drug abuse prevention with multiethnic youth (pp. 1–22). Thousand Oaks, CA: Sage.

*Lewis, C., Battistich, V., & Schaps, E. (1990). School-based primary prevention: What is an effective program? New Directions for Child Development, 50, 35–59.

*May, P. A., & Moran, J. R. (1995). Prevention of alcohol misuse: A review of health promotion efforts among American Indians. American Journal of Health Promotion, 9, 288 –299.

*Miller, B. C., & Paikoff, R. L. (1992). Comparing adolescent pregnancy prevention programs. In B. C. Miller & R. L. Paikoff (Eds.), Preventing adolescent pregnancy: Model programs and evaluations (pp. 265–284). Newbury Park, CA: Sage.

Morrissey, E., Wandersman, A., Seybolt, D., Nation, M., Crusto, C., & Davino, K. (1997). Toward a framework for bridging the gap between science and practice in prevention: A focus on evaluator and practi- tioner perspective. Evaluation and Program Planning, 20, 367–377.

Mrazek, P., & Haggerty, R. (1994). Reducing risks for mental disorders: Frontiers for preventive intervention research. Washington, DC: Na- tional Academy Press.

*Mulvey, E. P., Arthur, M. W., & Reppucci, N. D. (1993). The prevention and treatment of juvenile delinquency: A review of the research. Clin- ical Psychology Review, 13, 133–167.

*National Institute on Drug Abuse. (1997). Preventing drug use among children and adolescents: A research-based guide. Washington, DC: Author.

*Nitz, K. (1999). Adolescent pregnancy prevention: A review of inter- ventions and programs. Clinical Psychology Review, 19, 457–471.

*Norman, E., & Turner, S. (1993). Adolescent substance abuse prevention programs: Theories, models, and research in the encouraging 80�s. Journal of Primary Prevention, 14, 3–20.

*Ogletree, R. J., Riezno, B. A., Drolet, J. C., & Fetro, J. V. (1995). An assessment of 23 selected school-based sexuality education curricula. Journal of School Health, 65, 186 –191.

*Paglia, A., & Room, R. (1999). Preventing substance use problems among youth: A literature review and recommendations. Journal of Primary Prevention, 20, 145–156.

Price, R. H., Cowen, E. L., Lorion, R. P., & Ramos-McKay, J. (1989). The search for effective prevention programs: What we learned along the way. American Journal of Orthopsychiatry, 59, 49 –58.

*Ramey, S. L., & Ramey, C. T. (1992). Early educational intervention with disadvantaged children: To what effect? Applied and Preventive Psychology, 1, 131–140.

Resnicow, K., Solar, R., Braithwaite, R., Ahluwalia, J., & Butler, J. (2000). Cultural sensitivity in substance abuse prevention. Journal of Community Psychology, 28, 271–290.

*Ross, M. R., Powell, S. R., & Elias, M. (2002). New roles for school psychologists: Addressing the social and emotional learning needs of students. School Psychology Review, 31, 43–52.

*Sagrestano, L. M., & Paikoff, R. L. (1997). Preventing high-risk sexual behavior, sexually transmitted diseases, and pregnancy among adoles- cents. In R. P. Weissberg, T. P. Gullotta, R. L. Hampton, B. A. Ryan, & G. R. Adams (Eds.), Healthy children 2010: Enhancing children’s wellness (pp. 76 –104). Thousand Oaks, CA: Sage.

*Slavin, R. E., Karweit, N. L., & Wasik, B. A. (December 1992/January 1993). Preventing early school failure: What works? Educational Lead- ership, 10 –18.

Tobler, N. S., & Stratton, H. H. (1997). Effectiveness of school-based drug prevention programs: A meta-analysis of the research. Journal of Primary Prevention, 18, 71–128.

*Tolan, P. H., & Guerra, N. G. (1994). Prevention of delinquency: Current status and issues. Applied and Preventive Psychology, 3, 251–273.

*U.S. Department of Justice. (1995a). Delinquency prevention works: Program summary. Washington, DC: Author.

*U.S. Department of Justice. (1995b). Guide for implementing the com- prehensive strategy for serious, violent, and chronic juvenile offenders. Washington, DC: Author.

Wandersman, A., Morrissey, E., Davino, K., Seybolt, D., Crusto, C., Nation, M., et al. (1998). Comprehensive quality programming: The eight essential steps to effective community-directed prevention pro- grams. Journal of Primary Prevention, 19, 3–31.

Weissberg, R. P., & Greenberg, M. T. (1998). School and community competence-enhancement and prevention programs. In W. Damon (Se- ries Ed.) & I. E. Sigel & K. A. Renninger (Vol. Eds.), Handbook of child psychology: Vol. 4. Child psychology in practice (5th ed., pp. 877–954). New York: Wiley.

Weissberg, R. P., Walberg, H. J., O’ Brien, M. U., & Kuster, C. B. (2003). Long-term trends in the well-being of children and youth. Washington, DC: Child Welfare League of America Press.

*White, C. P., & White, M. B. (1991). The adolescent family life act: Content, findings, and policy recommendations for pregnancy preven- tion programs. Journal of Clinical Child Psychology, 20, 58 –70.

*Zigler, E., Taussig, C., & Black, K. (1992). Early childhood intervention: A promising preventative for juvenile delinquency. American Psychol- ogist, 8, 997–1006.

456 June/July 2003 ● American Psychologist