homework help

Effectiveness of the KEEP foster parent intervention during an implementation trial

Joseph M. Price a,c,⁎, Scott C. Roesch a,c, Natalia Escobar Walsh b,c a Department of Psychology, San Diego State University, 5500 Campanile Drive, San Diego, CA 92182, United States b SDSU/UCSD Joint Doctoral Program in Clinical Psychology, 6363 Alvarado, Ct., Suite 103, San Diego, CA, 92120, United States c Child and Adolescent Services Research Center, 3665 Kearny Villa Rd., Suite 200, San Diego, CA 92123, United States

a b s t r a c ta r t i c l e i n f o

Article history: Received 24 August 2012 Accepted 26 September 2012 Available online 2 October 2012

Keywords: Intervention Foster parents Behavior problems

Externalizing behavior problems are highly prevalent among children in foster care, placing them at risk for placement disruptions and later personal and social maladjustment. The KEEP foster parent intervention was designed to equip foster parents and relative caregivers with the parenting skills necessary for managing challenging behavior problems. In prior research, the KEEP intervention was found to be effective in reducing child behavior problems. In the current study, the KEEP foster parent intervention was implemented in San Diego County during a three-year trial. The intervention was delivered by paraprofessionals employed by a local community agency (Social Advocates for Youth, San Diego) to 181 foster parent and relative caregivers of boys and girls between the ages of 5 and 12. The control group from an earlier effectiveness study of the KEEP intervention that was also conducted in San Diego County was utilized as a historical comparison group. Regression analyses were used to examine the effects of the intervention on reducing levels of child behavior problems at treatment termination. Consistent with the findings from the previous KEEP effective- ness study, the intervention was found to be effective in reducing child behavior problems when delivered by a community agency. Furthermore, the KEEP intervention was found to be effective in reducing child behav- ior problems among children displaying various levels of initial behavior problems.

© 2012 Elsevier Ltd. All rights reserved.

1. Introduction

Children in foster care display emotion and behavior problems re- quiring mental health assessments and/or intervention at a rate higher than would be expected compared to normative samples or community studies (Landsverk & Garland, 1999; Landsverk, Garland, & Leslie, 2002). Among the various problems evidenced by children in foster care, externalizing behavior problems (e.g., aggressive, disruptive, de- structive, and oppositional behaviors) are highly prevalent and salient. Data from the National Survey of Child and Adolescent Wellbeing (NSCAW) study revealed that a high proportion (43% based on teacher report, 50% based on parent report) of children in foster care evidence some form of externalizing behavior problems (National Survey of Child & Adolescent Well-being Research Group, 2003). Findings from other studies reveal that the levels of antisocial behavior for children re- ceiving child welfare services are statistically indistinguishable from children in intensive mental health treatment programs (Trupin, Tarico, Low, Jemelka & McClellan, 1993). Similarly, in their examination of the mental health of Canadian children in foster care in comparison to community and clinical samples, Stein, Evans, Mazumdar, and Rae-Grant (1996) found that both the foster and clinical samples exhibited significantly more externalizing problems than the children in the community sample, with no differences between the foster and

clinical groups. Adding to the degree of impact of these findings is a body of research indicating that the risk for lifetime problems with an- tisocial behavior is especially high for children with an early onset of be- havior problems (e.g., Patterson, DeBaryshe, & Ramsey, 1989). Not surprisingly, many youth in foster care develop serious conduct prob- lems, including being arrested for violent crimes (Maxfield & Widom, 1996; Smith & Thornberry, 1995).

Within this population, externalizing behavior problems have been found to be linked to foster care placement instability. Not only have ex- ternalizing behavior problems been found to be predictive of placement disruptions and exits (Aarons et al., 2010; Chamberlain et al., 2006; Newton, Litrownik, & Landsverk, 2000), but placement disruptions have also been found to be predictive of increases in rates of child be- havior problems (Aarons et al., 2010; Newton et al., 2000). Thus, chil- dren who enter foster care displaying high levels of behavior problems have an increased likelihood of experiencing a change in placement, which, in turn, further increases the risk for continued and even escalating behavior problems.

In response to the need for addressing the behavior problems of chil- dren in foster care and to reduce the number of changes of placement, a foster parent training intervention entitled KEEP (Keeping Foster and Kinship Parents Trained and Supported) was developed and tested. The primary goal of the current investigation was to determine whether the effectiveness of the KEEP intervention in reducing child behavior problems could be maintained when delivered by a community agency during an implementation trial conducted in San Diego County.

Children and Youth Services Review 34 (2012) 2487–2494

⁎ Corresponding author at: Department of Psychology, San Diego State University, 5500 Campanile Drive, San Diego, CA 92182, United States.

0190-7409/$ – see front matter © 2012 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.childyouth.2012.09.010

Contents lists available at SciVerse ScienceDirect

Children and Youth Services Review

journal homepage: www.elsevier.com/locate/childyouth

1.1. Development of the KEEP intervention model

Based on the basic tenants of Parent Management Training (Kazdin & Wassell, 2000; Patterson, 2005), the KEEP intervention represents a modified version of Multidimensional Treatment Foster Care (MTFC) which was developed by Chamberlain and colleagues (Chamberlain, Leve, & DeGarmo, 2007; Chamberlain & Reid, 1991; Chamberlain & Reid, 1994; Eddy & Chamberlain, 2000; Leve & Chamberlain, 2004). An earlier version of the KEEP intervention was tested in Lane County, Oregon (Chamberlain, Moreland, & Reid, 1992). In this study, foster par- ents with a new child placement were randomly assigned to one of three conditions: parenting training using Parent Management Training components, payment and assessments only, and assessments only. Compared to the payment only group and the control group, parents in the parent-training group evidenced significantly greater decreases in child behavior problems, had fewer failed placements due to child be- havior or emotional problems, and were significantly less likely to quit foster parenting.

The next step in this line of research involved the development and testing of the KEEP intervention in an effectiveness study in San Diego County. The intervention, which began in the fall of 2000 and was com- pleted in early 2006, provided parent training and support to relative and nonrelative caregivers of children between the ages of 5 and 12 in regular foster care. Foster and kin caregivers in the intervention group completed a 16-week course of parenting training within the context of small facilitator-run groups. Caregivers in the control group received “services as usual,” which included yearly parenting classes and support groups for caregivers needing to meet state foster parent licensing re- quirements (i.e., licensed foster and kinship providers). The results of this study revealed that in comparison to the control group, children in the intervention group evidenced a significant decrease in behavior problems over the course of the 4-month intervention (Chamberlain, Price, Reid, et al., 2008). Moreover, the findings indicated that the effects of the intervention were maintained across developer-trained and non-developer-trained intervention staff (Chamberlain, Price, Reid, et al., 2008), suggesting that with appropriate training and supervision, the intervention remained effective as it moves away from the interven- tion developers. The intervention was also effective in increasing paren- tal use of targeted parenting strategies which, in turn, served to mediate the effects of the intervention on reductions in child behavior problems, especially for children displaying more than six behavior problems per day at baseline (Chamberlain, Price, Leve, et al., 2008). Finally, the inter- vention resulted in increasing the number of positive exits from the home (e.g., unification with biological parents, adoptions), and mitigat- ed the negative risk-enhancing effect of a history of multiple place- ments on negative exits (Price et al., 2008).

1.2. Implementation of KEEP in San Diego County

Following the completion of the original KEEP effectiveness study and publication of the initial findings, interest was generated among San Diego County Child Welfare leadership about how the KEEP in- tervention might be implemented into the regular in-service train- ing offered to foster and relative caregivers. As a result, a series of meetings took place between the Child and Adolescent Services Research Center (CASRC), the Oregon Social Learning Center (OSLC) re- search partners, and the San Diego County Child Welfare administrators on how to proceed toward the implementation of the KEEP intervention in San Diego County. The focus of these discussions centered on three key issues: (a) identifying funding for the intervention; (b) staffing fa- cilitators for the parenting groups; and (c) maintaining intervention fi- delity. Child Welfare administrators were able to obtain supplemental funding from the State of California to conduct a pilot of the implemen- tation of the KEEP intervention in San Diego County. Due to the work- loads of caseworkers, it was determined that KEEP intervention would need to be delivered by a community-based mental health service

contractor, one with a working relationship with the San Diego Child Welfare agency and currently delivering services in San Diego County. Social Advocates for Youth (SAY) San Diego, which served two of the six regions within the county, was contacted and expressed an interest in delivering the KEEP intervention. In order to maintain the fidelity of the intervention, OSLC-trained staff from the KEEP effectiveness study who were experienced in facilitating KEEP intervention groups (25+) and in supervising other KEEP group facilitators, provided weekly su- pervision of SAY San Diego group facilitators.

1.3. Cascading Dissemination Model

Moving the delivery of the KEEP intervention from research- based organizations (OSLC and CASRC) to a community-based provider represents the next phase in the Cascading Dissemination Model (Chamberlain, Price, Reid et al., 2008). In this model, the delivery, man- agement, and supervision of the intervention is moved away from the intervention developers at each iteration (in this case, OSLC) and to- ward the implementation of the intervention by individuals that were independent of its original developers. Phase 1 of the cascade is repre- sented by the initial development and testing of the intervention in an efficacy study that took place in three Oregon counties (Chamberlain et al., 1992). The parent training groups were conducted by an experi- enced foster parent who had extensive training in the OSLC PMT model and was supervised by the treatment developer. Phase 2 of the cascade was implemented in San Diego County as the first part of the KEEP effectiveness study. In this phase, the intervention was delivered by paraprofessional staff hired by the Child and Adolescent Services Re- search Center (CASRC) research partners. These facilitators were super- vised by an OSLC-trained on-site supervisor and an experienced OSLC clinical consultant. During Phase 3, which was the second part of the KEEP effectiveness trial, the CASRC intervention staff trained and super- vised a second cohort of paraprofessional interventionists. The OSLC clinical consultant had no direct interaction with the group facilitators in this phase, but did consult with the CASRC interventionists in their supervision of this new group of facilitators. The current study repre- sents the next phase of the Cascading Dissemination Model moving clos- er to large scale implementation, with delivery of the intervention through a community agency with no particular ties to OSLC and with funding provided by a Child Welfare agency rather than a research entity.

The primary goal of this investigation was to examine the effec- tiveness of the KEEP intervention in reducing child behavior prob- lems as it was being delivered by a community-based mental health provider in San Diego County. Since the delivery of the KEEP intervention by SAY San Diego did not include any type of control group, data from the original KEEP effectiveness study was utilized to create a quasi-experimental design. In this design, the baseline and post-treatment data from the randomized effectiveness study of the KEEP intervention conducted in San Diego County (1999 to 2006) was integrated and analyzed with the baseline and post-treatment data collected from the implementation trial of the KEEP intervention within San Diego Child Welfare Services (2005 to 2008). Data from both the intervention and control group from the KEEP effectiveness study were utilized to in order to integrate findings from the effective- ness study with the findings from the implementation trial. Within this design, the original control condition from the effectiveness trial served as the control group when comparisons were made with the original KEEP effectiveness study intervention group. This same control group also served as a historical comparison group (nonequivalent control) when comparisons were made with the KEEP implementation trial group. The use of a historical comparison group from earlier clinical tri- als has been used as a component of research designs to examine the ef- fectiveness of treatments delivered in a community setting (e.g., Costin & Chambers, 2007). This type of research design illustrates how re- searchers might utilize data from prior clinical trials of an intervention

2488 J.M. Price et al. / Children and Youth Services Review 34 (2012) 2487–2494

to examine the potential effectiveness of the intervention when it is implemented in a service setting without any type of control group. It is hypothesized that relative to the children in the control group from the KEEP effectiveness study, children in the community agency admin- istered KEEP SAY implementation trial group would also demonstrate greater reductions in child behavior problems over the course of the 4-month period of the intervention.

An additional goal of this investigation was to determine whether the KEEP intervention was effective in reducing child behavior prob- lems at termination among children with various levels of behavior problems at baseline. That is, is the KEEP intervention effective in reduc- ing child behavior problems regardless of the initial level of child behav- ior problems observed at baseline? It was hypothesized that the intervention would be effective in reducing child behavior problems at termination for children with behavior problems at or above the mean in baseline behavior problems. This hypothesis was examined using both the KEEP effectiveness study intervention group and the KEEP SAY implementation trial group.

2. Method

2.1. Participants

2.1.1. KEEP effectiveness study participants These participants were randomly assigned to either the interven-

tion (KEEP parent training) or to the control group in an earlier effec- tiveness trial of the KEEP foster parent intervention (see Chamberlain, Price, Reid et al., 2008). In this earlier study, eligible study partici- pants included all foster and relative caregivers with a child between the ages of 5 and 12 who was received from the San Diego County Health and Human Services Agency, Child Welfare services sometime between 1999 and 2004. Eligibility requirements were (a) the child was between the ages of 5 and 12, (b) the child had been in the place- ment for at least 30 days (in order to minimize selecting children in temporary shelters or emergency foster placements), and (c) the child was not considered to be “medically fragile” (that is, not severe- ly physically or mentally handicapped — only one child met this criteria). The resulting sample was comprised of 700 foster families (34% kinship placements, 66% non-relative placements). California state law requires foster parents to participate in parent training and support group each year in order to be licensed. The parents in the intervention group (n=359) were allowed to apply participation in the KEEP intervention group toward state licensing requirements. Parents in the control group (n=341) participated in routine parent training and support provided by San Diego County services. Table 1 shows the baseline demographic characteristics of the participants from KEEP effectiveness study.

2.1.2. KEEP SAY implementation trial participants Similar to the original KEEP effectiveness study, eligible participants

included all foster and relative caregivers with a child between the ages of 5 and 12 who was received from San Diego County Child Welfare Ser- vices. In addition, because Social Advocates for Youth (SAY) also ser- viced relative substitute caregivers (e.g., grandparents, aunts, and uncles) who were not dependents under the care of the San Diego HHSA, participants were also recruited from eligible relative caregivers served by SAY San Diego. Similar to the KEEP effectiveness study, eligi- bility requirements included that (a) the child was between the ages of 5 and 12, (b) the child had been in the placement for at least 30 days, and (c) the child was not considered to be “medically fragile.” Table 1 provides the demographic characteristics of the group of participants for whom background information was available (n=181), although the n's varied by demographic category. For those parents who received their children from San Diego County Child Welfare services, participa- tion in the KEEP intervention was allowed to fulfill yearly state licensing requirements. To examine potential differences between the KEEP

effectiveness study participants and the KEEP SAY implementation trial group, ANOVA and Chi-Square analyses were employed. The re- sults of the ANOVA analyses on continuous background variables re- vealed significant differences between the two samples on the following demographic variables: parent age, F(2,812)=6.96, p= .001; age of the focal child, F(2,872)=12.5, p=.001; and household in- come level (1 to 10 rating scale, with 1=household income of less than $14,999 and 10=$95,000 and up), F(2,856)=4.33, p=.01 Chi Square analyses on dichotomous background variables revealed significant group differences on percentage of relative vs. non-relative caregivers, c2(4, N=859)=23.7, p=.001, and primary language of foster parent, c2(2, N=859)=47.3, p=.001. Chi Square analyses also revealed signif- icant group differences on percentage of ethic group composition, c2

(10, N=854)=44.2, p.=.001. In particular, there was a higher per- centage of Latino caregivers in the KEEP SAY implementation group than in either of the KEEP effectiveness study groups.

2.2. Recruitment methods

For both sets of participants, recruitment was facilitated by the use of data systems from the San Diego County HHSA that were reviewed on a quarterly basis to identify eligible children and foster and kinship families. For the KEEP SAY implementation trial group, contact informa- tion on relative caregivers from SAY San Diego was also utilized to iden- tify eligible children and relative caregivers not served by the San Diego County Child Welfare Services. For both groups, similar recruitment procedures were used. First, eligible families were contacted by phone to determine their level of interest in the study. Next, interested families received a home visit, at which time a detailed project description, con- sent information and Institutional Review Board (IRB) approved con- sent form was provided. Those interested in participating verified their willingness to participate by signing the consent form. The inves- tigation was conducted in compliance with appropriate IRB (San Diego State University and the Oregon Social Learning Center for the KEEP ef- fectiveness study and from San Diego State University for the KEEP SAY implementation trial). Participation in both studies was voluntary. In addition, no solicitation or incentives were provided by San Diego County Child Welfare Services or Social Advocates for Youth for families

Table 1 Demographic information on foster and kin parents by group.

Demographic information KEEP effectiveness study KEEP SAY implementation (n=181)

Intervention (n=359)

Control (n=341)

Parent gender Female 94% 93% 98% Male 6% 7% 2%

(n=117) Mean parent age 49.86 (11.7) 47.29 (11.7) 51.28 (10.75) Parent ethnicity (n=154)

African American 27% 24% 21% Asian/Pacific Islander 4% 2% 2% Caucasian 21% 34% 14% Latino 41% 33% 59% Native American 1% 1% 1% Mixed ethnicity 6% 6% 3%

Parent preferred language (n=159) English 67% 79% 49% Spanish 33% 21% 51%

Mean household income 2.3 (1.3) 2.3 (1.3) 2.0 (1.4) Relationship to child (n=159)

Kinship caregiver 32% 36% 53.8% Non-Kinship caregiver 66% 64% 46.3%

Child gender (n=175) Female 50% 54% 46% Male 50% 46% 54%

Mean age of focal child 8.8 (2.2) 8.7 (2.3) 7.9 (2.3) Mean number of children in home

3.5 (1.8) 3.5 (2.0) 3.2 (1.8)

2489J.M. Price et al. / Children and Youth Services Review 34 (2012) 2487–2494

to participate in either of these groups. However, participants in the KEEP effectiveness study were provided incentives by the research pro- ject for completing baseline and termination assessments. However, participants in the KEEP SAY implementation trial were not provided incentives for completing any assessments.

2.3. Intervention model

Similar to parents in the KEEP effectiveness study, parents in the KEEP SAY implementation trial participated in parenting groups of 3 to 10 individuals led by a trained facilitator. Parents received 16 weeks of parent training, supervision, and support in behavior man- agement methods. The primary focus of the KEEP intervention was on increasing use of positive reinforcement, consistent use of non-harsh discipline methods, such as brief time-outs or privilege removal over short time spans (e.g., no playing video games for one hour, no bicycle riding until after dinner), and teaching parents the importance of close monitoring of the youngster's whereabouts and peer associations. In addition, strategies for avoiding power struggles, managing peer re- lationships, and improving success at school were also included. Ses- sions were structured so that the curriculum content was integrated into group discussions and primary concepts were illustrated via role-plays and videotaped recordings. Home practice assignments were given that related to the topics covered during sessions in order to assist parents in implementing the behavioral procedures taught in the group meeting. If foster parents missed a parent-training session, the material was delivered during a home visit. Such home visits have been found to be an effective means of increasing the dosage of the in- tervention for families who miss interventions sessions (Reid & Eddy, 1997).

Parenting groups were formed based on parent schedule, language preference (English or Spanish), and location. Parenting groups were conducted in community recreation centers, churches, or SAY facilities. Several strategies were used to maintain parent involvement, including (a) provision of childcare, using qualified and licensed individuals so that parents could bring younger children and know that they were being given adequate care, (b) credit was given for the yearly licensing requirement for foster care (HHSA foster parents, only), (c) parents were reimbursed $15.00 per session for traveling expenses, and (d) re- freshments were provided. Group session attendance/completion rates (including make-up sessions for absences) were high, with 92% of par- ents completing at least 14 sessions. The language of the materials (En- glish or Spanish) was determined by the language used in parenting groups.

As was the case with the KEEP effectiveness study, during the KEEP SAY implementation trial, the intervention was delivered by paraprofessionals. At least one of the group facilitators was bilingual in English and Spanish. Experience with group settings, interpersonal skills, and experience with diverse populations were given high priority in selecting interventionists. Interventionists were then trained over several weeks through a series of phases involving (a) viewing video re- cords of prior sessions run by experienced facilitators from the original KEEP effectiveness study, (b) role playing in mock group sessions, with the trainee as a group facilitator, (c) and co-facilitating group sessions with an experienced facilitator. The KEEP SAY personnel-led interven- tion groups were supervised on a weekly basis by an experienced KEEP facilitator from the KEEP effectiveness study. Supervisors reviewed video records of group sessions and met with group facilita- tors on a weekly basis to provide feedback. Consultation was also pro- vided by the Oregon Social Learning Center.

2.4. Measures

2.4.1. Child and parent characteristics Foster and kin parent-report of child and family characteristics and

demographics was assessed at study entry (baseline) via phone

interviews. Caregivers had known the target child for at least 30 days prior to the baseline assessment. Interviews were conducted in either English or Spanish, depending on the preference of the parents.

2.4.2. Child behavior problems The Parent Daily Report Checklist (PDR: Chamberlain & Reid, 1987)

was used to assess child behavior problems at baseline and four months later at termination in both groups. The PDR is a 30-item measure of child behavior problems administered via telephone to parents on a se- ries of consecutive or closely spaced days (approximately 1 to 3 days apart). During each call, a trained interviewer asked the foster/kinship parent the following question, “Thinking about (child's name), during the past 24 h, did any of the following behaviors occur?” Parents were then read the list of 30 behaviors and asked to indicate either “yes” or “no” as to whether the behavior had occurred in the last 24 h. Consis- tent with the KEEP effectiveness study, three PDR calls were adminis- tered at baseline (prior to the intervention) and at termination (following completion of the intervention) on different occasions across a two-week period. The PDR is structured so that parents focus on recalling only the past 24 h, thus avoiding aggregate recall or estimates of frequency thought to bias estimates (Stone, Broderick, Kaell, DelesPaul, & Porter, 2000). The PDR has been used in several previous outcome studies, including those with families referred because of child conduct problems (e.g., Kazdin & Wassell, 2000; McClowry, Snow, & Tamis-LeMonda, 2005) and families with children in regular foster care (Chamberlain, Price, Reid et al., 2008; Chamberlain, et al., 1992). The concurrent validity of the PDR has been demonstrated in connection with measures of child and family functioning, including live observations of family interactions in the home (Forgatch & Toobert, 1979; Patterson, 1976) and parents' ratings of child behavior (i.e.., Becker Adjective Checklist; Becker, Madsen, Arnold, & Thomas, 1967). Scores representing levels of child behavior problems were cal- culated for each child at baseline and termination by summing the number of behaviors reported per day on the PDR (out of the possible 30) divided by the number of calls made at each assessment period (typically three calls). Means and standard deviations for baseline and termination child behavior problems by intervention and control groups are provided in Table 2.

3. Results

3.1. Determination of covariates

As reported earlier, analyses to examine potential differences be- tween the KEEP effectiveness study groups and the KEEP SAY imple- mentation trial group on demographic variables revealed significant group differences on several continuous variables, including the age of the focal child, age of the primary caregiver, and income level. To deter- mine potential covariates for regression analyses, these variables were examined in relation to child behavior problems at termination using correlational analyses. The correlations between these variables and child behavior problems at termination were: r=−.093 (p=.013), .005 (ns), and .014 (ns), respectively. Prior analyses of the demographic data also revealed significant group differences on several categorical

Table 2 Means and Standard deviations of Baseline and Termination Behavior Problems by Group.

Group Variable Baseline Child Behaviors

Termination child behaviors

M SD M SD

KEEP effectiveness Study Intervention (n=356) 5.92 4.26 4.37 3.91 Control (n=341) 5.77 3.93 5.44 4.15

KEEP SAY implementation (n=159) 4.83 3.93 2.46 3.15

2490 J.M. Price et al. / Children and Youth Services Review 34 (2012) 2487–2494

and group status for those at the mean level of baseline behavior problems, b=− .24, p=. 001. In addition, there was a statistically significant and negative simple slope between the number of termi- nation behavior problems and group status for those at one standard deviation above the mean, b=−.36, p=.001, those at two standard deviations above the mean, b=− .49, p=.001, and those at three standard deviations above the mean, b=− .61, p=.001. In each case, relative to the control group, there was a significant reduction in termination behavior problems for children in the intervention group. Thus, as the number of baseline behaviors increased there was a greater reduction in the behavior problems in the intervention group in contrast to the control group. Fig. 1 depicts the differences between the control and intervention groups for the number of ter- mination behavior problems at a specific value (e.g., 1 SD above the mean) for baseline behavior problems.

Next, simple regression lines were computed for the relation be- tween number of termination behavior problems and group status 2 (KEEP SAY implementation trial group vs. KEEP effectiveness study control group) at specific values of number of baseline behavior problems: one standard deviation below the mean, at the mean, one standard deviation above the mean, and two standard deviations above the mean, and 3 standard deviations above the mean. The sim- ple slope between number of termination behavior problems and group status at one standard deviation below the mean, b=−.31, was statistically significant, p=.004. A statistically significant and negative simple slope was also found between number of termination behavior problems and group status for those at the mean level of base- line behavior problems, b=−.60, p=. 001. In addition, there was a sta- tistically significant and negative simple slope between the number of termination behavior problems and group status for those at one stan- dard deviation above the mean, b=−.89, p=.001, those at two stan- dard deviations above the mean, b=−1.18, p=.001, and those at three standard deviations above the mean, b=−1.47, p=.001. In each case, relative to the control group, there was a significant reduction in termination behavior problems for children in the intervention group. Thus, as the number of baseline behaviors increased there was a greater reduction in the behavior problems in the intervention group in contrast to the control group. Fig. 2 depicts the differences be- tween the comparison and intervention groups for the number of ter- mination behavior problems at a specific value (e.g., 1 SD above the mean) for baseline behavior problems.

4. Discussion

The primary goal of the current investigation was to examine the effectiveness of the KEEP intervention in reducing child behavior problems as it was being delivered by a community-based mental health provider during an implementation trial in San Diego County. A secondary goal was to determine whether the intervention was effec- tive in reducing child behavior problems at various levels as presented at baseline. The results from regression analyses replicated the findings of the KEEP effectiveness study in demonstrating that the KEEP inter- vention was effective in reducing child behavior problems in the KEEP effectiveness study intervention group (Chamberlain, Price, Leve, et al., 2008; Chamberlain, Price, Reid, et al., 2008). Furthermore, the find- ings from the same analyses revealed that the KEEP intervention was also effective in reducing child behavior problems when it was implemented in San Diego County by a community service provider rather than a research-based entity. This pattern of findings suggests that as the KEEP intervention moves away from the intervention devel- opers and into real world service settings the effectiveness of the inter- vention can be maintained. Moreover, not only was the effectiveness of the intervention maintained as it was adopted within a child welfare setting, but it was also found to be effective with a different composition of substitute caregivers. Whereas in the original KEEP effectiveness study 34% of caregivers were relatives (e.g., grandparents, aunts and un- cles), in the KEEP SAY implementation trial, 53.8% of the caregivers were relatives. Despite these differences, the cumulative findings from the current investigation and the original KEEP effectiveness study (Chamberlain, Price, Reid, et al., 2008) indicate that the KEEP foster par- ent intervention is effective in reducing child behavior problems, re- gardless of the type of caregiver relationship.

An additional goal of this investigation was to determine whether the KEEP intervention was effective in reducing child behavior prob- lems based on various levels of initial baseline behavior problems. That is, is the KEEP intervention effective in reducing child behavior problems, regardless of the initial level of child behavior problems observed at baseline? It was hypothesized that the KEEP interven- tion, as delivered during both the effectiveness study and during the implementation trial, would be effective in reducing child behav- ior problems at termination for children with behavior problems at or above the mean in baseline behavior problems as assessed by the PDR. The results of the regression analyses revealed significant effects for the interactions between group status (intervention vs. control/comparison) and baseline behavior problems. Follow-up

-0.75

-0.25

0.25

0.75

1.25

1.75

2.25

1 2

-1 SD Mean 1 SD 2 SD 3 SD

Control KEEP Intervention

-0.25

0.25

0.75

1.25

1.75

2.25

1 2

-1 SD Mean 1 SD 2 SD 3 SD

Control

Fig. 1. Differences between KEEP effectiveness study control and intervention groups for the number of termination behavior problems at a specific value for baseline behavior problems.

-1

-0.5

0

0.5

1

1.5

2

2.5

1 2

Comparison KEEP SAY Intervention

-1 SD Mean 1 SD 2 SD 3 SD -1 SD Mean 1 SD 2 SD 3 SD

Fig. 2. Differences between comparison (KEEP effectiveness study control) and KEEP SAY implementation groups for the number of termination behavior problems at a specific value for baseline behavior problems.

2492 J.M. Price et al. / Children and Youth Services Review 34 (2012) 2487–2494

simple slope analyses, one set of analyses for the effectiveness study comparisons and one set for the implementation trial comparisons, revealed that the KEEP intervention was effective in reducing chil- dren behavior problems at various levels of baseline behavior prob- lems, including at the mean level of baseline behavior problems (5.92 for the intervention group in the KEEP effectiveness study and 4.85 for the KEEP implementation study) and continuing up through levels of behavior problems as high as three standard devi- ations above the mean. During the implementation trial the inter- vention was also effective in reducing child behavior problems at levels as low as one standard deviation below the mean at baseline. The particular relevance of the findings from the simple slope analy- ses is that they demonstrate that the KEEP intervention is effective in reducing behavior problems at the levels that place children at risk for placement disruptions. Using data from the control group of the KEEP effectiveness study, Chamberlain et al. (2006) found that for each behavior above 6 behavior problems there was a 25% increased risk for a negative placement disruption (e.g., foster parent initiates request for change of placement because of child behavior problems or caseworker determines that the placement is no longer a good fit). Thus, by helping foster parents and relative caregivers to manage the behavior problems of the children in their care, the KEEP foster par- ent intervention has the potential to reduce the risk for negative placement disruptions and reduce the personal and economic bur- dens that result from placement disruptions.

4.1. Study limitations

One of the limitations of the current investigation was the demo- graphic differences between the participants in the KEEP effectiveness study and those who took part in the implementation trial. Even though the participants in both groups were from the same geographic regions within San Diego County, the samples differed in several ways (e.g.., proportion of foster parent vs. relative caregivers, proportion of English vs. Spanish speakers, ethnic composition, and age of the children). As noted earlier, group differences on relative proportion of type of caregiver (foster vs. relative) was attributable to the fact that the community agency delivering the KEEP intervention in this study (SAY San Diego) serviced relative substitute caregivers caring for chil- dren who were not dependents of San Diego County Child Welfare Services. Thus, in addition to recruiting caregivers with children who were dependents of child welfare, eligible relative caregivers served by SAY San Diego with children who were not dependents were also recruited into the intervention during implementation. Consequently, a higher proportion of relative caregivers were recruited into the KEEP intervention during implementation. In contrast, in the original KEEP effectiveness study only families (both nonrelative and relative) who were caring for child welfare dependents were recruited into the study. It is possible that the inclusion of the relative givers served by SAY San Diego to the KEEP SAY implementation trial contributed to the demographic differences between the samples.

Another limitation of the current study was the internal validity threat of history (Cook & Campbell, 1979), in particular local history, in contributing to differences between the historical comparison group (KEEP effectiveness study control group) and the KEEP imple- mentation trial group. It is possible that foster and kinship caregivers who participated in the KEEP intervention following implementa- tion received services unavailable to the foster and kinship care- givers who participated in the earlier KEEP effectiveness study, such as expanded pre-service or in-service training. However, regu- lar communications with our contacts at San Diego Child Welfare Services over the period of KEEP effectiveness study (1999 to 2006) and the implementation trial (2005–2008) did not reveal any sub- stantial changes to basic pre-service or in-service training for foster parents.

4.2. Conclusions

First, within the context of the Cascading Dissemination Model, the results of this investigation suggest that as the KEEP intervention moves away from the intervention developers, paraprofessionals from a community agency that are given adequate training and supervision can deliver the intervention to foster and relative caregivers in a man- ner that is effective in reducing behavior problems of children in foster care,. As mentioned earlier, the group facilitators hired by SAY San Diego received extensive training in the KEEP intervention model, the curriculum, and the management of group processes. This training was conducted by personnel trained by the intervention developers and took place over several weeks prior to start of the first parenting groups. In addition, all group sessions were video recorded and reviewed by the group facilitator and the clinical supervisor. Also, group facilitators contacted the parents in their group each week to as- sess levels of child behavior problems (via the Parent Daily Report — PDR) and to discuss application of session material. Group facilitators and the clinical supervisor met each week to discuss parents' progress in managing child behavior problems, the delivery of session material, and group processes during the prior session. It is within this context that the KEEP intervention was found to be effective in reducing child behavior problems, and at various levels of initial behavior problems. The training and supervision procedures were not burdensome for the group facilitators and likely contributed to the effectiveness of the inter- vention. The results of the implementation trial suggest that it is feasible to have the KEEP intervention delivered by a community mental health provider within a child welfare system of care and that it can be deliv- ered in a manner that leads to reductions in levels of children's external- izing behavior problems. Such reductions are likely to decrease the risk for unwanted placement changes in foster care (Chamberlain et al., 2006; Newton et al., 2000).

Second, this study illustrates a research strategy for examining the effectiveness of an evidence-based practice as it is implemented in a child welfare service setting by utilizing a research design that inte- grates data from a prior effectiveness study of the intervention and the data collected during the implementation of the intervention. With this design, the control group from an earlier efficacy or effective- ness study can provide a ready comparison group when it may not be possible to obtain a control or comparison group during the implemen- tation of an intervention. Researchers and service providers might even consider collaborating in conducting a randomized effectiveness trial in conjunction with a non-randomized implementation of an intervention in a community setting. The data from this research could be analyzed together to provide results on the effectiveness of the intervention under two types of conditions; (a) one with the delivery and moni- toring of the intervention carried out by researchers, and (b) the other with delivery and monitoring carried out by the community providers and/or the service agency personnel who are considering adopting the intervention. The findings generated from this type of design would address the effectiveness of the intervention and pro- vide valuable information on the potential challenges and barriers to large scale implementation of the intervention.

Acknowledgments

Support for the KEEP effectiveness trial was provided by Grant No. MH 60195 from the Child and Adolescent Treatment and Preventive Intervention Research Branch awarded to Dr. Patti Chamberlain. Sup- port for the KEEP SAY implementation trial was provided by a grant from the State of California awarded to San Diego County Child Wel- fare Services. The authors would like to thank San Diego County Child Welfare Services Directors: Yvonne Campbell, Mary Harris, and Debra Zonders-Willis; Deputy Directors: Patty Rahiser, Renee Smiley, and Roseann Myers; Social Advocates for Youth Supervisor: Shannon Throop; Research Project Directors: Courtenay Paulic, Jan Price, and

2493J.M. Price et al. / Children and Youth Services Review 34 (2012) 2487–2494

Norma Talamantes; OSLC consultant JP Davis; lead interventionists Norma Talamantes, Melissa Woods, Moniesha Cole, and Sonia Miramontes; and the foster and relative caregivers who participated in these studies.

References

Aarons, G. A., James, S., Monn, A. R., Raghavan, R., Wells, R. S., & Leslie, L. K. (2010). Behavior problems and placement change in a national child welfare sample: A prospective study. Journal of the American Academy of Child and Adolescent Psychiatry, 49, 70–80.

Becker, W. C., Madsen, C. H., Jr., Arnold, C. R., & Thomas, D. R. (1967). The contingent use of teacher attention and praise in reducing classroom behavior problems. Jour- nal of Special Education, 1, 287–307.

Chamberlain, P., Leve, L., & DeGarmo, D. S. (2007). Multidimensional treatment foster care for girls in the juvenile justice system: 2-year follow-up of a randomized clin- ical trial. Journal of Consulting and Clinical Psychology, 75(1), 187–193.

Chamberlain, P., Moreland, S., & Reid, K. (1992). Enhanced services and stipends for foster parents: Effects on retention rates and outcomes for children. Child Welfare, 71, 387–401.

Chamberlain, P., Price, J. M., Leve, L., Laurent, H., Landsverk, J., & Reid, J. B. (2008). Pre- vention of behavior problems for children in foster care: Outcomes and mediation effects. Prevention Science, 9, 17–27.

Chamberlain, P., Price, J. M., Reid, J. B., & Landsverk, J. (2008). Cascading implementa- tion of a foster and kinship parent intervention. Child Welfare: Journal of Policy, Practice, and Program, 87, 27–48.

Chamberlain, P., Price, J. M., Reid, J. B., Landsverk, J., Fisher, P. A., & Stoolmiller, M. (2006). Who disrupts from placement in foster and kinship care? Child Abuse & Neglect, 30, 409–424.

Chamberlain, P., & Reid, J. B. (1987). Parent observation and report of child symptoms. Behavioral Assessment, 9, 97–109.

Chamberlain, P., & Reid, J. B. (1991). Using a specialized foster care community treat- ment model for children and adolescents leaving the state mental hospital. Journal of Community Psychology, 19, 266–276.

Chamberlain, P., & Reid, J. B. (1994). Differences in risk factors and adjustment for male and female delinquents in treatment foster care. Journal of Child and Family Studies, 3(1), 23–39.

Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston: Houghton Mifflin Co..

Costin, J., & Chambers, S. M. (2007). Parent management training as a treatment for children with oppositional defiant disorder referred to a mental health clinic. Clinical Child Psychology and Psychiatry, 12, 511–524.

Eddy, J. M., & Chamberlain, P. (2000). Family management and deviant peer association as mediators of the impact of treatment condition on youth antisocial behavior. Journal of Consulting and Clinical Psychology, 68, 857–863.

Forgatch, M. S., & Toobert, D. J. (1979). A cost-effective parent training program for use with normal preschool children. Journal of Pediatric Psychology, 4, 129–145.

Kazdin, A. E., & Wassell, G. (2000). Therapeutic changes in children, parents, and families resulting from treatment of children with conduct problems. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 414–420.

Landsverk, J., & Garland, A. (1999). Foster care and pathways to mental health services. In P. Curtis, & G. Dale (Eds.), The foster care crisis: Translating research into practice and policy (pp. 193–210). Lincoln, NE: University of Nebraska Press.

Landsverk, J., Garland, A. F., & Leslie, L. K. (2002). Mental health services for children reported to child protective services. In J. E. B. Myers, L. Berliner, J. Briere, T. C. Hendrix, & C. Jenny (Eds.), The APSAC handbook on child maltreatment (pp. 487–507). (2nd ed.). Thousand Oaks, CA, US: Sage Publications, Inc..

Leve, L. D., & Chamberlain, P. (2004). Female juvenile offenders: Defining an early-onset pathway for delinquency. Journal of Child and Family Studies, 13, 439–452.

Maxfield, M., & Widom, C. (1996). The cycle of violence: Revisited six years later. Archives of Pediatric Adolescent Medicine, 150, 390–395.

McClowry, S. G., Snow, D. L., & Tamis-LeMonda, C. S. (2005). An evaluation of the effects of INSIGHTS on the behavior of inner city primary school children. Journal of Primary Prevention, 26, 567–584.

National Survey of Child and Adolescent Well-Being Group (2003). National survey of child and adolescent well-being (NSCAW One Year in Foster Care Wave 1 Data Analysis Report). Washington, DC: U.S. Department of Health and Human Services, Adminis- tration for Children and Families.

Newton, R. R., Litrownik, A. J., & Landsverk, J. A. (2000). Children and youth in foster care: Disentangling the relationship between problem behaviors and number of placements. Child Abuse & Neglect, 24, 1363–1374.

Patterson, G. R. (1976). The aggressive child: Victim and architect of coercive system. In L. A. Hamerlynck, L. C. Handy, & E. J. Mash (Eds.), Behavior modification and families. I: Theory and research. 11. Applications and developments (pp. 267–316). New York: Brunner/Mazel.

Patterson, G. R. (2005). The next generation of PMTO models. The Behavior Therapist, 28(2), 25–32.

Patterson, G. R., DeBaryshe, B. D., & Ramsey, E. (1989). A developmental perspective on antisocial behavior. American Psychologist. Special issue: Children and their development: Knowledge base, research agenda, and social policy application, vol. 44(2). (pp. 329–335).

Price, J. M., Chamberlain, P., Landsverk, J., Reid, J. B., Leve, L., & Laurent, H. (2008). Ef- fects of a foster parent training intervention on placement changes of children in foster care. Child Maltreatment, 13, 64–75.

Reid, J. B., & Eddy, J. M. (1997). The prevention of antisocial behavior: Some consider- ations in the search for effective interventions. In D. M. Stoff, J. Breiling, & J. D. Maser (Eds.), Handbook of antisocial behavior (pp. 343–356). New York: John Wiley & Sons, Inc..

Smith, C., & Thornberry, T. P. (1995). The relationship between childhood maltreat- ment and adolescent involvement in delinquency. Criminology, 33, 451–481.

Stein, E., Evans, B., Mazumdar, R., & Rae-Grant, N. (1996). Then mental health of chil- dren in foster care: A comparison with community and clinical samples. Canadian Journal of Psychiatry, 41, 385–391.

Stone, A. A., Broderick, J. E., Kaell, A. T., DelesPaul, P. A. E. G., & Porter, L. E. (2000). Does the peak-end phenomenon observed in laboratory pain studies apply to real-world pain in rheumatoid arthritics? The Journal of Pain, 1, 212–217.

Trupin, E. W., Tarico, V. S., Low, B., Jemelka, R., & McClellan, J. (1993). Children on child protective service caseloads: Prevalence and nature of serious emotional distur- bance. Child Abuse & Neglect, 17, 345–355.

2494 J.M. Price et al. / Children and Youth Services Review 34 (2012) 2487–2494