Evidence Based Practice

The Move to Evidence-Based Practice: How Well Does it Fit Child

Welfare Services? Richard P. Barth

ABSTRACT. Child welfare services are engaged in examining and applying concepts from evidence-based practice. This article pro- vides background on evidence-based practice in child welfare and suggests the areas of least and greatest fit between the methods of evidence-based practice and child welfare services. Implications for the emergence of more evidence-based appioaches to child welfare services are forwarded. Suggestions for social work education are also offered.

KEYWORDS. Evidence-based practice, child welfare, parent train- ing, home visiting

Evidence-based practice (EBP) has passed the tipping point of being a lad or rage. Federal initiatives such as the Program Assessment

Richard P. Barth is Professor and Dean al the School of Social Work, University of Maryland, Baltimore, MD.

This article benefited Ironi conversations with John Landsvcrk, Charles Wilson, Melissa Lim Brodkowski. and Lucy Berliner. 1 am gratefnl lo Sarah /lotnik and Mary Hodorowicz for iheir skilled a.ssistance with ihe preparation of (his article. An earlier version was given as a slide presentation ai Ohio State University, Columbus, OH, as the O"Lear>' leclure on October 18, 2007.

Address correspondence to; Richard P. Barth. School of Social Work, Uni- versity of Maryland. 525 W. Redwood Street. Baltimore. MD 21201. (E-mail: rbarth^ssw.umaryland.edu ).

Journal of Public Child Welfare, Vol. 2(2) 2008 Available online al htlp://jpcw.haworthpress.coni

© 2008 by The Hawonh Press. All rights reserved. doi: 10.1080/15548730802312537 '^^

146 JOURNAL OF PUBLIC CHILD WELFARE

Rating Tool (PART) implemented through the United States (US) Office of Management and the Budget, are emphasizing the link between policy, research, and funding. Legislative initiatives around the nation are developing ways to focus service provision toward the delivery of evidence-based interventions. There is now a Coalition for Evidence-Based Policy (Washington, DC; available online at http://coexgov.securesites.net/index.php?keyword=a432fbc34d71c7) and a Center for Evidence Based Policy (Portland, OR; available online at http://www.ohsu.edu/policycenter). In the summer of 2005, the federal government announced a request for proposals to sup- port the replication of evidence-supported child abuse prevention programs, signaling their interest in linking funding and evidence. The 2007 US Department of Health and Human Services (DHHS) Children's Bureau grantees meeting and the biannual National Con- ference on Child Abuse and Neglect were both on EBP. Evidence- based child welfare policies are near and child welfare services (CWS) professionals will need to be prepared to discuss the fea- sibility and reasonableness of policy initiatives, in the very near future.

CWS has not been as infiuenced by the evidence-based movement as education, medicine, or mental health. In all likelihood this is because child welfare is not as well understood—for example, rig- orous definitions of reasons for involvement with CWS still do not exist, although the titne is coming. Many states are now using the EBP lens to examine CWS. Several journals have had special issues on evidence-based CWS (e.g.. Research on Social Work Practice and Child Welfare in 2005 and 2006). Yet, much of the information provided in those journals was not about the fundamental activities of child welfare workers.

This article sets out to consider how CWS are affected by the current move to EBP in areas of human service provision, focusing particularly, on areas of least and greatest iit between EBP and CWS. First, components of CWS will be reviewed. Then, the issue of adding an evidence-based perspective in CWS will be discussed by providing an overview of the EBP movement and then detailing salient infor- mation about EBP in mental health and health care, education, and social work. How CWS could utilize and benefit from EBP will be mentioned in summary. Third, examples of standards of evidence for EBP in CWS will be provided. This will be followed by an analysis of where EBPs may have the best fil with CWS: family engagement and parent training. Finally, the challenges of implementing EBP in CWS will be outlined.

Richard P. Barth ^ H=>. 107

COMPONENTS OF CHILD WELFARE SERVICES

What Does Child Welfare Practice Involve?

Child welfare practitioners coordinate and provide prevention and intervention services to children and their families. Much of what they provide is assessment and intervention planning. In what is now a highly legalistic child welfare system, they make recommen- dations to the juvenile court about whether to substantiate victim- ization, where the child should reside, what services the child and parent(s) should receive, and what the permunent placement should be for the child. Depending on the unit of service provided (e.g., investigation or ongoing in-home services or foster care) they may provide relatively little direct intervention to change parent or child behavior and will, primarily, serve in a case management function. According to the Urban Institute (Washington, DC), the total cost for providing CWS is in excess of $22 billion dollars a year (Bruder et al., 2005).

Who Do Child Welfare Services Serve?

CWS supports children with a wide range of social, emotional, physical, behavioral, and familial needs. Only approximately 25% of CWS are provided to children who were physically battered or sexually assaulted, and this proportion appears to be declining (Jones, Finkelhor, & Kopiec, 2001). The other 75% of CWS are provided to children who are categorized as being victims of neglect (US DHHS, 2007). Neglected children include, but are not limited to, those who were emotionally abused or exposed to domestic violence, in need of higher levels of supervision than their caregiver can provide, in need of basic necessities that their parents are unable to provide, educationally neglected or whose behavior is beyond their paients' control, mentally ill, or experienced prenatal illicit substance exposure. The diversity of problems represented by child welfare-involved families makes the development and delivery of EBPs particularly difficult for most cases because evidence-supported interventions (ESIs) are generally designed for specific diagnostic groups. The complexity of present- ing problems for children who are in the category of having been neglected, and their families, may be quite substantial and varied—a poor fit with ihe origins of EBP in medicine, which was very specific to narrow diagnostic categories.

148 JOURNAL OF PUBLIC CHILD WELFARE

CWS encounters milhons of children and families, each year. CWS agencies received an estimated 5,000,000 calls in 2006. Of these calls, about 3,000,000 were accepted as reports of child maltreatment (US DHHS, 2007). Just less than 1,000,000 of these children are judged to be "victims" of abuse or neglect. Of these children, 250,000 new children enter foster care, and 180,000 children return home each year. 50,000 children are adopted, and 20,000 children "age out" of foster care each year (US DHHS, 2008).

These numbers are large, by any account. The significance of these numbers is demonstrated through the extensive reach that CWS has on the children within the United States. The average African American child has a 10% probability of entering foster care at some point during the first 6 years of his or her life (Berrick et al., 1998). More recent efforts (Magruder & Shaw, in press) establish that, among children born in California in 1996, nearly 20% bad been referred to CWS 7 years later and, of those, 3.5% had entered foster care. Given the number of children directly involved with CWS each year, their families, and those foster and adoptive families that care for them, the amount of people directly affected by CWS is a significant amount of our population. As a result, America's current CWS program is clearly a huge largely unevaluated social intervention—yet it is one deserving the strongest possible evidentiary base.

Adding an Evidence-Based Perspective in Child Welfare Services

Although child welfare research has blossomed, in recent years, very little is known about what kinds of services children and families receive during their encounters with CWS. Most of the research con- tinues to be about foster care; even though foster care involvement is the outcome of approximately 5% of all calls reporting alleged abuse or neglect (US DHHS, 2007). The majority of CWS involved fam- ilies remain intact—having their cases closed after an investigation. The most common "service" received by families is an investiga- tion followed by some form of parent training. Each year as many as 2,000,000 families receive an investigation and ol" those faniilies about 800,000 are referred to receive parent training (Barth et al., 2005). At the time of writing, there are no accurate estimates of how many mothers and fathers actually receive a reasonable dose of parent training, of their experience, or of what the outcome might be. Indeed, little is understood about the form and organization of parent training.

: Richard P. Barth i

Agencies vary widely in the method for which they contract, oversee, and receive feedback from parent-training providers (Hurlburt et al., 2007).

Some information indicates that parent training is delivered by child welfare workers for roughly 30% of cases, by community- based organizations not under CWS contract for 25%, and by mental health or other contract agencies for 40%, with a small percentage delivered under other auspices or contracting arrangements (Hurlburt et al., 2007). This variability in the mechanism of delivering parent training is matched by the variability of methods used. Indeed, most agencies indicated that the parent-training program that they used was not a previously developed manualized program and only 1% indicated using a program that has been evaluated as having a strong evidentiary base. CWS may have a good measure to learn from other fields also struggling to increase the proportion of their interventions that have a recognizable evidence base. Some of those lessons are provided below.

THE EVIDENCE-BASED PRACTICE MOVEMENT

The EBP perspective is increasingly being applied to assess and in- crease the effectiveness of services offered by agencies. The evidence- based perspective is being utilized in the fields of medicine, education, and inental health more frequently now than ever before. EBP exam- ines how practitioners can select the most appropriate and effective interventions when providing services. CWS have not transitioned to utilizing EBP as rapidly as other professions, but should recognize that if used correctly, the evidence-based perspective will provide opportunities to strengthen CWS by providing more successful, cost effective, and efficient interventions.

The work of Gibbs (2003) and Sackett (Sackett et al., 1996; 2000) are the most often cited when discussing the process of EBP. Their work provides a framework for practitioners to use in identifying ESIs and helping clients and social workers to make choices about which to use. The more readily identified element of the EBP framework is evidence-supported interventions (ESIs). These are the discrete interventions that have been developed, manualized, and tested in rigorous clinical trials. Sometimes these interventions are titled and copyrighted programs, such as "Multidimensional Treatment Foster Care" (MTFC), which have been judged lo have scientific findings (Chamberlain, 2003). It is a common tnisconception ihat these ESIs

¡50 JOURNAL OF PUBLIC CHILD WELFARE

are, in the aggregate, EBP. This is not the case. Indeed EBP does not require tbe existence of any ESIs, a fact that must be recognized—and for which we can be thankful—because there are very few ESIs in the cbild welfare field.

The process of EBP movement is intended to provide the best possible information, in a timely way, to guide the decision-making of clinicians and service recipients in selecting and implementing a treatment plan (Gibbs, 2003). According to Sackett et al. (1996), "evidenced-based (practice) is tbe conscientious, explicit and judicious u.se of current best evidence in making decisions about the care of individual patients. The practice of evidence-based (practice) means integrating individual clinical expertise with the best available external clinical evidence from empirical research" (p. 71). This information may result from randomized clinical trials or other research that offers reason to take a preferred course of action.

EBP is not the result of conducting numerous assessments or having a good management information system. Data collection tbat intends to inform the outcome of services may be important to the devel- opment of new evidence and may, ultitnately, help to generate ideas about areas needing the greatest change but asking clients to complete assessments that do not guide decision-making is not EBP. A brief look at how other professions have begun to implement EBP will be instructive. . ,

Evidence-Based Practice in Mental Health Care

The Cochrane Collaboration (http://www.cochrane.org), a British research group, evaluated the current research body of mental health EBP and found that only a minority of studies reviewed showed that innovative interventions were more effective than standard care (Cooper, 2003). The Collaboration completed 18 reviews comparing tbe impacts of innovative care to standard care for specialists working with people with severe mental health needs. Five of tbe 18 reviews failed to meet the necessary conditions to draw a conclusion. In eight reviews, there was no difference in outcomes between trial and comparison groups. In just five reviews (27%), significant advantages were found for the trial innovative practice groups.

Evidence-Based Practice in Education '•

Evidence-based education is gaining substantial ground and educa- tional research now has an institute to oversee its future development.

Richard P. Barlh v^

The Institute of Education Sciences (http://ies.ed.gov/), modeled after the National Institutes of Health, was founded on the dual pillars of empirical evidence and professional wisdom, as the ñeld of education recognizes the importance of evidence while still taking into account the paucity of current information (Whitehurst, 2002). Professional wisdom is the ability to make decisions developed from experience and commonly shared views about effective practice methods. Empir- ical evidence encompasses scientifically based research and empirical information and includes the incorporation of local conditions into the evidence used to guide instruction.

EBP in education relies far more heavily on professional wisdom than on tesearch. Professional wisdom in education requires experts to provide information where research evidence is missing or incomplete and to help transfer research findings to related practice settings. Empirical evidence is needed to help reconcile competing intervention approaches, to help generate a cumulative knowledge base of best practice, and to ensure educators are not perpetuating popular wisdom and individual biases (Whitehurst, 2002). Despite these needs, approx- imately 5% of all teacher activities have an informative evidetice base to them (Whitehurst, 2002).

Evidence-Based Practice in Social Work

EBP in social work seeks the transparent interlinking of evidentiary, ethical, and application concerns in all professional venues (Gambrill, 2003). EBP seeks ways to facilitate these concerns and to integrate them within a developmental and cultural framework attentive to ethical and philosophical issues. Efforts must be concurrently applied to practice, policy, research, and professional education. Examples of philosophical issues that must be encompassed are involving clients as informed participants and honestly communicating our lack of knowledge about important practice questions. No estimates could be located with regard to the proportion of activities that social workers engage in that have a strong evidence base.

HOW CHILD WELFARE SERVICES COULD USE AND BENEFIT FROM

EVIDENCE-BASED PRACTICE

As child welfare researchers, managers, and advocates the EBP movement cannot be ignored. The logic behind the saying "'you can

152 JOURNAL OF PUBLIC CHILD WELFARE

ignore politics but it will not ignore you." could also be applied to EBP. There is also much value in the work being done—this value depends, in part, on the fit between CWS and EBP. Understanding tbe fit between EBP is a necessity, for if efforts are not focused on better ways to achieve outcomes, policy makers and other individuals outside of the child welfare profession who do not have an accurate understanding ofthe issues will take on this task. It is imaginable tbat the manner in which they do so will not match child welfare's work requirements and generate advances in the field of child welfare.

Child welfare workers have the choice to clarify which aspects of tbeir work are being and are not being done well and which interventions are and are not supported by evidence. Gambrill (2003) bas emphasized the idea that the function of EBP is enhancing client's choice of methods designed to help them achieve tbeir goals. Ac- cording to Gambrill (2003), '"given the many burdens that social work clients confront, we could argue that accurate brokering of knowledge and ignorance is especially important in our field" (p. 19). The process of EBP includes a clear responsibility to inform clients about the limitations of researcher's knowledge of interventions and give an increasing number of clients tbe opportunity to participate in a more active role when selecting the form of services to be received.

Evidence-based reforms can be used to strengthen CWS if their implementation has been carefully and gradually structured and ad- dresses both reforming CWS programs and processes. Reforms of CWS must occur gradually. Agencies have to ensure that adequate resources are allocated to adapt interventions to each child welfare agency's specific population and practice parameters. Additionally, extensive supervision is necessary to ensure the interventions are effectively implemented through thorough staff training and ongoing support. The evaluation of process or system reforms must meet different criteria than reforms targeting specific ESIs, which describe manualized intervention elements.

Standards of Evidence for Evidence-Based Practice in Child Welfare Services

There are many discussions of what kinds of evidence should be considered—and how it should be applied—in determining the scientific soundness of an intervention. The American Public Human Services Association, National Association of Public Child Welfare

Richard P. Barth * r

Administrators (NAPCW, Washington, DC) (2005) acknowledges that there are many different methods in which organizations determine the scientific soundness of an intervention. However, the NAPCW also notes that there is an advantage to "using consistent language and terminology" (2005, p. 6) and as a result proposes guidelines and definitions for child welfare administrators. Another such discus- sion may not seem warranted, although the child welfare field has made some important advances of which to be aware. The Office of Child Abuse Prevention in Sacramento, CA, has contracted with San Diego Children's Hospital to implement the California Evidence- Based Clearinghouse (CEBC) for Child Welfare (San Diego; available online at www.cachildwelfareclearinghouse.org), which has developed standards for evaluating evidence and for dealing with the rigor of the intervention. The Clearinghouse also examines a programs applica- bility to CWS, and has identified select practices of greatest interest to child welfare directors and managers in child welfare. These are shown in Table I.

The CEBC has also developed an effectiveness scale (Figure 1 ). This scale is different from other scales in child welfare, most notably the scale of the Child Welfare League of America (CWLA, Arlington, VA), which was developed in 2001 as the Research to Practice Initia- tive and can be viewed online at www.cwla.org/programs/r2p/levels. htm (n.d.). The CWLA Research to Practice initiative evaluates each

TABLE 1. Practices of Greatest Interest to Child Welfare Di- rectors and Managers

Area of Practice

Domestic/Intimate partner violence: Batter intervention programs

Domestic/Intimate partner violerKe: Services tor women and their children

Motivation and engagement

Parent training

Placement stabilization

Reunification

Substance alausa (parental)

Trauma treatment for children

Youth transitioning into adulthood

Source: California Clearinghouse on Evidence Based Child Welfare Seivices (San Diego, CA). Available online at: http://www.cachildwellareclearinghouse.Ofg/seafch/ topical-area, retrieved October 10, 2007.

154 JOURNAL OF PUBLIC CHILD WELFARE

.•• •• --••• FIGURE 1. CEBC Scientific Rating Scale

The ctassificatjon system uses criteria regarding a practice's dtnicât and/or empirical support, documentation, acceptance within the field, and potential harm to assign a summary dassificabon score. A lower score indicates a greater level of support for Ihe practice protocol. The summary categories are:

1. Welt-Supported by Research Evidence 2. Supported by Research Evidence 3. ProfTiiytng Research Evidence 4 . Lxki Adqqgate Researdi Evidepcq 5. R^sejfçh gvidenoe Fais to Demonstrate Effisct 6; Çqnçefninfl Practice

Well Supported -«^^HHMMNMMMHMHHMHMHÍI^»- Concerning

Mote: http://www.cachild_welfare_clearinghouse.org/scÍen(iñc-raling/Ncale Retrieved July 7, 2008

practice only as exemplary, commendable, emerging, or innovative, which is an inadequate metric because it fails to cover the range of possibilities. The CEBC scale, unlike the CWLA scale, acknowledges when some practices have been tested and failed to show an impact or carry evidence that they may be harmful. American Public Ser- vices Association NAPCWA (2005) proposes a similar child welfare practice classification system, which is based on a version provided by the Department of Justice OfHce of Victims of Crime (Saunders, Berliner, & Hanson, 2003) and mentions the CEBC as a leader in conducting the search and analysis for EBP programs (p. 6) (available online at httpV/www.aphsa.org/home/doc/Guide-for-Evidence-Based- Practice.pdf; retrieved April 23, 2008). The developers of the Depart- ment of justice Office of Victims of Crime metric were involved in the creation of the CEBC scales, which can now be considered the industry standard.

Because of the limited amount of research that has been done witb children and families receiving CWS, the CEBC has also created a scale of child welfare relevance (not shown in this article), because

Richard P. Barth Hf:

most research on EBPs was not done with a child welfare population. Only one intervention has the highest rating for both scientific merit and relevance (abuse-focused cognitive-behavior therapy: Koiko & Swenson, 2002).

WHERE DO EVIDENCE-BASED PRACTICES FIT INTO CHILD WELFARE SERVICES DESIGN?

EBP is unlikely to be a direct route to significant CWS reform, and this will be important to keep in mind because, to enhance CWS outcomes, four target areas must be reformed: child welfare finances, systems, personnel, and interventions (Bickman, Reimer, Breda & Kelley, 2006). First, fiscal reform examines the amount and flexibility of the funding available for CWS, a substantial issue addressed by the Pew Charitable Trusts (Philadelphia, PA) because of the significant influence that the current funding constraints have on the development and delivery of quality services (Pew Commission, 2004). Second, reforms of the service systems incorporate how agencies are designed and how they coordinate and deliver services. Recent examples of sys- tems reforms include: the program improvement plans of the federal child and family service reviews, the multi- or alternative-response approaches, and concurrent planning and inleragency coordifiation. These initiatives do not specify specific practices but do change the rules of service eligibility, duration, and allowability. Third, child welfare outcomes can be strengthened by increasing the protessional- ism and skills of child welfare personnel. Through stronger training programs and more stringent hiring criteria, professionals' prepara- tion and skill level can be enhanced. University-agency partfierships and the child welfare collaboratives are examples of initiatives to strengthen the skill base of new child welfare personnel (Dickinson & Gilde Gibaja, 2007). If child welfare professionals build upon the work of medical and educational professionals, it could be observed that much of what is accomplished is based on clinical knowledge or educational wisdom, respectively. Clinical knowledge afid educa- tional wisdom can provide the basis for competency development and assessment to determine that this information has, indeed, been pas.sed on.

Last, the specific interventions implemented by CWS may need to be reforfiied—this would be the place where ESIs would most comfortably fit (although there could, in theory, be ESIs for develop- ing systems reform or funding). An example would be the work of

¡56 JOURNAL OF PUBLIC CHILD WELFARE

Glisson, Dukes, and Green (2006) to change organizational climate— a type of systems reform. If only one target area is changed then significant reform is unlikely—at best this would result in incremental reform. Thus, EBP is a relatively small but critical part of reforming CWS. It is also a component over which a considerable amount of control can be exercised.

Another way to think of the dominion of EBP is with regard to the central dimensions of what child welfare workers do. Examination ofthe competencies typically identified in child welfare training pro- grams indicates that much of what child welfare workers do involves assessment of risks, engagement of families, identification of safe and appropriate placements and services, referral to contract services; facilitating of visiting between children and their parents and siblings; liaison between the child, family, CASA, service provider and courts; decision-making regarding case plans and permanency plans. The decision-making aspects of child welfare have long been discussed as the central function of child welfare practice and continue to be a major performance challenge (Gambrill, 2005).

In simple terms, most of the activities performed by child welfare workers are not discrete interventions or prone to manualization or rigorous testing. The contributions ofthe EBP movement to improving these processes are unclear, although there are examples of ESIs (e.g., multi-sysiemic therapy IMSTl) that do include a wide-ranging analysis of field forces and do allow for multiple and coordinated responses from many providers. This is not to say that there are not elements of these service coordination packages that cannot be enhanced. For example, one area that has shown it can be improved and will be discussed later is family engagement.

In early intervention services much work has been done to concep- tualize the role of service coordination and engagement and to assess the related outcomes. This has been conceptualized by Bruder and colleagues (2005) most recently, for early intervention services. These activities are very close to what child welfare workers do—although child welfare workers have another dimension of involvement—the courts. If CWS were to become more professionalized and evidence based, CWS would have research and resultant EBP processes and ESIs for each of these components of service coordination. These would certainly include those activities that focus on improving sys- tems of care as the object of intervention, not just those with children and families as objects ofthe intervention.

In summary, much of what CWS is involved with remains squarely in the hands of practice wisdom, because there is so little research

Richard P. Barth 157

about CWS involvement interventions available. As a result, the re- mainder of this article will focus on two areas that have been re- searched and warrant more attention, exploration and elaboration: family engagement and parent education (also called parent training).

Improving Family Engagement * ^ J .

A key component of evidence-based children's mental health ser- vices is "family engagement" (McKay et al.. 2(X)4). This aspect of EBP has been largely overlooked, to date, in CWS. Although little discussed, the underlying assumption is, arguably, that court orders provide the impetus to encourage family engagement. Yet, there is evi- dence tbat suggests only approximately 50% of parents complete their parent training. In response to an inquiry via the International Society for Prevention of Child Abuse and Neglect (ISPCAN) child abuse and neglect listserv, a large international listserv. only two organizations reported their completion rates for parent training (http://ispcan,org/). One organization indicated that 80% of families completed parent- training programs, if they stayed for at least two sessions (George Bryan, Executive Director Exchange SCAN, Winston-Salem, NC, per- sonal communication, July 28, 2005). The other was quite surprised to find that only 19% of those who started the parent-training program completed it (Jenna Hargrove, personal communication. July 2, 2005). No published literature could be found indicating the completion rates for parent training delivered to parents involved with CWS. although the National Center for Injury Prevention and Control (2004) indicates that between 30% to 80% of those referred to parent training complete the program. Yet this topic remains little discussed—in a review of 52 Child and Family Services Review (CFSR) Hnal reports we found minimal discussion specilic to the engagement of parents and families in mental health service delivery—only 3 state reports mentioned it (McCarthy et al., 2007).

Even if we imagine that the court requirement of parent training— which is issued in more than 50% of all CWS cases in the US (Hurlburt & Barth 2007)—is such that only the non-court ordered cases are of concern with regard to family engagement, that is still a large group of cases. In National Survey of Child and Adolescent Well-Being (NSCAW) about twice as many families had cases closed by the end of the investigation as those that had cases opened after the investigation (US DHHS, 2005). We also know that the reabuse rate among families with children at home exceeded 33% (Kohl &

¡58 JOURNAL OF PUBLIC CHILD WELFARE

Barth, 2007). This demonstrates that there is a large group of families needing to be engaged in evidence-based services.

In social work it has often been argued that there are policy and funding reasons that represent barriers to adequate services. Yet, stu- dents of family engagement also indicate reasons such as therapeu- tic relationship problems, parental attitudes about services, family stress, and discouragement from social support networks to seek help (Harrison, McKay & Bannon, 2Ü04; McKay et al., 2004). Although most of this literature is about mental health services, rather than child welfare focused parent training, there also appear to be dif- ferences between Afiican American and white parents in attitudes about treatment. This could also be considered to be relevant to the receipt of newly developed trauma-focused interventions (e.g., Kolko & Swenson, 2002), which are far more focused on mental health issues than on parenting.

Family Engagement as an Evidence-Supported Intervention

Although family engagement strategies are less manualized and somewhat less well-tested than some of the best tested ESIs, there is evidence from McKay and colleagues (1996) that the number of sessions that families use is increased and drop out is reduced with the inclusion of the following tasks during the first face-to-face contact with the youth and his or her family:

1 ) clarification of the roles of the worker, agency, intake process, and possible service options;

2) setting the foundation for a collaborative working relationship; 3) identification of concrete, practical issues that can be immedi-

ately addressed, and 4) development of a plan to overcome barriers to service receipt.

Nock and Kazdin (2005) have shown that they can markedly increase attendance at parent management sessions if they take 5 to 15 minutes at every other session for motivational work. With a few exceptions, however, the evidence base for child welfare involved families is still weak (rating only a 3 on the CEBC Child Welfare Relevance Scale).

In McKay's (2005) study of urban youth's use of mental health services, 28% of the youth were in foster care or in an adoptive home. Service use decayed as a straight line function between 1 and 12 weeks from 72% of youth remaining in services to 9%. There

Richard P. Barth : .

was no relationship between trauma exposure and whether or not youth continued in services—suggesting that tbe effects of trauma on mental health are not so distinct that they represent a separate pattern of service use. This level of decay in service use may be common across a range of problem areas given the weak engagement strategies now used.

Improving Parent Training • ''j '

In examining the effectiveness of programs, it is essential to assess both the research findings and tbe quality of tbe research design. To accomplish this, an overview is provided of Multi-systemic Therapy (MST), The Incredible Years (TIY, Seattle, WA). and Parent-Child Interaction Therapy (PCIT), which include Parent Management Train- ing, Multidimensional Treatment Foster Care (MTFC). and (Project KEEP). These programs were selected due to their frequent nomi- nation in the mental health and juvenile services lists of evidence based practices, generally, and—in the case of KEEP—because they are emerging best practices.

Multi-Systemic Therapy (MST)

MST is a strongly researched intervention that provides short- term bome-based services that address the multi-determined causes of a youth's antisocial behavior. MST is broader than most par- ent training programs but represents what parent education programs might look like—long enough for change, stipervised practice in the home, addresses significant barriers to good parenting. MST aims to reduce youth's criminal and antisocial activity as well as reduce the rates of incarceration, out-of-home placement, and psychiatric hospitalization. At this point there are no published studies of MST with child welfare populations, altbougb they are under developmenL Indeed, MST does not authorize its use for child welfare popula- tions, per se. MST started in tbe 1970s and grew out of the "Home- builders" family preservation program. Since its inception, MST has moved away from its initial child welfare population to juvenile jus- tice, refining its practice model over time. Now distinct from tbe Homebuiiders model, tbe MST model lasts longer, is more focused around community partnersbips, and has stronger quality control. Ex- tensive evidence supports the effectiveness of MST, and the program has been identified as a Blueprints model by the Center for tbe

iñO JOURNAL OF PUBLIC CHILD WELFARE

Study and Prevention of Violence (Boulder, CO; available online at http://www.colorado.edu/cspv/blueprints/model/programs/MSThtml). Now MST trials are being conducted with child abuse and neglect populations. Recently, evaluation data from Youth Villages (Memphis, TN, a longtime MST provider for approved populations) suggests the efficacy of this approach with child welfare cases in showing that youth who are served with intensive in-home therapy (very much like MST) tend (p < .10) to have I-year outcomes that are better than a matched group of youth who receive group care.

MSTs claims of success are not uncontroversial. Littell (2005) critiqued MST, arguing that evaluations of MST have numerous design limitations. Littell (2007) concludes that there is no credible evidence that MST reduces out-of-home placements, recidivism, or improves family functioning in comparison with other services. She asserts that MST is not consistently better nor worse than other services and fully recognizes that it's effectiveness cannot be concluded. Indeed, the p- values from the analysis in Littell's review are almost all smaller than .20 and generally smaller than .10, favoring MST over conventional practice, and indicating that MST is fiiore likely to be providing benefit than to be having no effect.

The Incredible Years ' I"

The Incredible Years (TIY (available online at http://www.incredi bleyears.com/; retrieved August 5,2008) is a program of great promise for CWS. TIY was founded by Carolyn Webster Stratton at the Univer- sity of Washington (Seattle, WA). Webster Stratton is a nurse and psy- chologist who trained at the Oregon Social Learning Center (Eugene, OR). TIY was developed for children age 2 to 12 years with conduct problems. The Office of Juvenile Justice and Delinquency Prevention has identified TIY as an exemplary best-practice program. The goal of TIY is to reduce conduct problems and promote social competence in children through concurrent parent and child training. TIY works to reduce negative behaviors, classroom disruption, peer aggression, and noncompliance and promotes social, academic, and emotional cofiipetence. TIY also aims to foster strengths-based parenting. Both the child and parent receive training on the same topics in separate parent and child groups. The training is taught in a developmentally appropriate manner. Examples of topic areas are self control, anger management, and teaching about giving "time out" to parents and accepting "time out" for children.

Richard P. Barth ¡61

TIY parent training provides developmentally skill-based lessons to help parents' reduce their child's conduct behavior. TIY topics include; discipline and limit-setting, problem-solving, encouraging positive behaviors, increasing school engagement and achievement, and communication. Multiple teaching methods are used to engage parents, as the training incorporates videos, books, homework, and reminder notes for the refrigerator. TIY child training focuses on reducing aggression through school or counselor led groups. In therapist-directed programs, the group typically has six children and the curriculum lasts approximately 20 to 22 weeks. TIY also has school-based child training programs. Focus areas include fol- lowing the rules, problem-solving, and understanding feelings. The program is 60 lessons, administered two to three times per week. Each session incorporates having a circle time and small group activities.

Evaluations of TIY strongly support the benefits of this interven- tion. Multiple randomized trials of TIY all had large effect sizes. As a result of TIY, parents' aggression and use of corporal punishment decreased, while their effective discipline increased. For children, oppositional behavior decreased with and without diagnosed conduct disorders. All of these gains lasted up to 3 years. Outcomes were not associated with parent gender, ethnicity, class, or education. However, in examining the strength of these studies, it is important to note that the research on TIY has not yet been widely replicated by other investigators or with child welfare populations.

TIY's parent and child training model is a potentially strong fit with CWS. CWS has almost no parent training resources targeting both parents and children that do not require them to be in the same location. TIY has a separate yet simultaneous curriculum for the parent and the child, teaching children and parents the same skills in the groups. TIY has recently been developed with a downward extension to babies and toddlers—a substantial need in the field— although the evaluation data reported previously cannot be assumed to apply.

Parent-Child Interaction Therapy (PCIT)

PCIT was tested in Okalahoma in a randomized trial (Chaffin et al., 2004). Participating parents had a history of engaging in severe physically abusive behavior. Physical abuse re-report rates at a median of 850 days of follow-up were 19% for the PCIT group compared

!62 JOURNAL OF PUBLIC CHILI) WELFARE

with 49% for a standard, community, parenting group. The addition of individualized wrap-around services did not improve physical abuse re-report outcomes and may have been counterproductive. No differ- ences in outcomes were found by age, gender, or race/ethnicity. In addition, different therapists achieved comparable results. PCIT cost more than standard approach. The additional cost of averting a single re-report ranged from $371 to $ 1,326.

PCIT is increasingly being employed in CWS. PCIT is now in place as a resource to biological parents involved in CWS in several California sites as well as Oklahoma. West Virginia, and Washing- ton and achieving promising results with foster parents in California (Timmer et al., 2006) and New York (Linares et al., 2006). The PCIT program is designed for children ages 4 to 12 years. In PCIT, parents are taught specific skills, given the opportunity to practice these skills during therapy, and then continue practicing skills until mastery is acquired and the child's behavior has improved (Patterson, 1975; Reid, Patterson, & Synder, 2002). The two primary focus areas are relationship enhancement and discipline. With relationship enhance- ment, parents are taught and 'coached' how to decrease negativity and increase consistently positive communication with their child. With discipline, parents are taught and 'coached' the elements of effective discipline and child management skills. Distinct from most parent training programs which simply evaluate a parent's success based on attendance, therapists provide reports of parental competency at the end of PCIT. A recent meta-analysis of PCÍT also confirms that the effects of the full program are robust across child problems and parenting problems, according to parental self-report (Thomas, & Zimmer-Gem beck, 2007).

Parent Management Training

Parent Management Training is a technique with more than 30 years experience teaching parents how to manage their children's behavior. PMT teaches parents to provide rewards for positive behaviors and consequence for negative behaviors. The four key components of PMT are close supervision and monitoring of the child, consistent consequences by caregiver, promoting positive relationship with an adult, and maintaining low levels of interaction with deviant peers. The Oregon Social Learning Center developed the PMT model and continues to research and implement this practice. Randomized trials of PMT from the 1970s to 1990s found that PMT improved parenting

Richard P. Barth . ^ . M

and decreased adolescent delinquency and that these changes persisted over time (Chamberlain. 2003).

Project Safe Care , I-

A study of Project Safe Care demonstrated that this project reduced rates of reabuse when the program was implemented with ongoing staff supervision (Lutzker & Bigelow, 2002). Without ongoing super- vision to support the therapist and monitor the therapist's fidelity to the model. Project Safe Care's impact was quite similar to the child simply receiving typical CWS. Project Safe Care is now undeigoing a major implementation study in Oklahoma and is showing very promising results—only if intensive supervision is provided to help the practitioner properly deliver the intervention. If the supervision is not intensive, then the Project Safe Care intervention is not signiticanlly better than conventional methods of practice. (M. Chaffin. Professor of Pediatrics, University of Oklahoma Health Sciences Center, personal communication, October 4, 2006). - -,

Multidimensional Treatment Foster Care

Randomized trials of Multi-Treatment Foster Care (MTFC) reveal MTFC reduces youth's delinquent behavior and disruptions in place- ment. In a State Hospital Study (Chamberlain & Reid, 1991), youth randomly assigned to MFTC were placed more qtiickly and spent more days in cominunity placements than youth in the "treatment as usual" control group. Second, in a 1992 application to "regular" foster care (Chamberlain, Moreland, & Reid, 1992) children randomly assigned to the MTFC group had fewer placement disruptions, foster parents dropping out of providing care, and problem behaviors in follow-up. Last, Chamberlain (1998) assessed female delinquency processes and outcomes for 80 girls randotnly assigned to MTFC, rather than Group Caie, and found that the MTFC group had fewer aiTests.

Project KEEP . i<

An evaluation of Project KEEP studied MTFC's transferability to CWS. The study randomly assigned children to a MTFC treatment

¡64 JOURNAL OF PUBLIC CHILD WELFARE

group or control group. The study examined children in foster care ages 5 to 12 years who were experiencing a new placement and their foster parents. A total of 640 sets of children in foster care and their parents are enrolled—333 in the experimental treatment group and 305 in the control group; 40% of the children lived in kinship homes.

Project KEEP is distinct from MTFC and standard foster care in its training, support, and teaching methods. Project KEEP caregivers use parental daily report (PDR) to repeatedly assess behavior prob- lems. This tracking system helps identify the challenges faced by caregivers and assists child welfare personnel in tailoring training and interventions to the specific caregiver's needs. Project KEEP's parent training is distinct, providing weekly support and training in behav- ior management. Project KEEP staff relate problems the foster/kin parents are having in their homes to intervention strategies. As a result, attendance for parent training exceeds 80%. Additionally, the PDR system enables staff to follow up with caregivers on identified problems. Foster/kin parents receive $15/week to cover expenses plus day care and snacks during training.

The evaluation of Project KEEP provides insight to how to ap- proach CWS processes and impact CWS' outcomes. In examining Project KEEP's approach to the processes in CWS, researchers found foster/kin parents tolerate about as much child problem behaviors as non-system families, five behaviors a day. Additionally, caregivers said PDR data is feasible to collect and is well tolerated by foster/kin parents; therefore, PDR presents a potential tool for CWS. Project KEEP also provided insight in how to potentially strengthen CWS' outcomes. Children with foster parents who participated in Project Keep were less likely to be disruptive (Chamberlain et al., 2006). These two outcomes taken together produced a statistically significant positive effect on exits (Price et al., 2008). Children whose foster parents participated in Project KEEP were almost twice as likely to leave foster care for reunification or adoption. Children whose foster parents were not using Project KEEP were more likely to run away, disrupt, or have another negative exit from care.

CHALLENGES OF IMPLEMENTING EVIDENCE-BASED PRACTICES IN CHILD

WELFARE SERVICES

There are many questions about what the CWS worker should be able to do. For example, should a worker be able to do the following?

Richard P. Barth 165

• Assess which children need access to an evidence-supported (mental health) intervention;

• Be able to refer children to the correct ESI, based on assessment data collected as part of their investigation and on the interests of the child and family and on determination of which ESI is most acceptable to the family and most likely to enhance parent engagement;

• Be able to ensure that contract workers (e.g., vendors of parent training) are delivering an ESI; and

• Be able to deliver the ESI themselves?

Perhaps these tasks are not something that can be expected, or may not even be needed, depending on the organization of the services in each individual CWW's agency. A worker should, however, be able to understand and engage in the process of EBP so that he or she can determine what interventions are lacking and need to be developed.

Evidence-Based Practice, Systems Refonn, and -• - Practice Frameworks

Most of what child welfare workers do is guided by regulation (e.g., required number of home visits) or by practice frameworks. Practice frameworks are often considered to be enough to be the basis for training, even when entirely new programs such as alternative/multiple response are implemented. These programs may be implemented, as they often are, without any training on practice but with all the training focused on the implementation concepts and the forms or procedures. These guideline-based interventions are unlikely to have a significant influence on agency performance.

For EBP to have a greater influence on CWS the concept needs to be added to CWS practice frameworks and integrated into the implementation of new methods of providing services. Guidelines and practice frameworks are not likely to yield better outcomes un- less strong interventions are delivered (Bickman et al., 1999). There must be substantive opportunities to learn new ways of behaving as practitioners and these must be followed by parallel opportunities for children and families to learn.

CONCLUDING POINTS

Wisdom comes from experience and opportunities to learn from it. Excellent supervision, self-evaluation processes, and personnel poli-

¡66 JOURNAL OF PUBLIC CHILD WELFARE

cies that promote hiring of informed staff are key to CWS reform. In moving practice towards a more evidence-based approach, tbe profession must move through three stages. First practitioners and researcbers must discover new knowledge as researchers and practi- tioners uncover new approaches to CWS. Second is the deveiopmcnt of highly effective evidence based methods based on the new knowledge. Last, researchers and practitioners must figure out how to deliver the new knowledge and interventions in a manner that people can understand and apply.

EBP is a resource intensive effort (Whiting-Blome & Steib, 2004). Researchers and practitioners need resources to adapt known practices to different practice settings. Practitioners need resources including sufficient time, reduced case loads, adequate funding, and support to learn new practices. Also resources are necessary to implement new practices through external and internal supervision.

Evidence-based implementation requires reform of programs and processes. Good new ideas have been developed tbat could assist CWS. Parent training is the most developed and needed. Implementing these ideas cannot happen all at once and is only possible by allocat- ing sufficient resources in order to adapt them to CWS populations and practice parameters. Extensive supervision must also be provided during implementation. Self-evaluation strategies, strategy teams, and personnel policies to increase the longevity of tbe use of services are also needed to strengthen CWS and related mental health and substance abuse services.

Again, expanding EBPs must be addressed from multiple arenas including through funding and advocacy measures. Funding practices can be changed by linking funding and reimbursement for CWS to objective outcomes rather than outputs (Chaffin & Friedrieb. 2004) or by paying a premium for implementing ESIs with fidelity. EBP uptake grants can provide agencies with the necessary start-up capital to migrate to best practice models. Additionally, the profession can in- crease advocacy and social demand for best practices by disseminating cautiously derived information to funding organizations, governing boards, third-party payers, parents, and professional organizations.

The next step for CWS is to strengthen tbe quality and quan- tity of evaluations. CWS must adapt and test family engagement and parent education interventions witb strong evidentiary support witb related child welfare populations. The quality of methodical and cost-effectiveness reviews must also be improved. In addition, standards must be developed for providers and funders of evaluations to follow. Further, the profession has to develop competent third

: Richard P. Barth - ,

party evaluators. Moreover, CWS must plan systematic and long-term research programs to fill the service gaps and to start to focus on other elements of CWS that most need evaluation—especially supervising visits between parents and children in foster care. Last, as researchers, the longterm goal of infusing an evaluation climate in child welfare agencies must be considered (Chaffin, 2006). CWS continue to roll out new interventions on the basis of great anecdotes without a systematic and prospective plan to evaluate them. To paraphrase Matt Stagner (Stagner & Sinnott, 2007), the current child welfare service system is a social experiment in which putting the scientist in the labotatory to see what happened was all but forgotten. Yet, there is enough known about effective interventions that the use of EBP services for child treatment, parent support, substance abuse treatment, adult and child mental health services, and parenting classes should be expected. Developing policies that mandate the use of EBP should foster increased accountability as well as increase efforts for research and the incorporation of research into practice.

Once these interventions are developed, child welfare workers must have the knowledge, skill, motivation, and time to make referrals to them. Much has been learned about teaching ESIs so that they are used by the practitioners who learn them, but little is know about how to arrange for referrals to them. Very often, child welfare workers do not know very much about the services to which they refer—as these are community services, tnany of them not under the CWS agency's control (Hurlburt et al., 2007). An experienced provider of PCIT in Washington State recounts,

~ Many service providers in our state find themselves begging for referrals, doing dog and pony shows for caseworkers, to little avail. It is so absurd that a local agency that provides therapeutic child care, which is covered by Medicaid, actually has a fulltime person to troll the halls of the CWS to get them to remember to make referrals. Washington State had a Foster Care Assessment Program that was so underutilized that the legislature passed legislation requiring referrals. (L. Berliner, Director, Harborview Center for Sexual Assault and Traumatic Stress, Seattle, WA, personal communication, July 11, 2005). , '

If CWS develops more ESIs or they become more available to them and their clients, the problem of getting referrals to them will be a welcomed one.

168 JOURNAL OF PUBLIC CHILI) WELFARE

Social workers with bachelor's and master's degrees should under- stand the process of EBP and have basic familiarity with current ESIs and an understanding of how to identify emergent ESIs. Yet, the sci- ence of evaluation is a small part of what we emphasize in social work programs. If given the chance, EBP can become a unifying framework for social work education insofar as it integrates information about cultural competence, ethics, research, practice theory, development, macro- and micro-perspectives and practice skills (Drake et al., 2007). Social workers who are engulfed in the excitement of EBP may never be the same again.

REFERENCES

American Public Services Association, National Association of Public Child Welfare AümÍEiislrators. (2005), Guide for child welfare administrators on evidence based practice. Retrieved April 23, 2(X)4 (Vüiii httpV/www.aphsa.Di'g/hoinc/doc/Guide- ror-Bvidence-Ba.sed-Praclice.pdr

Barih, R. P., Landsvcrk, J.. Chamberlain, P., Reid, J., Rolls, J., Hurlbtirt. M., ct al. (2005). Parent iraininy in child welfare services: Planning for a more evidence based approach lo serving biological parents. Research on Social Work Practice. 7 5 , 3 5 3 - 3 7 1 .

Berrick. J., Needell, B., Barlh, R. P., & Jonson-Reid. M. (1998). The tender years: Toward deveiopmentaily sensitive cliiid weifare services. New York: Oxford.

Bickman, L.. Noser, K., & Sunimerfelt. W. T. (1999). Long-term ellects of a system of care on children and adolescents. Journal of Befiaviorai Health Science. 26, 185-202.

Bickman, L., Riemer, M., Breda, C , Kelley, S. D. (2006). CFIT: A system to provide a continuous quality improvement infrastructure through organizational responsiveness, measurement, training, and feedback. Report on Hniolional and Behavioral Disorders in Youih, 6, 86-87, 93-94.

Bruder, M. B., Harbin, G. L., Whitbrcad, K,. Conn-Powers, M., Roberts, R., Dunst, C. J., et al. (2(K)5). Establishing outconie.s for service coordination: A step toward evidence-based practice. Topics in Early Childhood Special Education, 25(3), 177-188.

Chaitin, M. (2006). Organizational culture and practice epistemologies. Clinical Psychology-Science and Practice, I3(i), 9 0 - 9 3 .

Cliallin, M., & Friedrich, W, (2004). Rvidence-based treatments in child abuse and neglect. Children & Youth Services Review. 26(11), 1097-1113.

Chaflin, M., Silovsky, J., Funderburk, B.. Valle, L. A., Brestan, E. V., Balaeliova, T., ct al. (2004). Parent-child interaction therapy with physically abusive parents: Klltcacy for reducing future abuse reports. Journal of Consulting and Clinical Psychology. 72, 500-510.

Richard P. Barth ¡69

Chaiiibcrlain, P. (1998 December). Treatment foster care. Washington, DC: US Depanmcin of Justice, OHice of Justice Programs, Oflice of Juvenile Justiee and IX'linquency Prevention.

Chamberlain. P. (2003). Muludimensiunal Treatment loster care program compo- nents and principles of practice. In P. Chamberlain (Rd.), Trealiiii- chnmic juvenile offenders: Advances made timmgh the Oregon nmlihiimensionul treatmeni foster care model (pp. 1-22). Washington, DC: American Psychological Association.

Chamberlain, P.. Moreland, S., & Reid, K. (1992). Hnhanced services and stipends lor loster parents: btïects on relention rales and outcomes tor children. Child Welfare, 7J, 387-401.

Chamberlain, P., Price, J. M., Reid, J. B.. Landsverk. J.. Fisher. P. A.. & Stoolmiller. M. (2006). Who disrupts from placement in foster and kinship care? Child Abuse & Neglect. 30(A), 4 0 9 ^ 2 4 .

Chamberlain, P.. & Reid. J. (1991). Using a specialized foster care communiiy treatineiii model tor ehildren and adolescents leaving the state mental luispiial. Journal of Community Psychology. (9(?>), 266-276.

Child Welfare League of America (n.d.). Research to practice initiaiive: About the program. Retrieved August 5, 2(H)8, trom hltp://www.cwla,t)rg/program.s/r2p/ r2paboutpage.htm

Cooper, B. (200.1). Evidence-based menial health policy: A critical appraisal. British Journal of Psychiatry, 183. 105-113.

Dickinson, N. S. & Gilde Gibaja. M. (2()04). University-child welfare partnerships: Familiar collaborations, new possibilities. Ptvtecting Children, /9(3). 27-36.

Drake, B.. Hovmand. P . Jon.son-Reid, M.. & Zayas, L.H. (2007). Adopting and teaching evidence-based practice in master's level s^K'ial work programs. Journal of Social Work Education, 43. 431-446.

Gambril!, E. D. (2()03). Evidence-based practice: Sea Change or the Emperor's New Clothes? Journal of Social Work in Etiucation, 39, 3-23.

Gambrill. H. D. (2003). [Décision making in child welfare: Errors and their context. Children and Youth Serx^ices Review, 27, 347-332.

Gihbs, L. B. (2003). Evidence-based practice for the helping p/vfessions. Pacific Grove, CA: Brooks/Cole and Thomson Learning.

Glisson. C . Dukes, D.. & Green. P (2006). 1 he etïects of Ihe ARC orgitnizational intervention on caseworker turnover, climate, and culture in children's service systems. Child Abuse Á. Neglect. 30ß), 833-880.

Harrison. M. R.. McKay. M. M., & Bannon, W. M. (2004). Inner-city child mental health service use: The real question is why youth and families do not use services. Communiiy Mental Health Journal, 40. 119-131,

Hurlburt. M. & Barth, K.P. (2007). Parent training in child welfare services: Findings from the National Survey of Child and Adolescent Well-Heing. In R. Haskins, F. Wulczyn, & M. B. Webb (Eds.). Child protection research fivm the Na- tional Survey of Child and Adolescent Well-Being (pp. 81-106). Washington, DC: Brciokings.

Jones. L. M . hinkelhor. D., & Kopiec, K. (2001). Why is sexual abuse declining? A survey of state child protection administrators. Child Abuse & Neglect. 25, 1139-1158.

¡70 JOURNAL OF PUBLIC CHILD WELFARE

Kohl, P. & Barth, K.P. (2007). Reentry, reabuse, and reinvolvement of maltreated children: h'indings from Ihe National Survey of Child and Adolescent Well- Beiiig. In R. Haskins, F. Wulczyii, & M. B. Webb (Eds.). Child pivtection: Using research to inipivve policy and practice (pp. 207-225). Washington, DC; Brook ings.

Kolko, I). J., & Swenson, C. C. (2002). Assessing and treating physically abused children and their families: a cognitive-behavioral approach. Thousand Oaks, CA: Sage Publications.

Linares, L. O., Montallo. D., Li, M. M., & Oza, V. S. (2006). A promising parenting intervention in foster care. Journal of Consulting and Clinical Psychology. 74( I ),

Littell, J. H, (2005), Lessons from a systematic review of eíTects ol" multisystemic therapy. Children and Youth Services Review, 27, 445-463.

Littell, J. H. (2007). Evidence-ha.sed practice? Where's the evidence. Paper presented to the SchiKil of Social Work, University of Maryland, February 15, 2007.

Lutzker, J. R., & Bigelow, K. M. (2(K)2). Reducing child maltreatment: A guidebook for parent services. New York, NY: The Guillord Press.

Magruder & Shaw, (in press). Children ever in care: An examination of cumulative disproportioiiülity. Child Welfare.

McKay, M., Hibbert, R, Hoagwood, K, Rodriguez, J, Murray, L, Legerski, J, & Fernandez, D. (2004). Integrating evidence-based engagement interventions into "real world" child mental health settings. Brief Treatment and Crisis Intervention 4(2), 177-186.

McKay, M. M., Lynn, C. J., & Bannon, W. M. (2005). Understanding inner city child menial health need and traimia exposure: Implications for preparing urban service providers. American Journal of Orthopsychiatry, 75(2). 201-210.

McKay, M., McCadam, K., & Gonzales, J. (1996). Addressing Ihe barriers to mental health services for inner cily children and their caretakers. Community Menial Health Journal. 32, 353-361.

McCarthy. J., Van Buren, H., & Irvine, M. (August 2007). Child and family service reviews 2001-2004: A mental health analysis. Washington, DC: National Tech- nical Assistance Center tor Children's Mental Health, Georgetown University Cenler for Child and Human Development.

National Center lor Injury Prevention and Control (2CK)4). Using Evidence-Ba.sed Parenting Programs to Advance CDC Efforts in Child Maltreatment Prevention Research Activities. Atlanta, GA: Centers for Disease Control and Prevention.

Nock, M. K. & Kazdin, A. H. (2005). Randomized controlled trial of a brief intervention for increasing participation in parent management training. Journal of Consulting atul Clinical Psychology. 73, 872-879.

Patterson, G. R. (1975). Families: Applications of social learning to family life. Champaign, IL: Research Press.

Pew Ct)mniission on Children in Foster Care (2002). Fostering the future: Safety, permanence, and well-being for children in foster care. Philadelphia, PA: Pew Charitable Trusts.

Price, J, M., Chamberlain, P., Landsverk, J., Reid, J. B., Leve, L. D., & Laurent, H. (2008). Eftects of a foster parent training intervention on placement changes uf children in foster care. Child Maltreatment, / i ( l ) , 64-75.

Richard P. Barlh ' . | 7 f

Reid, J. B., Patterson, G. R.. Snyder, J. (2002). Antisocial behavior in children and adolescents: A developmental analysis and model for intervention. Washington, DC: American Psychological Association.

Sackett, D. L., Rosenberg. W. M.. Muir Gray, J. A.. Haynes. R. B., & Richardson W. S. (1996). Rvidenced-based medicine: What it is and what it isn't. British Medical Journal. 312, 71-72.

Sacketl, D. L., Straus. S. E.. Riehard.son, W. S., Ro.senberg, W.. & Haynes. R. B. (2000). Evidence-ba.sed medicine: How to practice and teach EBM. Hdinburgh: Churchill Livingstone.

Saunders, B. H., Berliner, L., & Hanson. R. F. B. (2003). Child physical and sexual abuse: Guidelines for treatment (Final report: January 15. 2003). Charleston, SC: National C'rime Victims Research and Treatment Center.

Stagner. M. & Sinnott, M. R. (2007). Research provides broad context and measur- able value. Alliance for Children's and Families Magazine, Summer 2007. Re- trieved October 18. 2007, from http://www.alliancel.org/magazinc/Sumnicr20O7/ su()7_l n Ic rv i e w. pd f ;.

Thomas, R., & Zimmer-Gembeck, M. J. (2007). Behavioral outcomes of parent- ehild interaction therapy and triple p-positive parenting program: A review and meta-analysis. Journal of Abnormal Child Psycholugy, 350), 473-495.

Timnier, S. G,. Urquiza. A. J., Herschell, A. D.. McGrath. J. M., Zebell, N. M.. Porter, A. L..elal.(2(K)6). Parent-chiltilnteraction ihcrapy: Application of an empirically supported treatment to maltrealed children in foster care. Child Welfare, S5(6), 919-939.

United Stales (US) t>partment of Health and Human Services Administration for Children and Families. (2003). National survey of child and adolescent well- being: Children involved with the child welfare services (Baseline Report). Wash- ington, DC: Author.

US Department of Health and Human Services Administration on Children Youth and Families. (2007). Child maltreatment 2005. Washington, DC: US Government Printing Oftice.

US Department of Health and Human Services. (2008). Trends in foster care and adoption FY 2002-2006. Retrieved August 3, 2008, from hitp://www.acf.hhs.gov/ programs/cb/stats_research/afcars/trend.

Whitehurst. G. J. (2002). Evidence-based education (EBE). United Stales Deparlmenl of Education. Retrieved April 26, 2003, from http://www.ed.gov/nclb/mcthods/ whalworks/eb/edlite-slide021.html

Whiting-Blome, W. & S. Steib. (2(K)4). Whatever the problem. Ihe answer is evidence- based practice—or is il? Child Welfare. 83, 611-615.

RECEIVED; 12/12/07 REVISED: 04/27/08

ACCEPTED: 05/02/08