annotated bibliography

EvaluatingParentInvolvementandFamilySupportinEarlyInterventionandPreschoolPrograms.pdf

Home >Education homework help >annotated bibliography

FEATURE ARTICLE

Evaluating Parent Involvement and Family Support in Early Intervention and Preschool Programs

DONALD B. BAILEY, JR. University of North Carolina at Chapel Hill

Early intervention and preschool programs for children with disabilities are also accountable for providing certain types of support for families. How should these efforts be evaluated? This article describes three potential levels of accountability: (a) providing the legally required services for familiesf (b) providing services that are considered recommended, and (c) achieving certain outcomes as a result of working with families. Issues and considerations related to each level of accountability are discussed and recommendations are made for advancing policy and practice related to the evaluation of parent involvement and family support efforts.

A combination of legislative initiatives, fam- ily advocacy efforts, theory, and research has led to wide acceptance of the assumption that early intervention exists not just to support young children with disabilities, but also to support their families. Exactly what is meant by parent involvement and family support continues to be discussed, but at least three themes have emerged around which there is general consensus (Bailey et al., 1986; Bailey et al., 1998; Brewer, McPherson, Magrab, & Hutchins, 1989; Dunst, 1985; Shelton, Jepp- son, & Johnson, 1987). First, parent involve- ment and family support programs need to be individualized, given the diversity of family resources, priorities, concerns, and cultures. Second, parents should be given every oppor- tunity to participate as active partners in plan- ning services for their child and for them- selves, requiring professionals to engage in practices that recognize, value, and support this type of relationship. Third, since families are the ultimate decision makers and long- term care providers for their children, services

should be organized in ways that enable fam- ilies to feel and be competent in advocating for services and otherwise meeting the needs of their young child with a disability.

As states and local programs strive to provide a variety of family support initiatives, a funda- mental question remains unanswered: How should we evaluate whether parent involvement and family support efforts have been successful? In this paper I place this question in the context of accountability and propose three potential levels of accountability. Challenges associated with each level are presented, and I conclude with several potential recommendations for the field.

PROGRAM EVALUATION AND ACCOUNTABILITY

The principles and processes underlying pro- gram evaluation have been well described over the past few decades (Fink, 1995; Popham, 1993; Walberg & Haertel, 1990; Worthen, Sand- ers, & Fitzpatrick, 1997). Many definitions of

Bailey 1

evaluation have been offered, but the essence of each focuses on determining the worth of a pol- icy, program, or practice. Evaluation can and usually should involve multiple methods of data collection, analysis, and interpretation. Unlike more basic research, however, which seeks to understand fundamental laws or principles that underlie behavioral or biological phenomena, evaluation seeks to determine whether a partic- ular policy, program, or practice is worthwhile, better than other alternatives, affordable, ac- ceptable to others, and effective in meeting the needs of the individuals it is designed to serve.

Before an evaluation can be done, a clear de- scription of the policy, program, or practice be- ing evaluated is essential. An obvious challenge is that parent involvement and family support efforts in early intervention are multifaceted and can be conceptualized simultaneously as a set of policies, a set of program models, and a var- iable set of practices. As a policy, parent in- volvement and family support efforts are rooted primarily in the Individuals with Disabilities Ed- ucation Act (IDEA). Part C of IDEA explicitly acknowledges that a primary goal of early in- tervention is to help families meet the special needs of their infant and toddler with disabili- ties. Part B, which addresses preschoolers and school-aged children, is less explicit about fam- ily support as a primary goal, but contains a number of provisions regarding family rights and responsibilities in the context of deciding on goals and needed services for their children.

As a program, no one model characterizes parent involvement and family support pro- grams in early intervention and preschool pro- grams, as enormous variability exists (Beck- man, 1996; Harbin, McWilliam, & Gallagher, 2000). Although IDEA describes 16 compo- nents required of a statewide system of early intervention, within these guidelines states are given tremendous latitude in the way state and local programs are organized. Parent involve- ment and family support efforts are often part of a larger program of services, and might in- clude home visits, parent support groups, par- ent training activities, respite care, resource and referral, or service coordination.

As a set of practices, there is also enormous variation in the behaviors and activities of

professionals who are affiliated with parent in- volvement and family support programs. Working with families includes a wide variety of practices that cumulatively lead to a per- ceived and actual level of family support. Ear- ly intervention and preschool personnel estab- lish relationships with families, listen and re- spond to families' priorities and concerns, try to understand family perspectives, build on (and try not to supplant) informal support sys- tems, employ help-giving practices and atti- tudes that are consistent with current litera- ture, and assist families in accessing commu- nity resources (Bailey, 1994; Dunst, Trivette, & Deal, 1994).

A fourth dimension could also be added, namely that many of these programs and prac- tices are based on philosophical perspectives and assumptions which have consequences for the nature and type of services provided. For example, one program might have family em- powerment as its primary goal, based on the philosophical assumption that the primary goal of early intervention is to enable families to secure their own services and make major decisions about allocations of time and re- sources (e.g., Dunst, Trivette, & Deal, 1988). Another program, however, might have "par- ents as teachers" as its primary goal, based on the philosophy that families are the best and most important teachers of their children, but might need help in understanding and us- ing developmentally appropriate styles of in- teracting and communication with their chil- dren (e.g., Mahoney, Boyce, Fewell, Spiker, & Wheeden, 1998).

Historically, evaluation efforts have been categorized into two broad types of activity (Scriven, 1967). Formative evaluation is in- tended to provide staff with evaluation infor- mation that could be used to help change or improve the program. Usually formative eval- uation occurs during the implementation phas- es of a project and attempts to document whether the practices designed to constitute the program are indeed in place and whether any initial effects in the hypothesized direc- tion are evident. This information is then used to improve the program and to bring it in line with the originally proposed model. In some

2 JEI, 2001, 24:1

cases, this information actually might result in changes in the model.

In contrast, summative evaluation is typi- cally conducted at the end of a period of pro- gram implementation. The purpose of sum- mative evaluation is to determine whether the program did, in fact, accomplish its aims. This would include a focus on the practices in the program (e.g., Did the program provide what it said it would provide?) as well as on the outcomes of the program (e.g., Were the goals of the program achieved?).

Questions regarding program evaluation may also be considered in the context of ac- countability. Questions of accountability seek to determine that for which programs and ser- vice systems are responsible. Unlike a general summative evaluation model that asks, "What did this program accomplish?", accountability asks, "Did this program accomplish the spe- cific goals for which it was established?" In this article, I focus on summative evaluation questions couched in the context of program accountability. I argue that early intervention programs could be held accountable for three things: (a) providing the legally required ser- vices for families, (b) providing services that are consistent with current philosophy about recommended practices, and (c) achieving certain outcomes as a result of working with families. In describing and discussing these three potential levels of accountability, I draw parallels for each from prior efforts to evalu- ate child care and early intervention programs. The field has a great deal of experience in documenting these levels of accountability in the context of programs for children. Exam- ining the types of approaches that have been used at each level in evaluating child-focused efforts might help clarify some of the issues and approaches that will be needed in evalu- ating family-centered services.

THREE LEVELS OF ACCOUNTABILITY

Are We Providing the Services to Which Families Are Legally Entitled? The first question addresses the extent to which early intervention and preschool pro-

grams provide the parent involvement and family support activities for which they are legally responsible. In child care, this would be comparable to the basic licensing regula- tions for any early childhood program. All states have requirements for certain aspects of child care such as adult-child ratios, square footage per child, safety requirements for playground equipment, and so forth. Although a list of comparable specificity regarding fam- ily support activities does not exist in most states, the federally mandated components of Part C and Part B of IDEA, as well as any additionally legislated state requirements, pro- vide the basis for determining these respon- sibilities. A list of questions summarizing the major legislative requirements for early inter- vention and preschool programs is provided in Table 1.

Evaluating legal accountability to families assesses the extent to which early intervention programs comply with state and federal rules and regulations. This form of evaluation con- stitutes a monitoring function. Although di- mensions of quality could be assessed, at this level evaluation focuses primarily on compli- ance with explicitly required dimensions of practice, such as the requirement that the In- dividualized Family Service Plan (IFSP) be completed within 45 days of referral or the inclusion of all required domains on the IFSP.

This level of evaluation is essential, as fam- ilies must be offered all of that to which they are entitled. In some respects this is the sim- plest of the three accountability questions, but even here a number of challenges become ap- parent. Assuring this level of accountability requires a formal monitoring of practices by state and federal agencies. IDEA and the ac- companying federal regulations have in- creased the expectation that states be able to document full compliance with the require- ments of the law.

Who will do the evaluation? The U.S. De- partment of Education will provide one level of evaluation at the state level through the Continuous Improvement Monitoring Pro- gram (CIMP; Office of Special Education Programs, 2000). It is, however, each state's responsibility to establish the policies and pro-

Bailey 3

Table 1. Legal Accountability to Families: Federal Requirements for Parent Involvement and Family Sup- port Activities

A. Responsibilities of Part C programs serving infants and toddlers 1. Were families a part of each IFSP team? 2. Did the early intervention program use family-directed assessment of family resources, priorities, and

concerns? 3. Did all IFSPs contain, if the family so desired, procedures to address both child and family needs? 4. Was a full explanation of each IFSP provided in the family's native language and informed consent

obtained prior to the provision of early intervention services? 5. Were parents fully informed of their right to accept or decline any early intervention services? 6. Were all specified procedural safeguards in place? 7. Was a service coordinator identified for each family who was responsible for implementing the plan

and coordinating with other agencies and persons? 8. Was a review of the IFSP provided for each family at least every 6 months? 9. Was written prior notice provided to parents before initiating or changing any child's identification,

placement, or services?

B. Responsibilities of Part B programs serving 3- and 4-year-olds 1. Was informed parent consent attained before evaluating or reevaluating each child? 2. Were parents included as members of any group making decisions regarding eligibility for services? 3. Were parents included as members of any group making decisions about the placement of the child? 4. Were parents included as members of each IEP team? 5. Did parents agree to and sign all IEPs prior to the initiation of services? 6. Did the school use simple, understandable terms in each family's native language to describe family

rights? 7. Were parents given access to all records relating to their children? 8. Did the school provide an adequate set of procedural safeguards for families, including due process

and a voluntary mediation process? 9. Was written prior notice provided to parents before initiating or changing any child's identification,

placement, or services?

cedures by which these evaluations will occur at the local level. In states where education is not the lead agency for Part C, this could mean a dual system of evaluation for Part C and Part B services. Ideally, agencies would work together to assure comparability of stan- dards and guidelines for assessment.

How will compliance be assessed? Assess- ment procedures and guidelines will need to be developed. Under the CIMP program, the Office of Special Education Programs has de- veloped "cluster areas" in which indicators are listed as ways to provide evidence for doc- umenting that states are complying with fed- eral legislation. States are expected to design and implement an ongoing self-assessment process. In many states, this might result in a checklist for documenting whether or not the practice or regulation has been implemented.

However, a simple checklist might not ade- quately reflect practice, and gradients of im- plementation might be necessary. For exam- ple, a quantitative gradient could document whether each regulation has been implement- ed with all or only some families served by the program. More challenging will be the ad- dition of a qualitative gradient. Clearly there will be different levels of quality in imple- menting the requirements of the legislation. Take, for example, the requirement of "full explanation of the IFSP in the family's native language and informed consent obtained." A factual presentation of the IFSP might meet the letter of the law, but documentation of "full explanation" would also require some determination of whether or not families ac- tually understood the IFSP.

To complicate matters further, the CIMP

4 JEI, 2001, 24:1

process even includes evaluation criteria that are related to, but not explicitly part of the regulations listed in Table 1. For example, component CF.l states that "Community out- reach is provided in family-centered language, locations, and formats," and CF.2 states that "The needs of families with eligible infants and toddlers are identified and addressed through the family-centered orientation of pol- icies, procedures, and practices." Although some indicators of family-centered practices are provided (e.g., services and support sys- tems are flexible, accessible, comprehensive, and responsive to diverse family-identified needs), we see here a blurring of the lines be- tween this level of accountability (for the things to which families are legally entitled) and the next level of accountability (for pro- viding high quality services). This will un- doubtedly cause some potential conflicts as states and the federal government negotiate required versus optional components of ser- vice.

What will be the source of information? Documenting legal accountability to families will be a complex undertaking. It will likely require gathering information from a number of different sources, including a review of documents, information from staff and super- visors, and, ideally, input from families. Al- though the regulations seem straightforward at first glance and some can easily be docu- mented from a records review (e.g., Have 6- month reviews of IFSPs been conducted?), parent perspectives on guidelines such as fam- ily-directed assessments, being fully in- formed, and being included as part of the team will likely be important. Gathering informa- tion effectively from families, however, will require fully informing families of their rights so that they can determine for themselves whether they have received all required ser- vices.

Are Parent Involvement and Family Support Programs of High Quality? The next level of accountability refers to our obligation to provide parent involvement and family support services that are of high qual- ity and consistent with recommended practic-

es. In child care, a considerable amount of re- search has been devoted to determining how to measure quality, typically by using scales to rate the quality of environments and inter- actions (e.g., Harms, Clifford, & Cryer, 1998), describing the range of quality that currently exists in child care, examining factors that ac- count for variation in quality (e.g., education of teachers, private versus public childcare), and determining the relationship between quality of care and outcomes for children (e.g., Cost, Quality, and Child Outcomes Study Team, 1995). Generally this work has shown that multiple dimensions of quality can be measured, that considerable variation is ev- ident in quality, and that quality makes a dif- ference in developmental and behavioral out- comes for children,

Much has been written about quality in the ways we work with families, building on var- ious theories, constructs, and practices such as enablement, empowerment, help giving, par- ticipatory service planning, open communi- cation, collaborative goal setting, advocacy, and support (e.g., Beckman, 1996; Dunst et al., 1988; McWilliam, Winton, & Crais, 1996; Turnbull & Turnbull, 1997). Federal and state legislation provides the context in which these practices might or might not be exhibited, but the legislation does not mandate them.

A list of questions summarizing the major components of high-quality services described in the literature is provided in Table 2. Two essential dimensions of quality are reflected. The first dimension is the extent to which par- ent involvement and family support services are reflected in the overall philosophy and model of services provided by the local pro- gram. A family focus should be central to the program and shared by all team members. Ide- ally, families should have collaborated with professionals in the development of such a philosophy. Families should be viewed as competent and legitimate participants in the team, and interactions with families should generally be of a positive nature. All team members should recognize and respect the di- versity evident in family resources, priorities, and concerns, responding appropriately in ac- cordance with this variability. A logical con-

Bailey 5

Table 2. Are Early Intervention Programs Providing Parent Involvement and Family Support Services of High Quality?

A. Philosophy and program models 1. Was a family focus central to the program and shared by all team members? 2. Were families invited to collaborate in the development of the program philosophy? 3. Were families viewed as competent and legitimate participants on each early intervention team? 4. Were interactions with families positive? 5. Did all team members respect and respond appropriately to family diversity in beliefs, values, and

coping styles? 6. Were services flexible enough to meet individual family needs?

B. Practices with families 1. Did professionals engage in well-documented help-giving practices, such as active listening, clarifying

concerns and needs, and so forth? 2. Was family participation in decision-making processes actively supported throughout all phases of

intake, evaluation, program planning, program implementation, and program evaluation? 3. Were families actively supported in their efforts to serve as agents of intervention in promoting the

child's acquisition of competencies? 4. Were families supported in their efforts to create and use natural supports? 5. Were families supported and mentored in their efforts to develop skills in advocating for themselves

and their children? 6. To the extent they desire, were families supported in their efforts to assume primary responsibility for

service coordination?

sequence of this diversity is that services must be flexible enough to meet individual family needs.

The second dimension is the implementa- tion of certain practices identified as important to a family-centered approach. Professionals should engage in well-documented help-giv- ing practices, such as active listening, clari- fying concerns, and so forth. Family partici- pation in all aspects of decision making (e.g., intake, assessment, program planning, service delivery) should be actively sought and en- couraged. Families should have available to them appropriate assistance to enable them to provide developmentally appropriate and stimulating environments for their children and to access and use a range of formal and informal support systems. Services should help families develop skills in advocating for themselves and in identifying and accessing needed services.

As is evident in this list of factors, most of the practices described as part of a family- centered approach have to do with the nature and quality of the relationship between par-

ents and professionals (Dinnebeil, Hale, & Rule, 1996; Dunst, Johanson, Trivette, & Hamby, 1991). For example, in a study of pro- fessionals rated high on family-centered prac- tices and some of the families those profes- sionals served, McWilliam, Tocci, and Harbin (1998) found five underlying components of family-centered practice: positiveness, respon- siveness, orientation to the whole family, friendliness, and sensitivity.

Assessing the extent to which early inter- vention and preschool programs provide qual- ity services to families goes beyond monitor- ing compliance with regulations. This level of evaluation raises its own set of unique issues: What do we mean by quality? Are there di- mensions of quality on which we can all agree as being essential? What specific indicators should be used to document quality? How would one validate a quality of family servic- es scale? What about the subjective nature of quality (i.e., quality as perceived by families versus quality relative to some set of practices or standards)? Do cultural variations exist in assumptions about quality practices, and if so,

6 JEl 2001, 24:1

how can such variations be meaningfully in- corporated in a quality assessment? Who should be responsible for documenting quali- ty?

It is in this area of accountability that the most measurement work has been done thus far. A number of groups and individuals have developed or are in the process of developing scales to rate various dimensions of quality of family services. A brief description of several of these measures is provided in Table 3. The scales vary widely in terms of the number of items and the organization of items into clus- ters. All are ratings based on impressions and experiences, rather than direct observation of practice. Some include parents as respondents, whereas others involve ratings by professional staff. No single measure has been adopted by the field as the standard for assessing quality, and no studies have compared one scale with another. Perhaps most important, only limited research has addressed the relationship be- tween quality and outcomes for children and families, and the results to date have been mixed. For example, Trivette, Dunst, Boyd, and Hamby (1995) found that help-giving practices by professionals in early interven- tion programs were strongly related to par- ents' reports of personal control. Mahoney and Bella (1998), however, found that moth- ers' ratings of the family-focused quality of early intervention were not related to chil- dren's developmental functioning, maternal interaction styles, family functioning, or ma- ternal stress.

Cultural variation in families complicates issues regarding documentation of quality practices. Although it is clear that certain fun- damental aspects of practice (e.g., respect for the individual family's values and priorities, effective listening skills) generally transcend cultural contexts in their importance, it is equally clear that professional interactions with families will need to vary in accordance with cultural expectations and practices. Cre- ating an evaluation system that considers this important dimension of practice but also rec- ognizes the tremendous variability and indi- vidualization in the way these practices must

be implemented will be a considerable chal- lenge.

Most professionals and family members feel strongly that the provision of high-quality services is a moral imperative for early inter- vention. Yet the limited research in this area suggests that high standards of quality often are not being met, and numerous barriers (training, lack of administrative support, lim- ited resources) are often mentioned as imped- iments to quality (Bailey, Buysse, Edmond- son, & Smith, 1992). Thus a fundamental is- sue is the extent to which early intervention and preschool programs should be held ac- countable for providing a higher quality of family services.

Accountability for Family Outcomes A third level of accountability addresses the family outcomes of parent involvement and family support programs. The focus here is on the changes or benefits to families as a result of such services. This differs fundamentally from the first two levels of evaluation, both of which seek to determine the extent to which certain practices occur. Here we ask whether these practices have any functional conse- quences.

In the child care and early intervention lit- erature, much effort has been focused on doc- umenting outcomes of child care and early in- tervention programs. Typical child outcomes assessed in almost any study include devel- opmental progress, school achievement, social or behavioral outcomes, school placement, and the need for special services. In the family arena, however, this level of evaluation is challenging because there is relatively little consensus (or until recently even discussion) in the field as to what might constitute a de- sirable family outcome.

There have been a number of studies that have described particular outcome domains and have attempted to explore the relationship between early intervention programs or prac- tices and specific family outcomes, often de- pending upon the philosophy and assumptions underlying the program being tested. For ex- ample, studies have investigated the extent to which various parent training programs have

Bailey 7

*> >"•»* s 5 «5i i 5

s 0 0

•S •Si J< <S £ •5 Oa

*> o <J Q

^ Q* >> 05

<̂ t*> << K g «o

^ x. a

^ * * • *

•s § •s ^ H CO

*-« <D 03

O P H

s £

73 0 00

^ 3 CO

•8 11

13 0

0 0

<E s C/5

flj

If O > ,

11 0 ) , 0 3 H <4-4

P H

>> X ) 7 3

1 •a

o< <4-H

O GO

§•

( 4 - 1

0 00

13 0 00

0 0

1 g

' O

§ 0 a, S

§ OO

03 £ "3 0

<D > c

0 O

1 1

T3 <D

.S U S

^ 3 .s CN o q

5 *

* 1 . 3

o o a g a

s .2 ^_> > ~ » GO O 0

.h 3 «> £ fT C co O MH £ 00 Ui ^ «S 03 O H

o q

'55 o

> £ GO g

•̂ 5 c§ S - H J> o 00 , > Os

*> £ * 2

if I1

<4-H 0 0

22 T 3

—1 C

P PLH

! ! § 00

5 * Si •«2 O

•si o ^ W) o . c

>» ^̂

°Jg *H i-1 » O H O 0 8 a

1 § 5 s 2

S . 1

Osi PQ

ON i n

T3 i

03 >

» .2

03 fe . y

- . g 03

° -̂5 O H > , * - >> X5 VH

I I <L> w T 3 > 3 <U <U

« l g *2 S 2 a ^

l a g ca > «>

u .g -a

l-i § S S •*» -

f? ^ 1 ^ c co

•g-Bb 03

II a b

S -a a 03

s i s U g o ^ e '> •a o fc S U co

00 Q

0) <D

O H C M

C3 ©

^ ^

^ «G 03

C >> O § X) ~ D (D (U

"03 ^ c3

'3b

03 O

g *-r >» 'g a -g '5b 5 S

JEI, 2001, 24:1

been able to influence interactions between parents and their child with a disability (McCollum & Hemmeter, 1997). Other efforts have suggested that early intervention can help families build stronger formal and social support systems (Shonkoff, Hauser-Cram, Kraus, & Upshur, 1992) or help empower families (Trivette et al., 1995). Recent re- search by Thompson et al. (1997) suggested that there might be multiple pathways by which these outcomes are achieved, but the fact remains that attaining family outcomes through early intervention is indeed possible.

In several recent efforts, researchers have attempted to think more broadly about the ar- ray of possible family outcomes in early in- tervention and have suggested frameworks around which such an evaluation might be de- veloped. A summary of these recommenda- tions is provided in Table 4. Each group has chosen to describe potential family outcomes in a different way, but there is considerable overlap across the four models. The three most common domains of potential family outcomes of parent involvement and family support efforts are: (a) family satisfaction with services, (b) the family's knowledge of child development and their ability to provide a de- velopmentally supportive environment and advocate for their child's needs, and (c) the overall quality of the family's life and the changes that are needed in order to meet their child's needs.

A fundamental issue is whether we really want or expect to be held accountable for achieving outcomes for families beyond the traditional satisfaction with services. This area is fraught with conceptual, methodological, and resource challenges. One could argue that high-quality interactions with families that fulfill all of the state and federal requirements for early intervention and preschool programs are worthwhile and should be provided on the basis of their inherent merit. In an era of out- comes-based assessment, however, there might come a day when the field is asked to show whether family-centered practices have actually made any difference.

The evaluation of family outcomes will re- quire some consensus on what those outcomes

should be. The four models described in Table 4 each conceptualize family outcomes in dif- ferent ways, but there are no fundamental in- consistencies among them, rather just varying areas of emphasis. Perhaps the field should try to identify one or two core outcomes on which everyone can agree and for which we think there is consensus that, in fact, early interven- tion should and can result in those outcomes. A range of strategies then should be devel- oped to assess those outcomes and studies conducted to determine the extent to which they have been achieved. Current efforts by the Early Childhood Research Institute on Measuring Growth and Development (1998), as well as the National Early Intervention Longitudinal Study (a nationally representa- tive study of characteristics, services, and out- comes of early intervention programs funded by the Office of Special Education Programs and led by SRI International) should provide important information in this regard.

Of course, identifying common outcomes in some ways is contrary to the spirit of a fam- ily-centered approach, which argues for indi- vidualization of services based on each fam- ily's resources, priorities, and concerns. This perspective would suggest that the full range of family outcomes described above might not be equally applicable across all families, and thus assessing the extent to which they are achieved in early intervention might tell us very little unless we know whether families wanted or needed those outcomes. My own perspective is that it is still reasonable to ask if early intervention as a national endeavor has, for example, provided services that fam- ilies consider satisfactory or has enabled fam- ilies to feel more confident in their ability to support their child's development and to ac- cess needed formal and informal support sys- tems. If we cannot demonstrate these out- comes for the system as a whole, then we probably need to reexamine the assumptions and practices underlying the system.

CONCLUSIONS AND RECOMMENDATIONS

In this article I have addressed issues and con- siderations in evaluating parent involvement

Bailey 9

Table 4. Examples of Suggested Family Outcomes

Source Suggested Family Outcomes or Domains

Bailey et al. (1998)

Early Childhood Research Institute on Measuring Growth and De- velopment (1998)

Roberts, Innocenti, & Goetz (1999)

Turnbull, Turbiville, & Turnbull (2000)

Family perceptions of the early intervention (EI) experience 1. Does the family see EI as appropriate in making a difference

in their child's life? 2. Does the family see EI as appropriate in making a difference

in the family's life? 3. Does the family have a positive view of professionals and the

special service system?

Family impact 1. Did EI enable the family to help their child grow, learn, and

develop? 2. Did EI enhance the family's perceived ability to work with

professionals and advocate for services? 3. Did EI assist the family in building a strong support system? 4. Did EI help enhance an optimistic view of the future? 5. Did EI enhance the family's perceived quality of life?

1. Families will have a basic understanding of child development and will be able to identify needs for their child, including those related to cultural, linguistic, or disability specific issues.

2. Families will be able to assess how their child's development is progressing related to general outcomes identified on the IEP or IFSR

3. Families will be confident in their abilities to make choices about interventions for their child and will be able to implement those interventions effectively.

4. Families will feel that their beliefs and values are respected by other members of their child's team and will see themselves as equal and integral members.

1. Service-related outcomes (e.g., whether families received desired services, the level of difficulty in obtaining services, level of parents' influence over decisions)

2. Satisfaction outcomes (e.g., satisfaction with services, families' feelings of competence)

3. Quality of life outcomes (e.g., family participation in community activities, successful accommodations to family and communi- ty challenges)

1. Motivation outcomes (e.g., perceived self-efficacy, perceived control, hope, energy, persistence)

2. Knowledge or skill outcomes (e.g., information, problem-solving, coping skills, communication skills)

and family support programs in the context of early intervention. I have suggested that this evaluation could occur at three levels: (a) the extent to which programs meet federal and state standards for working with families, (b) the extent to which programs provide services that go beyond the required standards to pro- vide services widely acknowledged as consti-

tuting high quality, and (c) the extent to which families realize selected outcomes as a result of parent involvement and family support pro- grams.

T h e first level of evaluation is essential and states must develop strategies for assuring the provision of minimal requirements for servic- es. It will be difficult, however, to avoid issues

10 JEI, 2001, 24:1

of quality at this level, and thus it will almost be inevitable that some aspects of Level-2 evaluations (of program quality) will be con- ducted in almost every state. This work could be conducted in the context of broader efforts to determine quality in all aspects of early in- tervention, not just those aspects associated with working with families (Aytch, Cryer, Bailey, & Selz, 1999). Level-3 evaluations (documenting family outcomes) are contro- versial, but must be addressed. Without a clear statement of desired outcomes, efforts to doc- ument quality become less compelling, be- cause what we mean by quality might vary depending upon the outcome that is to be achieved.

What will be necessary to help states and local programs move toward a more system- atic evaluation of parent involvement and family support programs? At least five activ- ities seem important:

Develop Partnerships Between Parents and Professionals to Address Issues Related to Program Evaluation Families have an inherent interest in each lev- el of evaluation and would be directly affected as participants in the evaluation, as well as by the results of the evaluation. Parents need to be involved as key participants in discussions about and implementation of program evalu- ation efforts. Care will need to be taken to assure that the parents who participate in these activities reflect and can speak for the diver- sity of families currently participating in early intervention programs. In addition, parent per- spectives on issues related to evaluation meth- ods, goals, and findings should be solicited us- ing a variety of methods such as surveys, in- terviews, and focus groups.

Develop and Evaluate Assessment Instruments and Procedures Although considerable effort has been direct- ed in recent years towards measurement issues regarding quality of family services and doc- umentation of family outcomes, much work remains to be done. States are now embarking on quality assurance initiatives of varying de- grees of complexity and comprehensiveness.

Although there is something to be said for a number of groups working on the same task, some coordination of these efforts would be useful. Measurement issues such as reliability and validity will need to be addressed, with particular attention paid, however, to the unique reliability and validity issues associ- ated with assessment of family practices and outcomes (Henderson, Aydlett, & Bailey, 1993).

Develop, Implement, and Evaluate Models of Technical Assistance to Aid State and Local Programs in Developing Evaluation and Program Improvement Efforts Evaluation ideally should be conducted in the context of program improvement initiatives. It is, of course, at the local program level that parent involvement and family support activ- ities are provided. Local programs will look more favorably upon evaluation efforts if they feel that they can ultimately improve what they are doing in a positive manner. Some models for program improvement have been described (e.g., Bailey, McWilliam, & Win- ton, 1992; Olson, Murphy, & Olson, 1998; Snyder & McWilliam, 1999; Summers, McMann, & Fuger, 1997), and states have be- gun to tackle this issue from a number of per- spectives (McWilliam et al., 1996; Roberts, Innocenti, & Goetze, 1999). These models are only sporadically available, however, and a more systematic framework for evaluation and local support for improvement of practices will be necessary.

Convene a National Forum on Family Outcomes Enough conceptual work has now been done on family outcomes to warrant convening a national forum to reach some consensus on what, indeed, should be expected of early in- tervention. This forum, which should include parents, practitioners, policy makers, and re- searchers, should address the diversity of is- sues inherent in assessing family outcomes and make some recommendations to the field for a core set of outcomes that could be stud- ied more systematically.

Bailey 11

Conduct Research to Describe the National Status of Parent Involvement and Family Support Programs, and the Costs, Quality, and Outcomes of Such Programs We know very little from a national perspec- tive about the real nature and distribution of parent involvement and family support pro- grams. Efforts such as the ongoing National Early Intervention Longitudinal Study will provide some important information in this re- gard, but more work is needed to understand the variability in program models and practic- es, and factors contributing to that variability. Barriers to the full implementation of what the field considers are high quality practices need to be identified and strategies for overcoming those barriers need to be developed and tested. Ultimately research is needed in which the provision of key aspects of parent involve- ment and family support programs is linked with outcomes, to determine the extent to which our espoused practices are resulting in the outcomes that families desire.

REFERENCES

Aytch, L. S., Cryer, D., Bailey, D. B., & Selz, L. (1999). Defining and assessing quality in early intervention programs for infants and toddlers with disabilities and their families: Challenges and unresolved issues. Early Education and Development, 10, 7-23.

Bailey, D. B. (1992). Family Orientation of Com- munity and Agency Services. Chapel Hill, NC: Frank Porter Graham Child Development Cen- ter, University of North Carolina at Chapel Hill.

Bailey, D. B. (1994). Working with families of chil- dren with special needs. In M. Wolery & J. S. Wilbers (Eds.), Including children with special needs in early childhood programs (pp. 2 3 - 44). Washington, DC: National Association for the Education of Young Children.

Bailey, D. B., Buysse, V., Edmondson, R., & Smith, T. M. (1992). Creating family-centered services in early intervention: Perceptions of profession- als in four states. Exceptional Children, 58, 298-309.

Bailey, D. B., McWilliam, P. J., & Winton, P. J. (1992). Building family-centered practices in early intervention: A team-based model for change. Infants and Young Children, 5(1), 7 3 - 82.

Bailey, D. B., McWilliam, R. A., Darkes, L. A.,

Hebbeler, K., Simeonsson, R. J., Spiker, D., & Wagner, M. (1998). Family outcomes in early intervention: A framework for program evalu- ation and efficacy research. Exceptional Chil- dren, 64, 313-328.

Bailey, D. B., Simeonsson, R. J., Winton, P. J., Huntington, G. S., Comfort, M., Isbell, P., O'Donnell, K. J., & Helm, J. M. (1986). Fam- ily-focused intervention: A functional model for planning, implementing, and evaluating in- dividualized family services in early interven- tion. Journal of the Division for Early Child- hood, 10, 156-171.

Beckman, P. J. (1996). Strategies for working with families of young children with disabilities. Baltimore: Paul H. Brookes.

Brewer, E. J., McPherson, M., Magrab, P. R., & Hutchins, V. L. (1989). Family-centered, com- munity-based, coordinated care for children with special health care needs. Pediatrics, 83, 1055-1060.

Cost, Quality, and Child Outcomes Study Team (1995). Cost, quality, and child outcomes in child care centers: Technical report. Denver, CO: University of Colorado.

Dinnebeil, L. A., Hale, L. M , & Rule, S. (1996). A qualitative analysis of parents' and service coordinators' descriptions of variables that in- fluence collaborative relationships. Topics in Early Childhood Special Education, 16, 322- 347.

Dunst, C. J. (1985). Rethinking early intervention. Analysis and Intervention in Developmental Disabilities, 5, 165-201.

Dunst, C. J., Johanson, C , Trivette, C , & Hamby, C. (1991). Family-oriented early intervention policies and practices: Family-centered or not? Exceptional Children, 58, 115-126.

Dunst, C. J., Trivette, C. M., & Deal, A. G. (1988). Enabling and empowering families: Principles and guidelines for practice. Cambridge, MA: Brookline Books.

Dunst, C. J., Trivette, C. M., & Deal, A. G. (1994). Supporting and strengthening families: Meth- ods, strategies and practices (Vol 1). Cam- bridge, MA: Brookline Books.

Dunst, C. J., Trivette, C. M., & Hamby, D. W (1996). Measuring the helpgiving practices of human services program providers. Human Re- lations, 49, 815-835.

Early Childhood Research Institute on Measuring Growth and Development (1998). Family out- comes in a growth and development model (Technical Report no. 7). Minneapolis, MN: Center for Early Education and Development, University of Minnesota.

12 JEI, 200J, 24:1

Fink, A. (1995). Evaluation for education and psy- chology. Thousand Oaks, CA: Sage.

Harbin, G. L., McWilliam, R. A., & Gallagher, J. J. (2000). Services for young children with dis- abilities and their families. In J. P. Shonkoff & S. J. Meisels (Eds.), Handbook of early child- hood intervention (2nd ed., pp. 387-415). New York: Cambridge University Press.

Harms, T., Clifford, R. M., & Cryer, D. (1998). Ear- ly Childhood Environment Rating Scale—Re- vised. New York: Teachers College Press.

Henderson, L. W., Aydlett, L. A., & Bailey, D. B. (1993). Evaluating family needs surveys: Do standard measures of reliability and validity tell us what we want to know? Journal of Psycho- educational Assessment, 11, 208-219.

Mahoney, G., & Bella, J. M. (1998). An examina- tion of the effects of family-centered early in- tervention on child and family outcomes. Top- ics in Early Childhood Special Education, 18, 83-94.

Mahoney, G., Boyce, G., Fewell, R., Spiker, D., & Wheeden, C. A. (1998). The relationship of parent-child interaction to the effectiveness of early intervention services for at-risk children and children with disabilities. Topics in Early Childhood Special Education, 18, 5-17.

Mahoney, G., O'Sullivan, P., & Dennebaum, J. (1990). Maternal perceptions of early interven- tion services: A scale for assessing family-fo- cused intervention. Topics in Early Childhood Special Education, 10, 1-15.

McCollum, J. A., & Hemmeter, M. L. (1997). Par- ent-child interaction when children have dis- abilities. In M. Guralnick (Ed.), The effective- ness of early intervention (pp. 549-578). Bal- timore: Paul H. Brookes.

McWilliam, P. J. (1991). Brass Tacks: A self-rating of family-centered practices in early interven- tion (The Family Report). Chapel Hill, NC: Frank Porter Graham Child. Development Cen- ter, University of North Carolina at Chapel Hill.

McWilliam, P. J., & Winton, P. (1990). Brass Tacks: A self-rating of family centered practices in early intervention (staff versions). Chapel Hill, NC: Frank Porter Graham Child Development Center, University of North Carolina at Chapel Hill.

McWilliam, P. J., Winton, P. J., & Crais, E. R. (1996). Practical strategies for family-centered intervention. San Diego: Singular Publishing Group.

McWilliam, R. A., Tocci, L., & Harbin, G. (1998). Family-centered services: Service providers' discourse and behavior. Topics in Early Child- hood Special Education, 18, 206-221.

Murphy, D. L., Lee, I. M., Turnbull, A. P., & Tur- biville, V. (1995). The Family-Centered Pro- gram Rating Scale: An instrument for program evaluation and change. Journal of Early Inter- vention, 19, 24-42.

Office of Special Education Programs (2000). Con- tinuous Improvement Monitoring Process. Washington, D.C.: U.S. Department of Educa- tion.

Olson, J., Murphy, C. L., & Olson, P. D. (1998). Building effective successful teams: An inter- active teaming model for inservice education. Journal of Early Intervention, 21, 339-349.

Popham, W J. (1993). Educational evaluation (3rd

ed.). Boston: Allyn and Bacon. Roberts, R. N., Innocenti, M. S., & Goetze, L. D.

(1999). Emerging issues from state level eval- uations of early intervention programs. Journal of Early Intervention, 22, 152-163.

Scriven, M. (1967). The methodology of evalua- tion. In R. E. Stake (Ed.), Perspectives of cur- riculum evaluation: American Educational Re- search Association Monograph Series on Eval- uation, no. 1. Chicago: Rand McNally.

Shelton, T. L., Jeppson, E. S., & Johnson, B. H. (1987). Family-centered care for children with special health care needs. Washington, DC: Association for the Care of Children's Health. (ERIC Document Reproduction Service No. ED 288 321).

Shonkoff, J. P., Hauser-Cram, P., Krauss, M. W, & Upshur, C. C. (1992). Development of infants with disabilities and their families. Mono- graphs of the Society for Research in Child De- velopment, 57 (6, Serial No. 230).

Snyder, P., & McWilliam, P. J. (1999). Evaluating the efficacy of case method instruction: Find- ings from preservice training in family-cen- tered care. Journal of Early Intervention, 22, 114-125.

Summers, J. A., McMann, O. T, & Fuger, K. L. (1997). Critical thinking: A method to guide staff in serving families with multiple challeng- es. Topics in Early Childhood Special Educa- tion, 17, 27-52.

Thompson, L., Lobb, C , Elling, R., Herman, S., Jurkiewica, T, & Hulleza, C. (1997). Pathways to family empowerment: Effects of family-cen- tered delivery of early intervention services. Exceptional Children, 64, 99-113.

Trivette, C. M., Dunst, C. J., Boyd, K., & Hamby, D. W (1995). Family-oriented program models, helpgiving practices, and parental control ap- praisals. Exceptional Children, 62, 237-248.

Turnbull, A. P., Turbiville, V., & Turnbull, H. R. (2000). Evolution of family-professional mod-

Bailey 13

els: Collective empowerment as the model for the early 21st century. In J. P. Shonkoff & S. J. Meisels (Eds.), Handbook of early childhood intervention (2nd ed., pp. 630-650). Cambridge, MA: Cambridge University Press.

Turnbull, A. P., & Turnbull, H. R. (1997). Families, professionals, and exceptionality: A special partnership (3rd ed.). Columbus, OH: Merrill.

Walberg, H. J., & Haertel, G. D. (Eds.) (1990). The international encyclopedia of educational eval- uation. Oxford: Pergamon Press.

Worthen, B. R., Sanders, J. R., & Fitzpatrick, J. L. (1997). Program evaluation: Alternative ap- proaches and practical guidelines (2nd ed.). New York: Longman.

Address correspondence to Donald B. Bailey, Jr., Frank Porter Graham Child Development Center, CB# 8180, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina 27599-8180. Email: [email protected].

14 JEI, 2001, 24:1

mailto:[email protected]