human resource management

A pilot evaluation of the Family Caregiver Support Program

Ya-Mei Chen a,*, Susan C. Hedrick b, Heather M. Young c

a School of Nursing, University of Washington, United States b Health Services, School of Public Health, University of Washington, Research Career Scientist, VA Medical Center, United States c University of Washington, Grace Phelps Distinguished Professor and Director of Rural Health Research Development, Oregon Health and Sciences University, United States

Evaluation and Program Planning 33 (2010) 113–119

A R T I C L E I N F O

Article history:

Received 26 November 2008

Received in revised form 30 July 2009

Accepted 8 August 2009

Keywords:

Family Caregiver Support Program

Program evaluation

Caregiver

Support services

A B S T R A C T

The purposes of this study were to evaluate a federal and state-funded Family Caregiver Support

Program (FCSP) and explore what types of caregiver support service are associated with what caregiver

outcomes. Information was obtained on a sample of 164 caregivers’ use of eleven different types of

support service. Descriptive and comparative analyses were used to detect the differences between users

and nonusers of caregiver support services. Six measures included were caregiving appraisal scale,

caregiving burden, caregiving mastery, caregiving satisfaction, hour of care, and service satisfaction.

Using consulting and education services is associated with lessening of subjective burden; using

financial support services is associated with more beneficial caregiver appraisal, such as better caregiver

mastery. The findings are practical and helpful for future caregiver service and program development

and evaluation and policy making for supporting caregivers. In addition, the evaluation method

demonstrated in the study provided a simple and moderately effective method for service agencies

which would like to evaluate their family caregiver support services.

Published by Elsevier Ltd.

Contents lists available at ScienceDirect

Evaluation and Program Planning

j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a t e / e v a l p r o g p l a n

1. Introduction

An estimated 52 million Americans function as informal caregivers of ill or disabled individuals, and 23 percent (22.4 million) of U.S. households are caring for a relative or friend who is at least 50 years old (AARP, 2004; Coleman and Pandya, 2002). One fifth of all family members of seriously ill patients have to quit work or make another major life change in order to provide care, and almost one third report the loss of their entire savings (GAO, 1994). Furthermore, financial or other unmet needs may impede caregivers’ ability to function effectively, both in their own day-to- day lives and in their role as an ongoing support system for their patients (Kristjanson, Atwood, & Degner, 1995; Tringali, 1986). As a result, the need to provide support to caregivers has gradually gained societal attention, and many publicly and privately funded services have been developed to achieve this goal. The National Family Caregiver Support Program, for example, authorizes local Area Agencies on Aging (AAAs) to provide caregivers with various support services, including caregiver training, respite care, and supplemental services, among others. However, caregiver support services vary a great deal, and research findings regarding the

* Corresponding author at: Psychosocial & Community Health, Box 357263,

University of Washington, Seattle, WA 98195, United States. Tel.: +1 206 685 0819;

fax: +1 206 685 9551.

E-mail address: [email protected] (Y.-M. Chen).

0149-7189/$ – see front matter . Published by Elsevier Ltd.

doi:10.1016/j.evalprogplan.2009.08.002

effects of these services have shown mixed results. The research has also shown the need for a uniform method of evaluating caregiver support services. Feinberg and Newman (2006) studied administrators’ experiences of implementing the National Family Caregiver Support Program in all 50 states in the United States, and showed that there is still a great unevenness in services programs in different states. Because of this, they suggested that a uniform assessment and evaluation tool is necessary in order to better provide services to family caregivers.

1.1. Background of the problem

Caregiver support services most commonly provide informa- tion access, caregiver education and training, and respite and supplemental services. Research findings regarding the effects of these services have shown mixed results. Some studies, including those with rigorous designs such as randomized and controlled trials, showed caregiver support services either to have little or no impact on caregivers’ outcomes, or to be effective only for a subgroup of the caregiver population. Other studies, however, showed these services to be effective in different perspectives in supporting family caregivers (Brodaty, Green, & Koschera, 2003; Burns, Nichols, Martindale-Adams, Graney, & Lummus, 2003; Gallagher-Thompson et al., 2003; Lee & Cameron, 2004; Maas et al., 2004; Newcomer, Yordi, DuNah, Fox, & Wilkinson, 1999; Roberts et al., 1999; Toseland, Blanchard, & McCallion, 1995; Zank & Schacke, 2002). To prepare for the current study, we completed a

Table 1 Summary of 34 studies reviewed.

Intervention studied N Sig. positive

effects

No

effects

Adult day care/respite services 10 6 4

Caregiver training/counseling/support group 13 6 7

Supplemental services (i.e. meal delivery

service, transportation, homemaker,

or home aide care)

3 2 1

Coordinated program (i.e. all-inclusive

care for elderly, which contains more

than one of the three categories

described above)

8 3 5

References reviewed: (Berry et al., 1991; Brodaty et al., 2003; Burns et al., 2003;

Chang, 1999; Coon et al., 2003; Cox, 1997; Edelman & Hughes, 1990; Fox et al.,

2000; Gallagher-Thompson et al., 2003; Gaugler et al., 2003a,b; Gitlin et al., 2003;

Gottlieb & Johnson, 1993; Hepburn et al., 2001; Kemper, 1988; Kosloski &

Montgomery, 1993; Kosloski & Montgomery, 1994; Krout, 1995; Lawton et al.,

1989; Maas et al., 2004; Miller et al., 1999; Montgomery & Borgatta, 1989;

Montgomery et al., 1985; Newcomer et al., 1999; Newcomer et al., 1999;

Quayhagen et al., 2000; Roberts et al., 1999; Toseland et al., 1995; Toseland et al.,

2004; Toseland et al., 2001; Tourigny et al., 2004; Yordi et al., 1997; Zank & Schacke,

2002; Zarit et al., 1998).

Y.-M. Chen et al. / Evaluation and Program Planning 33 (2010) 113–119114

systematic review of 34 previous studies of caregiver interven- tions. The studies reviewed were conducted between 1984 and 2004 and used either randomized controlled trials or quasi- experimental controlled designs. Each of these studies investi- gated one of four categories of services as shown in Table 1. Our review of these studies found that only a little more than half of them showed services to result in any benefit for family caregivers.

Our review also indicated that different services might be associated with different caregiver outcomes. For example, research studies that found a positive effect of caregiver education and training showed these services to increase caregiver effec- tiveness in solving problems, improve caregiver feelings of competence, and reduce caregivers’ subjective and objective burdens (Brodaty et al., 2003; Burns et al., 2001, 2003; Chang, 1999; Coon, Thompson, Steffen, Sorocco, & Gallagher-Thompson, 2003; Gallagher-Thompson et al., 2003; Gitlin et al., 2003; Hepburn, Tornatore, Center, & Ostwald, 2001; Montoro-Rodriguez, Kosloski, & Montgomery, 2003; Quayhagen et al., 2000; Toseland, McCallion, Smith, & Banks, 2004; Toseland et al., 2001; Weuve, Boult, & Morishita, 2000). In regard to the effect of respite and supplemental services, on the other hand, studies that found a positive effect showed these services to decrease caregiver stress; decrease feelings of role overload, depression, burden, and time commitment; and improve overall psychological well-being (Berry, Zarit, & Rabatin, 1991; Cox, 1997; Gaugler et al., 2003a,b; Gottlieb & Johnson, 1993; Krout, 1995; Montgomery & Borgatta, 1989; Okamoto, Murashima, & Saito, 1998; Zarit, Stephens, Townsend, & Greene, 1998).

These results demonstrate the difficulty of evaluating caregiver support services. Most evaluation tools used in previous studies were likely to assess one particular aspect of the services’ outcomes, such as caregiver burden, more than other outcomes, such as caregiver mastery. These methods of evalua- tion may result in nonsignificant findings where the tool chosen does not focus on the appropriate caregiver outcomes. Therefore, developing a uniform evaluation method that is broad enough to cover multiple facets of caregiver outcomes is a challenging but important task. In addition, understanding whether different services relate to different caregiver outcomes, and which services might best support particular caregiver outcomes, could be very helpful for choosing or developing evaluation methods and tools.

1.2. Purpose of study

The purposes of this study were twofold: the first purpose was to test a simple evaluation method that would be easy for service agencies to adopt and that could be adopted on a wide scale. The second purpose was to determine whether different types of caregiver support services are associated with different caregiver outcomes. We collaborated with Aging and Disability Services (ADS) in Seattle, the local AAA, to achieve our purpose through a pilot study that evaluated a federal- and state-funded project, the Family Caregiver Support Program (FCSP), in King County in Washington State. In this region, the FCSP provides various caregiving support services including adult day care, in-home respite, information services, and financial assistance to the caregiver (ADS, 2003).

2. Methods

2.1. Design, setting, and participants

This study was a descriptive and one-time survey of caregivers living in King County who were reported as having received services from local service agencies of ADS’. Four local agencies agreed to send out an invitation letter and questionnaire to all caregivers who had received FCSP-funded services between 2001 and 2003. The University of Washington Human Subjects Division approved this study.

2.2. Questionnaire development

The researchers assisted the FCSP team in selecting tools appropriate for evaluating the FCSP. Several tools were selected for review, including the ‘‘Caregiver Appraisal Scale’’ (Lawton & Brody, 1969), the ‘‘Subjective and Objective Burden Scale’’ (Montgomery, Gonyea, & Hooyman, 1985), and the ‘‘Mastery Scale’’ (Pearlin & Schooler, 1978). To ensure the usefulness of the evaluation tool, and with the intention of selecting a tool on the basis of both successful scientific evidence and hands-on experience, the team invited the four local caregiver service agencies who agreed to send out an invitation letter and questionnaire to all caregivers to contribute their expertise. After thorough discussion, the team selected the ‘‘Caregiver Appraisal Scale’’ (CAS) developed by M.P. Lawton and E.M. Brody (1965) for the appropriateness of its language and its coverage of the broad scope of relevant caregiver experiences (Vitaliano, Young, & Russo, 1991). Other tools reviewed target only a single facet of caregiving experiences, and single-perspective tools were less adequate for the purposes of the current study. The FCSP is a program with multiple components, which include various types of services, and it is likely that caregivers’ experiences are multifaceted as well. The agencies consulted further suggested reducing the length of the CAS questionnaire in order to not overly stress caregivers. Consequently, three subscales were chosen for use in the study: ‘‘Subjective Burden’’ (e.g., ‘‘Your health has suffered because of the care you must give to care receiver’’ or ‘‘Very tired as a result of caring for care receiver’’), ‘‘Caregiving Mastery’’ (e.g., ‘‘I can fit in most of the things I need to do in spite of the time taken by caring for care

receiver’’), and ‘‘Caregiving Satisfaction’’ (e.g., ‘‘Helping care receiver has made you feel closer to him/her’’ or ‘‘Care receiver shows real appreciation of what you do for her/him’’). The length of the revised Caregiver Appraisal Scale (hereafter referred to as ADS-CAS) was thereby reduced from 47 to 34 items, with 13 items, 12 items, and 9 items each for the ‘‘Subjective Burden,’’ ‘‘Caregiving Mastery,’’ and ‘‘Caregiving Satisfaction’’ subscales, respectively.

The two subscales of the CAS that were not used in this study are ‘‘Impact of Caregiving’’ and ‘‘Cognitive Reappraisal.’’ The former

Y.-M. Chen et al. / Evaluation and Program Planning 33 (2010) 113–119 115

was excluded because of its high correlation with the ‘‘Subjective Burden’’ subscale (Deeken, Taylor, Mangan, Yabroff, & Ingham, 2003; Lawton, Kleban, Moss, Rovine, & Glicksman, 1989). The latter was excluded as not reflecting the purposes of the FCSP.

Participants responded to each item on the ADS-CAS based on a 5-point scale, from ‘‘Rarely or never (1)’’ to ‘‘Most of the time (5).’’ Higher total and subscale scores represent more positive caregiv- ing appraisals, except for the Subjective Burden subscale, where higher scores indicate caregivers perceived higher subjective burden.

In addition to the three subscales, the survey gathered information on caregivers’ age, gender, and relation to care receivers; the type of care provided; the hours of care (including hands-on and supervisory care) provided in the week prior to the survey; and the types of services that the caregivers received. Agencies reported providing a list of services, including (1) information about services, (2) assistance in accessing services, (3) caregiver counseling, (4) caregiver education and training, (5) financial assistance, (6) respite services/adult day care, (7) help with housework, (8) delivered meals, (9) transportation, (10) cash support. Caregivers were asked whether they had received each of these services. As a result of discussions with the FCSP team and with local service agencies, we further grouped these 10 services into three categories based on the nature of the services: (1) counseling and education services, (2) respite and supplemental services, and (3) financial support services. The counseling and education services category included information about services, assistance in accessing services, caregiver counseling, and care- giver education and training. The respite services category included respite services/adult day care, help with housework, delivered meals, and transportation. The financial support services category included financial assistance and cash support for caregiving. A general service satisfaction question was also included in the ADS-CAS, with a 4-point scale response: ‘‘Poor (1),’’ ‘‘Fair (2),’’ ‘‘Good (3),’’ or ‘‘Excellent (4).’’

2.3. Data collection methods

Each agency sent each of their clients a cover letter, a questionnaire, and a postage-paid return envelope addressed to ADS. To protect the clients’ confidentiality, the questionnaires were anonymous, and no follow-up occurred. A total of 866 survey packets were sent out, and 177 questionnaires (20.4%) were returned.

2.4. Quantitative data analysis

Data analyses were conducted using the Statistical Package for the Social Sciences (SPSS-PC) version 12.0. Prior to analyzing the data, all items were examined to assess the accuracy of variable calculations and missing values. If variables were missing at a rate larger than 5%, multiple imputation was applied (Rubin, 1977; Schafer, 1997, 1999, 2000). Cronbach’s alpha was used to evaluate the internal consistency of each subscale on the ADS-CAS.

Descriptive analyses were used to depict the characteristics of caregivers and the services they received. Two steps were included in the evaluation method. First, we examined the gaps between the types of care that the caregivers provided and the types of services that the caregivers received. Second, MANCOVA were used to compare caregivers’ appraisals in the following categories: (1) those who reported using any of the 10 services versus those who did not; (2) caregivers who used one particular service versus those who did not report using that particular service (For example, in comparing caregivers who had received financial services with caregivers who reported they had not received

financial services, ‘‘users’’ may have received other services as well as

financial services; ‘‘nonusers’’ of financial services may have received other services or may have reported not receiving any services); and (3) those who used only one out of the three categories of services versus those who did not use that particular service category. (For example, caregivers who had received services in the financial services category only, and no services from other categories, were

compared with caregivers who reported they had not received any

services from the financial services category. ‘‘Nonusers’’ in a category may have received services in other categories, or may have reported

not receiving any services at all.) Clients’ age, gender, and the number of care activities they provided were controlled as covariates. The outcome measures were: (1) the item mean of ADS-CAS, (2–4) the item means of each of the three subscales of ADS-CAS, (5) the total hours the caregiver spent on caregiving during the previous week, and (6) the caregiver’s satisfaction with services received.

2.5. Text summary

Caregivers’ text feedback was summarized and analyzed for common themes using content analysis.

3. Results

The response rate was 20.4%. Five questionnaires were returned blank, and two were returned with only text information. Furthermore, 6 caregivers stated that they were not providing any care at this point. As a result, only 164 questionnaires were entered for quantitative data analysis, an 18.9% usable response rate.

3.1. Psychometric properties of ADS-CAS

Most items in ADS-CAS were missing cases at a rate between 9% and 13%. Therefore, multiple imputation was used (Rubin, 1977; Schafer, 1997). After reverse coding and multiple imputation, Cronbach’s alphas for ADS-CAS were 0.90. The power to detect statistically significant differences in ADS-CAS between caregivers who reported having received at least one of the services and those who reported not receiving any services was .78.

3.2. Description of care provided by caregivers and of services

provided to caregivers

About 74% of caregivers were female, 17% were male, and 9% did not specify their gender. Their ages ranged from less than 20 to more than 81 years old, with an average age of 57. The majority stated that they were caring either for a spouse/partner (48.8%) or for parents (41.2%).

Caregivers provided from one to nine types of care to their care receivers, with an average of 6.8 (SD = 2.24). About one third provided all nine kinds of caregiving activities listed in the questionnaire, including personal care, safety/supervision, house- keeping and laundry, meal preparation, medication monitoring, transportation, shopping, financial management, and standby help. The most common type of care provided was transportation (83.5%). Caregivers reported receiving a range of zero to seven ADS services, with an average of 1.91 (SD = 1.54). The service most commonly used was information about services (52.4%). A surprising percentage (14.6%) stated that they had not received any services, even though all the caregivers surveyed had been identified by agencies as service recipients. These caregivers were labeled ‘nonusers’ and used as the comparison group in the first analysis. However, they cannot fully represent the real nonusers in the U.S. caregiver population. We discuss this issue further in the discussion section.

Table 2 MANCOVA results: marginal mean differences between users of any services and nonusers, users of a particular service and nonusers of that particular service, and users of a

single service category and nonusers of that service category (N = 164).

ADS-caregiver appraisal scale

CAS SB CM CS HOUR SS

Individual services

Use or non-use of services �3.89 �1.58 �3.00** �3.46* 21.65 �0.03 Services information �1.39 �0.38 �2.02* �0.11 �11.27 �0.04 Assistance in accessing services 1.91 �1.53 �1.28 1.66 �11.72 0.28* Caregiver counseling 1.48 �1.09 0.08 0.30 �12.03 0.32*

Caregiver training or education 2.36 �2.28 0.82 �0.74 �14.08 0.09 Financial assistance �10.82 3.57 �4.67* �2.49 �23.15 0.35 Respite services �7.69 2.02 �2.97 �2.70* 45.91*** 0.16 Help with housework �3.05 �0.40 �4.45* �1.0 4.46 0.17 Delivered meals �0.15 0.48 �0.25 �0.58 3.63 0.07 Transportation �5.81 1.06 �1.42 �3.33* �23.76 �0.15 Cash to support caregiving �2.40 3.70 0.02 1.28 �20.49 0.51*

Service catagories

Counseling and education 2.81 �2.80 �0.33 0.33 �15.39 �0.11 Respite �3.24 1.23 0.27 �2.28 39.28*** �0.05 Finance 28.17 �11.66 12.57* 6.63 7.38 0.06

[Bold] = 0.05 < p < 0.070; (users’ scores – nonusers’ scores). Note: CAS, CAS score; SB, subjective burden score; CM, caregiver mastery score; CS, caregiver satisfaction score;

HOUR, hours of care; SS, service satisfaction. Controlled for caregivers’ age, gender, and number of caregiving activities that they have provided as covariates. * p < 0.05 [bold]. ** p < 0.01 [bold]. *** p < 0.001 [bold]; (users’ scores – nonusers’ scores).

Table 3 Text summary.

Category Keywords Frequency

Exhausted caregivers Problem 4

Hard 3

Frustrate/stress/tire 6

My health 1

Collapse 1

Appreciation Thank/appreciate/grateful 16

Wonderful 4

Help needed Available 9

Need/need. . .help 32

Aware 1

Financial 6

[Staff] change 2

Y.-M. Chen et al. / Evaluation and Program Planning 33 (2010) 113–119116

3.2.1. Gaps between care provided by caregivers and services

provided to caregivers

Potential gaps were found between care provided and services received. The most common types of care provided by the caregivers were transportation, financial management, and medication monitoring. However, the most common services that these caregivers claimed to have received were services informa- tion, respite care, and assistance in accessing services. On the one hand, this indicates a good availability of the former three services. On the other hand, the services provided may not match what caregivers need most, such as help with transportation. For example, only 9.8% of caregivers received transportation services, while 83.5% of caregivers provided such services to their relatives. Another potential gap worth noting is the high rate of medication management assistance (79.9%) provided by caregivers versus the low rate of training and education received by caregivers (14%).

3.3. Mean score differences in outcome measures (ADS-CAS) between

users of any services and nonusers, users of a particular service and

nonusers of that particular service, and users of a single service

category and nonusers of that service category

After controlling for caregivers’ age, gender, and the number of care activities provided, caregivers who received assistance in accessing services, who used caregiver counseling services, or who obtained cash to support caregiving showed significantly higher satisfaction toward the services that they received (p < 0.05). Caregivers who received information about services, financial assistance, and help with housework reported lower caregiver mastery than did caregivers not using such services (p < 0.05). Analysis of service categories revealed additional relationships. The caregivers who received only financial support services showed significantly higher mastery (p < 0.05) than did those who did not use such services. The caregivers who received only respite services spent an average of 39.28 more hours caring for care recipients in the week prior to our survey (p < 0.01) than did those not using respite services. Analysis also showed that caregivers who received only counseling and education services perceived less subjective caregiving burden (p = 0.056) than did others, and that caregivers who received only financial support services showed better overall caregiver appraisal (p = 0.058).

Analysis both of individual services and of the three service categories revealed similar findings; therefore, a discussion of individual services will not be given here. Table 2 presents full results for both individual services and the three service categories. The results of the three analyses are presented in Table 2 as mean score differences between users of services and nonusers, users of a particular service and nonusers of that particular service, and users of a single service category and nonusers of that service category.

3.4. Content analysis of open-ended comments

Seventy-two caregivers entered textual comments on their questionnaires. Content analysis yielded 12 keywords and three themes: (1) exhausted caregivers (keywords: problem, hard, frustrate/stress/tire, my health, collapse), (2) appreciation of services received (keywords: thank/appreciate/grateful, wonder- ful), and (3) services needed (keywords: availability, need/ need. . .help, aware, financial, [staff]. . .change). Many caregivers reported fatigue due to their caregiver responsibilities. There were services that they indicated should be developed or improved in order to provide better support. The need for integrative services was frequently cited: ‘‘I was frustrated though that there is no one person and agency available to counsel or ‘pull the picture’ together.’’ Caregivers also reported difficulties in assessing their needs and in determining which services were available to meet these needs:

Y.-M. Chen et al. / Evaluation and Program Planning 33 (2010) 113–119 117

‘‘Often services tell you what they don’t do rather than ask what you need help with in caring for the person. By the time you sort through it

all you find you don’t/can’t use services.’’ Some caregivers reported that it took too long to search for and then wait for services and to solve their immediate problems: ‘‘I wish I had taken this class before I got so worn out.’’ ‘‘I wish I knew these services were available before my mom passed away.’’ ‘‘If we can get transportation I expect and hope this will help [to solve the immediate conflict between this caregiver and his/her spouse].’’ (Please see Table 3 for the frequency of each key word in caregivers’ text feedback).

4. Discussion

Findings from the current study can contribute not only to the growing body of research in the area of caregiving support but also to the future development of caregiving support services in the King County region.

4.1. Different caregiver appraisals between service users and nonusers

The use of different services was associated with different caregiver appraisals, and these findings add a great deal to caregiving research literature. Using counseling and education services, such as caregiver counseling services, was associated with a lessening of subjective burden; and using financial support services, such as cash support, was associated with a more beneficial caregiver appraisal. Although in this study using respite and supplemental services was not associated with any beneficial outcome from the caregivers’ point of view, it is still important to provide such support services. Details about each service category are discussed below.

4.2. Counseling and education services

Research findings have shown that counseling and education services are effective in helping caregivers to deal with their own psychological needs and in improving caregivers’ relationships with care recipients (Brodaty et al., 2003; Burns et al., 2003; Coon et al., 2003). The findings in the current study support this literature. By using counseling and education services, caregivers reduced their subjective burden. Our participants provided numerous remarks explaining that having someone to talk to or attending a support group or counseling class can prevent caregiver ‘‘burn out,’’ and they said that because of these services they saved time that would otherwise have been spent dealing with their negative emotions. However, it is also important to point out the possibility that caregivers who already perceive fewer burdens would be more likely to use this type of service. Those caring for someone with more severe disabilities and those who have fewer sources of caregiving help may perceive a higher burden and therefore have less energy to use these services (Markle-Reid & Browne, 2001; Toseland, McCallion, Gerber, & Banks, 2002). It is important to consider whether or not counseling and education services are more useful for caregivers with light care loads. A full examination of this topic will require future studies with more rigorous methodologies (such as randomized controlled trials or quasi-experimental designs).

4.3. Financial services

Compared to the other two categories of service, using financial support services was associated with more positive caregiving appraisals. These services provide a flexible pool of funds to Medicaid-eligible persons to purchase goods or services for family caregivers. Since, according to one study, one third of family caregivers reported the loss of all of their family savings (GAO, 1994), providing financial services is likely to give caregivers the

opportunity to focus on their caregiving activities and to develop higher confidence and satisfaction. However, we should not ignore another possible explanation—that caregivers who were able to gain access to these funds were competent users of the system who already had higher caregiving mastery and caregiving appraisal. Providing financial support to caregivers is a relatively new service developed in the last 15 years (Doty, Jackson, & Crown, 1998). Only a limited number of research studies have examined the effect of financial support services for caregivers (Eckert, Morgan, & Swamy, 2004; Mahoney, Simon-Rusinowitz, Loughlin, Desmond, & Squil- lace, 2004). The findings of this study encourage further investigation of the cost-effectiveness of providing financial support services to caregivers.

4.4. Respite and supplemental services

In contrast to findings in previous literature (Cox, 1997; Gaugler et al., 2003a,b; Krout, 1995; Zarit et al., 1998), this study found that caregivers using respite and supplemental services spent more hours on caregiving than nonusers and they did not show any positive caregiver appraisals. Although caregivers in the current study who used this type of service did not report any beneficial outcomes, it is important to recognize that this group of caregivers might be under a great deal of stress due to their care responsibilities and that they might still have a significant need for such support. Further analysis showed that older caregivers were the group who most used respite services and who most often requested help with housework; this group also spent more time on caregiving. This group of caregivers was more likely to be made up of spouses than of children, and they may also have more health problems of their own. Thus, this group might be more likely to be on the edge of giving up caregiving out of exhaustion. The services that they received apparently did not meet their needs. It is crucial to learn more about the needs of this group of caregivers and to modify these services to meet their needs.

4.5. Services that should be developed

Helping caregivers to ‘‘pull the picture together’’ should be the first task for case managers and service providers when first contacting caregivers. Caregivers are already exhausted from their caregiving tasks, and it is an added burden for them to try to find support from different resources. It will be crucial to develop a single window that could both provide all the information that caregivers need and help them access services in a more efficient manner. This would assure that available services are used by those who need them. Service availability is an important issue, and it requires more attention from providers and policy makers.

Several potential gaps were noted between care activities and services received by caregivers, such as in transportation and medication management. These potential gaps may either indicate low service availability or accessibility, which needs improvement, or may simply indicate that caregivers were confident in their ability to provide such care activities and had no need for additional support services. Given that transportation was the most common type of care activity provided by caregivers in the current study, and was commonly mentioned in the text feedback from our caregivers, it would likely be one of the services that caregivers would use if it were more available to them. The results showed that caregivers need access to transportation services and expect that getting such services would solve their current problems. Furthermore, in a recent national study, the rate of transportation services used by caregivers was almost twice the rate reported in the current study (9.8% in the current study vs. 18% in the AARP study) (AARP, 2004). This suggests that there is a need to make this service more available to caregivers in King County.

Y.-M. Chen et al. / Evaluation and Program Planning 33 (2010) 113–119118

There is also a potential gap between the high rate of medication management assistance and the low rate of training services on this subject. The percentage of caregivers in the current study who provided medication monitoring was much higher than the percentage of caregivers in the national study who did so (80% vs. 41%) (AARP, 2004). This strongly suggests the importance of providing more medication management education programs for caregivers in this region. The low rate of caregiver education and training programs in this region indicates a gap needs to be filled. For future study on caregivers living in Seattle/King County, it might be important to investigate their knowledge about the medications they give to their care receivers and what kind of support they need to help them perform this care activity better.

Another potential gap worth noting is that 14.6% of caregivers stated that they did not receive any services. All the caregivers surveyed were listed by the agencies as having received some form of services. It is intriguing that this group either did not remember receiving or did not believe they had received services. It could be that the amount of services received was not substantial enough for caregivers to note, or that the services provided were not what these caregivers were looking for. Both possibilities indicated inadequate services on this topic and required research study to further investigate. Also, further study of the differences between this group of caregivers, who likely received some services they did not remember or report, and real nonusers, who are in need but do not receive any services, will be important.

5. Limitations

There are several limiting methodological issues in this study. The cross-sectional design made it impossible to draw causal inferences. The long and variable time between when caretakers received services and when they responded to the ADS-CAS was a threat to validity. Adding a variable to assess the time between service use and survey response is recommended for future studies. Moreover, past research has found that caregivers’ perceptions of distress may be influenced by different factors at different stages of their caregiving (Vitaliano et al., 2002). Therefore, a longitudinal follow-up would help to determine the optimal time to provide caregivers with certain kinds of services, and this information would be valuable for making future policy.

Another limitation of this study was the low response rate of 20.4%. We have explored potential reasons for the low response rate. The first challenge that might have contributed to the low response rate is the fact that many caregivers do not self-identify with the term ‘‘caregiver.’’ That is probably why five questionnaires were returned blank. This has been a recurring theme and a challenge for the implementation and evaluation of family caregiver supports in the United States (Feinberg & Newman, 2006). Furthermore, the low response rate may well have to do with the substantial length of the study’s questionnaire. Jepson, Asch, Hershey, & Ubel (2005) studied the correlation between response rate and length of questionnaires and suggested that questionnaires above a threshold of 1000 words have lower response rates. Our survey questionnaire was over 5000 words, even after we removed two subscales from the CAS.

Detailed information about care recipients was not collected for the same reasons stated above. This may limit the study finding to be generated to the caregiver population. We believe that the characteristics of caregivers in the current study may be close to the general caregiver population; the current study’s demographics show similar composition of age, gender, and number of services provided and received compared with caregiver demographics in the National Family Caregiver Study (AARP, 2004). There may well be differences in other variables, of course. Increasing the response rate in future work will be important and can be addressed by further decreasing the length of the questionnaire. Other methods

that might increase the response rate in future studies include offering incentive payments, performing a follow-up survey, or providing a token for increasing response rates.

6. Lessons learned

6.1. Lesson learned for health care professionals

Findings from this study provide information for community service providers, such as community nurses, case managers, and social workers, to better understand the relationships between caregivers’ service use and caregiver appraisals. Sometimes caregiving responsibilities begin without any warning, and care- givers have no time to prepare themselves before assuming these responsibilities. They may not know what services are available or what services could be the most helpful. Knowledge generated from the current study can help case managers and service providers to help caregivers and care recipients anticipate and prioritize their needs, and to better support caregivers with the services they need most. For example, if a family caregiver expresses great subjective burden, our study findings suggest that case managers should think about offering counseling and education services first.

6.2. Lesson learned for area agencies of aging in the United States

Our experience of evaluating the FCSP program at the county level will be beneficial for other AAAs, particularly with respect to our collaboration experience with local service agencies. This study helps to address Feinberg and Newman’s (2006) call for a uniform assessment tool that can help us to understand and redress the unevenness in current caregiver service programs. The tool that we developed as a result of this study has great potential to become a standard tool for other states or AAAs to use. Our report of the gaps between the care provided and the services received by caregivers in King County is another important method of looking at service adequacy in a region, and this also could be easily adopted in other areas.

6.3. Lesson learned for future questionnaire development

The evaluation of individual services revealed similar findings to the evaluation of the three service categories. Therefore, we recommend listing three service categories in future question- naires instead of listing all of the detailed services provided by agencies. Listing all the individual services not only increases the time required to complete the questionnaire, but might also unnecessarily confuse caregivers, since questionnaires that list individual services require the caregiver to be able to identify what particular service(s) they received.

7. Conclusion

This pilot study was designed to evaluate the FCSP, but it also provided valuable information about the effect of caregiver support services as well as an understanding of what types of services might be associated with particular caregiver outcomes. These findings are useful to community care professionals and are also of practical value to program planners, policy makers, and formal care providers. The study’s findings can also serve as a basis for more rigorous future evaluations of caregiver support services.

Acknowledgements

The authors greatly appreciate the support and advice of Rosemary Cunningham, Margaret Casey, and all of the team members on the Family Caregiver Support Program at Aging and

Y.-M. Chen et al. / Evaluation and Program Planning 33 (2010) 113–119 119

Disability Services. The authors would also like to extend their gratitude to Senior Services, the Evergreen Healthcare-Geriatric Regional Assessment Team, the Northshore Senior Center, and the Kin On Community Caregiver Network-Caregiver Support. Their gracious help made this study possible.

References

AARP. (2004). Caregiving in the U.S.: National Alliance for Caregiving and AARP. ADS. (2003). Cash and Counseling Pilot Project. Seattle: Aging and Disability Services. Berry, G. L., Zarit, S. H., & Rabatin, V. X. (1991). Caregiver activity on respite and nonrespite

days: A comparison of two service approaches. Gerontologist, 31(6), 830–835. Brodaty, H., Green, A., & Koschera, A. (2003). Meta-analysis of psychosocial interven-

tions for caregivers of people with dementia. Journal of the American Geriatrics Society, 51(5), 657–664.

Burns, L. R., Walston, S. L., Alexander, J. A., Zuckerman, H. S., Andersen, R. M., Torrens, P. R., et al. (2001). Just how integrated are integrated delivery systems? Results from a national survey. Health Care Management Review, 26(1), 20–39.

Burns, R., Nichols, L. O., Martindale-Adams, J., Graney, M. J., & Lummus, A. (2003). Primary care interventions for dementia caregivers: 2-year outcomes from the REACH study. Gerontologist, 43(4), 547–555.

Chang, B. L. (1999). Cognitive-behavioral intervention for homebound caregivers of persons with dementia. Nursing Research, 48(3), 173–182.

Coleman, B. P., & Pandya, S. M. (2002). Family caregiving and long-term care. Washing- ton, DC: Public Policy Institute; AARP.

Coon, D. W., Thompson, L., Steffen, A., Sorocco, K., & Gallagher-Thompson, D. (2003). Anger and depression management: Psychoeducational skill training interventions for women caregivers of a relative with dementia. Gerontologist, 43(5), 678–689.

Cox, C. (1997). Findings from a statewide program of respite care: A comparison of service users, stoppers, and nonusers. Gerontologist, 37(4), 511–517.

Deeken, J. F., Taylor, K. L., Mangan, P., Yabroff, K. R., & Ingham, J. M. (2003). Care for the caregivers: A review of self-report instruments developed to measure the burden, needs, and quality of life of informal caregivers. Journal of Pain and Symptom Management, 26(4), 922–953.

Doty, P., Jackson, M. E., & Crown, W. (1998). The impact of female caregivers’ employment status on patterns of formal and informal eldercare. Gerontologist, 38(3), 331–341.

Eckert, J. K., Morgan, L. A., & Swamy, N. (2004). Preferences for receipt of care among community-dwelling adults. Journal of Aging & Social Policy, 16(2), 49–65.

Edelman, P., & Hughes, S. (1990). The impact of community care on provision of informal care to homebound elderly persons. Journal of Gerontology, 45(2), S74–S84.

Feinberg, L. F., & Newman, S. L. (2006). Preliminary experiences of the states in implementing the National Family Caregiver Support Program: A 50-state study. Journal of Aging & Social Policy, 18(3–4), 95–113.

Gallagher-Thompson, D., Coon, D. W., Solano, N., Ambler, C., Rabinowitz, Y., & Thomp- son, L. W. (2003). Change in indices of distress among Latino and Anglo female caregivers of elderly relatives with dementia: Site-specific results from the REACH national collaborative study. Gerontologist, 43(4), 580–591.

GAO. (1994). Long Term Care Population. Washington, D.C.: United States General Accounting Office.

Gaugler, J. E., Jarrott, S. E., Zarit, S. H., Stephens, M. A., Townsend, A., & Greene, R. (2003a). Adult day service use and reductions in caregiving hours: Effects on stress and psychological well-being for dementia caregivers. International Journal of Geriatric Psychiatry, 18(1), 55–62.

Gaugler, J. E., Jarrott, S. E., Zarit, S. H., Stephens, M. A., Townsend, A., & Greene, R. (2003b). Respite for dementia caregivers: The effects of adult day service use on caregiving hours and care demands. International Psychogeriatrics, 15(1), 37–58.

Gitlin, L. N., Winter, L., Corcoran, M., Dennis, M. P., Schinfeld, S., & Hauck, W. W. (2003). Effects of the home environmental skill-building program on the caregiver-care recipient dyad: 6-month outcomes from the Philadelphia REACH Initiative. Ger- ontologist, 43(4), 532–546.

Gottlieb, B., & Johnson, J. (1993). Impact of Day Care Programs on Family Caregivers of Persons with Dementia. Guelph, Ontario: Gerontology Research Centre, University of Guelph.

Hepburn, K. W., Tornatore, J., Center, B., & Ostwald, S. W. (2001). Dementia family caregiver training: Affecting beliefs about caregiving and caregiver outcomes. Journal of the American Geriatrics Society, 49(4), 450–457.

Jepson, C., Asch, D. A., Hershey, J. C., & Ubel, P. A. (2005). In a mailed physician survey, questionnaire length had a threshold effect on response rate. Journal of Clinical Epidemiology, 58(1), 103–105.

Kosloski, K., & Montgomery, R. J. V. (1993). The effects of respite on caregivers of Alzheimer’s patients: One year evaluation of the Michigan model respite programs. Journal of Applied Gerontology, 12(1), 4–7.

Kosloski, K., & Montgomery, R. J. (1994). Investigating patterns of service use by families providing care for dependent elders. J Aging Health, 6(1), 17–37.

Kristjanson, L. J., Atwood, J., & Degner, L. F. (1995). Validity and reliability of the family inventory of needs (FIN): Measuring the care needs of families of advanced cancer patients. Journal of Nursing Measurement, 3(2), 109–126.

Krout, J. A. (1995). Senior centers and services for the frail elderly. Journal of Aging & Social Policy, 7(2), 59–76.

Lawton, M. P., & Brody, E. M. (1969). Assessment of older people; self-maintaining and instrumental activities of daily living. The Gerontologist, 9, 179–186.

Lawton, M. P., Kleban, M. H., Moss, M., Rovine, M., & Glicksman, A. (1989). Measuring caregiving appraisal. Journal of Gerontology, 44(3), P61–71.

Lee, H., & Cameron, M. (2004). Respite care for people with dementia and their carers. Cochrane Database of Systematic Reviews, 2, CD004396.

Maas, M. L., Reed, D., Park, M., Specht, J. P., Schutte, D., Kelley, L. S., et al. (2004). Outcomes of family involvement in care intervention for caregivers of individuals with dementia. Nursing Research, 53(2), 76–86.

Mahoney, K. J., Simon-Rusinowitz, L., Loughlin, D. M., Desmond, S. M., & Squillace, M. R. (2004). Determining personal care consumers’ preferences for a consumer-direc- ted cash and counseling option: Survey results from Arkansas, Florida, New Jersey, and New York elders and adults with physical disabilities. Health Services Research, 39(3), 643–664.

Markle-Reid, M., & Browne, G. (2001). Explaining the use and non-use of community- based long-term care services by caregivers of persons with dementia. Journal of Evaluation in Clinical Practice, 7(3), 271–287.

Montgomery, R. J., & Borgatta, E. F. (1989). The effects of alternative support strategies on family caregiving. Gerontologist, 29(4), 457–464.

Montgomery, R. J. V., Gonyea, J. G., & Hooyman, N. R. (1985). Caregiving and the experience of subjective and objective. Family Relations, 34, 19–26.

Montoro-Rodriguez, J., Kosloski, K., & Montgomery, R. J. (2003). Evaluating a practice- oriented service model to increase the use of respite services among minorities and rural caregivers. Gerontologist, 43(6), 916–924.

Newcomer, R., Yordi, C., DuNah, R., Fox, P., & Wilkinson, A. (1999). Effects of the Medicare Alzheimer’s Disease Demonstration on caregiver burden and depression. Health Services Research, 34(3), 669–689.

Okamoto, M., Murashima, S., & Saito, E. (1998). Effectiveness of day care service for elderly patients with dementia and their caregivers as observed by comparison of days with and without day care services. Nippon Koshu Eisei Zasshi, 45(12), 1152–1161.

Pearlin, L. I., & Schooler, C. (1978). The structure of coping. Journal of Health and Social Behavior, 19(1), 2–21.

Quayhagen, M. P., Quayhagen, M., Corbeil, R. R., Hendrix, R. C., Jackson, J. E., Snyder, L., et al. (2000). Coping with dementia: Evaluation of four nonpharmacologic inter- ventions. International Psychogeriatrics, 12(2), 249–265.

Roberts, J., Browne, G., Milne, C., Spooner, L., Gafni, A., Drummond-Young, M., et al. (1999). Problem-solving counseling for caregivers of the cognitively impaired: Effective for whom? Nursing Research, 48(3), 162–172.

Rubin, D. B. (1977). Formalizing the subjective notions about the effect of nonrespondents in sample surveys. Journal of the American Statistical Association, 72(35.), 538–543.

Schafer, J. L. (1997). Analysis of incomplete multivariate data. London: Chapman & Hall. Schafer, J. L. (1999). Multiple imputation: A primer. Statistical Methods in Medical

Research, 8, 3–15. Schafer, J. L. (2000). Software for multiple imputation, 2000. Retrieved May 5, 2002, from

http://www.stat.psu.edu/�jls/misoftwa.html. Toseland, R. W., Blanchard, C. G., & McCallion, P. (1995). A problem solving intervention

for caregivers of cancer patients. Social Science & Medicine, 40(4), 517–528. Toseland, R. W., McCallion, P., Gerber, T., & Banks, S. (2002). Predictors of health and

human services use by persons with dementia and their family caregivers. Social Science & Medicine, 55(7), 1255–1266.

Toseland, R. W., McCallion, P., Smith, T., & Banks, S. (2004). Supporting caregivers of frail older adults in an HMO setting. The American Journal of Orthopsychiatry, 74(3), 349–364.

Toseland, R. W., McCallion, P., Smith, T., Huck, S., Bourgeois, P., & Garstka, T. A. (2001). Health education groups for caregivers in an HMO. Journal of Clinical Psychology, 57(4), 551–570.

Tourigny, A., Durand, P., Bonin, L., Hebert, R., & Rochette, L. (2004). Quasi-experimental Study of the Effectiveness of an Integrated Service Delivery Network for the Frail Elderly. Can J Aging, 23(3), 231–246.

Tringali, C. A. (1986). The needs of family members of cancer patients. Oncology Nursing Forum, 13(4), 65–70.

Vitaliano, P. P., Scanlan, J. M., Zhang, J., Savage, M. V., Hirsch, I. B., & Siegler, I. C. (2002). A path model of chronic stress, the metabolic syndrome, and coronary heart disease. Psychosomatic Medicine, 64(3), 418–435.

Vitaliano, P. P., Young, H. M., & Russo, J. (1991). Burden: A review of measures used among caregivers of individuals with dementia. Gerontologist, 31(1), 67–75.

Weuve, J. L., Boult, C., & Morishita, L. (2000). The effects of outpatient geriatric evaluation and management on caregiver burden. Gerontologist, 40(4), 429–436.

Yordi, C., DuNah, R., Bostrom, A., Fox, P., Wilkinson, A., & Newcomer, R. (1997). Caregiver supports: outcomes from the Medicare Alzheimer’s disease demonstra- tion. Health Care Financing Review, 19(2), 97–117.

Zank, S., & Schacke, C. (2002). Evaluation of geriatric day care units: Effects on patients and caregivers. The Journals of Gerontology, Psychological Sciences and Social Sciences, 57(4), P348–357.

Zarit, S. H., Stephens, M. A., Townsend, A., & Greene, R. (1998). Stress reduction for family caregivers: Effects of adult day care use. The Journals of Gerontology, Psychological Sciences and Social Sciences, 53(5), S267–277.

Ya-Mei Chen, Ph.D. MPH, Dr. Chen’s research focus is the development of community- based long-term care services for elders and their family. With her expertise in program intervention and evaluation, Dr. Chen has been involved in several federal and state-funded projected projects to help develop and evaluate programs specific for community elders.

Susan C. Hedrick, Ph.D., Dr. Hedrick’s research focus is the cost-effectiveness of interventions to improve care for persons with chronic illnesses.

Heather M. Young, Ph.D., Dr. Young’s research and clinical interests focus on envir- onments that promote healthy aging. She has played an instrumental role in shaping long-term care policies in Washington State through her evaluation research.