Proposal for a Volunteer Program

CHAPTER 15

Evaluating the Volunteer Program Contexts and Models

Jeffrey L. Brudney, PhD Cleveland State University

Tamara G. Nezhina, PhD DePaul University

The importance of volunteers and the effects of their donated services on the pros-pects of clients, organizations, society, and the volunteers themselves have be- come important matters of discussion and measurement. With resources in short supply and funders continually stressing organizational accountability for grants, con- tracts, and other financial support and the results achieved from these initiatives, the nature of evaluation of volunteer programs has changed over recent decades. Whereas, previously, counting the number of volunteers and the number of hours they contribute to an agency over a given period (such as a year) may have been considered sufficient “evaluation,” organizations have turned to more elaborate meth- ods to assess and demonstrate the contributions of their volunteers. The implications for the field of volunteer administration are evident: Safrit (2010) shows that conduct- ing evaluations has become an accepted (and expected) competency of volunteer re- source managers (also known as volunteer administrators).

Volunteer administrators and their host organizations need to be concerned about evaluating volunteer programs to satisfy the information needs of various constituen- cies. These constituencies or so-called stakeholders are persons or groups who have a stake in, or a claim on, the program, whether perceived or actual. For example, as one of the most prominent stakeholders, funders are no longer content merely with an organization having volunteers onboard but wish to know the results or outcomes or even the long-term impact of their involvement. Another important set of stakeholders, board members are interested in whether all organizational resources, including volunteers, have been put to good, if not best, use. Similarly, a third stake- holder group, organizational leadership, is eager to derive the most benefit from the volunteer program. At a more operational level, managers would like to

363 Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

make sure that volunteers are helping their departments, and the organization, achieve programmatic goals. For their part, volunteers may derive motivation from learning about the value of their efforts and the results they help to bring about for organizations and their clients.

Satisfying all of these stakeholders through the same evaluation of the volunteer program is not easy, and perhaps not even feasible. Accordingly, in this chapter we present an evaluation framework for assisting the volunteer resource manager with understanding and conducting different types of evaluation based on stakeholder involvement. Based on the evaluation literature, we then describe how volunteering might be valued by host organizations, volunteers themselves, and agency clients. We then present a logic model framework to guide the evaluation of a volunteer program. We begin by considering the meaning and purpose of evaluation.

Defining Evaluation

Evaluation entails an assessment or judgment of the value or worth of an endeavor or initiative (Carman & Fredericks, 2008; McDavid & Hawthorne, 2006; Wholey, Hatry, & Newcomer, 2004). Fitzpatrick, Sanders, and Worthen (2004) argue that the purpose of evaluation is to “render judgments about the value of whatever is being evaluated.” These assessments or judgments of value can be put to different uses, but the central purpose is to determine “the significance, the merit or worth” of something of interest (Scriven, 1991). We adopt Scriven’s definition for our goal of providing a framework for the evaluation of volunteer programs. By conducting an evaluation of volunteer participation and contribution, the evaluator attempts to assess the worth or value of volunteer efforts for various stakeholders.

Measuring volunteer value can be undertaken to meet the evaluation needs of various stakeholders. As other researchers have noted, clients, organizations, volun- teers themselves, and the community are the recognized beneficiaries of volunteer contribution (Brown, 1999; Handy & Srinivasan, 2004; Quarter, Mook, & Richmond, 2003). We argue that the needs of stakeholders must dictate to evaluators the purpose (s) of evaluation, and the kinds of methods to be used for measuring volunteer value. Volunteer resource managers charged with conducting, or assisting, an evaluation of the volunteer program should be aware of the influence of these various constituen- cies in the evaluation.

Role of Stakeholders in Evaluation

Some evaluation scholars attribute great importance to stakeholders in defining the purpose of the evaluation and setting evaluation goals (Berk & Rossi, 1990; Fitzpatrick et al., 2004; Patton, 1997; Rossi, Lipsey, & Freeman, 2004; Rutman, 1984). Indeed, stakeholders may hold varying degrees of interest in knowing the value or effectiveness of volunteering for society, a particular organization, or a specific pro- gram. Therefore, before the evaluation begins, the volunteer resource manager (as evaluator) should determine the stakeholders most concerned about the evaluation and establish communication with them to identify their goals for this endeavor (Rossi et al., 2004). Of course, ready communication with stakeholders will almost

364 Evaluating the Volunteer Program

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

certainly prove useful to the volunteer program for other purposes as well, such as promotion, support, outreach, etc. Involving interested stakeholders in the evaluation builds support for the evaluation process and commitment to the results. Patton (1997) contends that individuals, rather than organizations, use evaluation informa- tion. Thus, evaluation information should be targeted to specific persons or groups of identifiable persons or stakeholders, rather than to what was traditionally identified as the general “audience” for evaluation; He considers audiences rather amorphous and anonymous (Patton, 1997)

Rossi et al. (2004) provide a comprehensive listing of potential stakeholders in the evaluation process, including policymakers and decision makers, program sponsors, evaluation sponsors, target participants, program managers, program staff, program competitors, contextual stakeholders, and the evaluation and research community. They argue that these groups and individuals most often pay attention to evaluation. We refine this listing of stakeholders for the volunteer resource manager in the evalua- tion framework we present below.

Patton (1997) is similarly concerned about the utility of evaluation for the stake- holders. His focus is on the practical use of evaluation results. In one study, he found that among stakeholders, 78% of responding decision makers and 90% of responding evaluators felt that the evaluation had an impact on the program (Patton, 1997). Through consultation prior to undertaking the evaluation, the volunteer resource manager can facilitate this positive practical effect by learning about the purpose of the evaluation from the primary intended user(s) of the findings (Patton, 1997; Bingham & Felbinger, 2002). Thus, identifying specific stakeholder groups and under- standing their goals for the evaluation are important to the evaluator for increasing potential application of the findings.

Unfortunately, though, when evaluators attempt to serve too many audiences (stakeholders), they rarely manage to serve all of them well (Horst, Nay, Scanlon & Wholey, 1974). Horst et al. (1974) encourage program evaluators to identify those offi- cials or managers who have a direct influence on program decisions and to design the evaluation goals based on their points of view. This approach is supported by Patton’s (1997) recommendations to identify the group of primary users and to focus on their intended use of the evaluation. Such an approach improves the prospects for the eval- uation results to be utilized by program managers. When involved early in the process of designing the evaluation study, program managers feel ownership of the evaluation process and findings and are more likely to use them for program improvement (Patton, 1997; Posavac & Carey, 1992). Even so, the evaluation might, alternatively, be tailored to the meet the information needs of other stakeholders of the volunteer pro- gram, such as funders, organizational leadership, or the broader society. Accordingly, the model we propose below for volunteer resource managers calls for the involve- ment of primary stakeholders early in the evaluation process.

A further advantage of involving stakeholders in the evaluation is that their partic- ipation can help to determine whether a volunteer program has reached a stage of maturity where it is ready to be evaluated; such an effort is sometimes termed an “evaluability” assessment (McDavid & Hawthorne, 2006; Rutman, 1984; Wholey et al., 2004). Undertaking a full-blown impact evaluation of a program that is too new, unstable, or resource-poor to achieve results is a waste of precious organiza- tional time and energy; effective implementation of the program must occur first.

Role of Stakeholders in Evaluation 365

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

Patton (1997) argues that the evaluator can facilitate the program’s readiness for eval- uation by involving intended users in generating meaningful evaluation questions. This initial scrutiny and discussion among stakeholders and the evaluator can be very useful in reinforcing the need to support implementation and ongoing administration of the volunteer program, so that it would then be more capable of achieving its end goals. At that point, an evaluation of impact would be more appropriate.

Purposes of Evaluation

Following Rutman (1984), we want to understand the role of purpose in designing an evaluation of a volunteer program. Evaluation purpose shapes the evaluation design and helps to focus the results. The purpose of the evaluation is defined in consultation with program stakeholders, such as funders, board members, organizational leader- ship, managers, program evaluators, clients, and the volunteers themselves, each of whom may have different information needs. The volunteer resource manager as evaluator needs to work with relevant stakeholders to clarify the purpose, to design the evaluation process, and to help the stakeholders utilize the findings to their benefit (Patton, 1997; Patton & Patrizi, 2005; Posavac & Carey, 1992; Rossi et al., 2004). The evaluation literature suggests that evaluators need to pay attention to stakeholders’ perceptions and beliefs about the program so that they understand the stakeholder’s purpose and formulate a specific evaluation question(s) aimed toward the need of the consumer for certain information.

Identification of stakeholders, then, is the first step to elucidate goals and define the purpose of the evaluation. However, the determination of relevant stakeholders is often out of the hands of the volunteer resource manager. Funders may demand an evalua- tion of program impact as a condition of their financial allocation; board members may request an audit of volunteer activities; organizational leadership may solicit an analysis of the costs and benefits of volunteer involvement; operational managers may want to learn how to deploy volunteers more productively. More diffuse stakeholders may exert their preferences for the evaluation as well. For example, policymakers, the media, or the public (through individual citizen inquiries or through other stakeholders acting on their behalf) may inquire as to the benefit to the larger community of the activities of volunteers or the agency as a whole. In many nonprofit organizations, the volunteer administrator is charged with preparing a periodic evaluation (e.g., annually) of the volunteer program. This effort should take into account the information needs of the central stakeholders. Rehnborg, Barker, and Pocock (2006) encourage organizations first to clarify their purpose(s) for placing a value on volunteer efforts, and then to choose the appropriate evaluation method.

Evaluation literature suggests that evaluation of volunteers, like evaluation of other phenomena, should be conducted for the purpose defined by users of evalua- tion findings, and with approaches and techniques shaped by this purpose. Talmage (1982) defines three major purposes of evaluation information:

1. Meeting a political function 2. Assisting decision makers responsible for a policy or a program 3. Making judgments about the worth of the program

366 Evaluating the Volunteer Program

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

Her function-based approach shows considerable overlap among the functions. Rutman (1984) develops the purpose-based approach to program evaluation further. He also defines three purposes for evaluation: accountability, management, and knowledge. These three purposes are distinct and help to identify evaluation informa- tion users such as policymakers and organizations, managers, and the research com- munity. Drawing on Chelimsky’s (1997) definition of evaluation purposes, which follows Rutman’s classification with some variation,1 Rossi et al. (2004) elaborate the list of users of the three types of evaluation. All of these approaches are very similar conceptually, suggesting political, knowledge, and organizational purposes for evalu- ation. We elaborate these purposes next.

Political Purpose

Following the evaluation purposes described by Talmage (1982), Rutman (1984), Che- limsky (1997), and Rossi et al. (2004), the first purpose for undertaking an evaluation is political. From this perspective, the external audience for evaluation of volunteer effort is society at large and political decision makers. These stakeholders want to know, for example, how to create and refine policies toward volunteering, how vol- unteer programs benefit society, and whether such policy instruments as a tax subsidy to the nonprofit sector are justified by the results of its activities.

Society at large evaluates volunteer involvement based on value judgments con- cerning volunteering as a social phenomenon. If volunteering is recognized as an in- herent end goal in the society, the conclusion follows that “the more volunteers the better.” Yet, the value produced by volunteers for the benefit of the society is hard to measure in economic terms because it is neither bought nor sold in the marketplace. To the contrary, it is given, which renders its price beyond economic or monetary value. Goods produced by volunteers surpass market price for comparable goods be- cause these goods are infused with value added, such as good intentions; they are given wholeheartedly, which makes them “priceless.” They are in sharp contrast to goods and services sold in the market for the purpose of gaining profit. At the societal level, then, learning about the aggregate volume and value of volunteering for society, and how to stimulate them, are important evaluation purposes.

Knowledge Purpose

According to Rutman (1984), a second purpose for conducting an evaluation of volun- teers is the generation of basic knowledge about the value of volunteer contributions. This general knowledge is produced by those who study the phenomenon and seek to understand, explain, and predict its future development. The knowledge pro- duced may not be of immediate use, but it contributes to better understanding of

1 Chelimsky (1997) and Rossi et al. (2004) identify the same three major purposes for evaluation as Rutman (1984), but they use different terminology to describe them. For example, Rutman defines one purpose for evaluation as managerial, and Chelimsky and Rossi et al. define the same purpose as program improvement. In Rutman, the last purpose of evaluation that is not mentioned in our discussion is defined as the covert purpose. Rossi et al. also identify the same purpose as hidden agendas.

Purposes of Evaluation 367

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

volunteering and its role in personal life, the community, society at large, and the economy.

Economic approaches to valuing volunteers can be considered knowledge- related because they add to our knowledge of volunteering from a novel viewpoint. Yet, Graff (2005) and Smith and Ellis (2003) advise researchers to be cautious about relying exclusively on economic measurement, such as dollar valuation, to assess vol- unteer contributions. Graff recommends the use of a more complex and complete ap- proach that involves the identification of outcomes of volunteer work, calculation of the full costs of achieving those outcomes, and a consideration of whether the invest- ment in volunteers is a productive use of resources. She maintains that if members of the community are informed about the true social value that volunteers create, they are more willing to support the mission of an organization.

Organizational and Managerial Purpose

Talmage (1982) identifies assisting policy and program decision makers, guiding changes and innovations in a program, and informing managers about the worth of a program as possible functions of evaluation. This purpose converges with the manage- rial purpose for evaluation defined by Rutman (1984) and Rossi et al. (2004), which serves those needs of the organization. For example, Hotchkiss, Fottler, and Unruh (2009) in a study of the value of volunteers in hospitals find that more hours of volun- teering positively correlates with significant cost savings and higher level of patients’ satisfaction at the hospitals. This type of evaluation is helpful to agency insiders, includ- ing chief executives or directors, volunteer managers, program managers, and other staff, who may be interested in improving volunteer involvement and deployment, and the return on organizational investment in these important human resources. Rutman describes the management perspective on program evaluation as serving as a tool for making improved decisions about the design and delivery of programs and about the type and amount of resources that should be devoted to them.

From the organizational or management perspective, the measuring instruments for evaluating volunteer programs need to be adjusted or attuned to a set of manage- rial purposes. For example, it is not very useful for a manager to know the value of the aggregate hours contributed by volunteers, or the full-time equivalent labor force that volunteers constitute to make informed decisions about how to manage the program. Instead, managers need to know about such matters as how many volunteers are available, how many are needed for achieving departmental or organizational goals, how they are deployed, and the results that the volunteers help the organization achieve. This information helps them to plan strategically, and to set recruitment pur- poses and estimate expenses. Knowing the tasks performed by volunteers allows vol- unteer resource managers to develop their recruitment tactics based on information about the available and potential pool of volunteers; make decisions about deploy- ment of volunteers to maximize their effectiveness; and balance involvement of paid staff with a volunteer workforce. Program managers are interested in identifying where and how volunteers can extend the capacity of paid staff and augment paid labor. Useful to this group is evaluation information pertaining to the effectiveness of volunteers in performing assigned tasks and in assisting departments and the organi- zation toward goal achievement.

368 Evaluating the Volunteer Program

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

Purpose-Based Evaluation Framework for Valuing Volunteers

The evaluation literature provided useful insights into the purposes for conducting an evaluation. Building on this foundation, we have developed a framework for analyz- ing the purposes of evaluation of volunteer programs which incorporates the informa- tion needs of important stakeholder groups.

The purpose approach that we suggest helps to clarify evaluation needs by grouping multiple stakeholders into three major categories: society, organizations, and managers. At the societal level we define the following interests—society at large, policymakers, and researchers. At the general organizational level we define as stake- holders the following groups—the local community, agency board of directors, orga- nizational decision makers, and funders. Finally, the third, managerial level is represented by program managers, volunteer coordinators, paid staff, and volunteers. We derived these categories of stakeholders by modifying earlier approaches of Tal- mage (1982), Rutman (1984), Chelimsky (1997), and Rossi et al. (2004) to defining the major evaluation purposes for various groups of stakeholders.

Exhibit 15.1 presents the continuum of evaluation purposes captured in the new framework from the most general and abstract, to the more applicable, and finally to the most practical. The figure suggests questions to guide the evaluation of volunteer value at the different levels. At the most general societal level, the relevant questions pertain to the value of volunteering for society and the attendant methodologies. At the organizational level, the questions are similar but more circumscribed. They con- cern, for example, the relative costs and benefits of volunteering to the organization. Finally, at the managerial level, the questions address the actual deployment of vol- unteers, their interactions with paid staff and clients, and issues of operational effec- tiveness. At this level, the interests of stakeholders guide evaluators to gather

Societal Organizational Managerial

!!

Society/Knowledge

How much is volunteering worth to society? How should volunteering be valued? What approaches are available for evaluation?

Organization/Board/Funders

What is the value of volunteers to the organization? What are the costs of volunteering to the organization? Are volunteers used responsibly? Does the community support the organization through volunteering?

Manager/Staff/Volunteers

How effective is volunteer involvement in each program? Do volunteers achieve the expected results in each area? How can volunteers extend and enhance the work of paid staff? What kind of training do volunteers need?

EXHIBIT 15.1 Continuum of Evaluation Purposes

Purpose-Based Evaluation Framework for Valuing Volunteers 369

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

program-specific, detailed information for the practical use of organizational managers.

Exhibit 15.2 elaborates the purpose-based evaluation framework. It depicts the different levels of evaluation, primary stakeholders, focus of analysis, and major value or purpose of the evaluation. At the societal level, the focus is on the nonprofit/volun- tary sector or field, and the central purpose is an assessment of aggregate worth or the contribution to general knowledge. At the organizational level, the focus is centered on the host agency, with the purpose of accountability and stewardship for the volun- teer program. Finally, at the managerial level, the focus is the volunteer program, and the purpose encompasses operational effectiveness. We describe the different levels more fully in the sections below.

Societal Interests

The first level of evaluation includes important general interests. We define this level as societal and knowledge-based. At this level such stakeholders as society at large, legislators, political decision makers, and researchers need evaluation information to assess public and economic usefulness of volunteering, and to understand the aggre- gate worth of volunteering as a social phenomenon. Researchers endeavor to produce knowledge about volunteering and its value, and to propose various methodologies and measures to advance this inquiry. Policymakers can potentially use such general information about volunteering to guide public policy. Answers to these questions help to assess the aggregate worth of volunteering and improve our understanding of this phenomenon and the nonprofit sector more generally.

Organizational Interests

The second level of evaluation is more specific. At this level we find organizations and boards of directors as primary stakeholders, as well as funders. Their interests include the assessment of the value of volunteering to an organization, the cost of agency investment in volunteers, and the contribution to the organizational mission yielded through volunteer labor. Volunteer value evaluation at this level is focused

EXHIBIT 15.2 Evaluation Framework

Evaluation Levels/ Approaches Primary Stakeholders Focus Value/Purpose

Societal/ Knowledge

Society Legislators Political decision makers Researchers

Sector/Field Worth (aggregate) Learning (knowledge)

Organizational Organizational decision makers Board of directors

Organization Accountability Stewardship

Managerial/Staff Managers Volunteer resource managers Paid staff

Program Deployment Effectiveness

370 Evaluating the Volunteer Program

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

on the organization and its volunteers. Major concerns of primary stakeholders at this level are securing accountability, providing governance, controlling finances, and pur- suing the organizational mission.

Managerial Interests

The third level of evaluation is tailored to the practical interests of program managers, volunteer coordinators, paid staff, and volunteers. Their ultimate purpose is to im- prove program implementation through the involvement of volunteers and the expe- rience of these participants. Information about the current use of volunteers can assist these stakeholders in making decisions about deployment and effective involvement of lay citizens, the types of skills and knowledge required of volunteers to perform their tasks, the number of volunteers needed, the recruitment and retention rates of volunteers, etc.

The framework shown in Exhibit 15.2 is useful to define the evaluation purpose with regard to the various stakeholders. Although program goals tend to be vague and conflicting (Wholey, 1983), Poister (2003) argues that every program can be eval- uated. When the purpose of the evaluation and the end-user have been defined, the evaluator can select and design the kind of measures to be used, the level of detail required, and the frequency of measuring and reporting to enhance the decision-mak- ing process (Poister, 2003). The proposed framework assists volunteer resource managers as evaluators in making these determinations.

Valuing Volunteering for Organizations, Volunteers, and Clients

As we have showed, the evaluation literature presents a persuasive case for the impor- tance of identifying the purpose and goals for conducting an evaluation. Given these different purposes, the volunteer resource manager as evaluator may need to assess the value of volunteering for the organization, volunteers, and clients. We summarize relevant studies and methods below.

Value of Volunteering to the Host Organization

Measuring volunteer value is important to organizations for managerial, financial, and fundraising purposes. The most common method for valuing volunteer contributions is economic—an assessment of the dollar value of volunteer hours given to an organi- zation. The Independent Sector (IS) organization suggested attributing the average hourly wage for civilian, nonagricultural, nonsupervisory workers to account for vol- unteer hours, increased by 12% to account for benefits. The IS average hourly volun- teer compensation was calculated at $20.85 (in 2009). The total dollar value of volunteer time for 2009 is estimated at $169 billion to organizations, excluding in- formal volunteering (Independent Sector, 2011). The advantage of the IS approach is that it uses a widely cited and available statistic (the average hourly wage for civilian, nonagricultural, nonsupervisory workers in the United States). The disadvantage is that it does not distinguish among volunteers’ variable tasks and their respective mon- etary values.

Valuing Volunteering for Organizations, Volunteers, and Clients 371

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

Several knowledgeable observers make a compelling argument for correcting the IS valuation method to take into account the variety of tasks performed by volunteers in the attribution of hourly dollar values to their labor (e.g., Andersen and Zimmerer, 2003; Brudney, 1990; Ellis, 1996; Gaskin, 2003). Brown (1999) suggested devoting at- tention to the nature of the tasks performed by volunteers and the equivalent compen- sation for the same (paid) work; she proposed basing these economic measures on the average compensation in the service sector, because volunteers were assumed to be mostly engaged in producing services rather than goods. By fixing the hourly dol- lar value at the level of social workers’ wages, this approach lowered the average vol- unteer contribution value dramatically (from the IS [2011] estimation) to $9.87 per hour (in the late 1990s).

Handy and Srinivasan (2004) describe four approaches to valuing hours contrib- uted by volunteers to the organization based on the opportunity cost and the replace- ment cost of volunteer time. The opportunity cost approach looks at the value of a volunteer hour from the perspective of a volunteer and what that hour is worth to this person. The replacement cost method evaluates the value of volunteers from the per- spective of the organization, as if the agency had to pay market wage rates when pur- chasing such a service. Handy and Srinivasan describe the opportunity cost evaluation method based on the average wages foregone for volunteers who are employed ($16.42) and hypothetical wages for those who are unemployed ($12.58) donating their time in hospitals, the organizations investigated in their study. This approach, however, does not consider the fact that volunteers often perform very different tasks from those on their regular jobs.

The second opportunity cost approach is based on the perceived value of lei- sure time as assessed by the volunteers themselves. Handy and Srinivasan (2004) call this approach the L opportunity cost. The L opportunity cost produces a lower estimation of hourly value contributed to the hospitals by volunteers, in line with the volunteers’ own assessment of their leisure time. The inconsistency of the L opportu- nity cost method lies in its reliance on the subjective evaluation of leisure time by volunteers.

The replacement cost approach assumes that the value of a volunteer hour is equal to that of a staff hour when volunteers perform the same tasks that paid staff would have conducted. In the hospital volunteering study Handy and Srinivasan de- scribe this approach as unrealistic because of the hospitals’ fiscal constraints. They contend that the staff would not have provided services delivered by volunteers in the absence of volunteers under conditions of fiscal stringency. Another variation of the replacement cost method is proposed by Ross (1994), who suggests the industry wage application method, which takes the hourly wage in that sector(e.g., the aver- age hospital wage as equal to $19.69) and adjusts it for benefits. Following his ap- proach, the adjusted hourly value of volunteering for the hospital is calculated at $23.23.

Several studies discuss whether organizations would choose to pay employees to provide the same services if volunteers were not available and/or treat the general topic of the possible interchangeability of paid staff and volunteers (Bowman, 2009; Handy & Mook, 2010; Handy, Mook, & Quarter, 2008; Simmons & Emanuele, 2010). Examining volunteers as fair substitutes for paid labor in Canadian organizations, Handy et al. (2008) find that the organizations are more likely to use volunteers and

372 Evaluating the Volunteer Program

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

staff interchangeably under conditions that both volunteers and employees have com- parable skills and knowledge, and when volunteers are numerous due to such factors as fewer jobs in the (paid) labor market. Their study suggests that organizations may improve their efficiency by relying on volunteer labor.

Quarter et al. (2003) choose to use the replacement cost method in their estima- tion of volunteering value. They argue that this approach is appropriate because the replacement-cost framework allows calculating volunteer services at the value of similar services in the market. However, they admit, the debate continues whether volunteers substitute for paid labor or supplement paid labor (Brudney, 1990; Ferris, 1984). The replacement-cost approach also fits well with the labor division among volunteers and staff that is typical for some organizations (Handy and Srinivasan, 2004).

Gaskin (1999) also uses a replacement cost method to construct the Volunteer Investment and Value Audit (VIVA). VIVA takes into account the variety of tasks that volunteers perform by analyzing and measuring actual activities and matching them to paid work in the market, an approach introduced by Karn (1982–1983, 1983) in the early 1980s. Simultaneously, VIVA addresses the issues of benefit-cost and cost- effectiveness by examining the organizational inputs, defined as resources used to support volunteers, in relation to the outputs, defined as the pound (monetary) value of volunteer time. Dividing the output by the input allows calculation of the VIVA ratio, which states that for every pound invested in support of volunteers, X pounds in the value of their services are returned to the organization (Gaskin, 1999). The VIVA method also bases its assessment of volunteers’ value to the organization by imputing the value of contributed hours.

Anderson and Zimmerer (2003) discuss five methods for valuing the hours contributed by volunteers using the average wage, comparable worth (similar to the replacement-cost method), the Independent Sector (2011) approach (described previ- ously in this chapter), the living wage, and the minimum wage. Most of these methods can be classified either as opportunity-cost approaches or as replacement-cost approaches. Anderson and Zimmerer (2003) criticize the replacement-cost method for the implicit assumption that volunteers and paid employees are perfect substitutes. In addition, this method does not take into account the level of compensation for “vol- unteer substitutes:” it can be either on the entry level or the advanced level of com- pensation (Hopkins, 2000). The minimum wage method is easy to use, but it does not value volunteer activity respectfully (i.e., according to the work performed). Ellis (1996, 1999) contends that most volunteer assignments are above the minimum wage level, perhaps even higher than the median wage. In her view, this method does not reflect the value of volunteer expertise.

The living wage method is based on dollars required to subsist; it is closer to the cost of living approach (Anderson & Zimmerer, 2003). Much like the minimum wage approach, the cost of living method lacks any relationship to the nature of the particu- lar tasks performed by volunteers. It inherently places a low value on volunteer ser- vice, as if the service is performed by an unskilled worker. Having recognized these many disadvantages of the minimum wage and the living wage methods, most practi- tioners prefer the average wage or similar methods adjusted to include benefits. For more specific assessments, local average wages can be used, a procedure that allows contextualizing the value of volunteering (Anderson & Zimmerer, 2003).

Valuing Volunteering for Organizations, Volunteers, and Clients 373

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

Ross (1994) proposes the computation of “person-years” to account for volunteer hours, a method that differs conceptually from the dollar-based approaches. This method suggests valuing volunteer time in terms of full-time, year-round positions or person-years, equivalent to dividing the total hours contributed by volunteers by the average annual hours worked by a full-time employee (Ross, 1994). This approach is well-known as the full-time-equivalent (FTE) method (the maximum number of hours for an FTE might be set at 52 weeks £ 40 hours D 2,080 hours per year). This method makes no effort to differentiate the type of work performed by volunteers, or to esti- mate the dollar value of the volunteer time. It implies that volunteers provide for the extension of paid staff in an organization.

Goulbourne and Embuldeniya (2002) have developed several measures of volun- teer contributions that can be used for presenting the effect of volunteers on enhanced organizational revenues and decreased expenses. They propose looking at the ratio of volunteers to paid staff; the ratio can inform funders, managers, the general public, and other stakeholders regarding how much of the organizational success can be attributed to volunteers’ effort. Goulbourne and Embuldeniya (2002) also consider community involvement in the activities of a nonprofit organization as an important measure of success. This approach treats participation of volunteers as an inherently valued goal in itself. They measure the “community investment ratio” (CIR) by assess- ing the volunteer contribution, expressed as the dollar value of donated time, relative to the total dollar amount a particular funder has contributed or may contribute to the volunteer program or a specific event. This ratio allows comparing community in- volvement in various localities. Another economic measure they present is community support, calculated as “volunteer capital contribution” (VCC), which considers the amount of resources brought by volunteers—donations added to nonreimbursed out- of-pocket expenses of volunteers.

In their study of hospitals, for example, Hotchkiss et al. (2009) present empirical evidence that regardless of the cost of training and management, volunteers provided financial benefits exceeding costs. Irrespective of hospital size, many of the programs “reported that they had savings of more than a million dollars when they used volunteer service.” With respect to generalization of results, the researchers acknowl- edge that hospitals often use medical students as volunteers, whose skills allow the administration to save on paid labor for particular services (Hotchkiss et al., 2009, p.125).

Handy and Srinivasan (2004) present a calculation of volunteer contribution based on the benefit-cost approach. All benefits accruing to hospitals (the unit of anal- ysis in their study) and to volunteers are summed, and the costs of volunteering to the hospital and to volunteers themselves are subtracted. Thus, the authors arrive at an estimate of the net benefits of volunteering to the hospital, which they find to be very significant, and net benefits to the volunteers themselves, which they find to be nega- tive. In their study the volunteer contributions are estimated on the basis of the hourly value of volunteer work.

Another study by Handy and Mook (2010) evaluates direct and indirect benefits and costs of volunteers to an organization. With respect to benefits, they argue that in times of crisis many organizations undergoing budget cuts would rely on volunteers more heavily. In addition, volunteers may help organizations connect with communi- ties and receive recognition for the services they provide. Involving volunteers may

374 Evaluating the Volunteer Program

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

also pose problems, however, such as conflict with labor unions, liability issues, or tension with the paid staff (Handy & Mook, 2010). Thus, from the organizational per- spective, valuing volunteers entails an assessment of benefits and costs.

Using a noneconomic methodology, Hager and Brudney (2005) introduce a summary measure of “net benefits” of a volunteer program to an organization. This approach combines benefits and challenges realized by a volunteer program into a single barometer of (net) benefits to the organization. The method asks volunteer program managers to rate the benefits received by an organization from volunteer participation and the challenges encountered. “Net benefits” are calcu- lated as the difference between the benefits and the challenges. This approach has certain advantages: It allows identifying the benefits and problems emanating from volunteer participation, balancing the benefits against the problems in a single mea- sure, and calculating and comparing results across volunteer programs and host organizations.

In many studies, volunteers are viewed as producers of outputs and outcomes as well as beneficiaries. Multiple studies suggest that volunteers consider participation in nonprofit organization activity as personally beneficial to them (Brudney, 1990; Ellis, 1999; Handy & Mook, 2010; Handy & Srinivasan, 2004; Quarter et al., 2003). In the next section, we consider suggested methods for calculating the benefits that accrue to volunteers as a result of participation.

Value of Volunteering to the Volunteers

Many researchers and practitioners conceive of volunteers as providers of services as well as beneficiaries of participation. To measure the value accruing to volunteers themselves, Brown (1999) suggests two alternative methods. One method proposes calculating benefits in terms of opportunity cost. This approach assumes that volunteers benefited by gaining satisfaction measured in monetary terms in the amount of wages that they agree to forgo minus taxes (25%) and fringe benefits (i.e., the material gains that volunteers willingly give up to volunteer). The second way is to measure out-of- work volunteer time according to valuation given to this time by volunteers themselves. Based on other research, the value of free time is measured at the rate of half the employment wage. In addition, Brown argues that volunteers’ hourly value depends on the motivation to volunteer. When volunteers agree to endure as much stress while producing donated services as they would experience on their jobs, the level of motiva- tion is higher. Hence, the volunteer hour must be measured closer to the amount of the hourly wage (six-sevenths of a regular hourly wage). In such cases, the portion of the wage, properly adjusted for fringe benefits and taxes, is the measure of the volunteers’ cost of volunteering. Pro bono volunteering is an example of such services. When the volunteering environment is less stressful and more pleasant, the level of motivation is lower. In this instance, the economic value (or cost to the volunteer) can be measured at half the rate of the regular hourly wage.

Handy and Srinivasan (2004) measure benefits to volunteers by means of a sur- vey. Survey questions include attitude scales asking for ratings from 1 to 5 for specific benefits gained through volunteering, such as new skills, social contacts, references for employment, and job and career opportunities. Most volunteers rated these

Value of Volunteering to the Volunteers 375

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

benefits as high—4 or 5 on the respective scales. Handy and Srinivasan use willing- ness to pay to acquire such benefits as a measure of the monetary value of the volun- teering experience in the hospital: In their study, the average volunteer professed a willingness to pay $179.24 (2004) to attain these benefits. As Handy and Srinivasan recognize, this measurement requests a very subjective assessment and, hence, has limited reliability and generalizability. Handy and Mook (2010) discuss other non- economic benefits to volunteers, such as increasing social status and knowledge (especially from service on agency boards of directors) and the “warm glow” emanat- ing from volunteering.

In order to evaluate the benefits that volunteers receive from participation, Quarter et al. (2003) use a different approach based on identifying and pricing a “surrogate” (i.e., proxy or comparison) in the private marketplace. For example, gains to volunteers include such non-material benefits as the development of personal lead- ership skills. Quarter et al. propose as a surrogate measure the cost for a student to learn these same skills in a university course. (They estimated the cost of an appropri- ate course at $500 based on college courses that taught similar skills.) The surrogate methodology allows approximating the value of the benefit received by volunteers from their participation. Total benefits to volunteers are assessed by multiplying the percentage of volunteers who report receiving the benefit by the economic value of the benefit calculated according to the surrogate method. Quarter et al. suggest that the surrogate methodology can be extended to other benefits that might be gained by volunteers; depending on the program, volunteers might obtain skills in counseling, first aid, and so on. To arrive at net benefit estimation for volunteers, the cost of any training provided by the organization would be subtracted from the total amount of benefits received by these participants.

Although these methods are useful, the organization and the volunteers are not usually considered the main beneficiaries of volunteer activity by those who donate funds, and by those who assess the effectiveness of volunteer programs and nonprofit organizations. The primary beneficiaries are most often considered the cli- ents of the organization. The next section discusses the issue of measuring benefits to clients.

Value of Volunteering to Clients

Because clients typically receive services provided by volunteers at no cost to them, it can be especially difficult to evaluate benefits to this group. Instead of asking how much money clients would be willing to pay for the service that they receive for free, Murray (1994) asked how needy clients would trade off the in-kind services provided to them against a hypothetical gift of “cold hard cash” (equivalent valuation method). Murray used this approach to measure the in-kind goods value to recipi- ents of social security services such as Food Stamps and Temporary Assistance for Needy Families (TANF) benefits against the offered cash. Murray argued that welfare recipients were ready to trade such in-kind benefits for a smaller amount of cash, which equaled the value of in-kind transfer to 73% of the cash value. Economists call the difference between in-kind goods value and the amount of cash preferred by the recipients “deadweight loss,” because this difference produces no utility and satisfies no preferences. Economists maintain that this loss accounts for the desire of

376 Evaluating the Volunteer Program

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

policymakers to change recipients’ behavior. Following Murray’s findings, Brown (1999) suggests that the value of services to clients should be approximated through wage-based estimates of the market value of volunteer-produced services adjusted for the inefficiency of the in-kind resource transfer (i.e., at approximately 73% of the in-kind service market value).

In their model, Quarter et al. (2003) define direct client benefits as a primary out- put. They maintain that since recipients of the services do not pay for them, the mar- ket lacks a signal to help identify the value of the output. Therefore, they advise attributing a surrogate value to the output by finding and pricing a comparable service in the market; the method is analogous to the procedure described above to estimate the value of benefits received by volunteers. Quarter et al. conclude that the value of a volunteer service to a client is defined by the price for the comparable service or good in the market. For example, if the intended outcome for clients is independent living for the elderly, a surrogate measure is the cost of a nursing home.

In their study, Handy and Srinivasan (2004) define the value of volunteer ser- vices to enhance the quality of care to hospital patients (clients) as nonmaterial. They argue that volunteers are able to provide many soft, or intangible, services that are essential to the comfort of patients. Volunteers also reduce the workload of paid staff by taking on certain tasks and leaving staff members more free to concen- trate on other tasks (Brudney, 1990). By providing help to patients and supporting paid staff, volunteers enhance the quality of care, which is an important component of health care, although difficult to measure in monetary terms (Handy & Srinivasan, 2004). Handy and Srinivasan offer a nonmonetary measurement of quality of care by volunteers, which we refer to as impact rather than output. We discuss this inno- vative measurement below.

Other benefits that clients of health care organizations receive from volunteer services have been documented by Hotchkiss et al. (2009) in their study of Florida hospitals. They find that higher volunteer hours in patient care areas strongly corre- late with patient satisfaction, indicating that “hospitals with a significant volunteer component are likely to provide positive patient experiences.” (Hotchkiss et al., 2009, p.126).

As we have seen, economic evaluation of volunteer contributions continues to be a major interest of researchers. The various economic approaches yield great insight into the value of volunteer effort, yet they do not capture the gamut of volunteers’ contributions and the value they generate, or the ultimate impacts of their activity. Graff (2005) calls on researchers to be cautious about relying solely on economic measurement to assess volunteer value. She maintains that the dollar valuation method underestimates the actual value of volunteer work (Graff, 2005).

A panel discussion on the e-Volunteerism Web site (Fryar, Mook, Brummel, & Jalandoni, 2003) demonstrates a range of views on the value of economic measure- ment of volunteer contributions. Fryar begins by stating that “the most enduring and controversial question within the field of volunteerism is the one that relates to the ‘value’ of volunteers and the hours they contribute.” Brummel suggests that focusing on the monetary value of volunteering is harmful in the long run because, in his view, a narrow “economic focus” distracts from social valuation of volunteering and trivial- izes volunteer impact. Other participants in this discussion find it useful to conduct and present an economic valuation in combination with alternative indicators of

Value of Volunteering to the Volunteers 377

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

volunteer value (Mook, Jalandoni). Jalandoni takes a middle position that “[b]oth the quantitative as well as qualitative, anecdotal, and societal value of volunteering are important depending on what you need to use them for.” As Graff (2005) advises, a more complex and complete approach would involve the identification of outcomes or results of volunteer work, calculation of the full costs of achieving those outcomes, and finally evaluation of whether it is a worthwhile investment of resources.

As in other areas of social life, evaluation of the impact of volunteering is a chal- lenging task that is rarely measurable by using quantitative tools alone. Impact refers to the effects or results of an activity, program, or initiative in the larger community or society. As Safrit (2010) writes, “Thus, impact may be considered the ultimate effects and changes that a volunteer-based program has brought about upon those involved with the program (i.e., its stakeholders), including the program’s targeted clientele and their surrounding neighborhoods and communities, as well as the volunteer organiza- tion itself and its paid and volunteer staff” (p. 321, emphasis in original). The following section discusses attempts to measure the impact of volunteer activity.

Impact Measurement

Here we return to an innovative noneconomic approach that Handy and Srinivasan (2004) suggest for measuring the impact of volunteers in their study of the hospital services provided by volunteers. To do so, they identify the lack of overlap in roles between paid professionals and volunteers in the hospital setting, and document the division of labor between paid professionals and volunteers. Interviews with volun- teer managers identified enhanced quality of patient care as the most important contri- bution of volunteers. In order to identify volunteers’ impact on this dimension, Handy and Srinivasan asked the managers to rank 26 quality programs offered by hospitals that were described in the literature. To measure the quality of care, the survey asked managers, paid staff, and volunteers to rate the impact of volunteer services in each of these domains on a scale of 1 to 10. On average, managers rated volunteers’ impact on quality of care as 9.0, staff members rated it as 8.43, and volunteers as 8.7. The con- verging values supported the validity of measurement. Because this evaluation method is based on stakeholders’ assessment of the impact of volunteer services, it is often referred to as a “stakeholder approach.” It allows measuring the impact of volun- teer effort directed to achieve a specific inherently valued goal such as quality of pa- tient care.

Quarter et al. (2003) present a new approach to impact measurement for volun- teers and nonprofit organizations: social accounting. They suggest giving recognition to volunteers by measuring the social impact of volunteer services, and including this value in the accounting statement of an organization. Quarter et al. propose using the Expanded Value Added Statement (EVAS) to include the contribution of volunteers to the total value added produced by the organization. Three types of outcomes identified by the Community Return on Investment model are incorporated in the EVAS. For the EVAS statement, Quarter et al. include and measure three types of outputs:

1. Primary outputs. The value of direct services of an organization to clients 2. Secondary outputs. The value of indirect outputs that accrue to the organization’s

members (staff and volunteers) and customers (e.g., skills development)

378 Evaluating the Volunteer Program

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

3. Tertiary outputs. The value of indirect outputs that accrue to those other than the organization’s members and clients (e.g., consultations provided to other cooperatives)

Primary outputs to clients can be evaluated according to the surrogate valuation methods described above. Benefits that accrue to volunteers are classified as second- ary outputs. Quarter et al. (2003) suggest including these outputs in the EVAS as hours contributed plus out-of-pocket expenses not reimbursed. In order to value the hours of volunteers, Quarter et al. again use the replacement-cost approach based on the price of equivalent paid labor. Tertiary benefits to third parties are calculated by attrib- uting the market value to the services the third party received for free from the organi- zation, again using the surrogate method. The summation of all three outputs yields the value added by volunteering to the organization.

An important part of the EVAS is the idea of distribution of the value added among stakeholders. Primary services go to clients, secondary benefits to participants (staff and volunteers), and tertiary to the community. Calculation of the value to the first two of these groups of stakeholders is more straightforward; estimation of volun- teer value to the community requires greater ingenuity. Quarter et al. (2003) suggest using several approaches to measure the impact of volunteers:

1. Surrogate valuation. The comparable service can be found in the market, and the price for this service will be a surrogate value of the unpriced volunteer service.

2. Survey technique. Worth to clients is measured by compiling the list of either prices or consumer items and asking the respondents to situate the service in rela- tion to others on the list.

3. Avoidance cost. Calculated as the cost of undoing the damage. For example, the loss of or damage to outdoor recreational facilities that are publicly available has been assessed by the fees needed to replace the facilities (Crutchfield, 1962).

4. Attribution. Assigning a weight to various factors that influence results. This is achieved by means of using comparison groups, and longitudinal studies that help to collect information about change. Often it is impossible to determine precisely the causal effects, but information increases understanding and knowledge.

5. Stakeholder input. Stakeholders are defined and systematically asked in open meetings, interviews, confidential focus groups, and surveys about their views on the desirability of the service and its impact on clients.

Cahn and Rowe (1992) offer a conceptually different view on valuing volunteer time for the larger community. They propose the time dollar value as an instrument for valuing volunteers’ contribution, based on the idea of reciprocity. The time dollar method holds that when a person volunteers an hour of his or her time to help another person, his or her time-dollar bank is credited one hour. The time-dollar ac- count may accumulate multiple hours, which indicates that the owner is entitled to receive an equivalent amount (in hours) of services or goods from other people, or to receive in-kind compensation, such as reduced tuition cost in college. The value of a contributed hour is not differentiated by the type of services provided; all hours count the same regardless of the qualifications of the service providers and the

Value of Volunteering to the Volunteers 379

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

content of the service itself. The philosophical foundation for this method is equality of good intentions, and not the differentiated effort or service value. Accumulated credit hours would pay for provision of reciprocal assistance, but will not allow measuring the impact, even though the impact may be socially and economically significant (e.g., increased literacy). Cahn and Rowe’s (1992) time-dollar method requires very accurate recording and crediting of volunteer hours contributed and a well-defined group of people or “members” who participate, such as residents in a community, neighbor- hood, co-op, or college dormitory. For these authors, the “impact” of volunteer activity is community participation fostered through reciprocity.

Poister (2003) offers practical tips for public and nonprofit organizations on how to conduct evaluations that measure outputs and outcomes. He elaborates the “logic model” approach, which helps to specify the resources, activities, and outputs of a pro- gram or organization, and the outcomes (initial, intermediate and long-term) it is meant to achieve. The logic model begins with a careful preliminary analysis of the goals of the organization or program and the conceptual framework for their attainment; in this ap- proach, outcomes are the expected results conforming to the goals. When the goals are clear, and the expected results in the short-term and in the long-term periods are speci- fied, the task of a manager is to define the performance measures. Performance mea- sures typically include measures of output, efficiency, productivity, service quality, effectiveness, cost-effectiveness, and customer satisfaction. Application of this compre- hensive approach would assist the volunteer resource manager as evaluator to capture the multidimensional nature of the outcomes and the long-term impact created with the active help of volunteers. We turn to that topic now.

Logic Models

Logic models are often used as a framework for evaluation. A logic model depicts graphically a causal interpretation of how a program operates and is thought to achieve its intended results. The logic model is usually presented as a figure or diagram that shows connections between the important elements of a program. Organizational or program goals or objectives drive the logic model so that it is first necessary to agree on what the program is attempting to achieve—a useful process that can require significant discussion and consultation among stakeholders and program and organizational staff. As discussed above, the views of important stake- holders must be taken into account in this determination. However, the more specific the goal(s) of a program or initiative, the more straightforward will be the ensuing evaluation task.

The first of the elements in the logic model is the inputs or “resources” invested in the program, including monetary, personnel, equipment, technological, etc. Program managers and leaders are charged with converting these resources into pro- gram “activities,” such as services provided by the organization to clients or outreach efforts. The first tangible, measurable results of program activities are labeled “outputs,” which indicate the work the program has performed. Depending on the goal(s) of the program, outputs may include the number of sessions conducted with clients, the number of visits to clients, the number of promotional activities carried out by the program, and so forth.

380 Evaluating the Volunteer Program

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

Some evaluations go no farther than examination of the interconnections between program resources, activities, and outputs. Because these elements are largely under the control of the organization, it is of considerable interest to some stakeholders (al- though not all) to fix the evaluation on them. As we described briefly above, an evalu- ability assessment concentrates on the resources allocated to the program to determine whether they are sufficient to achieve the results intended. Such a perspective is critical not only to the organization—so that decision makers may understand any limitations of resources uncovered and hopefully address the situation—but also to the type of evaluation that may ultimately be conducted. Unless resources are in place to make the achievement of program effects for clients conceivable, undertaking more ambi- tious forms of evaluation (e.g., assessment of impact) makes little sense. Other forms of evaluation center on the conversion process of program inputs, to activities, to out- puts. Because these forms look primarily on the operation of the program rather than its results, they are typically termed “process evaluations.”

By way of example, consider a program that enlists volunteers and paid staff in mentoring middle school students, with the goal of improving school attendance and academic performance. The inputs or resources allocated to the program might con- sist of a single, full-time paid staff person and a large number of part-time volunteers, and some amount of budgetary resources. In addition to the key mentoring activities provided by volunteers, other activities undertaken by the program may consist of recruitment, orientation, placement, learning, supervision, and assessment of the vol- unteers. Outputs may encompass the number of matches of volunteer mentors and student mentees, the number of meetings between the mentors and mentees, the types of activities that they pursue in these relationships, and the amount of time de- voted to them.

An evaluability assessment would concentrate on the question of whether the re- sources allocated to the program—paid, volunteer, budgetary, and otherwise—were sufficient to operate the program. It would consider the relationship between the inputs available to the program in light of the extent of interest in the program (as reflected, for example, in the number of students seeking a mentor) as well as the program features that must be in place to offer mentees a quality experience. Such an assessment would scrutinize the balance between program resources on the one hand and program demands on the other in terms of the volume of prospective mentees and the activities necessary for their welfare. A process evaluation would consider the inter-relationship among resources, activities, and outputs. It might ask, for example, whether: the resources allocated to the program were used most efficiently; the train- ing offered to mentors was of high quality; the operations of the program were suffi- cient to generate, orient, and place qualified volunteers with mentors; and paid staff and volunteers were organized and coordinated in a way to achieve most outputs (e. g., mentor-mentee matches). Although these elements do not pertain to program out- comes—the changes or results realized by clients (here, middle school students)—the questions (and answers) are of interest to certain stakeholders, particularly program managers, organizational decision makers, and the volunteer mentors.

Outcomes can be understood as the effects of a program on its targeted clientele, especially changes anticipated and desired by the sponsoring organization. As opposed to the other elements of the logic model considered thus far, outcomes are not under the control of the program or organization. Rather, they represent

Logic Models 381

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

changes or effects to be realized in the external environment or audience of clients and communities. They are more ambitious both to achieve and to measure. Out- comes are related to the overall long-term impact of a program, but they have a more specific, short-term focus on immediate changes or results for clients (Safrit, 2010). Even so, as Poister (2003), McDavid and Hawthorne (2006), Wholey et al. (2004), and others point out, outcomes are normally categorized as initial, inter- mediate, and long-term. If the evaluator has done her or his job correctly, the out- comes will correspond to the objectives of the program, i.e., they will measure the achievement of the goals that are sought for the program in the client population or community.

Returning to the example of a volunteer mentor program guided by goals of improving school attendance and performance of middle-school students, we might conceive of initial outcomes centering on the mentor-mentee relationship. For exam- ple, does the client (student) appreciate the mentor and attend to the suggestions of the mentor? With respect to intermediate range outcomes, we might look for more behavioral indicators (measures) of the positive effects of mentoring: Do the student mentees participate in school activities, such as recognized groups or associations? Do they devote more time to their studies? Finally, in the longer term, the outcome mea- sures might comprise attendance and grades: Do the students show improvement in school attendance and grade reports? By contrast, the intended impact of the program might be a change in school culture to incorporate greater emphasis on student learn- ing, participation, and achievement. Although the initial, intermediate, and longer- term program outcomes would contribute to this impact, the impact is much more broad, far-reaching, and difficult both to attain and measure.

Summative Evaluations

Evaluations that attempt to link program resources and activities to outcomes (or im- pact) are usually termed “summative” evaluations, perhaps because they attempt to sum up the end results or achievements of a program or initiative. To prepare a sum- mative evaluation, the volunteer resource manager as evaluator must obtain data from or about the intended client group regarding outcomes (as well as background factors concerning the clients). The data collection effort should not be minimized: Develop- ing the necessary measurement strategy and instruments can be quite time-consuming and require significant background in measurement. Direct measurements from cli- ents are frequently sought in evaluations, such as interviews, focus groups, and sur- veys; supplementing such information with other data sources is highly desirable. Access to organizational records, for example, can be very valuable.

In the example of the volunteer mentoring program, the relevant data might ema- nate from independent surveys of the middle school students and the volunteer men- tors; it might also prove possible to conduct some in-depth interviews. In addition, official school records pertaining to student attendance and grades would be highly valuable additions to the store of data to evaluate the effects on the target population. Preparation of the data collection instruments will be guided by the need to obtain outcome information.

Summative evaluations incorporate a research design component. The topic of research design is both vast and technical, and we shall provide only a brief

382 Evaluating the Volunteer Program

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

introduction here. A research design specifies the methodology, or set of steps or pro- cedures utilized, to test anticipated causal relationships or hypotheses so that the re- sults can be accepted as valid (Meier, Brudney, & Bohte, 2012). In our now-familiar example of the volunteer mentoring program, the hypotheses are that the program will improve student attendance and grade reports. The research design must detail how we can arrive at valid or credible conclusions concerning these hypotheses. To do so, in general, the research design will call for a comparison or benchmark to eval- uate whether or not the program has succeeded in making an improvement in the outcome measures. Typically, the bases for comparison are either in reference to data collected at a point earlier in time for the same group of subjects, or with respect to information collected from a control or comparison group chosen to be as similar as possible to the program group, or both.

Thus, in our example, the research design might call for a collection of outcome data on student attendance and grade reports for those in the mentor program over time—ideally, data collected before the students were involved in the program and then again after their involvement. Comparison of these two sets of measures can be used to detect improvement (over time) in the outcome measures of attendance and grade reports. Alternatively, the research design might specify collecting and compar- ing data on the outcomes for the group of students in the mentor program (experi- mental or treatment group) versus a comparable group of students who do not have mentors but who are otherwise similar to the mentored students (control or compari- son group). This comparison, too, would allow the evaluator to identify improvement, in this case between the two groups of students (i.e., mentor program versus control group). Either form of comparison, over-time measurement or access to a control group, allows test of the hypotheses that the mentor program led to improved student performance in the outcome variables of school attendance and grade reports. Testing these comparisons based on data will require some familiarity with basic statistics (Meier et al., 2012).

Note that the logic model approach allows the volunteer resource manager as evaluator to assess the achievement of outcomes in a volunteer program or nonprofit organization without reference to their economic cost or valuation. This observation is important, especially in light of the differing opinions presented above in regard to the value of economic information for an evaluation of a program that includes a signifi- cant volunteer component. Based on the logic model framework, the volunteer re- source manager can use the findings from the outcome evaluation to demonstrate that the program helps the organization to achieve worthwhile objectives. In addition to program managers, clients, and volunteers, other stakeholders, such as the pubic, funders, and organizational leaders and board members, would likely be interested in evaluation results that showed, for example, that a volunteer mentor program had achieved desirable outcomes, such as improvements in the attendance of middle school students and their grade reports. Accordingly, the volunteer resource manager can use this information to press for greater recognition and support of the volunteer program.

It is quite possible, moreover, to supplement the results of the outcome evaluation with economic valuation of the volunteer program and its results. Such an enhance- ment can prove even more persuasive for the volunteer resource manager in making the case for the program to relevant stakeholders. If some form of cost/benefit

Logic Models 383

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

analysis of the volunteer program were necessary or desired, for example, the methods elaborated on economic valuation in the sections above can be put to good use. On the cost side, through maintaining records and files, organizations can usually estimate the expenses incurred by a department, program, or initiative. On the benefit side, al- though the positive effects of program or organizational activity are much more elusive to comprehend and measure, this chapter has presented methodologies for economic valuation of volunteer participation for organizations, clients, and the volunteers them- selves, as well as some approaches to impact assessment. Based on these methods the volunteer resource manager as evaluator can venture considerably beyond the ques- tion of whether improvements have been achieved in the client population to incorpo- rate a sophisticated analysis of the associated economic costs to the organization and benefits to various stakeholder groups.

Conclusion

Effective evaluation combines the rigor of the best methodological techniques with the realities of organizational context. It represents a rich blend of science and politics. The volunteer resource manager as evaluator requires some fluency in both domains. The evaluator seeks to provide methodologically valid answers to the questions that she or he believes the relevant stakeholders have posed for the program or the organization.

The earlier portions of the chapter are devoted to the organizational context. They point out that no single evaluation is likely to meet the information needs of the myr- iad stakeholders to a volunteer program, including funders, agency leadership, board of directors, program managers and other staff, volunteers, clients, and even public decision makers, citizens, and the media. Thus, choices will have to be made concern- ing evaluation purposes and stakeholders.

The volunteer resource manager as evaluator need not, and should not, guess or make assumptions about the stakeholders crucial to the evaluation process, their in- formation needs, and overall purposes. Instead, she or he should be much more pro- active in inquiring about the purposes of the evaluation and the stakeholders most invested in it. The volunteer resource manager should involve these groups in formu- lating the purposes for the evaluation, the questions to be answered by it, and the corresponding evaluation methodologies. This consultation should continue to encompass issues of measurement. If, as is frequently the case, the evaluation is to assess program results, then relevant stakeholders should participate in identifying the crucial measures of outcomes and impact to be used in the study. For the volun- teer resource manager as evaluator, turning their (stakeholders’) measures into our measures for an evaluation team helps to secure buy-in to the study and its results.

The consultation process is useful, too, to ferret out hidden agendas (i.e., situa- tions in which certain stakeholders hold pre-determined views of the worth of a pro- gram, initiative, or organization that they seek only to reinforce, and not disturb, through evaluation). Such ritualistic evaluations are enervating to organizational actors and resources and fail to advance knowledge. Bringing the relevant stakehold- ers on board prior to the evaluation can be effectual in identifying such covert pur- poses and, it is hoped, committing those involved to a genuine openness to the

384 Evaluating the Volunteer Program

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

evaluation questions and confidence in the results. These efforts require sensitivity to the organizational environment and central actors on the part of the volunteer re- source manager as well as political savvy.

Also needed for conducting evaluations are skills in methodological techniques, and the later sections of the chapter have provided an introduction to them. Most of these procedures turn on the ability of the volunteer resource manager as evaluator to incorporate into the analysis economic valuation of volunteer-based services for the organization, clients, and the volunteers themselves. Not all approaches are oriented to economic valuation, though. The logic model framework depicts the volunteer pro- gram as a causal sequence linking inputs to the program, the activities it carries out, and the outputs it generates, with program outcomes and impact (i.e., measures of the goals sought by the program. The logic model need not use economic measures), but it does require some knowledge of research design and statistical analysis. Thus, the volunteer resource manager as evaluator requires some background in the rele- vant techniques.

This chapter has focused on the role of stakeholders in the evaluation process. Yet meeting the needs and demands of stakeholders for information is only part of the evaluation task. We encourage volunteer resource managers to conceive of evalu- ation more generally for their own purposes as bringing data to bear on questions about volunteer involvement that can increase its value to the organization and grab the attention of key decision makers. For example, where are the organizational op- portunities to extend volunteer efforts? How can volunteers be used more produc- tively? In what new initiatives might volunteers make a valuable contribution? Astute volunteer resource managers likely think about such questions. When they begin to use the tools of evaluation to address them more systematically, they will find that their answers gain more force and acceptance.

References

Andersen, P., & Zimmerer, M. (2003). Dollar value of volunteer time: A review of five estimation methods. Journal of Volunteer Administration, 21(2), 39–44.

Berk, R. A., & Rossi, P. H. (1990). Thinking about program evaluation. Newbury Park, CA: Sage.

Bingham, R., and Felbinger, C. (2002). Evaluation In Practice: A Methodological Approach (2nd ed.) Chatham, NJ: Chatham House Publishers.

Bowman, W. (2009). The economic value of volunteers to nonprofit organizations. Nonprofit Management and Leadership, 19 (4), 491–506.

Brown, E. (1999). Assessing the value of volunteer activity. Nonprofit and Voluntary Sector Quarterly, 28(1), 3–17

Brudney, J. L. (1990). Fostering volunteer programs in the public sector: Planning, initiating, and managing voluntary activities. San Francisco, CA: Jossey-Bass.

Cahn, E., & Rowe, J. (1992). Time dollars: The new currency that enables Americans to turn their hidden resource—time—into personal security and community renewal. Emmaus, PA: Rodale Press.

Carman, J.G., & Fredericks, K.A. (Eds.). (2008). Nonprofits and evaluation. San Francisco, CA: Jossey-Bass.

References 385

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

Chelimsky, E. (1997). The coming transformations in evaluation. In E. Chelimsky and W. R. Shadish (Eds.). Evaluation for the 21st century: A handbook. Ch 1, 1–24. Thousand Oaks, CA: Sage.

Crutchfield, J. (1962, May). Valuation of fishing resources. Land Economics, pp. 145–154. Ellis, S. J. (1996). From the top down: The executive role in volunteer program success

(revised ed.). Philadelphia, PA: Energize, Inc. Ellis, S. J. (1999). The dollar value of volunteer time. Focus on volunteering. (Kopykit,

2nd ed.). Philadelphia, PA: Energize, Inc. Ferris, J. (1984). Coprovision: Citizens time and money donation in public service pro-

vision. Public Administration Review, 44(4), 324–333. Fitzpatrick, J., Sanders, J., & Worthen, B. (2004). Program evaluation. Alternative

approaches and practical guidelines (3rd ed.). Boston, MA: Pearson Education. Fryar, A., Mook, L., Brummel, A., & Jalandoni, N. (2003). Is assessing a financial

value to volunteering a good idea? E-Volunteerism, 3(2). Retrieved from www.e-volunteerism.com/quarterly/03jan/03jan-keyboard

Gaskin, K. (1999). Valuing volunteers in Europe: A comparative study of the volunteer investment and value audit. Voluntary Action, 2(1), pp. 35–49.

Gaskin, K. (2003). VIVA in Europe: A comparative study of the volunteer investment and value audit. Journal of Volunteer Administration, 21(2), 45–48.

Goulborne, M., & Embuldeniya, D. (2002) Assigning economic value to volunteer ac- tivity: Eight tools for efficient program management. Toronto, Ontario: Canadian Centre for Philanthropy.

Graff, L. (2005). Best of all: The quick reference guide to effective volunteer involve- ment. Linda Graff and Associates Inc.

Hager, M. A., & Brudney, J. L. (2005). Net benefits: Weighing the challenges and bene- fits of volunteers. Journal of Volunteer Administration, 23(1), 26–31.

Handy, F., & Mook, L. (2010, November 8). Volunteering and volunteers: Benefit-cost analysis. Research on Social Work Practice. Retrieved from http://rsw.sagepub. com/content/early/2010/11/01/1049731510386625.full.pdfChtml

Handy, F., Mook, L., & Quarter, J. (2008). The interchangeability of paid staff and vol- unteers in nonprofit organizations. Nonprofit and Voluntary Sector Quarterly, 37(1), 76–92.

Handy, F., & Srinivasan, N. (2004). Valuing volunteers: An economic evaluation of the net-benefit of hospital volunteers. Nonprofit and Voluntary Sector Quarterly, 33(1), 28–54.

Hopkins, S. (2000). Researching VET and the voluntary sector: Dealing with ambigui- ties. Retrieved from Retrieved from www.eric.ed.gov/PDFS/ED470938.pdf

Horst, P., J.N. Nay, J.W. Scanlon, and J.S. Wholey. (1974). “Program management and the federal evaluator.” Public Administration Review. 34(4), July- August, 1974, pp. 300–308.

Hotchkiss, R. B., Fottler, M. D., & Unruh, L. (2009). Valuing volunteers: The impact of volunteerism on hospital performance. Journal of Healthcare Management, 34(2), 119–128.

Independent Sector. (2011). Value of volunteer time. Retrieved from www.independent sector.org/volunteer_time

Karn, G. N. (1982–1983). Money talks: A guide to establishing the true dollar value of volunteer time, part I. Journal of Volunteer Administration, 1 (Winter), 1–17.

386 Evaluating the Volunteer Program

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

Karn, G. N. (1983). Money talks: A guide to establishing the true dollar value of volun- teer time, part II. Journal of Volunteer Administration, 1 (Spring), 1–19.

McDavid, J. C., & Hawthorn, L. R. L. (2006). Program evaluation and performance measurement: An introduction to practice. Thousand Oaks, CA: Sage.

Meier, K. J., Brudney, J. L., & Bohte, J. (2012). Applied statistics for public and non- profit administration (8th ed.). Boston, MA: Wadsworth Cengage Learning.

Murray, M. (1994). How efficient are multiple in-kind transfers? Economic Inquiry, 32, 209–227.

Patton, M. Q. (1986) Utilization Focused Evaluation. (2nd Ed.) Beverley Hills, CA: Sage

Patton M. Q. (1997). Utilization-focused evaluation (3rd ed.). Beverly Hills, CA: Sage. Patton, Q. P., & Patrizi, P. (2005). Case teaching and evaluation. New Directions for

Evaluation 2005 (105–Special Issue: Teaching evaluation using the case method), 5–14.

Poister, T. (2003). Measuring performance in public and nonprofit organizations. San Francisco, CA: Jossey-Bass.

Posavac, E., & Carey, R. (1992). Program evaluation (4th ed.). Englewood Cliffs, NJ: Prentice Hall.

Quarter, J., Mook, L., & Richmond, B. (2003). What counts: Social accounting for nonprofits and cooperatives. Upper Saddle River, NJ: Prentice Hall.

Rehnborg, S. J., Barker, C., & Pocock, M. (2006, January–March). How much is an hour of volunteer time worth? Various methods to monetize the contributions of a volunteer’s time. E-Volunteerism, Electronic Journal of the Volunteer Commu- nity. 6(2), 14–19. Retrieved from www.e-volunteerism.com/quarterly/06jan/ 06jan-rehnborg

Ross, D. (1994). How to estimate the economic contribution of volunteer work. Ottawa, Ontario: Department of Canadian Heritage. Retrieved from www.nald. ca/fulltext/heritage/ComPartnE/estvole.htm

Rossi, P. H., Lipsey, M.W., & Freeman, H. E. (2004). Evaluation: A systematic ap- proach (7th ed.). Thousand Oaks, CA: Sage.

Rutman, L. (1984). Evaluation research methods: A basic guide (2nd ed.). Beverly Hills, CA: Sage.

Safrit, R. D. (2010). Evaluation and outcome measurement. In K. Seel (Ed.), Volunteer administration: Professional practice 313–317 Markham, Ontario: LexisNexis Canada.

Scriven, M. (1991). Key evaluation checklist. In M. Scriven (Ed.), Evaluation thesaurus, 204–210. Thousand Oaks, CA: Sage.

Simmons, W. O., & Emanuele, R. (2010). Are volunteers substitute for paid labor in nonprofit organizations? Journal of Economics and Business, 62(1), 65–77.

Smith, D., & Ellis, A. (2003). Valuing volunteering. Journal of Volunteer Administra- tion, 21(2), 49–52.

Talmage, H. (1982). Evaluation of programs. In H. E. Mitzel (Ed.), Encyclopedia of educational research (5th ed.) 592–611 New York, NY: Free Press.

Wholey, J. S. (1983). Evaluation and effective public management. Boston, MA: Little, Brown.

Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (Eds.). (2004). Handbook of practical program evaluation (2nd ed.). San Francisco, CA: Jossey-Bass.

References 387

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

Connors, T. D. (Ed.). (2011). The volunteer management handbook : Leadership strategies for success. John Wiley & Sons, Incorporated. Created from ashford-ebooks on 2022-05-20 10:19:35.

C o p yr

ig h t ©

2 0 1 1 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .