U1D1-28- Describe a public needs assessment that you have worked on as part of your job, etc. see details below. answer all questions
PART I Assessing Current Practices
Chapter 1 Contemporary Issues in Human Service Program Planning and Administration
Chapter Overview The purpose of this chapter is to introduce:
The major governmental and other initiatives that have prompted the need for program planning The relationship between the logic model and program planning How agencies and community-wide networks relate to each other in addressing social problems The steps that are involved in effectiveness-based program planning The elements of a program that are critical to measuring effectiveness
The following topics are covered in this chapter:
The Era of Accountability Designing for Monitoring, Performance Measurement, and Program Evaluation The Logic Model Community Focus The Issue of Effectiveness Assessing an Existing Program What Is Effectiveness-Based Program Planning? Using Effectiveness Principles to Understand Existing Programs
Defining Programs Assessing Diverse Populations Problem Analysis Needs Assessment Selecting a Strategy and Establishing Objectives Program Design Data Requirements for Performance Measurement Monitoring and Using Information Technology Program Evaluation Budgeting
Implementation Considerations Summary Review Questions
The Era of Accountability When it comes to planning, funding, and implementing human service programs at the federal, state, and local levels, times have changed dramatically over the last few decades. Since the concepts of accountability, measuring results, and cost-benefit analysis have emerged over the past few decades, human services have been brought under a good deal more scrutiny (Hatry, 2010). Instead of funding for service delivery, the focus shifted to funding for results and return on investment.
The Government Performance and Results Act (GPRA) of 1993 (Public Law No. 103-62; Office of Management and Budget, 1993), reaffirmed by the GPRA Modernization Act of 2010 (Public Law 111-352), has become a major driver of performance accountability at the federal level (Hatry, 2014). GPRA requires federal departments
and agencies to annually report their performance to the president and Congress. A second driver of performance accountability at the federal level is the performance contracting requirements of the Federal Acquisition Regulation (FAR). The FAR represents the formal contracting policies and procedures of the federal government. The FAR requires that “all federal service contracts (including human service contracts) be performance based to the maximum extent possible” (FAR, 2015).
At the state and local government levels, two major drivers of performance accountability are the reporting initiative of the Governmental Accounting Standards Board (GASB) and the performance measurement requirements imposed by governors and legislatures. GASB sets financial and reporting standards for state and local governments. GASB has long advocated that state and local governments adopt performance accountability systems that track and report on the outputs, quality, and outcomes of government programs. GASB (1993) calls its recommended performance accountability system service efforts and accomplishments (SEA) reporting. Private sector funding organizations, such as foundations and the United Way (1996), have also adopted performance accountability systems. Most nonprofit human service agencies that receive government, foundation, or United Way funding through grants and contracts have likewise adopted performance accountability systems in order to satisfy the reporting requirements of their funders (United Way, 2003).
In 2013 the federal Office of Management and Budget (OMB) issued new regulations governing federal grants (Martin, 2013). The new regulations require that, in order to receive federal funding in the future, grant recipients must specify performance and outcome measures. Additionally, some grant recipients may be paid for achieving performance expectations and results, not activities, and these must be certified in writing (U.S. Government Printing Office, 2015). If monitored and enforced, the new regulations will essentially require that the steps we describe in the following chapters be standard expectations in order to receive federal funding for human service programs. These requirements may create a tension between what the funding source expects and what the local program hopes to accomplish, so in creating data collection systems it is important to consider as many perspectives (e.g., client, staff, management, board, community interests) on the definition of effectiveness as possible.
Designing for Monitoring, Performance Measurement, and Program Evaluation Designing programs and services for effectiveness involves collection of the kinds of data that will support responsiveness to funding source mandates for accountability and at the same time will allow program evaluators to determine whether the programs are getting the desired results. This is an important point. If monitoring, performance measurement, and program evaluation information is anticipated at the end of service provision, this information will be available only if certain design elements are incorporated at the beginning of the planning process. The Urban Institute pioneered an effort to establish criteria by which programs could be assessed in terms of whether they could be evaluated. They called this effort “Evaluability Assessment.” Here are some of the criteria they specified in order for a program to be considered “evaluable”:
Evidence required by management can be reliably produced. Evidence required by management is feasible to collect. Management’s intended use of the information can realistically be expected to affect performance. (Schmidt, Scanlon, & Bell, 1979)
The term evidence refers to the collection of data around specified variables that define the details of service provision and results. The term evidence-based practice has come into prominent use in many professions, including social work, to emphasize that clinical decisions must be based on the best available evidence from systematic research wherever possible (Johnson & Austin, 2006; McNeill, 2006). Management’s use of evidence simply takes this concept to the next level, defining the parameters within which evidence will be collected, how it will be aggregated, and how it will be used for monitoring, performance measurement, and program evaluation. One thing should be made perfectly clear: Designing programs that can be evaluated means collecting quantified data on service provision and client response. It also requires that data be captured in a spreadsheet or database
format to facilitate analysis. Only when data are displayed in some way that permits comparisons can the program planner or analyst begin to understand what works and what doesn’t.
Figure 1.1 The Logic Model
The Logic Model As discussed above, government programs were focused for so many years on process or provision of services that it has become a long and arduous process to shift the conceptual framework to one that emphasizes outcomes and results. A framework that has been very influential and helpful in understanding the basic concepts associated with effectiveness-based planning is the logic model. Introduced in the 1960s and 1970s, it was intended to shorten what was then a very cumbersome process of evaluating federally funded programs (Wholey, 1983, 1994). The widespread adoption of the concepts was given a significant boost by the United Way of America’s publication Measuring Program Outcomes in 1996.The logic model borrows concepts from systems theory to create and build upon a foundation in a way that helps to see the relationships between and among the resources invested, the services provided, and the results achieved.
Effectiveness-based program planning uses program as the unit of analysis. The value of the logic model is that it permits us to depict the sequence or flow of events that identifies program resources, matches resources to needs, quantifies the service process, tracks completion of the service process, and measures results. The sequence is depicted in Figure 1.1.
This model allows the planner to see the rational flow of addressing a problem and applying a process, while maintaining a focus on the purpose of the entire effort: effecting positive changes in the lives of clients and reducing the size and scope of a problem in a community. Definitions and applications of inputs, process, outputs, outcomes, and impact will be addressed throughout this book to emphasize the importance of understanding the logical, conceptual flow of the program-planning process. While many examples will be provided, it is important to remember that when applied, the principles should be used as guiding concepts and not as a rigid formula.
Community Focus One result of the emphasis on accountability has been to shift the focus from what an agency is providing and accomplishing to what happens to a target population within a community as a result of having received a
particular set of services. Most problems addressed by human service programs are community problems, such as children in need of a safe environment, homeless people in need of shelter and rehabilitation, victims of crime and violence in need of protection, family breakdown, addictions, and others. Over the past few decades there has also been an increasing recognition that meeting the needs of various ethnic and special populations, including African Americans, Asian Americans, Latino Americans, Native Americans, gays, lesbians, bisexual and transgendered people, low socioeconomic-status people, and other special populations may require adaptations. Research findings on success with these populations are limited, and requirements for outcome-oriented approaches have the potential to begin to meet the need for more complete findings (U.S. Department of Health and Human Services, 2001).
The kinds of problems brought to human service agencies are so complex that it is unlikely that services from just one agency can ever have a significant and measurable impact on an entire community. For this reason, funding sources have learned that they must put resources (primarily money) into a community with the expectation that a number of agencies will collaborate in addressing the problem in the interest of achieving a more comprehensive impact.
This community-wide focus means that early in the planning and proposal writing stages of a program, planners need to study the problem and identify the characteristics of the target population. Characteristics should anticipate data elements that will be needed to answer questions that may arise about the effectiveness of the program. In the following chapters we will examine in great detail the many developmental phases a program must go through and the many elements that need to be included if it is to be considered effectiveness-based. We will also attempt to illustrate how agency information systems and agency budgets can be structured in a way that allows them to fit into a larger community system, thus enabling state- and community-level organizations to measure the impact on the problem.
Basic to all program monitoring, performance measurement, and evaluation is the need for appropriate information technology support in terms of computer hardware, software, and expertise capable of tracking clients and generating program reports. Yet to achieve the full benefits of effectiveness-based program planning, there must be a strong commitment from the top and throughout the staff that evaluation of effectiveness is critical to the success of any client service program and that evaluation data will be used to initiate continuous quality improvement in the program as it becomes clear that changes are needed.
The Issue of Effectiveness Both efficiency and effectiveness have become major considerations in human service program and agency administration. Efficiency can be defined as the volume of service a program provides in relation to its costs (how much service is the agency getting for its dollar?). Effectiveness refers to the achievement of client outcomes (quality-of-life changes) as the result of receiving services (how much better off are clients as a result of having received this service?). Measuring outputs and outcomes requires that service providers track such factors as the amount of service a client received, whether the client completed the program or dropped out, and how much the client improved between entry into and exit from the program.
The termprogram also has some very specific meanings. A program is defined as a prearranged set of activities designed to achieve a stated set of goals and objectives (Netting, Kettner, & McMurtry, in press). This is an important and sometimes elusive concept for a newcomer to the profession to grasp. All measurement relies on first defining the elements that will go into program construction. If there is an expectation that stated objectives will be met and that monitoring, performance measurement, and/or evaluation will take place at some point, then an agency must plan in advance to establish goals and objectives and to provide clear definitions of services to be provided and results expected (all, of course, in writing). Client requests for help must be categorized in some way, after which clients are directed into services that offer the best possibility of meeting their needs and resolving their problems.
Figure 1.2 Organizational Chart Displaying Programs
The Commonwealth of Australia has done a particularly good job of applying the concept of effectiveness-based program planning for government-sponsored services. For example, in defining police services, rather than simply operating under the generic heading of crime-fighting, law enforcement efforts are broken down into four areas or programs: (1) community safety and support, (2) crime investigation, (3) road safety and traffic management, and (4) services to the judicial process (Steering Committee for the Review of Government Service Provision, 2006). The point is that services need to be categorized, defined, matched to client need, and delivered to determine whether they have been successful. Data collection systems are closely tied to program definitions. When programs have been conceptualized and have a set of goals and objectives that they are expected to achieve, then they ought to have visibility on the organizational chart, as depicted in Figure 1.2, and have staff and resources assigned to them.
Programs may be staffed by specialists who work exclusively with a defined population or problem, or staff responsibilities may cut across more than one program. The important issue is that client data must be associated with specific programs and services so that valid and reliable measures are produced.
Assessing an Existing Program In this opening chapter we will attempt to illustrate that designing effective programs requires a careful, detailed thought process that begins with an understanding of a social problem and ends with analysis of data on effectiveness. Chapters 2 through 6 will focus on assessing and understanding problems and needs. Chapters 7 through 9 will present a step-by-step process on writing goals and objectives, designing programs, and collecting and using data for analysis of efficiency and effectiveness. Chapters 10 through 13 will explain alternative budgeting and evaluation systems.
The tasks and processes of program development that will be proposed are by their very nature complex, simply because social, family, and personal problems are complex and sometimes well entrenched. The problems and the populations to be served will require thoughtful study and analysis. The purpose of delving into the complexities of social problems and human service programs is to ensure that there is a good fit of service to need. When this happens, services can be more precisely focused on getting the kinds of results intended. In short, it is a more proactive approach that is more assertive in ensuring service providers produce results rather than merely hoping that things turn out well for the client.
Perhaps in the same way that an understanding of the law is of critical importance to a practicing attorney or an understanding of the body is important to a physician, so an understanding of social problems and programs is central to the practice of social work planning and administration. This understanding will require that old assumptions be challenged and new approaches to serving clients be implemented as we learn more about effectiveness. In a sense, we will be proposing that programs go through periodic checkups, including data analysis, use of benchmarks, and comparison to best practices and the results they are achieving. These indicators should be helpful in determining a program’s continuing effectiveness and relevance in a changing environment. Failure to convert to these types of data and information becomes much like competing in an athletic competition without keeping score.
What Is Effectiveness-Based Program Planning? The idea of conducting periodic checkups is, in essence, what effectiveness-based program planning is all about. Designing effective programs requires that social service professionals develop a thoroughly researched understanding about social problems, people in need, and human services. A commitment to effectiveness requires the collection of new kinds of data—data that will provide information about client conditions at entry into and exit from services, thereby making clear the impact of services on problems. This approach will make
clear what is working and where changes in programs are needed so that services provided do more of the things that help and fewer of the things that do not. The system is designed to be useful for both direct service and management purposes.
Effectiveness-based program planning involves taking a program through a series of steps designed (1) to produce a clear understanding and definition of the problem to be addressed, (2) to measure client problem type and severity at entry, (3) to provide a relevant intervention, (4) to measure client problem type and severity at exit, and (5) to examine selected indicators in a follow-up study to determine short- and long-range outcomes. The purpose of all of these activities is to provide a basis for continual improvement of services to clients and to provide a common database for both clinical and administrative staff for analysis and decision making about program changes. This way, instead of asking clinicians to fill out forms useful only for completing management reports, clinical staff can record data useful for understanding the progress of their clients and, at the same time, provide data and information necessary to good program management.
Using Effectiveness Principles to Understand Existing Programs In the following chapters we will present a step-by-step process that will enable the reader to begin with a social, family, or personal problem experienced by clients, end with a program designed to get results, and be able to document them. Before beginning these chapters, however, we propose that you take a few minutes to take stock of current agency practices in a human service agency with which you are familiar as a means of becoming acquainted with some of the basic concepts of effectiveness-based program planning. This may be helpful in drawing a contrast between the way human service programs are often designed and the way they must be designed for measurement purposes. The purpose of this exercise it to introduce you to the concepts and issues that are critical to understanding effectiveness-based program planning. The remaining chapters will be devoted to explaining these in detail.
The planning process can be divided into sections on each of these topics:
Defining programs Assessing diverse populations Problem analysis Needs assessment Selecting a strategy and establishing objectives Program design Data requirements for performance measurement Monitoring and using information technology Program evaluation Budgeting
It should be evident at this point that each step of the planning process corresponds, in a way, to the sequence depicted in the Logic Model. Figure 1.3 illustrates these relationships. Note how much of the planning process is devoted to understanding and analyzing program inputs. If this part of the process is approached methodically, the rest should flow logically, as will be explained in the following chapters.
A program assessment instrument is provided online at study.sagepub.com/kettner5e. This instrument is designed to provide a quick overview of a program’s strong and weak areas. The program-planning model discussed is designed for those programs that provide a direct service to clients. It is not applicable for support programs such as fund-raising, lobbying, and advocacy.
Figure 1.3 The Logic Model Applied to the Planning Process
Defining Programs
An important first step in program planning is to be sure that the program is clearly defined. Services may be organized into programs that offer a specialized set of services to a defined population (e.g., detox, family counseling, employment services). Others may be designed in a way that all clients come through a common intake point and are systematically assigned to case managers who have room in their caseloads rather than to specialists. This is an important distinction in applying the principles of effectiveness-based program planning and one of the first elements of design that need to be assessed. The online questions on defining programs are intended to encourage you to think through where your agency stands on its definition and specialization of programs.
Assessing Diverse Populations
A surgeon general’s report published in 2001 identifies a number of disparities in mental health care for racial and ethnic minorities: (1) minorities have less access to services and are therefore less likely to receive needed services, (2) minorities often receive a poorer quality of mental health care, and (3) minorities are underrepresented in mental health research (U.S. Department of Health and Human Services, 2001). There is ample evidence that these disparities also apply to a wide range of health and human services to minority groups and other special populations (Lam & Sue, 2008). As states and communities attempt to address these disparities, program planners will need to take steps to ensure that programs and services are designed in a way that is inclusive and meets the needs of all populations.
Furthermore, many of the counseling professions have established standards that require that their members be inclusive and culturally competent in the provision of counseling and other services (Calley, 2011). While it is usually premature at this point to try to narrow down and define specific populations to be served, it is not too early to ensure that the complete range of possible target populations is taken into consideration in assessing needs. The online questions on assessing diverse populations should help to sensitize planners to all of the populations that are represented in the community so that they will be included in assessments and in planning for services.
Problem Analysis
In Chapters 2 and 3, we deal with the need for a thorough understanding of the theoretical underpinnings and the etiology (cause-and-effect relationships) of the problem(s) the program is intended to address. Sound practice requires that programs be based on a thorough study and analysis of the problem, but that is not always the case. Programs are sometimes planned and funded for political or ideological reasons without regard for the facts or
the realities of the situation. However, as we hope to demonstrate, those programs that have the greatest probability of success will be those that develop a clear understanding of the type, size, and scope of the problem as well as its relevant historical highlights, theory, research findings, and etiology.
For example, if a program is to be designed to treat drug users, it would be important to understand that people use drugs for different reasons, and treatment must be carefully tied to these reasons. Prescription drug abusers, junior high school kids experimenting with marijuana, and street gang members selling and using methamphetamines, for example, each need to be understood in context. Program planners, therefore, must set out to discover how many of each type are in the community, their reasons for using drugs, where they live, and how severe their problems are. Ethnic minority and other special populations also need to be identified and etiology examined to determine whether there are different factors that contribute to the problem or need. This approach provides a solid foundation on which to build an effective and precisely targeted program or intervention. Online questions on problem analysis are designed to help understand the extent to which a given program is incorporating background information into its understanding of the problem.
Needs Assessment
When someone is experiencing a problem, that individual has a need. Sometimes the need is obvious: Someone who is homeless needs a home; someone who is unemployed needs a job. At other times the need is more subtle and more difficult to meet—for example, the need for a permanent and loving relationship with a nurturing parent substitute, the need for a mentor to help build self-confidence, or the need to learn a work ethic in order to succeed in employment.
Accuracy and skill in matching needs to services comes from thorough study of the problem. Once you are comfortable that you have an understanding of need, it is time to turn to techniques of needs assessment. There are four different perspectives from which we look at need: normative need (as defined by experts in the field), perceived need (as seen by those experiencing the need), expressed need (from those who seek out services), and relative need (needs and resources in one geographic area compared with needs and resources in another; Bradshaw, 1972). The online questions on needs assessment will help you understand each of these perspectives on need and think through the extent to which your programs have taken these perspectives into account.
Selecting a Strategy and Establishing Objectives
Once the problem analysis and the needs assessment have been completed, it is time to begin to think about a strategy for reducing or eliminating the problem by meeting the identified needs. This involves a number of steps. By this point in the program-planning process, we are well grounded in history, theory, research, and etiology of the problem and population and are in a position to propose an appropriate intervention. This involves proposing one or more program hypotheses—statements about what outcomes are expected if a person with the problems we have defined receives appropriate service(s). Program hypotheses, then, provide a framework for the development of precisely stated goals, objectives, and activities. The online questions should help in assessing your understanding of a program’s underlying assumptions and expectations, and the fit of need to the proposed services.
Program Design
It is one thing to understand a need; it is quite another matter to design an intervention that will meet that need. Research in the field has made it clear that certain problems will respond better to certain, more precise interventions (e.g., Abreu, Consoli, & Cypers, 2008; Morris, 2008; Weaver, 2008). The purpose of the program design phase is to put together that service or combination of services that appears to have the best possible chance of achieving the program’s objectives. Program design involves careful consideration of the resources needed to address the needs of clients and attention to the ways in which these resources will be organized. It is a critical point in the planning and management of programs.
If we simply consolidate a great deal of program design under the heading of “casework,” we leave decisions about client assessment, service provision, service completion, and outcome assessment to the professional judgment of each caseworker. When this happens, it becomes difficult to establish precise objectives, to examine program effectiveness, and to modify program design in the interest of improving services to clients. On the other hand, bringing precision to each element of program design allows for constant examination and constructive program change as data and information about effectiveness become available to guide our refinements. The online questions should help you assess the level of precision achieved in specifying the elements of your program design.
Data Requirements for Performance Measurement
Data collection is the sine qua non of effectiveness-based program planning. All the effort put into the development of a program hypothesis, goals and objectives, and design will mean little if the correct data are not collected, aggregated, analyzed, reported, and used to make adjustments in the next cycle of services. Data collection systems must be designed to (1) answer questions about meeting community need, (2) identify diverse populations, (3) measure how much each has been helped, (4) measure the achievement of outputs and outcomes, and (5) calculate the costs associated with process, output, and outcomes. Principles associated with performance measurement should be understood before attempting to design an information system. The online questions may be useful in understanding the data requirements of effectiveness-based program planning.
Monitoring and Using Information Technology
Once program data elements have been designed and implemented in accordance with the guidelines established for effectiveness-based program planning, they can be collected, processed, and aggregated in a manner that informs both clinical staff and administrators. Programs can be said to meet objectives and to bring about positive changes in clients’ lives only if the data generated from the program provision process can support such statements.
In contemporary human service agency and program management, computerized data management is absolutely essential. Narrative case recording is useful for individual case analysis, planning, supervision, and documentation, but it is of very little use for purposes of measuring effectiveness. In effectiveness-based program planning, we propose a client data system that is capable of producing data and information about the progress of clients throughout each episode of service and the effects of these services at termination and follow-up. This information, we believe, should be used by all levels of staff, each from its own perspective. The online questions may be useful in assessing the strengths and weaknesses of an existing monitoring system.
Program Evaluation
One of the most exciting features of effectiveness-based program planning is that it produces information that informs staff about how successful the program was in relation to expectations as expressed in objectives. How many abusing and neglecting parents completed parent training? How many can demonstrate improved parenting skills? How many have stopped abusing and neglecting and are progressing toward more effective relationships with their children? How effective are these methods across all ethnic populations? This information can bring together direct service staff, supervisors, managers, administrators, and board members around a common set of concerns and interests. It is always more satisfying to be able to say, at the end of a program year, “We helped 75% of our clients master at least 10 techniques of effective parenting” than simply to be able to say “We provided services to 100 abusing and neglecting families.” Furthermore, the database produced can provide the raw material for an ongoing research and development function within the agency, a function usually reserved for only the wealthiest organizations. The online questionnaire explores whether various methods of evaluating human service programs are currently being used.
Budgeting
All programs and services depend on funding for their start-up and continuation, and for many funding sources there are no guarantees that the same level of support will continue year after year. It is, therefore, in the interests of clients and staff to ensure that the best possible results are being achieved for the lowest possible cost. A well- designed budgeting system is capable of generating important and valuable information for use in making program changes in the interest of providing better quality services for clients at a lower cost.
Unfortunately, many budgets in human service agencies reflect only categories for which dollars are to be spent. These are called line-item budgets. In effectiveness-based program planning, we propose—instead of or in addition to this simplistic type of budgeting—methods for calculating costs for items such as provision of a unit of service (e.g., an hour of counseling), completion of the full complement of prescribed services by one client (e.g., 10 parent-training sessions), achievement of a measurable outcome by one client (e.g., improved parenting skills), and achievement of a program objective (e.g., at least a 50% reduction in child abuse reports on those who complete the class).
For example, by costing out services, we may learn that it costs $1,500 per trainee to complete a training program. However, if we also find that there is a 50% dropout rate, the cost then doubles to $3,000 per “graduate.” These kinds of calculations help staff keep focused on using resources in a way that steers clients in the direction that offers them the best possible chance of success at the lowest cost. These types of calculations should ultimately lead to more cost-effective and cost-efficient operation of human service programs.
The online questions should help in assessing the strengths and weaknesses of your current budgeting system. These questions are intended to provide a very general overview of a particular program in terms of its fit with the principles of effectiveness-based program planning.
The following chapters are intended to explain each of the phases of effectiveness-based program planning in detail. As you proceed through these chapters we encourage you to think through and apply the concepts to a specific program. While the most ideal application of these principles is in designing new programs, you may also find that existing programs can be converted with careful attention to the details of each phase of the planning process.
Implementation Considerations We try in this book to make the case for performance measurement, monitoring, program evaluation, and precise forms of budgeting. We will cite many federal, state, and local mandates as well as ongoing efforts to develop and use these types of data collection and reporting. As the planning process evolves, it will be clear that what is being presented in this book and the requirements generally associated with this topic can be time-consuming and expensive. Carillo (2005) found that management information is greatly underutilized in human service agencies, and Hatry (2012, p. 23) suggests that, “overall, the feasibility of undertaking full, in-depth program evaluations in more than a very small percentage of HSOs [human service organizations] is unlikely.” He cites several reasons for this, including questions about the validity and reliability of data, compatibility between outcome measurement data and program evaluation data, and the quality and comparability of data across all levels of government and private agencies. That is not to say, however, that effectiveness-based planning is not important or that steps can’t be taken or progress made toward the goal of monitoring and measuring program performance.
Of critical importance is the culture and mind-set of the agency, including a commitment from the top administrators to entry-level workers to the collection and use of data to help clients resolve problems in the most effective and efficient way. Enough has been said about the need for leadership in developing performance management systems that we feel safe in stating that, without strong direction from the top, there is little chance that the necessary conversions can be successful (see, e.g., Packard & Beinecke, 2012).
Another important issue is the amount of and resources (including funding, computer hardware and software, technology consultation, etc.) available to the thousands of small, independent human service agencies around the country and the world. While it may be feasible for large federal, state, county, and private agencies with
extensive and specialized staffs to undertake a conversion to the kind of performance measurement, monitoring, and program evaluation that we describe, many agencies do not have these resources or capacity.
Often these small agencies are serving in highly specialized areas with underserved populations and meeting a critical need. Some even avoid applying to funding sources because of the requirements associated with performance measurement and program evaluation.
If the commitment is there from board, executive, middle management, and staff, many agencies have demonstrated that strategies can be developed through the use of volunteers, interns, existing staff, consultants, and other resources, dividing the process into three phases: preplanning, defining, and implementing. Not all phases require the same resources. Smaller agencies with limited resources may be able to make the necessary changes without a great up-front investment, embarking on a pre-planning effort and postponing expenses such as outside consultants until they achieve the staff readiness and have the resources to complete the process.
Summary Planning programs designed to achieve results is a very complex process. For human service professionals who have, for many decades, built programs around process, working with the concepts of results or outcomes requires some radical shifts in thinking. Yet it is completely understandable why various levels of government, the United Way, and other grant and contracting organizations need to be able to calculate the value of investments in programs in terms of results. Fortunately, the logic model provides a clear flow of phases in the program development process, so that if it is carefully operationalized and followed, program planners will be able to measure results, costs, and other important indicators. This chapter attempts to lay out the steps in that process and to define the elements critical to measurement. An online questionnaire is designed to guide the program planner through the various phases of effectiveness-based program planning as applied to an existing program.
This analysis is intended to introduce the phases and the concepts, with the understanding that each step in the process will be covered in more detail in a later chapter.
Review Questions 1. What are the critical differences between the old focus on process in human services and the more recent
focus on outcomes? Why are all levels of government concerned about this? 2. How does the logic model help in understanding and applying the elements of effectiveness-based program
planning? 3. Define and give examples of efficiency and effectiveness in relation to a particular program of service with
which you are familiar. 4. Why is it important to identify target populations as part of the planning process? 5. Identify some measurable outcome indicators for a program or service with which you are familiar.
References Abreu, J., Consoli, A., & Cypers, S. (2008). Treatment issues with Hispanic clients. In G. Gamst, A. Der-
Karabetian, & R. Dana (Eds.), Readings in multicultural practice (pp. 181–198). Thousand Oaks, CA: Sage.
Bradshaw, J. (1972). The concept of social need. New Society, 30, 640–643.
Calley, N. (2011). Program development in the 21st century: An evidence-based approach to design, implementation, and evaluation. Thousand Oaks, CA: Sage.
Carillo, T. (2005). Management information systems: Why are they underutilized in the social services? Administration in Social Work, 29(2), 43–61.
Federal Acquisition Regulation (FAR). (2015). Retrieved from http://www.acquisition.gov/far
Government Accounting Standards Board. (1993). Proposed statement of the Governmental Accounting Standards Board on concepts related to service efforts and accomplishments reporting. Norwalk, CT: Author.
Hatry, H. (2010). Looking into the crystal ball: Performance management over the next decade. Public Administration Review, 70(Suppl. 1), s208–s221.
Hatry, H. (2012). Outcomes measurement in the human services: Lessons learned from public and private sectors. In J. Magnabosco & R. Manderscheid (Eds.), Outcomes measurement in the human services: Cross- cutting issues and methods (2nd ed., pp. 17–29). Washington, DC: NASW Press.
Hatry, H. (2014). Transforming performance measurement for the 21st century. Washington, DC: Urban Institute.
Johnson, M., & Austin, M. (2006). Evidence-based practice in the social services: Implications for organizational change. Administration in Social Work, 30(3), 75–104.
Lam, A., & Sue, S. (2008). Client diversity. In G. Gamst, A. Der-Karabetian, & R. Dana (Eds.), Readings in multicultural practice (pp. 47–57). Thousand Oaks, CA: Sage.
Martin, L. (2013). The OMB supercircular: Grant reform or something more? Human Service Organizations: Management, Leadership & Governance, 38, 394–404.
McNeill, T. (2006). Evidence-based practice in an age of relativism: Toward a model for practice. Social Work, 51(2), 147–156.
Morris, E. (2008). Clinical practices with African-Americans: Juxtaposition of standard clinical practices and Afrocentrism. In G. Gamst, A. Der-Karabetian, & R. Dana (Eds.), Readings in multicultural practice (pp. 199– 216). Thousand Oaks, CA: Sage.
Netting, F., Kettner, P., & McMurtry, S. (in press). Social work macro practice (6th ed.). Upper Saddle River, NJ: Pearson.
Office of Management and Budget. (1993). Government Performance and Results Act of 1993. retrieved from http://www.whitehouse.gov/omb/mgmt-gpra/gplaw2m
Packard, T., & Beinecke, R. (2012). Leadership in the human services: Models for outcomes-driven organizational cultures. In J. Magnabosco & R. Manderscheid (Eds.), Outcomes measurement in the human services: Cross-cutting issues and methods (2nd ed.). Washington, DC: NASW Press.
Schmidt, R. E., Scanlon, J. W., & Bell, J. B. (1979). Evaluability assessment: Making public programs work better. Washington, DC: U.S. Department of Health, Education and Welfare.
Steering Committee for the Review of Government Service Provision. (2006). Report on government services 2006. Canberra, Australia: Productivity Commission.
United Way of America. (1996). Measuring program outcomes: A practical approach. Alexandria, VA: Author.
United Way of America. (2003). Program outcome measurement: Outcome measurement in national health and human services and accrediting organizations. Alexandria, VA: Author.
U.S. Department of Health and Human Services. (2001). Mental health: Culture, race, and ethnicity. A supplement to mental health: A report of the surgeon general. Rockville, MD: Author.
U.S. Government Printing Office. (2015). Part 200—Uniform administrative requirements, cost principles, and audit requirements for federal awards. In Electronic Code of Federal Regulations. Retrieved from http://www.ecfr.gov/cgi-bin/text-idx?tpl=/ecfrbrowse/Title02/2cfr200_main_02.tpl
Weaver, H. (2008). Native Americans. In G. Gamst, A. Der-Karabetian, & R. Dana (Eds.), Readings in multicultural practice (pp. 217–239). Thousand Oaks, CA: Sage.
Wholey, J. S. (1983). Evaluation and effective public management. New York, NY: Little, Brown.
Wholey, J. S. (1994). Assessing the feasibility and likely usefulness of evaluation. In J. S. Wholey, H. P. Hatry, & K. E. Newcomer (Eds.), Handbook of practical program evaluation. San Francisco, CA: Jossey-Bass.