U7A1-28 - **TUTOR IT IS REQUIRED THAT YOU FOLLOW ALL INSTRUCTIONS AS OUTLINED. - SEE ALL DETAILS BELOW*** Planning, Designing, Foundation, Tools, and Strategy Action Planning, Designing, Theoretical Foundation, Data Gathering Tools, and Intervention Stra
4/20/2018 Bookshelf Online: Designing and Managing Programs
https://online.vitalsource.com/#/books/9781483388328/cfi/6/46!/4/2/4/2@0:0 1/16
Chapter 8 Designing Effective Programs
Chapter Overview The purpose of this chapter is to explain:
The rationale for breaking programs into elements The need for a precise definition for each element The logic model and the ways in which it provides a framework for program design How the program hypothesis and program objectives fit together with the elements of program design
The following topics are covered in this chapter:
The Significance of Program Design Program Design and Diversity Defining the Elements of a Program
Inputs Throughputs Outputs Outcomes
Specifying the Program Hypothesis Summary Case Example Review Questions
The Significance of Program Design Program design refers to identifying and defining the elements that go into the delivery of a service. It is at this point that the program planner gets highly specific about what types of clients with what types of problems or needs will receive precisely what types of services, in what volume, and with what expected result. To put it into perspective, it is important to understand a bit about the history and development of the provision of social services to clients.
For many years, the whole notion of program design was somewhat taken for granted in planning programs. For the vast majority of human service programs, program design involved simply hiring caseworkers, assigning clients, and instructing the workers to provide whatever type of casework services were called for, depending on the program. Most decisions about services were left to the discretion of the caseworker. Although this approach is simple in its design and allows maximum flexibility for practitioners, it fails to deal with critical questions of relevance of services provided, accountability, and measurement of effectiveness, and fails to take advantage of research knowledge and best practices. Let us explore these issues by contrasting two programs for elderly, isolated seniors.
What differences do these designs make? The answer to that question depends on the intent of the program and whether measurement of effectiveness is an issue. Program A clearly provides for a stronger and more comprehensive relationship between caseworker and client. Program B, however, is clearly superior when effectiveness and accountability are the issues. Program B’s design is based on a hypothesis that the major barriers to full participation for isolated elderly include transportation and socialization and recreation opportunities, including congregate meals. Regular analysis of the data will enable Program B’s staff to discover whether this hypothesis is correct and to make adjustments as needed to make the program more effective.
The major differences between these programs are differences of precision, specificity, and detail. Program A caseworkers may also discover that socialization, recreation, and congregate meals are important factors in success for isolated elderly, but each caseworker would have to make this discovery independently and decide to act on it. For Program B, the problem analysis produced the findings, and the goals and objectives established the direction. Let us examine how planners, using effectiveness-based, program-planning principles, move from goals and objectives to program design.
Program Design Illustration Program A
4/20/2018 Bookshelf Online: Designing and Managing Programs
https://online.vitalsource.com/#/books/9781483388328/cfi/6/46!/4/2/4/2@0:0 2/16
Program A has five caseworkers, each of whom carries 30 cases. Caseworkers decide how often to meet with clients. When they meet, they attempt to determine needs and to find resources in the community to meet those needs. After each session, caseworkers write narrative recordings of case activity in the case record. When a client completes Program A, a caseworker writes a narrative explanation of why the client is no longer in the program and describes the client’s progress and status as of the point of termination.
Program B
Program B is structured differently. Program B has an intake and screening worker, two case managers, a meal coordinator, a transportation coordinator, and a socialization and recreation specialist. Program B staff identify at-risk seniors in the community, transport them to the Senior Center, screen and assess to determine unmet needs using scaled instruments, and build a case plan designed to meet unmet needs to the greatest extent possible, including (at a minimum) a daily noon meal and transportation to and from the Center.
On completion of Program B, a case manager administers a posttest on nutrition, social isolation, and general mobility and secures agreement from the client to participate in a 6-month follow-up evaluation. In Program B’s case records, case managers record test scores, itemize barriers, and code the special units from which each client receives services. The case manager then enters all data into the computer. Aggregated data reveal monthly patterns of participation and improvement on the part of clients.
Program Design and Diversity There is now a wealth of research and scholarly literature that documents the need for specialized interventions based on ethnicity, gender, socioeconomic status, sexual orientation and other variables. To cite just one example out of thousands of texts, Readings in Multicultural Practice (Gamst, Der-Karabetian, & Dana, 2008) includes readings on Hispanic clients; African Americans; Native Americans; Asian Americans; older adults; gender issues; lesbian, gay, and bisexual people; the poor; and the disabled. A one-size-fits-all approach to program design is clearly no longer viable in many types of services. Few if any current population-based studies make claims to be comprehensive or exhaustive, but all at least present beginning concepts and constructs, some of which can be incorporated into program design.
Calley (2011) identifies six factors that have strongly influenced this movement toward special population–sensitive program design:
1. The number of professional associations specifically committed to both broad and specific aspects of multiculturalism. As examples, she cites four divisions of the American Counseling Association and five divisions of the American Psychological Association, each focused on a special population. In addition, a number of professional organizations include expectations for cultural competence in their codes of ethics.
2. The amount of scholarship dedicated to cultural competence. As examples, she cites the Journal of Multicultural Counseling and Development and the Journal of Lesbian, Gay, Bisexual, and Transgendered Issues, among others.
3. Cultural competence as a core part of academic preparation in mental health disciplines. As examples she cites a number of accreditation standards.
4. The promulgation of national standards of cultural competence by the federal government. As an example she cites the National Standards on Culturally and Linguistically Appropriate Standards (CLAS) of the U.S. Department of Health and Human Services, Office of Minority Health.
5. The inclusion of cultural competency-specific accreditation standards for mental health and human service programs. As an example she cites a Council on Accreditation standard requiring that assessments be made in a strengths-based culturally responsive manner.
6. The requirement of addressing cultural competence in proposals for funding new program development. As an example she cites requirements of the Administration on Aging for technical assistance for national minority aging organizations (pp. 106–115).
Practitioners are sure to be influenced by one or more of these factors. More importantly, these factors reflect significant changes in the way professionals are approaching program design in the interest of providing more relevant and more effective services.
Defining the Elements of a Program Bringing precision and understanding to a phenomenon involves breaking it down into some basic elements. The logic model gives us a framework for examining and defining program elements in each of its five phases, as follows:
Inputs: resources (staff, funding, etc.) and raw materials (clients or consumers) Process: activities that use inputs to achieve objectives with raw materials Outputs: measurements of services provided and completion of all services prescribed Outcomes: demonstrated benefits to those receiving service
4/20/2018 Bookshelf Online: Designing and Managing Programs
https://online.vitalsource.com/#/books/9781483388328/cfi/6/46!/4/2/4/2@0:0 3/16
Impact: measurable changes occurring in organizations, communities, or systems as a result of services
Each of these elements will need to be defined in a quantitative way that will allow them to be entered into a computer program. The program should be designed following the principle of “enter each item only once—use in combinations as many times as needed to measure performance and effectiveness.”
Inputs
Inputs to a program include five elements representing an agency’s resources and raw materials: (1) clients or consumers, (2) staff, (3) material resources, (4) facilities, and (5) equipment. Clients represent the raw materials in a human service system; the other four elements represent the resources that will be used to perform the activities needed to convert the clients from persons with problems and needs to persons who have been able to resolve problems and meet needs.
Each element needs to be further defined, and we provide examples of how they might be defined in Table 8.1. As each element is defined, it is important to remember that at some point we will need to collect data on that element for use in monitoring, performance measurement, and evaluation. For this reason, it is useful to define each of the above elements in terms that will be useful for analytical and reporting purposes. For example, what breakdown of such factors as age, ethnicity, or income will be useful later on when it is necessary to analyze the population served? The important thing to remember is that for each client or case that is entered, these factors will vary, and it is these differences that become the focus of our evaluation and research. Is a client’s ethnicity, age, gender, marital status, number of children or other factor related to the types of problems experienced? Are staff with more experience or advanced degrees more effective in helping clients resolve their problems? These are the types of inquiries that will be possible when data elements are precisely defined. Table 8.1 may be used as a guide in defining elements specific to a program and understanding how each element may vary. Not all elements listed in Table 8.1 will be used in every program.
4/20/2018 Bookshelf Online: Designing and Managing Programs
https://online.vitalsource.com/#/books/9781483388328/cfi/6/46!/4/2/4/2@0:0 4/16
4/20/2018 Bookshelf Online: Designing and Managing Programs
https://online.vitalsource.com/#/books/9781483388328/cfi/6/46!/4/2/4/2@0:0 5/16
The relationship of program elements to phases of the logic model is illustrated in graphic form at the end of each section. Figure 8.1 depicts the elements associated with program inputs.
Throughputs
Throughputs refer to the procedures that will be implemented to carry out the program. It is during the service provision process that resources, including staff, material resources, facilities, and equipment, are used to help clients so that they may complete the service process (output) and, it is hoped, resolve their problems (outcome).
Throughputs in social service programs usually involve such treatment, rehabilitation, or support methods as counseling, job training, provision of day care, residential treatment, shelter, dispensing of food baskets, and provision of information and referral. To bring some degree of uniformity to the process and to incorporate the possibility of measurement, it is necessary to identify and define data elements that make up throughputs. These elements include service definition, service tasks, and method of intervention.
Service Definition
The service definition is usually a simple one- or two-sentence definition of services to be provided. Its function is to narrow down the service from something that might cover a whole range of client problems and needs to something that is focused on a specific aspect of client problems and needs. For example, the broad category “drug treatment” can include the following: detoxification; inpatient or outpatient medical treatment; individual, group, or family counseling; job training and placement; and follow-up supportive services. Simply using “drug treatment program” as a service definition does not sufficiently narrow its scope in a way that informs relevant parties what the program is intended to accomplish. A definition such as “This program or service is intended to provide outpatient detoxification to cocaine addicts ages 18 and older” helps those who need to know who the program is designed to serve and what services will be provided.
Figure 8.1 Logic Model as Applied to Program Inputs
A comprehensive listing of service definitions was developed by the United Way of America (1976) in a volume titled UWASIS II: A Taxonomy of Social Goals and Human Service Programs, in which more than 230 services are labeled and defined. Even though this volume was published 40 years ago, it is still available online and can be used to guide the program planner toward crafting specific service definitions to ensure accurate description of the service. An example from the UWASIS directory is the definition of pre-job guidance: “Pre-job guidance is a program designed to help individuals who need to learn the basic tools of obtaining employment to suit their particular skills and talents” (p. 208). Another listing of service definitions is the Arizona SFY 2010 Dictionary and Taxonomy of Human Services (Arizona Department of Economic Security, 2010). A number of examples help clarify ways in which service definitions can be written:
Adoption placement is defined as the selection of a family and placement and supervision of a child until the adoption is finalized Shelter services is defined as the provision of temporary care in a safe environment available on a 24-hour basis. Case management is defined as a process in which an individual who is determined to be in need of and eligible for services works with a professional staff person to identify needs, plan for services, obtain services, collect data on the service process, monitor and evaluate services, terminate the process, and follow up as needed
Service definitions should be written by experienced staff who are familiar with the service and should be distributed widely for review and comment before being finalized.
4/20/2018 Bookshelf Online: Designing and Managing Programs
https://online.vitalsource.com/#/books/9781483388328/cfi/6/46!/4/2/4/2@0:0 6/16
Service Tasks
Service tasks help to define the activities that go into the provision of the service. If, for example, a program or service is intended to increase independence and self-sufficiency for single mothers by training them in job-finding skills, what tasks might make up the service? Such tasks as screening and assessment, resume preparation, interviewing skills, work habits, job training, job placement, and follow-up might be considered to be part of a complete package. Payne (1997) summarizes the social work process in eight phases, each phase having its own associated activities or tasks. For example, the first phase, Assessing Problems, includes the tasks of stating the problem, analyzing the system, setting goals, setting strategy, and stabilizing the change effort. To ensure some degree of comparability in what clients receive, it is important that some thought be given to identifying and defining service tasks. In most cases, service tasks tend to follow a chronological order of services to a client.
Flowcharting Service Tasks
One technique that can be helpful in conceptualizing and documenting the service delivery process is flowcharting. This technique illustrates each task, each decision to be made, each form to be completed, and what must be accomplished before proceeding to the next step. Making service delivery flowcharts available to all staff and supervisors involved in providing services can not only guide the practitioner but also help ensure uniform practices from worker to worker.
Figure 8.2 illustrates a client flow through the Safe Haven Shelter.
Specifying tasks in this way and bringing increasing degrees of precision to their definition helps to introduce at least some degree of uniformity to what helping professionals provide and can serve a function similar to that of protocols in medicine, child protection, and other disciplines. A protocol for a particular service can be developed by beginning with the flowchart of client services. A narrative chart can then be added to accompany the flowchart. A narrative chart will include an explanation of what activities are to be carried out at each step in the process and what documentation is necessary.
Identification and listing of tasks may be a bit tedious in the beginning, but tasks serve to bring a clearer focus to the question of who does what with clients, for what purpose, and under what conditions. Tasks not only address the accountability question but also permit ongoing evaluation of effectiveness. If a particular approach is effective, the treatment can be repeated. If it is not, problem areas can be pinpointed by identifying their locations on the flowchart, and the treatment can be modified as needed. An illustration of a narrative chart is included in Table 8.2.
Method of Intervention
The third element of throughput is the method of intervention. Defining the method of intervention requires that program planners specify in advance the ways the service may be delivered. For example, meals for the elderly can be provided in a congregate setting or can be delivered to the elderly person’s home. Job training can be carried out in a classroom setting or on the job. Counseling can be offered to individuals, in groups, or in families. Payne (1997) describes three roles for social workers when the aim is helping people make transitions (p. 148):
1. Enabling (e.g., strengthening the client’s motivation, validating and supporting the client, helping to manage feelings) 2. Teaching (e.g., helping clients learn problem-solving skills, clarifying perceptions, offering appropriate information,
modeling behavior) 3. Facilitating (e.g., maintaining clients’ freedom of action from unreasonable constraints, defining the task, mobilizing
environmental supports)
Figure 8.2 Flowchart of Client Processing
4/20/2018 Bookshelf Online: Designing and Managing Programs
https://online.vitalsource.com/#/books/9781483388328/cfi/6/46!/4/2/4/2@0:0 7/16
4/20/2018 Bookshelf Online: Designing and Managing Programs
https://online.vitalsource.com/#/books/9781483388328/cfi/6/46!/4/2/4/2@0:0 8/16
4/20/2018 Bookshelf Online: Designing and Managing Programs
https://online.vitalsource.com/#/books/9781483388328/cfi/6/46!/4/2/4/2@0:0 9/16
Specification of these types of roles helps guide the worker through the helping process. The intent is to achieve at least some degree of uniformity in the service process experienced from one client to another rather than leaving decisions about interventions to individual judgment and choice. Since the method of treatment is based on an understanding of the problem and on the program hypothesis, it is important to identify best practices and specify a proven method of treatment or service delivery (e.g., behavior modification, cognitive therapy, crisis intervention) if such a method exists. As the federal government moves more toward focusing on and paying for performance over accountability, specifying methods of intervention may take on less importance than outcomes or results for funding purposes (Council on Financial Assistance Reform, 2014). However, for this same reason service tasks become even more important at the program level. It would be difficult to replicate a success if those providing the service didn’t bring at least some level of precision to the helping process. Table 8.3 illustrates throughput elements.
The diagram in Figure 8.3 illustrates the next phase of the logic model associated with program process or throughputs.
Outputs
The United Way of America (1996) defines outputs as the direct products of program activities and provides as examples the number of classes taught, the number of counseling sessions conducted, or hours of service delivered. Brody (2005) adds that outputs measure the volume of work accomplished. The purpose of measuring output is to determine (a) how much of an available service a client actually received and (b) whether the client completed treatment or received the full complement of services as specified in the program design. The “how much” question is answered by defining units of service, and the answer is referred to as an intermediate output. The “service completion” question is answered by defining what we mean by completion, and the answer is referred to as a final output. For example, if a training program consists of 10 sessions, one unit of service would be one session. Calculating intermediate outputs would require taking attendance and recording how many sessions each trainee actually completed. The purpose of this type of tracking is to learn whether those who drop out or are inconsistent in their attendance achieve the same results as those who are regular in their attendance.
4/20/2018 Bookshelf Online: Designing and Managing Programs
https://online.vitalsource.com/#/books/9781483388328/cfi/6/46!/4/2/4/2@0:0 10/16
Figure 8.3 Logic Model as Applied to Program Throughputs
Final output in this example refers to successful completion of the course, however that is defined. Participants are divided into “completers” and “dropouts.” This means that there will need to be a formal definition of completion. How many sessions can a person miss and still be considered a completer? Aggregated final output is the rate of completion (a percentage) for all the trainees in a given class. Isolating the group of completers allows program planners to determine whether the training made a difference in achieving the outcome projected in the program hypothesis and the outcome objectives. If dropouts and completers are equally successful in achieving training objectives, then the training cannot be said to have made a difference. Training courses are among the easier outputs to define, because there is usually a specified beginning and ending point. Completion of counseling, therapy, long-term care, and many others are more challenging when it comes to specifying final outputs, and they usually require some discussion and reaching a consensus before they can be considered formal definitions.
A third dimension of output—quality of service provided—has also emerged in current discussions of output measures. Quality performance measures are discussed later in this chapter.
Units of Service
Units of service can be measured in three different ways: (1) as episodes or contact units, (2) as material units, or (3) as time units (Martin & Kettner, 2010). An episode or contact unit is defined as one contact between a worker and a client. It is used when the recording of client contact information is important but when the actual duration (time) of the contact is not considered important. Information and referral services often use a contact unit, simply counting the number of requests for information they receive. Some mentor programs might also use number of contacts as their unit of service.
A material unit of service is a tangible resource provided to a client and can include such items as a meal, a food basket, a trip, an article of clothing, cash, or a prescription. Material units are generally considered to be the least precise of the three types of units of service because variation may or may not exist between individual units. For example, the number and types of items in two food baskets or two boxes of clothing can vary widely, but each basket or box is still counted as one unit of service. However, with other types of material units such as cash or a trip (measured in miles) the units are precise and comparable.
A time unit can be expressed in minutes, hours, days, weeks, or months, depending on the information needs of individual human service programs. Minutes might be used to measure the length of a phone call. An hour is often the unit used for a counseling session. Days might be used to measure the length of a stay in a homeless shelter. Weeks or months could be used for residential care. A time unit is the most precise of the three types of units of service because it is expressed in standardized increments. When time is used as a unit of service, it is important to state whether the time refers only to direct
4/20/2018 Bookshelf Online: Designing and Managing Programs
https://online.vitalsource.com/#/books/9781483388328/cfi/6/46!/4/2/4/2@0:0 11/16
client contact time or whether support activity time (e.g., completing paperwork, attending client staffings) is also included. Ultimately, units of service are used for a number of purposes, including a determination of cost per unit of service, so decisions about defining units of service should be made with a good deal of care and attention to their purpose.
The ways in which units of service may be used become evident when we calculate how many units a program can provide in a year. Let us assume that there are five counselors and that each one can see 20 clients per week at 48 weeks per year (allowing for 2 weeks for vacation and 10 paid holidays for each counselor in a year). Each counselor, then, has a capacity to provide 48 × 20, or 960 units of counseling per year. The entire program, with five counselors, can provide 4,800 units per year (960 units per worker × 5 workers). (In a later chapter we will discuss how a unit cost is calculated.) In the same manner for any given program, examination of resources provides a basis for calculating the number of units to be provided in a given year.
Episode units, even though they are not exact measures, help provide a degree of comparability from program to program. For example, a “child care day” usually runs from 6:00 a.m. to 6:00 p.m., and even though some children may be in child care for 6 hours, some for 8 hours, and some for 12 hours, for the purpose of measuring volume, all are considered to have received an episode of one child care day of services. It may simply not be worth the time and effort involved to calculate the exact number of hours and minutes for each child for the purposes for which the measure is used.
Material units may be precise and comparable, or they may vary. One newborn care kit may consist of exactly one package of diapers, two blankets, assorted soaps and lotions, and other necessities, with all being exactly the same. One food basket, on the other hand, might consist of a mix of canned and other nonperishable goods, but there could be wide variation from basket to basket, depending on the needs of the family. The rule of thumb is that the degree of precision and comparability in using episode and material units should be guided by the ways in which the information will be used. Table 8.4 illustrates time, episode, and material units of service.
Service Completion
The second data element involved in defining output is that of service completion and is referred to as a final output. The question that must be answered is: When is a client finished with a service? Service completion is defined at the time the program is designed. For some services, final output is easily and clearly defined; for others, it is problematic. In most training programs, for example (or even in a university setting, for that matter), a number of training sessions or class sessions are required for successful completion of the course. One intermediate output unit might be attendance at one class, and a final output might be defined as one client’s completing all the requirements of an automobile mechanics training course. A prenatal care program might consist of at least six monthly prenatal visits with a physician, and successful completion of a detoxification program might be defined as completion of a 60-day stay in an inpatient detox unit. Table 8.5 provides examples of intermediate and final outputs.
Note: FTE = full-time equivalent.
aWeeks can be adjusted for holidays and vacations.
4/20/2018 Bookshelf Online: Designing and Managing Programs
https://online.vitalsource.com/#/books/9781483388328/cfi/6/46!/4/2/4/2@0:0 12/16
bIf material resources are provided irregularly, adjustments will need to be made to this formula.
For some services, final outputs can be difficult to define. In ongoing services such as day care or long-term residential care for the elderly, it is not useful to define final outputs in terms of exit or completion of a program because these programs are not designed to move clients in and out at a steady and predictable pace. Clients in these types of long-term programs may remain for many years, and effective monitoring and evaluation cannot occur if measurements are taken only after an extended period of years. Effective monitoring and evaluation requires much more frequent examination.
In these types of instances, the program designers should define the final output in terms of completion of a fixed-term treatment plan. For example, one might define a final output as “completion of an individual care plan” for day care or “completion of the prescribed service plan for a 3-month period” in the case of long-term residential care for the elderly. Here we are dealing with the equivalent of achieving milestones as required in individual education plans.
Building milestones into long-term service designs permits measures to be taken at certain selected points to determine whether the treatment plan is having the desired effects. Its purpose is to ensure that the full service mix, as intended by the designers of the program, has been received by a client.
Defining output prior to implementation of a program also enables evaluators to distinguish between someone who completes the program and someone who drops out. These two groups need to be evaluated separately, but if output is defined simply as an exit from the program, it is impossible to distinguish completers from dropouts for evaluation purposes. Table 8.6 illustrates calculation of intermediate and final outputs.
Quality
As units of service have become more clearly conceptualized and defined, there has been a tendency by some funding sources to base contracts and reimbursement plans on the number of intermediate output units actually delivered (e.g., the number of counseling hours provided). This emphasis has, in turn, led to a concern about what is often referred to as “bean counting”—an exclusive focus on quantity to the exclusion of quality.
It is easy to see how this emphasis can come about. If an agency is reimbursed and rewarded for the number of interviews its staff conducts, a program manager for the agency may be tempted to cut down on the time of an interview and squeeze in 10 or 12 interviews per worker per day, regardless of the quality of those interviews. On the other hand, if an agency is held responsible for meeting certain standards of quality in its services, and is reimbursed on the basis of both efficiency (the number of units) and quality (meeting the standards), then the agency must find a way to balance both of these factors. This is the principle behind measuring quality. It should be pointed out that measurement of quality is not a typical component of contract expectations. However, when concerns are raised at the agency level about a particular dimension of service quality, these measurements can be handy tools to use.
4/20/2018 Bookshelf Online: Designing and Managing Programs
https://online.vitalsource.com/#/books/9781483388328/cfi/6/46!/4/2/4/2@0:0 13/16
Note: FTE = full-time equivalent.
Quality, unlike quantity (units of service), tends to be somewhat elusive and is defined differently depending on one’s perspective. In business and industry, customers have tended to be the final arbiters of what constitutes quality (Crosby, 1980, 1985; Deming, 1986; Juran, 1988, 1989). In human services, customer (client) perspectives are important but are not the sole criterion for determination of quality. Other perspectives, including those of administrators, professionals, board members, and funding sources, may also have important input to offer in determining what constitutes quality. If quality dimensions are to be a part of the program design, it is necessary that they be defined, that quality data be collected, and that service quality be monitored and evaluated on a regular basis.
Quality is frequently addressed through the use of standards. A standard is a specification accepted by recognized authorities that is regularly and widely used and has a recognized and permanent status (Kettner & Martin, 1987). For many of the elements of program design, standards will be imposed by outside sources. For example, wherever licensing, certification, or accreditation is a concern, standards must be identified and incorporated as a part of the program. If food is served, standards will be imposed by the health department. If medical services and facilities are a part of the program, the Joint Commission on Accreditation of Healthcare Organizations will impose standards.
In most instances, it is necessary to identify and operationalize standards. In some instances, however, it will be necessary to develop them. When considering competence, for example, selecting credentials for casework or counseling staff is often a judgment call. Some drug treatment programs operate exclusively with ex-addicts, regardless of educational background. Some counseling programs insist on a staff member’s having at least a master’s degree and prefer a doctorate. Some positions require bilingual staff, and defining a qualification such as “bilingual” depends on pre-established standards for the ability to speak two languages. In many ways, standards serve as protection for clients or consumers in that they affect the services provided. Martin (1993) made an important contribution to the measurement of quality by identifying 14 generally recognized quality dimensions, as illustrated in Table 8.7.
If it is determined that one or more dimensions of quality need to be tracked, program planners (in conjunction with clients and other stakeholders as needed) can determine which of the quality dimensions listed in Table 8.7 are the most important for a given program. Quality dimensions to be used in the program must then be operationally defined. For example, the quality dimension of accessibility could be defined as the amount of time it takes clients to get from their residences to the agency. The quality dimension of responsiveness could be defined as ensuring that at least 75% of clients who come to the agency for services are seen within 20 minutes of their scheduled appointment times. The dimensions selected will depend on perceived problems.
Once the quality dimensions are selected and defined, they must be melded with units of service (intermediate outputs) and tracked. For example, in tracking responsiveness, each time a client comes to the agency for services, it will be necessary to record whether that client was seen within 20 minutes of the scheduled appointment time. Or if the quality dimension of
4/20/2018 Bookshelf Online: Designing and Managing Programs
https://online.vitalsource.com/#/books/9781483388328/cfi/6/46!/4/2/4/2@0:0 14/16
competency is used and defined in terms of having an MSW and 3 years of counseling experience, it will be necessary to record the number of client counseling sessions that met this standard and the number that did not.
Source: Adapted from Martin, L. (1993). Total quality management in human service organizations. Newbury Park, CA: SAGE Publications, Inc. Reprinted with permission.
In measuring quality over the course of a year, two different sets of units will be recorded and tracked: (1) the number of units of a given service provided to clients and, (2) of those units provided, the number that met the pre-established quality standard. Following this format, any quality standard established in a field can be used in conjunction with units of service provided to determine the extent to which quality dimensions are being achieved in a program.
In instances in which quality dimensions are too difficult, time-consuming, or cumbersome to track, the client satisfaction approach may be used. When this option is selected, it is still necessary to select quality dimensions, but they are measured by translating them into questions to be asked of clients, for example, “Did your home-delivered meals arrive on time (within 20 minutes of scheduled delivery time)?” and “Do your home-delivered meals arrive hot?” The findings are then turned into percentages of clients who answer “yes” to determine whether the quality standard has been achieved. The diagram in Figure 8.4 illustrates the next phase of the logic model as applied to program outputs.
Outcomes
4/20/2018 Bookshelf Online: Designing and Managing Programs
https://online.vitalsource.com/#/books/9781483388328/cfi/6/46!/4/2/4/2@0:0 15/16
An outcome is defined as a measurable change in quality of life achieved by a client between entry into and exit from a program. Outcome measures can be placed into one of four categories: numerical counts, standardized measures, level-of- functioning scales, or client satisfaction (Kuechler, Velasquez, & White, 1988; Martin & Kettner, 2010). The questions that must be answered are: (1) Do clients improve as a result of services? (2) How do you define and measure improvement? The following sections discuss four approaches to measuring client improvement.
Figure 8.4 Logic Model as Applied to Program Outputs
Numerical Counts
Numerical counts are nominal measures related to client flow. They require yes or no answers to specific questions, such as the following: Was the client placed in a job on completion of training? Did the child return home following residential treatment? Was another crime committed by the juvenile subsequent to treatment? The answers are then converted into percentages to determine the extent to which the expected outcome was achieved. Numerical counts are relatively easy to define and interpret, and many programs already collect these data. This approach to measurement simply requires recording a client’s status (e.g., unemployed) at the point of entry into the system and then recording it again at completion of service and, in some cases, following up to record it again at selected intervals after completion of service. Aggregating all findings within a specified period (usually a year) will establish a success rate derived from numerical counts.
Standardized Measures
Standardized measures are objective instruments that have been validated and are widely used by practitioners. Examples include the Minnesota Multiphasic Personality Inventory (MMPI) and standardized intelligence tests such as the Stanford- Binet. Several volumes have been devoted to standardized measures of quality-of-life factors. For example, Kane and Kane (1981) developed measures for the elderly, and Fischer and Corcoran (1994) developed measures for families and children. Martin and Kettner (2010) identified a variety of different perspectives from which standardized measures have been developed (see Table 8.8).
The following is an illustration of a question from a Generalized Contentment Scale developed by Hudson (1982):
I feel that I am appreciated by others:
1 = rarely or none of the time
2 = a little of the time
3 = some of the time
4 = a good part of the time
5 = most or all of the time
Tracking these scores can be useful not only at the program level, but also for the caseworker or counselor to assess progress for individual clients. Getting the most out of standardized scales will require periodic reassessments and entering them into the client’s profile. Documenting client improvement at the program level will require that data on all clients receiving a particular service be aggregated (usually on a yearly basis), with results displayed in some type of table or graph.
Level-of-Functioning Scales
Level-of-functioning scales are instruments developed by staff and other local experts familiar with a particular population and problem and are specific to a program or service. They require that practitioners rate their clients on several aspects of functioning. For example, persons who are chronically mentally ill may be rated on such factors as self-care, decision- making ability, and interpersonal interaction. Persons with developmental disabilities may be rated on activities of daily
4/20/2018 Bookshelf Online: Designing and Managing Programs
https://online.vitalsource.com/#/books/9781483388328/cfi/6/46!/4/2/4/2@0:0 16/16
living, functional communication, interaction skills, and other factors. For each scale, indicators are specified and clients are rated at intake, at intervals during their participation in the program, and at exit from a program on a multi-point scale ranging from low to high functioning on each item. Table 8.9 is an illustration of a level-of-functioning measure for the safety of a person at risk of domestic violence.
Source: From Martin & Kettner (2010). Measuring the performance of human service programs (2nd ed.). Thousand Oaks, CA: SAGE Publications, Inc. Reprinted with permission.