SINGLE SYSTEM RESEARCH DESIGN

1

Session Four Bloom Chapter 2 BASIC PRINCIPLES OF CONCEPTUALIZATION AND MEASUREMENT The helping professions run on energy supplied by theories and research and are guided by professional training and ethics, as expressed in one’s personal style and experiences. WHAT IS CONCEPTUALIZATION? The process of finding general behavior patterns among the bits and pieces of specific situations. Theories are neither true nor false; they may provide fruitful derivations (predictions) that can be tested for their correctness. A major purpose of single-system designs, the topic of this text, is to test the correctness of practice hypotheses derived from theories, research, or practice wisdom, but to do this in the context of helping our clients. WHAT IS MEASUREMENT? Measurement is generally thought of as the process of assigning labels to certain characteristics of things according to a set of rules Levels of Measurement

Nominal level measures provide information about whether one observation is qualitatively different from another or a person is in one category or another for example, male or female

Ordinal Level measures provide the same information as nominal-level measures, plus information about the relative amount of one observation compared to another. For example, each week the number of martial arguments for a couple might be classified into:

1. Rarely or never 2. Some or little of the time 3. Occasionally 4. Most of the time

Interval level measures provide the same information as ordinal-level measures plus the adjacent intervals are equal so it’s possible to determine how much more of some characteristic is possessed by a person. Few examples in social work

2

Ratio Level measures provide the same information as interval-level measures plus they have a defined zero point that has some intrinsic meaning. Examples include the amount of time spent caring for an elderly parent, number of years married, etc.

Advantages of Measurement In a way, measurement is just a more formalized way of clarifying the observations that you make every day in practice, no matter what the level of measurement is used. DEFINATION AS A FIRST STEP IN MEASURMENT Conceptual Definitions - sometimes referred to as a nominal definition is found in the dictionary, defining a concept using one set of words-like synonyms-to define another word or set of words. Operational Definitions - An operational definition is one that assigns meaning to a concept (e.g., child neglect) in terms of the activities or operations necessary to measure it. Operating definitions are “working definitions” of what you are trying to measure CAN EVERYTHING BE MEASURED? Many years ago, the psychologist Robert Thorndike said that if something exists, it exists in some quantity; if it exists in some quantity, then it can be measured. If a problem is too subtle or complex to measure, it just may be too subtle or complex to work with in practice. Because many cleints present their concerns in rather vague and amorphous terms, your job is to help clients clarify and specify their concerns. One way of doing this is to identify those parts of the problem that can serve as measurable indicators of the problem, so that you can measure them to gather feedback on the success of your intervention. KEY CHARACTERISTICS OF ALL MEASURES Measurement Error

Random Measurement error refers to the possibility that responses could be affected by chance or haphazard occurrences

Systematic Measurement Error is a tendency to err in a particular way (e.g., few people would admit to a socially reprehensible action

Importance of Measurement Error – we would like to know how much measured or observed change in a client’s problem is due to actual change in the problem.

3

Reliability - The word reliability is a general term for the consistency of measurements and unreliability basically means inconsistency due to random error Test-Retest Reliability is determined by independently measuring the same group of people (or things) with the same measure on two different occasions under the same circumstances

Alternative-Forms Reliability is primarily concerned with the consistency with which we measure concepts

Internal-Consistency Reliability basically refers to the extent to which parts of a measure (items or even entire halves of the measure) are homogenous or measure the same thing. Interobserver reliability AKA interrater reliability refers to the extent to which two or more observers are consistent with each other when they independently observe or judge the same thing (

Standard Error of Measurement (SEM) usually should not be used to compare the adequacy of different measures.

What is “High” Reliability? See table in text

What Type of Reliability Should You Use? When you select existing measures for use with your clients you should determine whether evidence exists in the literature for the reliability of the measures.

Validity

Face Validity refers to the opinion by the developer of a measure, by some expert, or by you that an instrument measures what it intends to measure

Content Validity refers to the extent to which the questions or behaviors selected for measurement are representative or are a biased or limited sample of what you intend to measure

Criterion Validity refers to its ability to predict an individual’s performance or status on certain outcomes

Construct Validity refers to the extent to which empirically observed relationships among measures of concepts agree with the interrelationships among concepts predicted by theories. Construct validity, then, can be thought of as the use of multiple measures of the same concept

Reactivity – Reactive measures are those about which the client “reacts.”

4

Which Type of Validity Should You Use? In evaluating the validity of a measure for possible selection, one can probably place most confidence in the measure that has information about all the forms of validity.

The Relationship between Reliability and Validity An unreliable measure cannot be valid. Conversely, if a measure has poor validity, it is irrelevant that it may have high reliability. As a general rule, though, any lack of reliability in a measure tends to lessen its validity. The reliability of a measure is relatively easier to determine than its validity…an unreliable measure cannot be valid. Factors Influencing Reliability and Validity

Administration Procedures; any condition that’s irrelevant to the purpose of a measure can potentially introduce random or systematic measurement error.

Populations; the validity with which a target is measured can differ depending on numerous characteristics of clients, including racial and ethnic identity, national origin, cultural background, language preference, gender, age, sexual orientation, disability status, religion, extent of spirituality, and socioeconomic status.

Utility

Relevance to Intervention Planning – the most basic reason for measuring client targets is to improve client outcomes.

Ease of Use – Single-system designs require the repeated measurement of client targets over time.

Accessibility – the accessibility of a measure is also an important criterion. Direct and Indirect Measures Reliability, validity, and utility are important criteria by which to evaluate a measurement strategy, but one final consideration involves the extent to which your measure is a direct measure of the client’s target.

5

Bloom Chapter 3 SPECIFYING PROBLEMS AND GOALS INTRODUCTION: FROM GENERAL PROBLEMS TO SPECIFIC TARGETS OF INTERVNETION What client problems or issues need to be resolved and what goals need to be achieved? We use the term target to refer to the specific object of preventive or interventive services that is relevant in a given situation. By encouraging the specification of problems, we do not suggest that meaning be sacrificed for rigor. SPECIFYING CLEINT CONCERNS: IDENTIFYING AND CLARIFYING PROBLEMS AND POTENTIALS [The] task is to move from an area of concern felt by the client as problematic to a target on which client and practitioner can agree to work.. This target may involve interventive action to decrease a given set of events, to build strengths, or both. Preliminary Steps in Identifying Targets There are several steps you might take in the process of specifying a problem

Survey Client Concerns: Starting Where the Client Is – The first step on the path toward specifying problems begins when you attempt to review all the matters of concern to the client, referring agency, and/or relevant others. This is not the assessment proper; rather, it is a preliminary survey of problems and a discussion with those concerned, involved, or affected about how they perceive these problems.

Select a Target – Unless you are lucky enough to be meeting with an individual or group that has presented only one problem for you to work on, it is likely you will have to make some initial selections from among several concerns presented earlier by the client – focus on one problem and attempt to be specific as possible defining it.

Prioritize Targets – The first step in selecting a problem or area of concern on which to work is to review the written list of concerns that you developed earlier with the client. The task then becomes one of negotiating with the concerned parties regarding what concern(s) to work on.

Guideline for Going from the Vague to the Specific: Operational Definition of the Target of Intervention

6

The task of the practitioner is to help the client operationalize problems, that is, redefine them in more precise, measurable (observable) terms.

Clarity – Most clients have a great deal of difficulty being clear about describing the problem or about describing where, when, or how often it occurs. The job of the practitioner in these cases is to help translate vague or even inaccurate descriptions into specific and accurate ones. Think of how often you’ve heard the terms hostile, suspicious, immature, anxious, aggressive. What do you do with those terms? Countability – Anything that occurs, occurs with some frequency and can therefore be counted.

Verifying Sources – A way of knowing, when possible, that the problem does in fact exist, and a way for documenting when, where, and how often it occurs.

Dereify – One major impediment to defining problems in specific terms and with real-world referents is called reification, which refers to the treating of abstractions or constructs as though they were real.

Increasing and Decreasing – One way of helping you specify problems is to think of them in terms of concerns to be increased or decreased.

Measurability – Begin to consider different ways a given problem can be, or has been, measured.

Case Illustrations of Operationalized Targets See examples in text. SPECIFYING GOALS AND OBJECTIVES Goals usually emerge from discussion about client problems, but it is important to keep these ideas distinct. Goals indicate what the client would prefer to happen, to do, or to be, when the intervention is completed. Definition of Terms: Goals and Objectives The above definition of goals is the ultimate goal[s], or goal[s] related to ultimate outcomes for the client. Goals and objectives should have at least four components:

7

Components of Specific Goal Statements

Who? – Objectives and goals should be stated in terms of intermediate and ultimate outcomes for the client.

Will Do What? Attempt to state what clients actually will be doing to show that they have achieved the goal.

To What Extent? The purpose here is to attempt to establish how well and how often the target event occurs.

Under What Conditions? Specify the conditions and situations under which the desired behavior or activities will occur.

Using Statistics to Set Goals It’s generally a good idea, when possible, to be precise in identifying what you are shooting for in terms of an actual target. For example, a weight-loss program USING GOAL ATTAINMENT SCALING (GAS) TO ESTABLISH GOALS One widely used method for establishing and tracking goals is called Goal Attainment Scaling. Goal Attainment Scaling is a system to assist in goal definition and goal measurement. Basic Guidelines for Using GAS

Collecting Information – this step varies by practitioner, setting, and types of problem with which you may be working.

Designation of Problem Areas – Once assessment data are collected, the GAS format suggests you break these down into a series of “problem areas.”

Predictions for Each Problem Area – A specific follow-up point should be set for each scale or problem area. Then, a series of predictions is made for each scale. These predictions actually are the goals for the client.

Follow-Up “Interview” – The evaluation of outcome SETTING GOALS IN GROUPS We should distinguish true group problems and objectives from the individual problems and objectives of people who may be seen in groups. In general, we are proposing that, when possible, the group objectives and goals be identified and operationalized. In addition, it may be very useful to collect individual scores

8

on related topics so that the relative status of each individual may be compared with the group as a whole. PROBLEMS AND ISSUES IN SETTING GOALS There are several problems and issues in establishing objectives and goals that could impede progress. Anticipate Negative Outcomes You will want to help the client anticipate the impact of achieving his or her goals. You and the client will have to evaluate the effects that changes in the client are likely to have on others in his or her environment. Who Sets the Goals? Because goal setting is largely a value judgment—a statement of what is desired— whenever possible the client should have the major, if not the sole, say in setting goals. Anticipate Impediments Anticipate problems See list of twelve in text Case Illustrations of Goals in the Context of Problems and Interventions See text