D-Personal Progress Topic Interest

TT24

Reading-Single-CaseResearchDesignandAnalysis-CounselingApplications.pdf

Home >Psychology homework help >D-Personal Progress Topic Interest

Journal of Counseling & Development ■ October 2015 ■ Volume 93394

Best Practices

Received 08/15/14 Revised 12/03/14

Accepted 12/04/14 DOI: 10.1002/jcad.12037

Earn CE credit. Visit http://www.prolibraries.com/counseling to purchase and complete the test online.

Single-case research design (SCRD) is a type of research used to demonstrate experimental control within a single case and rigorously evaluate an intervention with one or a small number of cases (Kazdin, 2011). The single case being studied can be an individual person, a family, or a group of individuals (Morgan & Morgan, 2009). SCRDs are often known as n = 1, single-subject, small n designs, or single-case experimental designs. Gallo, Comer, and Barlow (2013) proposed that research designs that examine individuals, such as SCRDs, offer a greater understanding of the mechanisms of change in treatment. The use of a rigor- ous SCRD directly addresses the evidence-based question: What works for whom under what conditions? Counseling literature is replete with case studies that provide informa- tion about relationships between conditions and are helpful in understanding an individual client (Morgan & Morgan, 2009). SCRD is differentiated from case studies through its focus on the manipulation of the independent variable (i.e., counseling intervention). SCRDs are characterized by the manipulation of an independent variable—the hallmark of experimental design—and lead to causal inference that links treatment to effectiveness. Through the intensive applica- tion of repeated measures across the implementation of an intervention, SCRDs allow for the exploration of how, why, and when changes occur (Gallo et al., 2013).

Experimental psychologists began studying one or a few subjects in the 1880s and continued until the early 1900s. This individualized approach succumbed to the popularity of between-group design research because of the perception of greater generalizability associated

Dee C. Ray, Department of Counseling and Higher Education, University of North Texas. Correspondence concerning this article should be addressed to Dee C. Ray, Department of Counseling and Higher Education, University of North Texas, 1155 Union Circle, PO Box 310829, Denton, TX 76203 (e-mail: [email protected]).

Single-Case Research Design and Analysis: Counseling Applications Dee C. Ray

The application of single-case research design (SCRD) offers counseling practitioners and researchers a practical and viable method for evaluating the effectiveness of interventions that target behavior, emotions, personal characteristics, and other counseling-related constructs of interest. This article discusses general issues relevant to SCRD, as well as obstacles to its implementation in counseling settings. Steps to SCRD development, application, and analysis that are specific to the counseling field are also provided to encourage its use among counselors and researchers.

Keywords: single-case research design, visual analysis, intervention research

with group design and statistical analyses. However, the primary method of evaluating operant conditioning has traditionally been used by SCRD. In recent decades, SCRD experienced a resurgence in the literature, which has been attributed to the effect of the evidence-based intervention movement on applied behavioral analysis (Kazdin, 2011; Vannest, Davis, & Parker, 2013). The majority of SCRD reports are found in research on ap- plied behavioral analysis. However, SCRD reports are still rare in professional counseling journals (Barlow, Nock, & Hersen, 2009), and this lack of SCRD studies in the counseling literature has been noted by researchers who support its use (Lenz, 2013; Sharpley, 2007).

Although the rationale for the lack of SCRD publication is unclear, it is possible that counseling researchers may not be educated on its use, including the development and ap- plication of SCRD design and SCRD analysis. Because most SCRD literature is published with an emphasis on change in the overt behavior of individuals, the application of SCRD in counseling may be less evident. Counseling research typically requires the consideration of variables that are latent and less observable to the naked eye. Counseling researchers are likely to focus on the person of the client, person of the counselor, and the effect of the counseling relationship, among other variables that distinguish a counseling focus from a purely behavioral focus. The article provides an introduction to SCRD design and analysis as applied to the field of counsel- ing. I discuss the essential features of SCRD and basic steps to conduct an SCRD study. I also address typical barriers in SCRD application.

Journal of Counseling & Development ■ October 2015 ■ Volume 93 395

Single-Case Research Design and Analysis

Core Characteristics of SCRD SCRD is characterized by a specific coding system that is essential to its discussion and implementation (Riley-Tillman & Burns, 2009). A signifies the baseline condition; B is the intervention condition; C, D, and so forth denote subsequent different interventions; and when two interventions are deliv- ered simultaneously, the lettered interventions are presented together (e.g., B–C). These conditions are often referred to as phases in SCRD. In SCRD, the participant (for counseling purposes, this would be the client) is used as his or her own control for which comparisons are made. Hence, A (baseline) is a period of time in which the client receives no interven- tion, yet the client is monitored through assessment during the baseline to determine the stability of the dependent vari- able (e.g., anxiety symptoms) when no intervention is being delivered. Once stability has been established, the researcher introduces the B phase, which includes an intervention designed to affect the dependent variable of interest. In sim- plest terms, if the client showed no improvement or showed a decline during the baseline phase (A), yet demonstrates improvement during the intervention (B), there is reason to believe that intervention might be the cause of improvement.

According to Kazdin (2011), there are three essential characteristics of all SCRDs. The first characteristic of SCRD is the application of continuous assessment. The continuous assessment feature of SCRD is the true distinguishing char- acteristic of the design when compared with group designs. In SCRD, the researcher conducts repeated observations of the participant’s behavior, emotions, or a relevant construct over time. The single-case researcher identifies a target behavior that serves as the outcome variable for assessment. Frequent assessment or observation of the target behavior allows the researcher to determine the effect of treatment on the identi- fied behavior. In counseling research, the target behavior may be a specific observed behavior, such as increased heart rate or incidents of aggression, or an emotional variable assessed by client self-report or other-report, such as measures of anxiety or depression symptoms. The second characteristic, baseline assessment, involves the measurement of the target variable across the time period prior to intervention. Research- ers use the baseline phase to define a preintervention level of performance, providing the researcher with information about the extent of the identified problem and determining the stability of the target behavior without intervention (Ray & Schottelkorb, 2010). The third feature of SCRD is the stability of performance within phases. Stability of perfor- mance is indicated by an absence of variability in measured client behavior or self-report or other-report (Kazdin, 2011). A baseline in which data points remain at the same level or worsen throughout the phase signifies that the client’s behav- ior is not improving on its own, and provides stability from which to compare the intervention that will be introduced during the B phase.

Single-Case Experimental Designs The A-B design is the cornerstone of the experiment in SCRD. The initial manipulation of the independent variable (i.e., counseling intervention) is conducted by observing the stability of the dependent variable without interven- tion in the baseline (A phase), and then introducing the independent variable in the B phase while continuing to conduct observations. However, A-B designs are consid- ered limited in SCRD because of threats to internal valid- ity, such as the lack of a control for history, which may result in the influence of extraneous variables. Hence, the most popular SCRDs are withdrawal/reversal designs. In withdrawal/reversal designs, the initial A-B is followed by a withdrawal of intervention resulting in an A-B-A design. In traditional behavioral analysis research, effectiveness of a treatment in an A-B-A design would be concluded if a client demonstrated no improvement on the dependent vari- able during the A phase, showed reliable improvement in the B phase, and returned to a lesser state of improvement during the second A phase (see Figure 1). Interpretation of such a pattern would conclude that when treatment is being offered, problematic behaviors are reduced. Yet A- B-A is also cited as a limited design because of its lack of replication. The replication of reversal design is used to strengthen the conclusion that improvement is attribut- able to the intervention (Kennedy, 2005). Therefore, the A-B-A-B design is considered the most rigorous of the withdrawal designs in which the intervention is withdrawn and reintroduced multiple times (Kratochwill et al., 2010). Examples of multiple replication of the A-B-A-B design include A-B-A-B-A-B or A-B-A-B-C-A.

Currently, the use of A-B and A-B-A designs is contro- versial when interpreting treatment effectiveness. Given that

FiguRe 1

example of an A-B-A Design for Client 1

Note. Numbers on x-axis denote weekly data collection points. Scores on y-axis are T scores from a depression self-report. In the initial A phase, no intervention was provided. In the B phase, Client 1 participated in motivational interviewing sessions one time per week. In the subsequent A phase, the client participated in no intervention.

80 —

70 —

60 —

50 —

40 —

30 —

20 —

10 — 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

WeekD ep

re ss

io n

S el

f- R

ep or

t T

S co

A Phase B Phase A Phase

Journal of Counseling & Development ■ October 2015 ■ Volume 93396

Ray

they are difficult to replicate, Kratochwill et al. (2010) did not consider A-B or A-B-A designs to be contributory to evidence-based practices in their What Works Clearinghouse document. However, a large number of SCRD researchers (Barlow et al., 2009; Gallo et al, 2013; Kazdin, 2011; Ken- nedy, 2005; O’Neill, McDonnell, Billingsley, & Jenson, 2011; Riley-Tillman & Burns, 2009; Vannest et al., 2013) recognize A-B and A-B-A as valuable contributions to the examination of interventions, even with the inherent limitations. Morgan and Morgan (2009) stated that A-B-A is “a simple yet power- ful means of assessing the effects of the independent variable on behavior, and it has played a significant role in both basic and applied behavioral research” (p. 100).

In the field of counseling, replicated withdrawal/reversal designs are problematic in implementation. During the B phase, a counseling intervention is introduced. Typically, the goal of a counseling intervention is to have a lasting effect, such as clients experiencing fewer symptoms of depres- sion because they feel accepted and understood, or clients learning to identify the effect of thought on emotions and behaviors. Hence, theoretically, when the intervention is withdrawn, the client is likely to continue to show improve- ment. When the intervention is introduced once again (i.e., A-B-A-B), it would be difficult to discern if improvement is due to the first B intervention, personal reflection during the withdrawal of the A phase, or the second B intervention, and so on. To complicate the issue further, the introduction of another intervention (i.e., A-B-A-C) would cause further entanglement of effects. The effect of one treatment on the adjacent treatment of another, or on the withdrawal phase, is referred to as carryover effect (Barlow et al., 2009). Car- ryover effects indicate that the client has learned or experi- enced something that cannot be unlearned or unexperienced. Most counseling interventions are subject to these carryover effects and thereby present complications to the application of the A-B-A-B design. A-B-A-B designs are considered inappropriate for interventions that promote learning or experiences that cannot be reversed (Plavnick & Ferreri, 2013; Vannest et al., 2013). Gallo et al. (2013) noted that even though A-B design is not the most rigorous SCRD, it is valuable when other experimental methods are not possible. As a result of these challenges with replication withdrawal designs, Ray, Barrio Minton, Schottelkorb, and Garofano Brown (2010) encouraged the use of A-B-A designs for counseling interventions, allowing for the examination of only one intervention and avoiding the residual effects of multiple treatments.

Fortunately, the SCRD researcher has other options among designs. A second option considered to be accept- able in evidence-based review is the alternating treatment design (Kratochwill et al., 2010). In this design, different interventions are alternated following the baseline phase, and continuous measurement indicates the effect of each

intervention. However, this design is again problematic in counseling research because of the carryover effect. An- other more practical option for the counseling field is the multiple baseline design, the most well-known alternative to withdrawal/reversal designs (Barlow et al., 2009) and considered contributory to evidence-based review (Krato- chwill et al., 2010). Multiple baseline design is the strategy of choice when effects of an intervention or ethical concerns prohibit the removal of the intervention (Kennedy, 2005). Multiple baseline design involves the replication of the A-B experiment across participants, settings, or behaviors. Multiple baseline studies involve three possible applica- tions, including the measurement of three or more targeted behaviors for one participant, a target behavior across three or more settings, or the same target behavior across multiple participants. Just as the replication of A-B is essential to the rigor of a withdrawal design, the replication of A-B among multiple participants, in multiple settings or on multiple behaviors, allows for the increased credibility of results. In counseling research, multiple baseline design across behaviors may be challenging because the intervention is applied to specific behaviors, thereby linking the tailored intervention to three or more discrete behaviors. The holistic nature of some counseling interventions, such as humanistic approaches, may be misaligned with this behavioral focus. Multiple baselines across settings may also present prag- matic difficulties to the SCRD researcher because it may be challenging to collect data for a participant across three or more different settings.

Multiple baseline across subjects appears better aligned with counseling research, yet still offers obstacles. In the multiple baseline across participants design, the researcher identifies three or more individuals who are similar in pre- sentation, and then staggers the introduction of the interven- tion (B) phase. The baseline interval varies from participant to participant. Conclusions resulting from this particular design are based on replication in which across individuals, the targeted behavior did not improve over varied intervals of the baseline, and yet the intervention phase resulted in improvement for each participant (see Figure 2). Multiple baseline design across participants eliminates the problem of carryover effects within an individual and still requires few participants to demonstrate the replication of effect, making this design a viable alternative to withdrawal de- signs. However, multiple baseline design across participants requires a lengthy extension of a research study. In between- group studies, a researcher sets a pretesting point, presents a specified number of intervention sessions, then completes the study with a finite point of post testing. All SCRDs require a baseline phase that extends until stability of performance is met, which is unknown to the researcher at the initiation of the study. This particular feature of SCRD typically results in lengthier periods of research when compared with between-

Journal of Counseling & Development ■ October 2015 ■ Volume 93 397

Single-Case Research Design and Analysis

group studies. Multiple baseline design across participants will likely extend the research period even further, affecting the practical implementation of the design. However, to date, multiple baseline design appears to be the best SCRD option in matching rigorous design to counseling research.

Steps in Conducting SCRD The following section presents step-by-step instructions for conducting an SCRD, from origination of the design to analysis of the data. I also discuss possible obstacles in the application of SCRD to counseling settings and ways to overcome these challenges.

Step 1: Define the Research Question

The goal of SCRD is to examine the functional relationship between the implementation of an intervention and the pos- sible effects of that intervention. Kratochwill et al. (2010) sug- gested that SCRDs aim to answer, “Is this intervention more effective than the current ‘baseline’ or ‘business-as-usual’ condition?” (p. 3). The counseling researcher considers this when formulating the research question. Examples of appro- priate SCRD research questions include “What is the impact of child-centered play therapy on the empathy levels of young children exhibiting bullying behavior?” or “Is motivational interviewing effective in decreasing depressive symptoms among clinically depressed college students?” As part of the research question, the researcher will define the independent and dependent variables of interest. In SCRD, the independent variable is the intervention being examined. The dependent variable is historically an overt behavior that is problematic; however, the dependent variable can be symptoms, behaviors, or emotional states of the participants. Dependent variables in SCRD must be measurable and theoretically targeted by the intervention.

Step 2: Identify the Subject(s)

SCRD requires the participation of one subject, yet replication is a characteristic feature that lends credibility to SCRD inter- pretation of results. In multiple baseline across participants design, inclusion of three replications (i.e., three participants) is a minimal standard, whereas four or more adds strength to the design (Barlow et al., 2009; Gallo et al., 2013; Krato- chwill et al., 2010). Additionally, the inclusion of multiple participants protects an SCRD study from possible breakdown caused by individual attrition. Because SCRD requires the implementation of multiple phases, including phases in which the client is receiving no intervention, it is common to lose participants over a lengthy period of data collection.

In all experimental designs, specification of the sample is a key component whereby the researcher is charged with clarifying inclusion and exclusion criteria, providing detailed demographic data, and describing participants in relationship to their selection for the study (Chambless & Hollon, 1998). Specificity of description is especially salient in SCRD be- cause of the qualitative nature of the design. SCRD research- ers require full demographic and historical information on each participant, which is typically collected through initial and subsequent interviews. Such qualitative information lends context for the interpretation of data, a unique advantage of SCRD over between-group studies.

Step 3: Choose a Measurement/Instrument

Experimental design requires that the dependent vari- able is assessed through reliable and valid measures. An outcome assessment should be valid, reliable, and address “significant dimensions” of the problem being

FiguRe 2

example of a Multiple Baseline Across Participants Study

Note. Baselines are staggered and extended for each subsequent client.

90 —

80 —

70 —

60 —

50 —

40 —

30 —

20 — 1 2 3 4 5 6 7 8 9 10 11 12 13

Week

D ep

re ss

io n

S el

f- R

ep or

t T

S co

re C

lie nt

A Phase B Phase

90 —

80 —

70 —

60 —

50 —

40 —

30 —

20 — 1 2 3 4 5 6 7 8 9 10 11 12 13

C lie

nt 1

90 —

80 —

70 —

60 —

50 —

40 —

30 —

20 — 1 2 3 4 5 6 7 8 9 10 11 12 13

C lie

nt 2

Journal of Counseling & Development ■ October 2015 ■ Volume 93398

Ray

studied (Chambless & Hollon, 1998, p. 9). Additionally, SCRD requires that measurements can be administered frequently over a short period of time. Instrument manu- als report development and sampling procedures used to examine the psychometric properties of the assessment. Comprehensive assessment manuals also include pro- cedures used to establish reliability and validity of the instrument. The development of instruments purely for the purpose of measuring effects in specific single-case studies is discouraged. Measurement development is re- search in and of itself and typically, when a counseling researcher designs an instrument only for use in a current study, the measure has little reliability and validity and is created with bias (Ray & Schottelkorb, 2010).

Characteristically, SCRD includes observational mea- sures in which the overt behavior of participants is ob- served by objective raters. Examples of such observational measures may include the tally of inattentive behaviors over a period of 5 minutes or frequency counts of aggres- sive statements. When using direct-observation measures, interobserver/rater agreement or reliability is required (Kazdin, 2011). Kratochwill et al. (2010) reported that each outcome variable in SCRD must be measured systematically over time by more than one observer, agreement must be collected for each phase, and 20% of the data must meet minimal thresholds for agreement. Instructions for estab- lishing interobserver agreement can be found in several volumes on SCRD (Kazdin, 2011; Kennedy, 2005; O’Neill et al., 2011; Vannest et al., 2013).

In counseling research, the measurement of overt behaviors may not encompass the focus of the study. For example, in assessing the effect of a cognitive behavioral intervention on a client’s frequency of depressive thoughts, a researcher is unable to concretely observe thought patterns. In this ex- ample, a researcher will rely on client report of thoughts for data collection. In another example, a counseling researcher may be interested in the effect of person-centered counsel- ing on a client’s level of self-regard. Self-regard is a latent and internal construct that may not be observed in outward behaviors. However, the counselor researcher might select an established instrument with reasonable reliability and valid- ity to administer to the client following each session. Kazdin (2011) supported the utility of self-report and other-report in SCRD, especially in clinical settings.

A final consideration in assessment selection for SCRD is the inclusion of multiple assessments to support the focus of the study. A researcher may select one instrument that is used frequently and repeatedly to isolate the effect on the dependent variable, but other instruments can be administered periodi- cally to assess other effects tangential to the target focus (Ka- zdin, 2011). These additional instruments lend support and context to the focus of the study. For example, a counseling researcher may use a thought log for the client to self-report

the number of negative thoughts occurring throughout the day and may also administer an overall depression inventory after every four intervention sessions. In this case, data from the thought log would be used for visual inspection of change but periodic data from the depression inventory is reported to provide context linking negative thoughts to the client’s overall level of depression.

Two major obstacles impede measurement selection for SCRD. First, behavior observation is often difficult in set- tings where clients attend counseling. Counseling sessions are normally focused on the client and the client’s current concerns. Sessions are not conducive to the rating of specific overt behaviors. Observing behaviors in other adult client settings, such as work or home, can be intrusive and impracti- cal. One exception to this obstacle is the observation of child clients, which can easily take place in school settings with little intrusion or disruption to the client. When counseling researchers find that observation measures are unrealistic, they may choose self-report and other-report instruments. However, the second obstacle is a lack of available instruments that have reasonable psychometric properties and can be administered frequently. One option is to contact instrument authors to request additional information regarding the use of an instrument under repeated conditions or its reliability/ validity properties.

Step 4: Define the Intervention

Experimental designs require specificity of a treatment pro- tocol. For some, a treatment protocol indicates the use of a treatment manual that outlines the philosophy, methods, tech- niques, order of delivery, counselor qualifications, and setting (Chambless & Hollon, 1998). Yet, others (Kratochwill et al., 2010; Wampold, Lichtenberg, & Waehler, 2002) encourage the use of a basic protocol or levels of an intervention description. Consensus indicates that an intervention should be thoroughly described, offering clarity to the independent variable and its implementation. The implementation of the counseling in- tervention includes the delivery and monitoring of counselor training, along with supervision and accountability procedures to ensure adherence to treatment fidelity (Chambless & Hollon, 1998). Assessment of counselor training and fidelity ensures the integrity of the independent variable in SCRD.

Step 5: Select the SCRD

Numerous considerations are involved in the selection of the SCRD applied to a study. As presented earlier, the counseling researcher has several options, extending from the simple A-B protocol to complicated replication designs. Prominent considerations for selection include the address of carryover effects, ethical obligations, resources such as availability of interventionists and raters, time limitations, and attention to rigor. Withdrawal replication designs such as A-B-A-B and multiple baseline designs attend to the

Journal of Counseling & Development ■ October 2015 ■ Volume 93 399

Single-Case Research Design and Analysis

issue of rigor and are considered the most contributory to evidence-based intervention. However, A-B and A-B-A designs are considered valuable and informational, even with inherent limitations. Withdrawal/reversal designs, such as A-B-A and A-B-A-B, are inappropriate when the intervention is intended to have a lasting effect. Multiple baseline protocols typically extend the length of SCRD studies. The delay of an intervention to assess the possible deterioration of client status may be unethical and consid- ered client abandonment in some severe cases. Access to and participation of trained counselors and raters to imple- ment an intervention and data collection may be limited. Based on these considerations, the counseling researcher will select an SCRD that allows for the highest level of rigor in the context of practical limitations.

Once an SCRD has been selected, phase protocol should be determined prior to implementation. In the development of an SCRD, the counseling researcher outlines the amount of time delegated to each phase and what intervention will be implemented at what intervals throughout the phases (Ray & Schottelkorb, 2010). Because SCRD is an individualized approach to research, flexibility is characteristic of design implementation. Phase protocol can be altered in real time as the researcher analyzes data (Kennedy, 2005). For example, if a counseling researcher observes that data in the B phase, initially planned to be five data collection points, indicates a marked decrease in self-reported compulsive behaviors, the researcher may extend the B phase to 10 data collection points to observe when stability of behaviors is reached.

Step 6: Establish a Baseline

The unique and central feature of SCRD is its baseline, and its importance cannot be emphasized enough. Because each participant serves as his or her own control, the base- line phase is initiated to document the type of reports or behaviors demonstrated over time when no intervention is implemented. Demonstration of a stable baseline, in which the client’s behaviors or status remains the same or deterio- rates, is the comparison point for intervention. An unstable baseline, in which some data points are high while others are low, undermines the interpretation of any subsequent phases. Baseline phases must be stable for the successful completion and interpretation of SCRD.

Vannest et al. (2013) reported that few studies report stable baselines, which discredits authors’ attempts to in- terpret the results. Traditionally, three data points have been cited as the minimal standard for baseline data collection (Kennedy, 2005). However, Vannest et al. proposed that three points of data are not reliable, and suggested that although five may be enough, nine points of data are recommended for reliable representation of behavior. Most important, baseline data should continue to be collected until data are fairly stable and interpretable (Morgan & Morgan, 2009).

The lack of certainty regarding the length of the baseline is one of the drawbacks of SCRD. Researcher flexibility is crucial to establishing a usable baseline, which sets the foun- dation for the interpretation of intervention implementation. Establishing a stable baseline is predicated on the immediate graphing of data as it is collected. Inexperienced counseling researchers may be tempted to collect several points of data before they begin to graph results. Such a mistake leads to the assumption that stability has been reached and premature movement to the B phase, a decision that is invalidated when it is discovered that the baseline phase never provided stabil- ity from which to interpret intervention effects. In cases or settings in which time may be limited, a researcher will need to balance the positive and negatives effects of extending a baseline. If the extension of a baseline fatally jeopardizes the implementation of a study, the researcher may decide to move forward with less than the desired stability of a baseline to continue the study, and track future changes that may be more interpretable.

Step 7: Implement a Phase Protocol and Measure at Multiple Observation Points

Following the establishment of a stable baseline, planned phases are implemented according to the selected SCRD. All SCRDs require multiple data collection points, which are administered frequently. In traditional behavioral analysis SCRD, time intervals between data collection points may be as short as 1 minute. Counseling research seems better suited to data collection intervals of daily, biweekly, or weekly points. Any longer interval than 1 week may extend the length of an SCRD to an impractical period of time. Kratochwill et al. (2010) required that each phase have a minimum of three data points to be considered an attempt to demonstrate an effect. However, five data points per phase is necessary for a study to meet standards for evidence-based research. As data points increase within phases, the reliability of trend interpretation is improved, lending credibility to one’s findings.

Table 1 provides an example of protocol design for an A- B-A study. In this example study, the counseling researcher is investigating the relationship between mindfulness-based training and anxiety symptoms among participants who met inclusion criteria for being clinically anxious. The five data collection points for each phase of the study are intended to provide adequate data to determine the trend of change. The dependent variable of anxiety is being measured with a self- report anxiety symptom instrument on a weekly basis. The researcher is collecting additional data using a stress inventory at the beginning of each phase and following the completion of phases to provide additional information on the related variable of stress to the dependent variable of anxiety. Finally, to enhance the qualitative context, the researcher conducts a structured personal interview with the client at the first and last week of data collection.

Journal of Counseling & Development ■ October 2015 ■ Volume 93400

Ray

Step 8: Conduct Data Analysis and Interpret the Results

Visual analysis is the preferred and traditional method of data analysis in SCRD. Specifically, the analysis of baseline to intervention contrast is the most “authentic demonstration of intervention effect” (Vannest et al., 2013, p. 61). Kratochwill et al. (2010) proposed that visual analysis involves four steps and six variables for consideration. The four steps include documentation of a predictable baseline pattern, examination of data within each phase to assess patterns, comparison of data between phases to assess the effect of intervention, and integration of information from all phases to determine if there are at least three demonstrations of an effect. In Figure 3, data from Client 1 are examined through visual analysis. Client 1 demonstrated a stable baseline across five data points in which depression symptoms remained at a relatively high and consistent level across weeks of no intervention. Hence, the first step has provided a foundation from which to compare subsequent data.

There are six variables that should be considered to ex- amine within and between phase patterns in visual analysis (Kratochwill et al., 2010). The first feature is level and refers to the mean of each phase. Client 1’s self-report resulted in a mean of 75.4 for the baseline phase, which is well beyond threshold scores that meet the criteria for clinical depression.

The mean from Phase B was 62.75, well below the threshold scores for clinical depression. Thus, the level in this A-B design demonstrates a marked improvement between phases. The second feature of interest is the trend, referring to the slope of the data within phase. For Client 1, the slope indicates a slight increase in depression self-report across the baseline phase with R2 = .09, and then a strong trend downward during the B phase with R2 = .53. Again, the slope indicates a posi- tive trend during the intervention phase. Variability, the third consideration in visual analysis, is the amount of difference between the trend and each individual data point within a phase, often expressed through the range or standard devia- tion of data (Kratochwill et al., 2010). Data from Client 1 fall within the range 72 to 77, with a standard deviation of 2.07 in the baseline phase, whereas the range across the B phase was 60 to 70, with a standard deviation of 3.28. The ranges and standard deviations from Client 1’s data indicate low variability within phases, signifying stability of performance.

The fourth variable to consider in visual analysis is the immediacy of the effect, which provides information on how quickly the intervention demonstrated an effect, as evidenced by change in the data patterns when the interven- tion was offered. In the case of Client 1, Figure 3 indicates a substantial decrease in self-reported depression symptoms within the first three data collection points following the introduction of the intervention, thereby offering convinc- ing evidence that change was due to manipulation of the independent variable (Kratochwill et al., 2010). However, it should be noted that immediacy of effect might be delayed when intervention effects are cumulative, as in a counsel- ing relationship (Vannest et al., 2013). The fifth variable is consideration of overlap, involving the comparison of the proportion of data in one phase that overlaps with data in the previous phase. Low rates of overlap indicate a larger effect of treatment (Kratochwill et al., 2012). In Figure 3, there were no points of data within the baseline phase that overlapped with the intervention phase. The lowest point in the baseline, 72, was still higher than the highest point of 70 in the intervention phase, signifying that all points of intervention were an improvement over any of the points in baseline. The final variable of consideration in visual

Note. N = no intervention; MBT = mindfulness-based training; AS = anxiety symptom self-report; SI = stress inventory; PI = personal interview.

TABLe 1

A-B-A Design intervention and Data Collection Protocol example

Variable

Phase Intervention Data collection

1 2 3

A N

AS, SI, PI

4 5 6 7 8 9 10 11 12 13 14 15

Week

A N

B MBT

AS, SI

B MBT AS

A N

AS, SI

A N

AS, SI, PI

FiguRe 3

Visual Analysis graph for Client 1 Across A-B Phases Note. PND = percentage of all nonoverlapping data.

90 —

80 —

70 —

60 —

50 —

40 —

30 — 1 2 3 4 5 6 7 8 9 10 11 12 13

WeekD ep

re ss

io n

S el

f- R

ep or

t T

S co

A Phase B Phase

R2 = .09 PND = 100%

R2 = .53

M = 75.40 M = 62.75

Journal of Counseling & Development ■ October 2015 ■ Volume 93 401

Single-Case Research Design and Analysis

analysis is consistency of data patterns across similar phases. This variable requires the review of replication of phases. In the case of Client 1, the researcher would refer back to Figure 2 to compare results among Clients 1, 2, and 3, and examine the similarity of data patterns. Consistent data patterns across all three participants provide replication of results and support the credibility of findings.

Although visual analysis is largely supported as the most reliable method to interpret SCRD findings, the application of statistical analyses is discussed throughout SCRD litera- ture. The use of parametric analyses has been determined by some SCRD researchers as inappropriate for single-case data (Barlow et al., 2009; Vannest et al., 2013). Inferential statistics require certain assumptions of data, including normality of distribution and independence of observations. In SCRD, normal distribution is negatively affected by the low numbers of data points in small studies. This would be especially true of counseling SCRDs if data are collected on a weekly basis, thereby resulting in fewer points of data. Additionally, because SCRD requires repeated observa- tions of the same phenomenon over time, observations are most likely related (i.e., autocorrelation) and fail to meet the parametric assumption of independent observations (Barlow et al., 2009). The failure to meet assumptions for inferential statistical analysis is a likely scenario for small- scale counseling SCRDs.

However, the need for the objective analysis of data, along with the inappropriate application of inferential statistics, has given rise to the discussion and use of effect size esti- mations. Vannest et al. (2013) noted that there are currently 10 published nonparametric methods that examine overlap in data sets. Kratochwill et al. (2010) reported that there is no agreement on effect size methods for SCRD. There are several reviews of effect size estimations in the literature that closely examine the application of various procedures. Parker, Vannest, and Davis (2011) reviewed nine techniques and concluded that there are strengths and limitations to each method, whereas Wolery, Busick, Reichow, and Barton (2010) determined that there are no current suitable methods for calculating effect sizes in SCRD. There are three widely used effect size estimations that have been popularized in the literature and reviewed for adequacy by multiple research- ers. These three include percentage of nonoverlapping data (PND), percentage of all nonoverlapping data (PAND), and percent of exceeding the median (PEM; Kratochwill et al., 2010). Although researchers have noted significant limitations with each of these methods, they continue to be widely used and discussed. Parker and Hagan-Burke (2007) suggested that PND and PAND are intuitively ap- pealing and relate directly to visual analysis. Although Lenz (2013) found variable results when comparing PND, PAND, and PEM, he cited the strengths along with the limitations of each method. Full review and instructions on calcula-

tions of these three methods can be found in Lenz (2013) and Parker et al. (2011). However, a counseling researcher would benefit from reviewing all of the current effect size estimations before deciding on one method. Additionally, new information about the statistical applications for SCRD appear to be developed on a consistent basis, requiring that the SCRD researcher keep abreast of new findings and ap- plications. Barlow et al. (2009) cautioned that the use of statistics does not ensure that results are meaningful and should be used within the context of the statistics’ clinical relevance for interpretation.

Conclusion SCRD is a viable and worthy option for practitioners and researchers who wish to contribute valuable information on intervention effects. The ultimate goal of SCRD, as with all experimental designs, is to discover generalizations (Kazdin, 2011). Although case studies in counseling literature have been helpful in providing deeper understanding of clients and counseling relationships, SCRD uses experimental design to allow the interpretation of causal implications. The involve- ment of a small number of participants encourages the appli- cation of SCRD among practitioners who seek to evaluate an intervention. Because the traditional application of SCRD has been in the behavioral analysis sector, counseling research- ers may find the method difficult to translate to counseling settings. Experimental controls present some challenges to SCRD implementation, but most of these obstacles can be overcome through creative problem solving. The goal of this article was to provide the reader with a description of SCRD methods and a discussion of issues prominent in SCRD. Ad- ditionally, I hoped to encourage counselors to venture into the world of SCRD by providing steps for implementation of an SCRD. Although a detailed discussion of SCRD intricacies is beyond the scope of this article, the reader is introduced to the application of SCRD to the study of counseling intervention.

References Barlow, D., Nock, M., & Hersen, M. (2009). Single case experimental

designs: Strategies for studying behavior for change (3rd ed.). Boston, MA: Pearson Education.

Chambless, D., & Hollon, S. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66, 7–18. doi:10.1037/0022-006X.66.1.7

Gallo, K., Comer, J., & Barlow, D. (2013). Single-case experimental designs and small pilot trial designs. In J. Comer & P. Kendall’s (Eds.), The Oxford handbook of research strategies for clinical psychology (pp. 24–39). New York, NY: Oxford University Press.

Kazdin, A. (2011). Single-case research designs: Methods for clinical and applied settings (2nd ed.). New York, NY: Oxford University Press.

Journal of Counseling & Development ■ October 2015 ■ Volume 93402

Ray

Kennedy, C. H. (2005). Single-case designs for educational research. Boston, MA: Pearson.

Kratochwill, T., Hitchcock, J., Horner, R., Levin, J., Odom, S., Rindskopf, D., & Shadish, W. (2010). Single-case designs technical documenta- tion. Retrieved from http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf

Kratochwill, T., Hitchcock, J., Horner, R., Levin, J., Odom, S., Rindskopf, D., & Shadish, W. (2012). Single-case intervention research design standards. Remedial and Special Education, 34, 26–38. doi:10.1177/0741932512452794

Lenz, A. S. (2013). Calculating effect size in single-case research: A comparison of nonoverlap methods. Measurement and Evalu- ation in Counseling and Development, 46, 64–73.

Morgan, D. L., & Morgan, R. K. (2009). Single-case research methods for the behavioral and health sciences. Thousand Oaks, CA: Sage.

O’Neill, R., McDonnell, J., Billingsley, F., & Jenson, W. (2011). Single case research designs in educational and community settings. Upper Saddle River, NJ: Pearson Education.

Parker, R., & Hagan-Burke, S. (2007). Useful effect size interpre- tations for single case research. Behavior Therapy, 38, 95–105. doi:10.1016/j.beth.2006.05.002

Parker, R., Vannest, K., & Davis, J. (2011). Effect size in single-case research: A review of nine nonoverlap techniques. Behavior Modification, 35, 303–322.

Plavnick, J., & Ferreri, S. (2013). Single-case experimental designs in educational research: A methodology for causal analyses in teaching and learning. Educational Psychology Review, 25, 549–569. doi:10.1007/s10648-013-9230-6

Ray, D., Barrio Minton, C., Schottelkorb, A., & Garofano Brown, A. (2010). Single-case design in child counseling research: Implications for counselor education. Counselor Education and Supervision, 49, 193–208. doi:10.1002/j.1556-6978.2010. tb00098.x

Ray, D., & Schottelkorb, A. (2010). Single case design: A primer for play therapists. International Journal of Play Therapy, 19, 39–53. doi:10.1037/a0017725

Riley-Tillman, T. C., & Burns, M. (2009). Evaluating educational interventions: Single-case design for measuring response to intervention. New York, NY: Guilford Press.

Scruggs, T. E., & Mastropieri, M. A. (1998). Summarizing single subject research: Issues and applications. Behavior Modification, 22, 221–242. doi:10.1177/01454455980223001

Sharpley, C. F. (2007). So why aren’t counselors reporting n = 1 research designs? Journal of Counseling & Development, 85, 349–356. doi:10.1002/j.1556-6678.2007.tb00483.x

Vannest, K., Davis, J., & Parker, R. (2013). Single case research in schools: Practical guidelines for school-based professionals. New York, NY: Routledge.

Wampold, B., Lichtenberg, J., & Waehler, C. (2002). Principles of empirically supported interventions in counseling psychology. The Counseling Psychologist, 30, 197–216.

Wolery, M., Busick, M., Reichow, B., & Barton, E. (2010). Comparison of overlap methods for quantitatively synthe- sizing single-subject data. Journal of Special Education, 44, 18–28.

Copyright of Journal of Counseling & Development is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.

Copyright of Journal of Counseling & Development (John Wiley & Sons, Inc.) is the property of John Wiley & Sons, Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.