EDMG611Wk6

Journal of Counseling & Development ■ October 2015 ■ Volume 93412 © 2015 by the American Counseling Association. All rights reserved.

Received 08/05/14 Revised 11/10/14

Accepted 11/17/14 DOI: 10.1002/jcad.12039

The American Counseling Association (ACA) has called for counselors to be actively involved in research that informs practice. The ACA Code of Ethics (2014) states that “Coun- selors who conduct research are encouraged to contribute to the knowledge base of the profession and promote a clearer understanding of the conditions that lead to a healthy and more just society” (Section G., p.15). The ACA Code of Ethics (2014) also advises counselors to “plan, conduct, and report research accurately” (Section G.4.a., p.16). Thus, accurately reporting the results of one’s research is an important task for counselors and counselor educators. Furthermore, counseling researchers are expected to contextualize their findings and write results in a way that others can understand (Wester & Borders, 2011). Therefore, in addition to conducting rigorous, high-quality research, counselors are also expected to clearly communicate their findings to others.

Within the Results section of single-case research design (SCRD) studies, data are commonly presented graphically for each participant of interest and collectively for a study (Wolery, Dunlap, & Ledford, 2011). Graphs are essential in single-case research because visual analysis continues to be the traditional method for determining meaningful effects, although it is not without criticism (Parker & Hagan-Burke, 2007). Reporting visual inspection results along with a non- parametric or overlap metric indicating the degree of treat- ment effect is preferred (see Lenz, 2013; Vannest & Ninci, 2015). Furthermore, including a graph within the Results section provides readers with multiple pieces of information at one time and allows readers to make their own interpretation

Brittany L. Hott, Department of Psychology, Counseling, and Special Education, Texas A&M University–Commerce; Dodie Lim- berg and Jonathan H. Ohrt, Department of Educational Studies, University of South Carolina; Michael K. Schmit, Department of Counseling and Educational Pyschology, Texas A&M University–Corpus Christi. Correspondence concerning this article should be addressed to Brittany L. Hott, Department of Psychology, Counseling, and Special Education, Texas A&M University–Commerce, PO Box 3011, Commerce, TX 75429 (e-mail: [email protected]).

Reporting Results of Single-Case Studies Brittany L. Hott, Dodie Limberg, Jonathan H. Ohrt, and Michael K. Schmit

This article provides an overview of reporting and presenting single-case study results. The authors offer practical tips for sharing outcomes of studies, reporting visual inspection measures, conducting common statistical procedures, and disseminating results to stakeholders. Specific implications for counselors are provided.

Keywords: single-case research design, results

of treatment effects. Conventionally, the dependent variable is located on the ordinate or y-axis and the temporal variable is located on the abscissa or x-axis (Barlow, Nock, & Hersen, 2009). As with other research designs, there are numerous ways to effectively communicate the results of single-case studies. The purpose of this article is to provide an overview of the reporting procedures and essential components needed for SCRDs.

General Categories of Reporting Before reporting study outcomes, it is important to share information related to procedural integrity and data collection measures. These processes often include (a) interobserver agreement (IOA), (b) treatment fidelity, and (c) any unfore- seen circumstances that occurred during data collection. Next, measures of treatment effect are commonly reported.

Study Processes That Affect the Outcome

IOA. Counselors’ perceptions of a participant’s response can vary, and when this occurs, confidence issues are introduced in data recording that may lead to inaccurate conclusions. IOA assesses the precision of the measurement procedures used by the researcher and brings credibility to one’s findings. Therefore, it is helpful to have a second clinician check scores on permanent products (e.g., assessment protocols) or observe clients (i.e., behavioral observations). Typically, a minimum of 30% of permanent products (Kennedy, 2005) and at least 20% of behavioral observations (Kratochwill & Levin, 2010)

Journal of Counseling & Development ■ October 2015 ■ Volume 93 413

Reporting Results of Single-Case Studies

should be reviewed per phase. Numerous statistical methods are available to calculate IOA (e.g., permanent product agree- ment, Cohen’s kappa, prevalence and bias-adjusted kappa– ordinal scale). However, the selection of a specific method depends on the data recording procedures and the dimension of the target behavior being measured (Gast, 2010).

One common and relatively simple method used to cal- culate IOA for frequency data is applying a simple point- to-point agreement formula for each observation. The total number of agreements and disagreements are calculated, then agreements are divided by agreements plus disagree- ments, and then the value is multiplied by 100 (Kazdin, 2011). To analyze interval data, a total agreement formula can be used. Total agreement can be calculated by determin- ing the least amount of time divided by the greater amount of time, then multiplying by 100 (Alberto & Troutman, 2013). Researchers should randomly obtain IOA data on 20%–33% of their observations during the baseline, intervention, and each subsequent phase (Kazdin, 2011; O’Neil, McDon- nell, Billingsley, & Jensen, 2011). A high IOA between researchers is desirable in SCRD because it represents that there is an established level of confidence, which indicates that the target behavior is accurately defined operationally and changes in the dependent variable are a function of independent variable manipulation.

To illustrate IOA, consider how two counselors handled research related to Jerry, a fictional client with posttraumatic stress disorder (PTSD). To assess IOA in Jerry’s case, 20% of baseline and treatment sessions were randomly selected. During the observations, both counselors independently scored the PTSD-interview protocol and heart rate levels. The following is a sample of how to report IOA: To ensure accuracy of data, a second counselor reviewed a minimum of 20% of the observations and data collected. IOA for the PTSD interview was 100% during the baseline and treatment phases and 97% for the return-to-resting heart rate. IOA calculations indicated that client progress was accurately reported.

Fidelity of treatment. Fidelity of treatment refers to how faithful counselors are in delivering an intervention across the treatment phase. To assess fidelity of treatment, Adamson and Wachsmuth (2014) recommended establishing an opera- tionalized definition that clearly identifies the independent variable. A simple checklist including the key components of the interventions and materials can be used to assess treatment integrity. Fidelity data should be collected that is representative of the entire treatment phase (Lane & Beebe- Frankenberger, 2004). Additionally, O’Neill et al. (2011) sug- gested that a minimum of 25% of treatment sessions per phase be observed to calculate treatment fidelity, but recommended that researchers align with the quality established for IOA. Thus, a higher agreement between observations on treatment fidelity allows for clearer demonstration of systematic control by the independent variable. Counselors should strive for a

result of 90% or higher during each observation of treatment fidelity (Alberto & Troutman, 2013).

Consider Jerry’s case again: 20% of his sessions were taped, with client permission. A second counselor indepen- dently reviewed eye movement desensitization and repro- cessing (EMDR) treatment procedures using a checklist provided in the intervention protocol. A review of the checklist indicated that a 98% adherence to treatment protocols was attained—thus indicating a high rate of treatment fidelity.

Unforeseen participant and setting changes. Control- ling threats to internal validity increases the likelihood that changes in the dependent variable are a result of manipulat- ing the independent variable (Kennedy, 2005; O’Neil et al., 2011). Unforeseen changes in participants (i.e., maturation) may ultimately confound the results of a study. For example, suppose a study consisted of three participants, and two par- ticipants who showed positive results decided to drop out of the study because of personal reasons. If the one remaining participant suddenly responds to the intervention in a posi- tive way, results would indicate some type of bias. Similarly, unforeseen changes in the interventionist provider can also have an adverse effect. For example, suppose a study consisted of five participants and two interventionist providers, and one interventionist dropped out of the study midway. A similar bias result may occur, making interpretation of the findings difficult. Unpredicted changes in participants that are inconsistent with the previous implementation of the independent variable should raise concerns and be reported in the Results section.

Unexpected change in treatment settings, such as fire alarms, nonparticipant interruptions, equipment failure, and medication changes, can, and often do, influence the interpre- tation of results (Gast, 2010). Furthermore, external validity is compromised. External validity is related to the generalizabil- ity of results beyond the treatment conditions and setting (Bar- low et al., 2009; O’Neill et al., 2011). Counseling treatment is provided in applied settings, such as mental health agencies, private practice, schools, and clients’ homes; therefore, the potential for unexpected events to affect treatment sessions is increased.These threats may result in unexpected changes in participants and the interventionist provider. Such threats are often present in various counseling settings and may actually parallel some similar controlled settings; however, without accurate documentation of the results, counseling research- ers and the counselor may be unaware of these effects when implementing the interventions with clients.

Information Related to Treatment Effect

Although statistical significance tests are available (see Kra- tochwill & Levin, 2010; Parker, Vannest, & Davis 2011), visual analysis continues to be most common method used in single-case research. When reporting results, patterns of data are described within and across each phase of treatment. The variability or consistency of data within each phase, as well as

Journal of Counseling & Development ■ October 2015 ■ Volume 93414

Hott, Limberg, Ohrt, & Schmit

changes in level and trend, are described faithfully before and after each phase (Alberto & Troutman, 2013). Furthermore, a comparison of phase changes is described in the Results section, which enhances systemic control.

Measures of effect. The evaluation of statistical signifi- cance as a singular measure of effect alone is insufficient in counseling research (Thompson, 2002) or any viable social science research (Rapoff, 2009). For counselors to evaluate intervention effectiveness beyond statistical significance, and to assist clients in making an informed decision regard- ing their treatment, results that signify the importance of an intervention (i.e., measures of effect) need to be reported and identifiable in the Results section. Furthermore, coun- seling researchers must measure effects beyond statistical or even practical significance to sufficiently inform practice (Thompson, 2002).

Measures of effect are a critical piece of information used by researchers, practitioners, and the general public to make an informed decision about whether a particular treatment or intervention is useful, and to what degree. Effect size estimates are substantive, and can be standardized measures (i.e., Cohen’s d, h2, R2) or unstandardized measures (e.g., number of times a student speaks out of turn per class period), depending on the parameters of the research question and hypothesis. A growing and needed trend in the counseling literature is reporting estimates of treatment effects (Lenz, 2013). In concordance with the ACA Code of Ethics (2014), the accurate reporting of results (e.g., including the measure of effect) allows others to make informed decisions regarding practice and treatment, and helps to ensure the welfare of the public (G.4.a, p. 16). Precluding measures of effect in empiri- cal studies is not only unethical, but pertinent information is withheld, whether intentional or not, and misaligns with the spirit of research and the counseling profession (e.g., report- ing means without standard deviations; adjusting the scale of visual displays, which resorts in a distortion of the data).

Visual inspection. Historically, the evaluation of mean- ingfulness in single-case research depended on a graphical representation of data and visual inspection of the relationship between the intervention and dependent variable (Nock, Mi- chel, & Photos, 2007). To date, visual inspection remains the most common method in single-case design reporting (Fisher, Kelley, & Lomas, 2003; Ottenbacher, 1990). The graphical representation of data allows for the visual inspection of key components and provides a foundation for the interpretation and determination of an effect.

Visual inspection typically includes multiple analyses such as (a) level, (b) trend, (c) variability, (d) overlap, (e) imme- diacy, and (f) consistency. Level refers to the mean or median of the data within a phase (Kazdin, 2011). When reporting means within each phase, it is also important to include stan- dard deviations. After reporting levels, a discussion of trend, which refers to the slope of the line of best fit within a phase

and variability, is needed. Typically, the amount of deviation from the line of best fit is important (Kratochwill et al., 2010). Overlap involves a calculation of the percentage of data from one intervention phase that is included in the range of data from the previous phase (Scruggs & Mastropieri, 2013). Two common measures of overlap are percentage of nonoverlap- ping data (PND; Scruggs, Mastropieri, & Casto, 1987) and percentage of data exceeding the mean (PEM; Ma, 2006). Ad- ditionally, immediacy of effect is demonstrated when there is a lack of overlap between the last three data points in one phase and the first three points in the next phase (Horner, Swamina- than, Sugai, & Smolkowski, 2012). Therefore, summarizing both the overlap within the data and immediacy of effect are both important. Finally, an evaluation of consistency, which refers to the extent to which data patterns are similar across corresponding phases (Gast, 2010), is helpful. The following synthesis of results provides an example of how a clinician may choose to report treatment effects. It should be noted that an A-B design is used, in which the A phase represents the baseline data and the B phase displays the treatment data. Although A-B designs cannot be used to fully demonstrate a functional relationship between the intervention and baseline, they are practical and can be used to make treatment decisions in applied settings (Alberto & Troutman, 2013; see Barlow et al., 2009; Gast, 2010; and Kazdin, 2011, for other relevant single-case designs).

Returning back to Jerry’s case as an example, PTSD symptom scores were used to evaluate his symptom severity as a function of eye movement desensitization and his rest- ing heart rate. Baseline data indicated clinically significant PTSD symptoms (M = 47, SD = 1.87). After implementing EMDR, an abrupt and continuing decline in PTSD symptoms was evident, as demonstrated by a consistent PTSD Symptom Scale-Interview (PSS-I; Foa, Riggs, Dancu, & Rothbaum, 1993) decrease from 46 to 25. A clear relationship between the intervention and treatment outcomes was evident (PND = 100, PEM = 1.0). Furthermore, maintenance data indi- cated that symptoms continued to be at manageable levels posttreatment. Reponses to EMDR treatment were also ef- fective, as evidenced by an immediate and abrupt decline in Jerry’s return-to-resting heart rate. Data indicated a consistent functionality and meaningful difference between Jerry’s base- line and treatment (PND = 100, PEM = 1.0). Furthermore, Jerry’s heart rate remained stable over the 12 sessions and was maintained posttreatment. The outcomes of treatment appear to be successful.

Statistical approaches. The intervention effects should be strong and clear enough to preclude the need for statisti- cal analysis (Brossart, Parker, Olson, & Mahadevan, 2006; Nock et al., 2007). Within the Results section of SCRD, both metrics representing the visual interpretation and statistical analyses can be used to explicitly and easily convey data to others. Some common nonparametric procedures include (a)

Journal of Counseling & Development ■ October 2015 ■ Volume 93 415

Reporting Results of Single-Case Studies

improvement rate differences, (b) nonoverlap of all pairs, (c) Theil-Sen estimator (Theil, 1950), and (d) Tau-U (see Van- nest, Parker, & Gonen, 2011; Vannest & Ninci, 2015). Like visual inspection techniques, nonparametric procedures have strengths and limitations. Therefore, it is important to explain the formula for calculating treatment effect and the analytic procedure or procedures selected in detail. It may be helpful to include multiple analyses. For example, a study might include visual inspection as well as nonparametric measures, such as Tau-U (see Parker, Vannest, Davis, & Sauber, 2010; Vannest et al., 2011).

Social validity measures. Although reporting multiple measures of effect in the Results section of articles allows the reader to make informed decisions about intervention effectiveness, additional information is needed about both client and clinician perceptions of the intervention. Wolf (1978) identified the construct of social validity, referring to the social significance of an intervention, and the degree of importance indicated by clients, researchers, and society. Social validity is addressed on three different levels related to the degree that (a) an intervention is relevant and of interest to clients and society; (b) is feasible and socially important for clients; and (c) clients are satisfied with the intervention, at least enough to implement it beyond the treatment setting (Kazdin, 1977; Wolf, 1978). In counseling research, the term clinical significance (Thompson, 2002) is related to the broader concept of social validity and is concerned with the differences between groups as a function of the intervention and whether the intervention effects make a noticeable difference in people’s lives (Kazdin, 1999; Thompson, 2002; Wolf, 1978).

Within the Results section, measures of socially validity need to be explicitly addressed (Thompson, 2002) and easily identifiable by the readership (American Psychological As- sociation, 2010). Horner et al. (2005) indicated that measures of social validity are essential and are a quality indicator in SCRD. Furthermore, just as counselors are concerned with multicultural and social justice issues in counseling, research- ers’ attention to the selection of appropriate interventions for treatment is analogous with results from social validity measures. Kazdin (2011) and O’Neill et al. (2011) provided two common methods for assessing social validity. First is an assessment of the client’s subjective experiences of the importance, acceptability, and sustainability of the interven- tion impact. Subsequently, interventionists’ perceptions are also measured and documented, and should align with the client’s indicators of social significance. A second, less com- mon approach can be used through the social comparison of each participant to other individuals’ typical level of perfor- mance within a specific setting. The most common methods used to assess social validity often occur subjectively, either through the use of simple questionnaires or structured and unstructured interviews (Finn & Sladeczek, 2001; O’Neill et al., 2011). However, numerous standardized instruments

are readily available that can be used to assess social validity more objectively (Foster & Mash, 1999; Lane et al., 2009).

Translating Research Into Practice

Results are descriptive and provide statistical information that should parallel researchers’ intended hypotheses and align with the research question (Nock et al., 2007). The organiza- tion of single-case research results have differed in the coun- seling literature. For example, Cox, Lenz, and James (2015) organized results according to each participant; Heppner and Hendricks (1995) addressed results according to each one of their hypotheses; and Swan and Ray (2014) combined par- ticipant results into a single section and described each result accordingly. Given the evidence, the organization of results seems to be independent of the research design and more likely determined by the established research question and hypothesis. Although there are essential elements included in most Results sections, there are a plethora of options to effectively communicate the results of SCRD. Therefore, because of the differences in research questions, each study Results section is unique.

Statistical procedures used by a researcher are disclosed formatively within the Results section of an article with inter- vention effects best understood within their context. Results can be thought of as a culmination of intention and effort set forth by the researcher and transformed purposefully into concise information that indicates to others the outcome of a study. If a statistical analysis is conducted, the method used is clearly identified. A measure of effect should correspond to both the visual analysis and statistical results. Mean scores of each participant should be reported, along with standard deviations for each participant within and across intervention phases.

Counselor educators should engage in outcome-based research that informs practice. Counselors are ethically obligated to implement evidenced-based practices. (ACA, 2014). ACA has mandated that “Counselors plan, conduct, and report research accurately. Counselors do not engage in misleading or fraudulent research, distort data, misrepresent data, or deliberately bias their results” (ACA, 2014, p. 14). Thus, it is crucial for counseling research and practicing counselors to have guidelines when analyzing, interpreting, and sharing results of SCRD.

Evidence-based treatments are most commonly shared with counselors in academic journals, at professional con- ferences, and during professional development trainings. Authors, presenters, and trainers should be able to share the applicable components of research study results in a way that is feasible. Furthermore, an ethical obligation of practicing counselors is to understand how to interpret research and to be consumers of research. Therefore, translating research to practice is both the responsibility of the researcher and the practicing counselor; however, the primary responsibility belongs to the researcher to provide results that are unbiased,

Journal of Counseling & Development ■ October 2015 ■ Volume 93416

Hott, Limberg, Ohrt, & Schmit

clear, and understandable (APA, 2010). Regardless of their specialization, counselors who conduct research have a pro- fessional responsibility to share their findings. Sharing the results of single-case research is no exception, and results should be communicated with stakeholders. However, the interventions and measures may differ across specializations.

Conclusion Counselors are required to implement interventions that are grounded in empirical support. Likewise, counselors are expected to document their own effectiveness by evaluating the services they provide to students or clients. Most impor- tant, counselors must be able to effectively communicate the results to various stakeholders. SCRD is a clinically practical and methodologically rigorous approach that provides coun- selors with a way to evaluate their services and demonstrate the implications in a simplified format. In this article, we described how counselors can interpret the results of SCRD research and translate their findings to relevant stakeholders.

References Adamson, R. M., & Wachsmuth, S. T. (2014). A review of direct

observation research within the past decade in the field of emotional and behavioral disorders. Emotional and Behavioral Disorders, 39, 181–189.

Alberto, P. A., & Troutman, A. C. (2013). Applied behavior analysis for teachers (9th ed.). Upper Saddle River, NJ: Prentice Hall.

American Counseling Association. (2014). ACA code of ethics. Alexandria, VA: Author.

American Psychological Association. (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: Author.

Barlow, D. H., Nock, M. K., & Hersen, M. (2009). Single case ex- perimental designs: Strategies for studying behavioral change. Boston, MA: Pearson.

Brossart, D. F., Parker, R. I., Olson, E. A., & Mahadevan, L. (2006). The relationship between visual analysis and five statistical analyses in a simple AB single-case research design. Behavioral Modification, 30, 531–563. doi:10.1177/0145445503261167

Cox, R. m, Lenz, A. S., & James, R. K. (2015). A pilot evaluation of the ARRAY Program with offenders with mental illness. Journal of Counseling & Development, 93, 471–480.

Finn, C. A., & Sladeczek, I. E. (2001). Assessing the social validity of behavioral interventions: A review of treatment acceptability measures. School Psychology Quarterly, 16, 176–206.

Fisher, W. W., Kelley, M. E., & Lomas, J. E. (2003). Visual aids and structured criteria for improving visual inspection and in- terpretation of single‐case designs. Journal of Applied Behavior Analysis, 36, 387–406.

Foa, E., Riggs, D., Dancu, C., & Rothbaum, B. (1993). Reliability and va- lidity of a brief instrument for assessing post-traumatic stress disorder. Journal of Traumatic Stress, 6, 459–474. doi:10.1002/jts.2490060405

Foster, S. L., & Mash, E. J. (1999). Assessing social validity in clini- cal treatment research: Issues and procedures. Journal of Consult- ing and Clinical Psychology, 67, 308–319. doi:10.1037/0022- 006x.67.3.308

Gast, D. L. (2010). Single subject research methodology in behav- ioral sciences. New York, NY: Taylor & Francis.

Heppner, M. J., & Hendricks, F. (1995). A process and out- come study examining career indecision and indecisive- ness. Journal of Counseling & Development, 73, 426–437. doi:10.1002/j.1556-6676.1995.tb01776.x

Horner, R. H., Carr, E. G., Halle, J., McGee, J., Odom, S., & Wolery, M. (2005). The use of single subject research to identify evidenced based practice in spe- cial education. Exceptional Children, 71, 165–179. doi:10.1177/001440290507100203

Horner, R. H., Swaminathan, H., Sugai, G., & Smolkowski, K. (2012). Considerations for the systematic analysis and use of single-case research. Education and Treatment of Children, 35, 269–290.

Kazdin, A. E. (1977). Assessing the clinical or applied importance of behavior change through social validation. Behavioral Modifica- tion, 1, 427–451. doi:10.1177/014544557714001

Kazdin, A. E. (1999). The meanings and measurement of clinical significance. Journal of Consulting and Clinical Psychology, 67, 332–339.

Kazdin, A. E. (2011). Single-case research designs: Methods for clinical and applied settings (2nd ed.). New York, NY: Oxford University Press.

Kennedy, C. H. (2005). Single-case designs for educational research. Boston, MA: Allyn & Bacon.

Kratochwill, T. R., & Levin, J. R. (2010). Enhancing the scientific credibility of single-case intervention research: Randomization to the rescue. Psychological Methods, 15, 124–144. doi:10.1037/ a0017736

Lane, K. L., & Beebe-Frankenberger, M. (2004). School-based interventions: The tools you need to succeed. Boston, MA: Al- lyn & Bacon.

Lane, K. L., Kalberg, J. R., Bruhn, A. L., Driscoll, S. A., Wehby, J. H., & Elliott, S. N. (2009). Assessing social validity of school- wide positive behavior support plans: Evidence for the reliability and structure of the Primary Intervention Rating Scale. School Psychology Review, 38, 135–144.

Lenz, A. S. (2013). Calculating effect size in single-case re- search: A comparison of nonoverlap methods. Measurement and Evaluation in Counseling and Development, 46, 64–73. doi:10.1177/0748175612456401

Ma, H. H. (2006). An alternative method for quantitative synthesis of single subject research. Behavior Modification, 30, 598–617. doi:10.1177/0145445504272974

Nock, M. K., Michel, B. D., & Photos, V. I. (2007). Single-case research designs. In D. McKay (Ed.), Handbook of research methods in abnormal and clinical psychology, (pp. 337–350). Thousand Oaks, CA: Sage.

Journal of Counseling & Development ■ October 2015 ■ Volume 93 417

Reporting Results of Single-Case Studies

O’Neill, R. E., McDonnell, J. J., Billingsley, F. F., & Jensen, W. R. (2011). Single case research designs in educational and com- munity settings. Boston, MA: Pearson.

Ottenbacher, K. J. (1990). When is a picture worth a thousand $Rh values? A comparison of visual and quantitative methods to analyze single subject data. The Journal of Special Education, 23, 436–449.

Parker, R. I., & Hagan-Burke, S. (2007). Median-based overlap analysis for single case data: A second study. Behavior Modifica- tion, 31, 919–936. doi:10.1177/0145445507303452

Parker, R. I., Vannest, K. J., & Davis, J. L. (2011). Effect size in single-case research: A review of nine overlap techniques. Behav- ior Modification, 35, 303–322. doi:10.1177/0145445511399147

Parker, R. I., Vannest, K. J., Davis, J. L., & Sauber, S. B. (2010). Combining non-overlap and trend for single case research: Tau-U. Behavior Therapy, 42, 284–299. doi:10.1016/j.beth.2010.08.006

Rapoff, M. A. (2009). Editorial: Assessing and enhancing clinical significance/social validity of intervention research in pediatric psychology. Journal of Pediatric Psychology, 35, 114–119. doi:10.1093/jpepsy/jsp102

Scruggs, T. E., & Mastropieri, M. A. (2013). PND at 25: Past, pres- ent, and future trends in summarizing single subject research. Remedial and Special Education, 34, 9–19.

Scruggs, T. E., Mastropieri, M. A., & Casto, G. (1987). The quan- titative synthesis of single-subject research: Methodology and validation. Remedial and Special Education, 8, 24–33.

Swan, K. L., & Ray, D. C. (2014). Effects of child-centered play therapy on irritability and hyperactivity behaviors of children with intellectual disabilities. The Journal of Humanistic Coun- seling, 53, 120–133. doi:10.1002/j.2161-1939.2014.00053.x

Theil, H. (1950). A rank-invariant method of linear and polynomial regression analysis. Indigationes Mathematicae, 12, 85–91.

Thompson, B. (2002). “Statistical,” “Practical,” and “Clinical”: How many kinds of significance do counselors need to con- sider? Journal of Counseling & Development, 80, 64–71. doi:10.1002/j.1556-6678.2002.tb00167.x

Vannest, K. J., & Ninci, J. (2015). Evaluating intervention effects in single-case research designs. Journal of Counseling & Develop- ment, 93, 403–411.

Vannest, K. J., Parker, R. I., & Gonen, O. (2011). Single case re- search: Web based calculators for SCR analysis. (Version 1.0) [Web-based application]. College Station, TX: Texas A&M University.

Wester, K. L., & Borders, L. D. (2011). ACES research competen- cies for the counseling profession. Retrieved from http://www. acesonline.net/

Wolery, M., Dunlap, G., & Ledford, J. R. (2011). Single-case ex- perimental methods: Suggestions for reporting. Journal of Early Intervention, 33, 103–109.

Wolf, M. M. (1978). Social validity: The case for subjective mea- surement on how applied behavioral analysis is finding its heart. Journal of Applied Behavioral Analysis, 11, 203–214.

Copyright of Journal of Counseling & Development is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.