Week 9 Assignment

Volume 37/Number 4/October 2015/Pages 364-376-xxx/doi: 10 .17744/mehc.37.4.07

RESEARCH

Practice-Based Research: Meeting the Demands o f Program Evaluation through the Single-Case Design

Tiffany Nielson

The counseling profession is moving toward a foundation in outcome research to enhance the validity and efficacy o f clinical mental health counselors. A disparity between the roles o f researcher and counselor has resulted in counselors relying on other professions to validate their work. To remedy this gap, practice-based research using the single-case design is promoted as a suitable method for counselors. Current research is explored to highlight the value o f the single-case design in clinical practice and program evaluation. How the Accountability Bridge Counseling Program Evaluation Model (Astramovich & Coker, 2007) can be applied to sin­ gle-case design is described. The use o f feedback-informed treatment systems is shown to be a feasible measurement system for this method and for counseling practice. Suggestions are made for future research.

As mental health services move to primary funding from third-party payers, evidence-based practice is becoming essential to counselor survival (Granello & Witmer, 1998; Sexton, 2000). Third-party payers are expecting answers to validate counselor effectiveness (Granello & Granello, 1998), denoting the need for outcome-based research. The American Counseling Association (ACA) ethical code reserves an entire section for ethical research practices for the practicing counselor (ACA, 2014). Ethically, counselors are obligated to stay current with research to inform their counseling work (ACA, 2014; American Mental Health Counselors Association [AMHCA], 2010). Research is identified as one method for counselors to meet their ethical obli­ gation and display their commitment to the counseling profession (AMHCA, 2010). Conducting research as part of clinical practice will require the coun­ selor to have knowledge, skills, and practice in feasible research methods.

Promoting research in counseling and counselor training has been discussed and supported for over 20 years. The discussion has ranged from arguments for or against the use of various methodologies (e.g., Duffy &

Tiffany Nielson is affilia ted w ith the University o f Illinois a t Springfield. Correspondence about this article

should be addressed to Tiffany Nielson, Departm ent o f Hum an Development Counseling, Brookens Library 332, One University Plaza, University o f Illinois a t Springfield, Springfield, Illinois 62703-5407.

Email: tn ie l3@ uis.edu.

364 0 Journal of Mental Health Counseling

Practice-Based Research

Chenail, 2008; Kline, 2003; Lundervold & Belwood, 2000; Ray, Barrio Minton, Schottelkorb, & Brown, 2010; Sharpley, 2007; Southern & Devlin, 2010) to ways to bridge research and practice (Astramovich & Coker, 2007; Murray, 2009; Southern & Devlin, 2010) and models to enhance counselor training (Granello & Granello, 1998; Heppner et al., 1999; Huber & Savage, 2009; O’Brien, 1995; Okech, Astramovich, Johnson, Hoskins, & Rubel, 2006; Sexton, 2000). This article describes research developments within counsel­ ing and application of the current literature on single-case design (SCD) to research by the practicing counselor. The Ray et al. (2010) model for using SCD in child and adolescent research is adapted to clinical practice. The Accountability Bridge Counseling Program Evaluation Model (Astramovich & Coker, 2007) serves as a larger framework for research to display effectiveness. A description and examples of how SCD can be used are given, with specific applications of feedback-informed treatment (FIT) systems to promote clinical research.

OUTCOME RESEARCH CHASM

Murray (2009) described the “research chasm,” in which counselors do counseling and counselor educators conduct research as “separate, unrelated entities” (p.109), and proposed a theory-based method for counselors to infuse research findings into their practice. She concluded that the chasm between researcher and counselor would remain and urged greater focus on integrat­ ing research findings into counselor practice. Since the current approach to counselor training in research is focused on consuming the literature rather than producing research (Council for Accreditation of Counseling and Related Educational Programs, 2009), the chasm is not likely to disappear. While bridg­ ing current research and counseling is important, bridging the gap between researcher and counselor is equally important to clinically based research. The paradigm of practice-based research may remedy this gap and help practi­ tioners to assume roles as researchers (Henton, 2012).

Sexton (2000) argued that counselors are unsure of the role research plays in professional practice. This has led to counselors relying on other mental health professions for clinical research (Kaplan, 2010; Kaplan & Gladding, 2011; Kleist, 2003). Kaplan (2010) described the problem systemically: coun­ selor educators who are improperly trained, or who lack the desire to conduct research, train future counselor educators to conduct what he considers easy research to perpetuate a “profession of opinions rather than outcomes” (p. 26). In calling attention to the lack of research on client outcomes in hopes of pro­ moting discourse, Kaplan (2010) made the radical suggestion that counselors and counselor educators accept their dislike of research and rely exclusively on the research of psychology, social work, and other mental health professions.

While agreeing on the lack of outcome research in counseling, Guiffrida and Douthit (2010) responded to Kaplan’s proposal by instead urging that the counseling profession move to improve its outcome research. They proposed changes to make a place for research, such as using research methodologies

365

that fit the counselor paradigm, promoting professional presentations centered on research, and improving doctoral training. While Kaplan (2010) had argued that counselor personalities are not suitable for research, Guiffrida and Douthit (2010) noted the importance of producing literature within the wellness and developmental approach rather than relying on the medical model found in other mental health professions. However, both arguments center on coun­ selor educators and doctoral students rather than practicing counselors. Thus it appears that, although there is consensus on the need for outcome-based research in counseling, there is confusion about how and by whom the research should be conducted.

Mental health counselors approaching their duty to engage in research will encounter a variety of methodologies that may not resonate with their current counseling practice. Outcome research in counseling is comprised of methodologies that reveal how various factors affect client progress (Heppner, Wampold, & Kivlighan, 2008), such as models, counselor char­ acteristics, or specific interventions (Sheperis, 2010). Heppner et al. (2008) described the benefits and challenges of using a true experimental design as a method for outcome research in counseling. Henton (2012) emphasized the disparity within the field of counseling psychology between experimental designs and evidence-based practice, in contrast with practice-based evidence. Experimental designs are described as incongruent with typical counseling practice. Increased internal validity measures in experimental designs, such as theoretical purity and strict adherence to manualized treatment, decrease the generalizability of the results to typical counseling practice. In contrast, prac­ tice-based research takes place in the environment where counseling occurs, which creates a closer relationship between clinical practice and research (Henton, 2012). Henton described three forms of practice-based research: case studies, process research, and effectiveness research. The literature on the use of SCD has demonstrated the stance of this research design within the prac­ tice-based research paradigm (Foster, Watson, Meeks, & Young, 2002; Galassi & Gersh, 1993; Lundervold & Belwood, 2000; McDougall & Smith, 2006; Ray et al., 2010; Sharpley, 2007). Further descriptions of SCD for use in prac­ tice-based studies are discussed below as a means for counselors to do research.

Research Publications Current trends in ACA journals highlight the separation between cli­

nicians and academia: few clinicians are authors (Crockett, Byrd, Erford, & Hays, 2010; Erford, Miller, Duncan, & Erford, 2010) and there is little out­ come-based research (Kaplan, 2010; Sexton, 1996). Academics rather than practitioners continue to conduct research — as has been noted in a number of ACA publications: Erford et al. (2010) found few practitioner authors in Measurement and Evaluation in Counseling and Development and Crockett et al. (2010) similarly found a lack of practitioner authors in Counselor Education and Supervision and recommended collaboration between counselor educa­ tors and practicing counselors—although the authors did note a trend toward

366

Practice-Based Research

increased collaboration as evidenced by the increased number of authors for each article. While more coauthors does suggest greater collaboration, it is not clear whether the coauthors are educators, doctoral students, or practitioners.

Sexton (1996) explored outcome-based research reported in over 100 professional journals. While he found a number of valuable studies, he noted the practical difficulty of finding such reports, since outcome-based research is scattered across publications, with only a few articles in each publication. Similarly, Kaplan (2010) found only three outcome research articles within a two-year period in the Journal of Counseling and Development. The lack of out­ come research is evident throughout the ACA family of journals. Knowledge of appropriate practice-based methods to integrate into the clinical practice of mental health counselors is necessary to meet the need for evaluation in coun­ seling. This article describes how a program evaluation model integrated with use of SCD and FIT systems can be a manageable system for the practicing counselor.

BUILDING BRIDGES: FINDING THE RIGHT RESEARCH METHODOLOGY

As awareness of the lack of research into counseling outcomes has grown, an increasing number of models and methods have been proposed. Program evaluation is one way to systematically monitor the effectiveness of a program or of counseling services; the results can then be relayed to stakeholders (Astrainovich & Coker, 2007). Astramovich and Coker (2007) noted both the lack of training in program evaluation and the lack of feasible program evalu­ ation methods. To meet third-party payer demands, they emphasized the duty of the mental health counselor to contribute to program evaluation research. Their hope is that this duty to evaluate effectiveness comes from the desire to meet client needs and advocate for the profession.

To help clarify how to evaluate programs, Astramovich and Coker (2007) formulated the Accountability Bridge Counseling Program Evaluation Model, which operates as a step-by-step guide to planning, implementation, and assessment. They also underscored the importance of informing clients and third-party payers of outcomes to further integrate their feedback, planning, and goals. This is a continuous cycle in which implementation and feedback inform the sendees provided to meet the specified objectives. A description of how to use SCDs in practice shows how this method can meet the assessment needs of program evaluation.

Single-Case Design SCDs show promise as a research method that fits within the “construc­

tivist, diversity-focused epistemology” of counseling (Guiffrida & Douthit, 2010, p. 23). Many experimental designs, such as randomized controlled clinical trial or group experimental designs, that only answer specific research questions (Sharpley, 2007) may not be feasible for counselors (Guiffrida and Douthit, 2010; Lundervold & Belwood, 2000) or representative of counseling

367

practice (Lundervokl & Belwood, 2000); SCD is therefore considered to be a methodology that can contribute to outcome research while still grounded in practice (Foster et al., 2002; Galassi & Gersh, 1993; Lundervold & Belwood, 2000; McDougall & Smith, 2006; Ray et ah, 2010; Sharpley, 2007). Described as practice-based evidence or N = 1 design (Lundervold & Belwood, 2000; Sharpley, 2007), SCD consists of a single system (an individual or one group) that receives a treatment or intervention that is then subject to further anal­ ysis, in this case graphic, which can contribute to evidence and causation (Lundervold & Belwood, 2000). Ray et al. (2010) proposed the use of SCD to promote research in child counseling, describing it as an efficacious experi­ mental method that is a reasonable option for the practicing counselor.

The two primary uses of SCD are to promote the effectiveness of the individual counselor’s work and to illustrate the effectiveness of a program or services by documenting change in client behavior (Foster et ah, 2002; Lundervold & Belwood, 2000; Ray et ah, 2010). This aligns with the goals of program evaluation (Astramovich & Coker, 2007), creating a natural union of the two approaches. Lundervokl and Belwood (2000) described SCD as the “best kept secret in counseling” because it is not only a viable option for sci­ entific evaluation, it is also directly applicable to and grounded in counseling practice (p. 78). Those authors identify seven components for teaching this methodology to promote research training for master’s and doctoral students. Sharpley (2007) noted that many programs may already include these, and the absence of SCD reports in the literature may be due to a lack of understanding of procedure and analysis. He encouraged clarity in methodology and proce­ dures in addition to analysis to enable counselors to contribute what they have learned in working with individual clients to the literature.

Two forms of SCD are described in the literature; single-case experimen­ tal design and single-case evaluation; according to Ray et ah (2010) single-case experimental design can often be distinguished from single-case evaluation (AB design) by its intensity, experimental controls, and use of more than one baseline. Each form of SCD has a place in counseling practice research, though single-case evaluation is more practical for the counselor (Ray et ah, 2010). In this article, the term SCD encompasses the overarching concep­ tual framework of this methodology, although there are variations in how the method is used. Many counselors already gather data on client progress that can be easily adapted to an SCD. Ensuring methodological clarity from the outset of the research and in program evaluation is necessary to make this adaptation. A description and example of the use of the SCD within a program evaluation model highlights steps needed to complete the research.

APPLICATIONS IN CLINICAL PRACTICE

How mental health counselors can evaluate programs using SCD, whether experimental or evaluative, is outlined here. In acknowledging the rigor that is required to ensure that SCDs are of appropriate quality, Ray et al. (2010) out­ lined the use of single-case experimental design for work with children and for

368

Practice-Rased Research

applications to school settings. They spelled out steps in researching children to identify participants, instruments and points of measurement, phase proto­ col, ethical and design considerations, and data analysis. Components of these steps are described within the framework of the Accountability' Bridge model (Table 1). The four steps of the counseling program evaluation cycle from that model are integrated with the Ray et al. description of SCD (2010). Program evaluation methods are feasible when an SCD method is used and fit naturally with mental health counseling practice (Table 1). Including FIT systems offers a unique way for counselors to assess and monitor outcomes.

Tab le I . T h e A c c o u n ta b ility B rid g e M o d e l w ith S in g le -C ase Design

Accountability Bridge

Program Evaluation Cycle* Single-Case Design Implementation

1. Program Planning Participant Selection**

Instruments and Points o f Measurement**

Establishing Baseline

2. Program Implementation

3. Program Monitoring and Refinement FIT Systems

4. Outcomes Assessment Data Analysis**

•Astramovich & Coker, 2007. **Ray et at, 2010.

1. Program Planning The first step in program evaluation as outlined in the Accountability

Bridge Model (Astrainovich & Coker, 2007) is program planning. Here the objectives and needs of the clients guide the creation of a program or service, based on three aspects of creating an SCD:

Participant criteria. Identifying criteria for participants in program eval­ uation (Astramovich & Coker, 2007) and SCD research (Ray et ah, 2010) is foundational. The practicing counselor may choose to monitor a specific client or clients based upon presenting concern and demographic or other specific and identifiable criteria. In meeting the experimental demands of SCD, thor­ ough description of client criteria is encouraged (Galassi & Gersh, 1993; Ray et al., 2010). It may be beneficial to evaluate up to five clients to account for drop­ outs (Ray et al., 2010). When selecting participants who are currently clients, informed consent should clarify what the boundaries are and that participation is voluntary (ACA, 2014; AMHCA, 2010).

Instruments and points of measurement. Here the question of what is being measured is more clearly defined, and a specific measure for this is used (Ray et al., 2010). The practitioner-researcher will identify the independent and dependent variables. For SCD in program evaluation, the independent variable will be the intervention being offered; for example, an independent variable might be individual counseling using mindfulness-based techniques. Experimental forms of SCD encourage a specified manualized treatment protocol (Ray et al., 2010). Because this may not always be representative of

369

counseling practice (Henton, 2012; Ray et al., 2010) or meet the needs of the practitioner-researcher, as much description of the treatment as possible is encouraged. Identifying or describing the counseling procedures used is consistent with program evaluation methods and demands (Astramovich & Coker, 2007).

The dependent variable may be closely connected to the goals of coun­ seling or the needs of the client. For the mental health counselor this may be a variable such as anxiety symptoms or self-harming behaviors. This decision may also be connected to the assessment measures available. Ray et al. (2010) pointed out that assessment also needs to allow for multiple administrations without compromising the validity of the measure. FIT systems are promoted as a measurement system that may meet the needs of program evaluation and SCD (Miller, Duncan, Sorrell, & Brown, 2005). They are designed for repeated administration, often give access to measures for analysis, and have been found to be reliable and valid (Lambert, Hansen, & Harmon, 2010; Locke et al., 2011; Miller et al., 2005). The use of the FIT system can clarify when, how often, and by whom an assessment measure is administered within the program planning phase of program evaluation (Astramovich & Coker, 2007). Further discussion of the use of FIT systems is explained in step 3.

Establishing the baseline. Understanding variations in how SCD may be used is helpful in setting up the rest of the program evaluation procedures. Within SCD, A denotes a baseline or no-intervention phase and B or suc­ ceeding letters denote the intervention provided (Foster et al., 2002; Galassi & Gersh, 1993; Plavnick & Ferreri, 2013; Ray et al., 2010). There are numerous variations, but common designs are ABA, ABAB, which is also referred to as a reversal design (Plavnick & Ferreri, 2013), and ABC (Ray et al., 2010). Other variations are available (see, e.g., Foster et al., 2002; Plavnick & Ferreri, 2013). In a FIT system for SCD, data are gathered continuously. Ray et al. (2010) noted that it is possible to adapt the length and type of design. For example, if a client continues to improve through a service, it could continue to be offered until outcomes plateau. On the other hand, if one client is not progressing and a different program is trending as effective for other clients, it may be beneficial—and ethical —to transition the first client to other services or programs. After the treatment or intervention, it is common in experimental approaches to gather a post-intervention baseline (Foster et al., 2002; Ray et al., 2010). To reinforce a finding that change was due to the intervention, a second intervention phase is encouraged (Galassi & Gersh, 1993). With some client populations it may not be ethical or feasible to end services or fluctuate in this manner, particularly with pervasive or severe diagnoses. For those cases, an AB design may meet the need to examine change in the client (Foster et al., 2002). Foster et al. (2002) stated that causation cannot be determined in an AB design, and replication of intervention and baseline phases creates a stronger method to pinpoint change, but they noted that the AB design may be necessary when it is unhelpful or unethical to end treatment (Foster et al., 2002).

The goal of program evaluation and SCD is to illustrate how the specified intervention brings about a desired outcome. Establishing a baseline allows

370

Practice-Based Research

the analysis to demonstrate that change in a particular client may be primarily attributed to the intervention and is not due to other factors (Ray et ah, 2010). It further shows the effectiveness of the counselor’s work. Galassi and Gersh (1993) asserted the scientific nature of the SCD and encouraged detail in participant descriptions and interventions to support replication. They also encouraged replication with other clients and noted the strength added when a second baseline and intervention phase is introduced. As previously discussed, even the AB design has nrerit for displaying change for a specific client (Foster et ah, 2002). This method can help counselors to advocate for client participa­ tion in treatment and for the services provided.

After the variables are identified, the assessment measure is given before the intervention. The counselor may obtain baseline measures by administer­ ing an assessment when the client first seeks services, such as at intake, or before the first session. The counselor can make adaptations to gather a baseline based on client needs. It is useful to gather data for the baseline until a consistent pattern is found, which may require three to five measurement points (Ray et al., 2010). While this may not always be feasible due to client needs, (e.g., self- harm), to track change at least one measure is needed before an intervention. 1 he practitioner-researcher who committs to rigorous single-case experimental design needs multiple baseline measures (Galassi & Gersh, 1993; Foster et al., 2002; Plavnick & Ferreri, 2013; Ray et al., 2010). Seeking the greatest rigor possible is encouraged; when more baseline measurement points are gathered, a pattern of change or outcome during the intervention may become clearer. Once the participants, goals, measurements, and baseline are established, the counselor can move forward with the program.

2. Conducting the Program In this stage the services decided on in step 1 are provided; Astranrovich &

Coker (2007) noted that there are opportunities within this stage for adaptation and change to meet client needs or improve services. Throughout, data are gathered, preferably weekly, to monitor change and support analysis connected to the SCD.

3. Program Monitoring and Refinement It is advisable to monitor progress as services are provided and make any

necessary changes to meet the needs of the clients (Astranrovich & Coker, 2007). At this point, a client participant who may be better served through a different treatment is encouraged to seek consultation (ACA, 2014; AMHCA, 2010). Ethically, counselors are obligated to do what would be most beneficial and least harmful, even if that may compromise the client’s participation in research (ACA, 2014, AMHCA, 2010). The use of FIT systems to monitor cli­ ent progress is encouraged to meet the needs of both program evaluation and the SCD methodology. FIT systems are here described more fully in terms of models that may be used for program evaluation and SCDs:

371

Feedback-informed treatment (FIT) systems. In a meta-analysis, Lambert et al. (2003) described ways to promote client outcomes; their results showed that monitoring client progress reduced deterioration. They concluded that it may be time for more clinicians to adopt feedback-informed practices. FIT systems naturally lend themselves to SCD research (Miller et ah, 2005). They may not only prevent client deterioration (Lambert et ah, 2003) but also serve as a means to demonstrate counselor effectiveness (Miller et ah, 2005).

FIT systems are ready-made measures that allow for continual assessment of client progress. This allows the clinician to monitor client deterioration in the hope of preventing drop-out (Lambert et ah, 2003; Miller et ah, 2005). The system is brought into the counseling room and shared openly with the client to create a dialogue about concerns and progress. Information gathered from SCD is a natural application for these measures in addition to their ease of use in the clinical setting (Miller et ah, 2005).

A number of FIT systems are readily available. They often have assess­ ments and forms of measurement for monitoring client progress that are easy to use; the Partners for Change Outcome Management System (PCOMS) is an example (Miller et ah, 2005; PCOMS International, 2014). The PCOMS has an Outcome Rating Scale (ORS) and a Session Rating Scale (SRS). For the ORS, clients rate their own status on four domains: individual, interpersonal, social, and overall well-being (Miller et ah, 2005); adult, youth, young child, and group versions are available. The SRS assesses the therapeutic alliance and satisfaction with the counseling session. Each assessment takes clients only one minute to complete, can be completed at each session, and has reasonable reliability and validity (Miller et ah, 2005); it is listed in the National Registry of Evidence-Based Programs and Practices (PCOMS International, 2014).

These resources can be helpful for the practicing counselor who must deal with time constraints and has limited knowledge or skill in statistical analysis. Among other FIT systems are the Counseling Center Assessment of Psychological Symptoms (Locke et ah, 2011) and the OQ Measures system (Lambert et ah, 2010). The flexibility of the PCOMS system may be particu­ larly valuable for mental health counselors seeking feasible outcome measures. An example of the use of this system for analysis will be shown later because it seems particularly adaptable to mental health work with groups, individuals, adults, and children.

4. Outcomes Assessment The outcomes assessment phase of program evaluation measures client

progress at the conclusion of counseling (Astramovich & Coker, 2007). When SCD is used with a FIT system, this will be a fluid and continuing process. Once services are concluded, a second baseline can be established. The form of SCD used affects the degree to which the results can demonstrate that the outcome is due to the services provided (Foster et al., 2002; Galassi & Gersh, 1993). Once all data are gathered, they are analyzed for outcomes based upon previously set objectives (Astramovich & Coker, 2007).

372

Practice-Based Research

Data Analysis. Data analysis using SCD is a unique combination of quantitative information displayed in descriptive graphic form (Plavnick & Ferreri, 2013; Ray et al., 2010). A line graph is created that delineates baseline and intervention phases to visually display change (Lundervold & Belwood, 2000; Plavnick & Ferreri, 2013; Ray et al., 2010); Figure 1 is an example of how graphic analysis shows results for an ABA design. In this example, the ORS was used as the dependent measure. The scores are displayed on the ordinate with a scale of 0 to 40; higher scores represent greater well-being (Miller et al., 2005). The weeks are displayed on the abscissa, and baseline and intervention phases are delineated to mark when change occurred and how the services or program improved functioning. For this client, the counseling was mindfulness-based treatment. The graphic display clearly shows improvement in the ORS score.

F igure I . H yp o th e tica l d a ta d isp layed in a g raph ic analysis. Th e ORS score is m o n ito re d ove r a pe riod

o f 14 w eeks, w ith 3 w eeks fo r baseline, 8 w eeks o f in te rv e n t io n , and 3 w eeks fo r baseline

p o s t- in te rve n tio n .

It may also happen that the baseline does not return to its original level after intervention but rather a deterioration or decline is identified (Galassi & Gersh, 1993). In this example, the second baseline decreases minimally after intervention. I Iris may be an optimal goal, in that the client was able to maintain functioning yet may function better with counseling support. To strengthen this example, a second intervention phase could be provided, with the hope of showing a higher score in the intervention phase.

The simplicity of analyzing SCD data by creating a line graph makes this research method feasible for practitioners. Adding the intentional use and analysis of this method within program evaluation guidelines gives guidance on how the practicing counselor can demonstrate program effectiveness. The compatibility of SCD, FIT systems, and program evaluation guidelines thus offers counselors a manageable method for evaluating the effectiveness of an intervention.

373

Completing the Bridge To complete the program evaluation cycle as established by Astramovich

and Coker (2007), the mental health counselor will relay the results to stake­ holders. For the practitioner-researcher these may be third-party payers, such as managed care systems, administrators, or others involved in supporting services. Publication, presentations, or other means of sharing scholarship are encouraged to further advocate for counseling programs, client needs, and the counseling profession. In finalizing the Accountability Bridge Counseling Program Evaluation (Astramovich & Coker, 2007), adjustments and changes may be made in addition to assessing the needs of the clients before again cycling through the program evaluation steps.

CONCLUSION

Conducting practice-based research using SCD methods will enhance program evaluation and clinical research in counseling. Models are available to help the practicing clinician to succeed with this, such as program evalua­ tion models and readily available assessment measures. Counselors are encour­ aged not only to follow the guidelines in this article but also to seek out further information to support their program evaluation and research endeavors.

Further research might explore how mental health counselors can use FIT systems in implementing SCDs. A greater understanding of the challenges and concerns involved in evaluating programs may help to focus continuing education and other means to support the work of counselors. While research in the counseling profession has inherent value, how best to promote research production pedagogically is not clear because there has been little research on counselor training in research (Lambie & Vaccaro, 2011; Michalak, 2013). Continuing education may clarify the suggestions to focus on SCD, FIT sys­ tems, and program evaluation. Other models may evolve to improve the ability of future and current practicing counselors to effectively and ethically conduct practice-based research. As counselors integrate feasible research and program evaluation methods into their practice, it is anticipated that the mental health counselling profession may reinforce its identity and stance.

REFERENCES American Counseling Association (2014). ACA code o f ethics. Alexandria, VA: Author. American Mental Health Counselors Association. (2010). AMHCA code o f ethics. Alexandria, VA:

Author. Astramovich, R. L., & Coker, J. K. (2007). Program evaluation: The accountability bridge

model for counselors. Journal o f Counseling and Development, 85, 162-172. http://dx.doi. org/10.1002/j. 15 56-6678.2007.tb00459.x

Council for Accreditation of Counseling and Related Educational Programs. (2009). 2009 standards. Retrieved from: http://www.cacrep.org/2009standards.html

Crockett, S. A., Byrd, R., Erford, B. T., & Hays, D. G. (2010). Counselor Education and Supervision golden anniversary publication pattern review: Author and article characteristics from 1985 research training 2009. Counselor Education and Supervision, 50, 5—20. http://dx.doi. org/10.1002/j. 1556-6978.2010.tb00105.x

374

Practice-Eased Research

Duff)', M., & Chenail, R. J. (2008). Values in qualitative and quantitative research. Counseling and Values, 53, 22-38. http://dx.doi.Org/10.1002/j.2161-007X.2009.tb00111.x

Erford, B. 1., Miller, E. M., Duncan, K., & Erford, B. M. (2010). Submission patterns: Measurement and Evaluation in Counseling and Development author and article characteristics from 1990 to 2009. Measurement and Evaluation in Counseling and Development, 42, 296-307. http://dx doi org/10.1002/j. 15 56-6978.2010.tbOO 105.x

Foster, L. H., Watson, T., Meeks, C., & Young, J. (2002). Single-subject research design for school counselors: Becoming an applied researcher. Professional School Counseling, 6, 146-154.

Galassi, J. P., & Gersh, T. L. (1993). Myths, misconceptions, and missed opportunity: Single-case designs and counseling psychology, journal o f Counseling Psychology, 40, 52 5-5 31. http://dx.doi org/10.1037//0022-0167.40.4.525

Granello, P. F., & Granello, D. (1998). Training counseling students to use outcome research. Counselor Education and Supervision, 37, 224-237. http://dx.doi.Org/10.1002/j.1556-6978.1998. tb00547.x

Granello, P. F., & Witmer, J.M. (1998). Standards of care: Potential implications for the counseling profession, journal o f Counseling and Development, 76, 371-380. http://dx.doi. org/10.1002/j. 15 56-6676.1998.tb0269 5 .x

Guiffrida, D., & Douthit, K. (2010). Perspectives on the state of counselor education research: A response to Kaplan’s “A radical thought on counseling research: Let’s stop doing it.” ACES Spectrum, 70, 22-24. Retrieved from http://aces.lfchosting.com/

Henton, 1. (2012). Practice-based research and counselling psychology: A critical review and proposal. Counselling Psychology Review, 27, 11-21.

Heppner, P., Rooney, S., Flores, L. Y., Tarrant, J. M., Howard, f. K., Mulholland, A. M.,...Lilly, R. L. (1999). Salient effects of practice poster sessions on counselor development: Implications for research training and professional identification. Counselor Education and Supervision, 38, 205-217. http://dx.doi.org/10.1002/j. 1556-6978.1999.tb00571 .x

Heppner, P. P., Wampold, B. E., & Kivlighan, D. M. (2008). Research design in counseling (3rd ed.). Belmont, CA: Thomson Brooks/Cole.

Huber, C. H., & Savage, T. A. (2009). Promoting research as a core value in master’s-level counselor education. Counselor Education and Supervision, 48, 167-178. http://dx.doi. org/10.1002/j. 15 56-6978.2009.tb00072.x

Kaplan, D. (2010). A radical thought on counseling research: Let’s stop doing it. ACES Spectrum, 7, 25-26. Retrieved from http://aces.lfchosting.com/

Kaplan, D. M., & Gladding, S.T. (2011). A vision for the future of counseling: The 20/20 Principles for Unifying and Strengthening the Profession, journal o f Counseling and Development, 89, 367— 372. http://dx.doi.Org/10.1002/j.l 556-6678.201 LtbOOlOl.x

Kleist, D. M. (2003). Research on couple/family counselor training: A search for research. The Family journal, 11, 380-382. http://dx.doi.org/l0.1177/1066480703255463

Kline, W. B. (2003). The evolving research tradition in Counselor Education and Supervision. Counselor Education and Supervision, 43, 82-85. http://dx.doi.org/10.1002/jT556-6978.2003 tb01832.x

Lambie, G. W., &Vaccaro, N. (2011). Doctoral counselor education students’ levels of research self- efficacy, perceptions of the research training environment, and interest in research. Counselor Education and Supervision, 50, 243-258. http://dx.doi.Org/10.1002/j.1556-6978.2011.tb00122.x

Lambert, M. J., Whipple, J. L„ Hawkins, E. ]., Vermeersch, D. A., Nielsen, S. L., & Smart, D. W. (2003). Is it time for clinicians to routinely track patient outcome? A meta-analysis. Clinical Psychology: Science and Practice, 10, 288-301. http://dx.doi.org/10.1093/clipsy.bpg025

Lambert, M. J., Hansen, N. B., & Harmon, S. C. (2010). The OQ-45 system: Development and practical applications in health care settings. In M. Barkham, G. Hardy, & j. Mellor-Clark (Eds.), Developing and delivering practice-based evidence: A guide for the psychological therapies (pp. 141-154). New York, NY: Wiley-Blackwell. http://dx.doi.org/10.1002/9780470687994.ch6

Locke, B. D., Buzolitz, J. S., Lei, P.W., Boswell, J. F., McAleavey, A. A., Sevig, T. D., Hayes, J. (2011). Development of the Counseling Center Assessment of Psychological Symptoms 62 (CCAPS-62). Journal o f Counseling Psychology, 58, 97-109. http://dx.doi.org/10.1037/a0021282

375

Lundervold, D. A., & Belwood, M. F. (2000). The best kept secret in counseling: Single-case (N = 1) experimental designs, journal of Counseling and Development, 78, 92-102. http://dx.doi. org/10.1002/j. 15 56-6676.2000. tb02565.x

McDougall, D., & Smith, D. (2006). Recent innovations in small-N designs for research and practice in professional school counseling. Professional School Counseling, 9, 392-400.

Michalak, M. B. (2013). The scholarly process: A grounded theory exploring how counselor educators promote scholarship of counselors-in-training (unpublished doctoral dissertation). Idaho State University, Pocatello, ID.

Miller, S. D., Duncan, B. L., Sorrell, R., & Brown, G. S. (2005). The partners for change outcome management system. Journal of Clinical Psychology, 61, 199—208. http://dx.doi.org/10.1002/ jclp.20111

Murray, C. E. (2009). Diffusion of innovation theory: A bridge for the research-practice gap in counseling. Journal of Counseling and Development, 87, 108-116. http://dx.doi. org/10.1002/j. 15 56-6678.2009.tb005 56.x

O’Brien, K. M. (1995). Enhancing research training for counseling students: Interuniversity collaborative research teams. Counselor Education and Supervision, 34, 187-198. http://dx.doi. org/10.1002/j. 15 56-6978.1995 ,tb00241 .x

Okech, J., Astramovich, R., Johnson, M., Hoskins, W., & Rubel, D. (2006). Doctoral research training of counselor education faculty. Counselor Education and Supervision, 46, 131-145. http://dx.d0i.0rg/l 0.1002/j.l 5 56-6978.2006.tb00018.x

PCOMS International. (2014). What is PCOMS?. Retrieved December 16, 2014, from http://www. whatispcoms.com

Plavnick, J. B., & Ferreri, S. J. (2013). Single-case experimental designs in educational research: A methodology for causal analyses in teaching and learning. Educational Psychology Review, 25, 549-569. http://dx.doi.org/10.1007/sl0648-013-9230-6

Ray, D. C., Barrio Minton, C. A., Schottelkorb, A. A., & Brown, A. G. (2010). Single-case design in child counseling research: Implications for counselor education. Counselor Education and Supervision, 49, 193-208. http://dx.doi.Org/10.1002/j.1556-6978.2010.tb00098.x

Sexton, T. L. (1996). The relevance of counseling outcome research: Current trends and practical implications. Journal of Counseling and Development, 74, 590-600. http://dx.doi. org/10.1002/j.l 5 56-6676.1996.tb02298.x

Sexton, T. L. (2000). Reconstructing clinical training: In pursuit of evidence-based clinical training. Counselor Education and Supervision, 39, 218—227. http://dx.doi.Org/10.1002/j.1556-6978.2000. tbO 1234.x

Sharpley, C. F. (2007). So why aren’t counselors reporting N = 1 research designs? Journal of Counseling and Development, 85, 349—356. http://dx.doi.Org/10.1002/j.1556-6678.2007. tb00483.x

Sheperis, C. J. (2010). Contemporary issues in counseling research. In C. J. Sheperis, J. S. Young, & M. H. Daniels (Eds.), Counseling research: Quantitative, qualitative, and mixed methods (pp. 1-5). Upper Saddle River, NJ: Pearson Education.

Southern, S., & Devlin, J. (2010). Theory development: A bridge between practice and research. The Family Journal, 18, 84-87. http://dx.doi.org/10.1177/1066480709358422

376

Copyright of Journal of Mental Health Counseling is the property of American Mental Health Counselors Association and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.