SOCW wk 10 Discussion: Assessing Outcomes

Full Terms & Conditions of access and use can be found at https://www.tandfonline.com/action/journalInformation?journalCode=wasw21

Administration in Social Work

ISSN: 0364-3107 (Print) 1544-4376 (Online) Journal homepage: https://www.tandfonline.com/loi/wasw20

Outcomes-Driven Clinical Management and Supervisory Practices with Youth with Severe Emotional Disturbance

David King & Kay Hodges

To cite this article: David King & Kay Hodges (2013) Outcomes-Driven Clinical Management and Supervisory Practices with Youth with Severe Emotional Disturbance, Administration in Social Work, 37:3, 312-324, DOI: 10.1080/03643107.2012.691080

To link to this article: https://doi.org/10.1080/03643107.2012.691080

Published online: 17 May 2013.

Submit your article to this journal

Article views: 1357

View related articles

Citing articles: 1 View citing articles

Administration in Social Work, 37:312–324, 2013 Copyright © Taylor & Francis Group, LLC ISSN: 0364-3107 print/1544-4376 online DOI: 10.1080/03643107.2012.691080

Outcomes-Driven Clinical Management and Supervisory Practices with Youth with Severe Emotional Disturbance

David King School of Graduate and Professional Studies, Spring Arbor University, Spring Arbor,

Michigan, USA; LifeWays Community Mental Health, Jackson, Michigan, USA

Kay Hodges Institute for the Study of Children, Families, and Communities, Eastern Michigan

University, Ypsilanti, Michigan, USA

Data-informed and outcomes-oriented administrative practices are critical for achieving positive con- sumer results. This article describes a model for program managers and clinical supervisors that utilized a proactive, strengths-based approach, which was supported by a “real-time” outcome management system. Key factors critical to implementing and sustaining data-driven practices are presented.

Keywords: administration, CAFAS, children and adolescents, outcomes, outcomes management, social work, technology

INTRODUCTION

Administrators and managers in mental health or social service agencies are expected to measure outcomes for each client to determine the effectiveness of services and to inform stakeholders of their results. A variety of sources require this information, including federal and other governmental entities, various funders for nonprofit agencies, third-party payers, and accreditation bodies (Cohen, 2003; Martin & Kettner, 1997). Different types of service organizations require different outcome indicators (e.g., psychosocial functioning, number of episodes of out-of-home care). A step beyond outcome measurement is outcome management, which refers to aggregating, analyzing, and inter- preting outcome data in order to actively improve practices and outcomes (Lampkin & Hatry, 2003). Use of outcome data at this level can lead to more effective and efficient ways of serving clients, which is the goal of continuous quality improvement (CQI).

In fact, leaders in clinical social work have asserted that outcome measurement and manage- ment are now normal and expected parts of doing business in mental health care (Cohen, 2003;

The authors acknowledge the resources made available to the providers in Michigan through the Department of Community Health.

Correspondence should be addressed to David King, 1200 North West Avenue, Jackson, MI 49202, USA. E-mail: [email protected]

OUTCOMES-DRIVEN PRACTICES 313

Rosenberg, 2006). The current ethical code of the National Association of Social Workers states that social workers should monitor and evaluate policies, the implementation of programs, and practice interventions (National Association of Social Workers, 2008). Leaders in the field have called for making client outcomes and service effectiveness the most central agency foci (Mullen, 2004; Rapp & Poertner, 1988) and for using assessments for the benefit of the consumer (Cowger, 1994; Gutherie, McIntosh, Callaly, Trauer, & Coombs, 2008; Hodges, 2004b).

Building an outcomes management perspective and an effective outcome-driven service deliv- ery model within an agency is challenging, with many potential barriers. Such barriers include clinicians’ negative attitudes toward standardized assessment tools (Jensen-Doss & Hawley, 2010), administrators’ undervaluation of data-based program planning and management (Carrilio, Packard, & Clapp, 2004), and data-based technology that is difficult to navigate or otherwise unfriendly to users (English, Brandford, & Coghlan, 2000; Carrilio, 2005; Kapp & Stipp, 2010). Despite these challenges, some leaders have developed an outcomes management program in applied clinical settings (Neuman, 2003; Bliss, 2007; Hodges & Wotring, 2012).

Based on their review of the literature, Hodges and Wotring (2012) conclude that building the capacity to incorporate and sustain an outcomes management orientation depends on the presence of numerous processes within the organization. One such process is collection of consumer-level out- come data using a measure that is psychometrically sound (i.e., reliable, valid, sensitive to change). Other factors integral to an outcomes management orientation include technology that benefits all parties (i.e., consumer, practitioners, supervisors, and administrators), a learning organizational cul- ture, support by the organization’s top management, and an inclusionary approach that invites staff involvement (Poole, Davis, Reisman, & Nelson, 2001).

Despite considerable efforts by many organizations, it remains to be seen how the above princi- ples can be implemented to achieve successful consumer outcomes. Indeed, the ability to establish the right environment to sustain the effective use of data toward positive outcomes has also been identified as a significant concern in corporate settings (Spitzer, 2007, p. 4). In addition, human service organizations may spend considerable time, effort, and resources implementing evidence- based practice models in order to improve consumer outcomes only to find minimal returns for their investment compared to the total population served.

This article provides a qualitative analysis of a data-supported outcome management approach that utilizes clinical management and supervisory processes as mechanisms to introduce and main- tain an outcomes data orientation. The program and the key dimensions that were implemented over a three-year period are described. A qualitative approach is used here for analysis as this article represents a retrospective review of the program and the outcomes achieved. Empirical observa- tions of key components that appear to have led to successful outcomes are used to suggest general principles for use in other practice settings.

The approach described supports both the practitioners and the consumers while sustaining the organization’s vision for client success and rendering the agency more accountable, thus seeking to address the context as well as the content of data used. Research on the impact of supervision has shown that it has a positive effect on worker outcomes (Mor Barak, Travis, Pyun, & Xie, 2009), service effectiveness (Yoo & Brooks, 2005), use of client outcome data (Moore, Rapp, & Roberts, 2000), and attainment of client-specific goals (Poertner, 2006). In their meta-analysis of 27 research articles, Mor Barak and colleagues (2009) pinpointed three supervisor dimensions: task assistance, social and emotional supervisory support, and supervisory interpersonal interaction. The term task assistance refers to the supervisor’s case-specific, tangible advice and instruction to a supervisee. Of the three dimensions, task assistance was positively and statistically related to beneficial out- comes for workers, such as job satisfaction, effectiveness, and retention. By contrast, the other two dimensions were detrimental to outcomes for workers. Based on their findings with a large database, Yoo and Brooks (2005) concluded that supervisory management practices were statistically related

314 KING AND HODGES

to better child outcomes. This article discusses a process that includes task assistance supervision in addition to enhancing and expanding the application of this methodology throughout an entire department.

The population served in the program reviewed was youth with a serious emotional disturbance (SED), and the staff consisted primarily of social workers, including the director of the program (the first author, hereafter referred to as program director). The five components identified as criti- cal to achieving successful outcomes through the analysis of the program include: an organization dedicated to the mission of consumer success, an organizational leader who is passionate and driven to achieve positive results, a vision shared by leaders and the clinical team to achieve effective out- comes, data-informed supervision and risk management strategies within the context of supportive problem solving, and a framework for effective accountability.

AN ORGANIZATION OR PROGRAM DEDICATED TO CONSUMER SUCCESS

In 1988, Rapp and Poertner made the case that the consumer should take center stage and that the process of service provision should be preoccupied with clients and their well-being. Elsewhere, the importance of an organization’s mission and vision has been stressed as a vital component of outcomes measurement implementation (Fisher, 2005; Moore et al., 2000). Hodges and Wotring (2012) describe a dynamic model in which consumer outcomes are the central focus of all organiza- tional activities. These activities include a) general organizational operations, b) practice processes (i.e., interventions and data-informed case-specific supervision), c) data-informed management (i.e., use of aggregate for data for CQI endeavors, for monitoring performance goals, and for informing practices and procedures), and d) system support for maintaining evidence-based assessment and interventions. The model of Hodges and Wotring emphasizes that improving consumer outcomes requires embedding a meaningful and useful outcome measure at all levels of the organization and using aggregated data to inform policies and interventions via CQI efforts that are data-informed and are iteratively conducted at various levels (e.g., programs, departments, agency-wide, state-wide). With a similar vision in mind, the children’s services program in our review created a program design with the singular mission of being client-centered and making client success the central theme of program culture, service delivery, and program management. As a result, the program was continually directed toward service effectiveness, client outcomes, and results (Rapp & Poertner, 1988).

Although several evidence-based and promising practices were being implemented in the orga- nization at the time, the children’s leadership team believed that a broader clinical management framework was also needed to support these efforts to achieve successful outcomes for all youth served. The vision was to implement a model to guide and support all clinical practice, with the focus on identifying the need for evidence-based treatments, informing the use of these best prac- tices, and ensuring the progress of all youth being served. Regardless of which evidence-based practices were being implemented, the department goal was to improve functioning for all youth served and use outcome data to measure progress on this objective.

Furthermore, a sense of urgency in accomplishing this mission was consistently articulated and promoted as part of an outcomes-driven culture. This message of urgency grew out of concern that if services were not successful, the children served would only be at greater risk for harm to themselves or others or for negative life consequences later. The theme communicated was that the time to act and help was now and there was a moral and ethical imperative for the program and workers to respond.

An additional driver behind the desire for consumer results, as is common in service delivery environments today, was the increasing demand for services by children and families of even greater need. This demand for services, which coincided with the shrinking of the budget, further promoted

OUTCOMES-DRIVEN PRACTICES 315

a sense of urgency. The ability to achieve positive outcomes for children more quickly and effi- ciently would mean the ability to serve and make a difference in the lives of more children as a whole. Moving children and families through the system and into recovery became a priority. These elements of a clearly defined and urgent mission for achieving positive outcomes were translated as a need for the program and its staff to respond by taking full responsibility to make a difference. This theme permeated the vision and clinical dialogue of the program in delivery of services day to day.

ORGANIZATIONAL LEADERS WHO ARE DEDICATED TO ACHIEVING POSITIVE RESULTS

In their vision of the client-centered organization, Rapp and Poertner (1988) argued that the man- ager of the organization must be “myopic, single minded, and obsessed with clients” (p. 24) if this vision was to be achieved. Moore et al. (2000) further discuss the managerial skill needed to uti- lize aggregated data with staff to reinforce a shared vision, to stimulate creative problem solving, to improve performance, and to develop more personal “buy-in” into client outcomes. In fact, they conclude that this skill is not just an “add-on” (p. 495), but rather is “in some fundamental way a reformulation of management” (p. 495). This conclusion is consistent with Fisher’s (2005) analysis that transformational leadership at the team level is critical to making outcomes measurement an integral part of social work management.

In leading the charge for developing a client-centered program, the program director in our exam- ple displayed strong leadership, commitment, and passion of this nature. An example of a relentless leadership focus on consumer success was a specific initiative implemented to address the impor- tance of effective engagement of children and families in services. There were concerns that some staff frequently defined clients by labels such as “non-compliant” or worse and blamed children or families for lack of treatment success.

In response, the program director led a department discussion regarding the values needed for clinicians to effectively engage families in treatment. Beginning with a stated assumption that “we cannot help people if we cannot effectively engage them,” ideas were generated from clinicians to engage families from a strengths-based perspective that defined and clarified the meaning of this approach. These ideas were captured by the director and then distributed for input and editing by staff. As a result, a summary of engagement values, agreed to by all staff as part of their treatment approach, established that “failure is not an option” and that “we don’t give up on children and families.” These values further supported a culture of consumer success as primary. Examples of other engagement values developed include the following:

• We recognize that we can’t help children and families if we don’t effectively engage them and take full responsibility for making this our top priority.

• We will strive to be warm and welcoming to all children and families we serve, believing that children and families won’t care how much we know until they know we care.

• We will operate with a mental model that recognizes stages of change—taking children and families where they are at and designing interventions and approaches to match.

• We will be strength-based in our view and avoid judging, labeling, and identifying children and families as “non-compliant,” “resistant,” etc.

• We will assume that everyone has their reasons for being where they are, and we will seek to understand the journey of all children and families including past traumas and emotional injuries impacting their ability to move forward today.

• We will examine ourselves and our own reasons for resistance and frustration when they arise in engaging children and families.

316 KING AND HODGES

• We understand that it is vital to help engage children and families now. If we don’t, things are likely to get worse and they will only be back later. People are too important to let that happen.

A SHARED VISION BY SUPERVISORS AND THE CLINICAL TEAM

Fisher (2005) supports the idea that the formation of a team is essential to the development and implementation of an outcomes measurement framework, and that implementing an outcomes mea- surement system requires organizational change. Moore and colleagues (2000) also emphasize the importance of an organizational culture that supports learning and outcome achievement and in which everyone is focused on achieving a shared vision.

The first step taken in the development of the outcomes-driven management model used in the program reviewed was to involve clinical staff directly in the creation of a client-centered vision for the effective delivery of services. A shared vision for an effective delivery model would provide the foundation for organizational operations. Staff and supervisors were involved through department forums, surveys, and smaller team-based discussions to identify the core values that would drive the unit’s clinical approach. Clinicians were asked to describe values that they believed were most essential to achieving positive outcomes for children and families.

Once these core values were identified, they were summarized and categorized to capture core themes and concepts. In addition, staff and supervisors were involved in the creation of a department vision and mission statement that captured their broader desires as mental health clinicians working with children and families. The staff then reviewed again the values summaries, vision statements, and mission statements for final editing, input, and approval. The result of this collaboration was the creation of a department document entitled, “Child and Family Services Vision and Commitments” (See Table 1). Staff members were asked to express individually their alignment with and support for the commitments and vision as a way of doing business together in the department. Staff then agreed to post the vision and commitments summary at their desks and to review the ideas captured in it as part of clinical, team, and department conversations to keep the values at the heart of the department

TABLE 1 Children’s Services Vision, Mission, and Commitments

VISION To be recognized by our community and the state as a model of excellence in providing services to children with severe emotional disturbance and their families that get positive results.

MISSION To empower children and families through clinical excellence and a strength-based approach to develop positive skills, resources, and relationships that will help them achieve their goals and enrich their lives.

COMMITMENTS • Effective Collaboration: To partner effectively with families, co-workers, and the community in helping children and

families recover and to be honest, open, respectful, strengths-based and empathetic in our communication at all times. • Clinical Excellence: To provide excellent, effective, and timely services to children and families, and to be accountable

to families and each other for high-quality work that gets positive results. • Personal Integrity: To conduct ourselves with fairness and honor, seeking to do the right thing at all times, and to be

consumer- and solution-focused in all we do. • Genuine Care: To provide family-centered and family-driven care through our dedication to strengths-based treatment,

encouraging family voice and choice, understanding of each family’s unique cultural needs, and persistent pursuit of engagement that demonstrates our sincere care, concern, belief in, and support of the youth and families we serve.

OUTCOMES-DRIVEN PRACTICES 317

culture. As a result of this process, the outcome measures later identified were expressions of the department’s overall values and shared vision as described by Moore and colleagues (2000).

DATA-INFORMED SUPERVISION AND RISK MANAGEMENT STRATEGIES WITHIN A CONTEXT OF SUPPORTIVE PROBLEM SOLVING

Measure

In order to actively pursue the best outcome for each youth, it is essential to use a measure that is meaningful and useful to clinicians and families, that captures change over time and permits quantification for data-based decision making. The organization used the Child and Adolescent Functional Assessment Scale (CAFAS; Hodges, 2000, 2004a) as the data and outcome measurement source for the organization. There are over 80 articles substantiating this assessment tool’s reliabil- ity, validity, sensitivity to assessing change, and generalizability, which are summarized in Hodges (2004a, 2010). Numerous articles also describe how aggregated data on the CAFAS has been used to inform policy, practice, and training initiatives (Daleiden, Chorpita, Dondervoet, Arensdorf, & Brogan, 2006; Hodges & Wotring, 2004; Mueller, Tolman, Higa-McMillan, & Daleiden, 2010; Williams, 2009a, 2009b; Wotring, Hodges, Xue, & Forgatch, 2005).

The CAFAS is initially administered at intake for case planning, which includes determin- ing intensity and type of services. Thereafter, the CAFAS is administered periodically to monitor progress while in treatment (e.g., quarterly) and then at exit to examine service effectiveness. The youth’s day-to-day functioning is assessed in eight life domains: school/job, home, community (i.e., mostly delinquency), behavior toward others, moods/emotions (e.g., depression, trauma, anxiety), and self harmful behavior, substance use, and thinking (e.g., irrationality). Each domain consists of behavioral descriptions that are grouped into severe, moderate, mild, or minimal/no impairment, and assigned scores of 30, 20, 10, and 0, respectively.

Based on a routine clinical interview and collateral information, practitioners select descrip- tors that reflect the most severe problems observed during a chosen period, typically three months. In addition to these target problematic behaviors, each domain, or subscale, has a set of strengths and goals from which to select. The strengths and goals are used to set up a treatment plan and to generate a strengths-based report for the family. A total score is generated by summing all eight subscales, ranging from 0 to 240, with a higher score indicating greater impairment in day-to-day functioning. Therefore, as treatment progresses, lower CAFAS total and subscale scores indicate improvement.

The CAFAS measure is part of an outcome management web-hosted application referred to as FAS Outcomes (Functional Assessment Systems, 2009), which instantaneously generates assess- ment reports, sets up a treatment plan, and shows a summary of the youth’s results on his or her client dashboard. The displayed information includes total score, graphically presented domain subscale scores, and results for a variety of “clinical indicators” that assist with case decision making.

In addition, if a youth is re-assessed, the dashboard shows a comparison of the initial and most recent CAFAS results and provides the youth’s progress on several outcome indicators. These indi- cators include a) whether meaningful and reliable improvement was achieved, b) whether a youth who started services with one or more severe impairments across the eight domains no longer has severe impairments, and c) whether a youth who had pervasive behavioral impairment across set- tings (i.e., school, home, and community) at intake is no longer impaired in all three domains. These three outcome indicators were derived from research with aggregated databases (Xue, Hodges, & Wotring, 2004). The web-hosted system also includes features to permit managers and supervisors to view aggregated results for cases they oversee and to “drill down” to look at results for any of these youth. Aggregate reports and data export permit administrators and leaders to examine the needs and outcomes for youths served.

318 KING AND HODGES

Data-Informed Supervision Model

The web-based outcomes management system (FAS Outcomes, 2009) in which the CAFAS is embedded was an effective resource that proved to be an easily managed, real-time data collection system that supported this model and the active use of data by both staff and supervisors in the pro- gram reviewed. The data was an essential and effective resource for clinical monitoring, feedback, and accountability regarding outcomes. From this system, a variety of useful process and outcome reports were generated to support the data-driven management practices that were designed.

To ensure that the CAFAS was completed in a timely manner for all youth, the program director regularly ran a report to identify all cases with overdue assessments. Additionally, the director used the supervisor dashboard tools to review the data for youth with high CAFAS scores, youth who appeared to have high risk factors, and youth who were not making improvements. These data were downloaded into an Excel file, which was then used to track the clinical status and treatment progress for those youths whose outcomes appeared to be at-risk. Youth with high CAFAS scores as determined by the agency’s data profile were targeted as a priority for clinical consultation meetings and planning updates from supervisors and clinical staff. An example of this report, referred to as “Children’s Services Clinical Review and Follow-Up,” was sent to supervisors requesting follow-up (See Table 2).

To guide and inform clinical practice, specific procedures were developed to embed the use of the CAFAS in all clinical activities. In addition, processes were developed to maximize the use of CAFAS data in the daily clinical management and supervision of individual cases. See Figure 1, entitled “A Clinical Management Model for Consumer Success,” for an overview of the clini- cal management model used. As the model demonstrates, CAFAS data were used actively and consistently to identify consumer risks and progress. Data were also used to create a supportive, problem-solving focus in order to support staff and children continually toward treatment success.

The program director was responsible for reviewing the data weekly, assigning clinical support and follow-up tasks, and holding supervisors accountable through individual and group supervision and reporting. Supervisors were responsible for using case-specific data in individual and group supervision, providing clinical support and problem solving, and holding clinical staff accountable for follow-up on needed services and treatment recommendations. Clinicians were responsible for participating in regular individual and group supervision focused on improving consumer results and for following up on agreements reached during supervision.

Youth with the most severe CAFAS scores were required to have individual clinical consultation meetings arranged within specified time frames. These meetings included all clinical and support staff involved with the case, the psychiatrist (if available), the supervisor, and the program director. These consultation meetings ensured all needed services, supports, and strategies were being utilized to assist the youth and family in progressing toward their goals. Cases with high impairment, as indicated by CAFAS scores and clinical markers, in addition to cases that had made no progress during the last six months, were required to be addressed with an updated action plan. Moreover, such cases were identified for discussion at a biweekly clinical team consultation meeting during which a psychiatrist was present as part of the review. Recommendations made through the clinical consultation meeting were brought back to the family for discussion and approval before changing the youth’s plan. Together, all of these processes increased the ability of clinicians and supervisors to be intentional in responding to and planning for high-risk cases and increased accountability for achieving positive outcomes from the top down.

Using case-specific CAFAS data and tracking allowed supervisors to pinpoint areas of clin- ical need; to provide more effective support, follow-up, and problem solving for staff; and to hold staff accountable for the completion of tasks, including documentation and planning with clients. The CAFAS data further allowed for a more productive and streamlined supervisory pro- cess. Supervisors’ review with staff of the data on current child outcomes and progress improved

TA B

LE 2

C hi

ld re

n’ s

S er

vi ce

s C

lin ic

al R

ev ie

w an

d F

ol lo

w -U

p

C li

en t

na m

e P

ri m

ar y

cl ie

nt ID

A ge

To ta

l sc

or e

C A

FA S

ti er

Se ve

re im

pa ir

m en

t co

un t

R at

er A

ss es

sm en

t da

te A

dm in

is tr

at io

n

C ha

ng e

in to

ta l

sc or

e fr

om la

st C

A FA

S P

ro gr

am

St at

us an

d cl

in ic

al ac

ti on

pl an

fr om

su pe

rv is

or

12 15

0 Se

lf -h

ar m

/ se

ve re

m oo

d 4

SR 11

/ 18

/ 20

09 In

iti al

C A

FA S

In cr

ea se

d 60

W ra

pa ro

un d

7 12

0 Se

lf -h

ar m

/ se

ve re

m oo

d 2

A L

12 / 17

/ 20

09 12

m on

th C

A FA

S In

cr ea

se d

20 In

te ns

iv e

ca se

m an

ag em

en t

13 12

0 D

el in

qu en

cy 2

SR 12

/ 14

/ 20

09 In

iti al

C A

FA S

D ec

re as

ed 20

W ra

pa ro

un d

14 12

0 Su

bs ta

nc e

us e

2 R

N 10

/ 6/

20 09

3 m

on th

C A

FA S

In cr

ea se

d 10

In te

ns iv

e ca

se m

an ag

em en

t 10

14 0

T hi

nk in

g 4

W J

12 / 3/

20 09

6 m

on th

C A

FA S

D ec

re as

ed 30

H om

e ba

se d

15 12

0 Se

lf -h

ar m

/ se

ve re

m oo

d 2

W J

11 / 30

/ 20

09 In

iti al

C A

FA S

In cr

ea se

d 20

H om

e ba

se d

11 13

0 T

hi nk

in g

3 T

R 10

/ 14

/ 20

09 1

M on

th In

cr ea

se d

30 H

om e

ba se

d 15

12 0

D el

in qu

en cy

1 D

C 10

/ 23

/ 20

09 1

M on

th D

ec re

as ed

20 H

om e

ba se

d

319

C ul

tu re

o f S

ha re

d Vi

si on

Pr oa

ct iv

e, S

up po

rt iv

e, P

ro bl

em So

lv in

g Su

pe rv

is or

P ro

ce ss

es A

cc ou

nt ab

ili ty

– O

pe ra

tio na

liz in

g O

ut co

m es

D riv

en C

on tin

uo us

Q ua

lit y

Im pr

ov em

en t

C re

at e

pr oc

ed ur

es fo

r m

at ch

in g

as se

ss m

en t

re su

lts to

tr ea

tm en

t pl

an ni

ng (v

ia s

ta ff

in vo

lv em

en t a

nd le

ad er

sh ip

te am

)

•

M os

t a pp

ro pr

ia te

s

er vi

ce s

– st

ar tin

g

p oi

nt •

E

ffi ci

en cy

in u

se o

f

re

so ur

ce s

E m

be d

as se

ss m

en t

re su

lts in

a ll

pr oc

es se

s –

re po

rts , p

la ns

, m

ee tin

gs , s

up er

vi si

on

E ng

ag e

th e

yo ut

h an

d fa

m ily

in a

ss es

sm en

t pr

oc es

s

P ro

gr am

D ire

ct or

In di

vi du

al S

up er

vi si

on w

ith ea

ch s

up er

vi so

r (1

/w ee

k)

•

H om

e B

as ed

•

W ra

pa ro

un d

•

In te

ns iv

e C

om m

un ity

B as

ed •

C

as e

M an

ag em

en t

E ac

h S

up er

vi so

r m

ee ts

w ith

S up

er vi

se es

(1 /w

ee k)

Tw o

w ee

kl y

cl in

ic al

ca se

c on

su lta

tio n

m ee

tin gs

A tte

nd ee

s: •

S

up er

vi so

r •

C

lin ic

ia n

•

P sy

ch ia

tri st

•

P ro

gr am

D ire

ct or

•

A ll

S ta

ff at

te nd

at le

as t 1

m ee

tin g

pe r

w ee

k

In di

vi du

al C

lie nt

s

P ro

gr am

D ire

ct or

re vi

ew s

da ta

a nd

s en

ds s

ep ar

at e

re po

rt to

e ac

h su

pe rv

is or

re qu

iri ng

a ct

io ns

a nd

tre at

m en

t u pd

at es

o n

id en

tif ie

d yo

ut h

M os

ts e

ve re

ly im

pa ire

d (b

y sp

ec ifi

ed C

A FA

S s

co re

ra ng

e) w

ith v

er y

hi gh

ri sk

H ig

hl y

im pa

ire d

(b y

sp ec

ifi ed

C A

FA S

c lin

ic al

m ar

ke rs

) i nd

ic at

in g

hi gh

ri sk

Y ou

th w

ho h

av e

sa m

e C

A FA

S s

co re

fo r 6

m on

th s

or m

or e

In di

vi du

al c

as e

co ns

ul ta

tio n:

re qu

ire d

fo r C

A FA

S sc

or es

o ve

r 1 20

w ith

in 1

4 da

ys w

ith pr

og ra

m d

ire ct

or ,

cl in

ic al

te am

a nd

ps yc

hi at

ris t

M us

t u pd

at e

pl an

/ pr

es en

t c as

e at

a w

ee kl

y ca

se co

ns ul

ta tio

n m

ee tin

g

M us

t u pd

at e

pl an

/ pr

es en

t c as

e at

a w

ee kl

y ca

se co

ns ul

ta tio

n m

ee tin

g

A gg

re ga

te D

at a

E ac

h su

pe rv

is or

m on

ito rs

a gg

re ga

te da

ta fo

r t he

ir ow

n pr

og ra

m a

re a

- O ng

oi ng

A t q

ua rte

rly le

ad er

sh ip

te am

m ee

tin gs

, su

pe rv

is or

s re

po rt

ag gr

eg at

e da

ta a

nd ou

tc om

es fo

r C Q

I

U ni

t D ire

ct or

a nd

su pe

rv is

or s

us ed

a t a

to s

et p

ro gr

am /

te am

o ut

co m

e ta

rg et

s

R ep

ea t p

ro ce

ss a

fte r

re fin

em en

t

F IG

U R

E 1

A C

lin ic

al M

an ag

em en

tM od

el fo

r C

on su

m er

Su cc

es s

(c ol

or fig

ur e

av ai

la bl

e on

lin e)

.

320

OUTCOMES-DRIVEN PRACTICES 321

supervisory interactions and clarified expectations. Overall, the data-driven supervision model improved clinical focus, support for staff, accountability, clarity regarding follow-up on clinical and documentation tasks, and motivation for staff to continue to work toward positive outcomes for their consumers.

A FRAMEWORK FOR EFFECTIVE ACCOUNTABILITY

The final component in implementing the outcomes-driven clinical management model was to use aggregate data to define consumer-driven outcome performance targets and to design a process that uses aggregate data to continually inform practice. The ability to track a variety of youth outcomes for the organization was available through the benefits of the web-hosted CAFAS system, which included the functionality of outcomes reports that could be generated at any time. This functionality enabled each supervisor to check the progress of individual cases and the team aggregate outcomes in real time. Access to these supervisor dashboards was made available so that each team (e.g., home-based, wraparound, intensive case management services) could always know the status and outcomes of their cases, both aggregated and individually.

Additionally, individualized modifiers were added to each case in the web-based CAFAS sys- tem to identify when a consumer was receiving an evidence-based or best practice such as parent management training, trauma cognitive behavioral therapy, or wraparound. The modifiers helped track outcomes of children receiving these practices and the percentage of children who received them. The program administrator ran aggregate outcome reports monthly to compare data to tar- gets and shared this information with supervisors at leadership team meetings in which areas for improvement or clinical service delivery issues could be discussed.

Data patterns related to the CAFAS domains also were examined as part of annual needs assess- ments to identify trends in risk areas. These data were then used to help inform decisions regarding areas of need for evidence-based best practices or other necessary initiatives. For example, the community mental health center identified through the CAFAS profile that a significant portion of youth being served received special education services in the school system. This finding prompted additional collaboration and cross-training related to the use of positive behavioral supports. Data analysis also revealed a disproportionately low number of youth being identified as having substance abuse-related concerns. This discovery prompted an increased focus and training on awareness, as well as screening for substance abuse problems and possible co-occurring disorders.

The final use of aggregate data included setting target goals for outcomes to ensure continual striving for improvement via CQI efforts. For example, one of the goals set by the children’s services department leadership was for 60% of closed cases to achieve a meaningful and significant reduction in total CAFAS score, defined as reduction of 20 points or more (Hodges, Xue & Wotring, 2004). This target goal set in 2008 would represent a significant improvement from the department baseline data from 2006 and provide an attainable stretch goal for the clinical team. In addition, for youth who entered services with one or more severe ratings on the CAFAS, a target goal was set for 60% to exit services with no severe impairments in any domain. Supervisors were responsible for presenting their team progress on these targets quarterly at the department leadership meeting. In addition, supervisors were expected to provide analysis of barriers and offer solutions to enhance success where targets were not met. Lastly, at staff meetings, supervisors presented team outcome data quarterly to highlight progress and celebrate successes. This combination of reporting practices embedded the continuous quality improvement process at all levels of the program.

Two years after implementation of the clinical management system, the CAFAS outcomes data profile for Quarter 1 of 2010 revealed that, upon exit from services, 80% of the cases attained a 20- point or more reduction in their total CAFAS score. This reduction indicated meaningful and reliable improvement in functioning from intake to current CAFAS (Conklin, Hodges, & Revels, 2010).

322 KING AND HODGES

In fact, the results showed a substantial improvement in outcomes as indicated by an average change score of 38 for all cases (Conklin et al., 2010). This degree of change is one standard deviation, which represents a statistically significant improvement in functioning on average, and at least a moderate effect size. Additionally, these gains were made in spite of the first quarter data indicating a substantial increase in the severity of the needs of the youth coming into services compared to the average intake CAFAS score from the baseline data.

As part of the quarterly, agency-wide clinical care monitoring system, the final steps in this outcome monitoring component for quality improvement included reporting the data on current department-wide aggregate outcomes and progress toward performance measures. As a result, the use of the CAFAS and monitoring of results for consumer effectiveness were embedded into the agency’s performance improvement practices. CAFAS data and aggregate reports were also reviewed annually as part of program evaluation, planning discussions, and proposals for the coming year.

DISCUSSION

This article provides a qualitative review of a practice model in which clinicians and supervisors embraced a data-driven, clinical management approach with the goal of continuously striving to improve youth outcomes. The outcomes-driven management practices were greatly facilitated by utilizing an effective outcomes management system and a psychometrically sound measure of outcome. The article shows how outcome data can be effectively used to improve and monitor the quality of clinical treatment, manage risk, and keep treatment progress on track. Data trends indicated that staff became better at improving outcomes despite increased severity of youths’ needs.

A caveat about limitations of this analysis is in order. It is important to note that causation cannot be implied. These data trends may have no direct association to the use of this model, as other vari- ables or cohort effects could have been responsible or may have contributed to the results. In order to determine if this model played a causative role in improving outcomes, a comparison of this model to “supervision-as-usual” would be required. In addition, the model described may not be applicable to some clinical contexts, and other web-hosted outcomes management systems may be used besides the one described in this article.

Real-time technology played a critical role in the successful implementation of the model pre- sented by informing practice and monitoring outcomes. The technology available through the web-hosted system provided user-friendly, reliable, and easily accessed data crucial to outcomes management. English and colleagues (2000) asserted that two factors determine whether database technology will prove useful: the extent to which the database is integrated into the decision- making process and the nature and quality of the underlying data contained in the database. Both of these variables were addressed through the technology utilized in this implementation approach. Outcomes information was used to create and implement data-informed supervision practices, which in turn shaped service delivery and intervention strategies. These factors are crucial to embed- ding data-driven decision making and an empirical mindset into program management, program culture, and individual supervision and casework.

Another key component in this implementation of the outcomes-driven management practices was the role of a leadership approach based within a learning organization framework that created a supportive, solution-focused treatment culture. The literature also supports this approach as a cen- tral component of effectively implementing outcome management systems (Yoo & Brooks, 2005). The leadership provided in the case study ensured a focus on treatment and services that improve consumer outcomes while providing opportunities and processes for ongoing support and motiva- tion for clinical staff. This persistent focus on obtaining outcomes was accomplished through the

OUTCOMES-DRIVEN PRACTICES 323

co-creation of a vision, values, and culture to support this approach and also through accountabil- ity structures that maintain a focus on results within a context of supportive problem solving and continuous quality improvement.

Finally, of particular significance in the model presented was the value of data-informed super- vision. The approach supported previous conclusions by Mor Barak and colleagues, (2009) who emphasized task assistance as the key supervisory component in achieving positive outcomes. The data-informed supervision model presented in this article provided an important conduit for the effective and consistent implementation of supervisory task assistance grounded in the use of data. This observation is significant in light of the difficulties often encountered in achieving staff buy-in, as clinical staff often value relational treatment models over other approaches.

The field of social work could benefit greatly from more studies comparing management and supervision models and the corresponding impact on outcomes, as social workers administrate and direct many programs in mental health and child welfare. The current challenges in behavioral health care today require that we develop data-driven and outcomes-oriented approaches.

REFERENCES

Bliss, D. L. (2007). Implementing an outcomes measurement system in substance abuse treatment programs. Administration in Social Work, 34(4), 83–101.

Cohen, J. (2003). Managed care and the evolving role of the clinical social worker in mental health. Social Work, 48(1), 34–43.

Conklin, C., Hodges, K., & Revels, S. (2010, May). Using real time client level data to implement CQI practices that support EBTs and promising practices. Presented at Michigan Association of Community Mental Health Board Spring Conference, Dearborn, MI.

Cowger, C. (1994). Assessing client strengths: Clinical assessment for client empowerment. Social Work, 40, 506–514. Daleiden, E., Chorpita, B., Donkervoet, C., Arensdorf, A., & Brogan, M. (2006). Getting better at getting them better: Health

outcomes and evidence based practice within a system of care. Journal of American Academy of Child and Adolescent Psychiatry, 45(6), 749–756.

English, D., Brandford, C., & Coghlan, L. (2000). Data based organizational change: The use of administrative data to improve child welfare programs and policy. Child Welfare, 79(5), 499–515.

Fisher, E. (2005). Facing the challenges of outcome measurement. Administration in Social Work, 29(4), 35–49. Functional Assessment Systems (2009). FAS outcomes management system. Ann Arbor, MI: Functional Assessment

Systems. Retrieved from http://www.fasoutcomes.com Guthrie, D., McIntosh, M., Callaly, T., Trauer, T., & Coombs, T. (2008). Consumer attitudes toward the use of routine

outcomes measures in a public mental health service: A consumer-driven study. International Journal of Mental Health Nursing, 17(2), 92–97.

Hodges, K. (2000). Child and adolescent functional assessment scale (2nd revision) Ypsilanti, MI: Eastern Michigan University.

Hodges, K. (2004a). Child and Adolescent Functional Assessment Scale (CAFAS). In M. Maruish (Ed.), The use of psycho- logical testing for treatment planning and outcome assessment (3rd ed., pp. 405–441). Mahwah, NJ: Lawrence Earlbaum Associates.

Hodges, K. (2004b). Using assessment in everyday practice for the benefit of families and practitioners. Professional Psychology: Research and Practice, 35(5), 449–456.

Hodges, K. (2010). Child and Adolescent Functional Assessment Scale (CAFAS overview of reliability and validity). Ann Arbor, MI: Functional Assessment Systems. Retrieved from http://www.fasoutcomes.com/CAFASPsychometric Summary.htm

Hodges, K., & Wotring, J. (2004). The role of monitoring outcomes in initiating implementation of evidence-based treatments at the state level. Psychiatric Services, 55(4), 396–400.

Hodges, K., & Wotring, J. (2012). Outcomes managment: Incorporating and sustaining processes criticial to using outcome data to guide practice improvement. Journal of Behavioral Sciences and Research, 39(2), 130–143. doi:10.1007/s11414- 011-9262-y.

Hodges, K., Xue, Y., & Wotring, J. (2004). Use of the CAFAS to evaluate outcomes for youths with severe emotional disturbance served by public mental health. Journal of Child and Family Studies, 13(3), 325–339.

324 KING AND HODGES

Jensen-Doss, A., & Hawley, K. (2010). Understanding barriers to evidence based assessment: clnician attitudes toward standardiszed assessment tools. Journal of Clinical Child and Adolescent Psychology, 13(3), 885–896.

Kapp, S., & Stipp, K. (2010). Trolling for useful data in an automated information system: experiences of Kansas community mental health managers. Administration in Social Work, 34(2), 135–147.

Lampkin, L., & Hatry, H. (2003). Key steps in outcome management. Retrieved from Urban Institute, http://www.urban. org/url.cfm?ID=310776

Martin, L., & Kettner, P. (1997). Performance measurement. Administration in Social Work, 21(1), 17–29. Moore, T., Rapp, C., & Roberts, B. (2000). Improving child welfare performacne through supervisory use of client outcome

data. Child Welfare, 79(5), 475–497. Mor Barak, M., Travis, D., Pyun, H., & Xie, B. (2009). The impact of supervision on social work outcomes: A meta analysis.

Social Service Review, 83(1), 3–32. Mueller, C., Tolman, R., Higa-McMillan, C., & Daleiden, E. (2010). Longitudinal predictors of youth functional improve-

ment in a public mental health system. Journal of Behavioral Health Services & Research, 37(3), 350–362. Mullen, E. (2004). Outcomes measurement. Social Work in Mental Health, 2(2), 77–93. National Association of Social Workers (2008). Code of Ethics of the National Association of Social Workers. Washington,

DC: NASW. Retrieved from http://www.socialworkers.org/pubs/code/code.asp Neuman, K. (2003). Developing a comprehensive outcomes managment program. Administration in Social Work, 27(1),

5–23. Poertner, J. (2006). Social administration and outcomes for consumers. Administration in Social Work, 30(2), 11–24. Poole, D., Davis, J., Reisman, J., & Nelson, J. (2001). Improving the quality of outcome evaluation plans. Nonprofit

Management and Leadership, 11(4), 405–421. Rapp, C., & Poertner, J. (1988). Moving clients center stage through the use of client outcomes. Administration in Social

Work, 11, 23–28. Rosenberg, L. (2006). Improving our workforce. Behavioral Healthcare, 26(10), 39–40. Spitzer, D. (2007). Transforming performance measurement: Rethinking the way we measure and drive organizational

success. New York, NY: AMACOM Books. Williams, N. (2009a). Preliminary evaluation of children’s psychosocial rehabilitation for youth with serious emotional

disturbance. Research of Social Work Practice, 19(1), 5–18. Williams, N. (2009b). Dose-effect of children’s psychosocial rehabilitation on the daily functioning of youth with serious

emotional disturbance. Child and Youth Care Forum, 38(6), 273–286. Wotring, J., Hodges, K., Xue, Y., & Forgatch, M. (2005). Critical ingredients for improving mental health services: Use of

outcome, data stakeholder involvment, and evidence-based practices. The Behavior Therapist, 28, 150–158. Xue, Y., Hodges, K., & Wotring, J. (2004). Predictors of outcome for children with behavior problems served in public

mental health. Journal of Clinical Child & Adolscent Psychology, 33(3), 516–523. Yoo, J., & Brooks, D. (2005). The role of organizational variable in predicting service effectiveness: an analysis of a

multilevel model. Research on Social Work Practice, 15, 267–277.