Diversity and Cultures

CREATING CULTURALLY RESPONSIVE FAMILY THERAPY MODELS AND RESEARCH: INTRODUCING

THE USE OF RESPONSIVE EVALUATION AS A METHOD

Desiree M. Seponski University of Houston-Clear Lake

J. Maria Bermudez and Denise C. Lewis University of Georgia

Models of marriage and family therapy (MFT) typically reflect Western values and norms, and although cultural adaptations are made, many models ⁄ frameworks continue to be inappropriate or inadequate for use with non-Western cultures. Worldwide, therapists are examining ways of using MFT models in a culturally sensitive manner, especially when working with clients who are seen as having minority status or perceived as ‘‘other’’ by the dominant group. This essay suggests the use of responsive evaluation as a theoreti- cally consistent methodology for creating and evaluating culturally responsive therapies. This approach rigorously evaluates each unique client ⁄ therapist context, culture, power, needs, and beliefs. We describe responsive evaluation and discuss how each component addresses the research needs of examining culturally responsive family therapies. A case illustration is offered delineating the process of conducting culturally responsive therapy with a Cambodian sample using solution-focused and narrative therapy.

The field of marriage and family therapy (MFT) has made important strides toward apply- ing cultural sensitivity when working with minority populations. However, serious challenges remain with developing culturally responsive models of therapies. Many interventionists adapt preexisting models rather than develop a model that is designed to be theoretically and techni- cally responsive to the needs unique to a certain population. This is primarily because of a lack of awareness, resources, or adequate methods for creating culturally responsive models. Turner, Wieling, & Allen (2004) note that it is challenging but necessary to test an entire model or to create an emerging model based on the needs of the unique population.

We offer a framework for developing culturally responsive therapy (CRT) and research using responsive evaluation (RE) to assess or create a particular approach. Responsive evaluation is an interpretive methodology that ‘‘builds upon the experiential, personal knowing in real space and real time with real people’’ (Stake, 2004, p. xv). The interpretivist worldview allows the researcher to responsively focus on the issues that are of priority to the stakeholders and practitioners to develop a rich experiential understanding and provide information for improvement (Denzin & Lincoln, 1994). The foci are on the person’s experiences and views of the therapy and on the value of the therapy to that unique group. Coincidently, these foci also are consistent with CRT, as RE privileges the particular needs of a given population, is flexible to the unique needs of partici- pants, and relies on insider knowledge. It also is accountable to the community and works directly to improve the client’s context (Carlson, Erickson, McGeorge, & Bermudez, 2004) as well as assess the lives of participants from a strength-based perspective (Bermúdez, Zak-Hunter, & Silva, 2011). Integration of these two approaches, RE and CRT are compatible (see Table 1) and RE offer scholars and therapists a rigorous method for evaluating and conducting responsive intervention (see Table 2). The following provides an overview of RE and describes how each component addresses the research needs for examining culturally responsive family therapies.

Desiree M. Seponski, PhD, Family Therapy Program, University of Houston-Clear Lake, J. Maria Bermudez,

PhD, and Denise C. Lewis, PhD, Department of Child and Family Development, University of Georgia.

Address correspondence to Desiree M. Seponski, Family Therapy Program, University of Houston-Clear

Lake, 2700 Bay Area Boulevard, Houston, Texas, 77058; E-mails: [email protected], [email protected]

Journal of Marital and Family Therapy doi: 10.1111/j.1752-0606.2011.00282.x January 2013, Vol. 39, No. 1, 28–42

28 JOURNAL OF MARITAL AND FAMILY THERAPY January 2013

Table 1 Consistencies Between Culturally Responsive Therapy and Responsive Evaluation

Culturally responsive therapy Responsive evaluation

Epistemology Social constructionism Social constructionism Theoretical underpinnings

Interpretivism Interpretivism Feminism Naturalistic Just therapy Responsive Critical race theory Particularistic Relies on insider ⁄ local knowledge Qualitative Use of cultural advocated and advisor boards as accountability structures

Attention to context

Focus on socio-political and socio-cultural

Focus on socio-political and socio-cultural

Consideration of neighborhood, communities, and support systems

Recognition of social, cultural, and historical contexts

Recognize social, cultural, and historical contexts

Attention to culture

Consider traditions, values, worldviews, and life experiences

Attention to the culture of the organization, worldviews of each individual, and the experiences of each individual and in relation to one another

Attention to subordinations related to the intersection of race, class, gender, ethnicity, age, sexual orientation, nationality, and ability

Methods Client-centered Client-centered Use of nontraditional measures to address factors that affect recruitment and retention such as poverty, language barriers, structural racism and segregation, social isolation, and distrust toward institutions

Flexible, innovative, adapting, and emerging to fit the needs of the participants

Measures designed to fit population Responsive to concerns, key issues, problems, language, contexts, and standards of an array of stakeholder groups

Measures designed to fit population

Participants Clients and families Stakeholders at multiple levels Community members Various levels of hierarchy Therapists and supervisors Desire to find conflicting or a

wide array of views and experiences

Advisory board Accountable to minority community members and their experiences

Cultural advocates Relies on local knowledge Any stakeholders possible involved

January 2013 JOURNAL OF MARITAL AND FAMILY THERAPY 29

DEFINING CULTURALLY RESPONSIVE THERAPY

At the most basic level, being culturally competent is a necessary component of being a competent family therapist (Bean, Perry, & Bedell, 2001; Waites, Macgowan, Pennell, Carlton- LaNey, & Weil, 2004; Weaver & Wodarksi, 1995). Although multiple researchers describe CRT, there is no uniform definition nor are there concrete parameters for being culturally responsive. To the contrary, the terms culturally competent, culturally sensitive, and culturally responsive often are used interchangeably. This causes problems for researchers in operationaliz- ing, measuring, and cross-comparing CRT studies, as well as for therapists in understanding what is required to be culturally responsive.

A culturally competent service provider can provide services that address problems faced by people in multicultural contexts (Waites et al., 2004). Having a strength-based, nonpatholo- gizing stance is important, and family therapists are aware of the importance of offering inter- ventions that consider families within their socio-political and socio-cultural contexts (even within the same families) while paying special attention to neighborhoods, communities, and support systems (Waites et al., 2004). Attention to traditions, values, worldviews, and life expe- riences of the cultural groups in question is also paramount (Bernal, Bonilla, & Bellido, 1995; Castro, Barrera, & Martinez, 2004; Waites et al., 2004).

Being culturally competent as a researcher is also a necessary component of being a compe- tent researcher. At times, it can seem to be a monumental task to be culturally responsive as a researcher (Domenech-Rodrı́guez & Wieling, 2005). Traditional methods generally do not address factors that affect recruitment and retention such as poverty, language barriers, struc- tural racism and segregation, social isolation, and distrust toward institutions. Consequently, many services remain irrelevant to the needs of ethnic minorities (Parra-Cardona et al., 2009),

Table 1 (Continued)

Culturally responsive therapy Responsive evaluation

Findings Accountable to minority community members

A description of the quality of the process via the experiences of the stakeholders

Use of accountability structures to avoid the replication of dominant power structures

Feedback given throughout the process of evaluation for immediate change

Outcomes Understand the phenomenon An understanding of whether the participants perceive and experience the evaluand as valuable

Go beyond pathologizing participants in regards to external contexts to actively changing them

Users can use the data to alleviate, remediate, develop, or aspire within the evaluand

Explain roots of nonresponsiveness Does not aim for generalizability, but naturalistic generalization may occur

Criticize dominant power structures Empower researchers and participants Critique and change the situation Does not aim for generalization

30 JOURNAL OF MARITAL AND FAMILY THERAPY January 2013

Table 2 Applying Responsive Evaluation to Create Culturally Responsive Therapy

Responsive evaluation steps (Stake, 1975, 2005) RE applied to culturally responsive therapy

Talk with clients, program staff, audience, etc.

Talk with stakeholders including clients, friends and families of clients, therapists, supervisors, case managers, secretaries of the agencies, community members, insurance companies, etc. to gather information on their observations, perceptions, and experiences of the therapy model

Identify cultural advocates and create a cultural advisory board

Learn the culture, values, traditions, needs, and language of the stakeholders

Identify program scope Orient self to the community Observe therapy and community traditions and interactions Meet with various persons to find out what is of value to the stakeholders

Utilize documents and artifacts to gain background knowledge on the therapy and its use in the community

Overview program activities

Identify the activities surrounding therapy, including who attends sessions, how many sessions clients average, what is the cost of the therapy, how does the therapy influence the socio, cultural, and political environment, and how is impacted by these pressures?

Explore the role each stakeholder and the context play in the therapeutic process.

Determine the intentions and desired outcomes of the therapy for each stakeholder

Discover purposes, concerns

Identify the concerns of each of the stakeholders Work to amplify marginalized voices, especially of the clients and traditionally silenced stakeholders

Gather impressions of the therapeutic worth from various individuals whose opinions differ

Conceptualize issues, problems

Create a plan of potential issues and purposes of the evaluation. For example,

Are the theoretical model’s underlying epistemological and theoretical tenets consistent with cultural values?

How does the model respond to the unique needs and values of the client ⁄ therapist ⁄ family?

Do the interventions and goals meet client needs? Who is included in therapy? Why? How does this affect the client and system?

Are the therapeutic outcomes satisfactory? Review identified issues with the stakeholders, cultural advisory board, and cultural advocates

Redefine and revise the issues to be evaluated, if needed Identify data needs, re. issues

Determine the type of data needed for the evaluation Consider methods that are responsive to the needs of the stakeholders (role-playing, case notes, interviews, observation, measures, and focus groups)

Arrange data collection around the convenience of the clients, family, community, and therapists

January 2013 JOURNAL OF MARITAL AND FAMILY THERAPY 31

even though culturally responsive interventions are reported to be superior to traditional meth- ods with regard to client satisfaction, retention rates, and perceived credibility of the interven- tionist (Lu, Organista, Manzo, Wong, & Phung, 2001).

We use the definition provided by couple and family therapy researchers Carlson et al. (2004), who make several theoretical and conceptual distinctions between culturally competent and culturally responsive approaches. They assert that one can become culturally competent through learning, which can depoliticize power differentials, and can stand as an outsider expert on insider minority peoples’ needs. They contrast this with culturally responsiveness as being informed by feminism, just therapy, and critical race theory. Further, they stress the importance of cultural advocates and advisory boards to reduce power differentials. Competency, which relies on knowledge, awareness, and sensitivity, and responsiveness are similar in intention but diverge theoretically and conceptually in regard to the acquisition of knowledge, the distribu- tion of power, and the role of the expert. It is important to emphasize that culturally responsive therapists rely on local knowledge and are accountable to the local community and their unique needs, whereas culturally competent therapists assume they can acquire cultural knowledge and act as an expert in that community. For the purposes of synthesizing how culturally responsive

Table 2 (Continued)

Responsive evaluation steps (Stake, 1975, 2005) RE applied to culturally responsive therapy

Select observers, judges, instruments if any

Invite appropriate persons to observe and engage in the therapy and comment

Determine measures or create new instruments to meet the needs of the unique population

Ensure that the instruments measure what the stakeholders view as important, as opposed to being convenient for the evaluator

Observe designated antecedents, transactions, and outcomes

Begin more ‘‘formal’’ data collection Adapt and re-create methods based on stakeholder needs Continually check quality and accuracy of data; encourage member checking

Provide feedback throughout data collection; if something is or is not working, reveal

Thematize; prepare portrayals, case studies

Create narrative, portrayals, case studies, charts, graphs, etc. to depict tentative findings

Check quality of records and interpretations Gather reactions for stakeholders and identity the relevance and accuracy of findings

Winnow, match issues to audiences

Allow stakeholders to react to the various findings Work to represent concerns of each group (clients, therapists, families, community, etc.)

Format for audience use Provide feedback in useable format for various stakeholders Consider the use of various publication formats: client portfolios, pamphlets, presentations, role-plays, documentaries, art, radio, poem, narratives, and manuscripts

Assemble formal reports, if any

Create reports that are valuable and useable to the needs of the unique population

If publishing for a wider audience, provide in-depth descrip tion to promote vicarious experiencing and a potential for naturalistic generalization

32 JOURNAL OF MARITAL AND FAMILY THERAPY January 2013

family therapy can be studied using RE, key theoretical assumptions of feminism, just therapy, and critical race theory are briefly reviewed.

Feminism The overall purpose of the various branches of feminism is to rethink gender and power,

especially in regard to women’s subordination and other subordinations related to the intersec- tion of race, class, gender, ethnicity, age, sexual orientation, nationality, and physical ability. Consequently, feminist family therapy is a philosophy, political perspective, and lens for viewing the world that does not have a set of interventions or guidelines (Haddock, Zimmerman, & MacPhee, 2000). Feminist family therapy scholars and researchers have provided suggestions, which include exploring the effects of socialization on the problem, educating clients on egalitar- ian and mutually respectful relationships, teaching people to recognize and express their emotions, and encouraging clients to share responsibility for the relationship, family, and home (Braverman, 1988; Chaney & Piercy, 1988; Haddock et al., 2000; Silverstein & Goodrich, 2003; Wheeler, Avis, Miller, & Chaney, 1988). Table 1 delineates how these feminist values inherent in CRT are consistent with RE.

Just Therapy Just therapy is committed to the cultural, gender, and socio-economic equity of clients and

to eradicating social discrimination through attention to broad cultural, gender, social, spiritual, economic, and psychological contexts underlying the problems experienced by those with whom therapists work (The Family Center, 2009). Just therapists work to go beyond pathologizing clients suffering from external contexts, for helping in understanding and actively changing those contexts. As shown in Table 1, this focus on the socio-political, socio-economic, and socio-cultural contexts is consistent with RE.

Critical Race Theory Critical race theory is a framework that focuses on ways that injustice and subjugation

shape one’s world (Patton, 2002) and on providing explanation that empowers the subjugated to overcome the subjugation (Fay, 1987). Critical race theory can be applied (a) to explain how previous models have not been culturally responsive but have been ethnocentrically imposed upon minority populations, (b) to criticize the ways by which this lack of sensitivity has been kept in place, and (c) and to empower researchers and therapists through action research to learn about and identify a model that is culturally responsive. In conjunction with CRT, RE can be used to evaluate whether a therapy model is culturally responsive, as opposed to other methodologies that focus on a given intervention or on therapists’ worldviews. Next, we offer an overview of evaluation as a method of examination and how the factors mentioned earlier inform the use of RE as a way to assess or create a model of therapy that is culturally responsive.

GOALS OF EVALUATION

The goals of program evaluation are specific. The following roles are central to the purpose of evaluation: assessing goal attainment, aiding organization development, assessing contextual quality, studying policy, aiding social action, and legitimatizing the program and deflecting crit- icism. Secondary aims include assuring accountability, accrediting programs, estimating cost- effectiveness, and granting awards or standings (Stake, 2004). This process allows one to use evaluation methods to assess the use of the CRT model, examine how it works within a given context, to influence policy and social action surrounding the community and funding for the implementation of the CRT, and to estimate the cost-effectiveness and cost-efficiency of the CRT model. These goals and methods of logic do not always meet the needs of the stakehold- ers, do not take into consideration stakeholder context, culture, power, biases, needs, or beliefs, and are not responsive to the emerging issues or needed changes. Logic and the goals and process of evaluation are not universal constructs. To address this reality, Stake (1975) pro- posed the methodology of RE, which builds on basic evaluation principles and responds to

January 2013 JOURNAL OF MARITAL AND FAMILY THERAPY 33

stakeholder issues and contextual properties. RE can be used to consider the client’s, family’s, and therapist’s contexts, culture, power, needs, and beliefs and is more appropriate than basic models of evaluation, which often neglect marginalized stakeholders’ needs. This is especially important in CRT research, as the purpose of CRT is to respond to the participants, so the methods should do the same.

GOALS OF RESPONSIVE EVALUATION AND COMPATIBILITY WITH CULTURALLY RESPONSIVE THERAPY

Similar to the goals of CRT, RE focuses on specific problems of a particular program in a singular program context. As an interpretive, naturalistic, responsive, particularistic, and quali- tative approach (Stake, 2004), RE allows the researcher to responsively focus on issues that are of priority to stakeholders and practitioners, to develop a rich experiential understanding, and to provide information for improvement (Denzin & Lincoln, 1994). The evaluator seeks to understand how people perceive and experience their lives (Stake, 2004). Abma (2005) describes RE as a holistic qualitative method that derives its foci from multiple stakeholders’ perspectives and issues. It is a disciplined inquiry that enables a greater understanding of contextualized human behavior through inclusion of multiple voices.

Responsive evaluation seeks an understanding of the nature, merit, and worth of the evalu- and (Stake, 2004), to gain that understanding from as many stakeholders as possible, and to respond in a way that is natural to the people involved (Stake, 1975). In CRT research, this process may include gathering information from supervisors, trainers, therapists, adult clients, children clients, families and friends of clients, the front desk workers who schedule appoint- ments, caseworkers, and perhaps even funding agencies.

It is important to understand that the focus is on program quality via perspectives of par- ticipants; it is not so much whether the program has inherent value itself, but whether the par- ticipants perceive and experience it as valuable. In RE, the evaluator is considered a stakeholder and so his or her perceptions are also taken into account. RE is a way for evaluat- ing and documenting program quality by using interpretation and criteria measurements (Stake, 2004). It is especially responsive to key issues or problems identified by people at the sites. Stake states that the ‘‘understanding of goodness rather than the creation of goodness is its aim. Users may go on to alleviate or remediate or develop or aspire, but the purpose of this evaluation is mainly to understand (p. 89).’’

This is a divergence from traditional evaluation where the focus is on the value of the pro- gram itself, not necessarily if the program is valuable to a given population. In the case of minority or marginalized populations in need of CRT, a model has little value if it is not appli- cable and usable for the intended population. This is especially important for therapists, as CRT is accountable to the minority community members and their experiences, as opposed to empirically validating or researching a model for the sake of knowledge, and it relies on local knowledge to inform the clinical work.

Epistemology and Theoretical Framework The epistemological and theoretical framework described herein is aligned with the post-

positivist views of many couple and family therapists. This type of action research focuses on the context, time, and place and is never value free (Small & Uttal, 2005). It is important to consider and openly acknowledge this value-based epistemology, especially the ‘‘‘epistemologi- cal’ perception of quality’’ (Stake, 2004, p. 6). Stake (2004) stresses the importance of concen- trating on people’s perceptions, choices, and judgments.

When a researcher ⁄ therapist is responsive and focuses on the individuals opinions, con- texts, perspectives, etc., they receive rich data on the inside perspective of the program but lose the credibility of standardized criterion measurements that are provided to compare and contrast standardized measurements between groups (Stake, 2004). This may affect the use and acceptance of the findings, depending on the audience. Concentration on one unique population will promote CRT for that community, but will not necessarily be generalizable to other groups. This is important to note as it meets the CRT tenets of being accountable and

34 JOURNAL OF MARITAL AND FAMILY THERAPY January 2013

concerned about a unique group but does not necessarily empirically validate the model; instead, it shows that it is valuable for that particular population.

Interpretivism and Social Constructionism Responsive evaluation is rooted in interpretivism (Denzin & Lincoln, 1994; Stake, 2004,

2005), which is epistemologically informed by social constructionism. Social constructionism posits, ‘‘all meaningful reality, precisely as meaningful reality, is socially constructed’’ (Crotty, 1998, p. 55; emphasis in original). Thus, our culture and social interactions are both the source of our human thought and behavior and are bi-directionally affected by our thought and behavior (Crotty, 1998). These dominant discourses of our culture and society often determine our reality (Wetchler, 1996). In this worldview, individuals interpret and understand the world in which they live via subjective experiences based on the situation, and social, cultural, and historical contexts (Creswell, 2007). Thus, one cannot think, act, or behave independently of one’s background and personal interpretations of the current situation. Interpretivist research- ers desire to understand how others make sense of their world and recognize that their own personal, historical, and cultural backgrounds tint their interpretations (Creswell, 2007). Just therapy, feminist, and critical race theory underpinnings of CRT are consistent with the RE social constructionism epistemology. Both recognize that clients are affected and view experi- ences via their own context and social, cultural, and historical contexts. RE methods reveal these perceptions and experiences for an emerging CRT. Further, using RE ⁄ CRT pushes researchers and therapists to recognize their own background and context and how that influ- ences emerging knowledge and interpretations.

Defining Responsiveness Similar to CRT, to be responsive in RE means to be ‘‘client-centered’’ (Stake, 2004, p. 101)

and to respond to concerns, key issues, problems, language, contexts, and standards of an array of stakeholder groups. RE studies emphasize social issues and cultural values as well as per- sonal and programmatic dilemmas (Stake, 2004). The aim of RE is to respond to the needs of the clients rather than test the model to find outcomes.

The researcher holds a predetermined, intentional stance to become immersed in the con- text, personally know stakeholders, and to be action oriented and dedicated to accommodating the stakeholders’ needs and issues. This is similar to the idea of cultural competency and sensi- tivity in attitude and to culturally responsiveness in action—meaning that the researcher has background knowledge (cultural competency) and sensitivity but actually takes action to respond to client needs. In the field of CRT research, one would take action to change the methods, make the participants more comfortable, work within the context, develop culturally appropriate measures and instruments, and include more family members or clients. In RE and in CRT’s feminist, just therapy, and critical race theory underpinnings, researchers have inten- tional worldviews to actively respond to the stakeholders.

A Case Study of a Flexible, Responsive Design A case illustration of a flexible, responsive design is presented here. Seponski and Lewis

(2010) in collaboration with multiple stakeholders in Cambodia created a narrative and solu- tion-focused therapy informed outreach program using a booklet, ‘‘My Grandmother and Me’’ was based on the community-addressed need for reducing stigmatization and improving inter- generational relationships of families affected by HIV ⁄ AIDS and was originally developed by Seponski and Lewis (2010). Active and continuous input from the other collaborators regarding cultural sensitivity, languaging for the intended audience, and translation and back-translation of materials aided the refinement of the outreach program process. The researchers, host site faculty, and students also discussed academic needs, which included clinical education, experi- ence, and supervision, as well as the resources of the researchers, which included research and clinical expertise, education in gerontology and MFT, and grant funds. Each collaborator agreed that responding to the crisis of over 300,000 children being orphaned by HIV ⁄ AIDS and working with a local nongovernmental organization (NGO) that supports this population would be most beneficial project for all involved. At this point, NGO staff members were

January 2013 JOURNAL OF MARITAL AND FAMILY THERAPY 35

invited as collaborators. Once the collaborative group was formed, Seponski conducted a 1-day university wide training on solution-focused therapy and a half-day training on the therapy booklet for the host faculty and students and for the NGO staff members.

Because no Cambodian-normed instruments preexisted to measure therapeutic change ⁄ out- comes, and time was limited, the stakeholders agreed on the use of interviews, focus groups, document analysis of the children’s art, and live observations to accommodate time and loca- tion preferences of the stakeholders (i.e., the children, their grandparents who were raising them, student therapists, the clinical supervisor (Seponski), the faculty, and the NGO). Com- bining each of these methods allowed optimal collection of information. Data collection and therapy occurred simultaneously, with Seponski conducting live observations while supervising, obtaining reflections from student therapists between sessions, answering questions raised by student therapists and NGO staff, and providing needed educational feedback. Giving and get- ting feedback was an essential daily process for adapting the therapy.

As outsiders, Seponski and Lewis (2010) purposefully addressed their roles and personal biases to the other stakeholders. To this end, Seponski worked to balance the roles of educator, supervisor, and researcher, while Lewis worked to establish collaboration and promote partner- ships of equal investment with the university and NGO. Often, because of their privileges of education, wealth, and US citizenship, local stakeholders deferred to the researchers from the US. Thus, the researchers continually strove to share the power, validate local knowledge in front of local stakeholders, and amplify the marginalized voices of the children and student therapists. Further, Seponski and Lewis had to be clear on their biases and needs as stakehold- ers themselves: they were time-limited in their travel, restricted in funding, and invested in pro- ducing a publishable manuscript to meet academic, funding, and tenure requirements. The experiences of Seponski and Lewis in that collaborative project led us to conceptualize the pres- ent framework for using RE to develop and inform culturally responsive intervention. The RE design is adapted and adjusted regularly to accommodate the local context and cultural, social, behavioral, and economical factors (Abma, 2005) and to accommodate changing needs (Abma, 2005; Stake, 2005). Thus, the initial design is tailored to the unique project, and revisions and modifications are made in response to the program needs revealed by the data that is integrated and synthesized from multiple sources at multiple levels as shown in the Seponski and Lewis (2010) project.

In the preceding paragraphs, we have delineated the commonalities between RE and CRT. We have provided a review of the foundational elements necessary for creating a CRT program that combines RE. Later, we highlight some of the pragmatic aspects of using RE as a method for creating CRT.

STEPS FOR USING RESPONSIVE EVALUATION

Creating a Contract and Establishing Goals The evaluator and hiring agency must negotiate the contract of the evaluation and be clear

on the evaluator’s role, the evaluand (Stake, 2004), and what legitimizes the evaluation for the stakeholder (i.e., test scores, behaviors, specific language, outcomes, etc.; Stake, 1975). In CRT research, this would include meeting with an advisory board, consultants, and stakeholders to determine each person’s or agency’s role. After roles have been outlined and agreed upon, prominent events in RE occur multiple times throughout the evaluation are not sequential or linear, often co-occur, and are returned to at various points (Stake, 1975, 2004, 2005). In early stages of research, including having initial discussions with the stakeholders and identifying the program scope, the evaluator often uses sub steps as shown in Table 2 (Stake, 2005, pp. 223– 224).

Contextual Assessment and Information Gathering For an evaluation to be successful and useful to stakeholders, the evaluator must know the

issues, culture, language, and needs of the stakeholders for whom the evaluation is being completed. Through time in the field, interviews, document analysis, and examining photo- graphs, artifacts, and local historical pieces, the evaluator can gain invaluable knowledge about

36 JOURNAL OF MARITAL AND FAMILY THERAPY January 2013

the program and gather potential questions to ask stakeholders before implementation (Stake, 2004). An interview guide, rather than direct questions, allows researchers to know and seek information on what the population is potentially facing. The background information should contain a wide array of issues, voices, and opinions about the intervention program. Consistent with CRT’s feminist, just therapy, and critical race theory worldviews, the evaluator should also identify power imbalances and marginalized persons in the community affected or not affected by the intervention.

Questions stakeholders are asked and background explorations are specific to the program in real time. When examining and evaluating the evaluand, it is important to consider and examine processes and personal, contextual, and historical needs (Stake, 2004). Local meaning is important (Stake, 2005) as REs, like case studies, focus on the experiential knowledge of the participants and the influences of multiple contexts, including social and political contexts.

Identify Issues and Scope of Problem The program problems (or issues) are identified after the background has been explored, in

particular, immediate, interactive, particular, and subjective issues felt by stakeholders (Stake, 2004). The evaluator then chooses a few issues as the focus of the study, which helps determine the merit of the situation without replacing the goal of evaluating the merit.

Choose Assessment Measures and Procedures Instruments and methods of data collection are chosen to best measure the evaluands’

merit and value. It is acceptable to have both standards-based (generally quantitative) and RE (generally qualitative) instruments and methods in the same study (Stake, 2004). Initial mea- surements should be used to gather criterial data, a primarily interpretive process, which helps determine the merit of the evaluand (Stake, 2004). The evaluator then uses various methods (focus groups, interviews, observations, role-playing, surveying, etc.) to evaluate the evaluands’ antecedents, transactions, or outcomes.

Measurements should respond to the observations of the unique program and the unique issues of the group, so the researcher can respond to the emerging issues and needs of the program and other stakeholders (Stake, 1975). While it may be easier to use predetermined or popular scales, both the culturally responsive therapist and responsive evaluator would be more interested in being flexible with the inventories or developing new measures that fit this particu- lar population. Observation, document analysis, and interviewing are not just a part of the data collection, but a way to plan and focus for a flexible, innovative, and adaptive program design (Stake, 2004). The responsive evaluator, like the culturally responsive therapist, recognizes that she or he does not have inside expert knowledge and therefore calls on the insider’s local knowledge to help design the study and gain a better understanding of their experience of the evaluand.

Giving and Getting Feedback It is appropriate and expected to give feedback to improve the process and to refine the

focus (Stake, 2004). If the evaluator becomes aware of something that is not working, she or he gives immediate feedback at that moment to implement change, as opposed to waiting until the end of the evaluation. The process is facilitated by the participation of the stakeholders, which builds their confidence in the information and the applicability of its use in their unique context. This is one of the key benefits of RE—providing immediate feedback to responsively fit the needs of the stakeholders. Consistent with CRT, the point is to respond to the needs of clients, to give immediate feedback to change the model, and to continually adapt to meet the needs of the clients ⁄ participants.

Participants Because RE is a method of inquiry whereby the design emerges from the issues and con-

cerns of the stakeholders, as many stakeholders as possible should be included. These issues and concerns are obtained through in-depth conversations with stakeholders, especially those who are marginalized and whose voices are silenced because of previously experienced systemic

January 2013 JOURNAL OF MARITAL AND FAMILY THERAPY 37

repercussions and institutionalized oppression. It is the researcher’s role to create an environment of openness, respect, inclusion, and engagement (Abma, 2005) where the participants can interact and engage in dialogue that is nonconfrontational. It is crucial to create an environment where the power balances and voices are equal. Abma (2005) suggests this can be achieved via in-depth interviews to gain thick descriptions. Securing thick descriptions provides factual details, allows one to generalize local, context-bound knowledge to a larger system, and assures that the biases of one group do not dominate the evaluation.

Because RE works to balance power, opens dialogue for all voices to be heard, examines the values, biases, and group power imbalances, is attentive to the context, and works to elimi- nate the domination of marginalized groups, it is consistent with the goals of CRT. Multiple voices, diverse points of view, and continuous observations are included to reduce biases (Stake, 1975). Such an inclusion provides an abundance of valuable data, and it helps identify exceptions, outliers that demonstrate complexity, and unrepresentative voices (Stake, 2004); therefore, the evaluation should represent the complexity of multiple truths and opinions. Ambiguity is tolerated because these data are from multiple perspectives, are interpretive, and are socially constructed (Stake, 2005). While it is not necessary for the stakeholders to reach a consensus on the criteria, it is necessary for each participant to have a share in the power and in the process for change (Abma, 2005).

In addition to the stakeholders sharing equal power, and consistent with CRT’s feminist’s attempts to flatten the hierarchy (Matheson, Benson, & Walsh, 2004), stakeholders have power equal to the evaluator. RE honors the previously unexposed knowledge of the participants (Stake, 2004) whereby voices of the local community generate knowledge and change within that community. Local voices are seen as having value and privileged knowledge that are equal to the academician’s contribution (Small & Uttal, 2005). The evaluator must work diligently to examine the standards set by those empowered, recognize how those standards influence stake- holders differently, and give equal voice and power to multiple perceptions when evaluating the program. While stakeholders are central to the evaluation and their voices and standards are not to be ignored, in times of disagreement, it is ultimately up to the evaluator to discern the main standards of the evaluand (Stake, 2004).

Methods Similar to other clinical studies, RE studies generally take the form of case studies using

open-ended interviews, on-site observations, and document reviews. However, this is not always the case, as RE relies less on formal, structured interviews and more on natural and casual con- versation (Stake, 1975, 2004, 2005). In accord with the model’s interpretivist stance, methods should be interactional and adaptive to best understand the context, experiences, and multiple perspectives (Denzin & Lincoln, 1994). Methods can include e-mail, telephone, and in-person conversations and interviews, group interviews, focus groups, live observations, surveys, client records, narratives, snapshots, and videotapes (Stake, 2004). Creative methods, such as journal- ing and art, are also encouraged. For example, Abma (2005) utilized conversational interviews and storytelling workshops to educate stakeholders on other stakeholders’ perspectives, allowed stakeholders to respond to one another in a safe manner, encouraged the participants to inter- act, promoted dialogue, and encouraged stakeholders to raise their own questions. Therapeuti- cally, one could use family photographs, art, heirlooms, and stories. Feminist-informed, critical, and multiculturalists family researchers McDowell and Fang (2007) have suggested that researchers use these diverse methodologies to support social equity and amplify marginalized voices.

The overall goal of RE to have quality data (Stake, 2004) requires interviewing the right people, using the right instruments. Researchers must find good indicators and responsively ground the interview, survey, and questionnaire in the literature, in information previously described by the stakeholders, or information obtained through review of historical artifacts and documents. Consistent with an interpretivist worldview (Creswell, 2007), it is best to ask global and general questions to get the overall meaning of an issue (e.g., How do you feel about the issue? How do you perceive it?). Reviewing and piloting methods are encouraged before the data gathering stage (Stake, 2004).

38 JOURNAL OF MARITAL AND FAMILY THERAPY January 2013

Role of the Researcher Like action researchers, responsive evaluators must work to establish the partnership,

including finding collaborators and identifying stakeholders. They must determine research questions for encouraging social action, anticipate changing research questions, make adjust- ments, and modify the research design. In addition, they must reveal their stake in the research questions and measures and balance authority and expertise so that there is a genuine partner- ship (Small & Uttal, 2005). Responsive evaluators must be comfortable with ambiguity and understand that they may not be able to predetermine appropriate measures and that they will have to readjust the design to the context (Abma, 2005).

Researchers also must have strong interpersonal, communication, and negotiation skills, as well as genuine empathy whereby they can relinquish the role of expert and adopt the collabo- rating and power-sharing roles of interpreter, facilitator, educator, and guide (Abma, 2005). It is beneficial for the evaluator to spend time in the field to best understand the community and build relationships. These roles are consistent with CRT underpinnings and researcher roles, as they work to give voice to the marginalized, to balance power within the research, and to create social change. Further, the emphasis on local knowledge and the willingness and dedication of the evaluator to adapt to the client needs is present in both CRT and RE.

Biases of Evaluator ⁄ Researcher The evaluator must be reflexive (Stake, 2005), aware, and open about her or his role as

stakeholder. An evaluator who is outside the culture of the evaluand may misinterpret the stakeholder group’s meanings and struggle between balancing power and completing the evalu- ation in a timely manner. Sensitivity to the needs of the group and its members’ boundaries and a healthy understanding that some stakeholders are skeptical of the evaluation process and may not accept the formal reports will help the evaluator in the reflexive process. This does not mean that the evaluator must comply with all wishes, but that self-correction and transparency are important and can be incorporated with peer reviewing and meta-evaluation (Stake, 2004).

The responsive evaluator’s interpretations are seen as a way to enhance the experience and evaluation of the program (Stake, 2004). However, one dangerous bias of evaluators is to treat the group or ideas in either an overly negative or positive light (Stake, 2004). This bias should be kept in the forefront of the evaluator and meta-evaluator’s minds. An emphasis on power and potential research biases is consistent with CRT, where the therapist ⁄ research works to bal- ance and be aware of differential power and of biases.

Analysis Data can be coded, stored, and analyzed in several ways. In general, the evaluator codes

and interprets continuously throughout the evaluation, keeping a detailed audit trail, logging, and checking all of the material including the people, places, narratives, quotes, and photo- graphs. Because of time and funding constraints and a limited need for exact quotes or langu- aging, the evaluator does not usually audio record the interactions. Notes are compiled and reviewed with participants after interviews (Stake, 2004). Codes, notes, logs, and files each con- tribute significant information in RE and provide a deeper interpretation that unifies the story (Stake, 2004).

The evaluator may consider each stakeholder’s perspectives, the cost of the therapy, the reduction in symptoms, the comfort of the client, and whether the criteria and standards were met when reviewing whether a family therapy model is culturally responsive. The evaluator codes the data and then looks for patterns for consistency, repetition, variations, or causality (Stake, 2004). Patterns are cross-checked for reflection. The synthesis of data gives meaning and interpretation to the analysis to pull it all back together (Stake, 2004).

The perceptions of merit should be considered from multiple levels. Comparisons should be made across groups and across time, instrument and procedure strengths and weakness should be considered, and response and participation rates taken into account (Stake, 2004). In both RE and CRT, it is necessary for the analysis to be systematic, clear, intentional, recorded, and repeatable to hold the researcher and therapist accountable, ensure methodological rigor, and to balance power.

January 2013 JOURNAL OF MARITAL AND FAMILY THERAPY 39

Validation ⁄ Trustworthiness Responsive evaluation must ensure quality through validation and trustworthy strategies.

These strategies include member checking (i.e., participants review the transcripts and analysis), triangulation of the data, and researcher journaling (i.e., reflexive writing of the evaluation pro- cess and the researchers role; Abma, 2005). Technical steps, such as replication, nonverbal oper- ationalization, triangulation, and creating a list of participants to observe, judge, and to whom responses are given can also be used to enhance reliability of data (Stake, 1975). Ongoing review is needed during the case study from all of the stakeholders and the researcher (Stake, 2005), as member check and quality control is essential to a successful RE (Stake, 2004).

Critical colleagues, review panels, and meta-evaluators should be an active part of this pro- cess (Stake, 2004). For example, Carlson et al. (2004) conducted a culturally responsive narra- tive ⁄ just therapy project for Latinos in their community (utilizing, but not identifying, many RE methods). Local community members educated the therapists on their experiences and insights on racism and oppression as Latinos. Cultural advocates were then utilized to provide an emic, local approach and provided an insider expert and consulter role to the therapists. Further, Carlson et al. (2004) used an accountability board of local Latinos to consult on the program. Based on the needs and views of all of the stakeholders, they worked to address the inequalities and pursued the goal of educating local Latinos about being marriage and family therapists and to examine and dismantle the historical inequities that have previously stopped this process. The use of clients, therapists, cultural advocates, stakeholders, and consultants permitted a truly CRT program, such as suggested by RE.

Additionally, as with other qualitative traditions, triangulation of data is important for assuring methodological rigor. In a CRT study, this would include comparing across persons (therapists, clients, client families, agency workers, etc.), families, the community, the evalua- tors, the model, the methods, and across feminist, just therapy, and critical theories. One may even compare across measures and inventories, perhaps measuring symptom reduction in sev- eral ways. Triangulation, especially in RE and CRT studies, goes beyond confirmation to possi- bly provide alternative meanings (Stake, 2004). While it is not necessary to triangulate all pieces of data or all assumptions, the more thorough the triangulation, the more opportunity for uniqueness to emerge and the higher quality of validation.

Generalizability Although the focus of RE is on a unique program and context, readers are able to make

naturalistic generalizations through the vicarious experiences written by the researcher (Stake, 2005). The evaluator, with all the data from all the sources, having completed the analysis, can make a synthesized judgment on the merit of the program (Stake, 2004). A description rich in detail describes the multitude of voices and shows how the complex processes work within the program (Small & Uttal, 2005). Formal generalizations and conclusions are made, and descrip- tions for vicarious experiencing and naturalistic generalizations are given (Stake, 2004). For example, therapist readers using CRT for one Southeast Asian group may identify whether she or he is able to use some of the findings to apply to another Southeast Asian group, from one collective culture to another, or from one native group to their refugee group counterpart, all based on vicarious experiencing and naturalistic generalizations of the reader.

CHALLENGES OF USING RESPONSIVE EVALUATION

Working with several criteria and issues is enlightening but may create resource constraints on the evaluator. There is a delicate balance between having enough or too much data. Simi- larly, excess triangulation and member checking creates a risk of too narrow a view (Stake, 2004). The determining factor may depend on the planned intent of the evaluation report—to publish in an academic journal, disseminate widely, or to use solely by that program. Family therapy researchers who use RE while working with minority and marginalized populations may find their work challenged by critics who value more generalizable results or may question the authenticity of their research methods.

40 JOURNAL OF MARITAL AND FAMILY THERAPY January 2013

Consistent with McDowell and Hernández’s (2010) essay on decolonizing practices in academia and family therapy, and Mock’s (2008) essay on social justice and accountability in MFT training, it is our hope that academia and MFT specifically will become more inclusive of postpositivist methodologies that directly involve and affect the lives of those whom we study and strive to help with our interventions. We also recognize that one must navigate the inherent challenges in doing so especially for scholars of color and others in marginalized positions in academia (i.e., sexual minorities, students, untenured faculty, etc.). One must also contend with the goals of stakeholders who value the role and use of empirically supported treatments. This method meets the tenets of CRT by being accountable and concerned about a unique group but does not necessarily empirically validate a model; instead, it shows that it is valuable and valid specifically for a particular population. Generalizations can be made only to the extent that the goals of one group are similar to the goals of another.

Evaluation counsel boards, such as the National Research Council, and researchers them- selves are heightening the importance of RE to be rigorous, intentional, and research oriented. These suggested RE guidelines are consistent with CRT, will promote stronger findings, and will push the evaluator to be clear with intent, consistent with theory, and rigorous with design and implementation to create a strong model of therapy. However, there seems to be a tension created between being responsive to the situation and creating rigor. Being responsive in itself is seen as a form of rigor in CRT. We propose utilizing this strength, as well as all of points listed in this section by both the National Research Counsel and Stake (2004), to create a rigorous research evaluation study on CRT.

BENEFITS FOR THE FIELD OF MARRIAGE AND FAMILY THERAPY

We assert that using a RE methodology is beneficial in creating culturally responsive family therapy and research methods. Both RE and CRT are epistemologically and theoretically aligned, are consistent with feminist, just therapy, and critical race theory worldviews, and aim to respond to the unique needs of a given population. Utilizing RE allows the researcher to adapt to the changing issues and needs of the participants, use flexible methods, rely on local knowledge to inform the research, be accountable to minority members, and depend on accountability structures including cultural advocates and advisory boards, all of which are essential to developing, implementing, and identifying a CRT. Finally, the emerging findings from RE can be instantaneously revealed and implemented during the study, which further meets the immediate needs of the participants. We contend that RE can have a legitimate, research-oriented place in family therapy as a means of learning more about the unique clinical needs of minority populations. This process and outcome offers the field of MFT with a useful means to move MFT forward in our efforts to offer culturally responsive research and intervention.

REFERENCES

Abma, T. A. (2005). Responsive evaluation: Its meaning and special contribution to health promotion. Evaluation

and Program Planning, 28, 279–289.

Bean, R. A., Perry, B. J., & Bedell, T. M. (2001). Developing culturally competent marriage and family thera-

pists: Guidelines for working with Hispanic families. Journal of Marital and Family Therapy, 27(1), 43–54.

Bermúdez, J. M., Zak-Hunter, L., & Silva, L. (2011). Experiences of parenting classes for mexican american

mothers parenting alone: Offering culturally responsive approaches. American Journal of Family Therapy, 39,

360–374.

Bernal, G., Bonilla, J., & Bellido, C. (1995). Ecological validity and cultural sensitivity for outcome research:

Issues for the cultural adaptation and development of psychosocial treatments with Hispanics. Journal of

Abnormal Child Psychology, 23, 67–82.

Braverman, L. (1988). Feminism and family therapy: Friends or foes. In L. Braverman (Ed.), A Guide to feminist

family therapy (pp. 5–9). New York, NY: Harlington Park Press.

Carlson, T. S., Erickson, M. J., McGeorge, C. R., & Bermudez, J. M. (2004). Just therapy with Latino Families in

the Midwestern United States. Poster session presented at the Narrative Therapy and Community Work

Conference, Oaxaca, Mexico.

January 2013 JOURNAL OF MARITAL AND FAMILY THERAPY 41

Castro, F. G., Barrera, M., & Martinez, C. R. (2004). The cultural adaptation of prevention interventions:

Resolving tensions between fidelity and fit. Prevention Science, 5, 41–45.

Chaney, S. E., & Piercy, F. P. (1988). A feminist family therapy behavior checklist. The American Journal of

Family Therapy, 16(4), 305–318.

Creswell, J. W. (2007). Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks,

CA: SAGE Publications.

Crotty, M. (1998). The foundations of social research: Meaning and perspective in the research process. Thousand

Oaks, CA: SAGE Publications.

Denzin, N. K., & Lincoln, Y. S. (Eds.) (1994). Handbook of qualitative research. Thousand Oaks, CA: SAGE.

Domenech-Rodrı́guez, M., & Wieling, E. (2005). Developing culturally appropriate, evidence based treatments for

interventions with ethnic minority populations. In M. Rastogi & E. Wieling (Eds.), Voices of color: First-

person accounts of ethnic minority therapists (pp. 313–334). Thousand Oaks, CA: Sage Publications.

Fay, B. (1987). Critical social science: Liberation and its limits. Ithaca, NY: Cornell University Press.

Haddock, S. A., Zimmerman, T. S., & MacPhee, D. (2000). The power equality guide: Attending to gender in

family therapy. Journal of Marital and Family Therapy, 26(2), 153–170.

Lu, Y. E., Organista, K. C., Manzo, S. Jr, Wong, L., & Phung, J. (2001). Exploring dimensions of culturally sen-

sitive clinical styles with Latinos. Journal of Ethnic and Cultural Diversity in Social Work, 10(2), 45–66.

Matheson, J. L., Benson, K., & Walsh, S. R. (2004). Flattening the hierarchy: The co-creation of a feminist

informed family therapy course in an MFT doctoral program. Journal of Feminist Family Therapy, 15,

45–74.

McDowell, T., & Fang, S.-R. S. (2007). Feminist-informed critical multiculturalism. Journal of Family Issues,

28(4), 559–556.

McDowell, T., & Hernández, P. (2010). Decolonizing academia: Intersectionality, participation, and accountabil-

ity in family therapy and counseling. Journal of Feminist Family Therapy, 22, 93–111.

Mock, M. R. (2008). Visioning social justice: Narratives of diversity, social location and personal compassion. In

M. McGoldrick & K. V. Hardy (Ed.), Re-Visioning family therapy: Race, culture, and gender in clinical prac-

tice (2nd ed., pp. 425–441). New York, NY: The Guilford Press.

Parra-Cardona, J. R., Holtrop, K., Cordova, D., Escobar-Chew, A. R., Horsford, S., Tams, L., et al. (2009).

‘‘Queremos aprender’’: Latino immigrants’ call to integrate cultural adaptation with best practice knowledge

in a parenting intervention. Family Process, 48(2), 211–231.

Patton, M. Q. (2002). Qualitative research & evaluation methods (3rd ed.). Thousand Oaks: Sage.

Seponski, D. M., & Lewis, D. C. (2010). My grandmother and me: International service-learning in Cambodia with

children infected and affected by HIV ⁄ AIDS. Information for Action: Journal on Service-Learning Research with Children and Youth, 3(2). Retrieved November 14, 2011, from http://www.service-learningpartnership.org/

site/DocServer/IFA-CambodianYouth.Vol3No2.pdf?docID=4204.

Silverstein, L. B., & Goodrich, T. J. (Eds.) (2003). Feminist family therapy: Empowerment in social context.

Washington, DC: American Psychological Association.

Small, S. A., & Uttal, L. (2005). Action-oriented research: Strategies for engaged scholarship. Journal of Marriage

and Family, 67, 936–948. doi: 10.1111/j.1741-3737.2005.00185.x.

Stake, R. E. (1975). To evaluate an arts program. In R. E. Stake (Ed.) Evaluation the arts in education: A respon-

sive approach (pp. 12–31). Columbus, OH: Merrill.

Stake, R. E. (2004). Standards-based & responsive evaluation. Thousand Oaks, CA: SAGE Publications.

Stake, R. E. (2005). Qualitative case studies. In N. K. Denzin & Y. S. Lincoln (Eds.) The SAGE handbook of

qualitative research (3rd ed.). Thousand Oaks, GA: SAGE Publications.

The Family Center. (2009). Family therapy: Just therapy. Retrieved December 23, 2009, from http://www.

familycentre.org.nz/Areas_of_Work/Family_Therapy/index.html?PHPSESSID=4674b0e9.

Turner, W. L., Wieling, E., & Allen, W. D. (2004). Developing culturally effective family-based research pro-

grams: Implications for family therapists. Journal of Marital and Family Therapy, 30(3), 257–270.

Waites, C., Macgowan, M. J., Pennell, J., Carlton-LaNey, I., & Weil, M. (2004). Increasing the cultural respon-

siveness of family group conferencing. Social Work, 49(2), 291–300.

Weaver, H. N., & Wodarksi, J. S. (1995). Cultural issues in crisis intervention: Guidelines for culturally compe-

tent practice. Family Therapy, 22, 213–223.

Wetchler, J. L. (1996). Social constructionist family therapies. In F. P. Piercy, D. H. Sprenkle & J. L Wetchler &

Associates (Eds.), Family therapy sourcebook (2nd ed., pp. 129–152). New York, NY: The Guildford Press.

Wheeler, D., Avis, J. M., Miller, L., & Chaney, S. (1988). Rethinking family therapy education and supervision:

A feminist model. In M. McGoldrick, C. Anderson & F. Walsh (Eds.), Women in families: A framework for

family therapy (pp. 53–71). New York, NY: Norton.

42 JOURNAL OF MARITAL AND FAMILY THERAPY January 2013

Copyright of Journal of Marital & Family Therapy is the property of Wiley-Blackwell and its content may not

be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written

permission. However, users may print, download, or email articles for individual use.