Week 3 discussion: Phenomenology Research Design;Observational Research

335

http://dx.doi.org/10.1037/14773-012 APA Handbook of Clinical Psychology: Vol. 2. Theory and Research, J. C. Norcross, G. R. VandenBos, and D. K. Freedheim (Editors-in-Chief) Copyright © 2016 by the American Psychological Association. All rights reserved.

C h a P t e r 1 2

QualITaTIVE mEThods Heidi M. Levitt

Over the past few decades, qualitative research methods have become recognized as essential approaches within clinical science. Not only are these approaches flourishing within research enclaves, but psychology as a field is embracing methodological pluralism and benefiting from it. This expansion of vision allows researchers to investigate new sets of problems and to reconsider questions about the goals of research writ large. It is an exciting time.

In this chapter, I describe what qualitative methods are and introduce readers to some com- mon methodological approaches within this rubric. As well, I outline some core principles of the design process and consider central limitations of these approaches. Landmark studies and key accomplish- ments in the area are described as well as future directions. Although many of the examples empha- size clinical research, the chapter is applicable to multiple areas of investigation throughout psychol- ogy because these methods are widely used.

DESCRIPTION AND DEFINITION

The term qualitative methods has arisen in the disci- plinary vernacular to signify the analysis of language in contrast to the reduction of experience to num- bers, which is seen as the domain of quantitative research. This term has been considered a misno- mer, however, for several reasons (e.g., Shadish, 1995). For instance, experiments can use qualitative

variables and subject them to statistical analyses. Also, qualitative researchers may incorporate quan- titative information to support their findings, such as when reporting the frequency of a given response within their databases or interrater reliability coef- ficients for coding. Partly because of this confusion, the term human science is sometimes preferred. This term ties to the foundational work of philosophers and forerunners of psychology (such as Dilthey, Mill, James, Vico, and Wundt; see Danziger, 1990; Rennie, 1995) and ties the distinction between these sets of methods to the problem of studying subjectivity and who arrived at the conclusion that a separate human science should be developed inde- pendently of natural science, which had the study of material objects as its domain.

Many qualitative methods have been developed to study subjective and interpersonal experiences, which has been referred to as the double hermeneutic, in which investigators are interpreting and under- standing subjects and their process of interpreting and understanding (Giddens, 1987). The distinctive rhetoric of qualitative research has evolved to address these concerns and to strengthen research within a subjectivist framework (Rennie, 1995). For instance, qualitative approaches may take into account the researchers’ subjectivity by demonstrating how the methods have maximized the researchers’ attun- ement to the experiences under study by collecting rich descriptions or having lived experiences that encourage a depth of understanding. In contrast to

I thank Cara Herbitter for her helpful comments on the manuscript of this chapter.

APA Handbook of Clinical Psychology: Theory and Research, edited by J. C. Norcross, G. R. VandenBos, D. K. Freedheim, and B. O. Olatunji Copyright © 2016 American Psychological Association. All rights reserved.

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

Heidi M. Levitt

336

quantitative methods, researchers are encouraged to explicitly acknowledge the role of their precon- ceptions in the scientific endeavor. Reflexive self- disclosure contributes to the sense of objectivity by laying out the perspectives of investigators and the ways these were managed or used within the analy- sis. These approaches also guide researchers to con- sider the effects of participants’ subjectivity, such as their ability to withhold or misrepresent experiences, and the influence of investigators’ expectations on participants’ responses. Unfortunately, this subjectiv- ist rhetorical approach differs from traditional psy- chology, and this unfamiliarity may have delayed the acceptance of qualitative methods.

In contrast, natural science has evolved from an objectivist rhetoric in which researchers persuade their audience of the rigor of their work by describ- ing human experience in behavioral terms, by quan- tifying experience, and by keeping the researcher’s personal experiences separate from the analysis (e.g., Danziger, 1990). These differences require researchers to develop new fluencies when evaluat- ing from a novel rhetorical position and recogniz- ing credibility in different forms. Understanding qualitative methods, then, is not merely a process of learning procedures or methods but a process of understanding the implications of objectivist and subjectivist frameworks in conceptualizing research.

A review of qualitative methods can become thorny because qualitative researchers vary in terms of epis- temologies, goals, and procedures—even when using the same methods (e.g., Gergen, 2014). For instance, although some methods work to set aside investiga- tors’ preconceptions (e.g., Giorgi, 2009), others meth- ods use researchers’ theories to structure analyses (e.g., Fine, 2013). Although some methods seek to develop theories (e.g., grounded theory; Glaser & Strauss, 1967), others seek to create social change (e.g., par- ticipatory action research; Fine, 2013). To further complicate things, rigorous methods are adapted to fit the characteristics of the individual study at hand—a process that I describe in the following sections.

CORE PRINCIPLES AND APPLICATIONS

Qualitative methods have been applied in the study of countless topics. The number of methods and

variations on these methods is increasing with their expanding use. In this section, I present central principles in the design of qualitative methods and describe a few of the dominant qualitative methods.

Core Principles A variety of excellent guidelines are available for conducting and reviewing qualitative research (e.g., Elliott, Fischer, & Rennie, 1999; Morrow, 2005). These guidelines have tended to focus on principles within a human science approach, likely because a natural science or objectivist rhetoric is already the received view in psychology and its prac- tices to establish rigor are widely known (e.g., using statistics such as interrater reliability, describing findings in behavioral language, using third-person language, avoiding discussion of the investiga- tors’ beliefs). The four following criteria have emerged as common principles from across these recommendations.

Researchers should own their perspectives when presenting their research. Researchers are often motivated to conduct research by both personal and professional reasons. Researchers often select a topic to study because they have had important personal experiences with that topic (e.g., sadness), because they care about a topic (e.g., concerns in the lives of significant others), or because it relates to their pro- fessional goals (e.g., obtaining a grant, advancing a theory). In qualitative methods, research precon- ceptions are assumed, and so a transparent descrip- tion of researchers’ perspectives strengthens the presentation of the research, especially as it relates to the topic under investigation. For instance, in my research on lesbian, gay, bisexual, and transgender identities (e.g., Levitt & Ippolito, 2014), I describe myself as a femme-identified lesbian, whereas in my research on psychotherapy process (e.g., Levitt & Williams, 2010), I tend to describe myself in terms of my psychotherapy orientation and interest in psychotherapy integration. In both cases, I am aware that my identities and experiences influence the ways in which I frame questions, word findings, and identify dynamics both within and outside of my awareness. The depth of self-reflection conveyed in an article could depend on the method in use as

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

Qualitative Methods

337

well. For instance, an autoethnography would entail much more detail on how different aspects of iden- tity influence the meaning-making process than a grounded theory analysis.

Also, the ways in which these preconceptions are managed differ in relation to the research goals at hand. Typically, researchers’ perspectives are dealt with in two ways: Some approaches encour- age researchers to become aware of their beliefs so they can set them aside when considering their data (e.g., Giorgi, 2009) or, even if they do not believe they can be entirely set aside, so they can find ways to limit their influence on their analysis (e.g., Rennie, 1995). Researchers might apply this principle by considering how to ask questions in an open-ended, nondirective manner or by actively interrogating their preconceptions as they engage with their data.

Other methods, however, use researchers’ perspectives as a lens for the analysis, such as in criti- cal, feminist, or multicultural methods that might seek to cast light on systems of power and oppres- sion (e.g., Fine, 2013; Ponterotto, 2013). Using these methods, researchers would make clear the nature of the theory being used and the corresponding study goals in the presentation of the research. In either case, transparency in researchers’ goals bolsters the trustworthiness of the study. The disclosure can demonstrate that the researchers are consciously approaching the analysis in a way that is coherent with the procedures at play, whether they are aiming to use the perspective as an analytic tool or to reduce the impact of their perspectives on the research process.

Research contributions should be meaningful in light of their purpose. Research findings should make a contribution to the problem the research is trying to solve. Often this contribution is assessed in the light of literature that came before it. Because qualitative research is often recommended as a way to begin a new line of inquiry, it is not uncom- mon that a study at hand is the first one of its kind. Indeed, qualitative research is often associated with feminist and multicultural research because of its ability to give voice to experiences that have not been represented (or have been misrepresented) in the psychological literature (Ponterotto, 2013).

Instead, the meaningfulness of research contribu- tions should be assessed in light of the researchers’ goals, so clarity about those goals is essential.

Qualitative research in the context of mixed- methods studies often has as its goals the develop- ment of theory for further evaluation or explanation of unintelligible findings from a prior study. In social justice research, goals for research often include the amelioration of social or institutional problems, the empowerment of the disenfranchised, or the resolution of local problems within a commu- nity. In many forms of clinical research, the goals are to shed light on a problem so that health and mental health care providers develop a greater sensitivity to the dynamics at play. In any case, the meaningful- ness of a contribution should be judged pragmati- cally, in relation to the research goals at hand.

Across these goals, human science research often contributes an understanding of what cannot be seen—interjecting into the psychological literature either the subjective internal process or the cultural process framing interpersonal interaction. To exem- plify the process of revealing concealed experience, I provide an example from one of my research proj- ects on pauses in psychotherapy (Levitt, 2001).

In the session, the client was describing feeling depressed and the psychologist–therapist had been asking her to consider what she could say to herself to help herself feel better. The client struggled to do this, and so the therapist attempted another tack.

T: What would a friend tell you? (8-second pause) A close friend.

C: Somehow that doesn’t help me. T: Okay. What would you like to hear from a

close friend, or what would you tell a close friend—sorry. (4-second pause) Who was in your shoes?

C: (34-second pause) I suppose I’d tell a person in my shoes to have enough confidence in their proven ability to set limits. (16-second pause)

T: Is that evidence against [the depressive thought]?

In the session transcript, it appears that the client is trying to engage with the questions being posed by the therapist and is struggling but

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

Heidi M. Levitt

338

finally succeeds in developing an answer to resist her depression. In the interview about her experi- ence, it becomes clear that the client’s subjective experience is markedly different. While describing the first pauses in this segment of therapy, the cli- ent said,

The idea is—what would a friend tell me—if you can’t think of what you would tell yourself—but they’ve said all the wrong things for a long time now. . . . So [I was] remembering that they hadn’t been terribly helpful. [When considering the next pauses,] now I’m in a position where I have to tell them something that will be quite positive. . . . I don’t have anything left for them [my friends] is what I was thinking. And then knowing this is a theoretical situation and actu- ally the friend I am talking to is myself and I probably don’t have that much love for myself either so (right) so then I’m stuck. . . . I know how I talk to myself, so if I say this, it feels like a lie. Because it’s not, it’s not what I tell myself. It’s what I should be telling myself, if I were my friend—[client laughs] if my friend was [supportive]. So it got very convo- luted. . . . I wasn’t forgiving my friends for the past year of letting me down and I wasn’t letting myself off the hook either . . . in order to do the leap [to com- plete the task].

When asked what she was experiencing dur- ing the final pause, the client said, “Oh, disbelief in what I just said.” She later described the pres- sure to come up with the right answer, comparing her experience of therapy to being a contestant on the television show Jeopardy. This client’s inter- view communicated concerns that might otherwise remain invisible and that have been found across psychotherapy orientations. Also, it emphasizes the role of client agency within psychotherapy as she contributes to the work of maintaining the alliance by engaging in therapy in ways that are unexpected and unobservable by her therapist (see Bohart, 2000).

Knowing that lengthy pauses of this sort might cloak internal strife, therapists might then ask clients in a similar situation to describe what is happening for them during these moments. This research generated a typology for coding silences in therapy as well as discursive cues that can be used by therapists or researchers to distinguish productive and obstructive silences (e.g., Frankel et al., 2006). In this way, revealing new understand- ings can benefit both science and clinical practice. Clarity about the goal at hand is key in any process of assessing qualitative methods because there are many legitimate research aims.

Findings are grounded in data. In qualitative research, there are multiple ways to demonstrate that findings have been developed to represent com- monalities found in the data. Researchers most often do this by describing their process of analysis and by exemplifying that process with quotations from the data and detailed descriptions of how findings were derived. Other ways of demonstrating that data are grounded entail providing descriptions of the basis for a finding in a social or interpersonal system (e.g., Fine, 2013).

The quest to ground findings in data also requires researchers to consider their recruitment method and to consider how to find participants who can generate strong and comprehensive descriptions of the phenomena under study (Morrow, 2005). Qualitative researchers are often interested in seeking a breadth of experiences within the confines of their question so that the findings can be as encompassing of differences as possible. For instance, in research on transgender identities (e.g., Levitt & Ippolito, 2014), I prioritized diver- sity within these identities (e.g., transmen, butch women, genderqueer people). Of course, many types of diversity can be relevant to a given question, but because qualitative studies often require a smaller number of participants, researchers tend to focus on the forms of difference that are most influential in relation to their topic of study.

Some of the ways that qualitative researchers seek to demonstrate groundedness are in keeping with their epistemological approaches. For instance, more objectivist approaches might value a large

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

Qualitative Methods

339

number of researchers coming to consensus about an interpretation or might value high interrater reliability coefficients. These processes can signify that the findings are not idiosyncratic interpreta- tions of the data and can bolster trustworthiness.

In contrast, more constructivist qualitative methods tend to value demonstrations that a single researcher or small number of researchers had a depth of engagement with the data. A sole inves- tigator has experience with all aspects of study design, literature review, interviewing, and analy- sis and so may have a depth of knowledge that is unattainable with a large team. Indeed, these researchers may share concerns that using consen- sus can weaken findings by forcing the analysts with the most intimate or penetrating knowledge of a dataset to shift their interpretation to accom- modate analysts who do not share the same level of attunement (Rennie, 2000).

As another alternative, consensus can be structured along the lines of epistemic privilege (Levitt, 2015a). For instance, in a group study, it might be that the interpretation of those research- ers who conducted an interview is given priority because of their experience in interacting with the participant and observing intonation and nonverbal cues, whereas methodological privilege is credited to the investigator who has the most experience with a given procedure. Processes of consensus can be developed that recognize the differing forms of methodological and interpretive expertise within a set of investigators, are sensitive to power dif- ferences, and are open to incorporating multiple perspectives on a dataset. Still, consensus is only one approach to demonstrating that findings are grounded in data, and there can be different ratio- nales for groundedness, as described previously.

Using checks to supplement the credibility of a central analytic process. Qualitative researchers often seek support for their findings beyond their analytic process (Elliott et al., 1999). Although this process may not be necessary, it can bolster the trustworthiness of the findings. This support can take many forms, including triangulation with other sources of data (e.g., psychotherapy outcome data) or forms of analysis (e.g., in a multimethod study).

It might include procedures such as conducting checks by asking the participants to provide feed- back on the findings to the investigators or inviting external auditors to review them and provide feed- back (e.g., Hill, 2012). Although these checks on the analysis can be helpful, they should be conceived as supplemental to the central analysis.

In working to articulate the logic underlying qualitative methods, David Rennie (2000, 2012) modified Peirce’s (1965) theory of inference and proposed an understanding of the inferential pro- cesses that can be used to understand the analytic process that is central to many qualitative methods. The cycle Rennie described, methodical hermeneu- tics, was framed within the hermeneutic circle—a process of developing understanding in which researchers bring to bear evidence from their experi- ence of a whole phenomenon on their understand- ing of its parts, and vice versa. The understandings derived from the study of parts and the whole come to inform each other, regardless of where an investigator begins. This meaning-making process occurs through a cycling of eduction (drawing forth intelligible understanding from inchoate experi- ence of the phenomenon), abduction (developing a tentative understanding), theorematic deduc- tion (deciding that the understanding should be evaluated via comparisons with further data), and induction (searching within other data to identify patterns). The process of induction might support the prior abduction or not. If not, it will lead to a new eduction, and the cycle begins again. In this way, the process is self-correcting, and the cycling of these logical moments makes qualitative research sufficient unto itself.

Still, even if not a requirement, checks can prove helpful by providing additional information that can be used by investigators in the cycle of meaning- making. Typically, new understandings from checks are best viewed as one source of information in relation to the larger primary analysis rather than as a final verdict on the value of an analysis. For instance, in a study applying a feminist critical anal- ysis to a set of data, an auditor or participant who has a different interpretive lens might arrive at dif- ferent conclusions, which might not cast a shadow on the original findings but provide an interesting

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

Heidi M. Levitt

340

counterpoint to consider. The researcher should consider the sources of checks, their experiences with the phenomenon, and the way the information from that source relates to the larger analysis (which individual participants often cannot access) when deciding if or how checks should be used to inform the findings. The evaluation of the method should then be focused on the logic within the analysis itself, any checks used, and their relation to the goals of the research project.

Cautions About Principles It can be helpful to articulate common principles underlying the design of qualitative studies because they can aid researchers and reviewers in under- standing the logic of qualitative methods. At the same time, there are concerns that the field is coalescing too rapidly on sets of guidelines that are misdirecting researchers and reviewers (Gergen, 2014). This misdirection may be a particular con- cern when guidelines direct researchers to a spe- cific set of procedures, given the many different procedural approaches that have been developed both between and within methods. Instead, many experts in qualitative methods have maintained that qualitative research is best approached as the development of an analytic solution to a spe- cific local problem (e.g., Guba & Lincoln, 2005). Instead of assessing studies by their adherence to a set of procedures, these experts have advocated that sophisticated researchers should adapt proce- dures creatively to better permit an understanding of the issue at hand.

Indeed, there is concern that that psychology can be seen as “methodolatrous”—idolizing sets of methods and adapting questions to fit them instead of developing methods to answer specific questions (Bakan, 1967). The fear is that this tendency may become the state of affairs in qualitative research as novice investigators grapple with learning these new research tools. Instead, researchers and reviewers should consider their participants’ characteristics, the investigators’ framework of understanding, the characteristics of the data at hand, and the goals and epistemology of the study and how these all inter- act with the logic of the analysis to best answer the questions at hand (Levitt, 2015a).

Applications There are far too many qualitative methods to address in a single chapter, but a description of a few methods will illustrate some of the variety in their processes and goals. The following descrip- tions are not intended to be comprehensive but to highlight five qualitative methodologies.

Phenomenological methods. The phenomenologi- cal tradition has enjoyed a longer history of research than many other modern psychological qualita- tive methods, mainly because of its promotion by researchers at Duquesne University since the late 1960s (Churchill & Wertz, 2001). This approach to qualitative methods has at its foundation the work of Edmund Husserl (1900/1970) and his interest in studying conscious experience and identifying the building blocks of subjectivity. From the basis of this approach, Amedeo Giorgi (2009) developed descrip- tive phenomenology and laid out a set of procedures to guide this process of inquiry. The method guides researchers to adopt a transcendental attitude of inquiry in which they set aside their own precon- ceptions, using tools such as bracketing, so that they can empathically reflect on their participants’ experiences without distortion. By cycling between their understanding of the participants’ experience and the data at hand, phenomenological research- ers describe essential aspects of the subject matter and the development of new insight into that lived experience.

Other approaches to phenomenological meth- ods include interpretive phenomenological analy- sis (e.g., Smith, Flowers, & Larkin, 2009), which encourages researchers to use their preconceptions deliberately (rather than to set them aside) and to engage with participants in codeveloping interpre- tations of their process of making-meaning. Both approaches have gained in popularity and are being used internationally.

Grounded theory methods. As with phenomenol- ogy, there are multiple versions of grounded theory that are based in different epistemologies and pro- cedures (e.g., Bryant & Charmaz, 2010; Rennie, 2012), and even the originators of the method (e.g., Glaser & Strauss, 1967) went on to argue for separate approaches. Grounded theory method

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

Qualitative Methods

341

approaches share a number of distinguishing fac- tors, however. Specifically, they share the analytic process of comparing each unit of data with every other unit and then grouping units of data into cat- egories on the basis of common meanings therein that are used to create labels for each category. The labeled categories are compared with one another in turn, and higher order categories are developed and labeled to reflect their shared meanings. This process continues until a data hierarchy is formed that has at its apex a core category, which is the cen- tral finding of the analysis. Other defining features of grounded theory include data collection being guided by theoretical sampling methods—in which investigators seek out participants strategically so as to flesh out their understanding within their ongo- ing analysis—and stopping at the point of satura- tion, which is when incoming data no longer seem to bring new understandings to the analysis. These procedures have gained acceptance in the field and are now routinely incorporated into other qualita- tive methods as well.

Narrative methods. The focus on narrative came into psychology as cognitive psychologists began to conceptualize the brain as encoding information in narrative rather than digital form (Bruner, 1990). These methods have been embraced in clinical psychology, and researchers have developed a wide range of coding systems and qualitative methods (Angus & McLeod, 2004). The narrative approach has expanded to include methodological variations that study the content of narratives, their function, their structural features, their evolution over time, and the process of producing narratives (e.g., Labov, 2006; Polkinghorne, 1988). Often, narrative is viewed as a rhetorical device through which people represent themselves to both themselves and others. Narratives can be studied in isolation, by study- ing their effect on listeners, and by studying their meaning within their historical and cultural context. Entire life stories can be examined, or studies can focus on stories told within or about people’s lives or stories related to specific topics or concerns.

Critical methods. In these approaches, research- ers begin their analyses with an interest in analyzing how interpersonal and sociopolitical structures and

processes function to support the privilege of domi- nant classes in society (e.g., gender, race, ethnicity, sexual orientation, immigration status). Researchers are interested in shedding light on otherwise invis- ible forces of oppression and injustice that have become status quo (e.g., Fine, 2013). They also critique the research process and the ways in which the discipline engages in acts that might perpetu- ate discriminatory practices and perspectives (e.g., Wendt & Gone, 2012). These approaches often take the form of interpretive lenses through which an analysis that incorporates other methods can be con- ducted (e.g., Chang & Yoon, 2011). They are often separated as a category of methods, however, because of their distinct goals, which center on social justice, institutional change, and empowerment in addition to the generation of new knowledge.

Discursive methods. Discourse analysis can be seen as a rubric that encompasses approaches with different epistemological foundations and practices (e.g., Riley, Sims-Schouten, & Willig, 2007), such as conversational analysis and critical discourse analysis. Unlike the previous methods, many of these approaches are interested in understanding the social function of discourse—without the goal of uncovering the internal experience underlying its production (Potter, Edwards, & Wetherell, 1993). Qualitative approaches to discourse analysis tend to incorporate a lens through which to explore how cultural and ideological assumptions make discourse intelligible. Critical discourse researchers engage in a close analysis of the words and phrases used in communication and their rhetorical impact. For instance, they often begin from feminist and mul- ticultural frameworks and, rather than seeking to set these perspectives aside, use them to shed light on oppressive or deceptive social practices and to further social justice aims. These approaches differ from quantitative approaches to discourse analysis that may use statistics to assess discursive features.

Even from a cursory examination of a few qualitative methods, the differences both across and within each method are striking. Within the same method, researchers have adopted different epistemologies, procedures, and goals. Given this variation, the application and evaluation of these

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

Heidi M. Levitt

342

methods requires researchers to deliberately select and adapt methods in such a way so as to best meet the characteristics and objectives of their study.

LIMITATIONS OF QUALITATIVE METHODS

Qualitative methods tend to focus on the content and context of subjective and interpersonal experi- ence. They do not answer questions about prob- ability or generate statistical information that can be used for the prediction of events within populations (Gergen, 2014). They are appropriate for developing understandings or explanations of how a phenom- enon is experienced as well as for studying forms of expression.

Assessing the limitations of qualitative methods can be challenging, in part because there are so many different methods, but also because limitations are necessarily understood in reference to research goals, and the goals of these methods can differ from those of quantitative research and from one another. The understanding of generalization within human science research is a case in point. If judged from a natural science perspective, generalization in quali- tative methods appears limited because these meth- ods often require a small number of participants and so random sampling cannot be used. Instead of having as goals prediction through the assessment of probability, however, these methods seek to develop comprehensive understandings and explanations of subjective experiences, social processes, and expres- sive communications. Intrinsic to many qualitative methods are alternative procedures that enhance generalization, such as using theoretical sampling (Glaser & Strauss, 1967) and deliberately selecting typical or variant instantiations of a phenomenon to study (e.g., Moustakas, 1994).

The process of generalizing qualitative research findings is located within readers (vs. the data), who will generalize those findings that seem to resonate and promise to shed light on their topic of interest and adjust them in relation to the patterns described in the findings. Rather than producing laws or probabilities, qualitative findings often present contextualized findings in such a way as to show how findings might be expected to change across situations. By grounding the findings in the data,

describing the sample characteristics, and provid- ing contextualized examples, readers are provided with information that permits them to assess the appropriateness of generalization.

There has also been considerable debate on whether qualitative methods can address causality. Most quantitative researchers believe that estab- lishment of causal statements requires controlled experiments. Some writers (e.g., Tacq, 2011), however, have argued that there is no difference in principle between causality in quantitative and qualitative research. A randomized experiment can be likened to methodical hermeneutics, and a multiple case study method that focuses on indi- vidual causal sequences can contribute information on causality. In short, because of the wide range of qualitative methods, their limitations should be assessed with reference to the goals underpinning the method in use.

LANDMARK STUDIES

Qualitative methods have been used in different areas of clinical psychology. In his intriguing his- torical account of the development of the field of qualitative inquiry, Wertz (2014) described forma- tive qualitative contributions to psychology by Wil- helm Wundt, Sigmund Freud, and William James and later contributions by influential figures such as Flanagan, Maslow, and Kohlberg, among many oth- ers. The following are a few of the many contribu- tions to the field that have broken new ground and had an important influence in psychology.

Maslow’s Self-Actualization Research Abraham Maslow began studying self-actualization as a way to understand his admiration of two of his mentors, Ruth Benedict and Max Wertheimer. He began reading biographies of and interviewing people who seemed to share similar qualities with these inspirational figures. His analysis of these data (Maslow, 1954) was extensive and organized in a hierarchical structure (similar to grounded theory; Rennie, 2007). It seems that Maslow, who conducted this work in an academic context that was strongly influenced by behavioral approaches to research, thought of his qualitative work as

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

Qualitative Methods

343

the prescience activity that led to his developing a questionnaire to assess self-actualization and conduct scientific research (Rennie, 2007; Wertz, 2014). Although he submitted this work for publi- cation and presented it in his presidential address to the American Psychological Association, it was rejected by a psychological journal, and he only published it in book format afterwards. Although he seemed ambivalent about publicly advocating for the place of qualitative methods, he continued this form of research in his work on peak experiences (Maslow, 1968).

Flanagan’s Critical Incident Research Not shy about advocating for the scientific value of his qualitative method, even in a behaviorist era, James Flanagan (1954) developed the critical incident technique (Wertz, 2014). This method was initially developed to identify successful pilot trainees in World War II and proved its effectiveness over his earlier survey research, because instructors had difficulty describing the reasons for the pilots’ failures. It was the largest psychological study of any kind to date, involving more than 150 psycholo- gists and 1,400 research assistants. Flanagan asked trainers to describe critical incidents in which pilots demonstrated excellence or failure to identify the most important characteristics. By identifying com- mon patterns in the descriptions, he both improved the pilot selection process and made suggestions to improve the design of the planes they flew. This method has gone on to be used in many areas of psychology, including job analyses in industrial and organizational psychology and the identification of best teaching practices within educational psychol- ogy and, through the analysis of commonalities across more than 1,000 ethical and unethical events reported by psychologists, was the foundation of the development of the ethics code developed by the American Psychological Association that guides clinical practice.

Rennie’s Study of Clients’ Experience Qualitative methods have been well received in the area of psychotherapy research, in part because of David Rennie’s research using and promoting qualitative methods (e.g., Rennie, 2012; Rennie,

Phillips, & Quartaro, 1988). His most influen- tial grounded theory study (i.e., Rennie, 1994) examined clients’ experience of the psychotherapy hour (and included some of his own clients) and was divided into a number of publications. This work uncovered aspects of clients’ internal experi- ence that typically remained hidden to therapists, including clients’ agentic decisions to be defer- ent to their therapists, to use storytelling to serve sometimes covert aims, and to exert influence on the session to protect themselves and maintain the therapeutic alliance. In his analysis, the goal of psy- chotherapy was to guide clients to engage in radical self-reflexivity so as to become aware of the effects of their self-awareness. His research as well as his articulation of epistemological bases for qualitative research (e.g., Rennie, 2012) inspired and sup- ported a generation of qualitative psychotherapy researchers (see Levitt et al., 2015 for a description of his legacy).

Fine’s Participatory Action Research Participatory methods can include both qualita- tive and quantitative methods, and their hallmark is the involvement of participants as research col- laborators and the study of participants from across multiple perspectives within the systems in ques- tion. The study question and design are shaped with participants toward goals that will inform, empower, and shape their community with aims tied to social justice concerns. Kurt Lewin (1946) and Morton Deutsch (Deutsch, Coleman, & Mar- cus, 2006)—the founding fathers of action research and conflict resolution, respectively—are credited with developing a historical foundation for this method. The Changing Minds Project studied the impact of establishing a college program in a maximum-security prison (Fine, 2013). In this project, participants included faculty, university presidents, the prison administrators, and a set of female prisoners as coresearchers. Although their quantitative analyses demonstrated that college programs reduce recidivism and save taxpayers money, the qualitative analyses showed how these programs also made the prison feel safer and more manageable and led to fewer disciplinary incidents transforming women’s lives on their release from

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

Heidi M. Levitt

344

prison. The work has had long- standing effects on prison cultures and inspired research and social policy in many other areas, including education and urban youths. Bringing diverse sets of investiga- tors together to share their expertise and share in the research process generated a different vision of research. Changing Minds is a landmark project because of the interest it stimulated in participa- tory and critical approaches to qualitative methods within psychology.

Greenberg’s Task Analysis Task-analytic research, developed by Leslie Green- berg (2007) and his colleagues, is a mixed-method approach to exploring how client processes within therapeutic tasks unfold (e.g., a therapeutic exercise, the resolution of a problem) and so has special rel- evance to clinical psychology. It is based on analyses of in-session patterns of client–therapist interactions and develops contrasting models of both good and poor outcome processes toward task completion so that therapists can learn optimal pathways to complete a task, the different acceptable routes, and potential missteps to avoid.

An innovation to this method was added in a study on the resolution of global distress (Pascual- Leone & Greenberg, 2007). Clients were found to uncover underlying experiences of either shame and fear or anger while en route, using different pathways, toward the eventual stage of assertion and self-soothing. In this study, qualitative model devel- opment was enhanced by the addition of hypothesis testing of sequential analyses to successfully evalu- ate whether the model’s stages proceeded in accor- dance with the qualitative predictions. Although the qualitative core of this method has been in use for decades, its further development as a mixed method has positioned it to gain more recognition.

KEY ACCOMPLISHMENTS

The most significant accomplishment in quali- tative methods has been their acceptance as an important methodological approach within psy- chology. Although these methods have long been in use, they tended not to be a visible part of the research endeavor because their application was not

formalized (Wertz, 2014). The articulation of the procedures in different qualitative methods has been crucial in this process. Now that these methods have become part of mainstream clinical psychology, the field is in a position to embrace increasingly com- plex qualitative designs.

Integration of Qualitative Research in Psychological Research Over the past 3 decades, the use of qualitative methods in psychology has increased many times over. In the 1980s, qualitative methods were unheard of by most clinical psychologists, with the notable exception of the phenomenological psy- chologists at Duquesne University (Churchill & Wertz, 2001). Each decade since, qualitative meth- ods have appeared to be growing rapidly in pub- lished research (Ponterotto, 2013), although they still make up a minority approach.

Teaching qualitative methods is becoming incor- porated into graduate research training, leading to higher proportions of qualitative dissertations. It appears that British clinical psychology programs are incorporating teaching of qualitative methods at a greater speed than North American programs (Harper, 2012). Nonetheless, the development of interest in funding multimethod projects by the National Institutes of Health (see http://sigs.nih.gov/ cultural/Pages/default.aspx) has been increasing U.S. national interest in these approaches as well.

The interest in forming professional associations and in publishing qualitative research is growing rapidly. A number of journals of relevance to clini- cal psychologists—Qualitative Research in Psychol- ogy, Qualitative Health Research, and Qualitative Inquiry—have launched of late. The American Psychological Association’s Division 5 (Quantitative and Qualitative Methods) recently formed a qualita- tive section, the Society for Qualitative Inquiry in Psychology, which founded Qualitative Psychology. These venues can be helpful outlets for qualitative research. Although the field is adjusting to qualita- tive methods, researchers still face challenges when seeking to publish in nonqualitative journals, such as the need to adapt their research to the journal page length requirements set for quantitative articles (Fassinger, 2005). Further signifying the support for

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

Qualitative Methods

345

qualitative methods, Division 5 recently has changed its name to Quantitative and Qualitative Methods from Evaluation, Measurement, and Statistics. These many changes suggest that qualitative methods have shifted from the marginality of a few decades past to a position firmly within the fold of psychological research.

Methodological Innovations New qualitative methods are emerging constantly as researchers adapt existing methods to explore more topics. The first is a more recent innovation in the field, and the second is a long-standing approach to qualitative methods that appears to be increas- ing as these methods become part of the canon of psychological research.

Metasyntheses. Enough qualitative research is accruing in different areas of psychology so that researchers are beginning to conduct qualitative metasyntheses (e.g., Timulak, 2007; Yick, 2008). These analyses demonstrate how qualitative research findings can accumulate to indicate robust findings that are found repeatedly. They deflate the criticism that qualitative research literature does not accu- mulate in the ways of normal science (Kuhn, 1962). Either by categorizing and describing repeated find- ings (as in a theoretical review) or by reanalyzing past findings (as in a quantitative meta-analysis), they can identify core findings in a field of study. As qualitative methods extend into many areas of clini- cal psychology, these metasyntheses will continue to help the field yield gains from qualitative research.

Tailored multimethod approaches. As research- ers become more familiar with qualitative methods, they can combine either methods or aspects of methods to tailor approaches to specific research questions. For instance, researchers might use a par- ticipatory action approach to develop a project with a community and within that project use a grounded theory analysis to develop an understanding of a problematic experience or a hermeneutic analysis of the hierarchy to articulate principles to address a problem. An experimental design can be recruited to implement the principles and assess their impact, or a researcher might recruit processes such as saturation and theoretical sampling to strengthen

a narrative method of analysis. The availability of a wider spectrum of methods can help researchers to design programs of research that incorporate mul- tiple qualitative and quantitative methods to meet their research goals. This sort of research program might lead to a number of independent publications (because it would not be possible to publish these many different analyses together in one article) that can build on one another to produce a greater understanding of a problem at hand.

FUTURE DIRECTIONS

Whereas psychology in the 1980s was characterized by qualitative and quantitative debates, psycholo- gists today are embracing and integrating the unique contributions that each research methodology has to offer. Qualitative methods, both independently and as part of a mixed-methods research program, have burgeoned. Qualitative methods can explicate the subjective, interpersonal, and cultural experi- ences that lead to statistical findings. Also, they have the potential to bridge the clinician–researcher gap by providing contextualized findings with rich description that clinicians can adapt to the moment- by-moment process of working with their clients. Following are several future directions that may further their development in clinical psychology.

Journal Review Practices

Review criteria that support qualitative research. The process of peer review in psychol- ogy has been developed with quantitative research in mind. In particular, the increasingly restrictive page limits pose problems for qualitative studies, which are not infrequently published in book form. Because qualitative methods are unfamiliar to many reviewers and because there are so many different variations of these methods, they tend to require lengthy method sections to explain their design. Also, strong qualitative studies rest on results sec- tions in which researchers have the space to exem- plify their analyses by providing quotations and demonstrations. Appropriate responses made by some journals include extending maximum page- length criteria for qualitative analyses. In the future,

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

Heidi M. Levitt

346

as journals increasingly place materials online, edi- tors might make available the option of placing parts of the method section online and asking authors to use an abridged version after review. This change would allow researchers to provide more of the information that is needed for an adequate review of a qualitative article.

Reviews that examine methods in light of goals versus fixed procedures. In addition, there is a need for journals to reconsider the ways qualitative reviews are administered and conducted because many psychologists still have not been educated about these approaches. The review process would ideally involve reviewers who have demonstrated skills to think flexibly about method, often neces- sitating focused qualitative method expertise. More important, though, it will be important to recruit action editors who have extensive expertise in thinking through multiple qualitative methods and epistemological approaches (Levitt, 2015b). Instead of reviews being based on assessments of whether researchers have conformed to the set of fixed procedures, reviewers will increasingly be asked to consider whether authors coherently adapted methods to fit their question and the study goals. A task force of the Society for Qualitative Inquiry in Psychology, a section of American Psychological Association Division 5, is developing a position paper with recommendations for designing and reviewing qualitative research to aid in this process (Levitt et al., 2014).

Training Innovations

Required qualitative coursework. Qualitative methods are being recognized in clinical psychol- ogy as approaches that can lead to empirically based hypotheses, shed light on quantitative findings, and answer questions that can be distinct from those investigated by quantitative research methods. These are tools that all researchers need. As this recognition grows, coursework in qualitative methods and educa- tion on these topics has been increasing (Ponterotto, 2013). In the future, competent training as a clinical scientist will require focused coursework in qualita- tive methods. Programs will seek out methodologi- cal expertise in both quantitative and qualitative

methods to provide their students with the training to flexibly pursue their programs of research.

Required training on the effects of epistemological diversity. Although required courses in philosophy and epistemology are beyond the scope of the typical clinical psychology program, coursework in methods that incorporates reflections on epistemology will become increasingly important in forward-thinking clinical programs. Developing understandings of the ways different research goals and epistemologies shift the ways methods are designed and assessed will be important for new generations of clinical sci- entists. This understanding could begin in history of psychology courses in which students can learn how the field arrived at its dominant epistemological approach and how it was selected from the compet- ing models soon after the turn of the last century (Danziger, 1990). In clinical theories courses, stu- dents could consider how and why distinct research cultures, embedded within different epistemological stances, developed within the major theoretical ori- entations. In multicultural courses, they could learn about the varied approaches to knowing that are valued across different cultures. Through systematic education on topics such as these, students can come to appreciate the challenges in viewing one episte- mology through the values of another. These insights would allow them to consider the importance of fostering intellectual and epistemological diversity within a field of science as well as across other areas of clinical psychology (e.g., Heatherington et al., 2012). This basis could help develop a sophisticated approach to design that allows students to pragmati- cally approach a wider range of research problems and to integrate qualitative and quantitative methods.

References Angus, L. E., & McLeod, J. (2004). The handbook of

narrative and psychotherapy: Practice, theory, and research. Thousand Oaks, CA: Sage.

Bakan, D. (1967). On method: Toward a reconstruction of psychological investigation. San Francisco, CA: Jossey- Bass.

Bohart, A. C. (2000). The client is the most important common factor: Clients’ self-healing capacities and psychotherapy. Journal of Psychotherapy Integration, 10, 127–149. http://dx.doi.org/10.1023/ A:1009444132104

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

Qualitative Methods

347

Bruner, J. S. (1990). Acts of meaning. Cambridge, MA: Harvard University Press.

Bryant, A., & Charmaz, K. (Eds.). (2010). The Sage handbook of grounded theory. Thousand Oaks, CA: Sage.

Chang, D. F., & Yoon, P. (2011). Ethnic minority clients’ perceptions of the significance of race in cross-racial therapy relationships. Psychotherapy Research, 21, 567–582. http://dx.doi.org/10.1080/10503307.2011 .592549

Churchill, S. D., & Wertz, F. J. (2001). An introduction to phenomenological research in psychology: Historical, conceptual and methodological contributions. In K. J. Schneider, J. F. T. Bugental, & J. F. Pierson (Eds.), The handbook of humanistic psychology: Leading edges in theory, research, and practice (pp. 247–262). http://dx.doi.org/10.4135/ 9781412976268.n19

Danziger, K. (1990). Constructing the subject: Historical origins of psychological research. http://dx.doi. org/10.1017/CBO9780511524059

Deutsch, M., Coleman, P. T., & Marcus, E. C. (2006). The handbook of conflict resolution: Theory and practice (2nd ed.). San Francisco, CA: Jossey-Bass.

Elliott, R., Fischer, C. T., & Rennie, D. L. (1999). Evolving guidelines for publication of qualitative research studies in psychology and related fields. British Journal of Clinical Psychology, 38, 215–229. http://dx.doi.org/ 10.1348/014466599162782

Fassinger, R. E. (2005). Paradigms, praxis, problems, and promise: Grounded theory in counseling psychology research. Journal of Counseling Psychology, 52, 156–166. http://dx.doi.org/10.1037/0022-0167.52.2.156

Fine, M. (2013). Echoes of Bedford: A 20-year social psychology memoir on participatory action research hatched behind bars. American Psychologist, 68, 687–698. http://dx.doi.org/10.1037/a0034359

Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51, 327–358. http://dx.doi.org/ 10.1037/h0061470

Frankel, Z., Levitt, H. M., Murray, D. M., Greenberg, L. S., & Angus, L. (2006). Assessing silent processes in psychotherapy: An empirically derived categorization system and sampling strategy. Psychotherapy Research, 16, 627–638. http://dx.doi.org/10.1080/ 10503300600591635

Gergen, K. J. (2014). Pursuing excellence in qualitative inquiry. Qualitative Psychology, 1, 49–60. http:// dx.doi.org/10.1037/qup0000002

Giddens, A. (1987). Social theory and modern sociology. Palo Alto, CA: Stanford University Press.

Giorgi, A. (2009). The descriptive phenomenological method in psychology: A modified Husserlian approach. Pittsburgh, PA: Duquesne University Press.

Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. London, England: Wiedenfeld & Nicholson.

Greenberg, L. S. (2007). A guide to conducting a task analysis of psychotherapeutic change. Psychotherapy Research, 17, 15–30. http://dx.doi.org/10.1080/ 10503300600720390

Guba, E. G., & Lincoln, Y. S. (2005). Paradigmatic controversies, contradictions, and emerging confluences. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (3rd ed., pp. 191–215). Thousand Oaks, CA: Sage.

Harper, D. J. (2012). Surveying qualitative research teaching on British clinical psychology training programmes 1992–2006: A changing relationship? Qualitative Research in Psychology, 9, 5–12.

Heatherington, L., Messer, S. B., Angus, L., Strauman, T. J., Friedlander, M. L., & Kolden, G. G. (2012). The narrowing of theoretical orientations in clinical psychology doctoral training. Clinical Psychology: Science and Practice, 19, 364–374. http://dx.doi.org/ 10.1111/cpsp.12012

Hill, C. E. (Ed.). (2012). Consensual qualitative research: A practical resource for investigating social science phenomena. Washington, DC: American Psychological Association.

Husserl, E. (1970). Logical investigations ( J. N. Findlay, Trans.). New York, NY: Humanities Press. (Original work published 1900)

Kuhn, T. S. (1962). The structure of scientific revolutions. Chicago, IL: University of Chicago Press.

Labov, W. (2006). Narrative pre-construction. Narrative Inquiry, 16, 37–45. http://dx.doi.org/10.1075/ni.16.1. 07lab

Levitt, H. M. (2001). Sounds of silence in psychotherapy: The categorization of clients’ pauses. Psychotherapy Research, 11, 295–309. http://dx.doi.org/10.1080/ 713663985

Levitt, H. M. (2015a). Interpretation-driven guidelines for designing and evaluating grounded theory research: A pragmatic-social justice approach. In O. Gelo, A. Pritz, & B. Rieken (Ed.), Psychotherapy research—Foundations, outcome and process (pp. 445–484). New York, NY: Springer.

Levitt, H. M. (2015b). Qualitative psychotherapy research: The journey so far and future directions. Psycho- therapy, 52, 31–37. http://dx.doi.org/10.1037/a0037076

Levitt, H. M., & Ippolito, M. R. (2014). Being transgender: Navigating minority stressors and developing authentic self-presentation. Psychology of Women Quarterly, 38, 46–64. http://dx.doi. org/10.1177/0361684313501644

Levitt, H. M., Lu, E. C., Pomerville, A., & Surace, F. I. (2015). Pursuing the question of reflexivity in

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .

Heidi M. Levitt

348

psychotherapy and qualitative methods: The contributions of David L. Rennie. The Journal of Counselling & Psychotherapy Research, 15, 3–11. doi: 10.1002/capr.12021

Levitt, H. M., Morrow, S. L., Wertz, F. J., Motulsky, S. L., & Ponterotto, J. G. (2014, August). Inviting comments on the evolving recommendations for reviewing qualitative research in psychology. Paper presented at the 122nd Annual Convention of the American Psychological Association, Washington, DC.

Levitt, H. M., & Williams, D. C. (2010). Facilitating client change: Principles based upon the experience of eminent psychotherapists. Psychotherapy Research, 20, 337–352. http://dx.doi.org/10.1080/ 10503300903476708

Lewin, K. (1946). Action research and minority problems. Journal of Social Issues, 2, 34–46. http:// dx.doi.org/10.1111/j.1540-4560.1946.tb02295.x

Maslow, A. H. (1954). Motivation and personality. New York, NY: Van Nostrand.

Maslow, A. H. (1968). Toward a psychology of being. New York, NY: Van Nostrand Reinhold.

Morrow, S. L. (2005). Quality and trustworthiness in qualitative research in counseling psychology. Journal of Counseling Psychology, 52, 250–260. http://dx.doi.org/10.1037/0022-0167.52.2.250

Moustakas, C. (1994). Phenomenological research methods. Thousand Oaks, CA: Sage.

Pascual-Leone, A., & Greenberg, L. S. (2007). Emotional processing in experiential therapy: Why “the only way out is through.” Journal of Consulting and Clinical Psychology, 75, 875–887. http://dx.doi. org/10.1037/0022-006X.75.6.875

Peirce, C. S. (1965). Collected papers of Charles Sanders Peirce. Cambridge, MA: Belknap Press.

Polkinghorne, D. E. (1988). Narrative knowing and the human sciences. Albany: State University of New York Press.

Ponterotto, J. G. (2013). Qualitative research in multicultural psychology: Philosophical underpinnings, popular approaches, and ethical considerations. Qualitative Psychology, 1, 19–32. http://dx.doi.org/10.1037/2326-3598.1.S.19

Potter, J., Edwards, D., & Wetherell, M. (1993). A model of discourse in action. American Behavioral Scientist, 36, 383–401. http://dx.doi.org/10.1177/ 0002764293036003008

Rennie, D. L. (1994). Clients’ deference in psychotherapy. Journal of Counseling Psychology, 41, 427–437. http://dx.doi.org/10.1037/0022-0167.41.4.427

Rennie, D. L. (1995). On the rhetorics of social science: Let’s not conflate natural science and human science.

Humanistic Psychologist, 23, 321–332. http://dx.doi. org/10.1080/08873267.1995.9986833

Rennie, D. L. (2000). Grounded theory methodology as methodical hermeneutics: Reconciling realism and relativism. Theory and Psychology, 10, 481–502. http://dx.doi.org/10.1177/0959354300104003

Rennie, D. L. (2007). Methodical hermeneutics and humanistic psychology. Humanistic Psychologist, 35, 1–14. http://dx.doi.org/10.1080/08873260709336693

Rennie, D. L. (2012). Qualitative research as methodical hermeneutics. Psychological Methods, 17, 385–398. http://dx.doi.org/10.1037/a0029250

Rennie, D. L., Phillips, J. R., & Quartaro, G. K. (1988). Grounded theory: A promising approach to conceptualization in psychology? Canadian Psychology/Psychologie canadienne, 29, 139–150. http://dx.doi.org/10.1037/h0079765

Riley, S. C. E., Sims-Schouten, W., & Willig, C. (2007). The case for critical realist discourse analysis as a viable method in discursive work. Theory and Psychology, 17, 137–145. http://dx.doi.org/10.1177/ 0959354307073156

Shadish, W. R. (1995). Philosophy of science and the quantitative-qualitative debates: Thirteen common errors. Evaluation and Program Planning, 18, 63–75. http://dx.doi.org/10.1016/0149-7189(94)00050-8

Smith, J. A., Flowers, P., & Larkin, M. (2009). Interpretative phenomenological analysis: Theory, method and research. London, England: Sage.

Tacq, J. (2011). Causality in qualitative and quantitative research. Quality and Quantity: International Journal of Methodology, 45, 263–291. http://dx.doi. org/10.1007/s11135-009-9293-0

Timulak, L. (2007). Identifying core categories of client- identified impact of helpful events in psychotherapy: A qualitative meta-analysis. Psychotherapy Research, 17, 305–314. http://dx.doi.org/10.1080/ 10503300600608116

Wendt, D. C., & Gone, J. P. (2012). Decolonizing psychological inquiry in American Indian communities: The promise of qualitative methods. In D. K. Nagata, L. Kohn-Wood, & L. A. Suzuki (Eds.), Qualitative strategies for ethnocultural research (pp. 161–178). http://dx.doi.org/10.1037/13742-009

Wertz, F. J. (2014). Qualitative inquiry in the history of psychology. Qualitative Psychology, 1, 4–16. http:// dx.doi.org/10.1037/qup0000007

Yick, A. G. (2008). A metasynthesis of qualitative findings on the role of spirituality and religiosity among culturally diverse domestic violence survivors. Qualitative Health Research, 18, 1289–1306. http:// dx.doi.org/10.1177/1049732308321772

Co py

ri gh

t Am

er ic

an P sy

ch ol og ic al A ss oc ia ti on . No t fo r fu

rt he

r di

st ri

bu ti

on .