When Should Mixed Methods Be Used?

Mixed_Methods_Research_Merging_Theory_with_Practic..._----_Pg_18--41.pdf

Home >Psychology homework help >When Should Mixed Methods Be Used?

Introduction to Mixed Methods Research 3

within the city. He wrote in 1926: “In so far as social structure can be defined in terms of position, social changes may be defined in terms of movement: and society exhibits in one of its aspects that can be measured and described in mathematical formulas.” He took an interest in land values, or street-car transfers, or the volume of traffic at intersections, as indexes for underlying social processes. (p. 153)

What are Mixed Methods?

In general, researchers who use mixed methods employ a research design that uses both quantitative and qualitative data to answer a particular question or set of questions. This combination of methods “involve[s] the collection, analysis, and integration of quantitative and qualitative data in a single or multiphase study” (Hanson, Creswell, Plano Clark, Petska, & Creswell, 2005, p. 224). The term “multimethods” refers to the mixing of methods by combining two or more qualitative methods in a single research study (such as in-depth interviewing and partici- pant observation) or by using two or more quantitative methods (such as a survey and experiment) in a single research study.

Mixed methods is a rich field for the combination of data because with this design “words, pictures, and narrative can be used to add meaning to numbers” ( Johnson & Onwuegbuzie, 2004, p. 21). In other words, what we generally consider qualitative data—“words, pictures, and narrative”—can be combined with quantitative, numerical data from a larger-scale study on the same issue, allowing our research results to be generalized for future studies and examinations.

Why Use Mixed Methods?

Greene, Caracelli, and Graham (1989) list five specific reasons that researchers should consider using mixed methods. The first, triangula- tion, seems to be the most commonly cited reason that mixed methods are incorporated into research. Triangulation—or, more specifically, methods triangulation, in the context of methods alone— refers to the use of more than one method while studying the same research ques- tion in order to “examine the same dimension of a research problem” ( Jick, 1979, p. 602). The researcher is looking for a convergence of the data collected by all methods in a study to enhance the credibility of the research findings. Triangulation ultimately fortifies and enriches a

Hesse-Biber, Sharlene Nagy. <i>Mixed Methods Research : Merging Theory with Practice</i>, Guilford Publications, 2010. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/ashford-ebooks/detail.action?docID=471119. Created from ashford-ebooks on 2019-07-16 17:15:09.

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

4 M I X E D M E T H O D S R E S E A R C H

study’s conclusions, making them more acceptable to advocates of both qualitative and quantitative methods.

The second reason to consider incorporating a mixed methods design is complementarity. Complementarity allows the researcher to gain a fuller understanding of the research problem and/or to clarify a given research result. This is accomplished by utilizing both quantita- tive and qualitative data and not just the numerical or narrative expla- nation alone to understand the social story in its entirety. Both com- plementarity and triangulation are useful “for cross- validation when multiple methods produce comparable data” (Yauch & Steudel, 2003, p. 466). Complementarity has proven useful in several research studies, and a strong example is Yauch and Steudel’s (2003) examination of the organizational cultures of two small manufacturers. In their work, Yauch and Steudel utilized both qualitative and quantitative research methods. Not only did the triangulation of the qualitative and quanti- tative data secure the validity of their study, but the complementarity of the two datasets produced a more thorough comprehension of the organizational cultures in question.

The researchers first used employee interviews to gather a wealth of narrative information and then used their qualitative findings to cre- ate a survey to collect numerical data. Yauch and Steudel’s hope was to use mixed methods to “identify key cultural factors that aided or hindered a company’s ability to successfully implement manufacturing cells [autonomous labor teams]” (2003, p. 467). Combining and cross- ing these data yielded rich results:

Despite the long delay between beginning the qualitative assessment and administering the survey, the OCI [survey] was an important means of triangulation for two of the cultural factors identified and had the potential to reveal additional cultural dimensions that the qualitative analysis might have missed. (Yauch & Steudel, 2003, p. 476)

This study illustrates the power and possibilities inherent in mixed methods research; the researchers unearthed the convergence of the data from interviews and surveys through triangulation as well as the complementarity of the qualitative and quantitative data, which sup- plied them with a greater understanding of the organizational cultures of small manufacturers.

Mixed methods assist the researcher’s total understanding of the research problem; this understanding represents the third reason

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

Introduction to Mixed Methods Research 5

for using mixed methods: development. Mixed methods often aid in the development of a research project by creating a synergistic effect, whereby the “results from one method . . . help develop or inform the other method” (Greene et al., 1989, p. 259). For example, statistical data collected from a quantitative method can often shape interview ques- tions for the qualitative portion of one’s study. Jenkins’s (2001) research on rural adolescents and substance abuse illustrates the potential of the development factor in a mixed methods study. Jenkins administered a structured questionnaire to quantitatively measure students’ drug use. Her study also included a follow-up set of focus-group interviews and open-ended questionnaires intended to capture students’ perceptions of “drug resistance difficulties” to a variety drugs ranging from alco- hol to LSD to various types of narcotics ( Jenkins, 2001, p. 215). The results from conducting both these studies sequentially contributed to Jenkins’s overall understanding of drug abuse among this population. Her initial use of a structured questionnaire provided her with a statisti- cal understanding of student drug use. A follow-up focus-group study provided her the opportunity to triangulate her data (asking whether the findings from both studies agreed), and, in doing so, she found that the results from the focus group were “consistent with the open- ended questionnaire findings . . . [and] provided further clarification and, in some instances, additional information” ( Jenkins, 2001, p. 219). In addition, by implementing a sequential design with the quantita- tive component first and the qualitative second, she was able to attain a “value added” understanding of the results from both studies. Her focus-group data allowed her to clarify and follow up on definitions and uses of terms, such as “peer pressure,” that were used in survey questions and to ground their meaning from the perspective of her respondents.

A fourth reason cited for using mixed methods is initiation; a study’s findings may raise questions or contradictions that will require clarifi- cation, thus initiating a new study. The desired effect of the new study would be to add new insights to existing theories on the phenomenon under examination (Greene et al., 1989). In fact, findings from this study might uncover a completely new social research topic and launch a new investigation, leading us to a fifth reason for doing mixed methods research: expansion. Expansion is intended to “extend the breadth and range of the inquiry” (Greene et al., 1989, p. 259). Producing detailed findings helps enable future research endeavors and allows researchers to continuously employ different and mixed methods in their pursuit of new or modified research questions.

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

6 M I X E D M E T H O D S R E S E A R C H

Greene and her colleagues (1989) provide a useful organizing framework for characterizing the ways researchers have used mixed methods. We can clearly see the positive power and synergy of using these methods to complement one’s research findings. Quantitative information delivered in a “hard data” format is amenable to statistical analyses and standardized tests of reliability and validity. Qualitative data add an in-depth understanding of research results and allow the researcher to explore anomalies or subgroups within the data. Work- ing with both methods gives many researchers a cross-check on their research results. Qualitative data illuminate the meaning of statistical results by adding a narrative understanding to quantitative research findings. Qualitative methods can also assist researchers who want to test the validity of their research questionnaires by sequentially utiliz- ing mixed methods. For instance, an initial qualitative study allows the development of research instruments, such as a questionnaire, that can be used in a large-scale quantitative research study. Similarly, quantita- tive data can assist qualitative researchers by providing them with a broader context within which to place their qualitative data, as well as providing, through survey samples, ways to identify representative cases for their in-depth research. It is in this sense that quantitative data can be useful for establishing generalizability of qualitative results.

All of these reasons provide strong arguments for a researcher to consider a mixed methods approach. The following in-depth example shows the promise of mixed methods research in tackling a complex social problem: obesity in children.

The Fight against Childhood Obesity: An Illustration of the Need for and Importance of Mixed Methods

America is facing a new range of problems in the 21st century. Poverty, war, and health care are among these challenges, but a deeper problem threatens families and children across the country. Childhood obesity has become a nationwide epidemic, and parents and physicians alike are raising concerns. Consider the following article that appeared in The New York Times:

Six-year-old Karlind Dunbar barely touched her dinner, but not for time- honored 6-year-old reasons. The pasta was not the wrong shape. She did not have an urgent date with her dolls.

The problem was the letter Karlind discovered, tucked inside her report card, saying that she had a body mass index in the 80th

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

Introduction to Mixed Methods Research 7

percentile. The first grader did not know what ‘‘index’’ or ‘‘percen- tile’’ meant, or that children scoring in the 5th through 85th percen- tiles are considered normal, while those scoring higher are at risk of being or already overweight.

Yet she became convinced that her teachers were chastising her for overeating.

Since the letter arrived, “my 2-year-old eats more than she does,” said Georgeanna Dunbar, Karlind’s mother, who complained to the school and is trying to help her confused child. “She’s afraid she’s going to get in trouble,” Ms. Dunbar said. (Kantor, 2007, pp. A1, A14)2

Many school districts across the country have adopted the body mass index (BMI) screening test as a weapon for fighting childhood obesity. On the surface, conducting BMI screening tests on students from kindergarten through eighth grade appears to achieve an impor- tant health goal by identifying the percentage of children who are over- weight, a crucial step in fighting childhood obesity. Yet there does not seem to be a research design in place that can fully address this objec- tive if the goal of these data is to assist in stemming the tide of obesity. In fact, it appears as though little thought was ever given as to how these data might empower parents and their children to be part of the prob- lem’s solution. Gathering descriptive quantitative data is not enough to complete the goals of this study. One needs a broader understanding of the social context within which this information is disseminated among families to ensure that both students and parents are prepared to pre- vent childhood obesity and to develop the best strategy for fighting the obesity epidemic now.

There are many questions we might ask to gain a clearer under- standing of the factors nurturing America’s obesity problem. Explor- ing the lived experiences of students and their families and the day-to- day experiences of children’s eating behaviors at school would provide beneficial information. What is the school environment like for stu- dents with respect to eating? What types of relationships do they have with food at home? To what extent, if any, would parents welcome the school’s input into their children’s issues with weight? These are only a few of the many social- context questions that should be addressed so that schools might utilize the best approach for conveying weight- related “bad news” to both students and their families.

Designing a stronger research strategy would play a significant role in this process. For example, one might use a qualitative focus-

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

8 M I X E D M E T H O D S R E S E A R C H

group-study design following the collection of BMI scores. The quan- titative data collected could be presented to a representative group of parents; the researchers, perhaps in conjunction with school and health officials, could assess the parents’ reactions to this aggregated data for their school and also receive input on how to disseminate, if at all, these results to families and students. A mixed methods study consisting of a quantitative component (the BMI score collection) fol- lowed by a qualitative component (focus group) might give research- ers a deeper understanding of parents’ feelings and attitudes toward childhood obesity. The focus group could also collect parents’ sugges- tions as to how these results could be utilized to combat the epidemic. As one mother quoted in The New York Times article commented, “The school provides us with this information with no education about how to use it or what it means” (Kantor, 2007, p. A14). A fictitious causal link exists in many researchers’ minds that letters to parents will trig- ger the logical response of parents taking personal action to stem the tide of weight gain in their children. This strategy has not succeeded, and, in some cases, the opposite reaction seems to have occurred. One report observed the reaction of some parents: “the letters sent home with report cards . . . [were] a shock. Many parents threw them out, outraged to be told how much their children should weigh or uncon- vinced that children who look just fine by local standards are too large by official ones” (Kantor, 2007, p. A14).

In the end, rather than fighting the effects of obesity, the letter made many families feel helpless and victimized by a flawed system. There appears to be little understanding of the physical and mental effects of bad BMI news on parents and children. The assumption that the problem stems from imprinted eating habits learned at home leads to the conclusion that BMI screenings conducted in school would be an effective deterrent to childhood obesity. Yet research has revealed that the problem runs deeper than family eating habits and screenings. In some schools, the food available in the cafeterias was not nutritious and, in fact, exacerbated the problem. In these specific schools, cafete- ria cuisine was the central root of the problem, as schools were “con- tinuing to feed them atrocious quality meals and snacks, with limited if any opportunities for phys-ed in school” (Kantor, 2007, p. A14).

This example illustrates the need for mixed methods research across a range of disciplines from the physical sciences to the social and behavioral sciences. The problems that followed the BMI screening tests and information letters could have been assuaged with a unique

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

Introduction to Mixed Methods Research 9

research method that would probe the numerical and statistical data and unite it with qualitative information collected through interviews with parents and students so that an effective means of childhood obe- sity prevention and reversal might be discovered. Scholars and research- ers are still debating the benefits of mixing methods, but there seems to be significant promise for this revolutionary design.

objectives of this Book

One objective of this book is to reconceptualize the approach to mixed methods research by offering a comprehensive approach that stresses the tight link between theory and research and that centers the research prob- lem in the design and analysis of mixed methods projects, whether they are derived from a quantitative or a qualitative approach.

A second aim of this book is to center qualitative approaches to mixed methods research. Typically, in mixed methods research discourse, quantitative approaches have primacy over qualitative ones. Qualitative approaches stem from a different research logic, one that privileges sub- jective experience and that is open to a multilayered view of the social world. In this book, we focus on the idea that centering a qualitative approach in mixed methods research can be illuminating, useful, and advantageous, especially as a means to get at subjugated knowledge— knowledge that has not been a part of mainstream research inquiry. A mixed methods approach also allows the researcher to get at “subjective experiences” of those researched while providing the means to test out theories generated from in-depth research samples.

A third objective is to provide researchers with a more detailed understanding of qualitative mixed methods perspectives and prac- tices. In addition, it is my hope that those researchers currently practicing mixed methods from a quantitative approach may dialogue with qualitative approaches by reflecting on the ways in which they might integrate a qualitative perspective into their mixed methods practice.

A final goal for this book is visionary and elusive. It is my wish that, in uncovering and centering qualitative approaches to mixed methods, we can open up a dialogue across current mixed methods research approaches and practices that may serve to fuel synergy and innovation in methods practice with the goal of providing a more complex view and understanding of the social world.

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

10 M I X E D M E T H O D S R E S E A R C H

a comprehensive Perspective on Mixed Methods

I argue that the current practice of mixed methods research is exem- plified by a “cart before the horse” approach. Mixed methods designs are driven by research techniques to the detriment of theory-based research. Figure 1.1 presents a “methods- centric” approach to mixed methods research that places methodology (theory) last in the process of choosing a particular mixed methods design. In this mixed meth- ods design (Figure 1.1), methodology (theory) is isolated from the rest of the research model and appears as the last in the design sequence. It is not clear how theory (whether it is implied or explicit) is linked to a specific methods design, but it appears as if the mixed methods design model is methods- centric in that the type of mixed method model selected drives the type of theory chosen.

In their review of trends and issues in mixed methods evalua- tion, Miller and Fredericks (2006) suggested that this misplacement of theory in a mixed methods project is a “problem of logic” that often plagues the choice of mixed methods approaches, especially in evalua- tion research. They noted:

FiGURe 1.1. Methods-centric approach to mixed methods research.

Methods (methods drive the research)

Data

Research Question

Methodology (theory)

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

Introduction to Mixed Methods Research 11

One can, of course, a priori, simply declare that any MM strategy is appropriate for the issue being studied. However, this thinking intro- duces an unwanted arbitrariness to the whole process, which poten- tially undermines the very purpose of advocating for the uniqueness of MM. The situation is further exacerbated by the use of vague rationales for the selection of a particular mixed method, suggesting that the use of MM will result in richer data or stronger inferences. (p. 569)

As I have stated, I believe that current practice of mixed methods research takes a “cart before the horse” approach. I propose, instead, a practice of mixed methods that is firmly rooted within a research con- text with the intention that the method or methods used foster a richer understanding of the research problem under investigation.

a comprehensive approach to Mixed Methods

Figure 1.2 provides a diagram of a comprehensive approach to research. The basic premise of the comprehensive approach is that methodol- ogy provides the theoretical perspective that links a research problem with a particular method or methods. Methodologies are derived from a researcher’s assumptions about the nature of existence (ontology). These assumptions, in turn, lead to their perspective philosophy or set of philosophies on the nature of knowledge building (epistemology) regarding such foundational questions as: Who can know? What can be known? We can think of methodology as a theoretical bridge that con- nects the research problem with the research method. Kushner (2002) underscored the importance of methodology in methods practice:

We cannot talk of “method” alone. To talk about the “interview” apart from its purpose is merely to picture two people engaged in ver- bal exchange. Only when we shift to the level of methodology where we talk about purpose and value does the instrument become suf- ficiently complex to sustain discussion. Method is like a glove which needs the human hand to give it shape and meaning. (p. 252)

Methodology leads the researcher to ask certain research ques- tions and prioritize what questions and issues are most important to study. Researchers within and across disciplines can hold a range of different methodologies that frame their methods practice. They might

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

12 M I X E D M E T H O D S R E S E A R C H

use methodologies that hold up the importance of studying the “lived experiences” of individuals (interpretative methodologies), those that privilege hypothesis testing and causality as the most important goals of social inquiry (positivist and postpositivist methodologies), or meth- odologies that stress issues of power, control, and social justice (trans- formative and critical methodologies).

A methodological perspective is not inherently quantitative or qual- itative in terms of its use of method. For example, those who practice more positivistic methodology— traditionally seen as quantitative—can use qualitative as well as quantitative methods. In fact, qualitative and quantitative methods are carried out within a range of methodologies (theoretical perspectives). Those espousing feminist methodologies— traditionally viewed as qualitative researchers—can use quantitative methods, including surveys and experiments, in their research. The

Ontology

Epistemology

Methodology

Research Problem

Mixed Methods Design

Paradigmatic Viewpoint

Economic factors

Serendipity

Stakeholder interests Review of literature

FiGURe 1.2. Comprehensive approach to mixed methods research.

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

Introduction to Mixed Methods Research 13

comprehensive perspective on mixed methods troubles the dualism between qualitative and quantitative methods when we remember that methods lie in the service of methodologies.

Jennifer Greene (2002) captured the important role that a meth- odological framework takes on in a research project:

Most . . . methodologies have preferences for particular methods, but methods gain meaning only from the methodologies that shape and guide their use. . . . An interview does not inherently respect the agency of individual human life; it only does so if guided by and implemented within a methodological framework that advances this stance. So, any discussions of mixing methods . . . must be discus- sions of mixing methodologies, and thus of the complex epistemo- logical and value-based issues that such an idea invokes. (p. 260)

Greene stated that methods are tools and that, in the hands of researchers with certain methodological persuasions, they can be used to promote social justice, maintain the status quo, or promote social transformation. There is a need, then, for researchers to be conscious of the methodological perspective(s) they employ within their research projects. Greene, Benjamin, and Goodyear (2001) referred to this as “thoughtful mixed method planning,” in which the researcher is cog- nizant of his or her particular methodological standpoint. They noted that each researcher should “figure out one’s stance on the ‘paradigm issues’ in mixed method enquiry” (pp. 29–30). Good mixed methods work requires “consciousness of this organizing framework and adher- ence to its guidance for enquiry practice” (p. 30).

It is also important to note, however, that although paradigmatic assumptions very often guide the researcher in a variety of ways— including the selection of certain research problems, the choice of methods selected, and how the researcher will analyze and interpret his or her data— research questions can be guided by a range of addi- tional factors. Figure 1.2 depicts these factors as arrows that can influ- ence the research question. Influential factors can include a review of the research literature, specific constraints the researcher is faced with (both economic and lifestyle choices, as well as those particular constraints imposed by funding agencies and/or one’s disciplinary practice), and even serendipity; these factors can preclude or favor the choice of particular research problems.

Sometimes a researcher is conscious about his or her paradigmatic worldview, but it can also remain implicit or even unconscious and taken for granted. Although there may be additional factors (economic

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

14 M I X E D M E T H O D S R E S E A R C H

concerns, time constraints, stakeholder interests) to take into account in the selection of a research method, methodology (theoretical per- spective) is the link that connects the research question with specific methods that address these questions (see Jackson, 2006). We take up these issues in depth in Chapter 2.

the Need for Qualitative approaches to Mixed Methods Research

The mixed methods research design that appears to be the favored design model incorporates a qualitative component into a primarily quantitative study, as this model is thought to satisfy the need for gen- eralization and to provide the illustrative power of narrative (Morse, 2003). This is not to discount the fact that there are mixed methods designs in which the qualitative component is primary, but even within these mixed methods studies, the goal of the qualitative component is often to “assist” the quantitative data in developing, for example, better measures for a survey or to assist in teasing out contradictory survey findings (Creswell, Shope, Plano Clark, & Green, 2006).

This popular type of mixed methods model is based on a quantita- tive positivistic approach to mixed methods research, one that privileges the scientific method of knowledge building. This type of science is built on a primarily deductive mode of knowledge building that relies on “theory testing” and that privileges value neutrality and objectivity over subjectively derived knowledge. Positivism holds the central belief that an objective reality exists that is independent of any individual’s subjective experience. Quantitative data and analyses (using surveys, experiments, and statistical analysis of data) are often at the center, with qualitative data used in the service of enhancing a primarily quan- titative mixed methods approach. Creswell, Plano Clark, Gutmann, and Hanson (2003) note that one of the most common mixed meth- ods designs is triangulation (QUAN + QUAL). This design is employed when a researcher seeks to validate quantitative statistical findings with qualitative data results. Yet the assumption underlying triangulation is the positivistic view that there is an objective reality in which a given truth can be validated. The most common type of sequential mixed methods design appears to place the qualitative study in a more sup- portive role, “where qualitative pilot work is likely to precede and be subservient to a larger survey” (Brannen, 2005, p. 15). Bahl and Milne

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

Introduction to Mixed Methods Research 15

(2006) note that within marketing research most researchers employ a “positivist orientation,” using qualitative research in a “supportive role” to a more privileged quantitative study (p. 198).

Giddings (2006) noted the extent to which much of the mixed methods research has been conducted under the guise of a positivistic methodology—what she called, “positivism dressed in drag”:

A design is set in place, a protocol followed. In the main, the ques- tions are descriptive, traditional positivist research language is used with a dusting of words from other paradigms, and the designs come up with structured descriptive results. Integration is at a descriptive level. A qualitative aspect of the study is often “fitted in.” The think- ing is clearly positivist and pragmatic. The message often received by a naïve researcher, however, is that mixed methods combines and shares “thinking” at the paradigm level. (p. 200)

According to Giddings, the idea that mixed methods research now combines the best of both qualitative and quantitative approaches is like a “new guise” for positivism and is really business as usual (2006, p. 200). In a sense, Giddings argued that mixed methods would serve only to strengthen the positivistic paradigm if in fact qualitative approaches were just “added and stirred” into a general positivistic methodological approach (p. 202). Although mixed methods approaches are not neces- sarily problematic in and of themselves, the consistent privileging of a quantitative approach in mixed methods research threatens to obscure the power and utility of qualitative approaches and the contributions they can make to mixed methods research.

There are few studies that analyze how mixed methods researchers are in fact using mixed methods designs in a range of empirical articles across the disciplines. What research exists suggests that researchers primarily use mixed methods at the “data collection” stage of a mixed methods project. Bryman (2006), in a content analysis of mixed meth- ods articles, notes that for the most part the quantitative and quali- tative portions of the studies were not integrated. In interviews with mixed methods researchers, Bryman (2007a) noted several reasons given by researchers for nonintegration. An important reason was the set of methodological biases researchers had toward one type of meth- odological approach. Researchers also mentioned the issue of the lag time between the two studies, especially with regard to the qualitative component of a study. Researchers also expressed confusion as to actu-

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

16 M I X E D M E T H O D S R E S E A R C H

ally how to mix methods, especially at the data analysis and interpreta- tion stages of a mixed methods project.

One missing piece in the discussion of mixed methods as a research design is the absence of its negative impact. We might critically ask ques- tions such as, When is applying more methods detrimental or unnec- essary to answering a given question? What are the economic costs of an additional methods component? What set of criteria helps the researcher decide when one method will do? What are the unintended consequences of applying a mixed methods design for the analysis and interpretation of data from both studies when the researcher is not adequately skilled in the application of both methods? These are but a few caveats regarding the employment of a mixed methods research design; what is more troublesome is that much of the mixed methods literature does not discuss the disadvantages of utilizing a mixed meth- ods design.

This book’s focus is on qualitative approaches, both mature and cutting edge, that enhance our understanding of the social world, par- ticularly the lived experiences of individuals and groups. A focus on qualitative approaches to mixed methods research also fits well with the movement toward interdisciplinary scholarship that is placing pres- sure on researchers to draw from many disciplinary methodologies and traditions. Focusing on a range of qualitative approaches can provide a broader theoretical lens through which to look at novel and often thorny interdisciplinary research problems and issues. Qualitative approaches also allow researchers to incorporate issues of social change, power, and authority that push on the boundaries of traditional positiv- ist and postpositivist research concerns and challenge what constitutes “legitimate” research.

Qualitative methodological approaches stress the importance of multiple subjective realities as an important source of knowledge build- ing. Epistemology in this paradigm holds that knowledge gathering and truth are always partial; that researcher values, feelings, and atti- tudes cannot be removed from the research relationship but instead should be taken into consideration when interpreting the data as part of the knowledge construction process; and some of these approaches also argue that the researcher should establish a reciprocal relationship with research participants to promote an interactional, cooperative co- construction of meaning.

In this book, I demonstrate both the strength of qualitative meth- odologies and how, when, and why these approaches may best be combined with quantitative methods in mixed method designs. This

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

Introduction to Mixed Methods Research 17

perspective is developed and illustrated throughout the book with in- depth examples of how researchers with a qualitative approach employ mixed methods designs.

Researchers originally trained in positivist approaches to research may want to explore the extent to which qualitative methods and methodologies can assist them in answering their research questions. Qualitative approaches to research questions can be valuable to posi- tivistic researchers in a variety of ways, from assisting with the genera- tion of theories and models to the testing of these models. Qualitative approaches, with their emphasis on participant meaning and experi- ence, can also provide a much- needed context for the development of a range of survey instruments from assessment and evaluation tools to question items on surveys.

Qualitative methodologies are a particularly sensitive means of capturing the lived experiences of groups and individuals, especially those often left out of traditional knowledge- building research proj- ects. In other words, qualitative approaches such as postmodernism and feminist critical theory desire to explore the subjective worlds of multiple realities, uncover perspectives of those who have been socially and politically marginalized, and upend positivism’s claims to objectiv- ity and traditional knowledge building as the source of truth.

Qualitative methodologies should not be mistaken for qualitative methods. Qualitative methodologies, as noted earlier, use quantitative as well as qualitative methods. The same rule applies for quantitative methodologies. After all, the method is but the tool; the methodology determines the way in which the tool will be utilized. This makes the issue of which design to use confusing and problematic for researchers working from a qualitative or quantitative methodological perspective. Early on, Lincoln and Guba (1985) noted that “naturalistic inquiry” was not necessarily antipositivistic and that, although the use of qualita- tive data allow the experiences of respondents to be voiced within the research project, there may be many ways in which quantitative data can be incorporated into this type of research approach (pp. 198–199).

This book highlights the ways in which a qualitative approach informs how mixed methods research is conducted. Not all qualitative approaches are the same, and I use this term as an umbrella category that encompasses several different approaches that have slightly differ- ent emphases. This book selects three specific qualitative approaches— an interpretative approach, a feminist approach, and a postmodern approach—to provide a more specific application of a qualitative approach to mixed methods research designs.

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

18 M I X E D M E T H O D S R E S E A R C H

For each approach (the interpretative, feminist, and postmodern- ist approaches), we are interested in how researchers with these spe- cific qualitative points of view integrate mixed methods into their work. How conscious or unconscious is the process of choosing to utilize mixed methods? More specifically, what motivations or purposes can be discerned as reasons for mixing methods? In what way(s), if any, are the qualitative and quantitative components of a project related, and at what stage in the research process? What are the issues and positive outcomes of using a mixed methods design?

Each of these approaches to research makes certain ontologi- cal, epistemological, and methodological assumptions regarding the nature of the social world. These different theoretical approaches are often subsumed under the term “qualitative methodologies.” However, these methodologies are just as diverse as the researchers who practice them, so let’s take a moment to examine these different perspectives in greater depth.

If a researcher employs an interpretative approach to research, the researcher’s epistemology assumes multiple subjective realities that consist of stories or meanings produced or constructed by individuals within their “natural” settings. Crabtree and Miller (1999) noted that interpretivists in particular

trace their roots back to phenomenology (Schutz, 1967) and herme- neutics (Heidegger, 1927, 1962). This tradition also recognizes the importance of the subjective human creation of meaning but doesn’t reject outright some notion of objectivity. Pluralism, not relativism, is stressed, with focus on the circular dynamic tension of subject and object. (p. 10)

Feminist perspectives are specifically interested in understanding knowledge “for women.” These perspectives, such as standpoint theory, are aware of the “male- centered” biases of traditional positivistic con- cerns, especially as they pertain to the issues of objectivity within the research process whereby individuals must place their own values and concerns outside the research endeavor. For feminist researchers, there is no knowledge that is without bias. There is no view from “nowhere”; knowledge is imbued with the power and authority of those who have it. They point to the long- standing androcentric (male) bias of early knowledge building, especially as practiced by early positivists, which often ignored women’s concerns in their research. Positivist research- ers have also historically overlooked issues of difference in terms of race, class, ethnicity, and sexual preference in their research problems

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

Introduction to Mixed Methods Research 19

and analyses, issues of difference that often centrally inform the work of feminist researchers. A feminist perspective moves the issues of those whose lives have been marginalized, overlooked, or misrepresented by traditional investigations to the “front and center” of the research agenda (see Hesse-Biber, 2007). Feminist research stresses the impor- tance of intersectionality— standpoints or worldviews created by the overlapping of a combination of locations within the social structure (i.e., race, class, gender, sexuality, geography, etc.).

Postmodern theorists stress a relativistic approach in their treat- ment of the social construction of reality. Postmodernism—as well as its related schools of thought, poststructuralism and critical theory— emerged in the late 20th century with the goal of challenging the status quo of modern enlightenment thought. One defining feature of post- modernist paradigms is that they assume that images, symbols, texts, and other representations have the power to create and sustain a given social reality. We use “postmodernism” as an umbrella term for these theories; however, that does not mean that they are the same thing, and at times these terms may not be internally consistent with one another.

Postmodernism challenges positivism’s emphasis on objective truth and instead posits how dominant discourse/language serves to oppress and maintain existing power relations within a society. Postmodern- ism focuses on deconstructing dominant images, symbols, and texts in order to disrupt their meanings and question their veracity, often by pointing out their class, race, and gender biases. Postmodernism favors a multilayered understanding of social reality that some critics of this perspective say borders on a relativistic view of the social world that obviates all of the scientific methods’ claims to truth.

Qualitative Mixed Methods approaches: some examples

In this section, I provide two examples of researchers who employ a qualitative approach to mixed methods and highlight the ways in which their specific researcher standpoints serve to guide their decision mak- ing at various stages throughout the research process.

A Feminist Approach

A feminist approach to knowledge building seeks to ask a set of research questions that often upend traditional forms of knowledge building by

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

20 M I X E D M E T H O D S R E S E A R C H

privileging the lived experiences of women at the center of research inquiry (feminist standpoint theory). Their theoretical perspective influences the type of research questions and the specific methods cho- sen, including the application of these methods in the data collection, analysis, and interpretation stages of the research process. A feminist methodology is firmly rooted in a specific theoretical standpoint on the nature of existence (ontology) and what can be known about it (episte- mology). A feminist perspective does influence one’s methods practice. For example, feminist researchers are particularly interested in issues of power, authority, and control while conducting research, and there- fore they specifically explore such questions as: What is studied and why? From whose perspective is it studied? Who is studied? Who is left out and needs to be included in this study? Looking at difference is an integral part of their research agenda. Realizing that not all women are the same, there is a particular interest in how race, class, sexual orienta- tion, and other differences intersect to impact specific women’s lives.

Feminist researchers utilize a range of both quantitative and quali- tative tools to answer research questions and are not wedded to one specific method or set of methods. They use whatever methods will best answer the research problem. The practice of research methods is mindful of the ways in which power and authority influence the research process.

Applying Mixed Methods: Behind the Scenes with David A. Karp

David A. Karp is a qualitative researcher who has done cutting-edge research on mental illness. He brings a symbolic interaction with the social world and a theoretical perspective that focuses on how people experience, define, and make meaning out of their daily worlds. He is particularly interested in the experiences of those individuals who are suffering from depression and other mental illnesses. I conducted a set of intensive interviews with him for a behind-the- scenes glimpse into how researchers talk about conducting their own research and the struggles they face in doing so. He highlights the importance of placing methodology first in our understanding of the research pro- cess. He demonstrates how it is the researcher’s theoretical perspective that leads her or him to ask certain questions, supporting the idea that the methodology of a researcher must be considered prior to deciding which methods to employ in order to acquire their answers.

In his research on depression, David Karp wants to understand depression on a micro rather than a macro level. His questions do

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

Introduction to Mixed Methods Research 21

not aim at the larger picture of depression—the numbers of people affected by this illness, the economic impact of the illness, and other grand-scale social problems. What matters to him instead is to under- stand this illness from the perspective of the individuals struggling with it on a daily, even minute-by- minute, basis—how it feels, what it means, how a person copes with it, how it shapes their interactions with others, and so on. There is a considerable challenge in this emphasis, because, according to Karp, a symbolic interaction perspective on depression is almost completely absent from the research literature on the subject:

what struck me looking at the statistical data was, here were people writing about affective disorders, writing about a feeling disorder, and in nowhere in all of this stuff, was I hearing the feelings of the people who had the disorder. It struck me as just strange, a strange contradiction, that people were writing about feelings without hearing the feelings of the people they were trying to write about.

It is the researcher’s methodological perspective that leads him or her to select a given research design. Karp’s methodological per- spective leads him to inquire about the “lived experiences” of those suffering from depression. He selects in-depth interviews as his pri- mary method because they allow him to pursue research questions that focus on understanding depression from the perspective of those afflicted. For instance, Karp observed, “most of my work in the con- text of the symbolic interaction point of view about things is that it’s an effort to see and to bring to the page the complexity of a messy phenomenon.” He feels that for this reason the in-depth qualitative interview “allows me to meet my goal, which is to bring to the reader a deeper understanding of an extremely complex matter, and I think to myself . . . what really would additional numbers bring to the books that are already written?” Although he considered gathering quanti- tative data, he mainly sees this type of data as “background” for his qualitative results.

the Perils of Mixed Methods

Another issue that is important to reflect on before embarking on a mixed methods study revolves around a researcher’s familiarity with both qualitative and quantitative research. Researchers must ask them-

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

22 M I X E D M E T H O D S R E S E A R C H

selves whether they have the training and resources necessary to carry out a mixed methods study.

For qualitative researchers, including quantitative data may be time- consuming and difficult to do, especially if one lacks training in the gathering and analysis of such data. Are the benefits worth the investment? David Karp talks about his hesitancy to take on another research method in his own work.

Sharlene heSSe-BiBer: When, if at all, might a qualitative researcher with a qualitative methodology think about using mixed methods?

DaviD Karp: You know, in some ways it’s a very uncomfortable question you’re asking because . . . what it is you tell students and what you do yourself [may be different]. Because I walk into a methods class, and . . . it doesn’t take more than about two hours into the semester to start talking about triangulation and multiple methods, and every method has strengths and every method has weaknesses. . . . But in the end, I’m always saying to the students that if you have the time and the energy and the money, and you want to get the best results in a piece of work, then you ought to have methods that complement each other . . . and that’s the best way that research ought to proceed and, if you can do it, that’s the way you ought to do it.

And then I go off in my own life and I don’t do much in the way of mixing methods and so, in some ways, the question makes me sort of uncomfortable. And . . . I think there are a couple of reasons I could mention [as to why I don’t]. One is that I think . . . you develop [ways of working] with which you feel comfortable, and it’s hard to break from that. . . . If you develop an expertise . . . with the way you’re doing things where you feel comfortable, where you’ve pulled it off before . . . it’s hard, it takes a certain courage to stray from . . . what you do well, what’s gotten you applause, and to start in some sense fooling around with [new] types of data. . . . It takes a little bit of a leap of courage . . . and maybe . . . a second dimension of this [is that] you haven’t challenged yourself to integrate that kind of data into the writing that you do. It’s partly a writing issue, I mean you get used to a certain kind of writing.

Jumping on the mixed methods bandwagon without really think- ing through the implications of doing so may yield research of dubious quality or research that does not add theoretical value and understand- ing to a research project. A quantitative researcher who lacks a basic understanding of qualitative methods but decides to use a qualitative

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

Introduction to Mixed Methods Research 23

method in his or her project runs the risk of “adding and stirring” his or her qualitative findings into a quantitative project. For example, in order to satisfy a funding agency that prefers a mixed method approach, a quantitative researcher might add a qualitative component to a study by including a few open-ended questions at the end of a (quantitative) survey. Karp described this approach, when used poorly or without the- oretical purpose, as “sprinkling.” Karp expanded on this point:

And you know, if you read work where a method is just a throw in, it’s just like going to the ice cream store and throwing a few M&Ms onto the top of the ice cream to make it look pretty, and it might taste a little bit better. When I see work like that, where the qualitative stuff is just thrown in for purposes of meeting an agenda, that doesn’t really forward your under- standing of the phenomenon or forward your theoretical thinking, that bothers me. I think all you’re doing is being pragmatic and sort of following someone else’s rules and falling in.

I think we should use these methods when they can really forward our understanding, I mean truly forward our understanding of things theo- retically, and any other reason strikes me as being problematic and in fact weakening the research that people do.

the Practice of Mixed Methods: Behind the scenes with stephen Borgatti

Stephen Borgatti is a professor of Management. He is both an anthro- pologist and a mathematician and is particularly interested in social network analysis. He conducts qualitative and quantitative research projects, as well as mixed methods projects. I talked with him on the importance of the link between one’s research problem and one’s method and some of the issues researchers may face when the fit between the two does not work. Borgatti is especially skeptical of forc- ing premade research designs into your research project. He believes that mixing qualitative and quantitative methods can be very produc- tive, and he does this regularly in his own work:

If I do a network analysis, I typically am going to do some kind of ethnog- raphy first to find out what even are the right questions or, if it’s theoreti- cally driven, I know what questions I want to ask, I still have to know what language to use so they will understand the questions that I’m asking. So I do a little ethnography, to try to understand what’s what, construct a sur-

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

24 M I X E D M E T H O D S R E S E A R C H

vey, then have people fill it out, and then, in the analysis, go back to people and say, “I’m finding this. How does this make sense to you, how do you interpret this picture?”

However, Borgatti believes that the growing movement among funding agencies to require the addition of a qualitative component to their quantitative studies—a form of mixed methods design—is mis- guided. He referred to this approach of force- fitting research design into a predetermined mold as a form of “scientism” and “magical think- ing” in which “the form of [the research] makes it scientific.” Borgatti noted that this problem can also work in reverse, such that quantita- tive methods are force-fit into a primarily qualitative design simply to increase the apparent legitimacy of the research:

I once saw a presentation by a postmodernist; he had done a factor analy- sis. And he put up these numbers and it was really interesting because it was sort of like a work of art, and the numbers were interpreted almost like a Rorschach, it [showed] little understanding about what factor analysis would involve. It was more like, I don’t know, it was more like totems of some kind that he pulled out. . . . I think . . . he was doing it to add legiti- macy to what he was doing. . . . He already had certain beliefs, and so he used the numbers to try to illustrate them, but the numbers weren’t telling the conclusions. He wasn’t drawing conclusions from the numbers. It was the other way around.

the Promise of Mixed Methods Research

An awareness of the importance of not “placing the cart before the horse” in mixed methods practice by centering the research problem holds the promise of enhancing existing theories and the discovery of new theoretical avenues. It is important that the research we do and the resulting data we collect relate back to our theoretical perspectives. Mixing methods can be an exciting process that might open our eyes to new realities and levels of understanding, but such a research design must be utilized only with the certainty that it will aid in “advancing our theoretical understanding” and not detract from our existing method- ologies.

My own mixed methods journey began many years ago when I first thought about writing a mixed methods book. In keeping with the

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

Introduction to Mixed Methods Research 25

importance of knowing your researcher standpoint as we go forward on this journey, the following are a few reflections on my book- writing journey.

My best recollection is that it all began as a nagging pain in the back of my mind each time I read articles and overall discussions about mixed meth- ods by those whose reputations have been made in promoting these ideas. I am not the creator of this new wave of mixed methods.

I am a feminist qualitative researcher who has a particular perspective on social reality. As a feminist, I am interested in asking a set of research questions that often trouble the waters of traditional knowledge building by including issues of difference in the research process.

I am interested in issues of power, authority, and control while con- ducting research as well as asking such questions as: What is studied? From whose perspective? Who is being studied? Who is left out and needs to be included in this study? I guess you might say that I am a methods interloper—an outsider and an insider to mixed methods research.

As a sociologist who has had traditional training in quantitative methods and the positivist paradigm, I am an insider in that I practice and teach both methods and have in fact conducted several mixed methods projects. As a feminist, I am often the outsider who asks new questions, yet I will utilize a range of tools— quantitative and qualitative—as needed to answer my questions. I am not wedded to one specific method or set of methods. I use whatever methods will facilitate getting answers to my research problem(s).

As a researcher, my agenda is one of promoting a comprehensive approach and understanding of the use of methods techniques by placing the practice of methods more firmly within a research context. If we are to realize the potential synergy of a mixed methods approach, it is important to understand the interconnections and philosophical groundings that con- nect our methods to research problems. I am cognizant of the importance of living within the contradictions and tensions of the research process. I enter into dialogue with this process. To dialogue means confronting our assumptions, suspending judgment, and embracing difference. To dialogue also means to hone our listening skills, with a stance toward understand- ing. I decided to write this book with the goal of providing researchers with a user- friendly guide to the practice of a qualitative approach to mixed methods research.

Sharlene heSSe-BiBer

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed

26 M I X E D M E T H O D S R E S E A R C H

G L o s s a R y

complementarity: one of the reasons for using mixed methods; accom- plished by utilizing both quantitative and qualitative data to under- stand the social story, allowing the researcher to gain a fuller under- standing of the research problem and/or clarify a given research result.

critical/postmodernist theories: qualitative approaches that examine how social life is produced and the privileges given to those in power, with a goal to emancipate and to expose social injustice.

development: aided by mixed methods when the “results from one method . . . help develop or inform the other method” (Greene et al., 1989, p. 259).

expansion: a reason for doing mixed methods research; used to “extend the breadth and range of the inquiry” (Greene et al., 1989, p. 259).

feminist approach: is specifically interested in understanding knowledge “for women”; moves the issues of those whose lives have been margin- alized, overlooked, or misrepresented by traditional investigations to the front and center of the research agenda.

initiation: use of mixed methods leads to findings that raise questions or contradictions that will require clarification, thus initiating a new study.

interpretative approach: researcher’s epistemology assumes multiple sub- jective realities that consist of stories or meanings produced or con- structed by individuals within their “natural” settings.

intersectionality: refers to standpoints or worldviews created by the over- lapping of a combination of locations within the social structure (i.e., race, class, gender, sexuality, geography, etc.).

methods triangulation: a reason for using mixed methods; the use of more than one method while studying the same research question. The researcher is looking for a convergence of the data collected by all employed methods in a study to enhance the credibility of the research findings.

mixed methods: a research design that uses both quantitative and qualita- tive data to answer a particular question or set of questions.

positivism: the central belief that there exists an objective reality and that “facts” are independent of any individual’s subjective experience and values.

C op

yr ig

ht ©

2 01

0. G

ui lfo

rd P

ub lic

at io

ns . A

ll rig

ht s

re se

rv ed