Week 5 Assignment 1
gd1222
MendingFences.pdf
Mending Fences: Defining the Domains and Approaches of Quantitative and Qualitative Research
Brittany Landrum and Gilbert Garza University of Dallas
In view of the increasing ubiquity of qualitative research, particularly mixed method designs, it is important to examine whether qualitative and quantitative models of research can be integrated and how this integration should take place. The recent adoption of best practices for mixed methods research by the NIH seems an opportune starting point for discussion of these questions. This article explores the notion that qualitative and quantitative research, while stemming from fundamentally different “approaches,” might yet find an appropriate complementary relationship. We argue, however, that such a complementary relationship depends on an understanding of the notion of approach and an insight into the fundamentally different guiding questions and domains of these 2 research models. Holding that “good fences make good neighbors,” this article explores the frontier between quantitative and qualitative research and the challenges attendant to designing and conducting mixed methods research.
Keywords: best practices, methodology, mixed methods research, qualitative research, quantitative research
Good fences make good neighbors. —Robert Frost (1919), “Mending Wall”
With the increasing ubiquity of qualitative research (Wertz, 2011) and the emergence of mixed methods research that utilizes both qual- itative and quantitative analysis (Creswell, Klassen, Plano Clark, & Smith, 2011; see also Creswell, 2009; Creswell & Clark, 2007; Tashakkori & Teddlie, 2003; Taskakkori, Ted- dlie, & Sines, 2013), there is a growing need to address the boundaries and differences between these two types of research. Both types of re- search have a set of usually implicit philosoph- ical suppositions (see Churchill & Wertz, 2002; Garza, 2004, 2007, 2011; Giorgi, 2009; von Eckartsberg, 1998; Wertz, 1985). Among oth- ers, Garza (2006) and Giorgi (2009) suggest that important differences exist between these two approaches to research. Following Giorgi, such differences would define different domains
of research motivated by fundamentally differ- ent questions and producing fundamentally dif- ferent knowledge claims. These different knowledge claims can “create a terrible mess” without an understanding of the philosophical foundations of both types of research (Greener, 2011, p. 3). Thus, this article seeks to delineate the domains of both approaches and discuss the combined use of quantitative and qualitative data and approaches in mixed methods research. An understanding of these differences with mu- tual respect for each domain will provide the necessary framework for discussing issues re- lated to mixing both types of research. Finally, we will discuss the complementarity of strengths of both approaches arguing for the necessity of methodological pluralism.
Defining Quantitative and Qualitative Domains and Approaches
Qualitative and quantitative research com- prise two different (but not opposed) interpre- tative frameworks. At a fundamental level, what distinguishes the domains of qualitative and quantitative research are the implicit interpreta- tive frames of reference that are brought to bear on their subject matter and methods (Giorgi,
Brittany Landrum and Gilbert Garza, Department of Psy- chology, University of Dallas.
Correspondence concerning this article should be ad- dressed to Brittany Landrum, Department of Psychology, University of Dallas, 1845 East Northgate Drive, Irving, TX 75062. E-mail: [email protected]
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an Ps
yc ho
lo gi
ca l
A ss
oc ia
tio n
or on
e of
its al
lie d
pu bl
is he
rs .
T hi
s ar
tic le
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
Qualitative Psychology © 2015 American Psychological Association 2015, Vol. 2, No. 2, 199–209 2326-3598/15/$12.00 http://dx.doi.org/10.1037/qup0000030
199
2009)—what von Eckartsberg (1998) and Giorgi (1970) have called ‘approach.’
In previous descriptions, qualitative and quantitative research have been defined by the type of data used (non-numeric and numeric, respectively; see Greener, 2011) as well as in- ductive and deductive frameworks (see Greener, 2011; Teddlie & Tashakkori, 2009). Another way to understand quantitative and qualitative approaches to research is in terms of the knowledge claims they make and the inter- pretive frameworks employed to bring these claims to light. At one end of a continuum describing the interface of knowledge and frame of reference are ‘purely’ quantitative studies. Such research examines relations of magnitude between variables measuring quantities1 (e.g., height, weight, number of behaviors, hippocam- pal volume, etc.) and uses the numeric analysis of data to test and verify these relations. At the other end of this continuum are ‘purely’ quali- tative studies. This sort of research makes de- scriptive knowledge claims about meaning us- ing ‘descriptive’ data typically expressing these findings in linguistic narratives.
However, all these definitions meant to dis- tinguish the two approaches are not mutually exclusive. Qualitative research does count and explore dimensions of magnitude (Sandelowski, 2001) and likewise quantitative research in- cludes non-numeric data (e.g., categorical data2) and makes inferences about meaning based on dimensions of magnitude (Teddlie & Tashakkori, 2009). Furthermore, all scientific inquiry draws on both inductive and deductive frameworks (see Merleau-Ponty, 1961/1964), and we would argue that the interplay between data and the interpretative frame of reference are not always mutually exclusively quantita- tive or qualitative. The boundaries between these two approaches, more often than not, are not a clearly defined fence but rather a mixing of both types of data and approaches. Indeed, in both ‘pure’ cases described above, the kind of knowledge claimed fits well with the frame of reference used to establish and communicate its findings. Verification or confirmation of such studies can be achieved in terms of replication within the same analytic model. However, it is with regard to the middle regions on the con- tinuum that epistemological clarity and explic- itness are needed to interpret research findings
and the light they shed on the topic under in- vestigation (see Figure 1).
One of these middle positions is called quan- titizing and occurs when research claims knowl- edge of an order of magnitude but uses a qual- itative interpretive framework as the basis of such claims (e.g., performing numerical analy- ses based on frequency of themes, or “ratings of strength or intensity” Teddlie & Tashakkori, 2009, p. 269; see also Sandelowski, Voils, & Knafl, 2009). The other ‘middle position’ is called qualitizing and occurs when research claims qualitative knowledge but uses a quan- titative interpretive framework as the basis of such claims (e.g., categories based on range in magnitude, frequency count taken as a dimen- sion of importance; Hesse-Biber, 2010; Sand- elowski et al., 2009; Teddlie & Tashakkori, 2009). Because the knowledge claims of such research and interpretive frames of reference used to establish and test them do not match, special care and epistemological knowledge must be used when interpreting such findings. For instance, Johnson and den Heyer (1980) emphasize the distinction between a statistical question and a psychometric question pointing to the necessity of understanding the rubric of measurement when interpreting IQ scores.
An example contrasting a ‘purely’ quantita- tive relationship with instances where data and approach are mixed data will help illustrate these concerns and the special care we are ad- vocating. A regression coefficient of 1 between number of friends on Facebook and number of photos on one’s profile means that an increase by one friend predicts an increase in one photo posted; both of these variables are measured using ratio data whereby 1 friend on Facebook and 1 photo are quantities and thus fall under the ‘purely’ quantitative approach. When the
1 We have deliberately chosen examples of measures whose relation to the scales which produce them are not under debate. There is widespread agreement that height and weight represent quantities on a ratio scale, for exam- ple. This is not always the case with scales such as the Likert type, which is discussed below.
2 Categorical data are often called qualitative or nominal data but are analyzed using specialized statistical methods within quantitative research (see Agresti, 2002). In this article, this scale of measurement classification is distinct from qualitative data and research which describe non- numeric data that are to be used with methodologies devel- oped by qualitative researchers.
200 LANDRUM AND GARZA
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an Ps
yc ho
lo gi
ca l
A ss
oc ia
tio n
or on
e of
its al
lie d
pu bl
is he
rs .
T hi
s ar
tic le
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
variable in question is on a Likert scale, the relationship is an increase or decrease in agree- ment based on the number people circle on average, not necessarily or directly with the construct it is taken to operationalize. Concerns have been raised against Likert type data con- cerning the appropriate use of parametric or nonparametric statistics resting upon whether it is interval or ordinal data, respectively (see, e.g., Carifio & Perla, 2008; Norman, 2010). For such data to be considered interval, one would have to be able to answer the question pointedly posed by Knapp (1990), “3 what?” in relation to a 3 circled on a Likert-type scale. This type of data is not quite a quantity like the number of friends or photos are; it is neither clear whether the steps on the scale are indeed equidistant from each other (see, e.g., Jamieson, 2004) nor whether the ‘degree’ of agreement is measuring a quantity of something and if this quantity is the same for everyone who completes it. The answers on a Likert-type scale cannot escape the subjective understanding of the participant. We are not saying that Likert type data should not be used in this way; rather we are advocat- ing for appropriately understanding the knowl- edge claims they make. Likert-type data fit somewhere between the two end points on the spectrum of the interface of knowledge and appear to be an example of quantitizing whereby a dimension of agreement (qualitative) is rendered in terms of quantity (quantitative).
While a psychometric question can be dis- tinct from a statistical question, Merenda (n.d.) points to a more troubling example of quantitiz- ing concerning a case when the question of what one is measuring cannot be separated from the
statistical problems that it raises. The case in point Merenda highlights is when data repre- senting dichotomous categories, such as male and female, are included with other continuous predictor variables through ‘dummy coding’ in a regression analysis. To be used in statistical analyses that require continuous variables, these dichotomous variables are treated as though they were continuous, as though there were values somewhere between male and female. This is a violation of the assumption of contin- uous and discrete predictor variables in a regres- sion analysis thus presenting a questionable sta- tistical result. He further adds that there is no substitute for conducting a separate analysis between males and females.
In an example of qualitizing, Cialdini et al. (1976) calculated the frequency of ‘we’ and ‘non-we’ statements used to describe team and personal outcomes for players on a sport team. A dimension of quantity (counts/frequency) is rendered in terms of subjective ownership of instrumentality in a sport team’s victory or de- feat. Similarly, in an example of quantitizing, Pollard, Nievar, Nathans, and Riggs (2014) counted the frequency of occurrences of various themes from qualitative narratives and con- cluded that based on nonsignificant chi-squared analyses that the experiences of Hispanic and Caucasian mothers did not differ thematically. In this example, a quantitative rubric is utilized to make claims regarding dimensions of expe- rience. In these examples, we see the need to take special care when interpreting the mean- ings of the statistical analysis and the operation- alization of the constructs given that the data
Middle Ground
Qualitizing Quantitizing
‘Pure’ Quantitative ‘Pure’ Qualitative
Numerical analysis of
data that are quantities
Descriptive analysis of
data that are non-numeric
Figure 1. The possible configurations of data and interpretative frame of references repre- sented as a continuum. The middle ground is of special concern regarding the practice of mixed methods.
201QUANTITATIVE AND QUALITATIVE DOMAINS
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an Ps
yc ho
lo gi
ca l
A ss
oc ia
tio n
or on
e of
its al
lie d
pu bl
is he
rs .
T hi
s ar
tic le
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
(quantitative or qualitative) and interpretation (qualitative or quantitative) do not coincide.
Although we argue that neither method holds a privileged perspective on the world, these two modes of description are distinguished, for the most part, by their respective approaches. We hold that no inquiry can be undertaken from a perspective-less position (Merleau-Ponty, 1945/ 1962) and thus even natural science is not value free (see Kendler, 2005 who asserts this and Garza, 2006, who refutes this position). Indeed, we would hold that an explicit acknowledgment of approach is necessary to assess the validity of any inquiry (Churchill, Lowery, McNally, & Rao, 1998; Garza, 2004).3 Specifically in qual- itative research, validity comprises a coherence between the researcher’s frame of reference, the research question, the data, and the findings. Next, we will turn to some specific concerns with mixed methods.
Concerns Regarding the Intersection of Quantitative and Qualitative Frameworks
The Question of Hegemony of Approach
In a qualitative research training meeting, conducted for researchers who were for the most part both well-versed in quantitative re- search models and inexperienced in qualitative research, one individual expressed a concern with the notion of ‘interrater reliability’ and a desire to make sure all the ‘coders’ were naming themes the same way. This individual felt that if one coder named a theme ‘reluctance’ and an- other named it ‘resistance,’ the analysis would not be reliable, that is, the same. This individual proposed providing a list of themes that all coders would share before conducting the anal- ysis. To run a quantitative interrater reliability analysis, it is commonly computed as a corre- lation coefficient describing the degree of over- lap between two variables (regardless of what scale of measurement the code comprises). The concern with the two words being the ‘same’ was a quantitative concern posed to a qualitative question. Here a qualitative claim is based on a quantitative rubric: the meanings are themati- cally related but the rubric is a numeric one of the codes used in reliability analysis. The knowledge claim here is one of corresponding magnitudes of evaluations. Judging the reliabil- ity of the responses based on their thematic
coherence instead allows us to recognize their ‘sameness’ while preserving the subtle and nu- anced differences captured in different ways of expressing it highlighting the different perspec- tives that are brought to bear when analyzing qualitative data. The potential of qualitative re- search to discern a complexity of meaning should not be hampered by the quantitative con- cern with reliability as correlation. Reliability in quantitative analysis rests on sameness, repeti- tion; in qualitative research it rests on related- ness (further discussed in Churchill & Wertz, 2002; Garza, 2004, 2007, 2011; Giorgi, 2009). This example presents an opportunity to illumi- nate the challenges that arise when the approach of one research model is applied to the practices of the other. Answering the concern raised here necessitates that we understand the differences in approaches that could be illustrative for prac- titioners in this area to avoid some of the com- mon pitfalls we are addressing.
In a particularly illustrative example, Fredrickson and Losada (2005) adopted formu- las created and suitable for fluid dynamics in physics to explain changes in attitudes over time. Resting on the presumption that attitudes are not only similar to but follow the same laws of nature as fluid, these researchers have not taken into account the differing philosophical approaches that shape both of these phenomena. Quite apart from whether attitudes are a physi- cal ‘thing’ like water for instance, the use and application of these mathematical formulas again highlights the hegemony of quantitative frameworks. Following the critiques raised by Brown, Sokal, and Friedman (2013), the utili- zation of these models can raise serious episte- mological and conceptual concerns.
Another example of hegemony of perspective is raised by Giorgi regarding the practice of some qualitative researchers to ‘verify’ their qualitative interpretative analyses by their par- ticipants or other ‘judges’ (Giorgi, 2008; Pollio, Henley, & Thompson, 1997). Giorgi (2008) as- tutely points out that participants are not versed in either the approach or procedures used for the analysis and thus could not assess its validity.
3 Creswell and Clark (2007) also point to the importance of laying out the philosophical underpinnings of research. However, in the literature, neither quantitative nor qualita- tive research uniformly does this.
202 LANDRUM AND GARZA
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an Ps
yc ho
lo gi
ca l
A ss
oc ia
tio n
or on
e of
its al
lie d
pu bl
is he
rs .
T hi
s ar
tic le
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
Similarly we would add that statistical results would not be verified by the participants be- cause we cannot presume sufficient statistical sophistication to make such a judgment. Al- though it might seem that we are singling out incursions of quantitative into qualitative prac- tice, we suspect this is because the highly spe- cialized language of statistics makes incursions in the other direction less likely; everyone speaks in narratives but not everyone speaks in statistical narratives. In either case, instances of either incursion point to the need for method- ological pluralism.
Counts
Our next concern is the use of counts in qualitative research (see Leech & Onwueg- buzie, 2011; Miles & Huberman, 1994; Sand- elowski, 2001), and there are a number of ‘qual- itative’ articles that include frequency counts of themes and ‘quantitizing’ or assigning a numer- ical value to qualitative data that is then subject to quantitative analysis (see Dutton & Win- stead, 2011; Sandelowski et al., 2009). It would be a mistake to equate frequency with impor- tance or worse yet to conduct statistical analysis with these counts as in what Sandelowski (2001) calls “acontextual counting.” Examples could include counting up the number of times a particular word is said taken to imply a greater importance of that dimension of meaning in that person’s life. Often what an individual does not say is just as revealing and important as what they do say and when counting something, this ‘absence’ is not taken into account. In a thesis workshop for senior undergraduates conducting phenomenological research, a participant pro- vided a description of losing her virginity and the most striking part was that she never men- tioned the partner once in the entire description (Garza, 2004, Spring). Here, the lack of any mention of the other party involved reveals much about this phenomenon as meaningfully lived by the participant. We argue that as soon as one begins to count themes, one is no longer conducting qualitative research and not really conducting quantitative research either. This, in our minds, fails to respect the proper domains for both types of research.
Another example of a heightened concern with numbers in qualitative research is what Sandelowski (2001) refers to as “analytic over-
counting.” This refers to the tendency by some qualitative researchers to count everything that could possibly be counted to the detriment of clear presentation of the qualitative findings. Examples of this include a focus on the precise number of themes identified whereby the actual count is given greater emphasis than a descrip- tion of the themes themselves. Sometimes even when patterns of meanings comprise the results, Sandelowski reports that researchers become preoccupied with the number of participants who exhibit the themes where the focus is on frequency and less so on the meaning of the themes or patterns. All of these examples point to the need for researchers to be mindful of the type of data being gathered and the analytic approach undertaken paying particular attention to the appropriate knowledge claims.
Confirmation and Validation
The practice of ‘(dis)confirming’ and ‘(non-)validating’ one set of findings with another set when the data and interpretive frameworks are not matched is widespread (see Ellis, Marsh, & Craven, 2009; Hastings, 2012; Riegel, Dickson, Kuhn, Page, & Wor- rall-Carter, 2010; Sechrist, Suitor, Riffin, Taylor-Watson, & Pillemer, 2011 for exam- ples). In all of these examples, quantitative and qualitative data are used to explicitly ‘confirm’ and ‘verify’ each other and to as- sess ‘concordance’ of findings.
Wagner et al. (2012) argue against ‘confir- mation’ of findings rooted in one approach by research rooted in the other because conflicting results might initially appear problematic. If we examine the hippocampus from a neurophysio- logical point of view and find there are differ- ences between those (including animal species) who hoard and those who do not hoard (see, e.g., Brodin & Lundborg, 2003; Hampton, Sherry, Shettleworth, Khurgel, & Ivy, 1995; Volman, Grubb, & Schuett, 1997), it would not be appropriate to use qualitative data to confirm differences in the hippocampus. On the other hand, would the hippocampal findings confirm differences in memory found in qualitative data? Although these two sets of findings from two approaches shed light on each other, we do not believe one can confirm the other without implicitly holding that one type of data is more valid and thus the basis for such confirmation. If
203QUANTITATIVE AND QUALITATIVE DOMAINS
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an Ps
yc ho
lo gi
ca l
A ss
oc ia
tio n
or on
e of
its al
lie d
pu bl
is he
rs .
T hi
s ar
tic le
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
hippocampal volume was found to be larger in those who hoard, survey data that revealed the importance of memory would not be surprising. But it would no more ‘confirm’ the findings related to hippocampal volume than a German translation of Shakespeare could confirm the Chinese translation; the point here is that a researcher must understand the differences in the languages used. The increase in hippocam- pal volume and the importance of memory are two complementary findings: they neither con- firm each other nor disaffirm each other. To- gether, they expand our understanding of the role of memory in those who hoard.
Another example of this practice of ‘confir- mation’ and ‘validation’ arises when attempting to interpret percentage of concordance between qualitative and quantitative findings. In a mixed methods study examining self care behaviors among patients with heart failure, Riegel et al. (2010) computed the percentage of agreement between identification of a self care theme in the participants’ narratives with a cutoff score on a quantitative survey. Although the two research- ers independently analyzed the two types of data respectively, the operationalization of self care has already been defined in advance with the use of a quantitative survey and the calcu- lation of ‘concordance’ rates presumes that the lived experiences provided through the narra- tives will touch on the same points raised by the survey items and vice versa. Furthermore, the concordance rates are taken to be an indication that the quantitative and qualitative methods are more valid and thus more trustworthy if a higher rate of concordance is reached. However, it is not immediately clear what this percentage of agreement means; for instance if self care main- tenance reached 75% agreement but self care confidence reached 95% agreement, what does the 20% difference mean? Assuming 100 par- ticipants in the sample, this difference would precisely mean that 20 more people provided evidence of this theme in their narratives and circled a higher number on the survey. Can this increase indicate that one piece of data is more valid? We suggest not because the validity of either quantitative or qualitative methods rests upon the respective philosophical approach un- dergirding both types of methods and that using one method cannot ‘confirm’ or ‘validate’ find- ings in the other. This practice renders the qual- itative data into a dimension of magnitude again
marking implicit adherence to a quantitative frame of reference. Additionally, this practice rests on the presumption that the number one circles for a group of items operationalized to measure a phenomenon will coincide with a description or narrative provided by the partic- ipants. How one narrates one’s experiences may or may not match with a list of items on this topic and one of the benefits of conducting mixed methods would be to examine this pos- sibility. However, holding this presumption of similarity across two types of data collected shuts down the possibility of examining this dimension when the goal is to assess ‘concor- dance.’ What these researchers have rightly dis- cerned is that there are similarities here as well as a relationship between these two methods; however, similarity has both qualitative and quantitative dimensions, and a change in one does not necessarily map onto a manifestation in both types. As Rollo May points out when describing the differences between memory ca- pacities in humans and sheep, a difference in terms of length of time or other quantitative distinctions also imply quality differences but given the distinct interpretative frames of refer- ence, these two changes cannot be assumed to be ‘the same’ (May, 1979). When these two methods are used to validate each other or to usurp one by the other, the strength of multiple perspectives is diminished and eradicates the possibility of exploring amplification, differ- ences, similarities, and so forth when both types of data are viewed from one perspective and thus conflated.
Likewise, we contend that neither method can be used to confirm or disconfirm the other. Instead, we suggest that the frame of reference here is ‘augmentation.’ Consider the ‘mountain’ task used to assess developmental egocentrism as an analogy here; a child sits at a square table with a three-dimensional mountain and is asked to describe the mountain from various view- points. The egocentric child cannot discern how a viewer sitting on the other three sides of the table would see anything different from what he or she sees from his or her own perspective. He or she might even be perplexed by the fact that such an observer could see something ‘at odds’ with what he or she sees. Similarly an ‘ap- proach-centric’ researcher might seek ‘confir- mation’ of his or her own perspective when conducting mixed methods. We argue that a
204 LANDRUM AND GARZA
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an Ps
yc ho
lo gi
ca l
A ss
oc ia
tio n
or on
e of
its al
lie d
pu bl
is he
rs .
T hi
s ar
tic le
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
methodologically pluralistic researcher would see that the complementary perspectives of other approaches, need not ‘confirm’ their own perspectival view but augment it, providing a more complex and full description of the phe- nomenon being investigated. Rather than have convergence or agreement as a goal of mixed methods, we advocate for complementarity in mixed methods.
Mixing Methods as Complementarity of Strengths: The Case for Methodological
Pluralism
Both quantitative and qualitative research methods are limited in scope assessing dimen- sions of meaning and magnitude, respectively (Giorgi, 2009). But it is perhaps more fruitful to think of these limits as domains of strength. These domains describe the frontiers of the two research models and set the stage for a comple- mentarity of strengths whereby our understand- ing of the phenomena we research is more com- plete in view of the differences than that proffered by ‘verification’ or ‘confirmation.’ Complementarity requires that research con- ducted from any point along the continuum (a) acknowledge the differences between the ap- proaches, (b) show respect for these differences, and (c) possess a mindfulness that the ‘middle ground’ we have described comprises complex intersections of knowledge claims, epistemo- logical assumptions, and approach. We advo- cate that no position on this continuum is priv- ileged and that methodological plurality allows researchers to more fully describe a phenome- non across this full continuum generating a wide array of knowledge.
In the recent Best Practices for Mixed Meth- ods Research in the Health Sciences, Creswell et al. (2011) describe three types of integrating qualitative and quantitative data. Two of the three types, connecting and embedding data, respect the boundaries of the two domains whereby one type of inquiry informs the other type of inquiry at a subsequent or concurrent time, respectively. The other type of integration, merging data, mixes up the ‘messy middle’ by using one type of data to compare and/or con- firm the findings from the other type.
When connecting data, one type of data anal- ysis is used to inform the collection of a second type of data at a subsequent time point. In this
way, the data gathered are analyzed using the methods appropriate to the type of data gath- ered. In our own mixed methods research ex- ample below, our qualitative analysis illumi- nated a transformed meaning of home that suggests an additional variable to examine in future quantitative research. The connecting process does not violate the boundaries as the type of data gathered (numeric vs. non-numeric) is appropriately analyzed (quantitative vs. qual- itative, respectively), enabling the two ap- proaches to mutually shed light on each other while neither confirming nor validating one ap- proach over the other.
Likewise in the embedding data method, one type of data analysis is deemed primary and the other as secondary. The primary method is cho- sen appropriately given the type of data being collected while the secondary method is chosen for supplemental and illuminating purposes. Like the connecting process above, the embed- ded process does not violate the boundaries between approaches.
However, the merging process can violate the boundaries we have outlined above. In this pro- cess, a researcher can transform a piece of qual- itative data into counts that are then subject to quantitative analyses. In our view, this violates a fundamental difference in the two approaches; namely the non-numeric qualitative data when transformed into number of times a theme is mentioned departs from a dimension of mean- ing (i.e., importance) and transforms it into the currency of magnitude (i.e., counts). This merg- ing process calls into question the boundaries that divide these two approaches and the differ- ent currencies that each trade in. As argued above, the act of using one type of approach to validate or confirm the other neglects how each has its own language, understanding, and phil- osophical foundations. However, this does not mean that the two approaches cannot be used concurrently in one research project. Rather than confirming or validating, where one ap- proach is more highly valued, we feel that when both approaches and domains are respected, the two types of results can shed light and illumi- nate the subject matter as well as provide a greater understanding than either approach could on their own.
Kendler (2005) argues that methodological plurality would create confusion and contradic- tion and argues for a strict natural science ap-
205QUANTITATIVE AND QUALITATIVE DOMAINS
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an Ps
yc ho
lo gi
ca l
A ss
oc ia
tio n
or on
e of
its al
lie d
pu bl
is he
rs .
T hi
s ar
tic le
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
proach to psychological research utilizing the methods of quantitative research. To our minds this is akin to saying that a meal could be accurately described either by a list of its ingre- dients or by the subjective experience of its deliciousness but not both; reporting both would be ‘confusing’ or ‘contradictory.’ De- spite these claims that the two types of research are incompatible (Kendler, 2005), we have il- lustrated that the goal of both types of research is to gain a more complete understanding of the phenomenon under investigation. Rather than rely on a ‘monomethod,’ we suggest that meth- odological plurality allows researchers to draw on the strengths of both quantitative and quali- tative research.
As a case in point, Trend (1979 as cited in Teddlie & Tashakkori, 2009) explored program implementation and found discrepant results in the quantitative and qualitative analyses. Spe- cifically, when examining the quantitative data, the program was rated positively across sites and it appeared successful. However the quali- tative data provided the researchers with a qual- itative impression that the implementation of the program was not successful and problems were encountered at the various sites. When attempting to reconcile these apparent differ- ences, the researchers discovered that a contex- tual variable, the site’s urban versus rural loca- tion, could account for the discrepancy revealing that dimensions of meaning associ- ated with this distinction in terms of costs, in- come of families, ethnicity, ease of recruitment, among others, revealed further nuances in the quantitative findings. By examining the qualita- tive data gathered for implementation of the program at each site rather than collapsing across sites as the initial quantitative analysis did, the researchers used both types of data gathered to augment each other and to illumi- nate the contextual factors specific to each pro- gram. Only when both data and thus both anal- yses were incorporated and examined together could the findings give a more comprehensive picture of implementation. This example illumi- nates how posing a qualitative question could lead to a reevaluation of a quantitative analysis providing further insights that would not have been possible if only one method had been applied.
Another example of the appropriately com- plementary relationship of quantitative and
qualitative analysis in mixed methods research is a study we conducted on Facebook usage and its relationship to satisfaction with college life. (Landrum & Garza, 2011). We began with a quantitative study by asking our participants to report on their Facebook (FB) usage and gath- ered measures of social capital among other measures of demographic and college experi- ence. We tested a Structural Equation Model (SEM) and found that when heavy users of FB were connecting with friends from high school, they reported less satisfaction with college life when compared to students who were connect- ing with fellow students and classmates at col- lege illuminating dimensions of magnitude. This quantitative finding suggested a fruitful avenue to explore dimensions of meaning, what FB means to them. Our structured interview focus group analysis revealed a theme that could not emerge from the quantitative analysis as we had conceived it. Our spontaneous inter- action with participants and open-ended analy- sis allowed us to discern that for some students the meaning of home had transformed from their parent’s home to their college residence. This qualitative finding shed new light on our interpretation of the SEM model suggesting that it was not so much how often students used FB but rather how they were using FB, whether they were connecting with those in their current milieu and past milieu and which of these mi- lieus was understood by participants as their home. This opens a whole new avenue of re- search of both kinds. The benefits of truly col- laborative mixed methods cannot occur when each or either model is corrupted to the pur- poses of the other. In both of these examples, the relationship between the two approaches is not one of confirmation or validation but of augmentation. Just as describing the mountain scene from two sides of the table yields a more comprehensive description, the full potential of mixed methods research becomes possible when the boundaries are respected, the strengths honored, and the two models are thus mutually and truly complementary across the entire con- tinuum of research approaches.
Like all who sojourn beyond their homes, methodological adventurers would be well ad- vised to learn the language and customs of the domains they visit. The necessity of this only comes to light when one recognizes that a fron- tier has been crossed. To achieve a truly appro-
206 LANDRUM AND GARZA
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an Ps
yc ho
lo gi
ca l
A ss
oc ia
tio n
or on
e of
its al
lie d
pu bl
is he
rs .
T hi
s ar
tic le
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
priate balance between quantitative and qualita- tive research methods as well as mixing the two approaches, we recommend methodological pluralism. Envisioned as a sort of methodolog- ical multiculturalism, we are calling for other researchers in the field to join this discussion and engage in dialogue with each other. We argue that together, quantitative and qualitative approaches are stronger and provide more knowledge and insights about a research topic than either approach alone. While both ap- proaches shed unique light on a particular re- search topic, we suggest that methodologically pluralistic researchers would be able to ap- proach their interests in such a way as to reveal new insights that neither method nor approach could reveal alone. When both quantitative and qualitative researchers reach out to each other across the fence, learn the language, and respect the boundaries outlined above, we can start to make great strides in the emerging field. Only when both sides understand and respect the domains can the differences and uniqueness of both approaches be appreciated.
References
Agresti, A. (2002). Categorical data analysis (2nd ed.). Hoboken, NJ: Wiley, John & Sons, Inc. http:// dx.doi.org/10.1002/0471249688
Brodin, A., & Lundborg, K. (2003). Is hippocampal volume affected by specialization for food hoard- ing in birds? Proceedings of the Royal Society of London B: Biological Sciences, 270, 1555–1563. http://dx.doi.org/10.1098/rspb.2003.2413
Brown, N. J. L., Sokal, A. D., & Friedman, H. L. (2013). The complex dynamics of wishful think- ing: The critical positivity ratio. American Psy- chologist, 68, 801–813. http://dx.doi.org/10.1037/ a0032850
Carifio, J., & Perla, R. (2008). Resolving the 50-year debate around using and misusing Likert scales. Medical Education, 42, 1150–1152. http://dx.doi .org/10.1111/j.1365-2923.2008.03172.x
Churchill, S. D., Lowery, J. E., McNally, O., & Rao, A. (1998). The question of reliability in interpre- tive psychological research: A comparison of three phenomenologically based protocol analyses. In R. Valle (Ed.), Phenomenological inquiry in psychol- ogy: Existential and transpersonal dimensions (pp. 63–85). New York, NY: Plenum Press. http://dx .doi.org/10.1007/978-1-4899-0125-5_3
Churchill, S. D., & Wertz, F. J. (2002). An introduc- tion to phenomenological research psychology: Historical, conceptual, and methodological foun-
dations. In K. J. Schneider, J. F. T. Bugental, & J. F. Pierson (Eds.), The handbook of humanistic psychology: Leading edges in theory, research, and practice (pp. 247–262). Thousand Oaks, CA: Sage.
Cialdini, R. B., Borden, R. J., Thorne, A., Walker, M., Freeman, S., & Sloan, L. (1976). Basking in reflected glory: Three (football) field studies. Jour- nal of Personality and Social Psychology, 34, 366–375. http://dx.doi.org/10.1037/0022-3514.34 .3.366
Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Thousand Oaks, CA: Sage.
Creswell, J. W., & Clark, V. (2007). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.
Creswell, J. W., Klassen, A. C., Plano Clark, V. L., & Smith, K. C. (2011). Best practices for mixed meth- ods research in the health sciences. Bethesda, MD: National Institutes of Health, Office of Behavioral and Social Sciences Research. Retrieved from http:// obssr.od.nih.gov/mixed_methods_research
Dutton, L. B., & Winstead, B. A. (2011). Types, frequency, and effectiveness of responses to un- wanted pursuit and stalking. Journal of Interper- sonal Violence, 26, 1129–1156. http://dx.doi.org/ 10.1177/0886260510368153
Ellis, L. A., Marsh, H. W., & Craven, R. G. (2009). Addressing the challenges faced by early adoles- cents: A mixed-method evaluation of the benefits of peer support. American Journal of Community Psychology, 44(1–2), 54 –75. http://dx.doi.org/ 10.1007/s10464-009-9251-y
Fredrickson, B. L., & Losada, M. F. (2005). Positive affect and the complex dynamics of human flour- ishing. American Psychologist, 60, 678 – 686. http://dx.doi.org/10.1037/0003-066X.60.7.678
Frost, R. (1919). Mending wall. In L. Untermeyer (Ed.), Modern American poetry. New York, NY: Harcourt, Brace and Howe. Retrieved from http:// www.bartleby.com/104/64.html
Garza, G. (2004). Thematic moment analysis: A di- dactic application of a procedure for phenomeno- logical analysis of narrative data. The Humanistic Psychologist, 32, 120 –168. http://dx.doi.org/ 10.1080/08873267.2004.9961749
Garza, G. (2004, Spring). Senior qualitative research workshop: Senior thesis. Lecture conducted from University of Dallas, Irving, TX.
Garza, G. (2006). A clarification of Heidegger’s phe- nomenology. American Psychologist, 61, 255– 256. http://dx.doi.org/10.1037/0003-066X.61.3 .255
Garza, G. (2007). Varieties of phenomenological research at the University of Dallas: An emerging typology. Qualitative Research in Psychology, 4, 313–342. http:// dx.doi.org/10.1080/14780880701551170
207QUANTITATIVE AND QUALITATIVE DOMAINS
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an Ps
yc ho
lo gi
ca l
A ss
oc ia
tio n
or on
e of
its al
lie d
pu bl
is he
rs .
T hi
s ar
tic le
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
Garza, G. (2011). Thematic collation: An illustrative analysis of the experience of regret. Qualitative Research in Psychology, 8, 40–65. http://dx.doi .org/10.1080/14780880903490839
Giorgi, A. (1970). Psychology as a human science: A phenomenologically based approach. New York, NY: Harper & Row.
Giorgi, A. (2008). Difficulties encountered in the application of the phenomenological method in the social sciences. Indo-Pacific Journal of Phe- nomenology, 8. Retrieved from http://www.ipjp .org/index.php?option�com_jdownloads& view�viewdownload&catid�33&cid�124&Itemid� 318
Giorgi, A. (2009). The descriptive phenomenological method in psychology. Pittsburgh, PA: Duquesne University Press.
Greener, I. (2011). Designing social research: A guide for the bewildered. Thousand Oaks, CA: Sage.
Hampton, R. R., Sherry, D. F., Shettleworth, S. J., Khurgel, M., & Ivy, G. (1995). Hippocampal vol- ume and food-storing behavior are related in parids. Brain, Behavior and Evolution, 45, 54–61. http://dx.doi.org/10.1159/000113385
Hastings, L. J. (2012). Generativity in young adults: Comparing and explaining the impact of mentor- ing (Doctoral dissertation, Paper 84, Educational Administration: Theses, Dissertations, and Stu- dent). Retrieved from http://digitalcommons.unl .edu/cehsedaddiss/84
Hesse-Biber, S. N. (2010). Mixed methods research: Merging theory with practice. New York, NY: Guilford Press.
Jamieson, S. (2004). Likert scales: How to (ab)use them. Medical Education, 38, 1217–1218. http:// dx.doi.org/10.1111/j.1365-2929.2004.02012.x
Johnson, R. W., & den Heyer, K. (1980). On the enduring untruth about measurement and paramet- ric statistics. Canadian Psychology/Psychologie canadienne, 21, 134 –135. http://dx.doi.org/ 10.1037/h0081084
Kendler, H. H. (2005). Psychology and phenomenol- ogy: A clarification. American Psychologist, 60, 318–324. http://dx.doi.org/10.1037/0003-066X.60 .4.318
Knapp, T. R. (1990). Treating ordinal scales as in- terval scales: An attempt to resolve the contro- versy. Nursing Research, 39, 121–123. http://dx .doi.org/10.1097/00006199-199003000-00019
Landrum, B. & Garza, G. (2011, April). Retention and Predictors of Satisfaction with College Life. Paper presented at the 57th Meeting of the South- western Psychological Association, San Antonio, TX.
Leech, N. L., & Onwuegbuzie, A. J. (2011). Beyond constant comparison qualitative data analysis: Us-
ing NVivo. School Psychology Quarterly, 26, 70– 84. http://dx.doi.org/10.1037/a0022711
May, R. (1979). Psychology and the human dilemma. New York, NY: W.W. Norton & Co.
Merenda, P. F. (n.d.). Common errors of omission and commission observed in proposals, theses, and dissertations, 1965–1985. Retrieved from https:// www.yumpu.com/en/document/view/12904682/ common-errors-in-analysis-and-writing-amsci- ammons-scientific-
Merleau-Ponty, M. (1962). The phenomenology of perception (C. Smith, Trans.). Mahwah, NJ: The Humanities Press. (Original work published 1945)
Merleau-Ponty, M. (1964). The phenomenology of perception (J. Wild, Trans.). Chicago, IL: North- western University Press. (Original work pub- lished 1961)
Miles, M. B., & Huberman, A. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: Sage.
Norman, G. (2010). Likert scales, levels of measure- ment and the “laws” of statistics. Advances in Health Sciences Education, 15, 625–632. http://dx .doi.org/10.1007/s10459-010-9222-y
Pollard, S. E., Nievar, M. A., Nathans, L. L., & Riggs, S. A. (2014). A comparison of White and Hispanic women’s stories of adjustment to the birth of a child. Infant Mental Health Journal, 35, 193–209. http://dx.doi.org/10.1002/imhj.21437
Pollio, H. R., Henley, T. B., & Thompson, C. J. (1997). The phenomenology of everyday life: Em- pirical investigations of human experience. Cam- bridge, UK: Cambridge University Press. http://dx .doi.org/10.1017/CBO9780511752919
Riegel, B., Dickson, V. V., Kuhn, L., Page, K., & Worrall-Carter, L. (2010). Gender-specific barriers and facilitators to heart failure self-care: A mixed methods study. International Journal of Nursing Studies, 47, 888–895. http://dx.doi.org/10.1016/j .ijnurstu.2009.12.011
Sandelowski, M. (2001). Real qualitative researchers do not count: The use of numbers in qualitative research. Research in Nursing & Health, 24, 230– 240. http://dx.doi.org/10.1002/nur.1025
Sandelowski, M., Voils, C. I., & Knafl, G. (2009). On quantitizing. Journal of Mixed Methods Re- search, 3, 208 –222. http://dx.doi.org/10.1177/ 1558689809334210
Sechrist, J., Suitor, J. J., Riffin, C., Taylor-Watson, K., & Pillemer, K. (2011). Race and older mothers’ differentiation: A sequential quantitative and qual- itative analysis. Journal of Family Psychology, 25, 837–846. http://dx.doi.org/10.1037/a0025709
Tashakkori, A., & Teddlie, C. (Eds.). (2003). Hand- book of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage.
Tashakkori, A., Teddlie, C., & Sines, M. C. (2013). Utilizing mixed methods in psychological re-
208 LANDRUM AND GARZA
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an Ps
yc ho
lo gi
ca l
A ss
oc ia
tio n
or on
e of
its al
lie d
pu bl
is he
rs .
T hi
s ar
tic le
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
search. In J. A. Schinka, W. F. Velicer, & I. B. Weiner (Eds.), Handbook of psychology: Vol. 2. Research methods in psychology (2nd ed., pp. 428–450). Hoboken, NJ: Wiley.
Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research. Thousand Oaks, CA: Sage.
Volman, S. F., Grubb, T. C., Jr., & Schuett, K. C. (1997). Relative hippocampal volume in relation to food-storing behavior in four species of wood- peckers. Brain, Behavior and Evolution, 49, 110– 120. http://dx.doi.org/10.1159/000112985
von Eckartsberg, R. (1998). Introducing existential- phenomenological psychology. In R. Valle (Ed.), Phenomenological inquiry in psychology: Existen- tial and transpersonal dimensions (pp. 3–20). New York, NY: Plenum Press. http://dx.doi.org/ 10.1007/978-1-4899-0125-5_1
Wagner, K., Davidson, P., Pollini, R., Strathdee, S., Washburn, R., & Palinkas, L. (2012). Reconciling
incongruous qualitative and quantitative findings in mixed methods research: Exemplars from re- search with drug using populations. International Journal of Drug Policy, 23, 54–61. http://dx.doi .org/10.1016/j.drugpo.2011.05.009
Wertz, F. (1985). Method and findings in a phenom- enological psychological study of a complex life event: Being criminally victimized. In A. Giorgi (Ed.), Phenomenology and psychological research (pp. 155–216). Pittsburgh, PA: Duquesne Univer- sity Press.
Wertz, F. J. (2011). The qualitative revolution and psychology: Science, politics, and ethics. The Hu- manistic Psychologist, 39, 77–104. http://dx.doi .org/10.1080/08873267.2011.564531
Received November 10, 2013 Revision received March 23, 2015
Accepted June 30, 2015 �
209QUANTITATIVE AND QUALITATIVE DOMAINS
T hi
s do
cu m
en t
is co
py ri
gh te
d by
th e
A m
er ic
an Ps
yc ho
lo gi
ca l
A ss
oc ia
tio n
or on
e of
its al
lie d
pu bl
is he
rs .
T hi
s ar
tic le
is in
te nd
ed so
le ly
fo r
th e
pe rs
on al
us e
of th
e in
di vi
du al
us er
an d
is no
t to
be di
ss em
in at
ed br
oa dl
y.
