Research methods unit VII web assignment

C O M M E N T A R Y

Best practices in mixed methods for quality of life research

Ann C. Klassen • John Creswell • Vicki L. Plano Clark •

Katherine Clegg Smith • Helen I. Meissner

Accepted: 17 January 2012 / Published online: 4 February 2012

� Springer Science+Business Media B.V. 2012

Introduction

There is a growing priority in all areas of health research to

develop new methodologies to improve the quality and

scientific power of data, and this is leading to an extraor-

dinary surge in methodological diversity. This diversity

reflects the nature of the problems facing health sciences

and health care delivery, such as disparities among popu-

lations, age groups, ethnicities, and cultures; poor adher-

ence to recommended treatments; behavioral risk factors

contributing to disability and health; and the translation of

research findings into applied settings. The diversity in

methodology also signals a growing acceptance of behav-

ioral and social science perspectives in clinical research,

the formation of interdisciplinary research teams, and use

of multi-faceted approaches. Such approaches are impor-

tant to investigations of complex health problems, which

call for incorporating patient and family point of view, and

cultural models of illness and health.

Contributing to this interest in methodological develop-

ment has been the increased methodological sophistication

of mixed methods research, and practices related to com-

bining quantitative and qualitative research. Researchers

are using approaches such as in-depth interviews, field

observations, and patient records to understand individual

experiences, participant involvement in interventions, and

barriers to and facilitators of treatment. These qualitative

approaches are often combined with data from clinical tri-

als, surveys of attitudes and beliefs, economic or medical

data to better understand health problems [1]. Evidence in

the published literature attests to the current use of mixed

methods approaches in health-related research, from car-

diology [2], pharmacy [3], family medicine [4], pediatric

oncology nursing [5], mental health [6, 7], disabilities [8]

and nutrition [9], in both clinical settings [10] and in the

social context of daily activities and relationships [11].

Scientists and clinicians working in the area of quality

of life broadly, and more specifically in health outcomes

assessment, have found mixed methods to be increasingly

important for both theoretical and methodological reasons.

Quality of life researchers often examine questions that

have multiple epistemological, scientific, and clinical foci

and are faced with integrating diverse perspectives, types

of evidence, and audiences or stakeholders. Data may

range from biological data from a patient’s clinical record,

to health care delivery indicators and costs, to household

and community-level outcomes such as loss of productiv-

ity, and regional or national policies. The journal Quality of

Life Research has a long-standing commitment to pub-

lishing high-quality research that brings both qualitative

A. C. Klassen (&) Drexel University School of Public Health,

Philadelphia, PA, USA

e-mail: [email protected]

J. Creswell

John University of Nebraska, Lincoln, NE, USA

e-mail: [email protected]

V. L. Plano Clark

University of Nebraska, Lincoln, NE, USA

e-mail: [email protected]

K. C. Smith

Johns Hopkins Bloomberg School of Public Health,

Baltimore, MD, USA

e-mail: [email protected]

H. I. Meissner

Office of Behavioral and Social Science Research National

Institutes of Health, Bethesda, MD, USA

e-mail: [email protected]

123

Qual Life Res (2012) 21:377–380

DOI 10.1007/s11136-012-0122-x

and mixed methodologies to bear on these complex and

multi-faceted research questions.

In their 2010 editorial in Quality of Life Research, Ring

and colleagues [12] noted the growing use of qualitative

methods to capture ‘‘the subtlety and distinctions experi-

enced by patients’’ and discussed the growing use of both

quantitative and qualitative methods to capture the com-

plexity of quality of life assessment. However, they also

note the need for methodological rigor, and therefore the

development of sufficient numbers of well-informed

teachers, mentors, and collaborators, as well as journal and

grant reviewers.

A study of funded NIH investigations revealed a dramatic

increase of terms such as ‘‘mixed methods’’ or ‘‘multi-

methods’’ in their abstracts since 1996 [1]. However, despite

the expanding interest in mixed methods research, no

guidelines for ‘‘best practices’’ existed to assist scientists

developing applications for funding or to aid reviewers

assessing the quality of mixed methods investigations. In

November 2010, The Office of Behavioral and Social Sci-

ences Research (OBSSR) of the National Institutes of Health

(NIH) commissioned the development of a resource that

would provide guidance to NIH investigators on how to

rigorously develop and evaluate mixed methods research

applications, as well as to guide peer review, and program

initiatives at NIH to maximize the contribution of mixed

methods in health research. This review summarizes key

recommendations from ‘‘Best Practices for Mixed Methods

Research in the Health Sciences’’, available at http://obssr.

od.nih.gov/scientific_areas/methodology/mixed_methods_

research.

Purpose

The guidelines are framed with a definition of mixed

methods as a research approach or methodology (1)

focusing on research questions that call for real-life con-

textual understandings, multi-level perspectives, and cul-

tural influences, (2) employing rigorous quantitative

research assessing magnitude and frequency of constructs

and rigorous qualitative research exploring the meaning

and understanding of constructs, (3) utilizing multiple

methods (e.g., intervention trials and in-depth interviews),

and (4) intentionally integrating or combining these

methods to draw on the strengths of each. Mixed methods

researchers use and often make explicit diverse philoso-

phies of science, from the strictly positivist perspectives

common in the biological and natural sciences to the more

post-positivist or constructivist perspectives of many of the

social and behavioral sciences. Researchers who hold dif-

ferent philosophical positions may find mixed methods

research to be challenging because of the tensions created

by their different beliefs [13], but this may also represent

an opportunity to transform these tensions into new

knowledge, through the integration of a variety of theo-

retical perspectives.

Mixed methods research begins with the assumption that

investigators, in understanding the social and health

worlds, gather evidence based on the nature of the question

and theoretical orientation, with inquiry targeted toward

various sources and many levels that influence a given

problem (e.g., policies, organizations, family, individual).

Quantitative (mainly deductive) methods are ideal for

measuring pervasiveness of ‘‘known’’ phenomena and

central patterns of association, including inferences of

causality. Qualitative (mainly inductive) methods allow for

identification of previously unknown processes, explana-

tions of why and how phenomena occur, and the range of

their effects. Mixed methods research, then, is more than

simply collecting multiple forms of qualitative evidence

(e.g., observations and interviews) or quantitative evidence

(e.g., surveys and diagnostic tests). It involves the inten-

tional collection of both quantitative and qualitative data

and the combination of the strengths of each to answer

research questions.

In mixed methods studies, investigators intentionally

integrate or combine qualitative and quantitative data, to

maximize the strengths and minimize the weaknesses of

each. This idea of integration distinguishes current views of

mixed methods from older perspectives in which investiga-

tors collected both forms of data, but kept them separate or

casually combined them rather than using systematic inte-

grative procedures. ‘‘Meta-inference’’ is the term used to

describe the purposeful consideration of the total evidence

about the questions of interest, provided by both types of

data, as well as the combined analyses [13]. Meta-inference

may identify contradictory as well as confirmatory ele-

ments of the evidence, and lead to new understanding of the

phenomena under study.

The use of mixed methods is most suitable when a

quantitative or qualitative approach, by itself, is inade-

quate to develop multiple perspectives and a complete

understanding about a research problem or question.

Researchers may seek to view problems from multiple

perspectives to enhance and enrich the meaning of a sin-

gular perspective. They also may want to contextualize

information, to take a macro picture of a system (e.g., a

hospital) and add in information about individuals (staff or

patients). Other reasons include to merge quantitative and

qualitative data to develop a more complete understanding

of a problem; to develop a complementary picture; to

compare, validate, or triangulate results; to provide illus-

trations of context for trends; to examine processes/expe-

riences along with outcomes; or to have one database

build on another.

378 Qual Life Res (2012) 21:377–380

123

Design and methods

There is no rigid formula for designing a mixed methods

study, but the following general steps should provide some

guidance, especially for an investigator new to mixed

methods. Preliminary considerations include considering

philosophy and theory, resources (e.g., time, financial

resources, skills), and the research problem and reasons for

using mixed methods.

Clarification of study aims and research questions that

call for qualitative, quantitative, and mixed methods is

important, to incorporate these into the reasons for con-

ducting a mixed methods study. It is also critical to

determine the methods of quantitative and qualitative data

collection and analysis (when it will be collected, what

emphasis will be given to each, and how they will be

integrated or mixed), and select a mixed methods design

that helps address research questions and the data collec-

tion/analysis/integration procedures. After collecting and

analyzing the data, meta-inference allows the researcher to

interpret how the combined quantitative and qualitative

approaches contribute to addressing the research problem

and questions, and to report findings while making explicit

the contribution of the mixed methods approach.

Basic considerations:

• Theoretical and conceptual orientation: The choice of a mixed methods design should be informed by one

or more theoretical and conceptual orientation(s) that

supports the overarching science and needs of the

study.

• Fixed versus emergent mixed methods designs: In a fixed design, the methods are predetermined at the

outset, because the investigators have made the specific

decision to mix qualitative and quantitative approaches.

In an emergent (or cyclical) design, the methods

emerge during the process of the research.

• Timing and analytical logic: Qualitative and quanti- tative data may be collected concurrently, which may

be attractive in studies where time in the field is costly,

or limited due to a time-sensitive phenomenon of

interest. Alternatively, a sequential approach may be

useful for single investigators who have ample time to

stretch data collection over a lengthened period, or if

results from an initial phase inform a subsequent phase.

• Priority: In some mixed methods studies, the quanti- tative and qualitative research is equally emphasized. In

other studies, priority is given to either the quantitative

or the qualitative research.

• Point of interface: The ‘‘point of interface,’’ or the point where mixing occurs, differs depending on the

mixed methods design [14]. This ‘‘point’’ may occur

during data collection (e.g., when both quantitative

items and qualitative open-ended questions are col-

lected on the same survey), during data analysis (e.g.,

when qualitative data are converted or transformed into

quantitative scores or constructs to be compared with a

quantitative dataset), and/or during data interpretation

(e.g., when results of quantitative analyses are com-

pared with themes that emerge from the qualitative

analysis).

Mixed methods designs

There are three basic types of mixed methods designs

[15], but more complex designs are commonplace and are

driven by the specific questions and aims in the particular

investigations.

• Convergent (or parallel or concurrent) designs are used when the intent is to merge concurrent quantitative and

qualitative data to address study aims, the data analysis

consists of merging data which are collected concur-

rently, and comparing the two sets of data and results.

• Sequential (or explanatory sequential or exploratory sequential) designs allow one data collection activity to

build on the results from the other. Qualitative data may

be collected to help to explain in more depth the

mechanisms underlying the quantitative results. Con-

versely, initial exploratory qualitative data collection

and findings may be used to design a quantitative

instrument for use with a larger population.

• Embedded (or nested) designs use quantitative and qualitative approaches in tandem and embed one in the

other to provide new insights or more refined thinking.

For example, in-depth interviews could be embedded

within an intervention to understand how experimental

participants experience the treatment.

Issues and special considerations

In mixed methods research, methodological and logistical

issues arise that need to be anticipated, including resources.

Because multiple forms of data are being collected and

analyzed, mixed methods research requires extensive time

and resources to carry out the multiple steps involved in

mixed methods research, including the time required for

data collection and analysis. In teamwork, different

approaches as well as different analytical or writing styles

might emerge. Leaders need to anticipate the challenges

and benefits of a team approach to mixed methods

research, and the ‘‘Best Practices’’ contains a section spe-

cifically on building a mixed methods research team.

Qual Life Res (2012) 21:377–380 379

123

Challenges specific to concurrent designs (i.e., merging

quantitative and qualitative research) include having ade-

quate sample sizes for analyses, using comparable samples,

and employing a consistent unit of analysis across the

databases. For sequential designs (i.e., one phase of qual-

itative research builds on the quantitative phase or vice

versa), the issues relate to deciding what results from the

first phase to use in the follow-up phase, choosing samples

and estimating reasonable sample sizes for both phases,

and interpreting results from both phases.

Issues arise during data analysis and interpretation when

using specific designs. When the investigator merges the

data during a concurrent design, the findings may conflict

or be contradictory. A strategy of resolving differences

needs to be considered, such as gathering more data or

revisiting the databases. For designs involving a sequential

design with one phase following the other, the key issues

surround the ‘‘point of interface’’ in which the investigator

needs to decide what results from the first phase will be the

focus of attention for the follow-up data collection. Making

an interpretation based on embedded results may be chal-

lenging because of the unequal emphasis placed on each

dataset by the investigator.

To explain mixed methods research plans to funders in

persuasive ways within page limitations, organizing

information into a table or presenting a figure of the mixed

methods procedures can aid in conserving space while

clearly identifying the expected contribution of each

activity, as well as the benefits of the integrated analysis

and interpretation [4]. Page and word limitations also affect

publication of mixed methods studies in scholarly journals

in which word limitations call for creative ways to present

material.

Conclusions

The ‘‘Best Practices for Mixed Methods Research in the

Health Sciences’’ expands on the topics we have reviewed

here, and contains additional sections on building teams,

writing an NIH R Series (research) application, develop-

ment of Career, Training and Program Project applications,

criteria for review, and suggestions for future activities. We

look forward to the continued discussion fostered by the

growing interest in mixed methods and congratulate the

journal Quality of Life Research for their early and con-

sistent contribution to this important conversation.

References

1. Plano Clark, V. L. (2010). The adoption and practice of mixed

methods: U. S. trends in federally funded health-related research.

Qualitative Inquiry, 16(6), 428–440. 2. Curry, L. A., Nembhard, I. M., & Bradley, E. H. (2009). Quali-

tative and mixed methods provide unique contributions to out-

comes research. Circulation, 119, 1442–1452. 3. Almarsdottir, A. B., & Traulsen, N. M. (2009). Multimethod

research into policy changes in the pharmacy sector—the Nordic

case. Research Social Administrative Pharmacy, 5(1), 82–90. 4. Stange, K. C., Crabtree, B. F., & Miller, W. L. (2006). Publishing

multimethod research. Annals of Family Medicine, 4, 292–294. 5. Wilkins, K., & Woodgate, R. (2008). Designing a mixed methods

study in pediatric oncology nursing research. Journal of Pediatric Oncology Nursing, 25(1), 24–33.

6. Creswell, J. W., & Zhang, W. (2009). The application of mixed

methods designs to trauma research. Journal of Traumatic Stress, 22(6), 612–621.

7. Palinkas, L. A., Horwitz, S. M., Chamberlain, P., Hurlburt, M. S.,

& Landsverk, J. (2011). Mixed-methods designs in mental health

services research: A review. Psychiatric Services, 62(3), 255–263.

8. Mertens, D. M. (2009). Research and evaluation in education and psychology: Integrating diversity with quantitative, qualitative, and mixed methods. Thousand Oaks, CA: Sage.

9. Klassen, A. C., Smith, K. C., Black, M. M., & Caulfield, L. E.

(2009). Mixed method approaches to understanding cancer-rela-

ted dietary risk reduction among public housing residents. Jour- nal of Urban Health: Bulletin of the New York Academy of Medicine, 86(4), 624–640.

10. McVea, K., Crabtree, B. F., Medder, J. D., Susman, J. L., Lukas,

L., McIlvain, H. E., et al. (1996). An ounce of prevention?

Evaluation of the ‘put prevention into practice’ program. Journal of Family Practice, 43(4), 73–81.

11. Pasick, R. J., Burke, N. J., Barker, J. C., Galen, J., Bird, J. A.,

Otero-Sabogal, R., et al. (2009). Behavioral theory in a diverse

society: Like a compass on Mars. Health Education Behavior, 36(5), 11S–35S.

12. Ring, L., Gross, C. R., & McColl, E. (2010). Putting the text back

into context: Toward increased use of mixed methods for quality

of life research. Quality of Life Research, 19, 613–615. 13. Greene, J. C. (2007). Mixed methods in social inquiry. San

Francisco: Wiley.

14. Morse, J., & Niehaus, L. (2009). Mixed method design: Princi- ples and procedures. Walnut Creek, CA: Left Coast Press.

15. Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Thousand Oaks, CA: Sage.

380 Qual Life Res (2012) 21:377–380

123

Copyright of Quality of Life Research is the property of Springer Science & Business Media B.V. and its

content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's

express written permission. However, users may print, download, or email articles for individual use.