QBRDM #6

Contents lists available at ScienceDirect

Currents in Pharmacy Teaching and Learning

journal homepage: www.elsevier.com/locate/cptl

Methodology Matters

Thematic analysis of qualitative research data: Is it as easy as it sounds?

Ashley Castleberrya,⁎, Amanda Nolenb

a University of Arkansas for Medical Sciences, College of Pharmacy, 4301 West Markham Street, Slot 522-4, Little Rock, AR 72205, United States b University of Arkansas at Little Rock, School of Education, 2801 S. University, Little Rock, AR 72204, United States

A R T I C L E I N F O

Keywords: Qualitative Thematic analysis

A B S T R A C T

Issue: We are seeing the use of qualitative research methods more regularly in health professions education as well as pharmacy education. Often, the term “thematic analysis” is used in research studies and subsequently labeled as qualitative research, but saying that one did this type of analysis does not necessarily equate with a rigorous qualitative study. This methodology review will outline how to perform rigorous thematic analyses on qualitative data to draw interpreta- tions from the data. Methodological Literature Review: Despite not having an analysis guidebook that fits every re- search situation, there are general steps that you can take to make sure that your thematic analysis is systematic and thorough. A model of qualitative data analysis can be outlined in five steps: compiling, disassembling, reassembling, interpreting, and concluding. My Recommendations and Their Applications: Nine practical recommendations are provided to help researchers implement rigorous thematic analyses. Potential Impact: As researchers become comfortable in properly using qualitative research methods, the standards for publication will be elevated. By using these rigorous standards for thematic analysis and making them explicitly known in your data process, your findings will be more valuable.

Issue

We are seeing the use of qualitative research methods more regularly in health professions education, as well as pharmacy education.1 Moreover, researchers recognize that qualitative methods provide “a source of well-grounded, rich descriptions and explanations of processes in identifiable local contexts”2 meaning the descriptions arise from the data but provide insight that goes beyond only numbers. Additionally, mixed-methods approaches combining qualitative and quantitative data analyses are becoming more popular in health professions practice and education.3

Qualitative research has been used in fields like education, sociology, and anthropology for some time and has, excitedly, gained more traction in the health research and health professions education fields but remains under utilized.2,4 This lag in adoption is likely due to the skepticism regarding the rigor of such methods from researchers, including pharmacists, who are more accustomed to quantitative research methods using statistical tests to “prove” an outcome and/or provide validity and reliability evidence.5 While quantitative research tends to focus on the frequency, intensity, or duration of a behavior, qualitative research methods allow us to explore the beliefs, values, and motives that explain why the behaviors occur. The primary aim of qualitative research is to gain a better understanding of phenomenon through the experiences of those who have directly experienced the phenomenon, recognizing

https://doi.org/10.1016/j.cptl.2018.03.019

⁎ Corresponding author. E-mail addresses: [email protected] (A. Castleberry), [email protected] (A. Nolen).

Currents in Pharmacy Teaching and Learning 10 (2018) 807–815

1877-1297/ Published by Elsevier Inc.

T

the value of participants' unique viewpoints that can only be fully understood within the context of their experience and worldview. The value of this approach to empirical research is that it provides a richer, deeper understanding of the meanings that people place on actions, events, and relationships.6 In academic medical education in general and in pharmacy education specifically, qualitative methods are used to explore the complex phenomenon encountered by faculty, students, patients, and policymakers.7

The descriptive nature of qualitative approaches allows the researcher to build a complex, holistic picture in a natural setting.8

This approach has been described as a bricolage of a wide range of interconnected methods to capture the essential essence of a phenomenon.9 The methodological literature is in agreement that qualitative research is a term that includes several research designs (e.g., case study, ethnography, grounded theory, narrative inquiry, and phenomenology) characterized by specific design assump- tions, sampling procedures, data collection, and data analysis protocols.10

One of the challenges to qualitative research is the open-ended nature of data as opposed to numbers only. Examples of qualitative data can include interview transcripts, newspaper articles, questionnaire responses, diaries, videos, images, or field observations. Text as data is often more difficult to reduce and identify patterns than numbers as data. Thematic analysis (TA) is a data analysis strategy that is a commonly used approach across all qualitative designs and is the subject of this methodology review. Often, TA is used in research studies and subsequently labeled as qualitative research, without providing the necessary details about how the analysis reduced the data into workable themes and the emerging conclusions. Collingridge and Gantt note “an understanding of the standards of rigorous qualitative research and familiarity with qualitative approaches has not kept pace with the growing presence of qualitative methods.”11

TA is a method of “identifying, analyzing, and reporting patterns (themes) within data”.12 It is described as a descriptive method that reduces the data in a flexible way that dovetails with other data analysis methods.13 It is used commonly because of the wide variety of research questions and topics that can be addressed with this method of data analysis.14 TA of open ended responses from surveys or transcribed interviews can explore the context of teaching and learning at a level of depth that quantitative analysis lacks while allowing flexibility and interpretation when analyzing the data, but it should be undertaken with special care and attention to transparency of the method in order to ensure confidence in the findings.12

This methodology review will outline how to perform TA on qualitative data. This research method can greatly benefit pharmacy education, therefore it is imperative that studies be designed and reported effectively to maintain the high standards of educational scholarship.15 By using sound and respected data collection and analysis techniques, the researchers can build trustworthiness and credibility with their readers.6

Methodological literature review

In pharmacy education, the most common type of qualitative data gathered is in the form of open-ended responses to ques- tionnaires or reflections in written form. Additionally, content from interviews and focus groups can be gathered. Designing questions to gather the data in these multiple formats is integral to ensure collection of good data. While collecting the data can be very fun and exciting, the real fun begins when the data are analyzed.

Yin's book, Qualitative Research from Start to Finish, outlines a general framework to design a qualitative research study: collect and record data, analyze the data, display and disseminate your findings.6 For the remainder of this section, the focus will be on the analysis portion of the research process.

In general, analysis of qualitative data can be outlined in five steps: compiling, disassembling, reassembling, interpreting, and concluding.6 The process of TA will be described within this framework.

Compiling

Compiling the data into a useable form is the first step to finding meaningful answers to your research questions. Compiling might mean transcribing so that the researchers can easily see the data. If your data needs transcribing from an interview or focus group, some experts recommend that you do the transcription yourself.16 While this takes much more time then paying someone to provide this service for you, the closeness to the data that you achieve during this process can jumpstart the other steps of the data analysis process. It seems intuitive, but the researcher needs to read and reread the data to become intimately familiar with it. This should occur many times throughout the analysis process. In this phase, the researcher is expected to transcribe interviews or focus groups, collate responses, and organize other textual data to be included in the analysis. Transcription services can help the researcher to save time but it is even more important that the researcher know the data intimately. In familiarizing themselves with the data,12 the researcher acquires a sense of the entirety of the data and allows a greater understanding of phrasing or the meaning of a term when viewed within the context of the whole. After getting your data in a consistent and organized format, you are ready to begin dissecting your data to discover its components.

Disassembling

After compiling and organizing the data, it must be separated. Disassembling the data involves taking the data apart and creating meaningful groupings. This process is often done through coding. Coding, in the realm of qualitative research, is defined as “the process by which raw data are gradually converted into usable data through the identification of themes, concepts, or ideas that have some connection with each other.”5 Coding simply involves researchers identifying similarities and differences in the data.16

Kuper describes how qualitative research differs from quantitative research in that “qualitative data analysis is largely inductive,

A. Castleberry, A. Nolen Currents in Pharmacy Teaching and Learning 10 (2018) 807–815

808

allowing meaning to emerge from the data, rather than the more deductive, hypothesis centered approach favored by quantitative researchers.”4 The meaning that “emerges from the data” is often first seen as the data is disassembled or coded.

The activity of coding involves identifying interesting features of the data systematically across the entire data set and occurs at multiple levels. Initially, codes are attached to units of data that could vary in size (i.e., phrase, sentence, paragraph) but usually codes encompass a complete thought. They can take the form of a descriptive label that directly describes or is taken from the text. However, codes can also be more abstract and complex in the form of metaphors or literary references.2 The code serves as a tag used to retrieve and categorize similar data so that the researcher can pull out and examine all of the data across the dataset associated with that code.

The action of coding requires the researcher to ask specific questions of the data as appropriate.17

• What is happening in the text? • Who are the actors and what are their roles? • When is it happening? (preceding event, during event, reaction to event, etc.) • Where is it happening? • What are the explicit and implicit reasons why it is happening? • How is it happening? (process or strategy)

A coding strategy can be established before coding begins (a priori) based on a careful review of previous research or theory.18 If a previous coding scheme has been used in other studies similar to yours, you may choose to use this as the starting point for your scheme. This can be accomplished by reviewing the literature inside and outside of your discipline to uncover a coding scheme used by others in similar context that could translate well to the new research situation. If the scheme does not fit your data well, modifications can be made. In contrast to a priori, the coding scheme can be open or emergent–meaning that the scheme is created as coding ensues because there is no beginning structure of the scheme but rather it develops during the coding process.

Saldana describes twenty-five coding methods categorized into seven groups.19 These approaches provide guidance to the re- searcher and answers the question, “How do I know what to code for?” While twenty-five coding methods may seem overwhelming, Saldana suggests that many of these methods overlap in intent and can be mixed and matched. For example, the “descriptive” code is a code applied to a basic topic from the data. Descriptive codes could be used to identify a role, process, action, place or something that is easily identified. Another coding approach is “In Vivo” coding that uses verbatim words or phrases from the participants’ narrative to describe the unit of data. In Vivo codes provide insight into how participants are talking about a phenomenon since it uses the participants’ voice when developing units of code. For example, a participant might use the word “scripts” to refer to prescriptions. This is a unique term that suggests a possible micro-culture such as pharmacy that uses specific terminology. Most likely, the researcher would choose a combination of these approaches to coding based on the purpose of the study and guided by their research questions.

As the researcher is coding, he or she is developing definitions for each code or groups of codes. These definitions could be thought of as inclusion and exclusion criteria in a quantitative study, and they ensure that codes are applied reliably throughout the data. As the code book containing code definitions is developed and refined, it is often necessary for the researcher to go back and re- code previously coded material to make certain that data examined early in the analysis is coded in the same manner and with the same coding definitions/criteria as data addressed later in the analysis. Researchers new to qualitative research methods often ask, “When can I stop coding and analyzing data?” When no new themes are identified upon reviewing new data, researchers can be confident in their coding scheme.5

Because TA draws upon a substantial amount of textual data, software programs are available to facilitate the organization process. Collectively they are called Computer-Assisted Qualitative Data AnalysiS or CAQDAS (pronounced “cactus” – like the plant in the desert). These can be helpful, but programs such as Microsoft Excel and Access can be as effective. NVivo® (QSR International Pty Ltd), MAXQDA (VERBI GmbH), and ATLAS.ti® (Scientific Software Development GmbH) are widely used tools that provide technological support to the qualitative research that streamlines the data analysis process and allows for more complex, deeper analysis of the data. We have used NVivo® for several projects and found it easy to use while creating beautiful graphical displays for the data.20 These software tools assist the researcher in looking at patterns of codes and links between codes across large fields of data. Linguistic and semantic algorithms detect sequencing and co-occurring phrasing in a reliable and systematic manner. A common misconception is that CAQDAS software can analyze the data for you–this cannot be farther from the truth. While software can assist researchers with organizing large amounts of qualitative data, the researcher's mind is the power behind analysis and not any software program. Fig. 1 provides example coding with NVivo®.

Reassembling

The codes, or categories to which each concept is mapped, are then put into context with each other to create themes. A theme “captures something important about the data in relation to the research question, and represents some level of patterned response or meaning within the data set”.12 Braun and Clarke use a house as an analogy to describe codes and themes-codes are the bricks that comprise the walls or themes.14 If simply put, themes are patterns in the codes; they take the numerous pieces of related code to show a bigger picture of what is being portrayed. Themes can be further divided into sub-themes.

Two common ways qualitative researchers put the data back together within themes are hierarchies and matrices. Thematic hierarchies provide a visual tool with which to articulate how themes are subordinate or superordinate to each other. Hierarchies are

A. Castleberry, A. Nolen Currents in Pharmacy Teaching and Learning 10 (2018) 807–815

809

constructed by clustering similar codes to produce higher-order codes. The researcher is then able to analyze the restructured data at multiple levels of granularity. The higher-order codes provide the researcher a view across the broad, thematic landscape of the data. More detailed codes allow the researcher to examine fine distinctions across instances. Matrices are constructed by arranging par- ticipant roles, themes, variables, emerging concepts, and data sources into rows and columns to provide a broad visual representation that grounds findings in the data and context.2 Heath et al. published an ‘insider's view’ to the method behind their study on hospital outpatient pediatric care.21 Table 1 is one of the matrices they constructed showing participants (rows) and codes (columns). They constructed separate matrices for each category. Each cell shows an excerpt of verbatim data from interview transcripts. The matrix provided an important analytical tool for the researchers to examine how the participants viewed the role of pediatric services in the community by their role.

Both hierarchies and matrices are tools used to provide a structure with which to reduce qualitative data as well as communicate relationships among groups, contexts, constructs, and codes. Other tools to help visualize the data include flowcharts, concept maps, and diagrams.

During reassembly, the analytical thinking of the researchers is evidenced. The researcher begins by gathering all relevant data into each potential theme and continuously reviews each theme to determine if it is robust in relation to the coded extracts and data set. Care must be taken to tell the story of the data and not arrange the data to support the researchers’ theory or overreach the data.7

When quality checking theme development, research should ask a few key questions14:

• Is this a theme (it could be just a code)? • If it is a theme, what is the quality of this theme (does it tell me something useful about the dataset and my research question)? • What are the boundaries of this theme (what does it include and exclude)? • Are there enough (meaningful) data to support this theme (is the theme thin or thick)? • Are the data too diverse and wide ranging (does the theme lack coherence)?

Review by multiple researchers can help to validate that the groupings of data are consistent with the raw data by establishing inter-coder reliability. This can be easily accomplished through NVivo® and other analysis software programs by comparing coding done by multiple researchers on the same text. A coding comparison query will calculate percentage agreement among coders as well as a Kappa coefficient that considers the amount of agreement that could be expected to occur due to chance. It is important to note that inter-relater reliability is not the “rule” when conducting qualitative research. If the data include unstructured, interactive interviews, Morse cautions against attempting to achieve inter-coder reliability due the reflexive and complex nature of the exchange between the interviewer and the participant(s).22 Doing so might unintentionally lead to an over-simplification of the coding,

Fig. 1. Example of thematic analysis coding with NVivo(®) software.

A. Castleberry, A. Nolen Currents in Pharmacy Teaching and Learning 10 (2018) 807–815

810

T ab

le 1

E xt ra ct

fr om

th e ‘P h il os op

h y of

C ar e’

m at ri x w it h it al ic s in d ic at in g ve

rb at im

te xt .2 1

Id eo

lo gy

of C C T H

P at ie n t- ce n tr ed

E qu

it y in

se rv ic e

E qu

iv al en

ce to

ap p ro ac h

p ro vi si on

h os p it al

ca re

M an

ag er

1 G en

P ae d s d oe

sn 't n ee d to

be in

h os p it al ; w it h ri gh

t in fr as tr u ct u re

C C T H

m ak

es se ns e [p 1,

24 ].

N ee d to

d el iv er

se rv ic es

ba se d on

w h at

fa m il ie s

n ee d ;a

t th e m om

en t fo cu se d on

w ha

t's ea si er

fo r us

Q Q Q

[p 16

, 46

7] .

Fo r so m e p eo

p le

a ci ty

ce n tr e h os p it al

is C T H

th an

a cl in ic

in th e co

m m u n it y [p 62

0, 21

].

N ee d to

in st il co nfi

de nc e th at

th ey

'r e ge

tt in g

sa m e le ve

l of

ca re , bu

t C T H

[p 2,

33 ].

C o n su

lt an

t 7

C C T H

is a go

od re co

m m en

d at io n ; on

ly p at ie n ts

w h o n ee d

sp ec ifi c in ve

st ig at io n s sh ou

ld at te n d h os p it al

fo r ou

tp at ie n ts

[p 1,

7] .

P re se rv at io n of

th e in st it ut io n (h os p it al ), ra th er

th an

ne ed s of

th e po pu

la ti on

th ey

ac tu al ly

se rv e,

se em

s to

be th e pr ed om

in an

t in te re st

Q Q

[p 3,

69 ].

M ak

e it cl ea r to

pa ti en ts

it 's ex ac tl y sa m e

se rv ic e in

sa te ll it e cl in ic , th ey

ar e se ei ng

m e

(s am

e co

n su lt an

t) [p ag e 1,

16 ].

E xe

cu ti v e 5

T h e m or e w e ke

ep p at ie n ts

ou t of

h os p it al , th e be

tt er ; d on

't w an

t p at ie n ts

in h os p it al

if d on

't n ee d to

[p 1,

14 ].

C C T H

is p ar t of

bi gg er

pi ct ur e ar ou

nd se lf ca re

an d se lf m an

ag em

en t

[p 11

, 35

0 ].

If ju st

tr an

sf er

cl in ic

fr om

h os p it al

to co

m m u n it y se tt in g,

im pr ov e ac ce ss

fo r so m e,

bu t re du

ce it fo r ot he rs

Q Q

[p 11

, 34

0 ].

A. Castleberry, A. Nolen Currents in Pharmacy Teaching and Learning 10 (2018) 807–815

811

analysis, and insights in an attempt to reach agreement across a team of researchers. If a researcher is working alone or shares the concern illustrated by Morse when working with unstructured and complex qua-

litative data, establishing intra-coder reliability along with member-checking or participant validation are a very suitable alter- natives.19,22 Intra-coder reliability allows you to assess the consistency with which you are coding similar data. Experts recommend coding a portion of the data right way, waiting a few days, then returning and re-coding the same data.2 Internal consistency should be in the range of 85–90% depending on the complexity of your coding scheme. It is also recommended to review emerging findings with participants in order to affirm that their experiences are represented accurately.22 Through this process you acquire valuable feedback throughout the analytic process that in turn supports your ability to defend interpretations and conclusions.5,23

Interpreting

Unfortunately, data do not “speak for themselves”.6 This critical stage in the research process involves the researcher making analytical conclusions from the data presented as codes and then themes. Even though the steps of data analysis are listed in linear sequence, interpretation does not have to wait until the end of the analysis process. In fact, interpretation by the researcher should be happening during the first three steps (compiling, disassembling, and reassembling).6

Yin explains that there is no checklist to constitute good interpretation, but there are five qualities that should be the goal of all qualitative interpretations. First, the interpretation should be complete. Readers should be able to see the beginning, middle, and end of how the interpretations were drawn. Second, the interpretations should be fair in that other researchers should reach the same interpretation if given the same data. Third, the interpretations should also be accurate and representative of the raw data. Fourth, in the context of current literature, good studies will add value to our understanding of the topic. Fifth, data methods and subsequent interpretations should be credible and gain respect from colleagues.6

Once data has been reassembled through coding, the researcher is then able to extract excerpts from the data and view them in relation to and in the concert with each other. Doing so allows the researcher to begin to start focusing on interpreting what is going on within and across varied experiences, beliefs, and histories and thus begin to identify thematic patterns across the data. Themes capture an essence of the phenomenon under investigation in relation to your research question or purpose of the study. These usually are abstract and difficult to identify from reading over raw data the first few times. Furthermore, in TA, the importance of the theme is not dependent upon how often it appears or how much data is contained within the theme. Rather, the importance is related to whether it captures something important in relation to the overall research questions.

As part of this level of the analysis, researchers will often develop a thematic map which is a visual representation of themes, codes, and their relationships (Fig. 2).24 These maps usually involved detailed descriptions of the themes including describing the patterns identified across the coded data. Braun and Clarke provide an example of how thematic maps evolve throughout a study as the researchers’ thinking about the themes and how they relate to each other evolve.12 This visual representation provides another level of analysis in TA that allows the researcher to place the themes in the larger context of the larger landscape of the phenom- enon.25

Qualitative data analysis software is particularly useful in developing data visualizations in the form of a three-dimensional map of code clusters. By viewing your re-assembled data in this way, you are able to see relationships and connections among constructs that are not readily apparent. Fig. 3 provides example data visualization in NVivo® showing how codes cluster together in the text across the data. The size of the nodes suggests how often that code is present across the data. The colors indicate families of codes or codes with meanings or functions. The diagram can be rotated in any direction to get a clearer view of any particular code. This particular figure is from a content analysis of academic department chair job postings across 20 years.26

These major themes become the starting point in interpreting how the themes relate to each other. Be cautious to not simply restate one's codes and themes as interpretations. Your interpretations should be at higher levels than themes. This means that interpretations should include discussions of relationships between themes and more global findings in the context of all codes, which is broader and more general than the specific codes, or themes. Another way to identify interpretations from the data is to answer research questions after the data has been coded. After all, research questions are where studies start and end. When you have enough clarity to write the subtitle for your manuscript, you know you have arrived at your interpretations.6 Interpretations should arise easily from your data and become the foundations for your conclusions.

Concluding

In the context of TA, raw data forms codes and codes form themes and thematic maps. Identifying and defining these themes leads to interpretations. Conclusions are the response to the research questions or purpose of the study.6 All research should start with a plausible research question and analysis should always answer a question; it just could be that the question shifted slightly over the course of the data analysis process.14

While the qualitative research community resists establishing a single set of evaluative standards to determine research quality, some journals may require adherence to certain guidelines in order to publish qualitative research. Qualitative researchers ascribe to common values of transparency of data analysis and recursive interpretations.27 Research must yield results that are open for careful scrutiny into the researchers’ decision making throughout the analysis process. This can be accomplished with a detailed description of coding procedures and criteria, detailed description of how codes and patterns of codes led to themes and resulting interpretation.

It is worth noting that conclusions from qualitative research are not usually generalizable.23,28 The conditions in which qualitative research is conducted can often not be replicated. This is not a hindrance or limitation to the research, but rather a feature of the

A. Castleberry, A. Nolen Currents in Pharmacy Teaching and Learning 10 (2018) 807–815

812

research to be acknowledged. Readers should assess how findings can be transferred and applied to their own area of practice.4,5 This process, termed analytical generalization, allows the reader to identify differences and similarities between the research context and their situations in order to determine relevance and applicability of study findings.11

Recommendations and their applications

1. Surround yourself with a team of researchers with expertise in qualitative research. Whether you are just starting this type of research or have been conducting it for years, the power of a team and mentorship cannot be overstated.5

2. To increase the consistency of the coding process, multiple coders should be used. This quality assurance process will ensure that your coding scheme and interpretations are illustrative of your data. If it is not possible to have multiple coders, keep detailed notes of decisions you make regarding coding the data and reassembling the data into themes.23

3. Know your own biases and report these openly in your manuscripts. In qualitative studies, the researcher is a part of the research process itself and thus brings their worldview in addressing the research questions.5 Therefore, it must be clear to the reader who the investigator is and how their background could bias findings.29

4. When reporting, include direct quotations from your data that are brief and targeted to show readers the quality of your coding, theme generation, and subsequent conclusions. This can be a large step towards transparency and dependability. Readers will be more likely to accept your conclusions when they see that they are rooted in empirical participant data.16 Try to avoid simply providing numbers to quantify how many participants mentioned each theme. This may be appropriate for certain data, but it should not be the only data represented in a qualitative study.

5. To increase reliability, use sound research methods for collecting and analyzing data as described in this review. According to Collinridge and Grantt, “qualitative researchers who adopt reliable, qualitative methods and conduct their analyses in a competent manner are expected to produce results that enrich our understanding of the meanings that people attach to social phenomena.”11

6. Answer your research questions. Aligning data collection methods with your research questions while obtaining results that cor- relate with other previously published data will increase validity.11,30

7. Take your time when coding for it is foundational to the data analysis process and should not be rushed. TA is hard work. If you are growing data weary, take a break from your work and return to it with fresh eyes. Be flexible in your analysis. If your inter- pretations are not coming easily, consider recoding the data to make sure you didn't miss key concepts.

Fig. 2. Example thematic map for visual representation of themes, codes, and their relationships.24

Reprinted from Concept Maps: Theory, Methodology, Technology Proc. of the First Int. Conference on Concept Mapping, Volume 1, Daley, B. J., Using concept maps in qualitative research, 191-197, Copyright 2004, with permission from Author.

A. Castleberry, A. Nolen Currents in Pharmacy Teaching and Learning 10 (2018) 807–815

813

8. Don't be afraid to start again. The data analysis process for TA is not linear nor does it necessarily occur once all of the data are collected. The process is recursive in nature where the researcher moves in and out of each phase of analysis. For example, upon examining patterns to establish a theme, the researcher might identify an unexpected pattern that can be further defined with additional coding and re-assembling.

9. Allow the readers to trust you and your work. Make sure that you are transparent about your analysis process (credibility), provide adequate detail of study content so the reader can determine if the findings are generalizable (transferability), indicate con- sistency in your research to show repeatability (dependability), and ensure that your results arise from the data and not your biases (confirmability).

Potential impact

By following these steps for TA of qualitative data, researchers can present work that is trustworthy and credible. An example of both using and explicitly describing sound qualitative analysis processes is found in Austin's article entitled “Continuous Professional Development: A Qualitative Study of Pharmacists’ Attitudes, Behaviors, and Preferences in Ontario, Canada.”31

In this study, forty-two pharmacists participated in small, 90-min focus group sessions with an experienced facilitator who facilitated discussion on continuing professional development. Data gathered included field notes and direct participant quotations. CAQDAS software was used to assist in the coding of raw data that was further refined into themes and subthemes. Confirmation of major themes was achieved through constant comparison with field notes and member checking. Direct quotes were used within the body of the manuscript to allow the reader to verify thematic coding and ensure that conclusions were then drawn from the in- terpretations of themes.

Benefits of following these recommendations include recognized validity and reliability of your qualitative work. Additionally, researchers who have not tried any type of qualitative methods previously could follow these steps and produce insightful work.12

There is no magic prescription for how to best analyze qualitative data. In fact, analysis can even change during analysis; this flexibility is often noted as one of the strengths and advantages of qualitative research. But despite not having an analysis guidebook that fits every research situation, these general steps will help you make sure that your analysis is systematic and thorough.27 As more researchers become comfortable in properly using qualitative research methods, the standards for publication will be elevated. By using these rigorous standards for TA and making them explicitly known in your data process, your findings will not only be valued in the arena of pharmacy education, but in other fields of study as well.

Fig. 3. Example of data visualization available with NVivo(®) showing how codes cluster together in the text across the data.

A. Castleberry, A. Nolen Currents in Pharmacy Teaching and Learning 10 (2018) 807–815

814

Disclosure

Authors have nothing to disclose in relation to this manuscript.

References

1. Rosenthal M. Qualitative research methods: why, when, and how to conduct interviews and focus groups in pharmacy research. Curr Pharm Teach Learn. 2016;8(4):509–516.

2. Miles B, Huberman A, Saldana J. Qualitative Data Analysis. 3rd ed. Thousand Oaks, CA: Sage Publications; 2014. 3. Creswell JW, Plano Clark VL. Designing and Conducting Mixed Methods Research. Thousand Oaks, CA: Sage Publications; 2006. 4. Kuper A, Reeves S, Levinson W. An introduction to reading and appraising qualitative research. BMJ. 2008;337:404–407. 5. Austin Z, Sutton J. Qualitative research: getting started. Can J Hosp Pharm. 2014;67(6):436–440. 6. Yin RK. Qualitative Research from Start to Finish. New York, NY: The Guilford Press; 2011. 7. Anderson C. Presenting and evaluating qualitative research. Am J Pharm Educ. 2010;74(8):141. 8. Creswell J. Qualitative Inquiry and Research design: Choosing among Five Approaches. Thousand Oaks, CA: Sage Publications; 2007. 9. Denzin NK, Lincoln YS. The Landscape of Qualitative Research. 3rd ed Thousand Oaks, CA: Sage Publications; 2008. 10. Nolen A, Talbert T. Qualitative assertions as prescriptive statements. Educ Psychol Rev. 2011;23(2):263–271. 11. Collingridge DS, Gantt EE. The quality of qualitative research. Am J Med Qual. 2008;23(5):389–395. 12. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101. 13. Vaismoradi M, Turunen H, Bondas T. Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nurs Health Sci.

2013;15(3):398–405. 14. Braun V, Clarke V. Thematic analysis. In: APA Handbook of Research Methods in Psychology: Vol. 2. Research Designs. 15. McLaughlin JE, Dean MJ, Mumper RJ, Blouin RA, Roth MT. A roadmap for educational research in pharmacy. Am J Pharm Educ. 2013;77(10):218. 16. Sutton J, Austin Z. Qualitative research: data collection, analysis, and management. Can J Hosp Pharm. 2015;68(3):226–231. 17. Bernard H, Wutich A, Ryan G. Analyzing Qualitative Data: Systematic Approaches. 2nd ed Thousand Oaks, CA: Sage Publications; 2017. 18. Boyatzis RE. Transforming Qualitative Information: Thematic Analysis and Code Development. Thousand Oaks, CA: Sage Publications; 1998. 19. Saldana J. The Coding Manual For Qualitative Researchers. 3rd ed. Thousand Oaks, CA: Sage Publications; 2016. 20. Castleberry A. NVivo 10 [software program]. Version 10. QSR International. Am J Pharm Educ. 2012;7(1):25. 21. Heath G, Cameron E, Cummins C. Paediatric ‘care closer to home’: stake-holder views and barriers to implementation. Health Place. 2012;18(5):1068–1073. 22. Morse J. “Perfectly healthy, but dead”: the myth of inter-rater reliability. Qual Health Res. 1997;7(4):445–447. 23. Barbour RS. Checklists for improving rigour in qualitative research: a case of the tail wagging the dog. Br Med J. 2001;322(7294):1115–1117. 24. Daley BJ. First Int. Conference on Concept Mapping. In: Cañas AJ, Novak JD, González FM (Eds.), Concept Maps: Theory, Methodology, Technology Proc. of the First

Int. Conference on Concept Mapping (Vo1. 1). Pamplona, Spain; 2004. http://cmc.ihmc.us/cmc-proceedings/. Accessed 11 April 2018. 25. Kuckartz U. Qualitative Text Analysis: A Guide to Methods, Practice, and Using Software. Thousand Oaks, CA: Sage Publications; 2014. 26. Nolen A, Dzwik L, Vander Putten J. Longitudinal analysis of department chair employment announcements: changing qualifications, roles, and responsibilities.

Paper Submitted for the 2018 Annual Conference of the American Educational Research Association. 2018; 2018. 27. Strauss AL. Qualitative Analysis for Social Scientists. Cambridge: Cambridge University Press; 1987. 28. Kvale S. Ten Standard objections to qualitative research interviews. J Phenomenolog Psychol. 1994;25(2):147–173. 29. Corbin J, Strauss A. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Thousand Oaks, CA: Sage Publications; 2014. 30. Cook DA. When I say… validity. Med Educ. 2014;48:948–949. 31. Austin Z, Marini A, Macleod Glover N, Croteau D. Continuous professional development: a qualitative study of pharmacists' attitudes, behaviors, and preferences

in Ontario, Canada. Am J Pharm Educ. 2005;69(1):4.

A. Castleberry, A. Nolen Currents in Pharmacy Teaching and Learning 10 (2018) 807–815

815