QBRDM #6

A Practical Guide to Collaborative Qualitative Data Analysis

K. Andrew R. Richards University of Alabama

Michael A. Hemphill University of North Carolina at Greensboro

The purpose of this article is to provide an overview of a structured, rigorous approach to collaborative qualitative analysis while attending to challenges associated with working in team environments. The method is rooted in qualitative data analysis literature related to thematic analysis, as well as the constant comparative method. It seeks to capitalize on the benefits of coordinating qualitative data analysis in groups, while controlling for some of the challenges introduced when working with multiple analysts. The method includes the following six phases: (a) preliminary organization and planning, (b) open and axial coding, (c) development of a preliminary codebook, (d) pilot testing the codebook, (e) the final coding process, and (f) reviewing the codebook and finalizing themes. These phases are supported by strategies to enhance trustworthiness, such as (a) peer debriefing, (b) researcher and data triangulation, (c) an audit trail and researcher journal, and (d) a search for negative cases.

Keywords: multiple analysts, qualitative methods, researcher training, trustworthiness

While qualitative research has been traditionally discussed as an individual undertaking (Richards, 1999), research reports have in general become increasingly multi-authored (Cornish, Gillespie, & Zittoun, 2014; Hall, Long, Bermback, Jordan, & Patterson, 2005), and the field of physical education is no exception (Hemphill, Richards, Templin, & Blankenship, 2012; Rhoades, Woods, Daum, Ellison, & Trendowski, 2016). Proponents of collaborative data analysis note benefits related to integrating the perspectives provided by multiple researchers, which is often viewed as one way to enhance trustworthiness (Patton, 2015). Collaborative data analysis also allows for researchers to effec- tively manage large datasets while drawing upon diverse perspec- tives and counteracting individual biases (Olson, McAllister, Grinnell, Walters, & Appunn, 2016). Further, collaborative ap- proaches have been presented as one way to effectively mentor new and developing qualitative researchers (Cornish et al., 2014).

Despite the potential benefits associated with collaborative qualitative data analysis, coordination among analysts can be challenging and time consuming (Miles & Huberman, 1994). Issues related to the need to plan, negotiate, and manage the complexity of integrating multiple interpretations while balancing diverse goals for involvement in research also represent challenges that need to be managed when working in group environments (Hall et al., 2005; Richards, 1999). Concerns have also been voiced about the extent to which qualitative data analysis involving multiple analysts is truly integrative and collaborative, rather than reflective of multiple researchers working in relative isolation to produce different accounts or understandings of the data (Moran- Ellis et al., 2006).

Challenges associated with collaboration become com- pounded when also considering the need for transparency in qualitative data analysis. Analysts need to develop, implement, and report robust, systematic, and defensible plans for analyzing qualitative data so to build trustworthiness in both the process and findings of research (Sin, 2007). Authors, however, often prioritize results in research manuscripts, which limits space for discussing methods. This leads to short descriptions of data analysis proce- dures in which broad methods without an explanation of how they were implemented (Moravcsik, 2014), and can limit the availability of exemplar data analysis methods in the published literature. This has given rise to calls for increased transparency in the data collection, analysis, and presentation aspects of qualitative research (e.g., Kapiszewski & Kirilova, 2014). The American Political Science Association (APSA, 2012), for example, recently published formal recommendations for higher transparency stan- dards in qualitative research that call for detailed descriptions of data analysis procedures and require authors support all assertions with examples from the dataset.

To help address the aforementioned challenges, scholars across a variety of disciplines have published reports on best practices related to qualitative data analysis (e.g., Braun & Clarke, 2006; Cornish et al., 2014; Hall et al., 2005). Many of these approaches are rooted in theories and epistemologies of qualitative research that guide practice (e.g., Boyatzis, 1998; Glaser & Strauss, 1967; Lincoln & Guba, 1985; Strauss & Corbin, 2015). Braun and Clarke’s (2006) highly referenced article provides a step-by-step approach to completing thematic analysis that helps to demystify the process with practical examples. In another similar vein, Hall and colleagues (2005) tackle challenges related to collaborative data analysis and discuss processes related to (a) building an analysis team, (b) developing reflexivity and theoretical sensitivity, (c) addressing analytic procedures, and (d) preparing to publish findings. Cornish and colleagues (2014) further this discussion by noting several dimensions of collaboration that are beneficial in

Richards is with the Department of Kinesiology, University of Alabama, Tuscaloosa, AL. Hemphill is with the Department of Kinesiology, University of North Carolina at Greensboro, Greensboro, NC. Address author correspondence to K. Andrew R. Richards at [email protected].

225

Journal of Teaching in Physical Education, 2018, 37, 225-231 https://doi.org/10.1123/jtpe.2017-0084 © 2018 Human Kinetics, Inc. RESEARCH NOTE

qualitative data analysis. The rigor and quality of the methodology may benefit, for example, when research teams include insider and outsider perspectives, multiple disciplines, academics and practi- tioners, international perspectives, or senior and junior faculty members.

In this paper, we contribute to the growing literature that seeks to provide practical approaches to qualitative data analysis by overviewing a six-step approach to conducting collaborative qual- itative analysis (CQA), which is grounded in qualitative methods and data analysis literature (e.g., Glaser & Strauss, 1967; Lincoln & Guba, 1985; Patton, 2015). While some practical guides in the literature provide an overview of data analysis procedures, such as thematic analysis (Braun & Clarke, 2006), and others discuss issues related to collaboration (Hall et al., 2005), we seek to address both by overviewing a structured, rigorous approach to CQA while attending to challenges that stem from working in team environ- ments. We close by making the case that the CQA process can be employed when working with students, novice researchers, and scholars new to qualitative inquiry.

Collaborative Qualitative Analysis: Building Upon the Literature

In our collaborative work, we began employing a CQA process in response to a need to balance rigor, transparency, and trustworthi- ness in data analysis while managing the challenges associated with analyzing qualitative data in research teams. Our goal was to integrate the existing literature related to qualitative theory, meth- ods, and data analysis (Glaser & Strauss, 1967; Patton, 2015; Strauss & Corbin, 2015) to utilize procedures that allowed us to develop consistency and agreement in the coding process without quantifying intercoder reliability (Patton, 2015). Drawing from recommendations presented in other guides for conducting quali- tative data analysis (Braun & Clarke, 2006; Hall et al., 2005), researchers adopting CQA work in teams to collaboratively develop a codebook (Gibbert, Ruigrok, & Wicki, 2008) through open and axial coding, and subsequently test that codebook against previously uncoded data before applying it to the entire dataset. There are steps embedded to capitalize on perspectives offered by members of the research team (i.e., researcher triangulation; Lincoln & Guba, 1985), and the process culminates in a set of themes and subthemes that form the basis for study results. The CQA process also embraces the tradition of constant comparison (Glaser & Strauss, 1967) as newly coded data are compared with existing coding structures and modifications are made to those structures through the completion of the coding process. This provides flexibility to modify generative themes1 in light of challenging or contradictory data.

The CQA process is grounded in thematic analysis, which is a process for identifying, analyzing, and reporting patterns in qualitative data (Boyatzis, 1998). Typically, thematic analysis culminates with a set of themes that describe the most prominent patterns in the data. These themes can be identified using inductive approaches, whereby the researcher seeks patterns in the data themselves and without any preexisting frame of reference, or through deductive approaches in which a theoretical or conceptual framework provides a guiding structure (Braun & Clarke, 2006; Taylor, Bogdan, & DeVault, 2015). Alternatively, thematic analy- sis can include a combination of inductive and deductive analysis. In such an approach, the research topic, questions, and methods may be informed by a particular theory, and that theory may also

guide the initial analysis of data. Researchers are then intentional in seeking new ideas that challenge or extend the theoretical perspectives adopted, which makes the process simultaneously inductive (Patton, 2015). The particular approach adopted by a research team will relate to the goals of the project, and particularly the extent to which the research questions and methods are informed by previous research and theory.

Trustworthiness is at the center of CQA, and methodological decisions are made during the research design phase to address Guba’s (1981) four criteria of credibility, confirmability, depend- ability, and transferability. In particular, we find that triangulation, peer debriefing, an audit trail, negative case analysis, and thick description fold into CQA quite naturally. In addition to the afore- mentioned researcher triangulation, data triangulation is often a central feature of design decisions as researchers seek to draw from multiple data sources to enhance dependability (Brewer & Hunter, 1989), and an outside peer debriefer (Shenton, 2004) can be invited to comment upon ongoing analysis so to add credibility. An audit trail can be maintained in a collaborative researcher journal to enhance confirmability (Miles & Huberman, 1994), and a negative case analysis can highlight data that contradict the main findings so to enhance credibility (Lincoln & Guba, 1985). Transferability is addressed by providing a detailed account of the study context and through rich description in the presentation of results (Shenton, 2004).

Overview of the Collaborative Constant Comparative Qualitative Analysis Process

The CQA process includes a series of six progressive steps that begin following the collection and transcription of qualitative data, and culminate with the development of themes and subthemes that summarize the data (see Figure 1). These steps include (a) preliminary organization and planning, (b) open and axial coding, (c) the development of a preliminary codebook, (d) pilot testing the codebook, (e) the final coding process, and (f) review of the codebook and finalizing the themes. While the process can be employed with teams of various sizes, we have found teams of two to four analysts to be most effective because they capitalize on the integration of multiple perspectives, while also limiting variability due to inconsistencies in coding (Olson et al., 2016). In larger teams, some members may serve as peer debriefers.

When considering the initiation of teamwork, we concur with the recommendations of Hall and colleagues (2005) related to the development of rapport among team members prior to beginning analysis. A lack of comfort may lead team members to hold back critique and dissenting viewpoints that could be important to data analysis. This is particularly true of faculty members working with graduate students where the implied power relationship can discourage students from being completely forthright. As a result, we recommend that groups engage in initial conversations un- related to the data analysis so to get to know one another and their relational preferences. This could include a discussion of com- munication styles, previous qualitative research experience, and epistemological views related to qualitative inquiry (Hall et al., 2005). The team leader may also provide an overview of the CQA process, particularly when working with team members who have not used it previously. As part of this process it should be made clear that all perspectives and voices are valued, and that all team members have an important contribution to make in the data analysis process.

JTPE Vol. 37, No. 2, 2018

226 Richards and Hemphill

D ow

nl oa

de d

by E

bs co

P ub

li sh

in g

ae ll

sw or

th @

eb sc

o. co

m o

n 05

/0 8/

18 , V

ol um

e ${

ar ti

cl e.

is su

e. vo

lu m

e} , A

rt ic

le N

um be

r ${

ar ti

cl e.

is su

e. is

su e}

Phase One: Preliminary Organization and Planning

Following the collection and transcription of data, the CQA process

begins with an initial team meeting to discuss project logistics and

create an overarching plan for analysis. This includes writing a

brief description of the project, listing all qualitative data sources to be included, acknowledging any theoretical or conceptual frame- works utilized, and considering research questions to be addressed. Members of the data analysis team should also have an initial discussion of and negotiate through topics, such as the target

Figure 1 — Overview of the six steps involved in collaborative qualitative analysis. Strategies for enhancing trustworthiness underpin the analysis process.

JTPE Vol. 37, No. 2, 2018

Qualitative Data Analysis 227

D ow

nl oa

de d

by E

bs co

P ub

li sh

in g

ae ll

sw or

th @

eb sc

o. co

m o

n 05

/0 8/

18 , V

ol um

e ${

ar ti

cl e.

is su

e. vo

lu m

e} , A

rt ic

le N

um be

r ${

ar ti

cl e.

is su

e. is

su e}

journal, anticipated authorship, and a flexible week-by-week plan for analysis. The weekly plan includes a reference to the data analysis phase, coding assignments for each team member, and space for additional notes and clarification (see Figure 2). Deci- sions related to the target journal and authorship, as well as the weekly plan for analysis, will likely evolve over time, but we find it helpful to begin such conversations early to ensure that all team members are on the same page.

Phase Two: Open and Axial Coding

To begin the data analysis process we use open coding to identify discrete concepts and patterns in the data, and axial coding to make connections between those patterns (Corbin & Strauss, 1990). While open and axial coding are distinct analytical procedures,

we embrace Strauss and Corbin’s (2015) recommendation that they can occur simultaneously as researchers identify patterns and then begins to note how those patterns fit together. Specifically, each member of the research team reads two to three different data transcripts (e.g., field notes, interviews, reflection journal entries) and codes them into generative categories using their preferred method (e.g., qualitative data analysis software, manual coding). The goal is to identify patterns common across transcripts, or to note deviant cases that appear.

Depending on the approach to thematic analysis adopted, a theoretical framework and research questions could frame this process. We find it helpful, however, to retain at least some inductive elements so to remain open to generative themes that may not fit with theory. Following each round of coding, team members write memos in a researcher journal, preferably through a

Project Overview and Data Analysis Timeline Project Overview: To understand how physical education teachers navigate the sociopolitical realities of the contexts in which they work and derive meaning through interactions with administrators, colleagues, parents, and students. This work is a qualitative follow-up to a large- scale survey that was completed by over 400 physical education teachers from the US Midwest.

1. Theoretical Framework: Occupational socialization theory 2. Target Journal:Physical education pedagogy specific journal, such as the Journal of

Teaching in Physical Education or Research Quarterly for Exercise and Sport 3. Anticipated Authorship:Researcher 1, Researcher 2, Researcher 3 4. Data Sources:30 individual interviews, 5 focus group interviews, field notes from

observations of teachers 5. Research Questions:

a. How do physical education teachers perceive that they matter given the marginalized nature of their subject?

b. How do interactions with administrators, colleagues, parents, and students influence physical educators’ perceptions of mattering and marginalization?

c. How do physical education teachers’ perceptions of mattering and marginalization influence feelings of role stress and burnout?

Weekly Plan for Data Analysis: Week Coding Phase Coding Assignment Notes July 11, 2016 Initial Meeting rof nalp eht ssucsiD enoN

analysis and review the data analysis timeline. Make changes and adjustments to the plan as necessary. Discuss the various phases of analysis and prepare to begin open coding.

August 1, 2016 Open Coding 1 Researcher 1: 1001, 1002

Researcher 2: 1003, 1004

Researcher 3: 1005, 1006

Open coding of each transcript into categories. Following coding, identify 3-4 generative themes and write a 1 page memo

August8, 2016 Open Coding 2 Researcher 1: 1022, 1023

Researcher 2: 1024, 1025

Researcher 3: 1007, 1027

Open coding of each transcript into categories. Following coding, identify 3-4 generative themes and write a 1 page memo

Figure 2 — Example of a project overview, code numbers (e.g., 1001) refer to interview transcripts.

JTPE Vol. 37, No. 2, 2018

228 Richards and Hemphill

D ow

nl oa

de d

by E

bs co

P ub

li sh

in g

ae ll

sw or

th @

eb sc

o. co

m o

n 05

/0 8/

18 , V

ol um

e ${

ar ti

cl e.

is su

e. vo

lu m

e} , A

rt ic

le N

um be

r ${

ar ti

cl e.

is su

e. is

su e}

shared online platform (e.g., Google Docs), in which they overview the coding and describe two or three generative themes supported by data excerpts. During research meetings, team members over- view their coding in reference to the memos they wrote, and the team discusses the coding process more generally. Phase two continues for three to four iterations, or until the research team feels they have seen and agree upon a variety of generative themes related to the research questions. The exact number of transcripts coded depends on the size of the dataset and the level of initial agreement established amongst the researchers. The team can move on when all coders feel comfortable with advancing to the devel- opment of a codebook. In our experience, this usually involves coding approximately 30% of all transcripts, but could be less when working with large datasets.

Phase Three: Development of a Preliminary Codebook

After the completion of phase two, one team member reviews the memos and develops a preliminary codebook (Richards, Gaudreault, Starck, & Woods, in press). An example codebook is included in Figure 3, and typically includes first- and second-order themes, definitions for all themes, and space to code quotations from the transcripts. Theme definitions provide the criteria against which quotations are judged for inclusion in the codebook, and thus should be clear and specific. We code by copy/pasting excerpts from the transcript files into the codebook and flagging each with the participant’s code number, the line numbers in the transcript file, and a reference to the data source (e.g., Interview 1001, 102–105).

This allows for reference back to the data source to gain additional context for quotations as needed. We always include a “General (Uncoded)” category where researchers can place quotations that are relevant, but do not fit anywhere in the existing coding structure. These quotations can then be discussed during team meetings. Once compiled, the draft codebook is circulated to the research team for review and discussed during a subsequent team meeting. Changes are made based on the team discussion, and a preliminary codebook is finalized. At this stage we enlist the assistance of a researcher who is familiar with the project, but not involved in the data analysis, to serve as a peer debriefer (Lincoln & Guba, 1985). This individual reviews and comments on the initial codebook, and appropriate adjustments are made before proceeding.

Phase Four: Pilot Testing the Codebook

After the initial codebook has been developed, it is tested against previously uncoded data. During this step, the researchers all code the same two to three transcripts, and make notes in the researcher journal related to interesting trends or problems with the codebook. Weekly research team meetings provide a platform for researchers to overview and compare their coding and discrepancies are discussed until consensus is reached. Entries in the researcher journal are also discussed. These discussions lead to the develop- ment of coding conventions, which function as rules that guide subsequent coding decisions. Conventions may be created for double coding excerpts into two generative themes in rare instances when both capture the content of a single quotation, and that quotation cannot be divided in a meaningful way.

Perceived Mattering Codebook

stpircsnarT morf selpmaxE snoitinifeD semehtbuS semehT

Subject Marginalization Lack of communication

Teacher believes physical education does not matter due to lack of communication about issues that affect thephysical education environment.

“My stressful day, um probably when things pop up that are not…A lot of my stresses get raised from being an activities director. If the school calls me and says now they have to— they have kids who are not coming, they change times, or I have a different schedule. My stuff is very organized and if it’s not where I think it’s supposed to be and I need it, that’s very stressful for me” (1019, 210–217, individual interview)

dna emit fo kcaL resources

Teacher believes physical education does not matter due to lack of teaching contact time and resources such as materials, equipment for PE, or teaching facilities.

“It’s kind of rough because I don’t have my own classroom. I don’t have my own computer up there. I don’t have a room that I can make into a welcoming environment so that’s kind of rough” (1018, 110–112, individual interview)

“Right now that class is more just like babysitting. It’s just a study hall, kind of boring. I don’t have a classroom I’m in the gym balcony where the bleachers are at. I don’t have space the kids complain” (1018, 120–122, focus group)

eileb rehcaeT troppus fo kcaL ves physical education does not matter due to situations in which the physical educator does not feel support for ideas or initiatives.

“I think the colleagues, it wouldn’t matter either way outside of the P.E. teachers, and I think the administration wouldn’t care either way.” (1018, 348–350, individual interview)

“At the elementary level that would be a big issue. As they get a little older, you know middle school, high school it’s not as much probably fun. They don’t see it in their eyes as much fun. The students themselves probably wouldn’t care, there’d be a handful.” (1019, 307–309, focus group)

Figure 3 — Example codebook including themes, subthemes, definitions of subthemes, and quotations from the dataset.

JTPE Vol. 37, No. 2, 2018

Qualitative Data Analysis 229

D ow

nl oa

de d

by E

bs co

P ub

li sh

in g

ae ll

sw or

th @

eb sc

o. co

m o

n 05

/0 8/

18 , V

ol um

e ${

ar ti

cl e.

is su

e. vo

lu m

e} , A

rt ic

le N

um be

r ${

ar ti

cl e.

is su

e. is

su e}

Conventions can also specify priority in the use of generative themes. In Figure 3, for example, there are generative themes for both “lack of support” and “lack of communication” related to subject marginalization. Lack of communication could be consid- ered a way in which support is limited, but because there is a specific category for lack of communication, it would receive priority when coding. Modifications are made to the codebook as needed during these meetings, and an updated codebook is produced to guide subsequent analysis. The pilot testing continues for three to four rounds of coding, or until the research team feels confident in the codebook. Once the team feels ready to move on, they have a final discussion of the codebook in light of the pilot testing and make adjustments. The peer debriefer (Lincoln & Guba, 1985) then reviews the evolving codebook and recommends changes prior to the final coding process.

Phase Five: Final Coding Process

In the final phase of coding the adjusted codebook is applied to all project data, including that which had been previously coded during the formative phases of codebook development. While the researcher triangulation involved when using multiple coders can increase “validity2” in qualitative research, some have argued that it has the potential to reduce “reliability” because of inconsisten- cies in coding across analysts (Olson et al., 2016). As a result, some qualitative researchers have introduced measures of inter-coder reliability in an attempt to quantify agreement between coders (Neuendorf, 2017). While acknowledging these perspectives, we struggle with efforts to apply the quantitative principles of reliabil- ity and validity to qualitative data analysis (Patton, 2015). We prefer to approach the issue of coder agreement, and the broader notions of trustworthiness and credibility, by establishing a clear protocol and codebook (Gibbert et al., 2008) through previous steps of CQA, and then dialogue through and reach consensus on coded data. This is done either through consensus coding or split coding. Regardless of the strategy chosen, coding conventions developed during previous phases are applied to the coding process. Analysts continue to make notes in the researcher journal related to problems with the generative themes, or interesting patterns in the data, and issues are discussed during weekly research meetings. We continue to apply the constant comparative method (Strauss & Corbin, 2015) at this stage as modifications are made to the code- book to reflect ongoing insights developed in the coding process.

Consensus coding is the more rigorous, but more time- consuming form of final coding. It is likely the more effective approach when working in larger groups where coding consistency concerns are more abundant (Olson et al., 2016). During each iteration of coding, team members code the same two to three transcripts into the codebook. Then, during research team meet- ings, each coded statement is compared across members of the research team. Disagreements are discussed until the group reaches consensus. Split Coding relies more heavily on the establishment of clarity through the preliminary coding phases and the coding conventions that have been developed (Gibbert et al., 2008). While less rigorous than consensus coding, split coding is also less time consuming and manageable within smaller teams. During each iteration of coding, team members code two to three different transcripts. As a result, only one member of the team will code each transcript. Then, during research meetings, questions or concerns related to particular excerpts are discussed. Split coding culminates with each team member reviewing all coded excerpts in the codebook, and disagreements are discussed to consensus.

Phase Six: Review the Codebook and Finalize the Themes

After all of the transcripts have been coded using consensus coding or split coding, the research team meets one final time to review the codebook. During the meeting, the codebook is developed into a thematic structure comprised of themes and associated subthemes that describe participants’ perspectives. The thematic structure is reviewed and approved by all members of the research team, and the final agreed upon structure forms the basis for the result that will be presented as part of the manuscript. Importantly, through the earlier stages of CQA, all members of the research team have had a hand in shaping and agree upon the themes that are presented. This process, therefore, capitalizes on the enhanced trustworthiness provided by multiple analysts, while minimizing issues related to coder variability, without attempting to quantify the qualitative data analysis process (Patton, 2015).

Conclusions and Final Thoughts

The purpose of this article is to provide an overview of a structured, rigorous approach to CQA while attending to challenges that stem from working in team environments. While this article has focused primarily on the data analysis process, effective analysis begins at the design phase when researchers pose research questions, decide on methods, and identify participants (Patton, 2015). After data have been collected, the six-phase CQA process is adopted to make meaning through the formation of generative themes. This process integrates existing approaches to qualitative research (Glaser & Strauss, 1967; Miles & Huberman, 1994; Patton, 2015), and contributes to the emerging literature that seeks to provide practical examples of qualitative data analysis (e.g., Braun & Clarke, 2006; Cornish et al., 2014; Hall et al., 2005). It provides a structured and rigorous approach that enhances transparency through the data analysis process (e.g., Kapiszewski & Kirilova, 2014; Moravcsik, 2014), while capitalizing on the development of a codebook and multiple researchers’ perspectives (Gibbert et al., 2008).

In considering qualitative data analysis, Woods and Graber (2016) explain, “ultimately, it is the responsibility of the investi- gator to select those procedures that best meet the philosophic orientation of the study, the purpose of the investigation, and the methods that were used to collect the data” (p. 30). Regardless of the particular approach taken, all qualitative researchers are chal- lenged to ensure methodological rigor and transparency, and CQA provides one way to demonstrate inclusive collaboration among researchers. The coding, memoing, and pilot testing of the code- book provide multiple layers where all researchers have opportu- nities to share their perspectives. The audit trail maintained through ongoing discussions and the researcher journal also enhances transparency and allows for the process to be documented and adapted for use across multiple research projects.

We find that CQA can aid in the management of large, qualitative datasets by providing a structured and phasic approach to analysis. This can be particularly helpful for graduate students, early career researchers, and diverse research teams who may be struggling to identify rigorous data analysis procedures that meet the needs of all researchers (Cornish et al., 2014). The step-by-step nature of the approach also has applicability for those coordinating groups of researchers, or analysts who want to adopt a rigorous, systematic, and defensible process that can be implemented with fidelity on a consistent basis. The process can further be adapted for

JTPE Vol. 37, No. 2, 2018

230 Richards and Hemphill

D ow

nl oa

de d

by E

bs co

P ub

li sh

in g

ae ll

sw or

th @

eb sc

o. co

m o

n 05

/0 8/

18 , V

ol um

e ${

ar ti

cl e.

is su

e. vo

lu m

e} , A

rt ic

le N

um be

r ${

ar ti

cl e.

is su

e. is

su e}

those who prefer to analyze data manually, or through qualitative data analysis software.

In order to enhance transparency, researchers should be spe- cific about the methods used when analyzing data (Moravcsik, 2014). This can be done, in part, by identifying and implementing with fidelity a practical guide to analysis, such as the one advocated in this paper, or other examples in the literature (e.g., Braun & Clarke, 2006; Cornish et al., 2014; Hall et al., 2005). The process can then be specifically identified and cited in the methods, along with an explanation of any adaptations or deviations from original articulation. To further transparency, researchers may also com- municate why they use collaboration in qualitative research, and how they believe it enhances study results. In future qualitative methodology discussions, researchers should continue to consider more nuanced understandings of how collaboration enhances qualitative research. These conversations have the potential to capitalize on the benefits associated with multiple analysts, and thus could aid the design of future research.

Notes

1. While many researchers use terms such as “emergent” or “emerging” when discussing themes and the processes through which they are devel- oped (Taylor & Ussher, 2001), this language implies that the researcher plays a generally passive role in the creation of themes, or “if we just look hard enough they will ‘emerge’ like Venus on the half shell” (Ely, Vinz, Downing, & Anzul, 1997, p. 205). We, therefore, refer to themes as being generative so to emphasize the active role researchers play in generating them through qualitative data analysis.

2. While we agree with the perspective of Patton (2015), who is reluctant to apply the quantitatively oriented terms of “reliability” and “validity” to discussions of qualitative data analysis, we use them here because they are adopted by Olson and colleagues (2016). Our intent is to differentiate our desire to enhance trustworthiness and credibility from inter-coder agreement, which is more quantitatively driven.

References

American Political Science Association. (2012). Guide to professional ethics in political science (2nd ed.). Washington, DC: Author.

Boyatzis, R.E. (1998). Transforming qualitative information: Thematic analysis and code development. Thousand Oaks, CA: Sage.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77–101. doi:10.1191/ 1478088706qp063oa

Brewer, J., & Hunter, A. (1989). Multimethod research: A synthesis of styles. Thousand Oaks, CA: Sage.

Corbin, J., & Strauss, A. (1990). Grounded theory research: Procedures, canons, and evaluative criteria. Qualitative Sociology, 13, 3–21. doi:10.1007/BF00988593

Cornish, F., Gillespie, A., & Zittoun, T. (2014). Collaborative analysis of qualitative data. In U. Flick (Ed.), The Sage handbook of qualitative data analysis (pp. 79–93). Thousand Oaks, CA: Sage.

Ely, M., Vinz, R., Downing, M., & Anzul, M. (1997). On writing qualitative research: Living by words. London, UK: Routledge.

Gibbert, M., Ruigrok, W., & Wicki, B. (2008). What passes as a rigerous case study? Strategic Management Journal, 29, 1465–1474. doi: 10.1002/smj.722

Glaser, B.G., & Strauss, A. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago, IL: Aldine.

Guba, E. (1981). Criteria for assessing the trustworthiness of naturalistic inquiry. Educational Technology Research and Development, 29(2), 75–91.

Hall, W.A., Long, B., Bermback, N., Jordan, S., & Patterson, K. (2005). Qualitative teamwork issues and strategies: Coordination through mutual adjustment. Qualitative Health Research, 15, 394–410. PubMed doi:10.1177/1049732304272015

Hemphill, M.A., Richards, K.A.R., Templin, T.J., & Blankenship, B.T. (2012). A content analysis of qualitative research in the Journal of Teaching in Physical Education from 1998 to 2008. Journal of Tea- ching in Physical Education, 31, 279–287. doi:10.1123/jtpe.31.3.279

Kapiszewski, D., & Kirilova, D. (2014). Transparency in qualitative security studies research: Standards, beliefs, and challenges. Security Studies, 23, 699–707. doi:10.1080/09636412.2014.970408

Lincoln, Y.S., & Guba, E. (1985). Naturalistic inquiry. New York, NY: Sage. Miles, M.B., & Huberman, A.M. (1994). Qualitative data analysis: An

expanded sourcebook (2nd ed.). Thousand Oaks, CA: Sage. Moran-Ellis, J., Alexander, V.D., Cronin, A., Dickinson, M., Fielding, J.,

Sleney, J., & Thomas, H. (2006). Triangulation and integration: Processes, claims and implications. Qualitative Research, 6, 45–59. doi:10.1177/1468794106058870

Moravcsik, A. (2014). Transparency: The revolution in qualitative research. Political Science and Politics, 47, 48–53. doi:10.1017/ S1049096513001789

Neuendorf, K. (2017). The content analysis guidebook (2nd ed.). Thousand Oaks, CA: Sage.

Olson, J.D., McAllister, C., Grinnell, L.D., Walters, K.G., & Appunn, F. (2016). Applying constant comparative method with multiple investi- gators and inter-coder reliability. The Qualitative Report, 21(1), 26–42.

Patton, M.Q. (2015). Qualitative research and evaluation methods (4th ed.). Thousand Oaks, CA: Sage.

Rhoades, J.L., Woods, A.M., Daum, D.N., Ellison, D., & Trendowski, T.N. (2016). JTPE: A 30-year retrospective of published research. Journal of Teaching in Physical Education, 35, 4–15. doi:10.1123/ jtpe.2014-0112

Richards, K.A.R., Gaudreault, K.L., Starck, J.R., & Woods, A.M. (in press). Physical education teachers’ perceptions of perceived matter- ing and marginalization. Physical Education and Sport Pedagogy.

Richards, L. (1999). Qualitative teamwork: Making it work. Qualitative Health Research, 9, 7–10. doi:10.1177/104973299129121659

Shenton, A.K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22, 63–75. doi:10.3233/ EFI-2004-22201

Sin, C.H. (2007). Using software to open up the “black box” of qualitative data analysis in evaluations. Evaluation, 13, 110–120. doi:10.1177/ 1356389007073684

Strauss, A., & Corbin, J. (2015). Basics of qualitative research: Techni- ques and procedures for developing grounded theory (4th ed.). New York, NY: Sage.

Taylor, G.W., & Ussher, J.M. (2001). Making sense of S&M: A discourse analytic account. Sexualities, 4, 293–314. doi:10.1177/136346001 004003002

Taylor, S., Bogdan, R., & DeVault, M.L. (2015). Introduction to qualita- tive research methods: A guidebook and resource (4th ed.). New York, NY: Wiley.

Woods, A.M., & Graber, K. (2016). Interpretive and critical research: A view through a qualitative lens. In C.D. Ennis (Ed.), Routledge handbook of physical education pedagogies (pp. 21–33). New York, NY: Routledge.

JTPE Vol. 37, No. 2, 2018

Qualitative Data Analysis 231

D ow

nl oa

de d

by E

bs co

P ub

li sh

in g

ae ll

sw or

th @

eb sc

o. co

m o

n 05

/0 8/

18 , V

ol um

e ${

ar ti

cl e.

is su

e. vo

lu m

e} , A

rt ic

le N

um be

r ${

ar ti

cl e.

is su

e. is

su e}

Copyright of Journal of Teaching in Physical Education is the property of Human Kinetics Publishers, Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.