brian 2-19

"Chapter 11 Data Analysis and Findings: Examining what was Found in a Qualitative Study

If you are similar to most consumers of research, then the main reason you read research is to find out what researchers have learned about topics and problems that are important to you. You are therefore probably most interested in examining the findings when you read qualitative research reports. To understand qualitative findings, however, you need to know how researchers describe their qualitative analysis process in the Method sections of their reports in addition to understanding the forms by which they report their qualitative findings in the Results sections. In this chapter, you will learn to interpret information about how researchers analyze data and present their findings in qualitative research studies.

BY THE END OF THIS CHAPTER, YOU SHOULD BE ABLE TO:

Identify and understand the data analysis and findings of a qualitative research study.

List the steps that researchers report using to analyze qualitative data.

Know how to identify results in the form of description and themes when reading a qualitative research report.

Evaluate the quality of the data analysis and findings in a qualitative research report.

Research would not be research without the analysis of collected data. It is an essential step and, in qualitative research in particular, it can be the step in the research process that requires the most time and effort from the researcher. Despite all the work that researchers put into their qualitative data analysis, they often report only a paragraph or two about how they analyzed the data and instead focus much attention in their reports on the actual findings that emerged from the analysis. As a reader of research, however, you need to understand both the process researchers use to analyze qualitative data and how the results of this process are reported to fully appreciate and evaluate the findings presented in a qualitative research report. Let’s start by considering how you identify data analysis and results when reading qualitative research.

How Do You Identify the Data Analysis and Findings in a Qualitative Study?

Recall from Chapter 1 that data analysis is the process that researchers use to make sense of and summarize the data that they gathered in order to address their research questions. In Chapter 8, you learned that quantitative researchers use procedures such as descriptive and inferential statistics to analyze the numeric data that they gathered in their studies. Because qualitative researchers gather extensive amounts of text and image data as part of their studies, they use qualitative analytic procedures that are appropriate for analyzing open-ended text and visual data. You can find information about the procedures that a researcher used to analyze the data by reading the Method section of the report. The qualitative data analysis process results in products such as descriptions of people and themes about the study’s central phenomenon. These products are the findings of the qualitative study, which you can locate by turning to the report’s Results section. That is, you need to look in two sections of the report to learn about data analysis and findings.

Look to the Method Section for an Overview of the Qualitative Analysis Process

You can find authors’ discussions of the data analysis process in the Method section of a qualitative research article, usually immediately following the discussion of the data collection procedures. In many studies, the authors use a subheading of Data Analysis to help you find the information, but in other studies you need to figure out where in the Method section data analysis is discussed. Generally, authors provide only an overview of the steps they took to analyze the data. Although this description may be brief, it informs you about key steps that occurred in the analysis process. Read the following information that Haldenby et al. (2007) provided in the Method section of the qualitative adolescent-homelessness study for a description of their study’s data analysis process. As you read, think about the different steps that the researchers report taking to analyze the data they gathered in the study.

Data Analysis

All interviews were audio-recorded and transcribed verbatim as soon as possible following the interview. Transcripts were reviewed by the interviewer for accuracy. Once transcription was completed, a narrative style of analysis was conducted with the assistance of Atlas-Ti, a qualitative software program. This process involved several readings of the transcripts to capture initial impressions (Lieblich et al., 1998). More focused codes were then developed as ideas surfaced from the narratives. The code list was continuously revised to accommodate new perspectives and to collapse overlapping categories. The focused code list guided the analysis, and more abstract themes evolved from the transcribed stories. Attention was paid to both the content of the story and the way in which it was told (Lieblich et al., 1998).

Zimmerman and West (1987) have argued that society “invisibly” guides people to behave socially within the dichotomous norms of femininity and masculinity. It is therefore thought that gender is embedded in our everyday experiences (Zimmerman & West, 1987), which influence how youth create different meanings out of similar circumstances. Social understandings of gender and their influence on the narratives were therefore considered throughout the analysis. Ideas that emerged from the transcripts that did not fit the evolving code list were recognized as important and considered throughout the analysis. In all cases except for the one boy/girl group interview, young women’s and young men’s transcripts were initially analyzed separately. Finally, dominant themes were identified, and conclusions were made. (paragraphs 29–30)

This passage conveys details about the general steps used by researchers to analyze qualitative data. These steps included:

Preparing the data. The researchers transcribed the interview conversations from the audiorecordings made during each interview. The person who conducted the interview checked to make sure that each transcription was accurate. The transcripts were loaded into a special computer software program (called ATLAS.ti) that was used during the analysis.

Exploring the data. Once the transcripts were prepared, the researchers then read through them several times to become familiar with the information and to form some initial ideas about the data.

Coding the data. The researchers next began to assign “codes” to the data and worked to develop a refined list of codes that identified the major ideas and perspectives in the data. They also describe how their conceptual framework about the importance of gender informed aspects of their analysis and coding.

Developing description and themes. As a result of their coding, the researchers identified “abstract themes” that emerged from the analysis of the data. Themes are the larger patterns or ideas found across different sources of qualitative data. The major themes found in the data became the major findings of the study.

Validating the findings. The researchers also provided a little information about how they ensured that the findings were credible and trustworthy. Specifically, they noted that they actively attended to any ideas found in the data that did not seem to fit their list of codes. By stating that they were open to considering information that was contrary to their early results, the researchers are assuring readers that they can trust that the final results represent the perspectives that were in the data, not just what the researchers were expecting to find.

Look to the Results Section for the Qualitative Findings Produced by the Analysis Process

Authors typically devote a great deal of attention to describing what was found as a result of the qualitative data analysis process. Because of the importance of the results in a qualitative research article, they are reported in their own major section, immediately after the Method section. This section may be called Results like in most quantitative studies, but qualitative investigators often prefer to use the word Findings as the heading for this section because they feel it better conveys that the researcher was open to having learned from participants. A good strategy for reading a Findings section is to look at the opening paragraph for an overview of what was learned in the study to help you understand the details presented in the section. For example, read how Haldenby et al. (2007) provided an overview of their major findings at the start of the Findings section in the qualitative adolescent-homelessness study:

Findings

All participants appeared eager to share their stories and did so in an insightful way. From their narratives five themes emerged: (a) the realities of exiting street life, (b) negotiating dangerous terrain, (c) rethinking family, (d) the hazards of being female, and (e) the elusive nature of health and the health care system. Because critical research invites reflection into the contextual factors that shape and influence a person’s experience, a separate analysis of current Canadian policy was completed. These findings will be addressed in relation to the participants’ stories. (paragraph 31)

From reading this short paragraph, we can expect to locate information about five themes in the Findings section. In addition, we learn that the researchers describe the contexts (e.g., being poor and having trouble continuing their education) that emerged as important for understanding the participants’ homeless experiences. If we keep reading, we will find six subheadings in the Findings section—one for each of the five themes and one for the contexts. Description and themes are the typical products that result from the qualitative analysis process. To understand and evaluate the results that authors report in their qualitative studies, you next need to know more about the qualitative analysis process.

Here’s a Tip!

Look for subheadings to help you identify the major results of a qualitative study when you read the Findings section of a report. Often researchers use a subheading for each major theme that was found.

How Do You Understand a Study’s Qualitative Data Analysis?

Individuals who are unfamiliar with qualitative data analysis procedures often think that it must be very simple: The researcher talks to a few individuals and then writes up a summary of what was said. Good qualitative data analysis, however, is not simply summarizing what people had to say. Qualitative data analysis should be a systematic, rigorous, and thoughtful process that researchers use to uncover detailed descriptions of and larger patterns about the central phenomenon from the collected data. You need to be able to recognize the different activities involved in this process as they are described in reports and to assess whether the reported information indicates the use of good qualitative procedures.

You can visualize the major activities in the qualitative data analysis process by examining the bottom-up approach to qualitative data analysis in Figure 11.1. This process is called bottom-up (also referred to as inductive) because the researcher works from all the detailed data (e.g., typed interview texts or observational field notes) up to a

Figure 11.1 The Bottom-Up Approach to the Process of Qualitative Data Analysis

few general patterns (e.g., codes and themes). As shown in the figure, once qualitative researchers collect data, they implement several activities to inductively build up from the data. These activities include preparing the data for analysis, developing a general sense of the data, coding the data, developing findings in the forms of description and themes from the codes, and using strategies to ensure the accuracy of the findings.

The activities shown in Figure 11.1 can be challenging to identify in many qualitative reports because a good qualitative analysis process is an interpretive and dynamic process that is not easily reported in clearly defined steps. The process is interpretive because qualitative researchers make personal assessments throughout the process to determine a description that fits the situation or themes that capture the major categories of information. This means that each qualitative researcher brings his/her own perspective to all aspects of the qualitative analysis process. The process is considered dynamic because qualitative researchers often implement the different activities simultaneously and iteratively. Implementing the process simultaneously means that researchers are involved in more than one activity at any point in time (e.g., preparing the data from one interview while building results from an earlier interview). Implementing the process iteratively means that researchers cycle back and forth between data collection and analysis to ensure they develop the best understanding. For example, researchers might have collected stories from individuals, and as the analysis of the stories proceeds, they may return to the participants for more information to fill in gaps in their stories.

Knowing that qualitative data analysis is inductive, interpretive, simultaneous, and iterative should help you identify some important differences between how researchers report analyzing qualitative data compared to analyzing quantitative data. For example, in quantitative research the investigator completes data collection first and then moves to data analysis. The traditional quantitative process is much more linear than the simultaneous and iterative process used in most qualitative studies. In quantitative research, the researcher also works to remove him- or herself from the process by using procedures such as objective criteria to determine the outcomes of hypothesis tests, which differs from the interpretive and inductive process used in good qualitative research. Keeping these features in mind as you read about how researchers analyzed their qualitative data helps you understand that qualitative researchers use a data analysis process that is different from that used in quantitative research.

Although the qualitative data analysis process reported in a study can be complicated because of its inductive, interpretive, simultaneous, and iterative nature, a good strategy for understanding the overall process is to identify how the researcher implemented each of the major steps shown in Figure 11.1. The steps to look for include how the researchers:

prepared the data;

explored the data;

coded the data;

refined the codes and developed findings from the codes; and

validated the findings.

Let’s consider key elements that appear during each of these steps and what kinds of information you can learn when reading researchers’ reports of their qualitative data analysis process.

Step 1—Identify How the Researchers Prepared Their Data

The first step to look for is data preparation. Because of the vast amount of open-ended information collected in most qualitative studies, researchers must take steps to prepare an accurate and detailed record of the data that can be used during the analysis process. You should remember from Chapter 10 that qualitative researchers obtain images and sounds from documents or audiovisual materials and obtain words through interviewing participants or by writing field notes during observations. When researchers collect images, audiovisual materials, or documents, they usually prepare these data by creating digital copies (such as by scanning a document) that can be used for analysis purposes. When the data are in the form of words, such as from interviews or field notes, then data preparation is more involved. In these situations, researchers need to convert the collected words into typed text.

When considering the preparation of text data in a research study, read carefully to learn how the data were prepared and who did the preparation; also, ascertain the accuracy of the data preparation and the completeness of the data preparation. The most common procedure that researchers use to prepare text-based data from interviews and observations is called transcription. Transcription is the process of converting audio recordings or field notes into typed text. Researchers may transcribe the information themselves or they may use a transcriptionist to type the text files. It is best if the person who gathered the data prepares the transcription, but it is not always feasible, particularly in large projects with extensive databases. The most accurate procedure is for the researcher to type the transcriptions verbatim. Verbatim means that the researcher types all spoken words as well as unspoken events such as [pause] to indicate when interviewees take a lengthy break in their comments or [laughter] when the interviewee laughs. A verbatim transcript of a 30-minute interview typically results in 10–15 pages of single-spaced text, and qualitative studies that include several interviews include the preparation of hundreds of pages of verbatim transcripts. The most complete procedure occurs when the researcher transcribes all of the gathered interviews and observational notes so that all the data can be included in the formal analysis. In some studies, researchers report that they prepared summaries of the collected data instead of verbatim transcriptions. The use of summaries may occur when a recorder failed to record an interview because of mechanical problems or when the researcher’s resources for transcription are limited. Although summaries are sometimes necessary, verbatim transcription of the entire database is best because that provides the researcher with the most accurate and complete record of the collected data.

When reading about data preparation, also note how the researcher organized the database to facilitate the analysis process. Qualitative researchers organize their database in order to analyze the data by hand or by using a computer. Researchers report this detail so that readers better understand how they worked with the data during the full analysis process. Here is an overview of how researchers implement these two options so you can recognize them when reading reports:

Analyzing qualitative data by hand indicates that researchers read the data, mark them by hand, and divide them into parts by hand. Traditionally, analyzing text data by hand involves using colored highlighting markers to mark parts of the text or cutting and pasting (with scissors and tape!) text sentences onto cards. For example, note how Rushton (2004) described analyzing his data by hand in a study of one preservice teacher’s experience: “I began to index themes with various highlighted colors and with notes in the margins” (p. 65). For this style of analysis, the researcher prepares printed copies of the transcripts so they are available for marking. A hand analysis is most appropriate in projects with relatively small databases (e.g., less than 500 pages of text).

Analyzing qualitative data by using a computer indicates that researchers use a computer program to facilitate the process of analyzing qualitative data. Unlike the statistical computer programs used to analyze quantitative data, qualitative computer programs do not analyze the data for researchers. They do, however, provide several features that help researchers analyze qualitative databases. These features include storing data, enabling the researcher to assign labels or codes to the data, and facilitating searching through the data and locating text assigned to specific codes. (We will learn more about coding shortly.) Computer-assisted analysis is ideal for studies that have large databases. For this style of analysis, the researcher prepares the transcripts and uploads them into a qualitative data analysis program. You can note that researchers analyzed the data by using a computer when they name a specific software program in their reports. For example, Churchill et al. (2007) identified the particular software program (called MAXqda) used for data analysis in their study on rural family well-being by stating: “The qualitative portions of the one-on-one interviews were audiotaped and transcribed verbatim and entered into a MAXqda database for analysis” (p. 278).

Step 2—Note Whether the Researchers Explored Their Data

Once you know how the researchers prepared the qualitative data for their analysis, you should expect to find that they started the formal analysis by exploring the data. A preliminary exploratory analysis in qualitative research consists of the researcher reading through the data to obtain a general sense of the data, recording initial ideas, thinking about the organization of the data, and considering whether more data are needed. This step is important in qualitative research because the researcher should have an understanding of the data as a whole before starting to break the data into the different perspectives and ideas. Look for evidence of this exploratory analysis in research reports. For example, Landreman, Rasmussen, King, and Jiang (2007) noted this step in their study about multicultural educators when they wrote: “Each team member read every transcript to gain a sense of the whole experience communicated by participants” (p. 279).

A key strategy that researchers use when exploring the data is to keep records of their thoughts and ideas. These records are often referred to as memos in qualitative research. Researchers write memos in the margins of transcripts, under photographs, in personal journals, or with their computers or tablets to help in initially exploring the data. These memos are short phrases, ideas, concepts, or hunches that occur to the researcher. In a good qualitative analysis, researchers continue to memo their ideas during the entire data analysis to document their interpretations throughout the process; these memos often become many pages or files of the researcher’s personal reflections. Because the use of memos is indicative of good qualitative analysis procedures, you should note their use when reading about a study’s data analysis. For example, read how Havstam and colleagues (2011) mentioned their use of memos when describing the data analysis in their qualitative study of young adults who had been born with a cleft lip: “Memos were written by the first author during the analysis process and shared with the co-writers” (p. 25).

Step 3—Discern the Researchers’ Use of Coding

After reading about how the data were prepared and explored in a qualitative study, you should next look for information about how the researchers coded the data. Coding is a procedure where a researcher identifies segments of text (or images), places a bracket around them or highlights them, and assigns a code that describes the meaning of the text segment. Codes are labels that the researcher uses to describe the meaning of a segment of text or an image in relation to the study’s central phenomenon. Codes can address many different topics, including participants’ feelings, perspectives, strategies, contexts, behaviors, or language. For example, in a study about how patients manage their chronic pain, the researcher might identify the following text segment from an interview with a participant: “I am so worried about what will happen if my pain gets any worse. How will I continue to work and support my family if that happens?” When considering the meaning of this segment, the researcher might choose to use a code label of anxiety about the future because the participant is expressing his concern about future events. As the researcher continues to code the data in the study, she uses this code whenever patients describe concerns about the future. She also develops several different codes including, for example, codes about participants’ feelings (e.g., anxiety about the future and frustration), perspectives (e.g., the role of physicians and medication preferences), strategies (e.g., using a pillbox and resting), and contexts (e.g., home environment and work setting). Coding is at the heart of qualitative data analysis and is used in all qualitative research designs. It helps to make qualitative data analysis rigorous because the researcher considers all the ideas in the data by coding all of the gathered information; because the codes represent the researcher’s interpretations of the data, these interpretations are necessarily linked to and build from the data.

Researchers determine their codes in different ways. You should therefore look for information about how they chose their codes when reading reports of the data analysis process. In some qualitative projects, researchers start with specific topics of interest, and they use these topics as codes to identify the data that relates to those topics. For example, in a study about student learning, the researcher’s approach might be informed by a theory of motivation based on students’ expectations of success, perceptions of the value of the information, and interest in the information. In this case, the researcher might plan to use three codes based on this theory, such as (1) success expectations, (2) value perceptions, and (3) interest. In many qualitative analyses, however, the researchers do not start with predetermined codes. These researchers read the data and create codes based on their interpretation of the meaning of the data. This is often referred to as open coding in reports because the researcher is open to the ideas that occur in the data. In open coding, the codes can be phrased in standard academic terms (e.g., a researcher referring to “academic achievement”) or expressed in the researcher’s own language (e.g., a statement about “teachers working together”). Researchers also state codes in the participant’s actual words, which are called in vivo codes. For example, in a study of teacher evaluation where a participant noted that she feels like “big brother is watching,” the researcher may choose to use “big brother is watching” as a code in the analysis for data that represents this perspective of the teacher evaluation process. Researchers may use all of these types of codes when analyzing their data.

To help you understand the process that researchers use to code their data, ­Figure 11.2 shows a sample transcript from an interview that has been coded by a researcher who used open coding to develop codes from the data. This transcript resulted from an interview for a project exploring changes in the curriculum in a rural high school (Jones, 1999). Overall, the interviewee talks about changes in the high school from a school based on a traditional curriculum to a school based on service learning in the community. Jones asks questions during the interview and “LU,” the interviewee, provides responses. Notice the following features of the coding process illustrated in this figure.

The researcher located sentences that seem to “fit together” to describe one idea and drew a bracket around them. These represent text segments.

The researcher assigned a code label to each text segment and recorded the codes on the left side. The researcher used only two or three words for each code label. In

Figure 11.2 Example of How a Researcher Coded a Page from an Interview TranscriptSource: Reprinted with permission of Jean E. Jones, Ph.D.

some cases, these were the actual words used by the participant, “LU,” which are examples of in vivo codes.

The researcher also recorded reflections (e.g., “getting a good sense here for the community and its values”) and potential big ideas (e.g., “community”) on the right side as memos.

Although qualitative researchers almost always code their data, they seldom include examples of coded transcripts or list all of their codes in their reports. By understanding what it means to code, however, you can better understand the process that the researcher describes using in a study. It is often helpful to note whether the author provided examples of codes that were used because even a few examples can help you to understand the types of codes in a study. For example, Whitney et al. (2012) listed several codes used in their qualitative data analysis in their study of how teachers wrote reports for publication. These codes included environments, audience, authority, confidence, identity, and risk-taking (pp. 418–419).

Step 4—Examine How the Researchers Refined Their Codes and Used Them to Build Their Results

Although coding is a key part of qualitative data analysis, simply coding all of the data does not provide useful results in qualitative research. Researchers use procedures to refine their codes and group them together into larger ideas to build their results. As you read the Method section of a qualitative report, you should look for information that explains how the researchers built their findings from the codes and coded data. The visual model in Figure 11.3 will help you understand the process that researchers use. The overall objective of this process is for the inquirer to make sense of the database by dividing it into many text or image segments, labeling the segments with codes, examining codes for overlap and redundancy, and grouping these codes into broad themes. This is why qualitative data analysis is considered to be an inductive process because the researcher starts with lots of data segments and builds up from the data to several codes and then to a few themes.

As Figure 11.3 indicates, in addition to coding the data, qualitative researchers continually evaluate the list of codes they have generated. As they code more of the data, they work to combine codes that represent redundant ideas. For example, a researcher who initially has the codes happiness and enjoyment might decide they actually represent the same idea in the data and combine the text segments under one code of happiness. Researchers also work to group similar codes together during the coding process. For example, the codes of happiness, feeling at peace, and excitement might be grouped together by the researcher, who then labels this group as positive emotions found in the data.

As the researchers make sense of the data, they try to reduce the total number of codes to a manageable number, such as 20 or 30, which represent the most important ideas about the central phenomenon. As the list of codes is refined, the researcher uses the codes to build the study’s findings to form answers to the research questions. Describing and developing themes from the data consists of answering the major research questions and forming an in-depth understanding of the central phenomenon through description, thematic development, and relating themes. You will find researchers using an assortment of terms to refer to variations of the process of refining codes and building findings, such as analytic induction, constant comparison, or thematic development, but each of these can be understood by considering the same general procedure

Figure 11.3 A Model of the Process of Coding, Refining Codes, and Building Qualitative FindingsSource: Adapted from Creswell (2008).

depicted in Figure 11.3. By understanding this general qualitative procedure, you can make sense of the specific process described in the Method section as well as the findings included in the report’s Results section. Let’s examine how researchers build these three types of qualitative findings in more detail.

Researchers Build Description.

A description in qualitative research is a detailed rendering of people, places, or events that provide the context for a qualitative research study. The researcher uses codes that include descriptive information about the people, events, and places to access the text segments in the data that provide the descriptive information. For example, in a study about classroom learning environments, the researcher might use descriptive codes such as seating arrangements, teaching approach, or physical layout of the room to locate all the text in the database that specifically relates to the classroom in order to build a description of the classroom where instruction takes place. Building description is an important type of result in many qualitative research studies. In providing a detailed description as part of the results, the researcher aims to transport the reader to a research site or help the reader visualize a person. In some forms of qualitative research design, such as in ethnography or in case studies, the researcher pays careful attention to building a description of the setting during the data analysis. Developing detail is important for these designs, and the researcher analyzes data from all sources (e.g., interviews, observations, documents) to build a portrait of individuals and events.

Researchers Build Themes.

In addition to description, the development of themes is another way that researchers build results as they analyze qualitative data. Themes (also called categories) are similar codes aggregated together to form a major idea about the central phenomenon in the database. They form a core result from qualitative data analysis. Like codes, themes have labels that typically consist of only a few words (e.g., “self-tensions” or “science identities”). They represent larger patterns in the data that emerged from the analysis. Examine Figure 11.4 for an example of how Kirchhoff and Lawrenz (2011), in a study about the role of teacher education in the careers of science

Figure 11.4 An Example of Grouping Codes into a Thematic Category in a Qualitative Research StudySource: Kirchhoff, A., & Lawrenz, F. (2011). The use of grounded theory to investigate the role of teacher education on STEM teachers’ career paths in high-need schools. Journal of Teacher Education, 62(3), 250, Fig. 1. Copyright © 2011 by Sage Publications. Reprinted by permission of Sage Publications, Inc.

teachers, aggregated many codes together into the thematic category of “support” during their analysis. In the left box in Figure 11.4, the authors give examples of some of the questions that were asked during interviews to gather data from the participating science teachers. In the middle box, they list many of the codes (e.g., “Content area preparation,” “Cohort/Peers,” and “Pragmatics”) that emerged as they initially analyzed the data from these questions. Upon reflecting on the many codes, the authors refined the codes and grouped several of them together to form a category that they named “support.” In the right box of the figure, the authors identify this larger category (i.e., “support”) as well as list the subthemes (or “properties”) and different perspectives (or “dimensions”) found within the data for the “support” category. They used a similar process to develop several categories during their analysis, including “preparation for high-need settings,” “desire to make a difference,” and “educational role models.”

When reading about how a researcher developed themes in a qualitative study, try to identify how many themes emerged from the data analysis. A good rule of thumb is to expect qualitative researchers to develop five to seven themes from their codes as their qualitative results. The development of a small number of themes is appropriate so that the researcher can provide detailed information about a few themes rather than superficial information about many themes. The reason that the number of themes is typically not less than five is so that the results include a sufficient number of major ideas to adequately convey the complexity of the central phenomenon under study.

Researchers Relate Multiple Themes.

Many qualitative studies in the literature stop their data analysis with the development of description and themes. In more sophisticated studies, however, the authors discuss using procedures for relating the themes to each other. The researchers may describe these procedures in their Method section, such as in many grounded theory studies where the researchers report using specific procedures (called axial and selective coding) to relate the themes into a theory. In many studies, however, the researchers do not provide much detail about how they related multiple themes, and you have to deduce the use of these procedures by examining the results reported in the Results section (we will learn more about qualitative results later in this chapter). Two examples of how researchers relate multiple themes are layering the themes and interconnecting the themes. Understanding these two approaches will help you make sense of the procedures reported in many qualitative studies.

Layering the themes means that the researcher represents the data using embedded levels of themes. This occurs when the researcher subsumes minor themes within major themes and then includes major themes within broader themes. The entire analysis becomes more and more complex as the researcher works upward toward broader levels of abstraction. For example, Asmussen and Creswell (1995) layered their themes in their case study about a campus reaction to a gunman incident by subsuming their five major themes (denial, fear, safety, retriggering, and campus planning) within two broader themes (organizational and psychological/social-psychological). In an ethnographic study of how parents of color support their children’s preparation for college, Auerbach (2007) subsumed her six major themes (e.g., mode of support and locus of support) within three types of parent roles that she named “moral supporters,” struggling advocates,” and “ambivalent companions” (p. 260).

Interconnecting the themes means that the researcher connects the themes into some sort of a larger order of ideas. For example, this larger order might be a chronology, such as when researchers connect the themes to tell a larger story found in the data. Plano Clark et al. (2002) interconnected their five major themes in a larger story in their study of adolescent tobacco use. Note how the following statement explains these connections:

As we report these themes in the next section, we organize them into a story line about adolescent use of tobacco. This story line reports how individuals begin smoking; how smoking becomes a pervasive influence in school lives; how attitudes are formed about smoking at school and in personal lives; how these attitudes, in turn, shape what it means to be a smoker; and, ultimately, how these experiences influence student suggestions for tobacco use prevention. (p. 1269)

Connecting themes into a chronology can be useful in different forms of qualitative research, including narrative research in particular. Researchers also interconnect themes into a sequence of relationships, such as when qualitative researchers generate a theoretical model in grounded theory studies. These interconnections convey how certain themes are found to influence or relate to other themes and researchers often develop a visual model to depict the connections. For example, Churchill and colleagues (2007) described interconnecting their themes about how rural low-income families have fun during analysis with the following statement:

As the thematic categories emerged from the data, we proceeded to the step of theory development. We continually refined the organization and interrelation of the thematic categories during the coding process and a conceptual visual model was developed based on the qualitative findings of this analysis. (p. 278)

Step 5—Identify the Strategies the Researchers Used to Validate Their Results

Once you have identified the previous aspects of the qualitative data analysis process, the last step is to look for evidence that the researchers used strategies to validate the quality of their findings. Validating findings means that the researchers use strategies to ensure the accuracy and credibility of the findings as part of the analysis process. In qualitative research, the findings that emerged from the data analysis process should be accurate and credible representations of the gathered data and participants’ experiences. Other terms that authors use to convey the quality of their results are trustworthiness and dependability. All of these terms aim to ensure that the researchers used a good data analysis process while also acknowledging that this process is very interpretive and subjective. Therefore, qualitative researchers use strategies to validate their findings that emphasize the nature of qualitative data and the qualitative data analysis process. Although you may find authors describing many different strategies for validating their qualitative findings in reports, our attention here is on four forms frequently reported by qualitative researchers: bracketing, triangulation, member checking, and auditing.

Because qualitative data analysis is an interpretive process, researchers reflect on their personal viewpoints and how they shape their interpretations of the data. One way that researchers address this issue is through bracketing. Bracketing is a process by which a researcher reflects on his or her own views and experiences related to the study’s central phenomenon, describes these perspectives in writing, and then works to set them aside (or “bracket” them) during the analysis process. Although personal bias can never be totally eliminated, bracketing helps to ensure that the researcher’s perspectives do not overwhelm the perspectives of the participants and therefore enhances the credibility of the study’s findings.

Qualitative inquirers also triangulate information from different data sources to enhance the credibility of a study. Triangulation is the process of corroborating evidence about a finding from different individuals (e.g., a principal and a student) or types of data (e.g., observational field notes and interviews). The inquirer examines each information source and finds evidence to support a theme. This helps to ensure that the themes found in a study are credible representations of people’s experiences and perspectives because the information draws on multiple sources of information or individuals.

Researchers check their findings with participants in the study to determine whether their findings are accurate. Member checking is a process in which the researcher asks one or more participants to check the accuracy of the findings. This process involves taking the findings back to participants and asking them (in writing or in an interview) about the accuracy of the report. Researchers ask participants about many aspects of the study, such as whether the description is complete and realistic, the themes are appropriate, and the interpretations are fair and representative of their perspectives.

Researchers may also ask a person outside the project to conduct an audit of the procedures used in a study. This can be as simple as having a second researcher independently code some of the data to see whether there is agreement on the major ideas found in the data. Some studies may include a peer review of the entire study’s methods. A researcher using peer review discusses his/her research process with a knowledgeable colleague to review the data collection and analysis procedures as they unfold during the study. The most formal procedure occurs when a researcher obtains the services of an individual outside the study to review different aspects of the research and report back, in writing, the strengths and weaknesses of the project. This formal process is called an external audit and is the most extensive of the audit strategies.

What Do You Think?

In a qualitative study aimed at understanding women’s experiences of divorce, Thomas and Ryan (2008) provided the following information about their data analysis process. As you read the passage, consider which sentences relate to each of the five steps for understanding a study’s data analysis process: (1) data preparation, (2) data exploration, (3) data coding, (4) code refinement and building results, and (5) validation strategies. Each sentence is labeled [A, B, and so on] to make it easier to refer to the individual statements.

[A] Following the interviews the researcher transcribed the audiotapes verbatim. . . . [B] The data analysis was completed using a method similar to Strauss and Corbin’s (1998) microanalysis techniques and procedures for grounded theory. [C] This method required the researcher to read the transcriptions line by line several times to become immersed in the data. [D] Notes were made to identify specific words or phrases that stood out to construct the initial categories that are termed microanalysis. [E] Specific themes were identified from the phrases to construct initial categories based on the coding process. [F] Validity was addressed in this study through checking the responses with the participants and by creating an atmosphere of openness for the researcher and participants. [G] Evidence was weighed with existing literature to validate findings. [H] Contrasts and comparisons were made based on feedback sought from informants to validate findings as recommended by Miles and Huberman (1994).

(Thomas & Ryan, 2008, p. 213)

Check Your Understanding

The authors provided information that relates to each of the five steps for understanding a qualitative data analysis process.

Data preparation. They indicated that the data were transcribed verbatim in [A], but did not provide any indication as to whether the analysis was conducted by hand or assisted by a computer in this extract.

Data exploration. They indicated the use of a preliminary exploratory analysis in [C] by referring to reading each transcript “to become immersed in the data.”

Data coding. They mentioned that they constructed “categories” in [D] and specifically referred to their use of a “coding process” in [E].

Code refinement and building results. The authors indicated the development of themes from the categories in [E]. If we keep reading the full article, we learn that this process identified eight themes that are reported as the study’s findings.

Validation strategies. The authors also discussed several strategies that they used to ensure the quality of the study’s findings. In [F] and [H], they referred to forms of member checking in which they had participants react to and provide feedback about the results of the analysis. In [G], they mentioned the use of a strategy of comparing the emergent findings with the existing literature.

How Do You Understand the Findings in a Qualitative Study?

Once you have read how researchers analyzed their qualitative data in the Method section of a qualitative report, you should next read their findings in response to their research questions. As mentioned earlier in this chapter, the section of a qualitative report where you can locate the findings of the study is typically labeled as Results or Findings. Although the use of one of these terms is most common, as you learned in Chapter 2, qualitative research reports tend to be more flexible in their structure than are quantitative research reports. This flexible structure means that you may read qualitative studies where the results are not so clearly labeled. In some qualitative reports, you may find that the researchers titled the results section based on the content of the findings and/or included more than one results section. For example, in a study of a high school theater program, Larson and Brown (2007) used both of these approaches and followed their description of the study’s methods with two sections where they reported the results of their analysis. They titled these results sections “The Experiential Setting of Les Misérables” (p. 1087) and “What Youth Learned and How” (p. 1091). There are even a few qualitative reports that include some results before the Method section. Although unusual, this may occur when a description of the people, setting, or event that is the focus of the study needs to be included early in the report so that the reader can understand the study’s context. For example, in a case study of the reaction of a campus to a specific gunman incident, Asmussen and Creswell (1995) reported their description of the event in a section titled “The Incident and Response” before they discussed the methods used in the study.

Once you have located the findings reported in a qualitative study, your next step is to read and understand these findings. Researchers report their qualitative findings in order to answer their qualitative research questions about the study’s central phenomenon. Therefore, it can be helpful to reread the study’s research questions before reading the results because knowing what questions are being answered may help you to understand the findings that are reported. It is also useful to specifically look for several key characteristics of qualitative research findings as you read. These characteristics include:

findings that report description,

findings that report themes,

tables and figures that convey additional details and complexity, and

findings that fit the study’s research design.

Let’s learn about each of these characteristics, which should help you understand the findings reported in a qualitative research study.

Read Descriptive Findings to Learn the Context of the Study and the Central Phenomenon

When researchers analyze their databases for description, they report this description as part of their results. Recall that description is a detailed rendering of people, places, or events that are examined in a qualitative research study. You may find description reported as part of the results of any qualitative study, but you should particularly expect to find it when reading a case study or ethnography because those approaches describe a case or a culture-sharing group. When you read a good qualitative description, it should effectively transport you into the context of the research and help you get to know the people, setting, or events that are important in the study. These descriptions are key results to read because they provide you with information about the context for the study so that you can better understand the study’s central phenomenon and better consider whether the results might be relevant to and useful for your own context.

When you read description about people in a qualitative study, you learn about the participants involved in the research. For example, in an ethnographic study about home literacy practices of Chinese Canadian families with young children, Li (2006) provided a description about the three families who were the focus of the study. These descriptions included many details, such as when each family immigrated to Canada, the background and professions of the parents, and the parents’ language proficiency and attitudes toward living in Canada. Likewise, when you read the description about a study’s setting, you learn about the environment in which the research took place. In a study about the stress experienced by music teachers at one high school, for example, Scheib (2003) included a description of the school that illustrated the larger community in which the school was located, including details such as the school’s size, student demographics, and how each school day is structured. You will also find that researchers report descriptions of events important to research studies, such as specific programs, a typical school day, or a particular activity. Padilla (2003) conducted a qualitative study about one woman’s experience of disability and included in his report a dramatic description of the events surrounding the accident that caused her serious head injury. In all of these examples, the descriptive results provide important contexts for more fully understanding the qualitative studies and their central phenomena.

Qualitative researchers use several features for reporting description. Knowing these features can help you read descriptive passages in reports, and understand and assess the presented information. To learn how to read qualitative description, examine part of a descriptive passage from a qualitative study about a professional development program for new teachers (Clayton, 2007). The passage in Figure 11.5 describes one of the teachers in the study, as taken from the author’s report. The discussion of this teacher and events in her classroom illustrates several features of description used in good qualitative research reports, which have been highlighted with the labels in the margin of the figure. The features of good qualitative description include:

A broad-to-narrow description. The descriptive passage starts with a broad focus of the day and teacher’s context; then the focus narrows to the classroom’s details and finally to the events in the classroom on this one day. Look for a broad-to-narrow description to understand the broader context of the study and to develop a sense of the real place where the study is located.

Vivid details. The author uses vivid details to create the description. We know, for example, what is posted on the bulletin boards and what announcements are written on the chalkboard. Noticing the vivid details helps you better understand the uniqueness of the particular people and setting in a study.

Figure 11.5 An Example of Description as a Finding in a Qualitative Report

Focus on facts and what occurred. The author does not make an interpretation or evaluate the situation—she simply reports the facts as recorded in the data sources. Keep in mind that descriptive results are different from the themes that researchers report as their interpretations of larger patterns in the data. (We will learn more about themes in the next section.)

Action words. The action comes alive with the use of action verbs and movement-oriented modifiers and adjectives. The teacher did not simply take the papers; she “ripped up” the papers. Good description should make you feel like you are in the setting as you as read, and action words help to bring the description to life.

Participant quotes. The passage includes quotes (i.e., the exact words of participants) to provide emphasis and realism in the account. These quotes are usually short because researchers are typically limited in the amount of space they have available to report their study. However, you should notice that good qualitative description includes specific details like quotes that are drawn from the researcher’s data collection.

By paying attention to these features of good qualitative description, you can consider to what extent the researcher has provided you with a compelling picture of the people, places, and events that illustrate the context for the qualitative research study.

Examine Themes to Learn the Larger Ideas Found About the Study’s Central Phenomenon

The most common form of qualitative results that you will find are themes. Recall that themes are major ideas about the central phenomenon that emerge when the researcher groups several codes together during the analysis. You will find themes reported as the results of qualitative studies using all of the common designs, including grounded theory, case study, ethnography, and narrative studies. Typically, researchers report five to seven themes in the Results section of their study to convey both the major ideas and the complexity of the central phenomenon under study. Remember that a key characteristic of qualitative research is to explore the complexity of topics and that complexity should be evident in the themes because they describe several major aspects of the central phenomenon. The themes that emerge from a qualitative study are the key results to examine to discern what the researchers discovered about the study’s central phenomenon. As you read about the themes in a qualitative study, consider whether each theme clearly relates to the study’s central phenomenon and represents a major idea about it.

Because themes are the primary findings of most qualitative studies, researchers typically state how many themes resulted from the analysis in their reports and give a name to each theme in the Findings section. When reading the Findings section, look at the major headings that are included, as they most often represent the name of the themes. For example, in the qualitative physical-activity-at-daycare study (Tucker et al., 2011) from Chapter 1, the authors used headings in their Results section to name the four major themes that emerged about ways to enhance preschoolers’ activity levels at daycare, including: “Enhanced staff training/workshops” and “Additional equipment and resources.” The names of themes can take many different forms, but a good theme name provides you with a clear sense of the larger idea that it represents in the researchers’ interpretation of the analyzed data. Theme names might be stated as nouns, verbs, phrases, or even questions. The names of themes can be as short as a single word. Brown and colleagues (2006) used this form for several themes in their study of patients waiting for a liver transplant. Their themes included “Transformation,” “Loss,” and “Coping.” Another common form occurs when researchers include participants’ own words in the name of their themes. We read an example of this strategy in the qualitative adolescent-homelessness study at the end of Chapter 4. Haldenby et al. (2007) combined participants’ quotes with their own interpretations to derive theme names, such as “‘I Can’t Really Feel Safe’: Negotiating Dangerous Terrain” and “‘They’re in the Same Situation as You’: Rethinking Family.”

As you learned for descriptive findings, qualitative researchers also use several features for reporting themes, and knowing these features can help you understand and assess results reported as themes. To learn how to read qualitative themes, examine the discussion of the “students’ experiences” theme found in a study about adolescent perceptions of tobacco use (Plano Clark et al., 2002) that is provided in Figure 11.6. This passage describes one of five themes reported in the study. The theme passage illustrates several of the features used in good qualitative research reports, some of which have been highlighted with the labels in the margin of the figure. The features of good qualitative themes include:

Subthemes within each major theme. Recognize that authors report major themes as well as subthemes subsumed under each major theme. These subthemes often represent the different codes aggregated together to form the theme during the analysis process. In Figure 11.6, the major theme is “What Are the Students’ Experiences with Tobacco in High Schools?” Under this major theme, the authors discuss the following

Figure 11.6 An Example of a Theme as a Finding in a Qualitative Report

subthemes: prevalence of tobacco use, where tobacco is used, and students feeling desensitized. In some reports, the authors use subheadings to indicate the subthemes, but often you need to recognize the different ideas within the theme passage as you read. The use of subthemes helps to convey the complexity of the findings, and you should look for the different concepts included when you read any theme.

Multiple perspectives and contrary evidence. Researchers include multiple perspectives about a theme (and subthemes) in their qualitative findings. The term multiple perspectives means that the researcher provides several viewpoints for a theme based on different individuals, different sources of information, or different views held by one person. In Figure 11.6, for example, the authors report the perspectives of many students, including those from different school settings. For instance, we learn different locations where participants perceive tobacco is used, such as public places, personal places, and school grounds. Multiple perspectives are important features of good themes because they convey the complexity of the central phenomenon in qualitative research. Good qualitative themes should include examples of the many different perspectives held by participants, not just the most common or popular perspective.

The use of different perspectives might also include contrary evidence, such as when participants or the information from different data sources disagree with each other. Although the theme presented in Figure 11.6 did not include contrary evidence, we can imagine what such evidence might have been. For example, had the authors found that some students perceived that very few people use tobacco at their school, they would have included that perspective as contrary to the general idea of tobacco use being prevalent. When reports include contrary evidence, you learn a complex and realistic picture of the central phenomenon through different individuals’ experiences. Good qualitative themes highlight the different perspectives that the author found in the data; by noticing these perspectives as you read, you learn about the range of views held about the study’s central phenomenon.

Here’s a Tip!

Good qualitative findings include multiple perspectives and points of view. This means that the researcher has learned about the complex perspectives that people hold about the topic. This is different from quantitative research that emphasizes determining average trends.

Participant quotes as evidence. As with qualitative descriptive findings, authors include the exact words from participants as quotes when reporting their qualitative themes. The presence of quotes in themes provides you with the actual words of participants, which makes the results seem more real and informs you of the language that participants use when talking about the topic. Researchers select quotes that capture feelings, emotions, and ways people talk about their experiences, such as learning that students refer to a place to smoke tobacco as “smoker’s corner.”

In addition, and perhaps more importantly, authors include quotes as their evidence for the information about a theme that they are reporting. Recall that themes emerge during data analysis based on the personal interpretations of the researchers. Therefore, researchers include actual excerpts from their data to support the interpretations that they report in their themes. For example, at the end of the passage in Figure 11.6, the authors report their interpretation that students are desensitized to tobacco use and provide two students’ quotes as supporting evidence for this claim. Therefore, when you read quotes reported about a theme, consider whether the information from participants supports the interpretation made by the authors.

The form of the quotes can vary across different reports. The passage in Figure 11.6 uses short quotes as evidence throughout the theme. In some reports, however, you may find that the authors include long quotes or even actual dialogue from participants to provide support for themes. For example, in a study about students’ perceptions of effective teachers, Howard (2002) provides this dialogue between a teacher (Hazel) and one of her students to support the theme “making school seem like home.”

Hazel [speaking to student]:

Where’s your book report?

Student:

I don’t have it. (long pause) I didn’t finish it.

Hazel:

You didn’t finish it? (with emphasis) What are you waiting for to get it done? Christmas?

Student:

No.

Hazel:

Alright then, get it done! (p. 433)

No matter the form of the quotes, pay close attention to quotes included for each theme to learn how participants talk about the theme and to judge whether the author has included sufficient evidence for you to feel that the theme represents participants’ experiences and perspectives.

Inclusion of literary devices such as metaphors, analogies, and tensions. Literary devices are useful for conveying the richness of qualitative findings, and you will read many qualitative reports that incorporate literary forms such as highlighting tensions and contradictions as part of their results. For example, in the study about adolescent perceptions of tobacco use in Figure 11.6, the authors go on to discuss “a number of contradictions” (p. 1276) that they found in the data and themes, such as adolescents perceiving that their schools support tobacco use despite the fact that all the schools had clear policies against its use. Another common literary device used in qualitative findings is the use of metaphors and analogies. For example, in reporting on the competition and concerns surfacing during the implementation of distance education in the state of Maine, Epper (1997) writes metaphorically about how student and citizen support is a “political football” game:

As the bickering went on, many students stood by watching their education dreams tossed around like a political football. Some were not content to sit on the sidelines. From the islands, valleys, backwoods, and far reaches of the state came letters to the faculty senates, legislators, and newspapers. (p. 566)

Noting the use of such literary devices when you read about a theme helps you to develop a more rich and in-depth picture of the theme and central phenomenon. The presence of such devices in a report of qualitative themes can help convince you that the researcher completed an in-depth exploration of the central phenomenon and has provided rich information about it.

By looking for these features of good qualitative themes when you read qualitative findings in a report, you can consider to what extent the themes seem based on the views of participants and have provided you with a rich understanding of the complexity of the central phenomenon of the study.

What Do You Think?

Consider how Haldenby et al. (2007) reported the findings in the form of themes in the qualitative adolescent-homelessness study from Chapter 4. Specifically, read the theme titled, “‘They’re in the Same Situation as You’: Rethinking Family,” located in paragraphs 40–45 and starting on page 148. List four features of good qualitative themes that you find in the discussion of this qualitative finding.

Check Your Understanding

This theme includes many good features of qualitative findings. Examples are as follows: (a) The larger theme includes several subthemes (e.g., feeling betrayed by their families, feeling disconnected from peers, and feeling connected with other homeless youth). (b) The theme includes multiple perspectives of participants (e.g., several types of violence experienced by the youth mentioned in paragraph 40). (c) The theme incorporates quotes from participants’ interview data for richness and as evidence for the researchers’ interpretations (e.g., short and long quotes included in paragraph 42). (d) The theme uses a metaphor to convey a key idea (e.g., referring to other homeless youth as “family” in the theme name).

Read Tables and Figures to Learn More About the Details and Complexity of the Findings

In addition to describing the findings in sentences and paragraphs, qualitative researchers often display their findings visually by using tables and figures that augment the discussion (Miles & Huberman, 1994). These displays provide important information about the findings in qualitative studies, often including details beyond what is provided in the text and conveying the complexity of the findings in a way that is difficult to do with words only. As you read qualitative findings, pay close attention to the displays included in the report to develop the most complete understanding of what the researchers found in their study. Although no two displays from different studies are exactly alike, there are a few common types of displays that researchers include in their qualitative reports. Learning to recognize these types makes it easier for you to understand the information presented visually. Five common visual displays that you will encounter include demographic tables, map figures, comparison tables, hierarchical tree figures, and thematic figures.

Demographic tables. Researchers include demographic tables in their reports to describe personal or demographic information for each person or site in the research. Demographic tables provide key information about the participants and settings to help you further understand the study’s context. Although demographic tables often appear in the Findings section of qualitative reports, some authors include them when describing the participants in the Method section. In either case, they enhance the description of the study’s participants and setting. For example, in a study of how high school teachers use technology in their classrooms, the researcher described each instructor and his or her primary delivery style in a demographic table, shown in Table 11.1. The seven participants in this qualitative study demonstrated different personal characteristics as well as diverse approaches to using technology, which the researcher wanted to highlight for the readers. This table provides readers with various demographic information for each teacher, such as number of years teaching, gender, class level of instruction, instructional approach used in the class, and his or her primary form of technology use. Use demographic tables like this to understand the important characteristics of the settings and participants in qualitative studies.

Map figures. Researchers may include maps to depict the physical layout of the study’s setting. Maps provide important details such as the relative placement of objects and how people interact within the study’s setting. As shown in Figure 11.7, Miller et al. (1998) display the physical setting of a soup kitchen in their study. The authors provided this figure so that readers could visualize where different activities happened as they were described in the report. Use figures that display maps to understand the important details of a setting and, as you read the results, examine how those details relate to the study’s description and themes.

Table 11.1 A Sample Demographic Table Used to Present Descriptive Information in a Qualitative Report

Name

Years Teaching

Gender

Class Level of Instruction

Instructional Approach in the Classroom

Primary Form of Technology in the Classroom

Amanda

25

Female

12

Discussion

Internet

Debbie

20

Female

11

Critique

Not used

Michelle

 5

Female

11

Discussion

Tablets

Nancy

10

Female

12

Interactive

Wireless laptops

Theresa

 6

Female

11

Discussion

Electronic whiteboard

Tim

 4

Male

10

Lecture

Internet

Yuchun

12

Female

10

Small Groups

Electronic whiteboard and tablets

Figure 11.7 A Sample Map Figure of the Physical Layout of a Setting in a Qualitative ReportSource: Miller, D. M., Creswell, J. W., & Olander, L. S. (1998). Writing and retelling multiple ethnographic tales of a soup kitchen for the homeless. Qualitative Inquiry, 4(4), 475. Copyright © 1998 by Sage Publications. Reprinted by permission of Sage Publications, Inc.

Comparison tables. In some studies, researchers create visuals of the information in the form of a comparison table, a table that compares groups on one or more of the study’s themes (e.g., freshmen and seniors in terms of “students’ experiences with tobacco”). These tables are particularly useful when they highlight different perspectives found in the data. For example, in a qualitative study of teachers, one theme that emerged was the meaning of “professionalism.” In developing this theme, the researcher analyzed statements gathered from both female and male teachers in a school and noticed that there were differences in the perspectives based on the gender of the participants. The table depicted in Table 11.2 compares the statements made by the female and male participants about their approaches to professionalism so that the reader can examine how they differed. When you find a comparison table in a qualitative report, use it to understand the different perspectives that the researcher found in the data about a theme and how those differences related to a certain grouping characteristic, such as gender.

Table 11.2 A Sample Comparison Table Used to Compare Two Groups for a Theme in a Qualitative Report

Statements About “Professionalism” From Female Participants

Statements About “Professionalism” From Male Participants

■ Helping fellow teachers is part of my day.

■ When another teacher asks for advice, I am generally a good listener.

■ It is important, once I achieve a certain level of experience, that I become a mentor to other teachers, especially new ones.

■ Caring about how other teachers employ high standards in their classroom is a sign of my own professionalism.

■ Being concerned about following the coordinator’s advice about curriculum shows a high level of professionalism.

■ It is important to be in charge in the classroom and to be aware of student off-task behavior.

■ I set standards for myself, and try to achieve these goals each year.

■ It is necessary that each teacher “pull” his or her weight in this school—a sure sign of professionalism.

Hierarchical tree figures. Recall that researchers often relate multiple themes as part of their qualitative analysis. Because these relationships can be difficult to describe clearly in words, researchers may include figures that convey the relationships among the themes as part of the results. One type of figure that relates multiple themes is the hierarchical tree. Researchers use hierarchical tree figures to visually represent themes and the multiple layers among the themes. The hierarchical tree diagram in Figure 11.8 illustrates the major themes (layer 1) and broader categories (layer 2) that describe a campus’s response to a gunman incident (layer 3) found in a case study of the incident (Asmussen & Creswell, 1995). Use the information displayed in hierarchical tree figures such as this example to understand how themes relate to each other, particularly which ideas combine together to form larger ideas about the central phenomenon.

Thematic figures. In some qualitative studies, researchers relate multiple themes by interconnecting them into a larger order. It is very common for researchers to use a thematic figure to display how the themes are interconnected by using boxes to convey the themes and arrows to convey the relationships among them. This type of figure is common in studies using a grounded theory design because the researcher analyzes the data with the explicit purpose of developing a theory of how ideas are interconnected. For example, Figure 11.9 depicts the findings from a grounded theory study by Harley and colleagues (2009) that examined the process by which African American women who are physically active adopt and maintain their physical activity practices. The authors identified numerous categories in the boxes, organized the categories within larger phases, and used arrows to show the

Figure 11.8 A Sample Hierarchical Tree Figure to Portray Thematic Layers in a Qualitative ReportSource: Adapted from Asmussen & Creswell (1995).

Figure 11.9 A Sample Thematic Figure That Interconnects Themes in a Qualitative ReportSource: Harley, A. E., Buckworth, J., Katz, M. L., Willis, S. K., Odoms-Young, A., & Heaney, C. A. (2009). Developing long-term physical activity participation: A grounded theory study with African American women. Health Education and Behavior, 36(1), 102, Fig. 1. Copyright © 2009 by Sage Publications. Reprinted by permission of Sage Publications, Inc.

connections among the boxes. When describing the results, the authors discussed each of the phases (i.e., initiation, transition, and integration), the two “loops” that describe how the process was ceased or modified, and the context and conditions for the overall process.

Another way that researchers use thematic figures is to show the interrelationships that exist among themes instead of a sequential order. In Auerbach’s (2007) study of parent roles, for example, she identified a typology of parent roles that emerged from the different perspectives across the major themes in her study. In Figure 11.10, she portrayed how these different roles (moral supporters, ambivalent companions, and struggling advocates) overlapped with each other as well as differed along important dimensions (such as how proactive the parents were in their role). When you find thematic figures such as these in reports, use them to not only identify the major themes of a qualitative study, but to also understand how the multiple themes relate to each.

Consider the Form of the Findings in Relation to the Research Design

All qualitative research reports include findings in the form of description, themes, and/or the relationships among multiple themes. As you read reports, you will notice that some studies tend to emphasize one of these types of findings (e.g., description)

Figure 11.10 A Sample Thematic Figure That Relates Themes in a Qualitative ReportSource: Auerbach, S. (2007). From moral supporters to struggling advocates: Reconceptualizing parent roles in education through the experience of working-class families of color. Urban Education, 42(3), 259, Fig. 1. Copyright © 2007 by Sage Publications. Reprinted by permission of Sage Publications, Inc.

and others emphasize another type (e.g., the relationships among multiple themes). Some of this variation reflects the style of different researchers and their disciplines, but it also reflects the different qualitative research designs that researchers use. Recall from Chapter 9 that reports tend to include different types of findings depending on the research design. Because of the flexible nature of qualitative research, there are no strict rules for how results are reported for each specific research design. Each of the major qualitative design types, however, tends to emphasize certain types of results. Here are some guidelines to keep in mind as you read different types of qualitative research:

If you are reading a study that used a general qualitative approach, expect that the findings simply include a report of the themes that emerged from the study. The report of basic themes is probably the most common form for qualitative research findings.

If you are reading a study that used narrative research, expect that the findings include a chronological retelling of the participant’s story, a description of the context for the story, and themes that emerged from the story.

If you are reading a study that used case study research, expect that the findings include an in-depth description of each case, themes that emerged about each case, and if multiple cases were studied, another layer of themes that emerged across all the cases.

If you are reading a study that used ethnographic research, expect that the findings include a detailed description of how a group behaves, thinks, and talks as well as the context, or setting, of the group.

If you are reading a study that used grounded theory research, expect that the findings include the themes (or categories) that emerged, a display of the interconnections among the themes, and a discussion of the theory that emerged about the interconnections and the hypotheses (or propositions) suggested by these connections.

How Do You Evaluate the Data Analysis and Findings in a Qualitative Study?

When evaluating the quality of a qualitative study’s data analysis and findings, keep in mind that these two aspects of the report are strongly tied to each other. Researchers describe how they implemented the data analysis process in the Method section and the results of that process in the Findings section. Good qualitative research involves the researcher systematically analyzing the qualitative data using a process that is inductive, interpretive, and dynamic to ensure results emerge that provide an accurate and credible portrayal of the information found within the collected data. Identifying the use of key aspects of the qualitative analysis process such as memoing and coding helps you judge the extent to which the researchers used good procedures in their study. Likewise, recognizing the features of good qualitative findings helps you assess the quality of the presented results as you consider whether they provide detailed and in-depth answers to the study’s research questions. Table 11.3 lists criteria that are useful to consider when evaluating the data analysis and findings in a qualitative study. This table also provides indicators of higher quality and lower quality for the criteria to help you make your own judgments when evaluating the information provided in a research report.

Figure 11.11 provides a convenient means for you to apply the quality criteria to evaluate the data analysis described within the Method section and findings presented in the Results section of any qualitative research report. For each of the criteria you locate, assign a quality rating from fair (1) to excellent (3) and document your evidence and/or reasoning behind the rating. If one of the criteria is missing or very poorly stated, then indicate poor (0) as your rating. Keep in mind that research reports vary in the extent of discussion about the data analysis process and that the nature of the qualitative findings varies depending on the study’s purpose and research design. Even with this variation, however, a good qualitative study should

Table 11.3 Criteria for Evaluating the Data Analysis and Findings in a Qualitative Research Report

Quality Criteria

Indicators of Higher Quality in Qualitative Research

Indicators of Lower Quality in Qualitative Research

The Key Elements

1. The analysis process used rigorous qualitative procedures.

+ The researcher prepared verbatim transcripts and electronic scans of all gathered data.

+ The researcher read the data to get a sense of the whole and recorded personal interpretations in memos.

+ The researcher developed and assigned codes to segments based on the meaning of the data.

+ The researcher refined the list of codes during the analysis and built results from the refined codes.

− The researcher prepared summaries of the gathered data or did not prepare all of the gathered data.

− The researcher started coding without first reading through the data and did record personal interpretations.

− The researcher used predetermined codes that did not come from the data.

− The process by which the researcher built results from the data is unclear.

2. Strategies were used to validate the findings.

+ The researcher used at least three strategies to ensure the accuracy and credibility of the findings such as bracketing, triangulation, member checking, or audits.

− The researcher did not employ multiple strategies to ensure the accuracy and credibility of the findings.

3. The findings include a description of the people, places, or events in the study.

+ The important context of the study is described with rich detail using a broad-to-narrow form, factual information, action words and modifiers, participant quotes, and tables or figures that provide additional details.

− The description of the context of the study is boring and dry, provides little detail, or is not even included.

4. The findings include appropriate themes about the central phenomenon.

+ There are five to seven themes reported.

+ The themes convey both major ideas about and the complexity of the central phenomenon using sub-themes, multiple perspectives and contrary evidence, participant quotes as evidence, literary devices to suggest the researcher’s interpretations, and comparison tables.

− There are too few (i.e., too simplified analysis) or too many (i.e., not enough analysis) themes.

− The themes lack subthemes within the larger themes, include only the common perspectives, have insufficient evidence in the form of quotes, or do not include literary devices that suggest the researcher’s interpretations.

5. The findings relate multiple themes to each other.

+ The researcher reports the relationships among the themes, such as the multiple layers of categories or interconnecting the themes into a sequence or larger order, conveys the relationships in the text and with a figure, and develops theme relationships that are consistent with the overall research design.

− The researcher reports themes without explaining how they relate to each other, does not adequately describe or depict the relationships, or relates the themes in a way that is inconsistent with the overall research design.

General Evaluation

6. The data analysis represents a good qualitative process.

+ The data analysis was an inductive, interpretive, and dynamic process from which credible and accurate results emerged based on the gathered data.

− The data analysis was a deductive (top-down), objective, fixed, or linear process from which results were produced that are not strongly tied to the gathered data.

7. The findings provide a good exploration of the central phenomenon.

+ The findings provide rich and detailed information that answers the study’s research questions.

− The findings provide superficial answers or do not clearly answer the study’s research questions.

still score well on most of the items listed in Figure 11.11. By adding up the rating scores for each of the criteria and using the suggested cutoff values provided at the bottom of the figure, you will have a quantitative measure that you can use to inform your overall assessment.

Figure 11.11 A Rating Scale for Evaluating the Data Analysis and Findings in a Qualitative Research Report

Reviewing What You’ve Learned To DoTo assess what you’ve learned to do, click here to answer questions and receive instant feedback.

■

Identify and understand the data analysis and findings of a qualitative research study.

The Method section of a qualitative research report contains information about the process that the authors used to analyze their qualitative data, which includes how they prepared the data, explored the data, coded the data, developed findings from the data, and validated the findings.

The Results section of a qualitative research report contains the findings that emerged from the data analysis process in response to the study’s research questions.

■

List the steps that researchers report using to analyze qualitative data.

Qualitative data analysis is a process that inductively builds from the data to larger patterns, that involves the researcher making personal interpretations, where several steps occur simultaneously, and that takes place iteratively with data collection as early results point for the need for more data.

Researchers prepare their data using procedures such as transcription to develop complete and accurate records of the gathered data.

Researchers explore the data by reading the information to get a sense of the whole and memoing their ideas about the data.

Researchers code the data by breaking the information into segments and assigning code labels to the segments that represent the meaning of the information in the segment.

Researchers refine their codes, aggregate them into larger ideas, and use the codes and coded data to build description, themes, and relationships among themes as the findings.

Researchers use strategies to validate their findings to ensure that they are credible and accurate.

■

Know how to identify results in the form of description and themes when reading a qualitative research report.

Description is a type of qualitative finding where the researcher reports a detailed description of the setting, people, or events that provide the context for the study.

Themes (or categories) are a type of qualitative finding in which the researcher reports a small number of major ideas that emerged about the study’s central phenomenon from the gathered data.

Tables and figures augment the descriptive and thematic findings by providing more detail or depicting the complexity within the findings, such as the layers or interconnections among themes.

The findings may emphasize description, themes, and/or relationships among themes, depending on the study’s overall qualitative research design.

■

Evaluate the quality of the data analysis and findings in a qualitative research report.

The evaluation of a study’s qualitative data analysis considers the completeness and accuracy of the data preparation; the extent to which the researcher developed emergent findings by exploring, coding, and refining the codes about the data; and the strategies used to validate the accuracy and credibility of the findings.

The evaluation of a study’s qualitative findings considers the extent to which the description is rich and detailed so that readers feels like they were present within the setting; the extent to which the themes and interconnections among the themes convey major ideas about the central phenomenon as well as the complexity of the central phenomenon; and the extent to which the findings answer the study’s research questions.

✓ To assess what you’ve learned to do, click here to answer questions and receive instant feedback.

Reading Research ArticlesCarefully reread the qualitative adoption-of-pedagogical-tools study by Leko and Brownell (2011) found at the end of Chapter 9 (starting on p. 306) and the qualitative adolescent-homelessness study by Haldenby et al. (2007) at the end of Chapter 4 (starting on p. 148).As you review each article, pay close attention to statements in which the authors described the data analysis process in the Method section and the results in the Findings section. Use the highlighting tool in the Pearson etext to indicate where the authors have provided information about the data analysis and findings, and use the notetaking tool to add marginal notes that name each element you highlighted and note how each one is related to the study’s qualitative approach. Among the elements you will want to find are:

Preparing the data

Exploring the data

Coding the data

Developing findings

Validating the findings

Results as description

Results as themes

Results as relationships among themesNote, however, that sometimes authors do not describe all of the steps of the data analysis process—for example, they might not mention preparing the data or exploring the data. In addition, authors do not always report all types of results—that is, they might not report description, they might not report themes, or they might not report relationships among themes. If one of these steps or type of results is missing, indicate that in your marginal notes.

Click here to go to the qualitative adoption-of-pedagogical-tools study by Leko and Brownell (2011) so that you can enter marginal notes about the study.

Click here to go to the qualitative adolescent-homelessness study by Haldenby et al. (2007) so that you can enter marginal notes about the study.

Understanding Research ArticlesApply your knowledge of the content of this chapter to the qualitative adoption-of-pedagogical-tools study by Leko and Brownell (2011) starting on p. 306.

What evidence can you find about how the authors prepared the gathered data for analysis in the adoption-of-pedagogical-tools study?

What evidence can you find that the authors explored the data, coded the data, and refined the codes and developed findings from the codes?

What strategies did the authors use to validate their findings?

The major types of qualitative findings include description, themes, and relationships among themes. Which of these types of findings did the authors report?

Consider the findings in paragraphs 46–49 under the heading of “Tricia.” What type of finding is this? What features of good qualitative research did the authors use in these paragraphs?

Consider the findings in paragraphs 68–69 under the heading of “Component Concept: Personal Qualities.” What type of finding is this? What features of good qualitative research did the authors use in these paragraphs?

Consider the tables and figures that appear in this article. (a) How does Table 1 help you learn more about the study’s findings? (b) How does Figure 1 help you to learn more about the study’s findings?

✓ Click here to answer the questions and receive instant feedback.

Evaluating Research ArticlesPractice evaluating a qualitative study’s data analysis and findings, using the qualitative adoption-of-pedagogical-tools study by Leko and Brownell (2011) starting on p. 306 and the qualitative adolescent-homelessness study by Haldenby et al. (2007) starting on p. 148.

Use the criteria discussed in Table 11.3 to evaluate the quality of the data analysis and findings in the adoption-of-pedagogical-tools study. Note that, for this question, the rating form includes advice to help guide your evaluation.

✓ Click here to open the rating scale form (Figure 11.11) to enter your ratings, evidence, and reasoning.

Use the criteria discussed in Table 11.3 to evaluate the quality of the data analysis and findings in the adolescent-homelessness study. Note that, for this question, the rating form does NOT include additional advice.

✓ Click here to open the rating scale form (Figure 11.11) to enter your ratings, evidence, and reasoning."