U8A1-68 - TUTOR PLEASE FOLLOW THE INSTRUCTIONS AS OUTLINED TO COMPLETE THIS HOMEWORK....Data Analysis Strategies...SEE ATTACHED FILES

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 1/39

CHAPTER

8 Qualitative Analysis and Interpretation

In set theory, an empty set is denoted by a pair of empty braces: { }. A set is defined as a collection of things that are brought together because they have something in common. In mathematical set theory, the items in a set must obey a clear, definitive rule, for example, whole numbers that are multiples of 2 (2, 4, 6, 8, 10, . . . ) or words that begin with the letters Qu and have only one syllable (Queen, Queer, Quinn, Quit, . . . ). The rule must be without ambiguity, so that it is clear and uncontested that an item does or does not belong in the set.

In qualitative analysis, in contrast, what items belong in a set (e.g., a grouping, a category, a pattern, a theme) is a matter of judgment. Judgments can vary depending on who is doing the judging, with what criteria, and for what purpose. Thus, unlike rules, judgments can be ambiguous. Love and hate may be considered in the same set (strong emotions) or may be judged to belong in different sets (things that bring humans together vs. things that divide us). What constitutes a set in the game of tennis is clear, defined by a rule. What constitutes a set in qualitative analysis must be defined and created anew each time the game, qualitative analysis, is played. Play on.

Chapter Preview Part 1 of the book provided an overview of qualitative inquiry, with chapters on the nature, niche, and value of qualitative inquiry; strategic themes in qualitative inquiry; a variety of qualitative inquiry frameworks (paradigmatic, philosophical, and theoretical orientations); and practical and actionable qualitative applications. Part 2 covered qualitative designs and data collection, with chapters covering purposeful sampling and design options, fieldwork strategies and observation methods, and qualitative interviewing. Part 3 presents the two final chapters: Chapter 8, “Qualitative Analysis and Interpretation,” followed by Chapter 9, “Enhancing the Quality and Credibility of Qualitative Studies.”

Module 65 in this chapter opens by covering the basics of analysis, with a focus on establishing a strong foundation for qualitative analysis. Module 66 presents the importance of thick description and constructing case studies. Module 67 turns to pattern, theme, and content analysis. Module 68 looks in depth at the intellectual and operational work of analysis. Module 69 presents logical and matrix analyses and explains how to synthesize qualitative studies. Module 70 takes on the critical processes of interpreting findings and determining substantive significance, with special attention to phenomenological and hermeneutic examples. Module 71 examines causal explanation thorough qualitative analysis. Module 72 opens the window on new analysis directions: contribution analysis, participatory analysis, and qualitative counterfactuals. Module 73 provides advice and examples on writing up and reporting findings, including using visuals. Module 74 addresses special analysis and reporting issues—mixed methods and focused communications—and provides a principles-focused report exemplar. Finally, Module 75 summarizes and concludes the chapter, plus providing case study exhibits. We begin with basic analysis.

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 2/39

SOURCE: Brazilian cartoonist Claudius Ceccon. Used with permission.

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 3/39

MODULE

65 Establishing a Strong Foundation for Qualitative Analysis:Covering the Basics

Good field methods are necessary, but not sufficient, for good research. You may be a skilled and diligent observer and interviewer and gather “rich data,” but, unless you have good ideas about how to focus the study and analyze those data, your project will yield little of value.

—William Foot Whyte (1984, p. 225) Learning From the Field

The Challenge Qualitative analysis transforms data into findings. No formula exists for that transformation. Guidance yes, but no recipe. Direction can and will be offered, but the final destination remains unique for each inquirer, known only when—and if—arrived at.

Medieval alchemy aimed to transmute base metals into gold. Modern alchemy aims to transform raw data into knowledge, the coin of the Information Age. Rarity increases value. Fine qualitative analysis remains rare and difficult—and therefore valuable.

Metaphors abound. Analysis begins during a larval stage that, if fully developed, metamorphoses from a caterpillar-like beginning into the splendor of the mature butterfly. Or this: The inquirer acts as a catalyst on raw data, generating an interaction that synthesizes new substance born anew of the catalytic conversion. Or this: Findings emerge like an artistic mural created from collage-like pieces that make sense in new ways when seen and understood as part of a greater whole.

Consider the patterns and themes running through these metaphors: transformation, transmutation, conversion, synthesis, whole from parts, and sense making. Such motifs run through qualitative analysis like golden threads in a royal garment. They decorate the garment and enhance its quality, but they may also distract attention from the basic cloth that gives the garment its strength and shape—the skill, knowledge, experience, creativity, diligence, and work of the garment maker. No abstract processes of analysis, no matter how eloquently named and finely described, can substitute for the skill, knowledge, experience, creativity, diligence, and work of the qualitative analyst. Thus, Stake (1995) writes classically of the art of case study research. Van Maanen (1988) emphasizes the storytelling motifs of qualitative writing in his ethnographic book on telling tales. Golden-Biddle and Locke (2007) make story the central theme in their book Composing Qualitative Research. Corrine Glesne (2010), a researcher and a poet, begins with the story analogy, describing qualitative analysis as “finding your story,” then later represents the process as “improvising a song of the world.” Lawrence-Lightfoot and Davis (1997) evoke “portraits” in naming their form of qualitative analysis The Art and Science of Portraiture. Brady (2000) explores “anthropological poetics.” Janesick (2000) uses the metaphor of dance in “the choreography of qualitative research design,” which suggests that, for warming up, we may need “stretching exercises” (Janesick, 2011). Hunt and Benford (1997) call to mind theatre as they use “dramaturgy” to examine qualitative inquiry. Denzin (2003) and Hamera (2011) call for ethnography to be “performative.” Richardson (2000b) reminds us that qualitative analysis and writing involve us not just in making sense of the world but also in making sense of our relationship to the world and therefore in discovering things about ourselves even as we discover things about some phenomenon of interest. In this complex and multifaceted analytical integration of disciplined science, creative artistry, skillful crafting, rigorous sense making, and personal reflexivity, we mold interviews, observations, documents, and field notes into findings.

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 4/39

The challenge of qualitative analysis lies in making sense of massive amounts of data. This involves reducing the volume of raw information, sifting the trivial from the significant, identifying significant patterns, and constructing a framework for communicating the essence of what the data reveal. In analyzing qualitative data, guidelines exist but no recipes; principles provide direction, but there is no significance test to run that determines whether a finding is worthy of attention. No ways exist of perfectly replicating the researcher’s analytical thought processes. No straightforward tests can be applied for reliability and validity. In short, no absolute rules exist, except perhaps this: Do your very best with your full intellect to fairly represent the data and communicate what the data reveal given the purpose of the study.

SIDEBAR

DISTINGUISHING SIGNAL FROM NOISE

When you try to locate a clear radio station signal through the static noise that fills the airways between signals, you are engaged in the process of distinguishing signal from noise. Nate Silver (2012) used that metaphor as the title of his best-selling book on “why so many predictions fail—but some don’t.” Silver found that the best predictions—whether of election outcomes, economic patterns, social trends, spread of disease, winning sports teams, stock market indicators, or any of the many arenas in which humans attempt predictions—are those that use both quantitative and qualitative data and use theory to turn data into a feasible, meaningful, and compelling story. Data are noise, lots and lots of noise. The more data, the more noise. Big data: loud noise. The story that detects, makes sense of, interprets, and explains meaningful patterns in the data is the signal.

But signals are not constant or static. They vary with context and change over time. So the quest to distinguish signal from noise is ongoing.

Frameworks for analyzing qualitative data can be found in abundance (Saldaña, 2011), and studying examples of qualitative analysis can be especially helpful, as in the Miles, Huberman, and Saldaña (2014) qualitative analysis sourcebook. But guidelines, procedural suggestions, and exemplars are not rules. Applying guidelines requires judgment and creativity. Because each qualitative study is unique, the analytical approach used will be unique. Because qualitative inquiry depends, at every stage, on the skills, training, insights, and capabilities of the inquirer, qualitative analysis ultimately depends on the analytical intellect and style of the analyst. The human factor is the great strength and the fundamental weakness of qualitative inquiry and analysis—a scientific two-edged sword.

That said, let’s get on with it. Exhibit 8.1 offers 12 tips for laying a strong foundation for qualitative analysis. These tips are neither exhaustive nor universal. They won’t all apply to everyone, but perhaps they will stimulate you to think of other things you might do to get yourself ready for the challenges of qualitative analysis. I’ll elaborate several of these tips and then delve into alternative ways of conducting qualitative analysis.

Elaboration of Some Tips for Ensuring That a Strong Foundation for Qualitative Analysis Begins During Fieldwork

Field methods have the advantage of flexibility, allowing us to explore the field, to refine or change the initial problem focus, and to adapt the data gathering process to ideas that occur to us even in late stages of our exploration.

—Whyte, (1984, p. 225)

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 5/39

Research texts typically make a hard-and-fast distinction between data collection and analysis. For data collection based on surveys, standardized tests, and experimental designs, the lines between data collection and analysis are clear. But the fluid and emergent nature of naturalistic inquiry makes the distinction between data gathering and analysis far less absolute. In the course of fieldwork, ideas about directions for analysis will occur. Patterns take shape. Signals start to emerge from the noise. Possible themes spring to mind. Thinking about implications and explanations deepens the final stage of fieldwork. While earlier stages of fieldwork tend to be generative and emergent, following wherever the data lead, later stages bring closure by moving toward confirmatory data collection—deepening insights into and confirming (or disconfirming) patterns that seem to have appeared. Indeed, sampling, confirming, and disconfirming cases require a sense of what there is to be confirmed or disconfirmed.

Ideas for making sense of the data that emerge while still in the field constitute the beginning of analysis; they are part of the record of field notes. Sometimes insights emerge almost serendipitously. When I was interviewing recipients of MacArthur Foundation fellowships (popularly dubbed “Genius Awards”), I happened to interview several people in major professional and personal transitions, followed by several in quite stable situations. This happenstance of how interviews were scheduled suggested a major distinction that became important in the final analysis—distinguishing the impact of the fellowships on recipients in transition from those in stable situations.

Recording and tracking analytical insights that occur during data collection is part of fieldwork and the beginning of qualitative analysis. I’ve heard graduate students being instructed to repress all analytical thoughts while in the field and to concentrate on data collection. Such advice ignores the emergent nature of qualitative designs and the power of field-based analytical insights. Certainly, this can be overdone. Too much focus on analysis while fieldwork is still going on can interfere with the openness of naturalistic inquiry, which is its strength. Rushing to premature conclusions should be avoided. But repressing analytical insights may mean losing them forever, for there’s no guarantee they’ll return. And repressing in-the-field insights removes the opportunity to adapt data collection to test the authenticity of those insights while still in the field and fails to acknowledge the confirmatory possibilities of the closing stages of fieldwork. In the MacArthur Fellowship study, I added transitional cases to the sample near the end of the study to better understand the varieties of transitions the fellows were experiencing—an in-the-field form of emergent, purposeful sampling driven by field-based analysis. Such overlapping of data collection and analysis improves both the quality of the data collected and the quality of the analysis, so long as the fieldworker takes care not to allow these initial interpretations to overly confine analytical possibilities. Indeed, instead of focusing additional data collection entirely on confirming emergent patterns while still in the field, the inquiry should become particularly sensitive to looking for alternative explanations and patterns that would invalidate the initial insights.

EXHIBIT 8.1 Twelve Tips for Ensuring a Strong Foundation for Qualitative Analysis

1. Begin analysis during fieldwork: Note and record emergent patterns and possible themes while still in the field. Add confirming cases to deepen analysis and possible disconfirming cases to test thematic ideas while still in the field.

2. Inventory and organize the data: Make sure you have all the interviews, observations, and documents that constitute the raw data of your qualitative inquiry. Check that the data elements and sources are labeled, dated, and complete.

3. Fill in gaps in the data: As soon as possible, fill in the gaps in the data while connections in the field are fresh. If later interviews turn up issues that need to be checked out with earlier interviewees, do so quickly. If documents are missing, take steps to get them.

4. Protect the data: Back them up. Make sure the data are secure. 5. Express appreciation: Thank those who have provided you with data. Fieldwork creates

relationships. Once out of the field, analysis and writing can take a lot of time. Don’t wait until it’s

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 6/39

all done to show your appreciation to those who have provided you with data. Follow up with appropriate expressions of appreciation sooner rather than later. Procrastination can too easily lead to never getting it done. Do it!

6. Reaffirm the purpose of your inquiry: Restate the purpose of your enquiry, and therefore the purpose of your analysis. Purpose drives analysis. Design frames and sets the stage for analysis. Be clear about why you’re doing this work, revisit and reengage with the questions and the purposeful sampling strategy that have guided your inquiry, and get clear about the primary product that you are producing that will fulfill your study’s purpose and answer priority questions in accordance with your design.

7. Review exemplars for inspiration and guidance: Reexamining classic works in your field can be a source of inspiration. They are classics for a reason. Keep those exemplars nearby to reinvigorate and motivate you when the drudgery of analysis sets in or doubts about what you’re finding emerge. Those who wrote the classics experienced analysis fatigue and doubts as well. They persevered. So will you.

8. Make qualitative analysis software decisions: If you’re using qualitative data management software, learn how to use it effectively. Locate technical support. Practice data entry and some simple analysis. All qualitative analysis software has a steep learning curve. Leave time and mental space to learn if you’re new to the software. If you’re an old hand, check out new versions and features that might be useful.

9. Schedule intense, dedicated time for analysis: Qualitative analysis requires immersion in the data. It takes time. Make time. Set a realistic schedule. Enlist the support of family, friends, and colleagues to help you stay focused, and give the analysis the dedicated time it deserves.

10. Clarify and determine your initial analysis strategy: Inductive qualitative analysis can follow a number of pathways. Various theoretical traditions (ethnography, phenomenology, constructivism, realism, etc.) provide frameworks and guidance. Data can be organized and reported in different ways: case studies, question-by-question interview analysis, storytelling, elucidating sensitizing concepts or principles, and thematic analysis, among others. Grounded theory provides a highly prescriptive framework for analysis. Decide what strategy fits your purpose, write it down with your rationale, and get started analyzing. This process involves reconnecting with the theoretical and strategic framework that presumably guided design decisions and the formulation of your inquiry questions.

11. Be reflective and reflexive: Monitor your thought processes and decision-making criteria. Be in touch with predispositions, biases, fears, hopes, constraints, blinders, and pressures you’re under. Qualitative analysis is ultimately highly personal and judgmental. You are the analyst. Observe yourself. Learn about yourself and your analysis processes, both cognitively and emotionally.

12. Start and keep an analysis journal: Document analysis decisions, emergent ideas, forks in the road, false starts, dead ends, breakthroughs, eureka moments, what you learn about analysis, what you learn about the focus of the inquiry, and what you learn about yourself. You may think you’ll remember these things. You won’t. Document the analytical process—in depth, systematically, and regularly. That documentation is the foundation of rigor. Qualitative analysis is a new stage of fieldwork in which you must observe and document your own processes even as you are doing the analysis.

See elaboration of these tips in the next sections.

In essence, when data collection has ended and it is time to begin the formal and focused analysis, the qualitative inquirer has two primary sources to draw from in organizing the analysis: (1) the questions that were generated during the conceptual and design phases of the study, prior to fieldwork, and (2) the analytic insights and interpretations that emerged during data collection.

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 7/39

Even when analysis and writing are under way, fieldwork may not be over. On occasion, gaps or ambiguities found during analysis cry out for more data collection—so, where possible, interviewees may be recontacted to clarify or deepen responses, or new observations are made to enrich descriptions. This is called member checking—verifying data, findings, and interpretations with the participants in the study, especially key informants. While writing a Grand Canyon–based book that describes modern male coming-of-age issues (Patton, 1999), I conducted several follow-up and clarifying interviews with my two key informants and returned to the Grand Canyon four times to deepen my understanding of Canyon geology and add descriptive depth. Each time I thought that, at last, fieldwork was over and I could just concentrate on writing, I came to a point where I simply could not continue without more data collection. Such can be the integrative, iterative, and synergistic processes of data collection and analysis in qualitative inquiry. A final caveat, however: Perfectionism breeds imperfections. Often, additional fieldwork isn’t possible, so gaps and unresolved ambiguities are noted as part of the final report. Dissertation and publication deadlines may also obviate additional confirmatory fieldwork. And no amount of additional fieldwork can, or should, be used to force the vagaries of the real world into hard-and-fast conclusions or categories. Such perfectionist and forced analysis ultimately undermines the authenticity of inductive, qualitative analysis. Finding patterns is one result of analysis. Finding vagaries, uncertainties, and ambiguities is another.

Inventory and Organize the Raw Data for Analysis

It wasn’t curiosity that killed the cat.

It was trying to make sense of all the data curiosity generated. —Halcolm

The data generated by qualitative methods are voluminous. I have found no way of preparing students for the sheer mass of information they will find themselves confronted with when data collection has ended. Sitting down to make sense out of pages of interviews and whole files of field notes can be overwhelming. Organizing and analyzing a mountain of narrative can seem like an impossible task.

How big a mountain? Consider a study of community and scientist perceptions of HIV vaccine trials in the United States done by the Centers for Disease Control. A large, complex, multisite effort called Project LinCS: Linking Communities and Scientists, the study’s 313 interviews generated more than 10,000 pages of transcribed text from 238 participants on a range of topics (MacQueen & Milstein, 1999). Now that’s an extreme case, but on average, a one-hour interview will yield 10 to 15 single-spaced pages of text; 10 two- hour interviews will yield roughly 200 to 300 pages of transcripts.

Getting organized for analysis begins with an inventory of what you have. Are the field notes complete? Are there any parts that you put off to write later but never got to doing that need to be finished, even at this late date, before beginning analysis? Are there any glaring holes in the data that can still be filled by collecting additional data before the analysis begins? Are all the data properly labeled with a notation system that will make retrieval manageable (dates, places, interviewee-identifying information, etc.)? Are interview transcriptions complete? Assess the quality of the information you have collected. Get a sense of the whole.

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 8/39

©2002 Michael Quinn Patton and Michael Cochran

Fill in Gaps in the Data The problem of incomplete data is illustrated by the experience of a student who had conducted 30 in-depth pre- and post interviews with participants in a special program. The transcription process took several weeks. She made copies of three transcripts and brought them to our seminar for assistance in doing the analysis. As I read the interviews, I got a terrible sinking feeling in my stomach. While other students were going over the transcriptions, I pulled her aside and asked her what instructions she had given the typist. It was clear from reading just a few pages that she did not have verbatim transcriptions—the essential raw data for qualitative analysis. The language in each interview was the same. The sentence structures were the same. The answers were grammatically correct. People in natural conversations simply do not talk that way. The grammar in natural conversations comes out atrocious when transcribed. Sentences hang incomplete, interrupted by new thoughts before the first sentence is completed. Without the knowledge of this student, and certainly without her permission, the typist had decided to summarize the participants’ responses because “so much of what they said was just rambling on and on about nothing,” the transcriber later explained. All of the interviews had to be transcribed again before analysis could begin.

Earlier, I discussed the transition from fieldwork to analysis. Transcribing offers another point of transition between data collection and analysis as part of data management and preparation. Doing all or some of your own interview transcriptions (instead of having them done by a transcriber), for example, provides an opportunity to get immersed in the data, an experience that usually generates important insights. Typing and organizing handwritten field notes offer another opportunity to immerse yourself in the data, a chance to get a feel of the cumulative data as a whole. Doing your own transcriptions, or at least checking them by listening to the tapes as you read them, can be quite different from just working off transcripts done by someone else.

Protect Your Data Thomas Carlyle lent the only copy of his handwritten manuscript on the history of the French Revolution, his masterwork, to philosopher John Stuart Mill, who lent it to a Mrs. Taylor. Mrs. Taylor’s illiterate housekeeper thought it was waste paper and burned it. Carlyle reacted with nobility and stoicism and immediately set about rewriting the book. It was published in 1837 to critical acclaim and consolidated Carlyle’s reputation as one of the foremost men of letters of his day. We’ll never know how the acclaimed version compared with the original, or what else Carlyle might have written in the year lost after the fireplace calamity.

So it is prudent to make backup copies of all your data, putting one master copy away someplace secure. Indeed, if data collection has gone on over any long period, it is wise to make copies of the data as they are collected, being certain to put one copy in a safe place where it will not be disturbed and cannot be lost, stolen, or burned. One of my graduate students kept all of his field notes and transcripts in the truck of his car.

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 9/39

The car was vandalized, and he lost everything, with no backup copies. The data you’ve collected are unique and precious. The exact observations you’ve made, the exact words people have spoken in interviews—these can never be recaptured in precisely the same way, even if new observations are undertaken and new interviews are conducted. Moreover, you’ve likely made promises about protecting confidentiality, so you have an obligation to take care of the data. Field notes and interviews should be treated as the valuable material they are. Protect them.

Beyond Thomas Carlyle’s cautionary tale, my advice in this regard comes from two more recent disasters. I was at the University of Wisconsin when antiwar protestors bombed a physics building, destroying the life work of several professors. I also had a psychology doctoral student who carried her dissertation work, including all the raw data, in the back seat of her car. An angry patient from a mental health clinic with whom she was working firebombed her car, destroying all of her work. Tragic stories of lost research, while rare, occur just often enough to remind us about the wisdom of an ounce of prevention.

Once a copy is put away for safekeeping, I like to have one hard copy handy throughout the analysis, one copy for writing on, and one or more copies for cutting and pasting. A great deal of the work of qualitative analysis involves creative cutting and pasting of the data, even if done on a computer, as is now common, rather than by hand. Under no circumstances should one yield to the temptation to begin cutting and pasting the master copy. The master copy or computer file remains a key resource for locating materials and maintaining the context for the raw data.

Qualitative data analysis software (QDAS) facilitates saving data in multiple digital locations, such as external hard drive, server, DVD copy, and flash drive. However, the researcher must be disciplined enough to update backups frequently and not destroy all old copies, especially preserving the original master copy.

Purpose Drives Analysis

“Data” linked to real human social worlds is where human social science, whatever else it does, has to start and has to finish.

—Michael Agar (2013, p. 19) The Lively Science

Purpose drives analysis: This follows from the theme of Chapter 5, that purpose drives design. Design gives a study direction and focus. Chapter 5 presented a typology of inquiry purposes: basic research, applied research, summative evaluation research, formative evaluation, and action research. (See Exhibit 5.1, p. 250.) These distinct purposes inform design decisions and inquiry focus and subsequently undergird analysis because they involve different norms and expectations for generating, validating, presenting, and using findings.

Basic qualitative research is typically reported through a scholarly monograph or published article, with primary attention to the contribution of the research to social science theory. The theoretical framework within which the study is conducted will heavily shape the analysis. As Chapter 3 made clear, the theoretical framework for an ethnographic study will differ from that for ethnomethodology, heuristics, or hermeneutics.

Applied qualitative research may have a more or less scholarly orientation depending on primary audience. If the primary audience is scholars, then applied research will be judged by the standards of basic research, namely, research rigor and contribution to theory. If the primary audience is policymakers, the relevance, clarity, utility, and applicability of the findings will become most important.

For scholarly qualitative research, the published literature on the topic being studied focuses the contribution of a particular study. Scholarship involves an ongoing dialogue with colleagues about particular questions of interest within the scholarly community. The analytical focus, therefore, derives in part from

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 10/39

what one has learned that will make a contribution to the literature in a field of inquiry. That literature will likely have contributed to the initial design of the study (implicitly or explicitly), so it is appropriate to revisit that literature to help focus the analysis.

Focus in evaluation research should derive from questions generated at the very beginning of the evaluation process, ideally through interactions with primary intended users of the findings. Too many times, evaluators go through painstaking care, even agony, in the process of working with primary stakeholders to clearly conceptualize and focus evaluation questions before data collection begins. But then, once the data are collected and analysis begins, they never look back over their notes to review and renew their clarity on the central issues in the evaluation. It is not enough to count on remembering what the evaluation questions were. The early negotiations around the purpose of an evaluation usually involve important nuances. To reestablish those nuances for the purpose of helping focus the analysis, it is important to review the notes on decisions that were made during the conceptual part of the evaluation. (This assumes, of course, that the evaluator has treated the conceptual phase of the evaluation as a field experience and has kept detailed notes about the negotiations that went on and the decisions that were made.)

In addition, it may be worth reopening discussions with intended evaluation users to make sure that the original focus of the evaluation remains relevant. This accomplishes two things. First, it allows the evaluator to make sure that the analysis will focus on needed information. Second, it prepares evaluation users for the results. At the point of beginning formal analysis, the evaluator will have a much better perspective on what kinds of questions can be answered with the data that have been collected. It pays to check out which questions should take priority in the final report and to suggest new possibilities that may have emerged during fieldwork.

Summative evaluations will be judged by the extent to which they contribute to making decisions about a program or intervention, usually decisions about overall effectiveness, continuation, expansion, and/or replication in other sites. A full report presenting data, interpretations, and recommendations is required. In contrast, formative evaluations, conducted for program improvement, may not even generate a written report. Findings may be reported primarily orally. Summary observations may be listed in outline form, or an executive summary may be written, but the timelines for formative feedback and the high costs of formal report writing may make a full, written report impractical. Staff and funders often want the insights of an experienced outsider who can interview program participants effectively, observe what goes on in the program, and provide helpful feedback. The methods are qualitative, the purpose is practical, and the analysis is done throughout fieldwork; no written report is expected beyond a final outline of observations and implications. Academic theory takes second place to understanding the program’s theory of action as actually practiced and implemented. In addition, formative feedback to program staff may be ongoing rather than simply at the end of the study. However, in some situations, funders may request a carefully documented, fully developed, and formally written formative report. The nature of formative reporting, then, is dictated by user needs rather than scholarly norms. For qualitative evaluators, a primary purpose of inquiry, analysis, and interaction around findings is to “foster learning”; a qualitative evaluator “serves as an educator, helping program staff and participants understand the evaluation process and ways in which they can use that process for their own learning” (Goodyear et al., in press).

Action research reporting also varies a great deal. In some action research, the process is the product, so no report will be produced for outside consumption. On the other hand, some action research efforts are undertaken to test organizational or community development theory, and therefore, they require fairly scholarly reports and publications. Action research undertaken by a group of people to solve a specific problem may involve the group sharing the analysis process to generate a mutually understood and acceptable solution, with no permanent, written report of findings.

Students writing dissertations will typically be expected to follow very formal and explicit analytical procedures to produce a scholarly monograph with careful attention to methodological rigor. Graduate students will be expected to report in detail on all aspects of methodology, usually in a separate chapter, including a thorough discussion of analytical procedures, problems, and limitations.

The point here is that the process, duration, and procedures of analysis will vary depending on the study’s purpose and audience. Likewise, the reporting format will vary. First and foremost, then, analysis depends on

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 11/39

clarity about purpose (as do all other aspects of the study). Knowing what kind (or kinds) of findings are needed and how those results will be reported constitutes the foundation for analysis.

Design Frames Analysis Purpose drives analysis through purposeful sampling. Design reflects purpose and therefore frames analysis.

You don’t wait until you’ve collected data to figure out your analysis approach. Design decisions (Chapter 5) anticipate what kind of analysis will be done. In particular, the purposeful sampling strategy you’ve followed is based on what kind of results you want to produce. The data you have to analyze are based on your design. What you have sampled determines what you will address in analysis. Exhibit 8.2 shows the connection between purposeful sampling strategy and analysis approach. The purposeful sampling strategies are taken from Exhibit 5.8 in Chapter 5 (pp. 266–272).

Take Guidance and Inspiration From Examples and Exemplars

Since we learn from examples, it pays to carefully select good examples to learn from. —Halcolm

Reexamining classic works in your field can be a source of inspiration. They are classics for a reason. Keep those exemplars nearby to reinvigorate and inspire you when the drudgery of analysis sets in or doubts about what you’re finding emerge. Those who wrote the classics experienced analysis fatigue and doubts as well. They persevered. So will you.

The first chapter presented several examples of important qualitative studies from different fields and disciplines:

• Patterns in women’s ways of knowing (Belenky et al., 1986)

EXHIBIT 8.2 Connecting Design and Analysis: Purposeful Sampling and Purpose-Driven Analysis

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 12/39

NOTE: The first and second columns correspond to purposeful sampling strategies presented in Exhibit 5.8, pp. 266– 272.

The methods section of a qualitative report can use this same connecting framework (Exhibit 8.2), showing how design informs analysis. An excellent example of such explicit connecting design and analysis is Kaczynski, Salmona, and Smith, (2014).

• Eight characteristics of organizational excellence (Peters & Waterman, 1982) • Seven habits of highly effective people (Covey, 2013) • Case studies and cross-case analysis illuminating why battered women kill (Browne, 1987)

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 13/39

• Three primary processes that contribute to the development of an interpersonal relationship (Moustakas, 1995)

• Case examples illustrating the diversity of experiences and outcomes in an adult literacy program (Patton & Stockdill, 1987)

• Teachers’ reactions to an oppressive school accountability system (Perrone & Patton, 1976)

Reviewing these examples of qualitative findings from the first chapter will ground this discussion of analytical processes in samples of the real fruit of qualitative inquiry. And this chapter will add many more examples.

Chapter 3 presented theoretical orientations associated with qualitative inquiry (ethnography, phenomenology, constructivism, etc.). These theoretical frameworks have implications for analysis in that the fundamental premises articulated in a theoretical framework or philosophy are meant to inform how one makes sense of the world. If you have positioned your inquiry within one of those traditions, exemplars will help immerse you in the way in which that tradition guides analysis. Later in this chapter, I’ll examine in more depth two of the major theory-oriented analytical approaches, phenomenology and grounded theory, as examples of how theory informs analysis.

Using Qualitative Analysis Software Computers and software are tools that assist analysis. Software doesn’t really analyze qualitative data. Qualitative software programs facilitate data storage, coding, retrieval, comparing, and linking—but human beings do the analysis. That said, one reviewer of this book added the following elaboration of how software analysis facilitates engagement with the data.

It is correct to emphasize that Qualitative Data Analysis Software (QDAS) is just a tool which the researcher as instrument must remain in control of. However, the tool is both a data management tool and a qualitative analysis tool. When working with QDAS the researcher is building relationships [with the data] which are a process that is more than just content analysis. The tool helps the researcher build connections which promote further development of complex insights. This ongoing process of meaning construction is analysis. It should be also noted that QDAS can be used with any number of theoretical approaches.

Software has eased significantly the old drudgery of manually locating a particular coded paragraph. Analysis programs speed up the processes of searching for certain words, phases, and themes; labeling interview passages and field notes for easy retrieval and comparative analysis; locating coded themes; grouping data together in categories; and comparing passages in transcripts or incidents from field notes. But the qualitative analyst doing content analysis must still decide what things go together to form a pattern, what constitutes a theme, what to name it, and what meanings to extract from case studies. The human being, not the software, must decide how to frame a case study, how much and what to include, and how to tell the story. Still, software can play a useful role in managing the volume of qualitative data to facilitate analysis, just as quantitative software does.

Quantitative programs revolutionized that research by making it possible to crunch numbers, more accurately, more quickly, and in more ways. . . . Much of the tedious, boring, mistake-prone data manipulation has been removed. This makes it possible to spend more time investigating the meaning of their data.

In a similar way, QDA [qualitative data analysis] programs improve our work by removing drudgery in managing qualitative data. Copying, highlighting, cross-referencing, cutting and pasting transcripts and field notes, covering floors with index cards, making multiple copies, sorting and resorting card piles, and finding misplaced cards have never been the highlights of qualitative research. It makes at least as much sense for us to use qualitative programs for tedious tasks as it does for those people down the hall to stop hand-calculating gammas. (Durkin, 1997, p. 93)

The analysis of qualitative data involves creativity, intellectual discipline, analytical rigor, and a great deal of hard work. Computer programs can facilitate the work of analysis, but they can’t provide the creativity and

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 14/39

intelligence that make each qualitative analysis unique. Moreover, since new software is being constantly developed and upgraded, this book can do no more than provide some general guidance. Most of this chapter will focus on the human thinking processes involved in analysis rather than the mechanical data management challenges that computers help solve. Reviews of computer software in relation to various theoretical and practical issues in qualitative analysis can help you decide what software fits your needs.

What began as distinct software approaches have become more standardized as the various software packages have converged to offer similar functions, though sometimes with different names for the same functions. They all facilitate marking text, building codebooks, indexing, categorizing, creating memos, and displaying multiple text entries side by side. Import and export capabilities vary. Some support team work and multiple users more than others. Graphics and matrix capabilities vary, but they are becoming increasingly sophisticated. All take time to learn to use effectively. The greater the volume of data to be analyzed, the more helpful these software programs are. Moreover, knowing which software program you will use before data collection will help you collect and enter data in the way that works best for that particular program.

SIDEBAR

QUALITATIVE ANALYSIS TECHNOLOGY REVOLUTION: PAST AND FUTURE

In the early 1980s, as qualitative researchers began to grapple with the promise and challenges of computers, a handful of innovative researchers brought forth the first generation of what would come to be known as CAQDAS (Computer Assisted Qualitative Data Analysis Software), or QDAS (Qualitative Data Analysis Software). . . . These stand-alone software packages were developed, initially, to bring the power of computing to the often labor-intensive work of qualitative research. While limited in scope at the beginning to text retrieval tasks, for instance, these tools quickly expanded to become comprehensive all-in-one packages. . . .

Close to 30 years later, QDAS packages are comprehensive, feature-laden tools of immense value to many in the qualitative research world. However, with the advent of the Internet and the emergence of web-based tools known as Web 2.0, QDAS is now challenged on many fronts as researchers seek out easier-to-learn, more widely available and less expensive, increasingly multimodal, visually attractive, and more socially connected technologies. . . .

Truly, qualitative research and technology is in the midst of a revolution.

QDAS 2.0 offers spectacular possibilities to qualitative researchers. . . . What is emerging: Web 2.0 tools with various capacities. . . . As we move more deeply into the digital age, their use, which was once a private choice, will become a necessity. (Davidson & di Gregorio, 2011, pp. 627, 639)

Qualitative discussion groups on the Internet regularly discuss, rate, compare, and debate the strengths and weaknesses of different software programs. While preferences vary, these discussions usually end with the consensus that any of the major programs will satisfy the needs of most qualitative researchers. Increasingly, distinctions depend on “feel,” “style,” and “ease of use”—matters of individual taste—more than differences in function. Still, differences exist, and new developments can be expected to solve existing limitations. Exhibit 8.3 lists resources for comparing and using qualitative software programs.

In considering whether to use software to assist in analysis, keep in mind that this is partly a matter of individual style, comfort with computers, amount of data to be analyzed, and personal preference. Computer analysis is not necessary and can interfere with the analytic process for those who aren’t comfortable spending long hours in front of a screen. Some self-described “concrete” types like to get a physical feel for the data, which isn’t possible with a computer. Participants on a qualitative listserv posted these responses to a thread on software analysis:

• The best advice I ever received about coding was to read the data I collected over and over and over. The more I interacted with the data, the more patterns and categories began to “jump out” at me. I never even

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 15/39

bothered to use the software program I installed on the computer because I found it much easier to code it by hand.

• I found that hand coding was easier and more productive than using a computer program. For me, actually seeing the data in concrete form was vital in recognizing emerging themes. I actually printed multiple copies of data and cut it into individual “chunks,” color coding as I went along, and actually physically manipulating the data by grouping chunks by apparent themes, filing in colored folders, and so on. This technique was especially useful when data seemed to fit more than one theme and facilitated merging my initial and later impressions as themes solidified. Messy, but vital for us concrete people.

So, though software analysis has become common and many swear by it because it can offer leaps in productivity for those adept at it, using software is not a requisite for qualitative analysis. Whether you do or do not use software, the real analytical work takes place in your head.

Being Reflective and Reflexive

Distinguishing signal from noise requires both scientific knowledge and self-knowledge. —Nate Silver (2012, p. 453)

The strategies, guidelines, and ideas for analysis offered in this book are meant to be suggestive and facilitating rather than confining or exhaustive. In actually doing analysis, you will have to adapt what is presented here to fit your specific situation and study. However analysis is done, analysts have an obligation to monitor and report their own analytical procedures and processes as fully and truthfully as possible. This means that qualitative analysis is a new stage of fieldwork in which analysts must observe their own processes even as they are doing the analysis. The final obligation of analysis is to analyze and report on the analytical process as part of the report of actual findings. The extent of such reporting will depend on the purpose of the study. Module 73, later in this chapter, will discuss reflexivity and voice in depth.

EXHIBIT 8.3 Examples of Resources for Computer-Assisted Qualitative Data Analysis Software Decisions, Training, and Technical Assistance

Reviews of qualitative analysis software and web-based programs in relation to significant theoretical and practical issues in qualitative analysis can help you decide what software, if any, fits your needs. Here are some examples of resources that provide information and training.

• The Computer-Assisted Qualitative Data Analysis Software (CAQDAS): The software provides practical support, training, and information in the use of a range of software programs designed to assist qualitative data analysis; platforms for debate concerning the methodological and epistemological issues arising from the use of such software packages; and research into methodological applications of CAQDAS (CAQDAS Networking Project, 2014).

• International Institute for Qualitative Methodology, University of Alberta, Canada: The institute facilitates the development of qualitative research methods across a wide variety of academic disciplines, offering training and networking opportunities through annual conferences and online workshops on qualitative software analysis programs (http://www.iiqm.ualberta.ca/AboutUs.aspx).

• Mobile and Cloud Qualitative Research Apps, The Qualitative Report: http://www.nova.edu/ssss/QR/apps.html

• Qualitative Research, Software & Support Services, University of Massachusetts, Amherst:

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 16/39

http://www.umass.edu/qdap/ • Coding Analysis Toolkit (CAT), a free service of the Qualitative Data Analysis Program (QDAP)

hosted by the University Center for Social and Urban Research at the University of Pittsburgh: http://cat.ucsur.pitt.edu/

• CAQDAS list of options and links: (a) Open source and free, (b) proprietary, and (c) web based (http://en.wikipedia.org/wiki/Computer- assisted_qualitative_data_analysis_software)

• Learning qualitative data analysis on the web: Comparative reviews of software (http://onlineqda.hud.ac.uk/Intro_CAQDAS/reviews-of-sw.php)

• Make inquiries to the qualitative research listserv, QUALRS-L: [email protected]/?

SIDEBAR

OBSERVATIONS AND ADVICE FROM A QUALITATIVE SOFTWARE CONSULTANT

Asher E. Beckwitt has built a consulting business advising graduate students and novice researchers on how to engage in qualitative research using software for analysis (www.qualitativeresearch.org). I asked her to share her experiences as a consultant.

Question: What are the most common challenges you encounter in using qualitative methods with clients and in advising graduate students on their qualitative dissertations?

Answer: The most common challenges are as follows:

1. Students do not understand the differences between qualitative and quantitative methods. 2. They do not understand that there are different types of qualitative analysis (e.g., grounded theory,

phenomenology, narrative analysis, etc.). 3. They do not understand that there are different methodologists’ approaches within these areas (e.g.,

Glasser and Strauss approach grounded theory differently than Strauss and Corbin). 4. They do not understand how to code and analyze their data according to their chosen approach.

Question: You work a lot with qualitative software. What do you tell clients and students that qualitative software does well—and doesn’t do?

Answer: Software is an excellent tool for organizing and coding information. It allows you to store all of the collected data and the codes in one place, as well as code multiple sources (papers/interviews/focus group transcripts/field notes). The organizational capacity of software permits you to code in more detail (e.g., you may have hundreds of codes).

Question: What are the most common misunderstandings about qualitative software? Answer: Most people assume software codes the data for you. This is incorrect, because the

software only stores the information. The researcher is still responsible for coding the text and making decisions about what (and how) to code the data. People also assume software is a method. Software is not a method, it is a software program. Clients and students also erroneously assume using software will make their

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 17/39

project more rigorous or valid. Software simply stores the information inputted by the researcher and provides a more efficient way to retrieve and query that information.

Question: What are the greatest advantages of qualitative software? Answer: It stores the information in one place and allows you to code in more detail. Question: Advice about using software? Cautions? Like with any software, it is important to

save your work often. Your wisdom about qualitative software? Answer: Most of the people I encounter think the software will do the work for them. As

mentioned, software does not code the data for you. The researchers must understand the type of qualitative method and methodology they are using and understand how to implement and translate this approach to code their data in software.

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 18/39

MODULE

66 Thick Description and Case Studies: The Bedrock of Qualitative Analysis

Bedrock is a solid, firm, strong, and stable foundation.

We need a bedrock of story and legend in order to live our lives coherently. —Alan Moore

English writer and graphic comic artist

Thick, rich description provides the foundation for qualitative analysis and reporting. Good description takes the reader into the setting being described. In his classic Street Corner Society, William Foote Whyte (1943) took us to the “slum” neighborhood where he did his fieldwork and introduced us to the characters there, as did Elliot Liebow in Tally’s Corner (1967), a description of the lives of unemployed black men in Washington, D.C., during the 1960s. In Constance Curry’s (1995) oral history of school integration in Drew, Mississippi, in the 1960s, she tells the story of an African American mother Mae Bertha Carter and her seven children as they faced day-to-day and night-to-night threats and terror from resistant, angry whites. Through in-depth case study descriptions, Angela Browne (1987) helps us experience and understand the isolation and fear of a battered woman whose life is controlled by a rage-filled, violent man. Through detailed description and rich quotations, Alan Peshkin (1986) showed readers The Total World of a Fundamentalist Christian School, as Erving Goffman (1961) had done earlier for other “total institutions,” closed worlds like prisons, army camps, boarding schools, nursing homes, and mental hospitals. Howard Becker (1953, 1985) described how one learns to become a marijuana user in such detail that you almost get the scent of the smoke from his writing.

SIDEBAR

ANALYZING AND REPORTING HOW PROGRAM DESCRIPTIONS VARY BY PERSPECTIVE

In describing and evaluating different models of youth service programs, Roholt, Hildreth, and Baizerman (2009) began by capturing and reporting different perspectives: (a) official programmatic descriptions, (b) youth programmatic descriptions, and (c) adult descriptions of the programs.

Evaluation is necessary for youth civic engagement (YCE) programs if they are to receive funding and if they want to improve their work. . . . We sought to understand the program from the points of view and experiences of the multiple participants. . . . We wanted to know what they did, how they made sense of it, and what consequences this had for them and others. We did this by asking young people, teachers, youth workers, volunteers, coordinators, principals, parents, and other non-involved young people to teach us as much as possible about the program and about their participation experiences.

Five questions guided the evaluation:

1. What does this project say it is about? 2. How is it organized and carried out? 3. What are young people doing in the program? 4. What meaning do they give to their work?

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 19/39

5. What consequences does this work have for them, others, the larger community?

What was different about our study was that we worked as if we did not understand what anyone was telling us. For example, when a young person said the word “citizenship,” we assumed that we did not understand what he or she meant. This strategy opened the door to a deeper interrogation of the insider’s experience and brought us to the many ways these projects are experienced and understood by different types of participants—young people, adult coaches, school principals, community leaders, teachers, group leaders, and by different individuals. This gave us a look at what was meaningful to them, as well as to what was done in the program and why. (Roholt et al., 2009, pp. 72, 73, 75)

The detailed descriptions were used to compare different program models and the different perspectives on those models of people in diverse roles and in varying relationships with the programs. This comparative analysis was possible because the descriptions were “thick” and rich. The bedrock of the analysis was description from different perspectives.

These classic qualitative studies share the capacity to open up a world to the reader through rich, detailed, and concrete descriptions of people and places—“thick description” (Denzin, 1989c; Geertz, 1973)—in such a way that we can understand the phenomenon studied and draw our own interpretations about meanings and significance.

Description forms the bedrock of all qualitative reporting, whether for scholarly inquiry, as in the examples above, or for program evaluation. For evaluation studies, basic descriptive questions include the following: How do people get into the program? What is the program setting like? What are the primary activities of the program? What happens to people in the program? What are the effects of the program on participants? Thick evaluation descriptions take those who need to use the evaluation findings into the experience and outcomes of the program.

Description Before Interpretation

A basic tenet of research is careful separation of description from interpretation. Interpretation involves explaining the findings, answering “why” questions, attaching significance to particular results, and putting patterns into an analytic framework. It is tempting to rush into the creative work of interpreting the data before doing the detailed, hard work of putting together coherent answers to major descriptive questions. But description comes first.

Alternative Ways of Organizing and Reporting Descriptions

Several options exist for organizing and reporting descriptive findings. Exhibit 8.4 presents various options depending on whether the primary organizing motif centers on telling the story of what occurred, presenting case studies, or illuminating an analytical framework.

These are not mutually exclusive or exhaustive ways of organizing and reporting qualitative data. Different parts of a report may use different reporting approaches. The point is that one must have some initial framework for organizing and managing the voluminous data collected during fieldwork.

Where variations in the experiences of individuals are the primary focus of the study, it is appropriate to begin by writing a case study using all the data for each person. Only then are cross-case analysis and comparative analysis done. For example, if one has studied 10 juvenile delinquents, the analysis would begin by doing a case description of each juvenile before doing cross-case analysis. On the other hand, if the focus is on a criminal justice program serving juveniles, the analysis might begin with description of variations in answers to common questions, for example, what were the patterns of major program experiences, what did they like, what did they dislike, how did they think they had changed, and so forth.

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 20/39

Likewise, in analyzing interviews, the analyst has the option of beginning with case analysis or cross-case analysis. Beginning with case analysis means writing a case study for each person interviewed or each unit studied (e.g., each critical event, each group, or each program location). Beginning with cross-case analysis means grouping together answers from different people to common questions or analyzing different perspectives on central issues. If a standardized open-ended interview has been used, it is fairly easy to do cross-case or cross-interview analysis for each question in the interview. With an interview guide approach, answers from different people can be grouped by topics from the guide, but the relevant data won’t be found in the same place in each interview. An interview guide, if it has been carefully conceived, actually constitutes a descriptive analytical framework for analysis.

A qualitative study will often include both kinds of analysis—individual cases and cross-case analyses— but one has to begin somewhere. Trying to do both individual case studies and cross-case analysis at the same time will likely lead to confusion.

Case Studies

Case study is not a methodological choice but a choice of what is to be studied. . . . We could study it analytically or holistically, entirely by repeated measures or hermeneutically, organically or culturally, and by mixed methods—but we concentrate, at least for the time being, on the case.

—Robert E. Stake “Case Studies” (2000, p. 435)

Case analysis involves organizing the data by specific cases for in-depth study and comparison. Well- constructed case studies are holistic and context sensitive, two of the primary strategic themes of qualitative inquiry discussed in Chapter 2. Cases can be individuals, groups, neighborhoods, programs, organizations, cultures, regions, or nation-states. “In an ethnographic case study, there is exactly one unit of analysis—the community or village or tribe” (Bernard, 1994, pp. 35–36). Cases can also be critical incidents, stages in the life of a person or of a program, or anything that can be defined as a “specific, unique, bounded system” (Stake, 2000, p. 436). Cases are units of analysis. What constitutes a case, or unit of analysis, is usually determined during the design stage and becomes the basis for purposeful sampling in qualitative inquiry (see Chapter 5 for a discussion of case study designs and purposeful sampling). Sometimes, however, new units of analysis, or cases, emerge during fieldwork or from the analysis after data collection. For example, one might have sampled schools as the unit of analysis, expecting to do case studies of three schools, and then, reviewing the fieldwork, one might decide that classrooms are a more meaningful unit of analysis and shift to case studies of classrooms instead of schools, or add case studies of particular teachers or students. Contrariwise, one could begin by sampling classrooms and end up doing case studies on schools. This illustrates the critical importance of thinking carefully about the question “What is a case?” (Ragin & Becker, 1992).

EXHIBIT 8.4 Options for Organizing and Reporting Qualitative Data

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 21/39

The case study approach to qualitative analysis constitutes a specific way of collecting, organizing, and analyzing data; in that sense, it represents an analysis process. The purpose is to gather comprehensive, systematic, and in-depth information about each case of interest. The analysis process results in a product: a case study. Thus, the term case study can refer to either the process of analysis or the product of analysis, or to both.

Analyzing patterns and identifying themes across multiple case studies has become a significant way of conducting qualitative analysis (Stake, 2006). Case studies may be layered or nested. For example, in evaluation, a single program may be a case study. However, within that single program case (n = 1), one may do case studies of several participants. In such an approach, the analysis would begin with the individual case studies; then, the cross-case pattern analysis of the individual cases might be part of the data for the program case study. Likewise, if a national or state program consists of several project sites, the analysis may consist of

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 22/39

three layers of case studies: (1) individual participant case studies at project sites combined to make up project site case studies, (2) project site case studies combined to make up state program case studies, and (3) state programs combined to make up a national program case study. Exhibit 8.5 shows this layered case study approach.

This kind of layering recognizes that you can always build larger case units out of smaller ones—that is, you can always combine studies of individuals into studies of a program—but if you only have program-level data, you can’t disaggregate it to construct individual cases.

Case Study Rule

Remember this rule: No matter what you are studying, always collect data on the lowest level unit of analysis possible.

Collect data about individuals, for example, rather than about households. If you are interested in issues of production and consumption (things that make sense at the household level), you can always package your data about individuals into data about households during analysis. . . . You can always aggregate data collected on individuals, but you can never disaggregate data collected on groups. (Bernard, 1994, p. 37)

Though a scholarly or evaluation project may consist of several cases and include cross-case comparisons, the analyst’s first and foremost responsibility consists of doing justice to each individual case. All else depends on that.

Ultimately, we may be interested in a general phenomenon or a population of cases more than in the individual case. And we cannot understand this case without knowing about other cases. But while we are studying it, our meager resources are concentrated on trying to understand its complexities. For the while, we probably will not study comparison cases. We may simultaneously carry on more than one case study, but each case study is a concentrated inquiry into a single case. (Stake, 2000, p. 436)

Case data consist of all the information one has about each case: (a) interview data, (b) observations, (c) the documentary data (e.g., program records or files, newspaper clippings), (d) impressions and statements of others about the case, and (e) contextual information—in effect, all the information one has accumulated about each particular case goes into that case study. These diverse sources make up the raw data for case analysis and can amount to a large accumulation of material. For individual people, case data can include (a) interviews with the person and those who know her or him, (b) clinical records and background and statistical information about the person, (c) a life history profile, (d) things the person has produced (diaries, photos, writings, paintings, etc.), and (e) personality or other test results (yes, quantitative data can be part of a qualitative case study). At the program level, case data can include (a) program documents, (b) statistical profiles, (c) program reports and proposals, (d) interviews with program participants and staff, (e) observations of the program, and (f) program histories.

EXHIBIT 8.5 Case Study: Layers of Possible Analysis

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 23/39

From Data to Case Study Once the raw case data have been accumulated, the researcher may write a case record. The case record pulls together and organizes the voluminous case data into a comprehensive, primary resource package. The case record includes all the major information that will be used in doing the final case analysis and writing the case study. Information is edited, redundancies are sorted out, parts are fitted together, and the case record is organized for ready access either chronologically or topically. The case record must be complete but manageable; it should include all the information needed for subsequent analysis, but it is organized at a level beyond that of the raw case data.

A case record should make no concessions to the reader in terms of interest or communication. It is a condensation of the case data aspiring to the condition that no interpreter requires to appeal behind it to the raw data to sustain an interpretation. Of course, this criterion cannot be fully met: some case records will be better than others. The case

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 24/39

record of a school attempts a portrayal through the organization of data alone, and a portrayal without theoretical aspirations. (Stenhouse, 1977, p. 19)

The case record is used to construct a case study appropriate for sharing with an intended audience, for example, scholars, policymakers, program decision makers, or practitioners. The tone, length, form, structure, and format of the final case presentation depend on audience and study purpose. The final case study is what will be communicated in a publication or report. The full report may include several case studies that are then compared and contrasted, but the basic unit of analysis of such a comparative study remains the distinct cases, and the credibility of the overall findings will depend on the quality of the individual case studies. Exhibit 8.6 shows this sequence of moving from raw case data to the written case study. The second step—converting the raw data to a case record before writing the actual case study—is optional. A case record is only constructed when a great deal of unedited raw data from interviews, observations, and documents must be edited and organized before writing the final case study. In many studies, the analyst will work directly and selectively from raw data to write the final case study.

The case study should take the reader into the case situation and experience—a person’s life, a group’s life, or a program’s life. Each case study in a report stands alone, allowing the reader to understand the case as a unique, holistic entity. At a later point in analysis, it is possible to compare and contrast cases, but initially, each case must be represented and understood as an idiosyncratic manifestation of the phenomenon of interest. A case study should be sufficiently detailed and comprehensive to illuminate the focus of inquiry without becoming boring and laden with trivia. A skillfully crafted case feels like a fine weaving. And that, of course, is the trick. How to do the weaving? How to tell the story? How to decide what stays in the final case presentation and what gets deleted along the way. Elmore Leonard (2001), the author of Glitz and other popular detective thrillers, was once asked how he managed to keep the action in his books moving so quickly. He said, “I leave out the parts that people skip” (p. 7). Not bad advice for writing an engaging case study.

EXHIBIT 8.6 The Process of Constructing Case Studies

Step 1. Assemble the raw case data

These data consist of all the information collected about the person, program, organization, or setting for which a case study is to be written.

Step 2. (optional) Construct a case record

This is a condensation of the raw case data, organized, classified, and edited into a manageable and accessible file.

Step 3. Write a final case study narrative

The case study is a readable, descriptive picture of or story about a person, program, organization, or other unit of analysis, making accessible to the reader all the information necessary to understand the case in all its uniqueness. The case story can be told chronologically or presented thematically (sometimes both).

The case study offers a holistic portrayal, presented with any context necessary for understanding the case.

In doing biographical or life history case studies, Denzin (1989a) has found particular value in identifying what he calls “epiphanies”—“existentially problematic moments in the lives of individuals” (p. 129).

SIDEBAR

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 25/39

HOLISTIC CASE STUDIES

Neuroscientists have long used case studies of victims of traumatic brain injuries to understand how the brain works.

Depending on what part of the brain suffered, strange things might happen. Parents couldn’t recognize their children. Normal people became pathological liars. Some people lost the ability to speak—but could sing just fine. These incidents have become classic case studies, fodder for innumerable textbooks and bull sessions around the lab. The names of these patients—H. M. Tan, Phineas Gage—are deeply woven into the lore of neuroscience. (Kean, 2014, p. SR8)

Science journalist Sam Kean (2014) has reflected on such outlier case studies and concluded that “in the quest for scientific understanding, we end up magnifying patients’ deficits until deficits are all we see. The actual person fades away” (p. SR8). He has concluded that more holistic case studies are needed and are even critical to a fuller understanding.

When we read the full stories of people’s lives . . . , we have to put ourselves into the minds of the characters, even if those minds are damaged. Only then can we see that they want the same things, and endure the same disappointments, as the rest of us. They feel the same joys, and suffer the same bewilderment that life got away from them. Like an optical illusion, we can flip our focus. Tales about bizarre deficits become tales of resiliency and courage. (p. SR8)

It is possible to identify four major structures, or types of existentially problematic moments, or epiphanies, in the lives of individuals. First, there are those moments that are major and touch every fabric of a person’s life. Their effects are immediate and long term. Second, there are those epiphanies that represent eruptions, or reactions, to events that have been going on for a long period of time. Third are those events that are minor yet symbolically representative of major problematic moments in a relationship. Fourth, and finally, are those episodes whose effects are immediate, but their meanings are only given later, in retrospection, and in the reliving of the event. I give the following names to these four structures of problematic experience: (1) the major epiphany, (2) the cumulative epiphany, (3) the illuminative, minor epiphany, and (4) the relived epiphany. (Of course, any epiphany can be relived and given new retrospective meaning.) These four types may, of course, build upon one another. A given event may, at different phases in a person’s or relationship’s life, be first, major, then minor, and then later relived. A cumulative epiphany will, of course, erupt into a major event in a person’s life. (p.129)

Programs, organizations, and communities have parallel types of epiphanies, though they’re usually called critical incidents, crises, transitions, or organizational lessons learned. For a classic example of an organizational development case study in the business school tradition, see the analysis of the Nut Island sewage treatment plant in Quincy, Massachusetts—the complex story of how an outstanding team, highly competent, deeply committed to excellence, focused on the organizational mission, and working hard still ended up in a “catastrophic failure” (Levy, 2001).

Studying such examples is one of the best ways to learn how to write case studies. The section titled “Thick Description,” earlier in this chapter, cited a number of case studies that have become classics in the genre. Chapter 1 presented case vignettes of individuals in an adult literacy program. An example of a full individual case study is presented as Exhibit 8.33, at the end of this chapter (pp. 638–642). Originally prepared for an evaluation report that included several participant case studies, it tells the story of one person’s experiences in a career education program. This case represents an exemplar of how multiple sources of information can be brought together to offer a comprehensive picture of a person’s experience, in this instance, a student’s changing involvement in the program and changing attitudes and behaviors over time. The case data for each student in the evaluation study included the following:

1. Observations of selected students at employer sites three times during the year 2. Interviews three times per year with the students’ employer-instructors at the time of observation

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 26/39

3. Parent interviews once a year 4. In-depth student interviews four times a year 5. Informal discussions with program staff 6. A review of student projects and other documents 7. Twenty-three records from the files of each student (including employer evaluations of students, student

products, test scores, and staff progress evaluations of students)

Initial interview guide questions provided a framework for analyzing and reviewing each source. Information from all of these sources was integrated to produce a highly readable narrative that could be used by decision makers and funders to better understand what it was like to be in the program (Owens, Haenn, & Fehrenbacher, 1976). The evaluation staff of the Northwest Regional Educational Laboratory went to great pains to carefully validate the information in the case studies. Different sources of information were used to cross-validate the findings, patterns, and conclusions. Two evaluators reviewed the material in each case study to independently make judgments and interpretations about its content and meaning. In addition, an external evaluator reviewed the raw data to check for biases or unwarranted conclusions. Students were asked to read their own case studies and comment on the accuracy of fact and interpretation in the study. Finally, to guarantee the readability of the case studies, a newspaper journalist was employed to help organize and edit the final versions. Such a rigorous case study approach increases the confidence of readers that the cases are accurate and comprehensive. Both in its content and in the process by which it was constructed, the Northwest Lab case study presented at the end of this chapter (Exhibit 8.33) exemplifies how an individual case study can be prepared and presented.

How one compares and contrasts cases will depend on the purpose of the study and how the cases were sampled. As discussed in Chapter 5, critical cases, extreme cases, typical cases, and heterogeneous cases serve different purposes. Once case studies have been written, the analytic strategies described in the remainder of this chapter can be used to further analyze, compare, and interpret the cases to generate cross-case themes, patterns, and findings. Exhibit 8.7 summarizes the central points I’ve discussed for constructing case studies.

SIDEBAR

DIVERSE CASE STUDY EXEMPLARS

Case Studies in This Book

• The story of Henietta Lacks. This in-depth case study tells the story of a poor African American tobacco farmer who grew up in the South. In 1951, when she died of cervical cancer, the cells from her tumor were taken for research, without her knowledge or permission, by an oncologist. The cells manifest unique characteristics: They could be cultured, sustained, reproduced, and distributed to other researchers, the first tissue cells discovered with these extraordinary characteristics. How this affected her family and the medical world shows how layers of case studies can be interwoven (Skloot, 2010). (See Chapter 2, Exhibit 2.7, pp. 78–80.)

• Story of Li. This case study presents highlights of a participant case study used to illuminate a Vietnamese woman’s experience in an employment training program; in addition to describing what a job placement meant to her, the case was constructed to illuminate hard to measure outcomes such as “understanding the American workplace culture” and “speaking up for oneself,” learnings that can be critical to long-term job success for an emigrant. (See Chapter 4, Exhibit 4.3, pp. 182–183.)

• Thmaris. A case study of a homeless youth and his journey to a more stable life, this case illuminates the challenges and long-term effects of dealing with childhood trauma and failed relationships. His experience in homeless shelters is central to the case study. (See Chapter 7, Exhibit 7.20, pp. 511– 516.)

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 27/39

• Mike’s story. This case study tells the story of one person’s experiences in a career education program. This case represents an exemplar of how multiple sources of information can be brought together to offer a comprehensive picture of a person’s experience, in this instance, a student’s changing involvement in the program and changing attitudes and behaviors over time. (See Chapter 8, Exhibit 8.33, pp. 638–642.)

Examples of Excellent Published Case Studies

• Education case studies. Brizuela, Stewart, Carrillo, and Berger (2000), Stake, Bresler, and Mabry (1991), Perrone (1985), and Alkin, Daillak, and White (1979)

• Family case studies. Sussman and Gilgun (1996) • International development cases. Wood et al. (2011), Salmen (1987), and Searle (1985) • Government accountability case study. Joyce (2011) • Case studies of effective antipoverty programs. Schorr (1988) • Case studies of research influencing policy in developing countries. (Carden, 2009) • Philanthropy case studies. Evaluation Roundtable (2014) and Sherwood (2005) • Public health cases. White (2014) • Business cases. Collins (2001a, 2009), Collins and Porras (2004), and Collins and Hansen (2011)

EXHIBIT 8.7 Guidelines for Constructing Case Studies

1. Focus first on capturing the uniqueness of each case. The qualitative analyst’s first and foremost responsibility consists of doing justice to each individual case. Don’t jump ahead to formal cross-case analysis until the individual cases are fully constructed.

2. Construct cases for smaller units of analysis first. You can always aggregate data collected on individuals into groups for analysis, but you can never disaggregate data collected only on groups to construct individual case studies.

3. Use multiple sources of data. A case study includes and integrates all the information one has about each case—interview data, observations, and documents.

4. Write the case to tell a core story. Structure the case with a beginning, middle, and end. 5. Make the case coherent for the reader. The case study should take the reader into the case situation

and experience—a person’s life experience, a group’s cohesion, a program’s coherence as a program, or a community’s sense of community.

6. Balance detail with relevance. A case study should be sufficiently detailed and comprehensive to illuminate the focus of inquiry without becoming boring and laden with trivia.

7. Readability and coherence check. Have someone read the case and give you feedback about its coherence and readability, and any gaps or ambiguities that need attention.

8. Accuracy check. For individual case studies, have the person whose story you’ve written review the case for accuracy. For other units of analysis (programs, communities, organizations) have a key informant review the case.

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 28/39

MODULE

67 Qualitative Analysis Approaches: Identifying Patterns andThemes

The ability to use thematic analysis appears to involve a number of underlying abilities, or competencies. One competency can be called pattern recognition. It is the ability to see patterns in seemingly random information.

—Boyatzis (1998, p. 7)

This module will present the kinds of findings that result from qualitative analysis. I’ll examine what is meant by content analysis and distinguish patterns from themes. I’ll contrast inductive and deductive analytical approaches and introduce some specific analytical approaches like grounded theory and analytic induction. I’ll discuss sensitizing concepts as a focus for analysis and differentiate indigenous concepts and typologies from analyst-created concepts and typologies. Exhibit 8.10, at the end of this module (pp. 551–552), will summarize the 10 analytical approaches reviewed in this module.

The next module will go into detail about the actual coding and analytical procedures for making sense of qualitative data. I could have started with those procedural processes for analysis, but I think it’s helpful to understand first what kinds of findings can be generated from qualitative analysis before delving very deeply into the mechanics and operational processes. Thus, this module will provide examples of patterns, themes, indigenous concepts and typologies, and analyst-constructed concepts and typologies—the fruit of qualitative analysis, a metaphor that harks back to Chapter 1. The next module will explain how you harvest qualitative fruit once you know more about the variety of fruit that can be harvested.

Content Analysis

No consensus exists about the terminology to apply in differentiating varieties and processes of qualitative analysis. Content analysis sometimes refers to searching text for and counting recurring words or themes. For example, a speech by a politician might be analyzed to see what phrases or concepts predominate, or speeches of two politicians might be compared to see how many times and in what contexts they used a phrase like “global economy” or “family values.” More generally, content analysis usually refers to analyzing text (interview transcripts, diaries, or documents) rather than observation-based field notes. Even more generally, content analysis refers to any qualitative data reduction and sense-making effort that takes a volume of qualitative material and attempts to identify core consistencies and meanings. Case studies, for example, can be content analyzed.

Patterns Are the Basis for Themes

The core meanings found through content analysis are patterns and themes. The processes of searching for patterns and themes may be distinguished as pattern analysis and theme analysis, respectively. I’m asked frequently about the difference between a pattern and a theme. The term pattern refers to a descriptive finding, for example, “Almost all participants reported feeling fear when they rappelled down the cliff,” while a theme takes a more categorical or topical form, interpreting the meaning of the pattern: FEAR. Putting these terms together, a report on a wilderness education study might state,

The content analysis revealed a pattern of participants reporting being afraid when rappelling down cliffs and running river rapids; many also initially experienced the group process of sharing personal feelings as evoking some fear. Those patterns make dealing with fear a major theme of the wilderness education program experience.

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 29/39

Inductive and Deductive Qualitative Analyses Qualitative deductive analysis: Determining the extent to which qualitative data in a particular study support existing general conceptualizations, explanations, results, and/or theories

Qualitative inductive analysis: Generating new concepts, explanations, results, and/or theories from the specific data of a qualitative study

Francis Bacon is known for his emphasis on induction, the use of direct observation to confirm ideas and the linking together of observed facts to form theories or explanations of how natural phenomenon work. Bacon correctly never told us how to get ideas or how to accomplish the linkage of empirical facts. Those activities remain essentially humanistic—you think hard. (Bernard, 2000, p. 12)

SIDEBAR

HUMAN PATTERN RECOGNITION

Pattern detection is an evolutionary capacity developed in and passed on from our Stone Age ancestors.

Human beings do not have very many natural defenses. We are not all that fast, and we are not all that strong. We do not have claws or fangs or body armor. We cannot spit venom. We cannot camouflage ourselves. And we cannot fly. Instead, we survive by means of our wits. Our minds are quick. We are wired to detect patterns and respond to opportunities and threats without much hesitation. (Silver, 2012, p. 12)

Both the capacity and the drive to find patterns is much more developed in humans than in other animals, explains Tomaso Poggio, a neuroscientist who studies how human brains process information and make sense of the world. The problem is that these evolutionary instincts sometimes lead us to see patterns when there are none. People do that all the time, Poggio has found—“finding patterns in random noise.” Thus, unless we work actively to become aware of our biases and avoid overconfidence when identifying patterns, we can fail to accurately distinguish signal (pattern) from noise (random occurrences and relationships) (Silver, 2012, p. 12).

So beware of the allure and dangers of apophenia: identifying meaningful patterns in meaningless randomness.

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 30/39

Bacon (1561–1626) is recognized as one of the founders of scientific thinking, but he also has been awarded “the dubious honor of being the first martyr of empiricism.” Still pondering the universe at the age of 65, he got an idea one day while driving his carriage in the snow in a farming area north of London. It occurred to him that cold might delay the biological process of putrefaction, so he stopped, purchased a hen from a farmer, killed it on the spot, and stuffed it in the snow. His idea worked. The snow did delay the rotting process, but he subsequently contracted bronchitis and died a month later (Bernard, 2000, p. 12). As I noted in Chapter 6, fieldwork can be risky. Engaging in analysis, on the other hand, is seldom life threatening, though you do risk being disputed and sometimes ridiculed by those who arrive at contrary conclusions.

Inductive analysis involves discovering patterns, themes, and categories in one’s data. Findings emerge out of the data, through the analyst’s interactions with the data. In contrast, when engaging in deductive analysis, the data are analyzed according to an existing framework. Qualitative analysis is typically inductive in the early stages, especially when developing a codebook for content analysis or figuring out possible categories, patterns, and themes. This is often called “open coding” (Strauss & Corbin, 1998, p. 223), to emphasize the importance of being open to the data. “Grounded theory” (Glaser & Strauss, 1967) emphasizes becoming immersed in the data—being grounded—so that embedded meanings and relationships can emerge. The French would say of such an immersion process, Je m’enracine (“I root myself”). The analyst becomes implanted in the data. The resulting analysis grows out of that groundedness.

From Inductive to Deductive

Once patterns, themes, and/or categories have been established through inductive analysis, the final, confirmatory stage of qualitative analysis may be deductive in testing and affirming the authenticity and appropriateness of the inductive content analysis, including carefully examining deviate cases or data that don’t fit the categories developed. Generating theoretical propositions or formal hypotheses after inductively identifying categories is considered deductive analysis by grounded theorists Strauss and Corbin (1998): “Anytime that a researcher derives hypotheses from data, because it involves interpretation, we consider that to be a deductive process” (p. 22). Grounded theorizing, then, involves both inductive and deductive processes: “At the heart of theorizing lies the interplay of making inductions (deriving concepts, their properties, and dimensions from data) and deductions (hypothesizing about the relationships between concepts)” (Strauss & Corbin, 1998, p. 22).

SIDEBAR

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 31/39

INDUCTIVE PROGRAM THEORY DEVELOPMENT

Inductive [program theory] development involves observing the program in action and deriving the theories that are implicit in people’s actions when implementing the program. The theory in action may differ from the espoused theory: what people do is different from what they say they do, believe they are doing, or believe they should be doing according to policy or some other principle.

The program in action could be observed at the point of service delivery in the field that is closest to the clients of the program. It could include observation of the program in action, including through participant observation. Interviews can be conducted with staff about how they implement the program and about why they undertake some activities that may appear to be at variance with the program design or omit parts of the program design. Program participants can be interviewed about how they experience the program (or have experienced it), if data are gathered through exit interviews; and how they would like to experience it. . . .

Theory in action could also be identified from looking at how program managers have interpreted the program, as indicated by the types of practices they adopt. However, it is important to confirm that inferences drawn are correct. For example, what they consider to be important about the program and how they interpret the program’s intent might be inferred from their choice of particular performance indicators and how they use them. This inference would need to be confirmed with program managers, since the selection of indicators may have been imposed on management and staff as, for example, part of national nonprogram specific monitoring requirements. Or the indicators may have been selected simply because they were available and easy to measure and report, but not necessarily considered by staff to be meaningful.

—Funnell and Rogers (2011, pp. 111–112)

From Deduction to Induction: Analytic Induction

Analytic induction as a distinct qualitative analysis approach begins with an analyst’s deduced propositions or theory-derived hypotheses and “is a procedure for verifying theories and propositions based on qualitative data” (Taylor & Bogdan, 1984, p. 127). Sometimes, as with analytic induction, qualitative analysis is first deductive or quasi-deductive and then inductive, as when, for example, the analyst begins by examining the data in terms of theory-derived sensitizing concepts or applying a theoretical framework developed by someone else (e.g., testing Piaget’s developmental theory on case studies of children). After or alongside this deductive phase of analysis, the researcher strives to look at the data afresh for undiscovered patterns and emergent understandings (inductive analysis). I’ll discuss both grounded theory and analytic deduction at greater length later in this chapter.

SIDEBAR

DEDUCTIVE ANALYSIS EXAMPLE: AGENCY RESISTANCE TO OUTCOME MEASUREMENT

Identifying factors that support evaluation use and overcoming resistance to evaluation have been two of the central concerns of the evaluation profession for 40 years (Alkin, 1975; Patton, 1978b). A great volume of research, much of it qualitative case studies of use and nonuse, point to the importance of high- quality stakeholder involvement to enhance use (Brandon & Fukunaga, 2014; Patton, 2008, 2012a). Strickhouser and Wright (2014) contributed to this arena of inquiry by interviewing the directors and staff of eight human service nonprofit agencies and their one common funder in a large southeastern metropolitan area. They found that agencies continue to resist, and in some cases sabotage, evaluation reporting requirements. They found that, as shown in previous studies, program evaluators often find it difficult to conceptualize and evaluate outcomes. Tensions around outcome measurement make

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 32/39

communication between agencies and their funders difficult and frustrating on both sides. This is an example of a primarily deductive qualitative analysis because the questions asked and the analysis conducted draw on issues and concepts that are already well established. The qualitative inquiry tests whether already identified factors continue to be manifest in a specific group not previously studied. The findings are confirmatory and illuminating, but they do not generate any new concepts or factors.

Because, as identified and discussed in Chapter 2, inductive analysis is one of the primary characteristics of qualitative inquiry, we’ll focus on strategies for thinking and working inductively. There are two distinct ways of analyzing qualitative data inductively. First, the analyst can identify, define, and elucidate the categories developed and articulated by the people studied to focus analysis. Second, the analyst may also become aware of categories or patterns for which the people studied did not have labels or terms, and the analyst develops terms to describe these inductively generated categories. Each of these approaches is described below.

Inductive Approaches: Indigenous and Analyst-Constructed Patterns and Themes

Indigenous Concepts and Practices

A good place to begin inductive analysis is to inventory and define key phrases, terms, and practices that are special to the people in the setting studied. What are the indigenous categories that the people interviewed have created to make sense of their world? What are the practices they engage in that can only be understood within their worldview? Anthropologists call this emic analysis and distinguish it from etic analysis, which refers to labels imposed by the researcher. (For more on this distinction and its origins, see Chapter 6, which discusses emic and etic perspectives in fieldwork.) “Identifying the categories and terms used by informants themselves is also called in vivo coding” (Bernard 1998, p. 608).

Consider the practice among traditional Dani women of amputating a finger joint when a relative dies. The Dani people live in the lush Baliem Valley of Irian Java, Indonesia’s most remote province, in the western half of New Guinea. The joint is removed to honor and placate ancestral ghosts. Missionaries have fought against the practice as sinful, and the government has banned it as barbaric, but many traditional women still practice it.

Some women in Dani villages have only four stubs and a thumb on each hand. In tribute to her dead mother and brothers, Soroba, 38, has had the tops of six of her fingers amputated. “The first time was the worst,” she said. “The pain was so bad, I thought I would die. But it’s worth it to honor my family.” (Sims, 2001, p. 6)

Analyzing such an indigenous practice begins with understanding it from the perspective of its practitioners, within the indigenous context, in the words of the local people, in their language, within their worldview.

According to this view, cultural behavior should always be studied and categorized in terms of the inside view—the actors’ definition—of human events. That is, the units of conceptualization in anthropological theories should be “discovered” by analyzing the cognitive processes of the people studied, rather than “imposed” from cross-cultural (hence, ethnocentric) classifications of behavior. (Pelto & Pelto, 1978, p. 54)

Anthropologists, working cross-culturally, have long emphasized the importance of preserving and reporting the indigenous categories of the people studied. Franz Boas (1943) was a major influence in this direction: “If it is our serious purpose to understand the thoughts of a people, the whole analysis of experience must be based on their concepts, not ours” (p. 314).

In an intervention program, certain terms may emerge or be created by participants to capture some essence of the program. In the wilderness education program I evaluated, the idea of “detoxification” became a

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 33/39

powerful way for participants to share meaning about what being in the wilderness together meant (Patton, 1999, pp. 49–52). In the Caribbean Extension Project evaluation, the term liming had special meaning from the participants. Not really translatable, it essentially means passing time, hanging out, doing nothing, shooting the breeze—but doing so agreeably, without guilt, stress, or a sense that one ought to be doing something more productive with one’s time. Liming has positive, desirable connotations because of its social group meaning—people just enjoying being together without having to accomplish anything. Given that uniquely Caribbean term, what does it mean when participants describe what happened in a training session or instructional field trip as primarily “liming”? How much “liming” could acceptably be built into training for participant satisfaction and still get something done? How much programmatic liming was acceptable? These became key formative evaluation issues.

In evaluating a leadership training program, we gathered extensive data on what participants and staff meant by the term leadership. Pretraining and posttraining exercises involved having participants write a paragraph on leadership; the writing was part of the program curriculum, not designed for evaluation, but the results provided useful qualitative evaluation data. There were small-group discussions on leadership. The training included lectures and group discussions on leadership, which we observed. We participated in and took notes on informal discussions about leadership. Because the very idea of leadership was central to the program, it was essential to capture variations in what participants meant when they talked about “leadership.” The results showed that the ongoing confusion about what leadership meant was one of the problematic issues in the program. Leadership was an indigenous concept in that staff and participants throughout the training experience used it extensively, but it was also a sensitizing concept since we knew going into the fieldwork that it would be an important notion to study.

Sensitizing Concepts In contrast to purely indigenous concepts, sensitizing concepts refer to categories that the analyst brings to the data. Experienced observers often use sensitizing concepts to orient fieldwork, an approach discussed in Chapter 6 (pp. 357–363). These sensitizing concepts have their origins in social science theory, the research literature, or evaluation issues identified at the beginning of a study. Sensitizing concepts give the analyst “a general sense of reference” and provide “directions along which to look” (Blumer, 1969, p. 148). Using sensitizing concepts involves examining how the concept is manifest and given meaning in a particular setting or among a particular group of people.

Conroy (1987) used the sensitizing concept “victimization” to study police officers. Innocent citizens are frequently thought of as the victims of police brutality or indifference. Conroy turned the idea of victim around and looked at what it would mean to study police officers as victims of the experiences of law enforcement. He found the sensitizing concept of victimization helpful in understanding the isolation, lack of interpersonal affect, cynicism, repressed anger, and sadness observed among police officers. He used the idea of victimization to tie together the following quotes from police officers:

• As a police officer and as an individual I think I have lost the ability to feel and to empathize with people. I had a little girl that was run over by a bus and her mother was there and she had her little book bag. It was really sad at the time but I remember feeling absolutely nothing. It was like a mannequin on the street instead of some little girl. I really wanted to be able to cry about it and I really wanted to have some feelings about it, but I couldn’t. It’s a little frightening for me to be so callous and I have been unable to relax.

• I am paying a price by always being on edge and by being alone. I have become isolated from old friends. We are different. I feel separate from people, different, out of step. It becomes easier to just be with other police officers because they have the same basic understanding of my environment, we speak the same language. The terminology is crude. When I started I didn’t want to get into any words like scumbags and scrotes, but it so aptly describes these people.

• I have become isolated from who I was because I have seen many things I wish I had not seen. It’s frustrating to see things that other people don’t see, won’t see, can’t see. I wish sometimes, I didn’t see the things. I need to be assertive, but don’t like it. I have to put on my police mask to do that. But now it is getting harder and harder to take that mask off. I take my work home with me. I don’t want my work to invade my personal life but I’m

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 34/39

finding I need to be alone more and more. I need time to recharge my batteries. I don’t like to be alone, but must. (p. 52)

Two additional points are worth making about these quotations. First, by presenting the actual data on which the analysis is based, the readers are able to make their own determination of whether the concept “victimization” helps in making sense of the data. By presenting respondents in their own words and reporting the actual data that were the basis of his interpretation, Conroy invites readers to make their own analysis and interpretation. The analyst’s constructs should not dominate the analysis, but rather, they should facilitate the reader’s understanding of the world under study.

Second, these three quotations illustrate the power of qualitative data. The point of analysis is not simply to find a concept or label to neatly tie together the data. What is important is understanding the people studied. Concepts are never a substitute for direct experience with the descriptive data. What people actually say and the descriptions of events observed remain the essence of qualitative inquiry. The analytical process is meant to organize and elucidate telling the story of the data. Indeed, the skilled analyst is able to get out of the way of the data to let the data tell their own story. The analyst uses concepts to help make sense of and present the data, but not to the point of straining or forcing the analysis. The reader can usually tell when the analyst is more interested in proving the applicability and validity of a concept than in letting the data reveal the perspectives of the people interviewed and the intricacies of the world studied.

Analyst-Created Concepts Sensitizing concepts are used during fieldwork to guide the inquiry and subsequent analysis. The analysis puts flesh on the bare bones of a sensitizing concept, deepening its meaning and revealing its implications. Concepts can also emerge during analysis that were not yet imagined or conceptualized during fieldwork. At a conference for fathers of teenagers aimed at illuminating strategies for dealing with the challenges of guiding one’s child through adolescence, the term reverse incest anxiety emerged in our analysis to describe some fathers’ fear of expressing physical affection for teenage daughters lest it be perceived as inappropriate.

SIDEBAR

TEMPLATE ANALYSIS

Template analysis is an approach being used in organizational research to organize and make sense of rich, unstructured qualitative data. The analytical framework provides guidance for defining codes, hierarchical coding, and parallel coding. Template analysis involves identifying conceptual themes, clustering them into broader groupings, and, subsequently, identifying “master themes” and subsidiary constituent themes across cases. In organizational research and program evaluation, template analysis “works particularly well when the aim is to compare the perspectives of different groups of staff within a specific context” (King, 2004, p. 257).

SOURCES: King (2012); King and Horrocks (2012); Waring and Wainwright (2008).

British social scientist Guy Standing (2011) created the term precariat to describe a new class of people in industrialized societies who are in the precarious position of only getting occasional short-term and part-time work and whose quality of life and living standards are made precarious. They have no career path and stability, and they experience multiple forms of economic and social insecurity.

Having suggested how singular concepts can bring focus to inductive analysis, the next level of analysis, constructing typologies, moves us into a somewhat more complex analytical strategy.

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 35/39

Typologies and Continua: Indigenous and Analyst-Constructed Frameworks

There are two kinds of people in the world: those who think there are two kinds of people in the world and those who don’t.

—Humorist Robert Benchley (1889–1945) Law of Distinction

Indigenous Typologies Typologies are classification systems made up of categories that divide some aspect of the world into parts along a continuum. They differ from taxonomies, which completely classify a phenomenon through mutually exclusive and exhaustive categories, like the biological system for classifying species. Typologies, in contrast, are built on ideal types or illustrative end points rather than a complete and discrete set of categories. Well- known and widely used sociological typologies include Redfield’s folk–urban continuum (gemeinschaft– gesellschaft) and Von Wiese’s and Becker’s sacred–secular continuum (for details, see Vidich & Lyman, 2000, p. 52). Sociologists classically distinguish ascribed from achieved characteristics. Psychologists distinguish degrees of mental illness (neuroses to psychoses). Political scientists classify governmental systems along a democratic–authoritarian continuum. Economists distinguish laissez-faire from centrally planned economic systems. Systems analysts distinguish open from closed systems. In all of these cases, however, the distinctions involve matters of degree and interpretation rather than absolute distinctions. All of these examples have emerged from social science theory and represent theory-based typologies constructed by analysts. We’ll examine that approach in greater depth in a moment. First, however, let’s look at identifying indigenous typologies as a form of qualitative analysis.

Illuminating indigenous typologies requires an analysis of the continua and distinctions used by people in a setting to break up the complexity of reality into distinguishable parts. The language of a group of people reveals what is important to them in that they name something to separate and distinguish it from other things with other names. Once these labels have been identified from an analysis of what people have said during fieldwork, the next step is to identify the attributes or characteristics that distinguish one thing from another. In describing this kind of analysis, Charles Frake (1962) used the example of a hamburger. Hamburgers can vary a great deal in how they are cooked (rare to well done) or what is added to them (pickles, mustard, ketchup, lettuce), and they are still called hamburgers. However, when a piece of cheese is added to the meat, it becomes a cheeseburger. The task for the analyst is to discover what it is that separates a “hamburger” from a “cheeseburger”—that is, to discern and report “how people construe their world of experience from the way they talk about it” (Frake, 1962, p. 74).

An analysis example of this kind comes from a formative evaluation aimed at reducing the dropout rate among high school students. In observations and interviews at the targeted high school, it became important to understand the ways in which teachers categorized students. With regard to problems of truancy, absenteeism, tardiness, and skipping class, the teachers had come to label students as either “chronics” or “borderlines.” One teacher described the chronics as “the ones who are out of school all the time, and everything you do to get them in doesn’t work.” Another teacher said, “You can always pick them out, the chronics. They’re usually the same kids.” The borderlines, on the other hand,

SIDEBAR

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 36/39

THE SIGNAL AND THE NOISE: LIVING WITH, LEARNING FROM, AND HONORING THE NOISE

The metaphor of distinguishing signal from noise is a powerful way to talk about pattern detection in qualitative analysis. In reporting findings, qualitative analysts typically focus on and highlight the patterns and themes found, what they mean, and their implications for theory and/or practice. The signal versus noise distinction can appear to connote that what is valuable is the signal. Of course, one person’s noise can be another person’s signal, so the distinction depends on perspective. But however distinguished, once the signal (pattern, theme, or meaning) is detected, the noise fades into the background. Still, much can be learned from dwelling with and understanding the noise. Describing, characterizing, making sense of, portraying, and understanding the noise can be, in and of itself, a qualitative analysis contribution.

• Evaluation professional Nora Murphy, cofounder of the TerraLuna Collaborative, described to me observing dinner meetings of teachers participating in an innovative initiative and just immersing herself in the predinner chatter and other activities going on—small groups forming and disbanding, people moving around and milling around (taking in the head nodding, head shaking, furled brows, and animated hands; reconnecting hugs and handshakes; and hearing the laughter)—literally experiencing the noise of the interactions to get a sense of how these teachers were coming together with each other.

• Seasoned educator Eleanor Coleman, of the Minnesota Humanities Center, told me how she and her team of district leaders went through an exercise of listing all the new initiatives that had been introduced in the school district over the past five years. The walls were soon covered with a list of more than 100 initiatives that had been introduced, demanding their attention and participation, plus ongoing demands from needy students, concerned parents, paper-pushing administrators, union leaders, and elected officials, while they tried to lead their lives, take care of their families, maintain relationships with friends and neighbors, and feed their spiritual needs. Messy lives. Noisy lives. How, through all that noise, would yet another initiative become an innovative and valued signal, catching the attention of and engendering commitment from teachers? Answering that question—indeed, even beginning to answer that question—meant dwelling more deeply with and understanding the noise.

In The Signal and the Noise, Nate Silver (2012) recounts a conversation with an international terrorism expert in which the expert distinguishes the challenge of finding a proverbial needle in a haystack from the even more daunting challenge of finding one particular needle in a large stack of needles. In both cases, the focus is on finding the needle—the thing you’re looking for, the signal, the pattern, the thing that stands out. And to find that one particular needle you’re looking for, you have to take apart the haystack or the stack of needles. But before doing so, imagine first inquiring into the stack, whether of hay or needles (How did the stack come to be there? What’s the context within which the stack has been stacked? What are the characteristics of the stack? What can be learned about and from the stack) before destroying it in search of the needle.

A comprehensive, holistic qualitative inquiry will describe, analyze, attend to, and attempt to understand both the signal and the noise. And sometimes, the noise is the signal.

skip a few classes, waiting for a response, and when it comes they shape up. They’re not so different from your typical junior high student, but when they see the chronics getting away with it, they get more brazen in their actions.

Another teacher said, “Borderlines are gone a lot but not constantly like the chronics.”

Not all teachers used precisely the same criteria to distinguish “chronics” from “borderlines,” but all teachers used these labels in talking about students. To understand the program activities directed at reducing high school dropouts and the differential impact of the program on students, it became important to observe

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 37/39

differences in how “borderlines” and “chronics” were treated. Many teachers, for example, refused even to attempt to deal with chronics. They considered it a waste of their time. Students, it turned out, knew what labels were applied to them and how to manipulate these labels to get more or less attention from teachers. Students who wanted to be left alone called themselves “chronics” and reinforced their “chronic image with teachers. Students who wanted to graduate, even if only barely and with minimal school attendance, cultivated an image as ‘borderline.’”

Another example of an indigenous typology emerged in the wilderness education program I evaluated. As I explained earlier, when I used this example to discuss participant observation, one subgroup started calling themselves the “turtles.” They contrasted themselves to the “truckers.” On the surface, these labels were aimed at distinguishing different styles of hiking and backpacking, one slow and one fast. Beneath the surface, however, the terms came to represent different approaches to the wilderness and different styles of experience in relation to the wilderness and the program.

Groups, cultures, organizations, and families develop their own language systems to emphasize distinctions they consider important. Every program gives rise to special vocabulary that staff and participants use to differentiate types of activities, kinds of participants, styles of participation, and variously valued outcomes. These indigenous typologies provide clues to analysts that the phenomena to which the labels refer are important to the people in the setting and that to fully understand the setting it is necessary to understand those terms and their implications.

SIDEBAR

TAO QUALITIES

When Beauty is recognized in the World Ugliness has been learned; When Good is recognized in the World Evil has been learned. In this way: Alive and dead are abstracted from growth; Difficult and easy are abstracted from progress; Far and near are abstracted from position; Strong and weak are abstracted from control, Song and speech are abstracted from harmony; After and before are abstracted from sequence.

Comparative Analysis

A newborn is soft and tender,

A crone, hard and stiff.

Plants and animals, in life, are supple and juicy;

In death, brittle and dry.

So softness and tenderness are attributes of life,

And hardness and stiffness, attributes of death.

—Tao Te Ching of Lao Tzu

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 38/39

Analyst-Constructed Typologies Once indigenous concepts, typologies, and themes have been surfaced, understood, and analyzed, the qualitative analysis may move to a different inductive task to further elucidate findings—constructing nonindigenous typologies based on analyst-generated patterns, themes, and concepts. Such constructions must be done with considerable care to avoid creating things that are not really in the data. The advice of biological theorist John Maynard Smith (2000) is informative in this regard: Seek models of the world that make sense and whose consequences can be worked out, for “to replace a world you do not understand by a model of a world you do not understand is no advance” (p. 46).

Constructing ideal types or alternative paradigms is one simple form of presenting qualitative comparisons. Exhibit 8.8 presents my ideal-typical comparison of “coming-of-age paradigms,” which contrasts tribal initiation themes with contemporary coming-of-age themes (Patton, 1999). A series of patterns are distilled into contrasting themes that create alternative ideal types. The notion of “ideal types” makes it explicit that the analyst has constructed and interpreted something that supersedes purely descriptive analysis.

In creating analyst-constructed typologies through inductive analysis, you take on the task of identifying and making explicit patterns that appear to exist but remain unperceived by the people studied. The danger is that analyst-constructed typologies impose a world of meaning on the participants that better reflects the observer’s world than the world under study. One way of testing analyst-constructed typologies is to present them to the people whose world is being analyzed to find out if the constructions make sense to them.

The best and most stringent test of observer constructions is their recognizability to the participants themselves. When participants themselves say, “yes, that is there, I’d simply never noticed it before,” the observer can be reasonably confident that he has tapped into extant patterns of participation. (Lofland, 1971, p. 34)

Exhibit 8.9, using the problem of classifying people’s ancestry, shows what can happen when indigenous and official constructions conflict, a matter of some consequence to those affected.

A good example of an analyst-generated typology comes from an evaluation of the National Museum of Natural History, Smithsonian Institution, done by Robert L. Wolf and Barbara L. Tymitz (1978). This has become a classic in the museum studies field. They conducted a naturalistic inquiry of viewers’ reactions to an exhibit on “Ice Age Mammals and Emergence of Man.” From their observations, they identified four different kinds of visitors to the exhibit.

EXHIBIT 8.8 Coming-of-Age Paradigms

Ideal-typical comparison of “coming-of-age paradigms,” which contrasts indigenous tribal initiation themes with contemporary, analyst-constructed coming-of-age themes

6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice

https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 39/39

EXHIBIT 8.9 Qualitative Analysis of Ancestry at the U.S. Census

To count different kinds of people—the job of the Census Bureau—you need categories to count them in. The long form of the 2000 census, given to one in six households, asked an open-ended, fill-in-the-blank question about “ancestry.” Analysts then coded the responses into 604 categories, up from 467 in 1980. The government doesn’t ask about religion, so if people respond that they are “Jewish,” they don’t get their ancestry counted. However, those who write in that they are Amish or Mennonite do get counted because those are considered cultural categories.

Ethnic minorities that cross national boundaries, such as French and Spanish Basques, and groups affected by geopolitical change, like Czechs and Slovaks or groups within the former Yugoslavia, are counted in distinct categories. The Census Bureau, following advice from the U.S. State Department, differentiates Taiwanese Americans from Chinese Americans, a matter of political sensitivity.

Can Assyrians and Chaldeans be lumped together? When the Census Bureau announced that they would combine the two in the same “ancestry code,” an Assyrian group sued over the issue but lost the lawsuit. Assyrian Americans trace their roots to a biblical-era empire covering much of what is now Iraq and believe that Chaldeans are a separate religious subgroup. A fieldworker for the Census Bureau did fieldwork on the issue.

“I went into places where there were young people playing games, went into restaurants, and places where older people gathered,” says Ms. McKenney. . . . She paid a visit to Assyrian neighborhoods in Chicago, where a large concentration of Assyrian-Americans lives. At a local community center and later that day at the Assyrian restaurant next door, community leaders presented their case for keeping the ancestry code the same. Over the same period, she visited Detroit to look into the Chaldean matter . . . .

“I found that many of the people, especially the younger people, viewed it as an ethnic group, not a religion,” says Ms. McKenney. She and Mr. Reed (Census Bureau Ancestry research expert) concurred that enough differences existed that the Chaldeans could potentially qualify as a separate ancestry group.

In a conference call between interested parties, a compromise was struck. Assyrians and Chaldeans would remain under a single ancestry code, but the name would no longer be Assyrian, it would be Assyrian/Chaldean/Syriac—Syriac being the name of the Aramaic dialect that Assyrians and Chaldeans speak. “There was a meeting of the minds between all the representatives, and basically it was a unified decision to say that we’re going to go under the same name,” says the Chaldean Federation’s Mr. Yono. (Kulish, 2001, p. 1)