U2D1-68 - Follow instructions below to answer the questions regarding Identifying Qualitative Inquire Themes in Research

CHAPTER

3 Variety of Qualitative Inquiry Frameworks Paradigmatic, Philosophical, and Theoretical Orientations

Personal coat of arms of Sir Isaac Newton: Two shinbones in saltaire argent crossed in silver

—Michon (2013)

I can calculate the motion of heavenly bodies, but not the madness of people.

—Sir Isaac Newton (1642–1726)

Chapter Preview There is no definitive way to categorize the various philosophical and theoretical perspectives that have influenced and that distinguish the types of qualitative inquiry. This chapter distinguishes theoretical perspectives by their foundational questions. We’re going to examine, compare, and contrast 16 different theoretical approaches to quality inquiry:

1. Ethnography 2. Autoethnography 3. Reality-testing, foundationalist epistemologies: positivism, postpositivism, empiricism, and objectivism 4. Grounded theory 5. Realism 6. Phenomenology 7. Heuristic inquiry 8. Social constructionism and constructivism 9. Narrative inquiry 10. Ethnomethodology 11. Symbolic interaction 12. Semiotics 13. Hermeneutics 14. Systems theory 15. Complexity theory 16. Pragmatism and generic qualitative inquiry

We’ll then look at 10 qualitative inquiry themes that cut across these 16 distinct and diverse theoretical orientations. That will set the stage for the next chapter, where we examine concrete and practical applications of qualitative methods. In effect, these two chapters, Chapters 3 and 4, look at theory and practice, first theory (this chapter), then practice (the next chapter), to integrate them: theory informing practice and practice informing theory.

To begin, I’m going to review the historic debate between qualitative and quantitative methods. The debate has waxed and waned over the years, and the precise nature of the debate has morphed from differences of opinion about which methods are the most credible to how best to combine the strengths of each through mixed methods. The intensity of the debate has varied among disciplines and across institutional settings. It cuts across the domains of research and evaluation, so this may be a good time to revisit that distinction since this book is about both. Research has as its primary purpose contributing to knowledge, and evaluation has as its primary purpose informing action: using data to support decision making; informing judgments about merit, worth, and significance; and improving interventions (programs, policies, and products). (For more on what evaluation is, see American Evaluation Association, 2014.)

As we’ll see, some elements of the debate endure and constitute part of the context for engaging in qualitative inquiry today for both research and evaluation purposes. That’s why we need to take this short walk through history to understand how we got to where we are now, including some of the lingering effects and continuing divisions that sometimes surface today when methods decisions are being made. Forewarned is forearmed.

MODULE

9 Understanding the Paradigms Debate: Quants Versus Quals

Intellectual Walls and Fences

We build too many walls and not enough bridges. —Sir Isaac Newton (1642–1726)

English physicist, mathematician, astronomer, natural philosopher

Don’t ever take a fence down until you know why it was put up. —Robert Frost (1874–1963)

American poet

Once upon a time, not so very long ago, a group of statisticians (hereafter known as quants) and a party of case study aficionados (quals) found themselves together on a train traveling to the same professional meeting. The quals, all of whom had tickets, observed that the quants had only one ticket for their whole group.

“How can you all travel on one ticket?” asked a qual.

“We have our methods,” replied a quant.

Later, when the conductor came to punch tickets, all the quants slipped quickly behind the door of the toilet. When the conductor knocked on the door, the head quant slipped their one ticket under the door, thoroughly fooling the conductor.

On their return from the conference, the two groups again found themselves on the same train. The qualitative researchers, having learned from the quants, had schemed to share a single ticket. They were chagrined, therefore, to learn that, this time, the statisticians had boarded with no tickets.

“We know how you traveled together with one ticket,” said a qual, “but how can you possibly get away with no tickets?”

“We have ever more sophisticated methods,” replied a quant.

Later, when the conductor approached, all the quals crowded into the toilet. The head statistician followed them and knocked authoritatively on the toilet door. The quals slipped their one and only ticket under the door. The head quant took the ticket and joined the other quants in a different toilet. The quals were subsequently discovered without tickets, publicly humiliated, and tossed off the train at its next stop.

Quants Versus Quals

Who are quants? They’re numbers people who, in rabid mode, believe that if you can’t measure something, it doesn’t exist. They live by Galileo’s admonition, “Measure what is measurable, and make measurable what is not so.” Their mantra is “What gets measured gets done.” And quals? They quote management expert W. Edwards Deming: “The most important things cannot be measured.” Quals find meaning in words and stories and are ever ready to recite Albert Einstein’s observation that “everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted”—relatively speaking, of course.

Quants demand “hard” data: statistics, equations, charts, and formulae. Quals, in contrast, are “softies,” enamored with narrative and case studies. Quants love experimental designs and believe that the only way to prove that an intervention caused an outcome is with a randomized controlled trial (RCT). Quants are control freaks, say the quals: simplistic, even simpleminded, in their naive belief that the world can be reduced to independent and dependent variables. The qual’s world is complex, dynamic, interdependent, textured, nuanced, unpredictable, and understood through stories, more stories, and still more stories. Quals connect the causal dots through the unfolding patterns that emerge within and across these many stories and case studies. Quants aspire to operationalize key predictor variables and generalize across time and space—the holy grail of truth: If x, then y, and the more of x, the more of y. Quals distrust generalizations and are most comfortable immersed in the details of a specific time and place, understanding a story in the richness of context and the fullness of thick description. For quals, the patterns they extrapolate from cross-case analyses are possible principles to think about in new situations but are not the generalized, formulaic prescriptions that quants admire and aspire to. Quants produce best practices that assert, “Do this because it’s been proven to work in rigorous studies.” Quals produce themes and suggest, “Think about this and what it might mean in your own context and situation.”

Are these overgeneralized stereotypes? In some contexts, yes; in others, not so much. Is there a middle ground? Absolutely. In this chapter, we will move beyond the extremes and find more than one place in what is sometimes called the radical middle. We will also look at core values shared by the full range of scientific researchers. But before doing all of that, we’re going to linger for a moment where the differences are manifest. Quals and quants do hold contrasting, even opposite, worldviews at times—and the history of those differences provides an important context for appreciating current efforts at détente and integration.

So do opposites attract? Indeed, they do. They attract debate, derision, and dialectical differentiation— otherwise known as the paradigms war. The story of the quals and quants offers a window into how the paradigms debate has ebbed and flowed. This debate about the relative merits of quantitative/experimental methods versus qualitative/naturalistic methods has periodically run out of intellectual steam, but like mythical zombies, it seems never to die. Some, like Carol Weiss (2006), one of the distinguished founders of the field of evaluation research, refused to engage in the quantitative/qualitative debate because she thought it was “pretty silly” (p. 483), an elegantly phrased scholarly judgment. But one person’s silliness is another person’s battleground. I understand and appreciate the advice of one reviewer of this book, who wrote, “I would prefer comment that the Paradigm Wars are ridiculous and so he won’t bother to give them space.” But I’ve found that it’s helpful to know a bit about the nature and history of the qualitative/quantitative debate because remnants remain and can surface unexpectedly at inopportune moments. Forewarned is forearmed, so some review seems prudent.

Competing Inquiry Paradigms

Ideology is to research what Marx suggested the economic factor was to politics and what Freud took sex to be for psychology.

—Michael Scriven (1972a, p. 94) Philosopher of science and evaluation pioneer

Controversy often accompanies and even engulfs qualitative methods. Students attempting to do qualitative dissertations can get caught up in and may have to defend philosophically as well as methodologically the use of qualitative inquiry to those who have little experience with or training in it. Evaluators may encounter policymakers and funders who dismiss qualitative data as mere anecdote. The statistically addicted may poke fun at what they call “the softness” of qualitative data. (In Western society, where anything can be and often is sexualized, the distinction between “hard” data and “soft” data has additional nuances of meaning and innuendo.) Such encounters derive from a long-standing methodological paradigms war. Not everyone has

adopted a stance of methodological enlightenment and tolerance, namely, that methodological orthodoxy, superiority, and purity should yield to methodological appropriateness, pragmatism, and mutual respect. Therefore, as I noted in introducing this section, a brief review of the paradigms debate is in order. (Elsewhere I have provided a more extensive review of the methodological paradigms debate: Patton, 2008b, chap. 12; 2012a, chap. 11).

SIDEBAR

THE KINGDOM OF WISDOM

The king of the Kingdom of Wisdom was growing old. He had two sons to whom he commissioned the task of expanding the kingdom. One traveled south and built Dictionopolis, the city of words; the other went north and built Digitopolis, the city of numbers.

The two brothers and their two cities entered into a fierce rivalry. The people of Dictionopolis believed that words were wisdom and much more important than numbers. The people of Digitopolis believed that numbers were wisdom and much more important than words. The rivalry intensified.

To avoid war and settle the matter, they agreed to arbitration by two independent thinkers, Rhyme and Reason, princesses who lived in the capital city of Wisdom. After lengthy and thoughtful deliberation, they pronounced that words and numbers were both essential and equally valuable. Outraged, the brothers banished Rhyme and Reason from the Kingdom of Wisdom.

Eventually, after a great and dramatic rescue, Rhyme and Reason were returned to the Kingdom of Wisdom. To find out how, and why this story matters, you’ll have to enter into the Kingdom of Wisdom through The Phantom Tollbooth (Juster, 1961).

Philosophers of science and methodologists have been engaged in a long-standing epistemological debate about the nature of “reality” and knowledge. That philosophical debate finds its way into research and evaluation in differences of opinion about what constitutes “good” research and high-quality evidence. In its simplest and most strident formulation, this debate has centered on the relative value of two different and competing inquiry paradigms: (1) using quantitative and experimental methods to generate and test hypothetical-deductive generalizations versus (2) using qualitative and naturalistic approaches to inductively and holistically understand human experience in context-specific settings. To be sure, the variety of inquiry approaches has expanded well beyond the simplistic dichotomy between quantitative and qualitative paradigms. Still, the paradigms debate is part of our methodological heritage, and still all too alive in some places, so knowing a bit about it may deepen appreciation for the importance of a strategic approach to methodological decision making.

SIDEBAR

THE HIDDEN BIASES IN BIG DATA

The business and science worlds are focused on how large [quantitative] datasets can give insight on previously intractable challenges. The hype becomes problematic when it leads to . . . “data fundamentalism,” the notion that correlation always indicates causation, and that massive data sets and predictive analytics always reflect objective truth.

Data and data sets are not objective; they are creations of human design. We give numbers their voice, draw inferences from them, and define their meaning through our interpretations. Hidden biases in both the collection and analysis stages present considerable risks, and are as important to the big-data equation as the numbers themselves.

—Kate Crawford (2013) Harvard Business Review

What Is an Inquiry Paradigm?

A paradigm is a worldview—a way of thinking about and making sense of the complexities of the real world. As such, paradigms are deeply embedded in the socialization of adherents and practitioners. Paradigms tell us what is important, legitimate, and reasonable. Paradigms are also normative, telling the practitioner what to do without the necessity of long existential or epistemological consideration. But it is this aspect of paradigms that constitutes both strength and weakness—a strength in that it makes decisions about what to do relatively easy, a weakness in that the very reason for a certain decision is hidden in the unquestioned assumptions of the paradigm. Thomas Kuhn, philosopher of science, explained how paradigms work in his influential classic The Structure of Scientific Revolutions (1970):

Scientists work from models acquired through education and through subsequent exposure to the literature often without quite knowing or needing to know what characteristics have given these models the status of community paradigms. . . . That scientists do not usually ask or debate what makes a particular problem or solution legitimate tempts us to suppose that, at least intuitively, they know the answer. But it may only indicate that neither the question nor the answer is felt to be relevant to their research. Paradigms may be prior to, more binding, and more complete than any set of rules for research that could be unequivocally abstracted from them. (p. 46)

So why does all this matter to the student interested in pursuing some research or evaluation question? It matters because paradigm-derived biases are the source of the derogatory distinctions between “hard data” and “soft data,” empirical studies versus “mere anecdotes,” and “objective” research versus “subjective” studies. These labels reveal value-laden prejudices about what constitutes credible and valuable contributions to knowledge. Such prejudices and paradigmatic blinders limit methodological choices, flexibility, and creativity. Adherence to a methodological paradigm can lock researchers into unconscious patterns of perception and behavior that disguise the biased, predetermined nature of their methods “decisions.” Methods decisions tend to stem from disciplinary prescriptions, concerns about scientific status, old methodological habits, and comfort with what the researcher knows best. Training and academic socialization tend to make researchers biased in favor of and against certain approaches.

Methodological Pragmatism, Eclecticism, and Mixed Methods

Methodology depends primarily on the subject matter of what is investigated, and on certain background assumptions. Debates about methodology are productive only if the subject matter is considered first.

—Ernest House (1994, p. 13) Evaluation theorist and methodologist

Many researchers and evaluators still conduct their studies with a predominantly quantitative or qualitative paradigm, but many others have become eclectic and creative, mixing methods. Exhibit 3.1 summarizes the methods paradigms debate by presenting 10 contrasting emphases between the quantitative/experimental, the qualitative/naturalistic, and the mixed-methods paradigms.

SIDEBAR

“QUALITATIVE METHODS ARE RUBBISH.”

What were you thinking?! Qualitative methods?

What kind of limp-wristed latte-sipping excuse for real science is this?

—World War II spoof depicting the vociferous emotional and dismissive reactions qualitative methods can provoke

YouTube (4 minutes): http://www.youtube.com/watch?v=FIdN8NKjqts

In the quantitative/qualitative debate, the mixed-methods paradigm is sometimes called “the radical middle.”

It is not enough for mixed methodology researchers to exist in an epistemological space that lies somewhere between the quantitative and qualitative epistemological spaces. Rather, mixed researchers should strive for what is the radical middle, which should not be a passive and comfortable middle space wherein the status quo among quantitative and qualitative epistemologies is maintained, but rather a new theoretical and methodological space in which a socially just and productive coexistence among all research traditions is actively promoted, and in which mixed research is consciously local, dynamic, interactive, situated, contingent, fluid, strategic, and generative. (Onwuegbuzie, 2012, p. 192)

©2002 Michael Quinn Patton and Michael Cochran

Hellish Inquiry

Are There Paradigms After Death?

A central argument for mixing methods comes from “critical multiplism” (Shadish, 1993), which argues that any single method is biased and that only by thinking critically about the strengths and weaknesses of various methods, and using multiple methods to overcome single-methods biases, can rigorous results be achieved. (See the sidebar on critical multiplism, p. 92).

EXHIBIT 3.1 The Methods Paradigms Debate: Ten Contrasting Emphases

Both qualitative and quantitative inquiry share basic values around and commitments to systematic inquiry, matching methods to questions, conscientious data collection, appropriate analysis, and detailed reporting of the procedures followed, including acknowledging both the strengths and the weaknesses of the inquiry approach taken and the results attained. Exhibit 9.1 (pp. 659–660) presents 10 general scientific research quality criteria that apply to all kinds of inquiries: qualitative, quantitative, and mixed methods. Still, beyond generally shared methodological standards, there are different emphases between qualitative, quantitative, and mixed-methods inquiry approaches. These are matters of emphasis within ideal types, not rigid orthodoxy.

Paradigm Flexibility and Methodological Appropriateness

While a paradigm offers a coherent worldview, an anchor of stability and certainty in the real-world sea of chaos, operating narrowly within any singular paradigm, including a mixed-methods paradigm, can be quite limiting. My practical stance involves judging the quality of a study by its intended purposes, available resources, procedures followed, and results obtained, all within a particular context and for a specific audience. When a new drug is tested before being made available to the general population, a triple-blind randomized experiment to determine efficacy is the design of choice, with careful attention to controlled and carefully measured dosage and outcome interactions, including side effects. But if the concern is whether people are taking the new drug appropriately, and if one wants to know what a group of people think about the new drug (e.g., an antidepressant), how they make sense of taking or not taking it, what they believe about themselves as a result of experiencing the drug, and how those around them deal with it, then in-depth interviews and observations are the more appropriate method of inquiry. The importance of understanding alternative research paradigms is to sensitize researchers and evaluators to the ways in which their methodological prejudices, derived from their disciplinary socialization experiences, may reduce their methodological flexibility and adaptability.

Being practical and flexible allows one to eschew methodological orthodoxy in favor of methodological appropriateness as the primary criterion for judging methodological quality, recognizing that different methods are appropriate for different situations. Situational responsiveness means designing a study that is appropriate for a specific inquiry situation or interest. A major purpose of this book is to identify the kinds of research questions and program evaluation situations for which qualitative inquiry is an especially appropriate method of choice. A wide range of possibilities exist when selecting methods. The point is to do what makes sense and report fully on what was done, why it was done, and what the implications are for findings.

SIDEBAR

CRITICAL MULTIPLISM AS A RESEARCH STRATEGY

Scientists who rely on disciplinary training for scientific strategy are bound to produce research that is biased by paradigmatic errors of omission and commission. If so, then one of the more pressing needs in science is for the development of strategies that can uncover the biases of omission and commission that are inevitably present in all scientific methods and then ensure that they do not operate in the same direction in a study or in a research literature to yield a biased conclusion. . . .

Critical multiplism is a research strategy aimed at doing just that: It advises scientists to put together packages of imperfect methods and theories in a manner that minimizes constant biases. . . . Briefly, multiplism refers to the fact that any task in science can usually be conducted in any one of several ways, but in many cases no single way is known to be uniformly best. Under such circumstances, a multiplist advocates making heterogeneous those aspects of research about which uncertainty exists, so that the task is conducted in several different ways, each of which is subject to different biases. Critical refers to rational, empirical, and social efforts to identify the assumptions and biases present in the options chosen. Putting the two concepts together, we can say that the central tenet of critical multiplism is this: When it is not clear which of several defensible options for a scientific task is least biased, we should select more than one, so that our options reflect different biases, avoid constant biases, and leave no plausible bias overlooked. That multiple options yield similar results across operationalizations with different biases increases our confidence in the resulting knowledge. If different results occur when we do the task in different ways, then we have an empirical and conceptual problem to solve if we are to explain why this happened, but we are saved from a premature conclusion that a particular piece of knowledge is plausible. (Shadish, 1993, p. 18)

MQP Rumination # 3

Fools’ Gold: The Widely Touted Methodological “Gold Standard” Is Neither Golden nor a Standard

I am offering one personal rumination per chapter. These are issues that have persistently engaged, sometimes annoyed, occasionally haunted, and often amused me over more than 40 years of research and evaluation practice. Here’s where I state my case on the issue and make my peace.

The Wikipedia entry for randomized controlled trials (RCTs), reflecting common usage, designates such designs as the “gold standard” for research. News reports of research findings routinely repeat and reinforce the “gold standard” designation for RCTs (e.g., New York Times, 2014). Government agencies and scientific associations that review and rank studies for methodological quality acclaim RCTs as the gold standard. For example, researchers at the University of Saint Andrews reviewed “what counts as good evidence” and found that

when the research question is “what works?”, different designs are often placed in a hierarchy to determine the standard of evidence in support of a particular practice or

programme. These hierarchies have much in common; randomised experiments with clearly defined controls (RCTs) are placed at or near the top of the hierarchy and case study reports are usually at the bottom. (Nutley, Powell, & Davies, 2013, p. 10)

The Gold Standard Versus Methodological Appropriateness

A consensus has emerged in evaluation research that evaluators need to know and use a variety of methods in order to address the priority questions of particular stakeholders in specific situations. But researchers and evaluators get caught in contradictory assertions: (a) select methods appropriate for a specific evaluation purpose and question, and use multiple methods—both quantitative and qualitative— to triangulate and increase the credibility and utility of findings, but (b) one question is more important than others (the causal attribution question), and one method (RCTs) is superior to all other methods in answering that question. This is what is known colloquially as talking out of both sides of your mouth. Thus, we have a problem. The ideal of researchers and evaluators being situationally responsive, methodologically flexible, and sophisticated in using a variety of methods runs headlong into the conflicting ideal that experiments are the gold standard and all other methods are, by comparison, inferior. Who wants to conduct (or fund) a second-rate study if there is an agreed-on gold standard?

The Rigidity of a Single, Fixed Standard

The gold standard allusion derives from international finance, in which the rates of exchange among national currencies were fixed to the value of gold. Economic historians share a “remarkable degree of consensus” about the gold standard as the primary cause of the Great Depression:

the mechanism that turned an ordinary business downturn into the Great Depression. The constraints of the gold-standard system hamstrung countries as they struggled to adapt during the 1920s to changes in the world economy. And the ideology, mentality and rhetoric of the gold standard led policy makers to take actions that only accentuated economic distress in the 1930s. (Eichengreen & Temin, 1997, pp. 1–2)

The gold standard system collapsed in 1971 following the United States’ suspension of convertibility from dollars to gold. The system failed because of its rigidity. And not just the rigidity of the standard itself but also the rigid ideology of the people who believed in it: Policymakers across Europe and North America clung to the gold standard despite the huge destruction it was causing. There was a clouded mind-set with a moral and epistemological tinge that kept them advocating the gold standard until political pressure emerging from the disaster became overwhelming.

Treating RCTs as the gold standard is no less rigid. Asserting a gold standard inevitably leads to demands for standardization and uniformity (Timmermans & Berg, 2003). Distinguished evaluation pioneer Eleanor Chelimsky (2007) has offered an illuminative analogy:

It is as if the Department of Defense were to choose a weapon system without regard for the kind of war being fought; the character, history, and technological advancement of the enemy; or the strategic and tactical givens of the military campaign. (p. 14)

Indeed, while inquiries into the effectiveness of new pharmaceutical drugs or agricultural production techniques may benefit from RCTs, such designs are both inappropriate and misleading for inquiring into the complex, dynamic phenomena that characterize much of the human condition. The problem is that treating experimental designs as the gold standard cuts off serious consideration of alternative methods and channels millions of dollars of research and evaluation funds into support for a method that has not only strengths but also significant weaknesses. The gold standard accolade means that funders and policymakers begin by asking, “How can we do an experimental design?” rather than asking, “Given the state of knowledge and the priority inquiry questions at this time, what is the appropriate design?” Here are examples of the consequences of this rigid mentality:

• At an African evaluation conference, a program director came up to me in tears. She directed an empowerment program with women in 30 rural villages. The funder, an international agency, had just told her that to have the funding renewed, she would have to stop working in half the villages (selected randomly by the funder) in order to create a control group going forward. The agency was under pressure for not having enough “gold standard evaluations.” But, she explained, the villages and the women were networked together and were supporting each other. Even if they didn’t get funding, they would continue to support each other. That was the empowerment message. Cutting half of them off made no sense to her. Or to me.

• At a World Bank conference on youth service learning, the director of a university program that placed student interns in rural villages in Cambodia offered her program for an exercise in evaluation design. She explained that she carefully selected 40 students each year and matched them to villages that needed the kind of assistance the students could offer. Matching students and villages was key, she explained. A senior World Bank economist told her and the group to forget matching. He advised an RCT in which she would randomly assign students to villages and then create a control group of qualified students and villages that did nothing to serve as a counterfactual. He said, “That’s the only design we would pay any attention to here. You must have a counterfactual. Your case studies of students and villages are meaningless and useless.” The participants were afterward aghast that he had completely dismissed the heart of the intervention: matching students and villages.

• I’ve encountered several organizations, domestic and international, that give bonuses to managers who commission RCTs for evaluation to enhance the organization’s image as a place that emphasizes rigor. The incentives to do experimental designs are substantial and effective. Whether they are appropriate or not is a different question. In effect, it is gold from those who enforce the gold standard to those who implement it. One senior economist told me, “We’re applying the basic economic principle of giving rewards for what we want. We pay for gold standard designs. We promote those who commission and conduct gold standard designs. So we get gold standard designs. You get what you reward.” This explanation of how the world works rolled off his tongue like an oft- told story, which I suspect it was. He was smiling broadly at his cleverness, awash in his wisdom and self-congratulatory certainty. What I heard was rigidity, narrow-mindedness, misguided and distorted incentives, and an utter incapacity to detect even a hint of a problem.

Those experiences, multiplied 100 times, are what have generated this rumination.

Heaven or Gold Standard

Professor David Storey (2006) of Warwick Business School, University of Warwick, has offered a competing metaphor to replace the gold standard. He has posited “seven steps to heaven” in conducting evaluations, where heaven is a randomized experiment. So materially oriented and worldly evaluators are admonished to aspire to the gold standard, while the more spiritually inclined can aspire to follow the path to heaven, where heaven is an RCT.

In contrast, the metaphors of naturalistic inquiry are more along the lines of staying grounded, looking at the real world as it unfolds, going with the flow, being adaptable, and seeing what emerges.

Evidence-Based Medicine and RCTs

Medicine is often held up as the bastion of RCT research in its commitment to evidence-based medicine (EBM). But here again, gold standard designation has a downside, as observed by the psychologist Gary Klein (2014):

Sure, scientific investigations have done us all a great service by weeding out ineffective remedies. For example, a recent placebo-controlled study found that arthroscopic surgery provided no greater benefit than sham surgery for patients with osteoarthritic knees. But we also are grateful for all the surgical advances of the past few decades (e.g., hip and knee replacements, cataract treatments) that were achieved without randomized controlled trials and placebo conditions. Controlled experiments are therefore not necessary for progress in new types of treatments and they are not sufficient for implementing treatments with individual patients who each have unique profiles.

Worse, reliance on EBM can impede scientific progress. If hospitals and insurance companies mandate EBM, backed up by the threat of lawsuits if adverse outcomes are accompanied by any departure from best practices, physicians will become reluctant to try alternative treatment strategies that have not yet been evaluated using randomized controlled trials. Scientific advancement can become stifled if front-line physicians, who blend medical expertise with respect for research, are prevented from exploration and are discouraged from making discoveries.

Parachutes and RCTs Gordon C. S. Smith and Jill P. Pell (2003) published a clever and widely disseminated systematic review of RCTs on parachute use in the British Medical Journal. They arrived at the following conclusions:

• No RCTs of parachute use have been undertaken. • The basis for parachute use is purely observational (qualitative). • “Individuals who insist that all interventions need to be validated by a randomized controlled trial

need to come down to earth with a bump” (p. 1460).

On the other hand, they added that RCT’s parachute research could still be done: “We feel assured that those who advocate evidence-based medicine and criticize use of interventions that lack an evidence base will not hesitate to demonstrate their commitment by volunteering for a double blind, randomized, placebo controlled, crossover trial” (p. 1460).

RCTs and Bias RCTs aim to control bias, but implementation problems turn out to be widespread:

Even in the most stringent research designs, bias seems to be a major problem. For example, there is strong evidence that selective outcome reporting, with manipulation of the outcomes and analyses reported, is a common problem even for randomized trails. (Chan, Hrobjartsson, Haahr, Gotzsche, & Altman, 2004, p. 2457)

The result is that “a great many published research findings are false” (Ioannidis, 2005).

Methodological Appropriateness as the Platinum Standard

It may be too much to hope that the gold standard designation will disappear from popular usage. So perhaps we need to up the ante and aim to supplant the gold standard with a new platinum standard: methodological pluralism and appropriateness. To do so, I offer the following seven-point action plan:

1. Educate yourself about the strengths and weaknesses of RCTs. (See resources below.)

2. Never use the gold standard designation yourself. If it comes up, refer to the “so-called gold standard.”

3. When you encounter someone referring to RCTs as the gold standard, don’t be shy. Explain the negative consequences and even dangers of such a rigid pecking order of methods.

4. Understand and be able to articulate the case for methodological pluralism and appropriateness, to wit, adapting designs to the existing state of knowledge, the available resources, the intended uses of the inquiry results, and other relevant particulars of the inquiry situation. (See the resources listed below.)

5. Promote the platinum standard as higher on the hierarchy of research excellence (and even closer to heaven).

6. Don’t be argumentative and aggressive in challenging gold standard narrow-mindedness. It’s more likely a matter of ignorance than intolerance. Be kind, sensitive, understanding, and compassionate, and say, “Oh, you haven’t heard. The old RCT gold standard has been supplanted by a new, more enlightened, Knowledge-Age platinum standard.” (Beam wisely.)

7. Repeat Steps 1 to 6 over and over again.

Resources

• For 10 limitations of experimental designs, see Patton (2008b, pp. 447–450). • For the case against a rigid methodological hierarchy, see Ornish (2014), Nutley, Powell, and Davies

(2013), Chen, Donaldson, and Mark (2011), Donaldson, Christie, and Mark (2009), Patton (2008b), Scriven (2008), and Julnes and Rog (2007).

• For official statements advocating methodological pluralism, see American Evaluation Association (2003), “Scientifically Based Evaluation Methods,” and European Evaluation Society (2007), “The Importance of a Methodologically Diverse Approach to Impact Evaluation.”

• For alternative and mixed methods, see Broadening the Range of Designs and Methods for Impact Evaluations (Department for International Development, 2012), Social Research Methods: Qualitative and Quantitative Approaches (Bernard, 2013), The SAGE International Handbook of Educational Evaluation (Ryan & Cousins, 2009), and the alternative approaches for impact evaluation described in DFID (2012).

• In platinum standard references, the designation “platinum standard” has been used by Deborah Lowe Vandell, chair of the Department of Education at the University of California at Irvine, to describe an evaluation design that uses a range of data collection approaches (e.g., observations, interviews, surveys) to collect qualitative and quantitative data on program implementation and outcomes (Evaluation Exchange, 2010). See also “Toward a Platinum Standard for Evidence-Based Assessment by 2020,” which incorporates case studies, comparative methods, triangulation, and alternative causal approaches in recognition of the wide array of goals and methodologies that are appropriate for assessing programs and policies in a dynamic and globalizing world (Khagram & Thomas, 2010); “Evaluating Outside the Box: Mixing Methods in Analyzing Social Protection Programmes” (Devereux & Roelen, 2013); and Scriven’s (2008) article, which refers to a higher standard than gold for causal research, a platinum standard (p. 18).

MODULE

10 Introduction to Qualitative Inquiry Frameworks

From Core Strategies to Rich Diversity

There’s more than one way to skin a cat. —Old English proverb

Alternative proverbial meanings:

• More than one way to do almost anything • Many ways to get something you want • Many ways to make money • More than one way to prepare a catfish (used in the southern American states, where catfish is

often abbreviated to cat, a fish that is indeed usually skinned in preparing it for eating) • A gymnastic feat that involves passing the feet and legs between the arms while hanging by the

hands from a horizontal bar (as if turning an animal’s skin inside out when separating the skin from the body)

—Michael Quinion (1999) Lexicographer

Qualitative inquiry is not a single, monolithic approach to research and evaluation. The remainder of this chapter moves beyond the qualitative/quantitative debate to present a rich menu of alternative possibilities within qualitative research. While there are 12 core strategies that undergird any qualitative inquiry, as discussed in the last chapter (see Exhibit 2.1, pp. 46–47), there has emerged a proliferation of approaches within the qualitative paradigm. Frameworks for undertaking qualitative inquiry manifest great diversity. Different philosophical, epistemological, and theoretical perspectives and traditions have given rise to competing ways of engaging in qualitative inquiry. Those new to qualitative inquiry are often confused and even discombobulated by the diverse terminology and contested practices they encounter—phenomenology, hermeneutics, ethnomethodology, semiotics, heuristics, phenomenography. Such language! Exhibit 3.2 reproduces a letter of lamentation I received from a graduate student following publication of the first edition of this book, which did not include the current chapter. She felt lost in the arcane terminology. This chapter clarifies the major alternatives. The increasing diversity of qualitative inquiry frameworks has created both opportunity and confusion. This chapter aims to elucidate the opportunities and reduce the confusion.

EXHIBIT 3.2 Plea for Help: Which Approach Is Right?

Dear Dr. Patton,

I desperately need your help. I am a graduate student in education planning to do my dissertation observing classrooms and teachers identified as innovative and effective. I want to see if they share any common approaches or wisdom that might be considered “best practices.” I took this idea to one

professor who asked me if I was proposing a phenomenological or Grounded Theory study. When I asked what the difference was, he said it was my job to find out. I’ve read about both but am still confused. Another professor told me I could do a qualitative study, but that asking about “best practices” meant that I was a positivist not a phenomenologist. Another grad student was told to “use a hermeneutic framing,” but she’s in a different department with a different topic. I’m a former school teacher and, I think, a pretty good observer and interviewer. I got very excited reading your book about the value of in- depth observations and interviewing, and that’s where I got the idea for my dissertation, but now I’m being told I have to fit into one of these categories. Please tell me which one is right for my study. I don’t care which one it is. I just want to get on with studying innovative classrooms. I feel lost and am on the verge of just doing a questionnaire where these philosophy questions don’t seem to get asked. But if you can tell me which approach is right, I might still be able to do what I want to do. Help!!!!!!!!!!

* * * * *

This chapter is my full response to her poignant plea for help. I noted in my original personal response that her dilemma was all too common. Distinctions among frameworks are contentious; not everyone agrees about what the inquiry labels and traditions mean. What is certain is that there is no “right” approach any more than there is a “right” fruit—apples, oranges, passion fruit. What you eat is a matter of personal taste, availability, price, history, and preference. Each qualitative inquiry framework offers a different emphasis, framework, or focus.

In the remainder of this chapter, we shall look at how varying inquiry traditions emphasize different questions and how these particular emphases can affect the analytical framework that guides fieldwork and interpretation. Understanding the divergent theoretical and philosophical traditions that have influenced qualitative inquiry is especially important in the design stage, when the focus of fieldwork and interviewing is determined. Weaving together theory-based inquiry traditions and qualitative methods will reveal a rich tapestry with many threads of differing texture, color, length, and purpose.

Qualitative Inquiry Frameworks: Distinguishing Core Questions There is no definitive way to categorize the various philosophical and theoretical perspectives that have influenced and that distinguish the types of qualitative inquiry. This chapter distinguishes theoretical perspectives by their foundational questions. A foundational or burning question, like the mythic burning bush of Moses, blazes with heat (controversy) and light (wisdom) but is not consumed (is never fully answered). Disciplines given birth by the mother of all disciplines, philosophy, can be distinguished by their core burning questions. For sociology, the burning question is the Hobbesian question of order: What holds society or social groups together? What keeps societal groups from falling apart? Psychology asks why individuals think, feel, and act as they do. Political science asks, “What is the nature of power, how is it distributed, and with what consequences?” Economics studies how resources are produced and distributed.

Disciplines and subdisciplines reveal layers of questions. Biologists inquire into the nature and variety of life. Botanists ask how plants grow, while agriculturists investigate the production of food, and agronomists narrow their focus still further to field crops. To be sure, reducing any complex and multifaceted discipline to a single burning question oversimplifies the focus of that discipline. But what is gained is clarity about what distinguishes one lineage of inquiry from another. It is precisely that clarity and focus I shall strive for in identifying the burning questions that distinguish major lineages of qualitative inquiry. In doing so, I shall displease those who prefer to separate paradigms from philosophies and distinguish theoretical orientations from design strategies. For example, social constructivism may be viewed as a paradigm, ethnography may be considered a research strategy, and symbolic interactionism may be examined as a theoretical framework. However, distinctions between the paradigmatic, strategic, and theoretical dimensions within any particular approach are both arguable and arbitrary. Therefore, I have circumvented those distinctions by focusing on and distinguishing core inquiry questions as the basis for understanding and contrasting long-standing and

emergent qualitative inquiry approaches. Exhibit 3.3 presents the core question for each inquiry framework reviewed here and serves as a guide for navigating through this chapter. We begin with ethnography’s inquiry into culture.

EXHIBIT 3.3 Alternative Qualitative Inquiry Frameworks: Core Questions and Disciplinary Roots

MODULE

11 Ethnography and Autoethnography

Culture as Central to Understanding Human Diversity

The purpose of anthropology is to make the world safe for human differences. —Ruth Benedict (1887–1948)

Ethnography pioneer

Human culture exists in large measure to restrain the natural desires of the species. —David Brooks (2011, p. 15)

The Social Animal

All 7 billion+ people alive on Earth today share a common female ancestor, dubbed Mitochondrial Eve, who lived in Africa about 140,000 years ago (Genomics, 2010; Oppenheimer, 2003). We all share her mitochondrial DNA. But we are not all the same. Culture explains the huge range of human diversity that cannot be attributed to genetic inheritance. Thus, I start this review of qualitative inquiry frameworks with ethnography, the study of culture.

Ethnography

Core inquiry questions: What is the culture of this group of people? How does culture explain their perspectives and behaviors?

Ethnography, the primary method of anthropology, is the earliest distinct tradition of qualitative inquiry. The notion of culture is central to ethnography. Ethnos is the Greek word for “a people” or cultural group. The study of ethnos, then, or ethnography, is “devoted to describing ways of life of humankind . . . , a social scientific description of a people and the cultural basis of their peoplehood” (Vidich & Lyman, 2000, p. 38). Ethnographic inquiry takes as its central and guiding assumption that any human group of people interacting together for a period of time will evolve a culture. Culture is that collection of behavior patterns and beliefs that constitute “standards for deciding what is, standards for deciding what can be, standards for deciding how one feels about it, standards for deciding what to do about it, and standards for deciding how to go about doing it” (Goodenough, 1971, pp. 21–22). The primary method of ethnographers is participant observation in the tradition of anthropology. This means intensive fieldwork in which the investigator is immersed in the culture under study. Ethnography becomes not just observation but a way of seeing (Wolcott, 2008).

Anthropologists have traditionally studied nonliterate cultures in remote settings, what were often thought of as “primitive” or “exotic” cultures. As a result, anthropology and ethnographers became intertwined with Western colonialism, sometimes resisting imperialism in efforts to sustain native cultures and sometimes as handmaidens to conquering empires as their findings were used to overcome resistance to change and manage subjugated peoples.

Modern anthropologists apply ethnographic methods to the study of contemporary society to understand phenomena such as Information Age culture, the culture of poverty, school culture, the culture of addiction, intercultural marriages, youth culture, and the spread of disease (ethno-epidemiology), to give but a few of many examples. Ethnography is being used to study international relations and global diplomacy (Lie, 2013), corporate environments (Jordon, 2012), and consumer behavior (Sutherland & Denny, 2007). Understanding how group culture affects individual behavior can be a core ethnographic question; for example, what makes a 17-year-old girl decide to wrap a bomb around her body, walk into a supermarket, and detonate it, killing herself and an 18-year- old girl shopping there (Handwerker, 2009)? Public ethnography aims to help people speak to issues that affect them and their communities and to support change (Beaver, 2012).

The importance of understanding culture, especially in relation to change efforts of all kinds, is the cornerstone of “applied ethnography” (Chambers, 2000; Pelto, 2013). This can be seen in the ongoing reports of members of the Society for Applied Anthropology since its founding in 1941.

Since the 1980s, understanding culture has become central in organizational studies (Morgan, 1986, 1989; Pettigrew, 1983) and in much organizational development work, including major efforts to change the culture of an organization or program. Organizational ethnography has a distinguished history that can be traced back to the influential Hawthorne electric plant study that began in 1927 (Schwartzman, 1993). Programs develop cultures, just as organizations do. A program’s culture can be thought of as part of the program’s treatment. As such, the culture affects both program processes and outcomes. Improving a program, then, may include changing the program’s culture (Patton, 2012a, pp. 144–147). An ethnographic evaluation would both facilitate and assess such change.

While traditionally ethnographers have used the methods of participant observation and intensive fieldwork to study everything from small groups to nation-states, what it means to “participate” or be in the “field” has changed with the advent of the Internet and social media. Nevertheless, whether doing ethnography in virtual space, a nonliterate community, a multinational corporation, or an inner-city school, what makes the approach distinct is the matter of interpreting and applying the findings from a cultural perspective. The Ethnographer’s Toolkit (Schensul & LeCompte, 2010) offers an introductory overview of ethnographic methods, as does Fetterman’s (2009) Ethnography: Step by Step, Walcott’s (2010) primer Ethnography Lessons, Murchison’s (2010) Ethnography Essentials, and O’Reilly’s (2012a) Ethnographic Methods. Skinner (2013) focuses on an ethnographic approach to interviewing. As the myriad references available for guiding ethnography indicate, it is a well-established approach. Ethnography has also spawned a derivative but substantially different way of looking at and inquiring into culture: autoethnography.

Autoethnography

Core inquiry question: How does my own experience of my culture offer insights about this culture, situation, event, and way of life?

We turn now from one of the earliest qualitative traditions, ethnography, to a recent and still emergent approach: autoethnography. Ethnography first emerged as a method for studying and understanding the other. It was their fascination with exotic otherness that attracted Europeans to study the peoples of Africa, Asia, the South Sea Islands, and the Americas. “The life world of the ‘primitive’ was thought to be the window through which the prehistoric past could be seen, described, and understood” (Vidich & Lyman, 2000, p. 46). In the United States, for educated, white, university-based Americans, the others were blacks, American Indians, recent immigrants, working-class families, and the inner-city poor (and, for that matter, anyone else not well educated, white, and university based). In recent times, when ethnography began to be used in program evaluations, the other became the program client, the student, the welfare recipient, the patient, the alcoholic, the homeless, the victim, the perpetrator, or the recidivist. In organizational studies, the other was the worker, the manager, the leader, the follower, and/or the board of directors. The others were observed, interviewed, and described, and their culture was conceptualized, analyzed, and interpreted. Capturing and being true to the perspective of those studied, what came to be called the emic perspective or the “insider’s perspective,” was

contrasted with the ethnographer’s perspective, the etic, or outsider’s view. The etic viewpoint of the ethnographer implied some important degree of detachment or “higher” level of conceptual analysis and abstraction. To the extent that ethnographers reported on their own experiences as participant-observers, it was primarily methodological reporting related to how they collected data and how, or the extent to which, they maintained detachment. To “go native” was to lose perspective.

©2002 Michael Quinn Patton and Michael Cochran

Cross-cultural Perspective

In the twenty-first-century postcolonial and postmodern world, the relationship between the observed and the observer has been called into question at every level. Postcolonial sensitivities raise questions about imbalances of power, wealth, and privilege between ethnographers and those they would study, including critical political questions about how findings will be used. Postmodern critiques and deconstruction of classic ethnographies have raised fundamental questions about how the values and cultural background of the observer affect what is observed, while also raising doubts about the desirability, indeed the possibility, of detachment. Then there is the basic question of how an ethnographer might study her or his own culture. What if there is no other as the focus of study but I want to study the culture of my own group, my own community, and my own organization and the way of life of people like me or people I regularly encounter, or my own cultural experiences?

These developments have contributed to the emergence of autoethnography—studying one’s own culture and oneself as part of that culture—and its many variations. David Hayano (1979) is credited with originating the term autoethnography to describe studies by anthropologists of their own cultures. Goodall (2000) calls this the new ethnography: “creative narratives shaped out of a writer’s personal experiences within a culture and addressed to academic and public audiences” (p. 9). In their extensive review of methods that focus on studying one’s own culture and oneself as part of that culture to understand and illuminate a way of life, Ellis and Bochner (2000) cite a large number of phrases that have been used and conclude that “autoethnography has become the term of choice in describing studies and procedures that connect the personal to the cultural” (p. 740).

Carolyn Ellis, an autoethnography pioneer, describes it this way:

Autoethnography is an autobiographical genre of writing and research that displays multiple layers of consciousness, connecting the personal to the cultural. Back and forth autoethnographers gaze, first through an ethnographic wide-angle lens, focusing outward on social and the cultural aspects of their personal experience; then, they look inward, exposing a vulnerable self that is moved by and may move through, refract, and resist cultural interpretations. As they zoom backward and forward, inward and outward, distinctions between the personal and cultural become blurred, sometimes beyond distinct recognition. Usually written in first-person voice, autoethnographic texts appear in a variety of forms—short stories, poetry, fiction, novels, photographic essays, personal essays, journals, fragmented and layered writing, and social science prose. In these texts, concrete action, dialogue, emotion, embodiment, spirituality, and self-consciousness are featured, appearing as relational and institutional stories affected by history, social structure, and culture, which themselves are dialectically revealed through action, feeling, thought, and language. (Ellis & Bochner, 2000, p. 739)

In autoethnography, then, you use your own experiences to garner insights into the larger culture or subculture of which you are a part. Autoethnography is both a perspective and a method (Chang, 2008; Muncey, 2010). As the Handbook of Autoethnography (Jones, Adams, & Ellis, 2013) demonstrates, great variability exists in the extent to which autoethnographers make themselves the focus of the analysis, how much they keep their role as social scientists in the foreground, the extent to which they use the sensitizing notion of culture, at least explicitly, to guide their analysis, and how personal their writing is. At the center, however, what distinguishes autoethnography from ethnography is self-awareness about and reporting of one’s own experiences and introspections as a primary data source. Ellis describes this process as follows:

I start with my personal life. I pay attention to my physical feelings, thoughts, and emotions. I use what I call systematic sociological introspection and emotional recall to try to understand an experience I’ve lived through. Then I write my experience as a story. By exploring a particular life, I hope to understand a way of life. (Ellis & Bochner, 2000, p. 737)

SIDEBAR

HONORING ONE’S OWN EXPERIENCE

One writes out of one thing only—one’s own experience. Everything depends on how relentlessly one forces from this experience the last drop, sweet or bitter, it can possibly give. This is the only real concern of the artist, to recreate out of the disorder of life that order which is art.

—James Baldwin (1990) Introduction

Tony Adams (2011) used autoethnography to elucidate of the meaning of “the closet” through his lived experience in the gay community. Anthropologist Mary Catherine Bateson’s (2000) autoethnographic description of teaching at a seminar at Spelman College in Atlanta includes detailed attention to the personal challenge she experienced in trying to decide how to engage with students of different ages, for example, calling older participants “elders.” Aaron Turner (2000) has explored using one’s own body as a source of data in ethnography—what he calls “embodied ethnography.” Doing autoethnography in collaboration has emerged as an alternative to working entirely alone (Chang, Wambura, & Hernandez, 2013; Norris, Sawyer, & Lund, 2012). Performing autoethnographically offers new ways of communicating findings (Spry, 2011).

Such personal writing is controversial because of its “rampant subjectivism” (Crotty, 1998, p. 48). Some critics object to the way it blurs the lines between social science and literary writing. One sociologist told me angrily that those who want to write creative nonfiction or poetry should find their way to the English department of the university and leave sociology to sociologists. Laurel Richardson (2000b), in contrast, sees the integration of art, literature, and social science as precisely the point, bringing together the creative and critical aspects of inquiry. She suggests that what these various new approaches and emphases share is that “they are produced through creative analytic practices,” which leads her to call “this class of ethnographies

creative analytic practice ethnography” (p. 929). While the ethnographic aspect of this work is constructed on a foundation of careful research and fieldwork (p. 937), the creative element resides primarily in the writing, which she emphasizes is itself “a method of inquiry, a way of finding out about yourself and your topic” (p. 923). Exhibit 3.4 summarizes Richardson’s criteria for judging the quality of an autoethnography.

Ellis warns that autoethnographic writing is hard to do:

It’s amazingly difficult. It’s certainly not something that most people can do well. Most social scientists don’t write well enough carry it off. Or they’re not sufficiently introspective about their feelings or motives, or the contradictions they experience. Ironically, many aren’t observant enough of the world around them. The self- questioning autoethnography demands is extremely difficult. So is confronting things about yourself that are less than flattering. Believe me, honest autoethnography exploration generates a lot of fears and doubts—and emotional pain. Just when you think you can’t stand the pain anymore, well, that’s when the real work has only begun. Then there’s the vulnerability of revealing yourself, not being able to take back what you’ve written or having any control over how readers interpret it. It’s hard not to feel your life is being critiqued as well as your work. It can be humiliating. And the ethical issues. Just wait until you’ve written about family members and loved ones who are part of your story. (Ellis & Bochner, 2000, p. 738)

EXHIBIT 3.4 Criteria for Judging the Quality of an Autoethnography

1. Substantive contribution. Does this piece contribute to our understanding of social life? . . . 2. Aesthetic merit: . . . Does the use of creative analytic practices open up the text, invite interpretive

responses? Is the text artistically shaped, satisfying, complex, and not boring? 3. Reflexivity: . . . How has the author’s subjectivity been both a producer and a product of this text? Is

there adequate self-awareness and self-exposure for the reader to make judgments about the point of view? . . .

4. Impact: Does this affect me? Emotionally? Intellectually? Does it generate new questions? Move me to write? Move me to try new research practices? Move me to action?

5. Expression of a reality: Does this text embody a fleshed out, embodied sense of lived experience? Does it seem a “true”—a credible account of a cultural, social, individual, or communal sense of the “real”?

SOURCE: Richardson (2000a, p. 254; 2000b, p. 937).

In my own major effort at autoethnographic inquiry (Patton, 1999), the struggle to find an authentic voice— authentic first to me, then to others who knew me, and finally to those who did not know me—turned what I thought would be a one-year effort into seven years of often painful, discouraging writing. And I was only writing about a 10-day period, a Grand Canyon hike with my son in which we explored what it means to come of age, or be initiated into adulthood, in modern society. My son started and graduated from college while I was learning how to tell the story of what we experienced together. To make the story work as a story and to make scattered interactions coherent, I had to rewrite the conversations that took place over several days into a single evening’s dialogue, I had to reorder the sequence of some conversations to enhance the plotline, and I had to learn to follow the novelist’s mantra to “show, don’t tell,” advice particularly difficult for those of us who make our living telling. More difficult still was revealing my emotions, foibles, doubts, weaknesses, and uncertainties. But once the story was told, the final chapter of the book, which contrasts alternative coming- of-age/initiation paradigms, emerged relatively painlessly.

©2002 Michael Quinn Patton and Michael Cochran

Confronting a Critic of Autoethnography

Autoethnography integrates ethnography with personal story, a specifically autobiographical manifestation of the more general “turn to biographical methods in social science” that strive to “link macro and micro levels of analysis . . . [and] provide a sophisticated stock of interpretive procedures for relating the personal and the social” (Chamberlayne, Bornat, & Wengraf, 2000, pp. 2–3).

By opening this chapter with the contrast between ethnography and autoethnography, we have moved from the beginnings of qualitative methods in anthropological fieldwork more than a century ago, where the ethnographer was an outsider among exotically distinct nonliterate peoples, to the most recent manifestation of qualitative inquiry in the postmodern age of mass communications, where autoethnographers struggle to find a distinct voice by documenting their own experiences in an increasingly all-encompassing and commercialized global culture. Next, we look at how quantitatively oriented inquiry traditions have influenced qualitative inquiry.

MODULE

12 Positivism, Postpositivism, Empiricism, and FoundationalistEpistemologies

Truth, Reality, and Objectivity

A judgment is said to be true when it conforms to the external reality. —Thomas Aquinas (1225–1274)

Philosopher and theologian

Truth, as any dictionary will tell you, is a property of certain of our ideas. It means their “agreement,” as falsity means their disagreement, with reality.

—William James (1842–1910) Philosopher and psychologist

Reality-Testing Inquiry Frameworks: Foundationalist Epistemologies

Core inquiry questions: What’s the nature of the real world? What’s true? How can one inquire objectively so that findings correspond to reality?

What these questions have in common is the presumption that there is a real world with verifiable patterns that can be observed and predicted—that reality exists and truth is worth striving for. Reality can be elusive, and truth can be difficult to determine, but describing reality and determining truth are the appropriate goals of scientific inquiry. Working from this perspective, researchers and evaluators seek methods that yield correspondence with the “real world”; thus, this is sometimes called a correspondence perspective.

Correspondence theory: A statement is true if it describes reality accurately.

—Epstein (2012, p. 20)

Reality-oriented inquiry and the search for truth have fallen on hard times in this skeptical postmodern age when honoring multiple perspectives and diverse points of view has gained ascendance in reaction against the oppressive authoritarianism and dogmatism that seemed so often to accompany claims of having found “truth.” Yet many people, especially policymakers and those who commission evaluation research, find it difficult to accept the notion that all explanations and points of view hold equal merit. Some people in programs seem to be helped more than others. Some students seem to learn more than others. Some claims of effectiveness are more plausible and have more merit than others. To test a claim of effectiveness by bringing data to bear on it, including qualitative data, is to be engaged in a form of reality testing that uses evidence to examine assertions and corroborate claims. How we can study, test, and come to know reality is a matter of considerable debate among philosophers of science and has led to distinctions among even those who share the belief that there is a fundamental reality to be discovered. I’ll briefly review the reality-focused traditions in the philosophy of science and then examine the implications for qualitative methods.

Positivism and Postpositivism Positivism, following August Comte (1798–1857), asserted that only verifiable claims based directly on experience could be considered genuine knowledge. Comte was especially interested in distinguishing the empirically based “positive knowledge” of experience from theology and metaphysics, which depended on fallible human reason and gullible belief. Logical positivism emerged in Austria and Germany in the early part of the twentieth century by combining Comte’s emphasis on direct experience with a rigorous and systematic application of rational thought based in logic. Logical positivism subsequently came to be associated with philosophical efforts to specify the basic requirements for what could be considered truly scientific knowledge, which included (a) the search for universal laws through (b) empirical verification of logically deduced hypotheses with (c) key concepts and variables operationally defined and carefully formulated to (d) permit replication, falsification, and, ultimately, generalization across time and space. Thus, real scientific knowledge (as opposed to mere beliefs) is limited to what can be logically deduced from theory, operationally measured, and empirically validated. Though influential in the first half of the twentieth century, logical positivism has been “almost universally rejected” as a basis for social science inquiry because of its narrow definition of both science and knowledge (Campbell, 1999a, p. 132). The legacy of the fleeting influence of logical positivism is that the term lives on as an epithet hurled in paradigm debates and routinely used incorrectly though persistently. Shadish (1995b) argues that one would be hard-pressed to find any contemporary social scientist, philosopher, or evaluator who really adheres to the absolute and narrow tenets of logical positivism. Rather,

the term has become the linguistic equivalent of “bad,” a rhetorical device aimed at depriving one’s opponent of credibility by name-calling. This is particularly true in the quantitative-qualitative debate where some qualitative theorists are fond of labeling all quantitative opponents as logical positivists, a fundamental but common “error.” (p. 64)

Postpositivism, which takes into account the criticisms against and weaknesses of rigid positivism, now informs much of contemporary social science research, including truth- and reality-oriented qualitative inquiry. As articulated by the eminent methodologist Donald T. Campbell in his collected writings about and vision for an “Experimenting Society” (Campbell & Russo, 1999), postpositivism tempers positivism by positing that

• discretionary judgment is unavoidable in science; • proving causality with certainty in human affairs is problematic; • knowledge is inherently embedded in historically specific paradigms and is therefore relative rather than

absolute; and • all methods are imperfect, so multiple methods, both quantitative and qualitative, are needed to generate

and test theory, improve understanding over time of how the world operates, and support informed policy making and social program decision making.

While they are modest in asserting what can be known with any certainty, postpositivists do assert that it is possible, using empirical evidence, to distinguish between more and less plausible claims, to test and choose between rival hypotheses, and to distinguish between “belief and valid [italics added] belief” (Campbell, 1999b, p. 151).

Logical Empiricism

Logical empiricism, another more moderate version of logical positivism, asserts that both natural and social sciences share the goal of describing, understanding, and explaining reality. Empiricists “maintain that objects exist regardless of whether we have any sense or consciousness of them—they are ‘out there’—quite apart from whether we know about them” (Epstein, 2012, p. 10). Schwandt (2001) helpfully explains the difference

between empiricism as philosophy of science and empirical research as any inquiry that is data based: Empiricism is founded on

the premise that knowledge begins with sense experience. . . . All forms of qualitative inquiry are empirical research to the extent that they deal in the data of experience. . . . To say that qualitative inquiry is empirical research, however, is not to say that it rests on the philosophy of empiricism. (pp. 67, 69)

Objectivism and Foundationalist Epistemologies

Objectivism is another philosophy that asserts “an independently existing world of objective reality that has a determinate nature that can be discovered” (Schwandt, 2001, p. 176).

Positivism, empiricism, and objectivism share a common foundation as “foundationalist epistemologies”:

These epistemologies assume the possibility and necessity of the ultimate grounding of knowledge claims. To be considered genuine, legitimate, and trustworthy (and not simply mere belief), knowledge must rest on foundations that require no further justification or interpretation. (Schwandt, 2001, p. 101)

Practice Implications for Qualitative Inquiry

What are the practical implications for qualitative inquiry of operating within a truth- and reality-oriented perspective? It means using the language and concepts of mainstream science to design naturalistic studies, inform data gathering in the field, analyze results, and judge the quality of qualitative findings. Thus, if you are a researcher or evaluator operating from a reality-oriented stance, you incorporate and use scientific terms like validity, reliability, causality, generalizability, and objectivity. You realize that completely value-free inquiry is impossible, but you worry about how your values and preconceptions may affect what you see, hear, and record in the field, so you wrestle with your values, try to make any biases explicit, take steps to mitigate their influence through rigorous field procedures, and discuss their possible influence in reporting findings. You may establish an “audit trail” to verify the rigor of your fieldwork and confirmability of the data collected because you want to minimize bias, maximize accuracy, and report impartially. In reporting, you emphasize the empirical findings—good, solid description and analysis—not your own personal perspective or voice, though you acknowledge that some subjectivity and judgment may enter in. You include triangulation of data sources and analytical perspectives to increase the accuracy and credibility of findings. Your criteria for quality include the “truth value” and plausibility of findings; credibility, impartiality, and independence of judgment; confirmability, consistency, and dependability of data; and explainable inconsistencies or instabilities. You may generalize case study findings, depending on the cases selected and studied, to generate or test theory, establish causality, or inform program improvement and policy decisions from the patterns established and lessons learned. In short, you incorporate the language and principles of twentieth-century science into naturalistic inquiry and qualitative analysis to convey a sense that you are dedicated to getting as close as possible to what is really going on in whatever setting you are studying. Realizing that absolute objectivity of the pure positivist variety is impossible to attain, you are prepared to admit and deal with imperfections in a complex, messy, and methodologically imperfect world, but you still believe that objectivity is worth striving for. As Kirk and Miller (1986) have asserted,

Objectivity, though the term has been taken by some to suggest a naive and inhumane version of vulgar positivism, is the essential basis of all good research. Without it, the only reason the reader of the research might have for accepting the conclusions of the investigator would be an authoritarian respect for the person of the author. Objectivity is a simultaneous realization of as much reliability and validity as possible. Reliability is the degree to which the finding is independent of accidental circumstances of the research, and validity is the degree to which the finding is interpreted in a correct way. (p. 20)

The truth- and reality-testing framework informs and undergirds most research and evaluation supported by government agencies, international organizations, scientific institutes, philanthropic foundations, and university programs. It will not be called that or labeled thus because the assumptions and perspective of this framework are deeply embedded at the level of a paradigm. It is equated to good “science.” Most quantitative

research operates from this traditional scientific perspective and only adds counting and measuring as operational procedures. What is counted and measured, if valid and reliable, is real and true. The underlying philosophical premises are not made explicit, because they are basic assumptions, deeply shared, that don’t need to be made explicit. They simply are. When this framework is transferred from quantitative research into qualitative research, the methods of data collection are changed, but the inquiry framework is not.

SIDEBAR

SPEAKING TRUTH: A DIFFERENT VIEW OF REALITY

More than any other time in history, mankind faces a crossroads. One path leads to despair and utter hopelessness. The other, to total extinction. Let us pray we have the wisdom to choose correctly. I speak, by the way, not with any sense of futility, but with a panicky conviction of the absolute meaninglessness of existence, which could easily be interpreted as pessimism. It is not. It is merely a healthy concern for the predicament of modern man.

—Woody Allen (1981) Filmmaker and humorist

What makes the underlying positivist framework hard to detect is that the qualitative literature, as represented by qualitative handbooks (e.g., Denzin & Lincoln, 2011), qualitative encyclopedias (e.g., Given, 2008), qualitative dictionaries (e.g., Schwandt, 2007), qualitative journals (e.g., Qualitative Inquiry, Qualitative Research), and major textbooks, presents a diverse array of inquiry frameworks, as represented by this chapter. Indeed, the diversity of inquiry frameworks is as impressive as it is confusing and contentious. But diversity of possibility is not diversity of practice. Most qualitative research and evaluation sponsored by official agencies and funders defaults to the truth- and reality-testing inquiry framework, expects objectivity to be valued and aimed for, and looks for findings that are presented as valid, reliable, generalizable, and, essentially, true in the commonsense meaning of true—that is, factual, credible, and supported by empirical evidence.

To emphasize this point, I close this review of truth- and reality-oriented qualitative inquiry with an excerpt from a medical journal in which health researchers are defending qualitative research to an audience known to be skeptical. Their approach is to associate qualitative research closely with accepted and credible forms of experimental research. Such a perspective epitomizes the truth- and reality-testing orientation:

What, then, does the qualitative researcher do once he or she accomplishes a careful and trustworthy understanding of the language and behavior of an individual human being? Here is where we rely on our positivist skills and methods. . . . Once we carefully examine and articulate that which we understand one human being to be doing, we attempt to collate the language and behavior of many human beings, so many that we might be able to test the relationships, for example, between setting and behavior or between age and hope. Included within the domains of qualitative science or narrative research, then, are efforts to generalize, to predict, and to relate initial states to outcomes. These efforts require the same evidence-based activities that are used in testing any hypotheses. (Charon, Greene, & Adelman, 1998, p. 68)

As Weiss and Bucuvalas (1980) found in their classic research, real-world decision makers as primary users of research and evaluation apply simple “truth tests” and “utility tests.” Is it true? If so, can we do something with it? The truth- and reality-testing inquiry framework aims to pass the truth test first and foremost. As we’ll see later, pragmatism picks up and focuses on the utility test. But, first, let’s look at qualitative frameworks that focus on generating and testing theory to understand reality.

SOURCE: © Chris Lysy—freshspectrum.com

MODULE

13 Grounded Theory and Realism

Theory Generation and Testing: Explaining How the World Works and Why It Works as It Does

It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.

—Arthur Conan Doyle Sherlock Holmes

I am using the word theory as a scientist means it: a set of ideas so well established by observations and physical models that it is essentially indistinguishable from fact. That is different from the colloquial use that means “guess.” To a scientist, you can bet your life on a theory. Remember, gravity is “just a theory” too.

—Philip C. Plait Astronomer

Grounded Theory

Core inquiry question: What theory, grounded in fieldwork, emerges from systematic comparative analysis so as to explain what has been observed?

Influential qualitative theorist and methodologist Norman K. Denzin (1997a) observed some time ago that the “grounded theory approach is the most influential paradigm for qualitative research in the social sciences today” (p. 18). That assessment remains accurate, in my judgment, as it does in the judgment of those involved in further developing and applying this approach: “Grounded Theory is probably the most commonly used qualitative method, surpassing ethnography, and it is used internationally” (Morse et al., 2009). Why is it so influential? In part, it may be because it’s an approach that comfortably incorporates and applies quantitative concepts like validity, reliability, causality, and generalizability to qualitative inquiry, as just discussed in the previous section; in part because the procedures for grounded theory are systematized and prescriptive as well as contextually adaptable (Bryant & Charmaz, 2010); in part because Glaser and Strauss (1967), who developed the approach (before later taking their ideas in different directions), mentored a generation of students who have practiced widely and published prolifically; in part because grounded theory has won widespread acceptance as sufficiently rigorous to serve as an acceptable framework for academic dissertations precisely because of the emphasis on data-based theory; and, finally, in part because it unabashedly admonishes the researcher to strive for objectivity.

It is important to maintain a balance between the qualities of objectivity and sensitivity when doing analysis. Objectivity enables the researcher to have confidence that his or her findings are a reasonable, impartial representation of a problem under investigation, whereas sensitivity enables creativity and the discovery of new theory from data. (Strauss & Corbin, 1998, p. 53)

Concern for theory development is often quite marked in the literature on qualitative methods. The classic writings of Glaser (1978, 2000), Strauss and Corbin (1998), Denzin (1978b), Lofland and Lofland (1984), Blumer (1969), Whyte (1984), and Becker (1970), to name but a few well-known qualitative inquiry advocates, take as a major contribution to knowledge the task of theory construction and verification. What distinguishes the discussion of theory in much of the qualitative methods literature is an emphasis on inductive strategies of theory development in contrast to theory generated by logical deduction from a priori assumptions.

In contrasting Grounded Theory with logico-deductive theory and discussing and assessing their relative merits in ability to fit and work (predict, explain, and be relevant), we have taken the position that the adequacy of a theory for sociology today cannot be divorced from the process by which it is generated. Thus one canon for judging the usefulness of a theory is how it was generated—and we suggest that it is likely to be a better theory to the degree that it has been inductively developed from social research. . . . Generating a theory from data means that most hypotheses and concepts not only come from the data, but are systematically worked out in relation to the data during the course of the research. Generating a theory involves a process of research. (Glaser & Strauss, 1967, pp. 5–6)

This theory–method linkage means that how you study the world determines what you learn about the world. Grounded theory depends on methods that take the researcher into and close to the real world, so that the results and findings are “grounded” in the empirical world. Herbert Blumer has offered a lifting-the-veil metaphor for explaining what it means to generate grounded theory by being immersed in the empirical world.

The empirical social world consists of on-going group life and one has to get close to this life to know what is going on in it. The metaphor that I like is that of lifting the veils that obscure or hide what is going on. The task of scientific study is to lift the veils that cover the area of group life that one purposes to study. The veils are not lifted by substituting, in whatever degree, preformed images for first-hand knowledge. The veils are lifted by getting close to the area and by digging deep in it through careful study. Schemes of methodology that do not encourage or allow this betray the cardinal principle of respecting the nature of one’s empirical world. . . . The merit of naturalistic study is that it respects and stays close to the empirical domain. (Blumer, 1978, p. 38)

All of the approaches to theory and research in this chapter use qualitative methods to stay grounded in the empirical world. Yet they vary considerably in their conceptualizations of what is important to ask and consider in elucidating and understanding the empirical world. While the phrase “grounded theory” is often used as a general reference to inductive, qualitative analysis, as an identifiable approach to qualitative inquiry, it consists of quite specific methods and systematic procedures (Glaser, 2000, 2001). In their book on techniques and procedures for developing grounded theory, Strauss and Corbin (1998, p. 13) emphasized that analysis is the interplay between researchers and data; so what grounded theory offers as a framework is a set of “coding procedures” to “help provide some standardization and rigor” to the analytical process.

In all versions, grounded theory begins with very early close coding of collected data. The initial coding aims to ask what is happening in these data and invokes short analytic labels in the form of gerunds to identify specific processes and treat them theoretically. . . . When researchers define a set of tentative codes, they use these codes to compare, sort, and synthesize large amounts of data. Throughout the process, grounded theorists write memos elaborating their codes by identifying their properties, the conditions under which the code arises, and comparisons with specific data and their codes. Memo writing (a) engages researchers with their data and emerging comparative analyses, (b) helps them to identify analytic gaps, (c) provides material for sections of papers and chapters, and (d) encourages researchers to record and develop their ideas at each stage of the research project. By writing successively more analytic memos, researchers raise the theoretical level of their work. (Charmaz & Bryant, 2008, p. 375)

Grounded theory emphasizes steps and procedures for connecting induction and deduction through the constant comparative method, comparing research sites, doing theoretical sampling (Morse, 2010), and testing emergent concepts with additional fieldwork. Grounded theory is meant to “build theory rather than test theory.” It strives to “provide researchers with analytical tools for handling masses of raw data.” It seeks to help qualitative analysts “consider alternative meanings of phenomena.” It emphasizes being “systematic and creative simultaneously.” Finally, it elucidates “the concepts that are the building blocks of theory.” Glaser

(1993) and Strauss and Corbin (1997) have collected together in edited volumes a range of grounded theory exemplars that include several studies of health (life after heart attacks, emphysema, chronic renal failure, chronically ill men, tuberculosis, and Alzheimer’s disease), organizational head-hunting, abusive relationships, women alone in public places, selfhood in women, prison time, and characteristics of contemporary Japanese society.

While grounded theory has become widely thought of as an approach specific to qualitative inquiry, Glaser does not limit it in that way.

Let me be clear. Grounded theory is a general method. It can be used on any data or combination of data. It was developed partially by me with quantitative data. It is expensive and somewhat hard to obtain quantitative data, especially in comparison to qualitative data. Qualitative data are inexpensive to collect, very rich in meaning and observation, and very rewarding to collect and analyze. So, by default, due to ease and growing use, grounded theory is being linked to qualitative data and is seen as a qualitative method, using symbolic interaction, by many. Qualitative grounded theory accounts for the global spread of its use. (Glaser, 2000, p. 7)