U7D1-68 - **READ AND FOLLOW ALL INSTRUCTIONS AS OUTLINED BELOW.. Research Methodology Peer Review.

Sensitivity of the Inquiry: Potential for Controversy

The degree to which fieldwork is inquiring into matters that are sensitive or potentially controversial will affect how the inquiry is conducted. Occasionally, I have been asked to help prepare university students for cross-cultural experiences in which they will be doing fieldwork on some topic of interest. Most have become accustomed to Western media norms where no questions are off-limits. They propose research on politics, sexual practices, gender relationships, sexual orientation, family power dynamics, income distribution, and religion as if these were straightforward topics for open study anywhere in the world. Quite the contrary. Most people in most cultures in the world do not openly and easily discuss these matters with each other, much less with strangers. Such obliviousness is potentially insulting and sometimes dangerous. Ultimately, engaging in fieldwork requires building trusting relationships with the people in the setting being studied. Inquiry into sensitive and potentially controversial issues requires delicacy, time, and judgment. Key informants who understand the local context will be crucial to learning how to approach sensitive issues.

Dimensions Along Which Fieldwork Varies: An Overview

We’ve examined 10 dimensions that can be used to describe some of the primary variations in fieldwork (Modules 44–45). Those dimensions are summarized in Exhibit 6.4. Each is a continuum. These dimensions can be used to help design observational studies and make decisions about the parameters of fieldwork. They can also be used to organize the methods section of a report or dissertation in order to document how research or evaluation fieldwork actually unfolded.

EXHIBIT 6.4 Dimensions of Fieldwork: Variations and Options Along Continua

MODULE

47 What to Observe: Sensitizing Concepts

A SENSITIZING FRAMEWORK

I keep six honest serving men.

They taught me all I knew:

Their names are What and Why and When

And How and Where and Who. —Rudyard Kipling

NASA (National Aeronautics and Space Administration) space scientist David Morrison (1999) once commented that, in astronomy, geology, and planetary science, observation precedes theory generation “and the journals in these fields never require the authors to state a ‘hypothesis’ in order to publish their results” (p. 8).

A recent example is the famous Hubble Space Telescope Deep Field in which the telescope obtained a single exposure of many days duration of one small field in an unremarkable part of the sky. The objective was to see fainter and farther than ever before, and thus to find out what the universe was like early in its history. No hypothesis was required—just the unique opportunity to look where no one had ever looked before and see what nature herself had to tell us.

In many other sciences the culture demands that funding proposals and published papers be written in terms of formulating and testing a hypothesis. But I wonder if this is really the way the scientific process works, or is this just an artificial structure imposed for the sake of tradition. (p. 8)

Part of the value of open-ended naturalistic observations is the opportunity to see what there is to see without the blinders of hypotheses and other preconceptions. Pure observation. As Morrison put it so elegantly, just the unique opportunity to look where no one has ever looked before and see what the world has to show us.

That’s the most open and emergent approach to fieldwork. A different approach is to enter the field to test theory and, especially, causal hypotheses (George & Bennett, 2005; Gerring, 2007; King et al., 1994).

How to focus fieldwork, then, is a matter of differing perspectives. Certainly it’s not possible to observe everything. The human observer is not a movie camera, and even a movie camera has to be pointed in the right direction to capture what is happening. For both the human observer and the camera, there must be focus. In fieldwork, this focus is provided by the study design and the nature of the questions being asked (Chapter 5). Once in the field, however, the observer must somehow organize the complex stimuli experienced so that observing becomes and remains manageable.

Sensitizing Concepts to Guide Fieldwork

Experienced observers often use “sensitizing concepts” to orient fieldwork. Chapter 5 discussed sensitizing concepts as an approach to sampling, essentially selecting cases that are exemplars of the concept of interest. As noted in the purposeful sampling discussion, qualitative sociologist and symbolic interactionist Herbert Blumer (1954) advocated using sensitizing concepts as a guide to fieldwork with special attention to the words and meanings that are prevalent among the people being studied. More generally, “a sensitizing concept is a starting point in thinking about the class of data of which the social researcher has no definite idea and provides an initial guide to her research” (van den Hoonaard, 1997, p. 2). Sensitizing concepts in the social sciences include loosely defined notions like victim, stress, stigma, and learning organization, which can provide some initial direction to a study as a fieldworker inquires into how the concept is given meaning in a particular place or set of circumstances being studied (Schwandt, 2001).

Rudyard Kipling’s poem about his “six honest serving men,” quoted above, constitutes a fundamental and insightful sensitizing framework identifying the central elements of good description. In social science, “group process” is a general sensitizing concept as is the focus on “outcomes” in evaluation. Kinship, leadership, socialization, power, and similar notions are sensitizing in that they alert us to ways of organizing observations and making decisions about what to record. Qualitative methodologist Norman Denzin (1978a) has captured the essence of how sensitizing concepts guide fieldwork:

The observer moves from sensitizing concepts to the immediate world of social experience and permits that world to shape and modify his conceptual framework. In this way he moves continually between the realm of more general social theory and the worlds of native people. Such an approach recognizes that social phenomena, while displaying regularities, vary by time, space, and circumstance. The observer, then, looks for repeatable regularities. He uses ritual patterns of dress and body-spacing as indicators of self-image. He takes special languages, codes and dialects as indicators of group boundaries. He studies his subject’s prized social objects as indicators of prestige, dignity and esteem hierarchies. He studies moments of interrogation and derogation as indicators of socialization strategies. He attempts to enter his subject’s closed world of interaction so as to examine the character of private versus public acts and attitudes. (p. 9)

The notion of “sensitizing concepts” reminds us that observers do not enter the field with a completely blank slate. While the inductive nature of qualitative inquiry emphasizes the importance of being open to whatever one can learn, some way of organizing the complexity of experience is virtually a prerequisite for perception itself. Exhibit 6.5 presents examples of common sensitizing concepts for various areas on inquiry. These concepts constitute ways of breaking the complexities of planned human interventions into distinguishable, manageable, and observable elements; they are by no means exhaustive of sensitizing concept possibilities but rather illustrate oft-used ways of organizing an agenda for inquiry. These concepts serve to guide initial observations as the inquirer watches for incidents, interactions, and conversations that illuminate these sensitizing concepts in a particular setting.

Overused Sensitizing Concepts Can Become Desensitizing

An Example of a Fieldwork Observation: Capturing the Interplay and Tension Between Two Evaluation Sensitizing Concepts

The Paris Declaration on Development Aid (OECD, 2005) posits five principles aimed at changing the relationships between donor countries and recipient countries. An evaluation was commissioned to assess implementation and impact. The credibility of the evaluation would depend, in part, on evaluator independence, a central principle of external evaluation and thus a sensitizing concept for observing how the evaluation was conducted. On the other hand, the relevance and utility of the evaluation would depend on the involvement of key stakeholders. To that end, an international advisory group was assembled to have input into the evaluation. Stakeholder involvement, then, is another central principle of evaluation and also a sensitizing concept for observing how the evaluation was conducted. These two principles, evaluator independence and stakeholder involvement, can come into conflict. Too much stakeholder involvement can undermine evaluator independence, and vice versa.

These two concepts are relational in that evaluator independence can only be meaningful in relation to some kind of real or potential effort to interfere with, undermine, intrude on, or otherwise threaten independence. Likewise, stakeholder involvement is a matter of degree, varies hugely in type and amount, and only has meaning in relationship to the evaluation and evaluator. Howard Becker (1998) has asserted, convincingly, I think, that

all terms describing people are relational—that is, they only have meaning when they are considered as part of a system of terms. . . . That seems obvious enough. But it’s one of those obvious things that people acknowledge and then ignore. (pp. 132–133)

So let’s look at the relationship between stakeholder involvement and evaluator independence in the evaluation of the Paris Declaration as an example of using sensitizing concepts to guide fieldwork.

Observation

The international advisory group for the evaluation met with the core evaluation team for two days in Copenhagen in 2011 to review and provide comments on the draft final evaluation report shortly before it was due to be publically issued. (There had been two prior meetings of the advisory group at earlier stages of the evaluation.) A critical and revealing exchange arose near the end of the meeting when, near the end of two days of intense interaction, two advisory members pushed for one more round of review before the report was finalized. At this point, the pressure on the Core Evaluation Team to complete the report on time was quite intense. To write yet another draft and send it out for review would involve delay and threaten delivering the final report on time. The leader of the Core Evaluation Team expressed concern about such a delay and added, “The final evaluation report is our report. We have listened to your feedback and will incorporate it as much as we think is appropriate, but our independence requires that we, and we alone, produce the final report.”

The tone of the demand from the advisory members who wanted to see one more draft became challenging. One said, in a loud voice, with finger pointing at the evaluation leader, “You tell us that you’ll take our feedback into account, but now you tell us that we won’t get a chance to review a new draft and react to whether you have accurately heard and incorporated our feedback.”

Silence enveloped the room after this exchange; the tension was palpable. The chair of the meeting called for a short break. Negotiations ensued in the hallway among key actors in the process from both the evaluation and the advisory group sides. Arguments from both sides were vehemently repeated. Options were offered and rejected. But after 20 minutes, a resolution emerged. The Core Evaluation Team agreed to circulate one part of the report for final commentary

and feedback, the Conclusions and Recommendations section. This was the part of the report that would likely receive the greatest attention when it was publicly issued. However, instead of producing the revised section in three languages (English, French, and Spanish, as was the custom), the revised conclusion would only be available in English, and there would be only one week for responses. The evaluation team agreed to take the responses seriously but reiterated that the final report would be informed by feedback, especially about clarity and understandability, but the substantive conclusions would be those of the evaluation team. All agreed that this solution was pragmatic and reasonable and that it protected evaluator independence while ensuring stakeholder involvement. That is the process that was followed. That critical incident was an opportunity to observe directly the intersection of two sensitizing concepts that are central to evaluation theory and practice (Patton & Gornick, 2011b, pp. 42–43).

EXHIBIT 6.5 Examples of Sensitizing Concepts in Various Contexts

Sensitizing concepts are terms, phrases, labels, and constructs that invite inquiry into what they mean to people in the setting(s) being studied. Quantitative inquiry begins by operationalizing concepts (determining measurement). Qualitative inquiry using sensitizing concepts leaves terms purposefully undefined to find out what they mean to people in a setting. Sensitizing concepts are windows into a group’s worldview.

The inquiry questions are as follows: What does this concept mean in this context to these people? What are the variations in meaning and the implications of those variations?

Here is a note of caution about sensitizing concepts. When they become part of popular culture, they can lose much of their original meaning. Philip Tuwaletstiwa, a Hopi geographer, relates the story of a tourist cruising through Native American areas of the Southwest. He overheard

the tourist, “all agog at half-heard tales about Hopi land,” ask his wife, “Where are the power places?”

“Tell her that’s where we plug-in TV,” he said (quoted by Milius, 1998, p. 92).

SIDEBAR

WISDOM AS A SENSITIZING CONCEPT

In 1950, renowned psychoanalyst Erik Erikson conceptualized the phases of life, identifying wisdom as a likely, but not inevitable, by-product of aging. As an old man writing the 4th edition of this book, I find myself strangely resonating to this developmental theory. Wisdom becomes ascendant during the eighth and final stage of psychosocial development, a time of “ego integrity versus despair.” Ego integrity counters the potential despair of increasing infirmity and approaching death, yielding mellowness-inducing wisdom. Erikson, however, never operationalized wisdom, and a half-century later, psychologists still don’t agree on what it is or how to measure it (Hall, 2007). This comforts me. The meaning and manifestation of wisdom no doubt vary by context. I experience wisdom as a usefulism—a sensitizing concept—something to ponder, look for, inquire into, and dialogue about. I confess that I find the possibility of at least this one positive outcome of aging reassuring (MQP).

Sensitizing Concepts Versus Operationalized Concepts

All terms describing people are relational—that is, they only have meaning when they are considered as part of a system of terms.

—Becker (1998, p. 132)

Chapter 5, on designing qualitative studies, contrasted purposeful sampling with probability sampling, making the case that these serve fundamentally different purposes that involve different logics, essentially leading to different types of inference. Social scientists trained only in random probability sampling have trouble grasping the value of small, information-rich purposeful samples. A fundamental difference between quantitative and qualitative inquiry strategies centers on these contrasting sampling strategies.

An equally great divide concerns how to inquire into concepts. Quantitative inquiry requires converting concepts into operationalized (measureable) variables. Qualitative inquiry leaves concepts open for exploration and illumination in the field. These are fundamentally different ways of undertaking an inquiry. Because these differences are a major source of misunderstanding and conflict between quantitative and qualitative researchers, it is worth taking a moment to look more deeply at this divide and its implications for fieldwork.

Operationalization involves translating an abstract construct into concrete measures for the purpose of gathering data on the construct. Through operationalization, a concept becomes a

variable, and if the operationalization gains wide acceptance and use, the operational definition becomes the standard measurement of the concept, for example, intelligence tests to measure intelligence. This constitutes the dominant approach to science. The Encyclopedia of Social Science Research Methods (2004), in an entry on operationalization, affirms the scientific goal of standardizing definitions of key concepts. It notes that concepts vary in their degree of abstractness, using as an illustration the concepts human capital versus education versus number of years of schooling as moving from high abstraction to operationalization. The entry then observes,

Social science theories that are more abstract are usually viewed as being the most useful for advancing knowledge. However, as concepts become more abstract, reaching agreement on appropriate measurement strategies becomes more difficult. (Mueller, 2004, p. 162)

Note here that a certain degree of abstraction is useful for advancing knowledge and building theory. The Encyclopedia entry goes on to discuss the controversy surrounding the relationship between the concept of intelligence and the operationalization of intelligence through intelligence tests, including the classic critique that the splendidly abstract and sensitizing concept of intelligence has been reduced by psychometricians to what intelligence tests measure.

Operationalization as a value has been criticized because it reduces the concept to the operations used to measure it, what is sometimes called “raw empiricism.” As a consequence, few researchers define their concepts by how they are operationalized. Instead, nominal definitions are used . . . and measurement of the concepts is viewed as a distinct and different activity. Researchers realized that measures do not perfectly capture concepts, although . . . the goal is to obtain measures that validly and reliably capture the concepts. (Mueller, 2004, p. 162)

It appears that there is something of a conundrum here, some tension between social science theorizing and empirical research. A second entry in the Encyclopedia of Social Science Research Methods sheds more light on this issue.

Operationalism began life in the natural sciences . . . and is a variant of positivism. It specifies that scientific concepts must be linked to instrumental procedures in order to determine their values. . . . In the social sciences, operationalism enjoyed a brief spell of acclaim. . . . [O]perationalism remained fairly uncontroversial while the natural and social sciences were dominated by POSITIVISM but was an apparent casualty of the latter’s fall from grace. (Williams, 2004, pp. 768–769)

The entry elaborates three problems with operationalization. First, “underdetermination” is the problem of determining “if testable propositions fully operationalize a theory” (p. 769). Examples include concepts like homelessness, poverty, and alienation, which have variable meanings in different social contexts. What “homeless” means varies historically and sociologically. A second problem is that objective scholarly definitions may not capture the subjective definition of those who experience something. Poverty offers an example: What one person considers poverty another may view as a pretty decent life. The Northwest Area Foundation, which has as its mission poverty alleviation, has struggled trying to operationalize poverty for outcomes evaluation; moreover, their researchers found that many quite poor people in states like Iowa and Montana, who fit every official definition of being in poverty, do not even see themselves as poor, much less in poverty. Third is the problem of disagreements among social scientists about how to define and operationalize key concepts. The second and third problems are related in that one researcher may use a local and context-specific definition to solve the second problem but that context-specific definition is likely to be different from and in conflict with the definition used by other researchers inquiring in other contexts.

One way to address problems of operationalization is to treat a concept of interest as a sensitizing concept and abandon the search for a standardized and universal operational definition. This means that any specific empirical study of a concept would generate a definition that fit the specific context for and purpose of the study, but operational definitions would be expected to vary. In mixed-methods inquiries, understandings that emerge from qualitative inquiry into a sensitizing concept could lead to a context-specific measurement of the concept for quantitative inquiry.

Context as a Sensitizing Concept

Throughout this book, I have noted that sensitivity to context is a strength of qualitative inquiry. Any particular qualitative inquiry is designed within some context, and researchers are admonished to take context into account, be sensitive to context, and watch out for changes in context. But what is context? It turns out that it works pretty well as a sensitizing concept but defies operationalization. This was especially evident at the 2009 annual conference of the American Evaluation Association (AEA; Rog, 2009), which had as its theme “Context and Evaluation.” Systems thinkers posited that system boundaries are inherently arbitrary, so defining what is within the immediate scope of an evaluation versus what is within its surrounding context will inevitably be arbitrary, but the distinction is still useful. Indeed, being intentional about deciding what is in the immediate realm of action of an evaluation and what is in the enveloping context can be an illuminating exercise—and different stakeholders might well provide different perspectives. In that sense, the idea of context is a sensitizing concept. Those conference participants seeking an operational definition of context ranted in some frustration about the ambiguity, vagueness, and diverse meanings of what they, ultimately, decided was a useless and vacuous concept. Why? Because it had not been (and could not be) operationally defined—and they displayed a low tolerance for the ambiguity that is inherent in such sensitizing concepts.

A sensitizing concept raises consciousness about something and alerts us to watch out for it within a specific context as we undertake fieldwork. That’s what the concept of context does. It says things are happening to people and changes are taking place as people in families, programs, organizations, and communities interact with people and processes in the surrounding environment, in their context(s). Watch out for those interactions and their effects. Pay attention. Something important may be happening. Units of observation (again, families, programs, organizations, communities, etc.) are not closed systems. Observe what’s going on both within the setting of interest and across the boundaries of that setting.

The bottom line: Don’t judge the maturity and utility of the concept by whether it has “achieved” a standardized and universally accepted operational definition. Judge it instead by its utility in sensitizing us to how people experience the worlds in which they live and work. This means that specific qualitative studies of important concepts may lead to operational definitions as appropriate. Over time, many empirical studies may use the same or similar operational definitions. Periodically, syntheses and comparisons will be undertaken. We can learn a great deal from how different researchers define a concept, whether operationally (deductively and quantitatively), nominally (as a sensitizing concept), or inductively (exploring emergent meanings and manifestations). What I am arguing against is the notion that arriving at some standard operational definition is the desired target, some kind of “achievement” indicating maturity, consensus, shared understanding, and professional acceptance.

MODULE

48 Integrating What to Observe With How to Observe

The what and how of qualitative inquiry are closely linked. Sources of data are derived from inquiry questions. Knowing what we want to illuminate helps us determine sources of data for that illumination. But there are also some general areas of observation that provide principles for engaging in qualitative fieldwork. In the sections that follow, we’ll examine the following:

• Describing the context: the observation setting

• Attending to historical perspectives as part of fieldwork

• Observing the human, social environment

• Observing planned, formal interactions

• Observing informal, spontaneous interactions

• Capturing the language used and its meanings

• Making sense of nonverbal communication

• Observing unobtrusively

• Using documents as sources of data in fieldwork

• Observing what does not happen

• Observing oneself

Describing the Context: The Observation Setting

Pay attention. It’s all about paying attention. Attention is vitality. —Susan Sontag (1933–2004)

American writer, filmmaker, and activist

Describing a setting, like a program setting, begins with paying attention to the physical environment within which the program takes place. The description of the program setting should be sufficiently detailed to permit the reader to visualize that setting. In writing a program description, the observer, unlike the novelist, should avoid interpretive adjectives except as they appear in quotes from participants about their reactions to and perceptions of that environment. Adjectives such as “comfortable,” “beautiful,” “drab,” and “stimulating” interpret rather than describe—and interpret vaguely at that. More purely descriptive adjectives include

colors—“a room painted blue with a blackboard at one end”;

space—“a 40-foot by 20-foot classroom with windows on one side”; and

purpose—“a library, the walls lined with books and tables in the center.”

SIDEBAR

TO DESCRIBE WELL, MINIMIZE INTERPRETATION

Everyone knows that there is no “pure” description, that all description, requiring acts of selection and therefore reflecting a point of view, is what Thomas Kuhn said it was, “theory laden.” That it is not possible to do away entirely with the necessity of selection, and the point of view it implies, does not mean that there aren’t degrees of interpretation, that some descriptions can’t be less interpretive (or perhaps we should say less conventionally interpretive) than others. We might even say that some descriptions require less inference than others. To say that someone looks like he is hurrying home with his shopping requires an inference about motivation that saying that he is walking rapidly doesn’t.

—Howard S. Becker (1998, p. 79)

Tricks of the Trade

If you are just beginning to learn to do systematic observations, you can practice writing descriptively by sharing a description of a setting you’ve observed with a couple of people and asking them if they can visualize the setting described. Another helpful exercise involves two people observing the same environment and exchanging their descriptions, watching in particular for the use of interpretive adjectives instead of descriptive ones. Vivid description provides sufficient information so that the reader does not have to speculate about what is meant. For example, simply reporting “a crowded room” requires interpretation. Contrast with this:

The meeting room had a three-person couch across one side, six chairs along the adjoining walls next to the couch, and three chairs along the wall facing the couch, which included the door. With twenty people in the room, all standing, there was very little space between people. Several participants were overheard to say, “This room is really crowded.”

Such descriptive writing requires attention to detail and discipline to avoid vague, interpretive phrases. But such writing can also be dull. Metaphors and analogies can enliven and enrich descriptions, helping readers connect through shared understandings and giving them a better feel for the environment being described. I once evaluated a wilderness education program that included time at the Grand Canyon. Exhibit 6.6 presents my feeble attempt to capture in words our first view of the Grand Canyon. Notice the metaphors that run through the description. Of course, this is one of those instances where a picture would be worth a mountain of words, which is why qualitative fieldwork increasingly includes photography and videography (Gubrium & Harper, 2013; Marion & Crowder, 2013). This excerpt aims at offering a sense of the physical environment more than it offers a literal description because, unless one has been there or seen pictures, the landscape is outside ordinary experience.

The physical environment of a setting can be important to what happens in that environment. In observing a program, what the reception area is like, the way the walls look in rooms, the amount of space available, how the space is used, the nature of the lighting, how people are organized in the space, and the reactions of program participants to the physical setting can be important information about both program implementation and the effects of the program on participants.

In neighborhoods and communities, the physical environment provides an essential context for understanding life in that setting. Chapter 2 featured Rebecca Skloot’s case study of The Immortal

Life of Henrietta Lacks (2010) to illustrate qualitative strategies in practice. Here is her description of entering the town of Clover in southern Virginia in search of Henrietta’s family and burial place.

Downtown Clover started at a boarded-up gas station with RIP spray-painted across its front, and ended at an empty lot that once held the depot where Henrietta caught her train to Baltimore. The roof of the old movie theater on Main Street had caved in years ago, its screen landing flat in a field of weeds. The other businesses looked like someone left for lunch decades earlier and never bothered coming back: one wall of Abbott’s clothing store was lined with boxes of new Red Wing work boots stacked to the ceiling and covered in thick dust; inside its long glass counter, beneath an antique cash register, lay rows and rows of men’s dress shirts, still folded starch-stiff in their plastic. The lounge at Rosie’s restaurant was filled with overstuffed chairs, couches, and shag carpet, all in dust-covered browns, oranges, and yellows. A sign in the front window said OPEN 7 DAYS, just above one that said CLOSED. At Gregory and Martin Super Market, half-full shopping carts rested in the aisles next to decades-old canned foods, and the wall clock hadn’t moved past 6:34 since Martin closed up shop to become an undertaker sometime in the eighties. (pp. 77–78)

EXHIBIT 6.6 Integrating Description and Metaphor to Provide a Sense of Place: A Participant Observation Example

The excerpt that follows was aimed at providing context for a Wilderness Leadership Education Program. This was the group’s first view of the Grand Canyon from Bright Angel Point, where the program was being launched. I was serving as participant-observer- evaluator.

We followed an asphalt path from the lodge a quarter-mile to Bright Angel Point, perhaps the most popular tourist site at the Grand Canyon because of its relatively easy access. With cameras aimed in all directions at the spectacular panorama, in a sea of domestic accents and foreign tongues, we waited our turn at the edge to behold the magnificent rock temples of Ottoman Amphitheater: Deva, Brahma, Zoroaster, and, in the distance, Thor. Each rises a half-mile above the undulating grayness of the stark Tonto Platform defining the eight-mile descent of Bright Angel Canyon, a narrow slit hiding the inner gorge that looks like it had been drawn in black ink to outline the base of the temples. Each begins as sheer Redwall that forms a massive foundation supporting a series of sloping sedimentary rock terraces, the Supai. These sweeping terraces, spotted green with sparse desert vegetation, point upward like arrow feathers to a white sandstone pedestal, the Coconino. A dark red pinnacle of Hermit shale uniquely crowns each temple. Eons of erosion have sculpted dramatic variations in every aspect save one: their common geologic history. I studied each separately, wanting to fix in my mind the differences between them, but the shared symmetry of strata melded them into a single, massive formation, a half-mile high and many miles around. Behind me I heard a participant say softly to no one in particular, almost under her breath, “It’s too awesome. I feel overwhelmed.” (Adapted from Patton, 1999)

A common mistake among observers is to take the physical environment for granted. Thus, an evaluator may report that the program took place in “a school.” The evaluator may have a mental image of “school” that matches what was observed, but schools vary considerably in size, appearance, and neighborhood setting. Even more so, the interiors of schools vary considerably. The same can be said for criminal justice settings, health settings, community mental health programs, neighborhoods, and, indeed, many places where humans congregate.

During site visits to early-childhood education programs, we found a close association between the attractiveness of the facility (child-made decorations and colorful posters on the walls, well-

organized learning materials, orderly teacher area) and other program attributes (parent involvement, staff morale, clarity of the program’s goals, and theory of action). An attractive, well- ordered environment corresponded to an engaging, well-ordered program. In observing, as well as conducting workshops, I have noted how the arrangement of chairs affects participation. It is typically much easier to generate discussion with chairs in a circle than in lecture style. The dim lighting of many hotel conference rooms seems to palpably drain energy from people sitting in those rooms for long periods of time. Physical environments clearly affect people.

Variations in the settings for a wilderness training program for which I served as participant- observer provide an interesting example of how physical environments affect a program. The explicit purpose of holding the “field conferences” in the wilderness was to remove people from their everyday settings in largely urban environments surrounded by human-made buildings and the paraphernalia of modern industrial society. Yet wilderness environments are no more uniform than the environments of human service programs. During the yearlong program, participants were exposed to four different wilderness environments: (1) the autumn forest in the Gila Wilderness of New Mexico; (2) the rough terrain of Arizona’s Kofa Mountains in winter; (3) the muddy, flooding San Juan River in the canyon lands of Utah during spring; and (4) among the magnificent rock formations of the Grand Canyon in summer, a desert environment. One focus of the evaluation, then, was to observe how participants responded to the opportunities and constraints presented by these different environments: forest, mountains, canyon-lined river, and Grand Canyon desert.

In addition, weather and seasonal differences accentuated the variations among these environments. Program activities were clearly affected by the extent to which there was rain, cold, wind, and shelter. In the program’s theory, weather uncertainties were expected to be a natural part of the program, offering unpredictable challenges for the group to deal with. But the program theory also called for participants to engage deeply with each other during evening group discussions. During one 10-day winter “field conference” that was unusually cold and wet, participants were miserable, and it became increasingly difficult to carry on group discussions, thus reducing considerably the amount of group process time available and rushing the interactions that did occur because of participants’ discomfort. Program staff learned that they needed to anticipate more the possible variations in physical environments, plan for those variations, and include the participants in that planning so as to increase their commitment to continuing the process under difficult physical conditions.

History as Context

I never forget things. I just don’t observe them in the first place. —Bauvard

Avant-garde artist

Historical information can shed important light on a setting. The history of a place, program, community, or organization is an important part of the context for research and evaluation. Distinguished qualitative sociologist William Foote Whyte (1984), sometimes called the father of sociological field research, has reflected on how he came to value historical research as a critical part of his fieldwork.

When we began our Peruvian research program, I viewed history as having little value for understanding the current scene. I thought I was only being sympathetic to the interests of our Peruvian researchers in suggesting that they gather historical data on each village for the last 50 years.

Fortunately, the Peruvians refused to accept the 50-year limit and in some cases probed up to 500 years in the history of villages or areas. Much of these data on rural communities would be of interest only to historians. However, understanding the paradox of the Mantaro Valley required us to go back to the conquest of Peru, and, in the Chancay Valley, we traced the beginnings of the differentiation of Huayopampa from Pacaros back more than a century. (p. 153)

Documenting and understanding the context of a program as part of an evaluation will require delving into its history. How was the program created and initially funded? Who were the original people targeted for program services and how have target populations changed over time? To what extent and in what ways have goals and intended outcomes changed over time? What have staffing patterns been over time? How has the program’s governance (board) been involved at various stages in the program’s history? What crises has the program endured? If the program is embedded within a larger organizational context, what is the history of that organization in relation to the program? How has the larger political and economic environment changed over time, and how have those changes affected program development? What are the stories people tell about the program’s history? These kinds of questions frame inquiry into the program’s history to illuminate context.

Damiano is a community-based poverty alleviation program in Duluth, Minnesota. It began as a temporary soup kitchen providing emergency meals during an economic crisis in 1982. Over time, it added a variety of services and is still operating today. Understanding that story constitutes a “retrospective developmental evaluation” (Patton, 2011, pp. 297–303) without which observing current operations would be devoid of meaningful context.

In the 1990s, I evaluated a “free high school” that had been created during the struggles and turmoil of the 1960s. Little about the program’s current programming could be understood outside the context of its historical emergence. The school’s image of itself, its basic approach, including curriculum and policies, had been handed down and adapted from that intense period of early development. Doing fieldwork in the 1990s could only be done by traversing the memories and legends of the school’s historical emergence in the 1960s.

The Human, Social Environment

To acquire knowledge, one must study; but to acquire wisdom, one must observe. —Marilyn Vos Savant

American author, formerly listed in the Guinness Book of World Records under “Highest IQ”

Just as physical environments vary, so too do social environments. The ways in which human beings interact create social ecological constellations that affect how participants behave toward each other in those environments. Rudolf Moos (1975) pioneered the social ecological view of human environments:

The social climate perspective assumes that environments have unique “personalities,” just like people. Personality tests assess personality traits or needs and provide information about the characteristic ways in which people behave. Social environments can be similarly portrayed with a great deal of accuracy and detail. Some people are more supportive than others. Likewise, some social environments are more supportive than others. Some people feel a strong need to control others. Similarly, some social environments are extremely rigid, autocratic, and controlling. Order, clarity, and structure are important to many people. Correspondingly, many social environments strongly emphasize order, clarity and control. (p. 4)

In describing the social environment, the observer looks for the ways in which people organize themselves into groups and subgroups. Patterns and frequency of interactions, the direction of communication patterns (Who controls? Who follows? Who isn’t heard?), and changes in these patterns tell us things about the social environment. How people group together can be illuminative and important. All-male versus all-female groupings, male–female interactions, and interactions among people with different background characteristics, racial identities, and/or ages alert the observer to patterns in the social ecology of a setting.

Decision-making patterns can be a particularly important part of a social environment. Who makes decisions about what gets done? To what extent are decisions made openly, so that others are aware of the decision-making process? How are decisions communicated? Answers to these questions are an important part of the description of the power dynamics of a social setting. These issues are particularly important in program evaluation to understand and illuminate the program’s decision-making patterns and environment.

An observer’s descriptions of a social environment will not necessarily be the same as the perceptions of that environment expressed by participants. Nor is it likely that all participants will perceive the setting’s human climate in the same way. At all times, it is critical that the observer records participants’ comments in quotation marks, indicating the source—who said what?—so as to keep perceptions of participants separate from the observer’s or evaluator’s own descriptions and interpretations.

Formal Interactions and Planned Activities

Never trust to general impressions, my boy, but concentrate yourself upon details. —Sir Arthur Conan Doyle (1859–1930)

The Adventures of Sherlock Holmes

In this section and the next, I distinguish observing formal, planned activities and occasions from informal, serendipitous interactions and events. Early fieldwork observations tend to focus on what is known to be occurring in advance. For example, program evaluations focus at least some significant observations on planned program activities. What goes on in the program? What do participants and staff do? What is it like to be a participant? These are the kinds of questions evaluators bring to the program setting to document program implementation. Likewise, in organizations and communities, there will be formal, planned activities that are important to see and understand. For example, in studying organizations, I find it valuable to observe staff meetings and orientation sessions for new staff; if a community is the focus, I’d want to observe any formal

neighborhood council meetings and community celebrations like Cinco de Mayo in a Hispanic neighborhood or Carnival in a Caribbean community.

Formal, planned activities have a kind of unity about them: a beginning, some middle point, and a closure point—things like a class session, a counseling session, mealtime in a residential facility, a meeting of some kind, a home visit in an outreach program, a consultation, or a registration procedure. Attending to such sequences illustrates how fieldwork progresses over the course of an observation. Initially, the observer will focus on how the activity is introduced or begun. Who is present at the beginning? What exactly was said? How did participants respond or react to what was said?

These kinds of basic descriptive questions guide the evaluator throughout the full sequence of observation. Who is involved? What is being done and said by staff and participants? How do they go about what they do? Where do activities occur? When do things happen? What are the variations in how participants engage in planned activities? How does it feel to be engaged in this activity? (The observer records his or her own feelings as part of the observation.) How do behaviors and feelings change over the course of the activity?

Finally, the observer looks for closure points. What are the signals that a particular activity is being ended? Who is present at that time? What is said? How do participants react to the ending of the activity? How is the completion of this unit of activity related to other program activities and future plans?

Each unit of activity is observed and treated as a self-contained event for the purpose of managing field notes. The observation of a single session of the Early Childhood Parent Education program presented in Chapter 1 is an example (pp. 29–31). Each observed event or activity can be thought of as a mini case study of a discrete incident, activity, interaction, or event. Choosing what to observe involves sampling decisions, and the options for sampling within a case are those discussed at length in Chapter 5. You can observe typical events, diverse interactions, or outlier occasions. Sensitizing concepts can guide what you select to observe. For example, if you are interested in power as a sensitizing concept, you will choose to focus observations on events and interactions that are organized by those with power but include those with less power. What is the nature of interactions on such occasions, for example, a political event or a ribbon-cutting ceremony with people in power on display?

Distinguish Cases From Observational Units Within Cases: Choosing What to Observe

Distinguish cases from observations. “Cases” are understood as the broader units, that is, the broader research settings or sites within which analysis is conducted; “observations” are pieces of data, drawn from those research sites, that form the direct basis for descriptive and causal inference. (Collier, Seawright, & Munck, 2010, p. 51)

During analysis, one looks across discrete observations for patterns and themes, but during the initial stages of fieldwork, the observer will be kept busy just trying to capture self-contained units of activity (observational units) without worrying yet about looking for patterns across activities.

How do you decide what to observe? Observe those things that relate most directly to the focus of your inquiry. This is called maximizing leverage, where you maximize your fieldwork time and resources to get the most relevant data to your inquiry. At the same time, you’re watching for opportunities to expand your understanding of your inquiry by being open to observations that

might at first appear tangential but, in fact, take you in an important, emergent direction. So you watch for observations that have similar characteristics (unit homogeneity) and those that are diverse (unit heterogeneity). Part of the cross-unit analysis will involve examining “unit homogeneity” versus “unit heterogeneity” in comparing observational units like discrete events, formal interactions, and planned activities (George & Bennett, 2005, pp. 171–172).

Observing and documenting formal activities will constitute a central element in fieldwork. In evaluation, the formal, planned activities are the core of evaluating planned program implementation (evaluating the “model”), but to fully understand a program and its effects on participants, observations should not be restricted to formal, planned activities. The next section discusses observation of the things that go on between and around formal, planned program activities.

SIDEBAR

REALITY CHECK APPROACH: “LIGHT TOUCH” PARTICIPANT OBSERVATION

The Reality Check Approach (RCA) seeks to understand the views, perspectives, and opinions of people living in poverty by spending several nights living with poor families and engaging in informal conversations with them, their neighbors, and local service providers. It may be viewed as a “light touch” participant observation. The RCA is considered to be a useful contribution to the overall study in that it provides contextual and candid information directly from people regarding their particular experience of change. (GRM International, 2013, p. 6)

For an example of RCA in practice, see the study of the long-term impact of development interventions in the Koshi Hills of Nepal.

The main focus of the study was to establish how people experienced change over the past 30+ years, what changes they considered were significant, and what their ideas were regarding the drivers of these changes. To this end, the host households included in the study were purposely selected to comprise different generations so that discussions and debate could be facilitated across generations.

The RCA was a small-scale study involving twenty seven host families, their neighbours, and local service providers (in total about 600 people). (GRM International, 2013, p. 6)

Informal Interactions and Unplanned Activities

I am turned into a sort of machine for observing . . . —Charles Darwin (1809–1882)

Naturalist and pioneer in the study of evolution

If you stop seeing and observing as soon as a planned, formal activity ends, you will miss a great deal of data. For example, most programs build in free or unstructured time between activities, with

the clear recognition that such periods provide opportunities for participants to assimilate what has occurred during formal programmatic activities as well as to provide participants with necessary breathing space. Rarely, if ever, can a program or institution plan every moment of participants’ time.

During periods of informal interaction and unplanned activity, it can be particularly difficult to organize observations because people are likely to be milling around, coming and going, moving in and out of small groups, with some sitting alone, some writing, some seeking refreshments, and otherwise engaging in a full range of what may appear to be random behaviors. How, then, can the observer collect data during such a time?

This scenario illustrates beautifully the importance of staying open to the data and doing opportunity sampling. One can’t anticipate all the things that might emerge during unplanned program time, so the observer watches, listens, and looks for opportunities to deepen observations, recording what people do, the nature of informal interactions (e.g., what subgroups are in evidence) and, in particular, what people are saying to each other. This last point is particularly important. During periods of unplanned activity, participants have the greatest opportunity to exchange views and to talk with each other about what they are experiencing in the program. In some cases, the evaluator will simply listen in on conversations, or there may be opportunities to conduct informal interviews, either with a single participant in natural conversation or with some small group of people, asking normal, conversational questions:

“So what did you think of what went on this morning?”

“What did you think of the session today?”

“How do you think what went on today fits with the rest of what you’ve been experiencing?”

Such questioning should be done in an easy, conversational manner so as not to be intrusive or so predictable that every time someone sees you coming, they know what questions you’re going to ask. “Get ready, here comes the evaluator with another endless set of questions.” Also, when doing informal, conversational interviewing, be sure that you are acting in accordance with ethical guidelines regarding informed consent and confidentiality. (See the earlier discussion in this chapter about overt vs. covert fieldwork.)

How something is said should be recorded along with what is said. At a morning break in the second day of a two-day workshop, I joined the other men in the restroom. As the men lined up to use the facilities, the first man to urinate said loudly, “Here’s what I think of this program.” As each man finished, he turned to the man behind him and said, “Your turn to piss on the program.” This spontaneous group reaction spoke volumes more than answers to formal interview questions and provided much greater depth of expression than checking “Very dissatisfied” on an evaluation questionnaire.

Everything that goes on in or around the program or other setting being observed is data. The fact that none of the participants talk about a session when it is over is data. The fact that people immediately split in different directions when a session is over is data. The fact that people talk about personal interests and share gossip that has nothing to do with the program is data. In many programs, the most significant participant learning occurs during unstructured time as a result of interactions with other participants. To capture a holistic view of the program, the evaluator- observer must stay alert to what happens during these informal periods. While others are “on break,” the observer is still working. No breaks for the dedicated fieldworker! Well, not really. You’ve got to pace yourself and take care of yourself, or your observations will deteriorate into

mush. Hopefully, you get the idea. You may be better off taking a break during part of a formal session, so you can work (collect data) while others are on break.

As happens in many programs, the participants in the wilderness education project I was observing/evaluating began asking for more free, unstructured time. When we weren’t hiking or doing camp chores, a lot of time was spent in formal discussions and group activities. Participants wanted more free time to journal. Some simply wanted more time to reflect. Most of all, they wanted more time for informal interactions with other participants. I respected the privacy of one- to-one interactions when I observed them and would never attempt to eavesdrop. I would, however, watch for such interactions, and judging body language and facial expressions, I would speculate when serious interpersonal exchanges were taking place. I would then look for natural opportunities to engage each of those participants in conversational interviews, telling them I had noticed the intensity of their interaction and inquiring whether they were willing to share what had happened and what significance they attached to the interaction. Most of the participants appreciated my role in documenting the program’s unfolding and its effects on participants and were open to sharing. It was on the basis of those informal interviews and observations that I provided formative feedback to staff about the importance of free time and helped alleviate the feeling among some staff that they had responsibility to plan and account for every moment during the program.

Participant observation necessarily combines observing and informal interviewing. Observers need to be disciplined about not assuming that they know the meaning to participants of what they observe without checking with those participants. During one period of unstructured time in the wilderness program, following a fairly intensive group activity in which a great deal of interpersonal sharing had taken place, I decided to pay particular attention to one of the older men in the group who had resisted involvement. Throughout the week, he had taken every available opportunity to make it known that he was unimpressed with the program and its potential for having an impact on him. When the session ended, he immediately walked over to his backpack, pulled out his writing materials, and went off to a quiet spot where he could write. He continued writing, completely absorbed, until dinnertime an hour later. No one interrupted him. With legs folded, notebook in lap, and his head and shoulders bent over the notebook, he gave off clear signals that he was involved, concentrating, and working on something to which he was giving a great deal of effort.

I suspected as I watched that he was venting his rage and dissatisfaction with the program. I tried to figure out how I might read what he had written. I was so intrigued that I momentarily even considered covert means of getting my hands on his notebook, but I quickly dismissed such unethical invasion of his privacy. Instead, I looked for a natural opportunity to initiate a conversation about his writing. During the evening meal around the campfire, I moved over next to him, made some small talk about the weather, and then began the following conversation:

“You know, in documenting experiences people are having, I’m trying to track some of the different things folks are doing. The staff has encouraged people to keep journals and do writing, and I noticed that you were writing fairly intensely before dinner. If you’re willing to share, it would be helpful for me to know how you see the writing fitting into your whole experience with the program.”

He hesitated, moved his food about in his bowl a little bit, and then said, “I’m not sure about the program or how it fits in or any of that, but I will tell you what I was writing. I was writing . . . ,” and he hesitated because his voice cracked, “a letter to my teenage son trying to tell him how I feel about him and make contact with him about some things. I don’t know if I’ll give the letter to him. The letter may have been more for me than for him. But the most important thing that’s been

happening for me during this week is the time to think about my family and how important it is to me and I haven’t been having a very good relationship with my son. In fact, it’s been pretty shitty, and so I wrote him a letter. That’s all.”

This short conversation revealed a very different side of this man and an important impact of the program on his personal and family life. We had several more conversations along these lines, and he agreed to be a case example of the family impacts of the program. Until that time, impacts on family had not even been among the expected or intended outcomes of the program. It turned out to be a major area of impact for a number of participants.

Capturing People’s Language and Meanings

The lunatic, the lover, and the poet

Are of imagination all compact.

One sees more devils than vast hell can hold;

That is, the madman. The lover, all as frantic,

Sees Helen’s beauty in a brow of Egypt.

The poet’s eye, in a fine frenzy rolling,

Doth glance from heaven to earth, from earth to heaven;

And as imagination bodies forth the forms of things unknown,

The poet’s pen turns them to shapes

And gives to airy nothing

A local habitation and a name. —William Shakespeare

A Midsummer Night’s Dream, Act V, Scene 1

As noted in Chapter 2, the Whorf hypothesis alerts us to the power of language to shape our perceptions and experiences. It’s worth repeating the story of Benjamin Whorf’s discovery. As an insurance investigator, he was assigned to look into explosions in warehouses. He discovered that truck drivers were entering “empty” warehouses smoking cigarettes and cigars. The warehouses often contained invisible but highly flammable gases. He interviewed truckers and found that they associated the word “empty” with “harmless” and acted accordingly. Whorf’s job, in Shakespeare’s terms, was to turn the perception of “airy nothing” into the shape of possible danger.

An anthropological axiom insists that one cannot understand another culture without understanding the language of the people in that culture. Language organizes our world for us by shaping what we see, perceive, and pay attention to. The things for which people have special

words tell others what is important to that culture. Thus, as students learn in introductory anthropology, indigenous Arctic peoples have many words for snow and desert-dwelling peoples may have many words for camel. Likewise, the artist has many words for red and to make space for the other changes kinds of brushes.

Roderick Nash (1986), in his classic study Wilderness and the American Mind, traces how changing European American perceptions of “wilderness” have affected at the deepest levels our cultural, economic, and political perspectives on deserts, forests, canyons, and rivers. He traced the very idea of wilderness to the eighth-century epic hero Beowulf, whose bravery was defined by his courage in entering the “wildeor”—a place of wild and dangerous beasts, dark and foreboding forests, and untamed, primordial spirits. In the Judeo-Christian tradition, wilderness came to connote a place of uncontrolled evil that needed to be tamed and civilized, while Eastern cultures and religions fostered love of wilderness rather than fear. Nash credits the Enlightenment with offering new ways of thinking about wilderness—and new language to shape that changed thinking.

Moving from the wilderness to the interior territory of communities, organizations, families, agencies, and programs, language still shapes experience and is therefore an important focus during fieldwork. People develop their own language to describe their concerns and the problems they deal with in their work. Educators who work with learning-disabled students have a complex system of language to distinguish different degrees and types of special needs and developmental disabilities, a language that changes as cultural and political sensitivities change. People in criminal justice generate language for distinguishing types of offenders or “perps” (perpetrators). Fieldwork involves learning the “native language” of the setting or program being studied and attending to variations in connotations and situational use. The field notes and reports of the observer should include the exact language used by participants to communicate the flavor and meaning of “native” language.

SIDEBAR

LANGUAGE INTENSITY IN THE SUPREME COURT

In decisions of the U.S. Supreme Court, justices writing dissents (on the losing side of the case) use more strident and intense language to express their disagreement than do those in the majority. Words like “clearly,” “absolutely,” and “undoubtedly” are used more often in dissents. These language “intensifiers” are accompanied by longer words.

Legal readers do not like intensifiers, long sentences, and long words. Nevertheless, when the legal reader becomes the legal writer and feels threatened with losing an appeal, or being on the dissenting side of a judicial opinion, the threatened legal writer will subconsciously resort to using more intensifiers, and maybe longer words and longer sentences, in an irrational attempt to “attack” the winning side, or to defend the losing argument. This response is explained by the . . . theory of argumentative threat . . . ; alienated from the majority, a Supreme Court Justice subconsciously (and irrationally) resorts to the universally censured intensifier in an attempt to bolster the losing argument. The theory of argumentative threat is consistent with social psychology theories suggesting that language use changes in response to a perceived threat. (Long & Christensen, 2013, pp. 958–959)

Language was especially important in the wilderness education project I evaluated. These were highly verbal people, well educated, reflective, and articulate, who spent a lot of program time in group discussions. Program staff understood how words can shape experiences. They wanted participants to view the time in the wilderness as a professional development learning experience, not a vacation, so staff called each week in the wilderness a “field conference”—hoping participants would see the program as a “conference” held in the “field.” Despite the determined efforts of staff, however, the participants never adopted this language. Almost universally, they referred to the weeks in the wilderness as “trips.” During the second “field conference,” the staff capitulated. Interestingly enough, that capitulation coincided with negative reactions by participants to some logistical inadequacies, unsuccessful program activities, and bad weather, all of which undercut the “conference” emphasis. Staff language reflected that change.

Other language emerged that illuminated participants’ experiences. One of the participants expressed the hope of “detoxifying” in the wilderness. He viewed his return to his everyday world as “poisonous retoxification.” The group immediately adopted this language of “detoxification” and “retoxification” to refer to wilderness time versus ordinary urban civilization time, ultimately shortening the words to “detox” and “retox.” This language came to permeate the program’s culture.

The discussions in the wilderness often reflected the physical environment in which program activities took place. Participants became skilled at creating analogies and metaphors to contrast their urban work lives with their wilderness experiences. After backpacking all day, participants could be heard talking about learning how to “pace myself in my work” or “shifting the burdens of responsibilities that I carry so that the load is more evenly balanced” (a reference to the experience of adjusting the weight of the backpack). In the mountains, after rock climbing, participants referred to “the danger taking risks at work without support” (a reference to the belay system of climbing, where someone supports the climber with a safety rope below). One discussion focused on how to “find toeholds and handholds” to bring about change back home, “to get on top of the steep wall of resistance in my institution.” They even assigned numbers to degrees of back-home institutional resistance corresponding to the numbers used to describe the degree of difficulty of various rock climbs. On the river, participant language was filled with phrases like “going with the flow,” “learning to monitor professional development like you read and monitor the current,” and “trying to find my way out of the eddies of life.”

Because of the power of language to shape our perceptions and experiences, most participants wanted to know the names of the rock formations, winding canyons, and river rapids we encountered, while others, following Desert Solitaire author Eddy Abbey (1968), set for themselves the goal of suppressing the human tendency to personify natural forms. Thus began a sustained personal interest in how names in the wilderness shaped our experiences there (Patton & Patton, 2001).

When I took my son into the Grand Canyon for a “coming-of-age” initiation experience (Patton, 1999), he reacted to the problem of finding words for that awesome environment by making up words. For example, on seeing the Canyon for the very first time, he whispered, “Buedüden,” which became our way of describing things too beautiful and awesome for ordinary words.

SIDEBAR

JARGON

Don Miller, in The Book of Jargon, tells of beginning the book with the intent of attacking and making fun of the jargon in various professions. To his surprise, he came to respect it. Whether within families, communities, peer groups, or professional associations, specialized language can add nuance and precision to communications and support connections.

I began to feel that there was something to be learned from the jargons, and that in fact it was much more interesting to have these different intellectual dialects in existence than to live in a perfectly homogeneous linguistic universe. . . . It is not only the major professions and subcultures that have their own styles and vocabularies; many smaller social units do, as well. Often within a single family, between brothers and sisters, or between lovers or old friends there exists a special, secret language, unknown to the rest of the world, which carries an intimate set of meanings and associations. In my own experience I have found these to be among the most beautiful forms in which language can be used. (Miller, 1981, p. xiii)

Capturing the precise language of participants honors the “emic” tradition in anthropology: recording participants’ own understandings of their experiences. Observers must learn the language of participants in the setting or program they are observing to faithfully represent participants in their own terms and be true to their worldview.

Nonverbal Communications

Verbal and nonverbal activity is a unified whole, and theory and methodology should be organized or created to treat it as such.

—Kenneth L. Pike (1912–2000) Linguist and anthropologist

Social and behavioral scientists have reported at length the importance of both verbal and nonverbal communication in human groups. While recording the language of participants, the observer should also attend to nonverbal forms of communication. For example, in educational settings, nonverbal communications would include how students get the attention of or otherwise approach instructors, like waving their hands in the air. In group settings, a great deal of fidgeting and moving about may reveal things about attention and involvement. How participants dress, express affection, and sit together or apart are examples of nonverbal cues about social norms and patterns.

Again, the wilderness program provides informative examples. Hugging emerged as a nonverbal way of providing support at times of emotional distress or celebration, for example, a way to recognize when someone had overcome some particularly difficult challenge, like making it up across a ledge along a cliff face. But subgroups differed in the amount of and comfort with hugging, and different “field conferences” manifested different amounts of hugging. When the group felt disparate, separated, with people on their own “trips” and isolated from each other, little hugging occurred either in pairs or around the group campfire. When the depth of connection was deeper, shoulder-to-shoulder contact around the campfire was common and group singing was more likely. Over time, it became possible to read the tenor of the group by observing the amount and nature of physical contact participants were having with each other—and participants in groups

with a lot of hugging and connectedness reported noticeably greater personal change and deeper reflection.

In evaluating an international development project, I observed that the three host country nationals (“locals”) had developed a subtle set of hand signals and gestures that the American staff never noticed. In meetings, the host country nationals regularly communicated with each other and operated as a team using these nonverbal signals. Later, having gained their confidence, I asked the local staff about the gestures. They told me that the Americans had insisted that each person participate as an individual on an equal footing in staff meetings, and to support an atmosphere of openness, the Americans asked them not to use their own language during staff meetings. But the locals wanted to operate as a unit to counter the power of the Americans, so they developed subtle gestures to communicate with each other since they were denied use of their own language.

Parameswaran (2001) has described how she relied on reading nonverbal cues to tell how potential interviewees reacted to her subject matter—the study of young middle-class women in India who read Western romance novels. She had to depend on observing and reading body language to pick up hostility, disapproval, support, or openness because the verbal formalities of some interactions offered fewer cues than nonverbal reactions. Among the young women, giggles, winks, animated interactions, lowered eyes, direct gaze—these became cues on how the fieldwork was progressing.

A caution is in order here. Nonverbal behaviors are easily misinterpreted, especially cross- culturally. Therefore, whenever possible and appropriate, having observed what appear to be significant nonverbal behaviors, some effort should be made to follow up with those involved to find out directly from them what the nonverbals really meant. I confirmed with other participants in the wilderness project the importance of hugging as a mechanism that they themselves used to sense the tenor of the group.

MODULE

49 Unobtrusive Observations and Indicators; and Documents and Archival Fieldwork

Being there while not being there, that’s really being there. —Halcolm

Being observed can make people self-conscious and generate anxiety, especially when the observations are part of an evaluation. Regardless of how sensitively observations are made, the possibility always exists that people will behave differently under conditions where an observation or evaluation is taking place than they would if the observer were not present.

Even when he is well-intentioned and cooperative, the research subject’s knowledge that he is participating in a scholarly search may confound the investigator’s data. . . . It is important to note early that the awareness of testing need not, by itself, contaminate responses. It is a question of probabilities, but the probability of bias is high in any study in which a respondent is aware of his subject status. (Webb, Campbell, Schwartz, & Sechrest, 1966, p. 13)

Concern about reactions to being observed has led some social scientists to recommend covert observations as discussed earlier in this chapter. An alternative strategy involves searching for opportunities to collect “unobtrusive measures” (Webb et al., 1966) or, qualitatively, to make unobtrusive observations. Unobtrusive measures and observations are those made without the knowledge of the people being observed and without affecting what is observed.

Robert Wolf and Barbara Tymitz (1978) included unobtrusive measures in their naturalistic inquiry evaluation of the National Museum of Natural History at the Smithsonian Institution, a study that has become a classic. They looked for “wear spots” as indicators of use of particular exhibit areas. They decided that worn rugs would indicate the popularity of particular areas in the museum. The creative observer can learn a number of things about patterns of behavior by looking for physical clues. Dusty equipment or files may indicate things that are not used. Areas that are used a great deal by children in a school will look different—that is, more worn—than areas that are little used.

In a weeklong staff training program for 300 people, I asked the kitchen to systematically record how much coffee was consumed in the morning, afternoon, and evening each day. Those sessions that I judged to be particularly boring had a correspondingly higher level of coffee consumption. Active and involving sessions showed less coffee consumption, regardless of time of day. (Participants could get up and get coffee whenever they wanted.)

In the wilderness program, the thickness of notebooks called “learning logs” became an unobtrusive indicator of how engaged participants were in self-reflective journaling. All participants were provided with “learning logs” at the beginning of the first “field conference” and encouraged to use them for private reflections and journaling. These three-ring binders contained almost no paper when first given to participants. Participants brought “learning logs” back each time they returned to the wilderness. (The program involved four different 10-day trips over the course of a year.) Despite differences in handwriting and spacing, the extent to which paper had

been added to the notebooks was one rough indicator of the extent to which the logs were being used. I subsequently interviewed each participant about his or her use of the notebooks.

The personnel of the National Forest Service and the Bureau of Land Management have a kind of unobtrusive measure they use in “evaluating” the wilderness habits of groups that go through an area such as the San Juan River in Utah. The canyons along the San Juan River are a very fragile environment. The regulations for use of that land state the following: “Take only photographs, leave only footprints.” This means that all garbage, including human waste and feces, are to be carried out. It takes several days to go down the river. By observing the amount and types of garbage groups carry out, park rangers could learn a great deal about the wilderness habits of various groups and their compliance with river regulations.

The creative observer, aware of the variety of things to be learned from studying physical and social settings, will look for opportunities to incorporate unobtrusive measures into fieldwork, thereby manifesting “sympathy toward multi-method inquiry, triangulation, playfulness in data collection, outcroppings as measures, and alternatives to self report” (Webb & Weick, 1983, p. 210). An example of such creative fieldwork is Jason De León’s Undocumented Migrant Project, which combines ethnographic research (collecting oral histories) and unobtrusive archeological research: collecting desert garbage left by illegal immigrants entering the United States from Mexico. He treats the garbage as cultural artifacts worthy of study in what he calls the “archaeology of undocumented migration”: viewing illegal immigration through desert debris (Wagner, 2011).

A particularly powerful example of unobtrusive fieldwork is Laura Palmer’s (1988) study of letters and remembrances left at the Vietnam Veterans Memorial in Washington, D.C., a work she called Shrapnel in the Heart. For the unobtrusive part of her fieldwork, Palmer sampled items left at the memorial, all of which are saved and warehoused by the U.S. government. She categorized and analyzed types of items and the content of messages. In some cases, because of identifying information contained in letters or included with objects (photographs, baby shoes, artwork) she was able, through intensive investigative work, to locate the people who left the materials and interview them. Their stories, the intrusive part of her study, combined with vivid descriptions of the objects that led her to them, offer dramatic and powerful insights into the effects of the Vietnam War on the lives of survivors. In one sense, her analysis of letters, journals, photos, and messages can be thought of as a nontraditional and creative form of document analysis, another important fieldwork strategy.

Documents and Documentation

If it isn’t documented, it didn’t happen. —Sohail Sangi (2009)

Fieldwork creates documentation. Systematic documentation of what is observed, heard, and experienced is what fieldwork is all about. Fieldwork also involves finding documents and documentation. Records, documents, artifacts, and archives, what has traditionally been called “material culture” in anthropology, constitute a particularly rich source of information about many organizations and programs. In contemporary society, all kinds of entities leave a trail of paper and

artifacts, a kind of spoor that can be mined as part of fieldwork. Families keep photographs, children’s schoolwork, letters, old Bibles with detailed genealogies, bronzed baby shoes, and other sentimental objects that can inform and enrich family case studies. People who commit suicide leave behind suicide notes that can reveal patterns of despair in a society (Wilkinson, 1999). Gangs and others inscribe public places with graffiti. Organizations of all kinds produce mountains of records, both public and private, on paper, digitally, and online. Indeed, an oft-intriguing form of analysis involves comparing official statements found in public documents (brochures, board minutes, annual reports) with private memos and what the observer actually hears or sees occurring in direct observation.

SIDEBAR

ARCHIVAL INQUIRY INTO TIMING

University of Minnesota business professor Stuart Albert (2013) studies how timing affects results. He describes himself as an archival hunter-gatherer. He collected case examples from The New York Times, The Wall Street Journal, and The Economist magazine by reading from cover to cover for more than 20 years and clipping every article that had anything to do with timing. That yielded more than 2,000 articles. In essence, timing was his sensitizing concept.

He sought examples of mistakes that could have been prevented based on what could have been known at the time. His qualitative analysis then involved looking for underlying temporal elements and structures. He found six elements that are part of every situation people face in making decisions: (1) sequences—what follows what; (2) punctuation—where things begin, end, stop, and start; (3) intervals—how much time there is between events; (4) rate—how fast things are going; (5) the shape or rhythm by which things unfold; and (6) polyphony—a lot of things happening at the same time and how they interact. He used these six dimensions to generate a seven-step process for analyzing why things happen when they do that provides guidance on when to effectively act.

No interviews. No fieldwork. Just immersion in his 20-year archive of news clippings.

Client files in programs are another rich source of case data to supplement field observations and interviews. At the very beginning of fieldwork, access to potentially important documents and records should be negotiated. The ideal situation would include access to all routine records on clients, all correspondence from and to program staff, financial and budget records, and organizational rules, regulations, memoranda, charts, and any other official or unofficial documents generated by or for the program. These kinds of documents provide the inquirer with information about many things that cannot be observed. They may reveal things that have taken place before the study began. They may include private interchanges to which the inquirer would not otherwise be privy. They can reveal aspirations, arrangements, tensions, relationships, and decisions that might be otherwise unknown through direct observation.

In evaluating the mission fulfillment of a major philanthropic foundation, I examined 10 years of annual reports. Each report was professionally designed, elegantly printed, and widely disseminated—and each report stated a slightly different mission for the foundation. It turned out that the president of the foundation wrote an annual introduction and simply stated the mission

from memory. The publication designer routinely lifted this “mission statement” from the president’s letter and highlighted it in bold font at the beginning of the report, often on the cover page. From year to year, the focus changed until, over the course of 10 years, the stated mission had changed dramatically without official board action, approval, or even awareness. Further investigation through years of board minutes revealed that, in fact, the board had never adopted a mission statement at all, a matter of considerable surprise to all involved.

As this example shows, documents prove valuable not only because of what can be learned directly from them but also as a stimulus for paths of inquiry that can only be pursued through direct observation and interviewing. As with all information to which an inquirer has access during observations, the confidentiality of program records, particularly client records, must be respected. The extent to which actual references to and quotations from program records and documents are included in a final report depends on whether the documents are considered part of the public record and therefore able to be publicized without breach of confidentiality. In some cases, with permission and proper safeguards to protect confidentiality, some information from private documents can be quoted directly and cited.

Program Records

Program records can provide a behind-the-scenes look at program processes and how they came into being. In the wilderness program evaluation, program staff made their files available to me. I discovered a great deal of information not available to other program participants: letters detailing both conceptual and financial debates between the technical staff (who led the wilderness trips) and the project directors (who had responsibility for the overall management of the program). Without knowledge of those arguments, it would have been impossible to fully understand the nature of the interactions between field staff and executive staff in the project. Disagreements about program finances constituted but one arena of communication difficulties during the program, including time in the wilderness. Interviews with those involved revealed quite different perceptions of the nature of the conflicts, their intensity, and their potential for resolution. While participants became aware of some arguments among staff, for the most part, they were unaware of the origins of those conflicts and the extent to which program implementation was hampered by them.

SIDEBAR

ARCHIVAL FIELDWORK: AN AFRICAN EXAMPLE

The progressive government of Tanzania under President Julius Nyerere (1961–1985) sought to settle the Wagogo people, a nomadic herding tribe in the Dodoma Region of Central Tanzania. In the late 1960s, conflicts between nomadic herders and settled agricultural farmers were increasing as both farms and herds grew larger. The former British colonial administration had tried unsuccessfully to settle the Wagogo. Before launching a new settlement scheme, the Tanzanian Ministry of Agriculture commissioned a study of the extensive colonial archives to inventory and assess what prior attempts had been tried. The Tanzanian government wanted to distinguish any new scheme from former colonial policies as well as learn from past failures. I spent three months going through years of detailed (and dusty) colonial reports to create an inventory of 35 different settlement schemes that had been tried, including a variety of financial incentives; harsh penalties; confiscation of cattle; disruption of herding routes; relocation of water sources, fences, and armed guards; and removing children to residential schools. It was

clear that the Tanzanian government would need a great deal of creativity, collaboration, and trust building to come up with a scheme that was not tainted by colonialism (Patton, 1970).

My review of files also revealed the enormous complexity of the logistics for the wilderness education program. Participants (college deans, program directors, administrators) were picked up at the airport in vans and driven to the wilderness location where the field conference would take place. Participants were supplied with all the gear necessary for surviving in the wilderness. Prior to each field trip, staff had many telephone and written exchanges with individual participants about particular needs and fears. Letters from participants, especially those new to the wilderness, showed how little they understood about what they were getting into. One seasoned administrator and hard-core smoker inquired, with reference to the first 10-day hike in the heart of the Gila Wilderness, “Will there be a place to buy cigarettes along the way?” Talk about being clueless! But by the end of the year of field trips, he had given up smoking. His letter of inquiry alerted me to the importance of this pre–post observation.

Without having looked over this correspondence, I would have missed the extent to which preparation for the one-week experiences in the wilderness consumed the time and energy of program staff. The intensity of work involved before the field conferences helped explain the behavior of staff once the field trips got under way. So much had gone into the preparations, virtually none of which was appreciated by or known to program participants, that program staff would sometimes experience a psychological letdown effect and have difficulty energizing themselves for the actual wilderness experience.

Learning to use, study, and understand documents and files is part of the repertoire of skills needed for qualitative inquiry. Exhibit 6.7 provides an inventory of documentation that can deepen fieldwork and qualitative analysis. For an extended discussion of “the interpretation of documents and material culture,” see Hodder (2000). For an exemplar of in-depth, carefully triangulated document analysis, you can’t do much better than the examination of the legend of President Abraham Lincoln’s last photograph before his assassination in 1865. Errol Morris (2013) examines multiple documents and photographs to separate the legend from the reality. It’s a fascinating story, readily available online and illustrating what qualitative documentary and investigative scholarship involve.

EXHIBIT 6.7 Types of Documentation and Artifacts to Support Qualitative Inquiry: A Suggestive Inventory

Observing What Does Not Happen

You can observe a lot by just watching. —Yogi Berra

Baseball player and pundit

Watch what’s there. And watch what’s not there. —Halcolm’s Yin and Yang of Observation

The preceding sections have described the things one can observe in a setting or program. Observing context, activities, interactions, what people do, and what they say is important in a comprehensive approach to fieldwork. But what about observing what does not happen?

The potential absurdity of speculating about what does not occur is illustrated by a Sufi story. During a plague of locusts, the wise fool Mulla Nasrudin, always looking on the bright side, went from village to village encouraging people by observing how fortunate they were that elephants had no wings. “You people don’t realize how lucky you are. Imagine what life would be like with elephants flying overhead. These locusts are nothing.”

To observe that elephants have no wings is indeed data. Moreover, elephants have no fins, claws, feathers, or branches. Clearly, once one ventures into the area of observing what does not happen, there are a near-infinite number of things one could point out. The “absence of occurrence” list could become huge. It is therefore with some caution that I include among the tasks of the observer that of noting what does not occur.

If social science theory, program goals, implementation designs, and/or proposals suggest that certain things ought to happen or are expected to happen, then it is appropriate for the observer or evaluator to note that those things did not happen. If a community where water is scarce shows no evidence of conflict over water rights, an anthropologist could be expected to report and explain this absence of expected tension. If a school program is supposed to, according to its funding mandate and goals, provide children with opportunities to explore the community but no such explorations occur, it is altogether appropriate for the evaluator to note the said implementation failure. If the evaluator reported only what occurred, a question might be left in the mind of the reader about whether the other activities had occurred but had simply not been observed. Likewise, if a criminal justice program is supposed to provide one-to-one counseling to juveniles but no such counseling takes place, it is entirely appropriate for the evaluator to note the absence of counseling.

Earlier, for a different purpose, I recounted observing early-childhood programs and finding in one program the absence of children’s art on the walls. Indeed, the absence of any colorful posters or art of any kind stood out because, in all other centers we observed, the walls were covered with colorful displays.

Thus, it can be appropriate to note that something did not occur when the observer’s basic knowledge of and experience with the phenomenon suggests that the absence of some particular activity or factor is noteworthy. This clearly calls for judgment, common sense, and experience. As eminent qualitative methodologist Bob Stake (1995) has asserted,

One of the principal qualifications of qualitative researchers is experience. Added to the experience of ordinary looking and thinking, the experience of the qualitative researcher is one of knowing what leads to significant understanding, recognizing good sources of data, and consciously and unconsciously testing out the veracity of their eyes and robustness of their interpretations. It requires sensitivity and skepticism. Much of this methodological knowledge and personality come from hard work under the critical examination of colleagues and mentors. (pp. 49–50)

Making informed judgments about the significance of nonoccurrences can be among the most important contributions an evaluator can make because such feedback can provide program staff or other evaluation users with information that they may not have thought to request. Moreover, they may lack the requisite experience or awareness to have noticed the absence of that which the evaluator observes. For example, the absence of staff conflict is typically noteworthy because staff conflict is common. Similarly, absence of conflict between administrative levels (local, state, and federal) would be noteworthy because such conflict is, in my experience, virtually universal.

In many such cases, the observation about what did not occur is simply a restatement, in the opposite, of what did occur. That restatement, however, will attract attention in a way that the initial observation might not. For example, if one were observing a program being conducted in a

multiracial community, it is possible that program goals would include statements about the necessity of staff being sensitive to the particular needs, interests, and cultural patterns of minorities, but there may not be specific mention of the desired racial composition of program staff. If, then, the evaluator knows that the staff of the program consist entirely of self-identified Caucasians, it is appropriate to report that the staff are all white; that is, no people of color are among the program staff, the importance of which derives from the location and nature of the program.

Observations of staff interaction and decision-making processes also provide opportunities for evaluators to note things that do not happen. If, over time, the observer notes that program-planning processes never include participants’ input in any systematic or direct way, it may well be appropriate for the evaluator to point out the absence of such input based on experiences indicating the significance of participant input in the planning processes of other programs.

My evaluation of the wilderness education program included observations about a number of things that did not occur. No serious injuries occurred on any of the six field conferences in the wilderness—important information for someone thinking about the possible risks involved in such a program. No participant refused to shoulder his or her share of the work that had to be done for the group to live and work together in the wilderness. This observation emerged from discussions with technical field staff who often worked with juveniles in wilderness settings where uneven sharing of cooking, cleaning, and related responsibilities often led to major group conflicts. The fact that the groups I observed never had to deal with one or two people not helping out was worth noting.

Perhaps the most important observation about what did not happen came from observing staff meetings. Over time, I noticed a pattern in which staff held meetings to make decisions about important issues, but no such decisions were made. Staff sometimes thought that a decision had been made, but closure was not brought to the decision-making process, and no responsibility for follow-up was assigned. Many subsequent implementation failures and staff conflicts could be traced to ambiguities and differences of opinion that were left unresolved at staff meetings. By hearing me describe what both was and was not occurring, staff became more explicit and effective in making decisions. Reporting what did happen in staff meetings was important; but it was also extremely important to observe what did not happen.

MODULE

50 Observing Oneself: Reflexivity and Creativity, and Review of Fieldwork Dimensions

Physician, heal thyself.

Observer, observe thyself. —Halcolm

Observations often tell you more about the observer than the observed. —Chris Geiger

Journalist and cancer survivor

In Chapter 2, I identified voice and perspective, or reflexivity, as one of the central strategic themes of qualitative inquiry. The term reflexivity has entered the lexicon as a way of emphasizing the importance of self-awareness, political/cultural consciousness, and ownership of one’s perspective. Reflexivity reminds the qualitative inquirer to observe herself or himself so as to be attentive to and conscious of the cultural, political, social, linguistic, and ideological origins of one’s own perspective and voice as well as, and often in contrast to, the perspectives and voices of those one observes and talks to during fieldwork. Reflexivity calls for self-reflection, indeed, critical self- reflection and self-knowledge, and a willingness to consider how who one is affects what one is able to observe, hear, and understand in the field as an observer and analyst. The observer, therefore, during fieldwork, must observe self as well as others, and interactions of self with others.

Exhibit 2.5 (p. 72) poses three interconnected, triangulated reflexive questions from three perspectives:

1. Myself, as inquirer: How do I know what I know? What shapes and has shaped my perspective?

2. People in the setting being studied: How do they know what they know? What shapes and has shaped their worldview? How do they perceive me? Why? How do I know? How do I perceive them?

3. Audiences for the study: How do they make sense of what I give them? What perspectives do they bring to the findings I offer? How do they perceive me? How do I perceive them?

These are questions to address explicitly in the methods section of a qualitative report. Reflecting on these questions informs the credibility of the conclusions you report from fieldwork.

SIDEBAR

REFLEXIVE/REFLECTIVE INQUIRY

Reflexivity involves “paying much attention to how one thinks about thinking . . . , a ‘reflexivity’ that constantly assesses the relationship between ‘knowledge’ and ‘the ways of doing knowledge.’”

Reflective [reflexive] research, as we define it, has two basic characteristics: careful interpretation and reflection. The first implies that all references—trivial and nontrivial—to empirical data are the results of interpretation. . . . Interpretation comes to the forefront of the research work. This calls for the utmost awareness of the theoretical assumptions, the importance of language and pre- understanding, all of which constitute major determinants of the interpretation. The second element, reflection, turns attention “inwards” towards the person of the researcher, the relevant research community, society as a whole, intellectual and cultural traditions, and the central importance, as well as the problematic nature, of language and narrative (the form of presentation) in the research context. . . . Reflection can, in the context of empirical research, be defined as the interpretation of interpretation and the launching of a critical self-exploration of one’s own interpretations of empirical material (including its construction).

—Alvesson and Sköldbery (2009, pp. 8–9)

Reflexive Methodology

Once again, for continuity, I would cite Parameswaran (2001), who has written a wonderfully self-reflective account of her experience returning to her native India to do fieldwork as a feminist scholar after being educated in the United States.

Because my parents were fairly liberal compared to many of my friends’ parents, I grew up with a little more awareness than many middle- and upper-class Indians of the differences between my life and that of the vast majority of Indians. Although I questioned some restrictions that were specific to women of my class, I did not have the language to engage in a systematic feminist critique of patriarchy or nationalism. Feminism for me had been unfortunately constructed as an illness that struck highly Westernized intellectual Indian women who were out of touch with reality. . . . It was my dislocation from India to the relatively radicalized context of United States that prompted my political development as a feminist and a woman of color. (p. 76)

Given this background and the controversial focus of her fieldwork (reading of Western romance novels by young Indian women), she identified reflective questions to guide her reflexive inquiry during and after fieldwork:

How do kinship roles assigned to native scholars shape social interactions in the field? How can commitments to sisterhood make it difficult for feminist ethnographers to achieve critical distance and discuss female informants’ prejudiced views? (p. 76)

Her personal inquiry into these questions, reflecting on her own fieldwork experiences (Parameswaran, 2001), is a model of reflexivity.

Many years ago, Indian philosopher J. Krishnamurti (1964) commented on the challenges of self-knowledge. Although his reflections were directed to the importance of lifelong learning rather than to being reflexive in fieldwork, his ruminations offer a larger context for thinking about how to observe oneself, a context beyond concern about methodological authenticity, though his advice applies to that as well.

Self-knowledge comes when you observe yourself in your relationship with your fellow-students and your teachers, with all the people around you; it comes when you observe the manner of another, his gestures, the way he wears his clothes, the way he talks, his contempt or flattery and your response

[italics added]; it comes when you watch everything in you and about you and see yourself as you see your face in the mirror. . . . Now, if you can look into the mirror of relationship exactly as you look into the ordinary mirror, then there is no end to self-knowledge. It is like entering a fathomless ocean which has no shore . . . ; if you can just observe what you are and move with it, then you will find that it is possible to go infinitely far. Then there is no end to the journey, and that is the mystery, the beauty of it. (pp. 50–51)

I realize that Krishnamurti’s phrase “There is no end to the journey” may strike terror in the hearts of graduate students reading this in preparation for dissertation fieldwork or evaluators facing a report deadline. But remember, he’s talking about lifelong learning, of which the dissertation or a specific evaluation report is but one phase. Just as most dissertations and evaluations are reasonably expected to contribute incremental knowledge rather than make major breakthroughs, so too the self-knowledge of reflexive fieldwork is but one phase in a lifelong journey toward self-knowledge—but it’s an important phase and a commitment of growing significance as reflexivity has emerged as a central theme in qualitative inquiry.

The point here, which we shall take up in greater depth in the chapters on analysis and credibility, is that the observer must ultimately deal with issues of authenticity, reactivity, and how the observational process may have affected what was observed as well as how the background and predispositions of the observer may have constrained what was observed and understood. Each of these areas of methodological inquiry depends on some degree of critical reflexivity.

Fieldwork Menu Summary

This lengthy review of options for what and how to observe during fieldwork constitutes a sensitizing framework. It is not a prescriptive guide—you must do this and all of this! Nor is it a formal checklist of the kind that airline pilots go through before takeoff. Rather, it is a menu of possibilities. You have to decide which items to incorporate into your own inquiry, adapt them to your own research or evaluation purpose and questions, and fill in the details within your own context. Exhibit 6.8 provides a summary graphic.

EXHIBIT 6.8 Dimensions of Fieldwork

Nested and Layered Case Studies During Fieldwork

A case study is expected to catch the complexity of a single case. The single leaf, even a single toothpick, has unique complexities—but rarely will we care enough to submit it to case study. We study a case when it itself is of very special interest. We look for the detail of interaction with its context. Case study is the study of the particularity and complexity of a single case, coming to understand its activity within important circumstances. (Stake, 1995, p. xi)

Months of fieldwork may result in a single case study that describes a village, community, neighborhood, organization, or program. However, that single case study is likely to be made up of many smaller cases, or observational units—the stories of specific individuals, families, organizational units, and other groups. Critical incidents and case studies of specific bounded activities, like a celebration, may also be presented within the larger case. The qualitative analysis process typically centers on presentation of specific cases and thematic analysis across cases. Knowing this, fieldwork can be organized around nested and layered case studies, which means that some form of nested case sampling must occur.

Let me briefly review the centrality of case studies as a qualitative inquiry strategy. Chapter 1 opened by citing a number of well-known and influential books based on case studies, for example, Peters and Waterman’s In Search of Excellence: Lessons From Americas’ Best-Run Companies (1982), Angela Browne’s important book When Battered Women Kill (1987), and Sara Lawrence- Lightfoot’s six detailed case studies of Respect (2000). Chapter 2 presented the construction of unique case studies as a major strategic theme of qualitative inquiry. Chapter 3 reviewed theoretical perspectives that are inductively case based. Chapter 4 reviewed at some length the importance in qualitative evaluation of capturing and reporting individualized outcomes based on case studies of how participants in programs change during a program and whether they maintain those changes afterward. Chapter 5 discussed alternative views of what a case is, examined what it means to do a case study (both an inquiry method and a product of such an inquiry), and provided an extended discussion of purposeful sampling strategies for selecting cases. Now we come to the point of distinguishing and integrating various observational units and data sources in fieldwork. William Foote Whyte’s (1943) classic study of Street Corner Society has long been recognized as an exemplar of the single (n = 1) community case study, even though his study of “Cornerville” includes the stories (case studies) of several individual lower-income youths, some of whom were striving to escape the neighborhood.

Integrating Cases: The Wilderness Education Program as an Example

Fieldwork can include a variety of kinds of data: formal interviews, conversational interviews, group interviews, spectator observations, participant observation, and document reviews. To illustrate this point, in the wilderness education program I’ve been using as an example throughout this chapter, our evaluation team generated case studies of participants (higher education leaders and administrators) using multiple sources of data from fieldwork: (1) background data gathered through interviews about participants’ situations and perspectives on entering the year of field conferences; (2) observations of their experiences during field conferences; (3) informal and conversational interviews with them during the four wilderness trips, each for 10 days, over the course of a year; (4) quotations from formal group interviews (focus groups) held at various times during the trips; (5) excerpts from their journals and other personal writings when they were willing to share those with us, as they often were; and (6) follow-up telephone interviews with participants after each field trip and after the entire program was completed to track the impact of the program on individuals over time.

The overall case being studied was the wilderness program, a single case. But it operated with two different cohorts of participants over two years, so each year was also a case, as was each trip within a year. And for the individual case studies, each year (n = 36), the various data sources noted above were combined to construct an integrated, holistic case study of individual personal and professional outcomes. Thus, when more than one object of study or unit of analysis is included in fieldwork, case studies may be layered and nested within the overall, primary case approach.

Let’s look at how the various qualitative pieces fit together. As noted, the three-year wilderness program constituted the overall, one might say macro, case study. Therefore, the final evaluation report presented conclusions about the processes and outcomes of the overall program, a case study of the three-year wilderness education initiative. As Exhibit 6.9 shows, however, within that overall evaluation case study were nested individual participant’s case studies documenting individual experiences and outcomes; case studies of each yearlong group cohort; and case studies of each separate field conference, for example, the 10 days in the Gila Wilderness of New Mexico and the 10 days in the Kofa Mountains of Arizona. Slicing through the fieldwork and analysis in other ways were case studies of particular incidents, for example, the emotional catharsis experienced by one participant when she finally managed to overcome her terror and rappel down a cliff face, the whole group watching and urging her on, a process that took some 45 tense minutes. Other mini cases consisted of different observational units. A full day’s hike could be an observational unit, as could running a specific dangerous rapid on the San Juan River. Each evening discussion constituted an observational unit, such that over the three years, we had notes on more than 80 discussions of various kinds. Staff meetings made for unique observational units, as could a specific decision made in a staff meeting.

In essence, extended fieldwork can and typically does involve many mini or micro case studies of various observational units (individuals, groups, specific activities, specific time periods, critical incidents), all of which together make up the overall case study, in this example, the final evaluation of the wilderness education program. Fieldwork, then, can be thought of as engaging in a series of multilayered and nested case studies, often with intersecting and overlapping units of analysis or observational units.

Creativity in Fieldwork

No checklist can be relied on to guide all aspects of fieldwork. A participant-observer must constantly make judgments about what is worth noting. Because it is impossible to observe everything, some process of selection is necessary. Plans made during design should be revised as appropriate when important new opportunities and sources of data become available. That’s where flexibility and creativity help. Creativity can be learned and practiced (Patton, 1987). Creativity comes into play in a major way in integrating and synthesizing multiple observations and sources of data into a coherent whole—the case study story. Creative fieldwork means using every part of oneself to experience and understand what is happening. Creative insights come from being directly involved in the setting being studied.