U7D1-68 - **READ AND FOLLOW ALL INSTRUCTIONS AS OUTLINED BELOW.. Research Methodology Peer Review.

EXHIBIT 6.9 Nested, Layered, and Overlapping Mini Case Studies During Fieldwork

The wilderness program evaluation for higher education leaders and administrators illustrates how case studies can be layered and nested. Evaluation of the three-year wilderness program constituted the overall macro case study, the primary evaluation case study. Nested and layered within that overall evaluation were various mini cases of overlapping and intersecting observation units that helped organize and frame fieldwork and note taking.

Nested, layered, and overlapping mini case studies (observational units)

I shall return to the issue of creativity in considering the interpretation of field notes later in this chapter and again in the analysis chapter (Chapter 8). For the moment, it is sufficient to acknowledge the centrality of creativity in naturalistic inquiry and to concur with Virginia Woolf:

Odd how the creative power at once brings the whole universe to order. . . . I mark Henry James’ sentence: observe perpetually. Observe the oncome of age. Observe greed. Observe my own despondency. By that means it becomes serviceable. (Quoted by Partnow, 1978, p. 185)

SIDEBAR

BOXED IN

Those who aspire to be creative are admonished to “think outside the box.” This presumes that one has exhausted the possibilities of learning within the box. Before moving outside the box, make sure you know the box. Observe it. Look deep within. Find out the history of the box, how it came to be the box. What has it held? What has been taken from it? Examine the corners. Look underneath, on top, on all sides. Know the box. Understand the box. Learn what the box has to teach. Think inside the box. Only then will you truly be ready to “think outside the box.”

—From Halcolm’s Boxing Guide

MODULE

51 Doing Fieldwork: The Data-Gathering Process

The purpose of the research has been clarified. The primary research questions have been focused. Qualitative methods using observations have been selected as one of the appropriate methods for gathering data. It is time to enter the field. Now begins the essential and arduous task of taking field notes.

Field Notes

Nota bene (Latin): “Note well, observe carefully, pay close attention.”

Many options exist for taking field notes. Variations include the writing materials used, the time and place for recording field notes, the symbols developed by observers as their own method of shorthand, and how field notes are stored. No universal prescriptions about the mechanics of and procedures for taking field notes are possible because different settings lend themselves to different ways of proceeding, and the precise organization of fieldwork is very much a matter of personal style and individual work habits. What is not optional is the taking of field notes!

Aside from getting along in the setting, the fundamental work of the observer is the taking of field notes. Field notes are “the most important determinant of later bringing off a qualitative analysis. Field notes provide the observer’s raison d’etre. If he is not doing them, he might as well not be in the setting.” (Lofland, 1971, p. 102)

Field notes contain the description of what has been observed. They should contain everything that the observer believes to be worth noting. Don’t trust anything to future recall. At the moment one is writing, it is very tempting, because the situation is still fresh, to believe that the details or particular elements of the situation can be recalled later. If it’s important as part of your consciousness as an observer, if it’s information that has helped you understand the context, the setting, and what went on, then as soon as possible that information should be captured in the field notes.

At the same time, you can’t and won’t capture everything. Reflecting his years of experience both doing qualitative studies and working with students on their dissertations, Bob Stake (2010) offers this comforting observation:

One of the largest worries of a new researcher is making an accurate record of what is happening. I sometimes think he or she worries too much about the accuracy. Yes, it has to be right, but there is more than one chance to get it right. The first responsibility of the observer is to know what is happening, to see it, to hear it, to try to make sense of it. That is more important than getting the perfect note or quote. Much of what we put down is an approximation that we can improve upon later—if we have a good idea what happened. (p. 94)

Description First

First and foremost, field notes are descriptive. They should be dated and should record basic information such as where the observation took place, who was present, what the physical setting was like, what social interactions occurred, and what activities took place. Field notes contain the descriptive information that will permit you to return to an observation later during analysis and, eventually, permit the reader of the study’s findings to experience the activity observed through your report.

Exhibit 6.10 presents different kinds of descriptive field notes. On the left side are vague and overgeneralized field notes. On the right side are more detailed and concrete field notes from the same observation. These examples illustrate the problem of using general terms to describe specific actions and conditions. Words like poor, anger, and uneasy are insufficiently descriptive. Such interpretive words conceal what actually went on rather than reveal the details of the situation. Such terms have little meaning for a person present for the observation. Moreover, the use of such terms in field notes, without the accompanying detailed description, means that the fieldworker has fallen into the bad habit of primarily recording interpretations rather than description. Particularly revealing are terms that can only make sense in comparison with something else. The phrase “poorly dressed” requires some frame of reference about what constitutes “good” dress. No skill is more critical in fieldwork than learning to be descriptive, concrete, and detailed.

EXHIBIT 6.10 Observations From Field Notes: Poorly Done Compared With Well Done

Field notes also contain what people say. Direct quotations, or as near as possible recall of direct quotations, should be captured during fieldwork, recording what was said during observed activities

as well as responses garnered during interviews, both formal and conversational. Quotations provide the “emic perspective” discussed earlier—the insider’s perspective—which “is at the heart of most ethnographic research” (Fetterman, 1989, p. 30).

Field notes also contain the observer’s own feelings, reactions to the experience, and reflections about the personal meaning and significance of what has been observed. Don’t deceive yourself into thinking that such feelings can be conjured up again simply by reading the descriptions of what took place. Feelings and reactions should be recorded at the time they are experienced, while you are in the field. Both the nature and intensity of feelings should be recorded. In qualitative inquiry, the observer’s own experiences are part of the data. Part of the purpose of being in a setting and getting close to the people in the setting is to permit you to experience what it is like to be in that setting. If what it is like for you, the observer or participant-observer, is not recorded in your field notes, then much of the purpose of being there is lost.

Finally, field notes include your insights, interpretations, beginning analyses, and working hypotheses about what is happening in the setting and what it means. While you should approach fieldwork with a disciplined intention not to impose preconceptions and early judgments on the phenomenon being experienced and observed, nevertheless, as an observer, you don’t become a mechanical recording machine on entering the field. Insights, ideas, inspirations—and yes, judgments too!—will occur while making observations and recording field notes. It’s not that you sit down early on and begin the analysis and, if you’re an evaluator, make judgments. Rather, it’s in the nature of our intellects that ideas about the meaning, causes, and significance of what we experience find their way into our minds. These insights and inspirations become part of the data of fieldwork and should be recorded in context in field notes. I like to set off field interpretations with brackets. Others use parentheses, asterisks, or some other symbol to distinguish interpretations from description. The point is that interpretations should be understood to be just that, interpretations, and labeled as such. Field-based insights are sufficiently precious that you need not ignore them in the hope that, if really important, they will return later.

Field notes, then, contain the ongoing data that are being collected. They consist of descriptions of what is being experienced and observed, quotations from the people observed, the observer’s feelings and reactions to what is observed, and field-generated insights and interpretations. Field notes are the fundamental database for constructing case studies and carrying out thematic cross- case analysis in qualitative research.

Procedurally Speaking

When field notes are written will depend on the kind of observations being done and the nature of your participation in the setting being studied. In an evaluation of a parent education program, I was introduced to the parents by the staff facilitator, who explained the purpose of the evaluation and assured the parents that no one would be identified. I then openly took extensive notes without participating in the discussions. Immediately following those sessions, I would go back over my notes to fill in details and be sure what I had recorded made sense. By way of contrast, in the wilderness education program, I was a full participant engaged in full days of hiking, rock climbing, and rafting/kayaking. I was so exhausted by the end of each day that I seldom stayed awake while the others slept, making field notes by flashlight. Rather, each night, I jotted down basic notes that I could expand during the time when the others were writing in their journals, but some of the expansion had to be completed after the weeklong field conference. In evaluating a leadership training program as a participant-observer, the staff facilitator privately asked me not to

take notes during group discussions because it made him nervous, even though most of the other participants were taking notes.

The extent to which notes are openly recorded during the activities being observed is a function of the observer’s role and purpose, as well as the stage of participant observation. If the observer or evaluator is openly identified as a short-term, external, nonparticipant observer, participants may expect him or her to write down what is going on. If, on the other hand, one is engaged in longer- term participant observation, the early part of the process may be devoted to establishing the participant-observer role, with emphasis on participation, so that open taking of notes is deferred until the fieldworker’s role has been firmly established within the group. At that point, it is often possible to openly take field notes since, hopefully, the observer is better known to the group and has established some degree of trust and rapport.

The wilderness program evaluation involved three 10-day trips (“field conferences”) with participants at different times during the year. During the first field conference, I never took notes openly. The only time I wrote was when the others were also writing. During the second field conference, I began to openly record observations when discussions were going on if taking notes did not interfere with my participation. By the third field conference, I felt I could take notes whenever I wanted to, and I had no indication from anyone that they even paid attention to the fact that I was taking notes. By that time, I had established myself as a participant, and my participant role was more primary than my evaluator role.

The point here is that evaluator-observers must be strategic about taking field notes, timing their writing and recording in such a way that they are able to get their work done without unduly affecting either their participation or their observations. Given those constraints, the basic rule of thumb is to write promptly, to complete field notes as soon and as often as physically and programmatically possible.

Writing field notes is rigorous and demanding work. Lofland (1971) has described this rigor quite forcefully:

Let me not deceive the reader. The writing of field notes takes personal discipline and time. It is all too easy to put off actually writing notes for a given day and to skip one or more days. For the actual writing of the notes may take as long or longer than did the observation! Indeed, a reasonable rule of thumb here is to expect and plan to spend as much time writing notes as one spent in observing. This is, of course, not invariant . . . but one point is inescapable. All the fun of actually being out and about monkeying around in some setting must also be met by cloistered rigor in committing to paper—and therefore to future usefulness—what has taken place. (p. 104)

Integrating Observations, Interviews, and Documentation: Bringing Together Multiple Sources for Triangulation

Fieldwork is more than a single method or technique. For example, evaluation fieldwork means that the evaluator is on-site (where the program is happening) observing, talking with people, and going through program records. Multiple sources of information are sought and used because no single source of information can be trusted to provide a comprehensive perspective on the program. By using a combination of observations, interviewing, and document analysis, the fieldworker is able to use different data sources to validate and cross-check findings. Each type and source of data has strengths and weaknesses. Using a combination of data types—triangulation, a recurring theme in

this book—increases validity as the strengths of one approach can compensate for the weaknesses of another approach.

Limitations of observations include the possibility that the observer may affect the situation being observed in unknown ways, people may behave in some atypical fashion when they know they are being observed, and the selective perception of the observer may distort the data. Observations are also limited in focusing only on external behaviors—the observer cannot see what is happening inside people, what they’re thinking and feeling. Moreover, observational data are often constrained by the limited sample of activities actually observed. Researchers and evaluators need other data sources to find out the extent to which observed activities are typical or atypical.

Interview data limitations include possibly distorted responses due to personal bias, anger, anxiety, politics, and simple lack of awareness since interviews can be greatly affected by the emotional state of the interviewee at the time of the interview. Interview data are also subject to recall error, reactions of the interviewee to the interviewer, and self-serving responses.

Observations provide a check on what is reported in interviews; interviews, on the other hand, permit the observer to go beyond external behavior to explore feelings and thoughts.

Documents and records also have limitations. They may be incomplete or inaccurate. Client files maintained by programs are notoriously variable in quality and completeness, with great detail in some cases and virtually nothing in others. Document analysis, however, provides a behind-the- scenes look at the program that may not be directly observable and about which the interviewer might not ask appropriate questions without the leads provided through documents.

By using a variety of sources and resources, the qualitative inquirer can build on the strengths of each type of data collection while minimizing the weaknesses of any single approach. This mixed- methods, triangulated approach to fieldwork is based on pragmatism and is illustrated by my attempt to understand some of the problems involved in staff communication during the wilderness education and leadership program evaluation. I mentioned this example earlier, but I’d like to expand it here. As noted, two kinds of staff worked in the program: (1) three people who had overall management and administrative responsibility (the program was their idea, and they procured the funding) and (2) the two technical staff who had responsibility for wilderness skills training, field logistics, and safety. The technical staff had extensive experience leading wilderness trips, but they also were skilled at facilitating group processes. During the trips, the lines of responsibility between technical staff and administrative staff were often blurred, and on occasion, these ambiguities gave rise to conflicts. I observed the emergence of conflict early on the first trip but lacked context for knowing what was behind these differences. Through interviews and casual conversations during fieldwork, I learned that all of the staff, both administrative and technical, had known each other prior to the program. Indeed, the program administrative directors had been the college professors of the technical staff, while the latter were still undergraduate students. However, the technical staff had introduced the directors to the wilderness as an environment for experiential education. Each of the staff members described in interviews his or her perception of how these former relationships affected the field operations of the program, including difficulties in communication that had emerged during planning sessions prior to the actual field conferences. Some of those conflicts were documented in letters and memos. Reading their files and correspondence gave me a deeper understanding of the different assumptions and values of various staff members. But the documentation would not have made sense without the interviews, and the focus of the interviews came from the field observations. Taken together, these diverse sources of information and data gave me a complete picture of staff relationships. Working back and forth among individual staff members and group staff meetings, I was able to use this information to

assist the staff in their efforts to improve their communications during the final field conference. All three sources of information proved critical to my understanding of the situation and that understanding enhanced my effectiveness as a formative evaluator.

The Technology of Fieldwork and Observation

The classic image of the anthropological fieldworker is of someone huddled in an African hut writing voluminously by light of a lantern. Contemporary researchers, however, have available to them a number of technological innovations, which, when used judiciously, can make fieldwork more efficient and comprehensive. First and foremost is the digital recorder. For some people, including myself, dictating field notes saves a great deal of time while increasing the comprehensiveness of the report. Learning to dictate takes practice, effort, and critical review of early attempts. Voice recorders must be used judiciously so as not to become obtrusive and inhibit program processes or participant responses. A voice recorder is much more useful for recording field notes in private than as an instrument to be carried about at all times, available to put a quick end to any conversation into which the observer enters.

Technology for taking photographs has become standard in fieldwork. Photographs can help in recalling things that have happened as well as vividly capturing the setting for others. Digital photography, as well as advances in printing and photocopying, now makes it possible to economically reproduce photographs in research and evaluation reports. In the wilderness education program evaluation, I officially became the group photographer, making photographs available to all of the participants. This helped legitimize taking photographs and reduced the extent to which other people felt it necessary to carry their own cameras at all times, particularly at times when it was possible that the equipment might be damaged. Looking at photographs during analysis helped me recall the details of certain activities that I had not fully recorded in my written notes. I relied heavily on photographs to add details to descriptions of places where critical events occurred in the Grand Canyon initiation story I wrote about coming of age in modern society (Patton, 1999). We’ll discuss the use of photographs at greater length in the analysis chapter (Chapter 8).

Video photography is another technological innovation that has become readily accessible and common enough that it can sometimes be used unobtrusively. For example, in a formative evaluation of a staff training program, I used video to provide visual feedback to staff. Videos of classrooms, training sessions, therapeutic interactions, and a host of other observational targets can sometimes be less intrusive than a note-taking observer. We had great success taking videos of mothers and children playing together in early-childhood education centers. Of course, use of such technology must be negotiated with program staff and participants, but the creative and judicious use of technology can greatly increase the quality of field observations and the utility of the observational record to others. Moreover, comfort with voice recorders and video has made it increasingly possible to use such technology without undue intrusion when observing programs where professionals are the participants. In addition, sometimes videos originally done for research or evaluation can subsequently be used for future training, program development, and communications to policymakers and the public, making the costs more manageable because of added uses and benefits.

Why should a social researcher wish to incorporate the analysis of images—paintings, photographs, film, videotape, drawings, diagrams and a host of other images—into their research? There are two good reasons. . . .

The first good reason is that images are ubiquitous in society, and because of this some consideration of visual representation can potentially be included in all studies of society. No matter how tightly or narrowly focused a research project is, at some level all social research says something about society in general, and given the ubiquity of images, their consideration must at some level form part of the analysis.

The second good reason why the social researcher might wish to incorporate the analysis of images is that a study of images or one that incorporates images in the creation or collection of data might be able to reveal some sociological insight that is not accessible by any other means. (Banks, 2007, pp. 3–4)

Here’s the caveat: Visual technology can add an important dimension to fieldwork if the observer knows how to use such technology and uses it well—for there is much to learn beyond how to point the camera or turn on the video recorder, especially about integrating and analyzing visual data within a larger fieldwork context. Moreover, a downside to visual technology has emerged since it is now possible to not only capture images on film and video but also change and edit those images in ways that distort. In his extensive review of “visual methods” in qualitative inquiry, Douglas Harper (2000) concluded that “now that images can be created and/or changed digitally, the connection between image and ‘truth’ has been forever severed” (p. 721). This means that issues of credibility apply to using and reporting visual data as they do to other kinds of data. Still, visual qualitative data will only become more important over time, and visual technology has already become a specialized method, with its own handbooks to offer guidance about how to make the best use of visual data (Margolis & Pauwels, 2011).

Whether using technology is more intrusive than taking notes by hand will depend on the setting, the people being observed, and the inquirer’s approach. Indeed, taking field notes can be nearly as intrusive as using visual technology, as illustrated in the fieldwork of anthropologist Carlos Castaneda. In the passage below, Castaneda (1972) reports on his negotiations with Don Juan to become his Native Indian key informant on sorcery and indigenous drugs. The young anthropologist records that Don Juan “looked at me piercingly.”

SIDEBAR

VISUAL QUALITATIVE DATA

Visual media, including photography, film, and more recently video, provide unprecedented opportunities for social science research. Consider video, for example: here is a cheap and reliable technology that enables us to record naturally occurring activities as they arise in ordinary habitats, such as the home, the workplace or the classroom. These records can be subject to detailed scrutiny. They can be repeatedly analyzed and they enable access to the fine details of conduct and interaction that are unavailable to more traditional social science methods. These records can be shown and shared with others not only fellow researchers, but participants themselves, or those with a more practical or applied interest in the activities and their organization. Unlike many forms of qualitative data, video can form an archive, a corpus of data that can be subject to a range of analytic interests and theoretical commitments, providing flexible resources for future research and collaboration. Video can also enable us to reconsider the ways in which we present the findings of social science research, not only to academic colleagues but more generally to the wider public.

—Heath, Hindmarsh, and Luff (2010)

Video in Qualitative Research: Analyzing Social Interaction in Everyday Life

Creative Visual Documentation

Distinguished evaluator Michael Bamberger is also a wood sculptor. He created a work of art to depict the daily activities of women in Morocco. He explains,

This (wood carving) was inspired by a series of planning workshops with Moroccan village women. They were asked to draw a story illustrating their typical day. The figures in the carving illustrate these activities including collecting firewood, drawing water from the well, taking the goats to pasture, preparing meals, taking care of the children and then all going to sleep on one of the beautiful carpets that the women make. The numbers indicate the time of day when different activities take place. The gnarled branches in the center are the olive trees. (Bamberger, 2014)

SOURCE: Bamberger (2014). Used with permission.

“What are you doing in your pocket?” he asked, frowning. “Are you playing with your whanger?”

He was referring to my taking notes on a minute pad inside the enormous pockets of my windbreaker.

When I told him what I was doing he laughed heartily.

I said that I did not want to disturb him by writing in front of him.

“If you want to write, write,” he said. “You don’t disturb me.” (pp. 21–22)

Whether one uses modern technology to support fieldwork or simply writes down what is occurring, some method of keeping track of what is observed must be established. In addition, the nature of the recording system must be worked out in accordance with the observer’s role, the purpose of the study, and consideration of how the data-gathering process will affect the activities and persons being observed. Many of these issues and procedures must be worked out during the initial phase (entry period) of fieldwork, to which we now turn.

SIDEBAR

NEW DIRECTIONS IN TECHNOLOGY-BASED FIELDWORK

The Human Speechome Project

The Human Speechome Project studies how children learn the meaning of words through “analysis of observational recordings of child–caregiver interactions in natural contexts.” Professor Deb Roy is recording his son’s development at home by collecting approximately 10 hours of high-fidelity audio and video on a daily basis from birth to age three. The resulting data amount to more than 100,000 hours of multitrack recordings. They constitute “the most comprehensive record of a child’s development made to date” (MIT Media Lab, 2014; Roy, 2009).

Google Glass

Google Glass is a pair of glasses that takes photographs by voice command and then transmits them by e-mail to a data collection site. It can be used in observations and interviews. It can also be intrusive, so extra care must be taken to respect people’s privacy and get permission when using any visual data collection technique as part of observational fieldwork.

MODULE

52 Stages of Fieldwork: Entry Into the Field

All truth passes through three stages. First, it is ridiculed. Second, it is violently opposed. Third, it is accepted as being self-evident.

—Arthur Schopenhauer (1788–1860) German philosopher

Thus far, fieldwork has been described as if it was a single, integrated experience. Certainly, when fieldwork goes well, it flows with certain continuity, but it is useful to look at the evolution of fieldwork through identifiable stages. Three stages are most often discussed in the participant observation literature: (1) the entry stage, (2) the routinization of the data-gathering period, and (3) the closing stage. The following sections explore each of these stages.

Entry Into the Field

Beginnings matter. Where you begin a journey and the route you take will determine what you see and the experience you have. It has always been so. Pay attention, then, to where and how you begin.

—Halcolm

The writings of anthropologists sometimes present a picture of the early period of fieldwork that reminds me of the character in Franz Kafka’s haunting novel The Castle. Kafka’s character is a wandering stranger, K., with no more identity than that initial. He doesn’t belong anywhere, but when he arrives at the castle, he wants to become part of that world. His efforts to make contact with the faceless authorities who run the castle lead to frustration and anxiety. He can’t quite figure out what is going on and can’t break through their vagueness and impersonal nature. He doubts himself; then he gets angry at the way he is treated; then he feels guilty, blaming himself for his inability to break through the ambiguous procedures for entry. Yet he remains determined to make sense out of the incomprehensible regulations of the castle. He is convinced that, after all, where there are rules—and he does find that there are rules—they must fit together somehow, have some meaning, and manifest some underlying logic. There must be some way to make contact, to satisfy the needs of the authorities, to find some pattern of behavior that will permit him to be accepted. If only he could figure out what to do, if only he could understand the rules, then he would happily do what he was supposed to do. Such are the trials of entry into the field.

Entry into the field for research or evaluation involves two separate parts: (1) negotiation with gatekeepers, whoever they may be, about the nature of the fieldwork to be done and (2) actual physical entry into the field setting to begin collecting data. These two parts are closely related, for the negotiations with gatekeepers will establish the rules and conditions for how one goes about

playing the role of observer and how that role is defined for the people being observed. In traditional scholarly fieldwork for the purpose of basic or applied research, the investigator unilaterally decides how best to conduct the fieldwork. In evaluation studies, the evaluator will need to take into account the perspectives and interests of the primary intended users of the evaluation. In either case, interactions with those who control entry into the field are primarily strategic, figuring out how to gain entry while preserving the integrity of the study and the inquirer’s interests. The degree of difficulty involved varies depending on the purpose of the fieldwork, the extent to which it may involve controversial or sensitive issues, and, accordingly, the expected degree of resistance to the study. Where the field researcher expects cooperation, gaining entry may be largely a matter of establishing trust and rapport. At the other end of the continuum are those research settings where considerable resistance, even hostility, is expected, in which case gaining entry becomes a matter of “infiltrating the setting” (Douglas, 1976, p. 167). And sometimes, entry is simply denied. A doctoral student had negotiations for entry end abruptly in a school district where she had developed good relationships with school personnel and negotiations appeared to be going well. She later learned that she was denied entry far into the negotiation process because of community opposition. The local community had had a very bad experience with a university researcher more than 20 years earlier and still viewed all research with great suspicion.

Explaining Why You Are Undertaking the Inquiry

A major difference between the entry process in scholarly research in contrast to entry for evaluation research is how the purpose of the study is explained. In cross-cultural scholarly research, the usual explanation is some variation of “I’m here because I would like to understand you better and learn about your way of life because the people from my culture would like to know more about you.” While anthropologists admit that such an explanation almost never makes sense to indigenous peoples in other cultures, it remains a mainstay initial explanation until mutual reciprocities can be established with enough local people for the observation process to become established and accepted in its own right.

Evaluators and action researchers, however, are not just doing fieldwork out of personal or professional interest. They are doing the fieldwork for some decision makers and information users who may be either known or unknown to the people being studied. It becomes critical, then, that evaluators, their funders, and evaluation users give careful thought to how the fieldwork is going to be presented. Because the word evaluation has such negative connotations for many people, having had negative experiences being evaluated, for example, at school or work, it may be appropriate to consider some other term to describe the fieldwork. In our onlooker, nonparticipatory observations for an implementation study of early-childhood programs in Minnesota, we described our role to local program participants and staff as follows:

We’re here to be the eyes and ears for state legislators. They can’t get around and visit all the programs throughout the state, so they’ve asked us to come out and describe for them what you’re doing. That way they can better understand the programs they have funded. We’re not here to make any judgments about whether your particular program is good or bad. We are just here to be the eyes and ears for the legislature so that they can see how the legislation they’ve passed has turned into real programs. This is your chance to inform them about your work and give them your point of view.

Other settings lend themselves to other terms that are less threatening than evaluator. Sometimes, a fieldwork project can be described as “documentation.” Another term I’ve heard used by community-based evaluators is process historian. In the wilderness education program, I was a

full participant-observer, and staff described my role to participants as “keeper of the community record,” making it clear that I was not there to evaluate individual participants. The staff of the project explained that they had asked me to join the project because they wanted someone who did not have direct ego involvement in the success or outcomes of the program to observe and describe what went on, both because they were too busy running the program to keep detailed notes about what occurred and because they were too involved with what happened to be able to look at things dispassionately. We had agreed from the beginning that the community record I produced would be accessible to participants as well as staff.

In none of these cases did changing the language automatically make the entry process smooth and easy. Earlier in this chapter, I described our attempt to be viewed as “educational researchers” in evaluating a community leadership program. Everyone figured out almost immediately that we were really “evaluators”—and that’s what participants called us. Regardless of the story told or the terms used, the entry period of fieldwork is likely to remain “the first and most uncomfortable stage of field work” (Wax, 1971, p. 15). It is a time when the observer is getting used to the new setting, and the people in that setting are getting used to the observer. Johnson (1975) has suggested that there are two reasons why the entry stage is both so important and so difficult:

First, the achievement of successful entree is a precondition for doing the research. Put simply, no entree, no research. . . . Published reports of researcher’s entree experiences describe seemingly unlimited contingencies which may be encountered, ranging from being gleefully accepted to being thrown out on one’s ear. But there is a more subtle reason why the matter of one’s entrance to a research setting is seen as so important. This concerns the relationship between the initial entree to the setting and the validity of the data that is subsequently collected. The conditions under which an initial entree is negotiated may have important consequences for how the research is socially defined by the members of the setting. These social definitions will have a bearing on the extent to which the members trust a social researcher, and the existence of relations of trust between an observer and the members of a setting is essential to the production of an objective report, one which retains the integrity of the actor’s perspective and its social context. (pp. 50-51)

Reciprocity as a Guiding Principle

While the observer must learn how to behave in the new setting, the people in that setting are deciding how to behave toward the observer. Mutual trust, respect, and cooperation are dependent on the emergence of an exchange relationship, or reciprocity, in which the observer obtains data and the people being observed find something that makes their cooperation worthwhile, whether that something is a feeling of importance from being observed, useful feedback, pleasure from interactions with the observer, assistance in some task, or compensation in some form. This reciprocity model of gaining entry assumes that some reason can be found for participants to cooperate in the research and that some kind of mutual exchange can occur, not only initially but as the inquiry unfolds.

SIDEBAR

RECIPROCITY

Qualitative studies intrude into settings as people adjust to the researcher’s presence. People may be giving their time to be interviewed or to help the researcher understand group norms; the researcher should plan to reciprocate. When people adjust their priorities and routines to help the researcher, or even just tolerate the researcher’s presence, they are giving of themselves. The researcher is

indebted and should be sensitive to this. Reciprocity may entail giving time to help out, providing informal feedback, making coffee, being a good listener, or tutoring. Of course, reciprocity should fit within the constraints of research and personal ethics and of maintaining one’s role as a researcher. (Marshall & Rossman, 2011, p. 121; see also Crow, 2008)

Infiltration

Infiltration lies at the opposite end of the continuum from a negotiated, reciprocity model of entry. Many field settings are not open to observation based on cooperation. Douglas (1976, pp. 167–171) has described a number of infiltration strategies, including “worming one’s way in,” “using the crowbar to pry them open for our observations,” showing enough “saintly submissiveness” to make members guilty enough to provide help, or playing the role of a “spineless boob” who could never possibly hurt the people being observed. He has also suggested using various ploys of misdirection where the researcher diverts people’s attention away from the real purpose of the study. There is also the “phased-entree tactic” by which the researcher who is refused entry to one group begins by studying another group until it becomes possible to get into the group that is the real focus of the researcher’s attention; for example, begin by observing children in a school when who you really want to observe are teachers or administrators.

Known Sponsor Approach

Often the best approach for gaining entree is the known sponsor approach. When employing this tactic, observers use the legitimacy and credibility of another person to establish their own legitimacy and credibility, for example, the director of an organization for an organizational study, a local leader, an elected official, or the village chieftain for a community study. Of course, it’s important to make sure that the “known sponsor” is indeed a source of legitimacy and credibility. Some prior assessment must be made of the extent to which that person can provide halo feelings that will be positive and helpful. For example, in an evaluation, using a program administrator or funders as a known sponsor may increase suspicion and distrust among program participants and staff.

Entry Anxiety Syndrome

The initial period of fieldwork can be frustrating and can give rise to self-doubt. The fieldworker may lie awake at night worrying about some mistake, some faux pas, made during the day. There may be times of feeling embarrassed, feeling foolish, questioning the whole purpose of the project, and even experiencing feelings of paranoia. The fact that one is trained in social science does not mean that one is immune to all the normal pains of learning in new situations. On the other hand, the initial period of fieldwork can also be an exhilarating time; a period of rapid new learning, when the senses are heightened by exposure to new stimuli; and a time of testing one’s social, intellectual, emotional, and physical capabilities. The entry stage of fieldwork magnifies both the joys and the pains of doing fieldwork.

Evaluators can reduce the stick-out-like-a-sore-thumb syndrome by beginning their observations and participation in a program at the same time when participants are beginning the program. In traditional fieldwork, anthropologists cannot become children again and experience the same socialization into the culture that children experience. Evaluators, however, can often experience

the same socialization process that regular participants experience by becoming part of the initiation process and timing their observations to coincide with the beginning of a program. Such timing makes the evaluator one among a number of novices and substantially reduces the disparity between the evaluator’s knowledge and the knowledge of other participants.

The Observer Is Also the Observed

Beginning the program with other participants, however, does not assure the evaluator of equal status. Some participants may be suspicious that real difficulties experienced by the evaluator as a novice participant are phony—that the evaluator is playacting, only pretending to have difficulty. On the first day of my participation in the wilderness education program, we had our first backpacking experience. The staff leader began by explaining that “your backpack is your friend.” I managed to both pack and adjust my “friend” incorrectly. As a result, as soon as we hit the trail, I found that the belt around my waist holding the backpack on my hips was so tight that my “friend” was making my legs fall asleep. I had to stop several times to adjust the pack. Because of these delays and other difficulties I was having with the weight and carriage of the pack, I ended up as the last participant along the trail. The next morning when the group was deciding who should carry the map and walk at the front of the group to learn map reading, one of the participants immediately volunteered my name. “Let Patton do it. That way he can’t hang back at the end of the group to observe the rest of us.” No amount of protest from me seemed to convince the participants that I had ended up behind them all because I was having trouble hiking (working out my “friendship” with my backpack). They were convinced that I had taken that position as a strategic place from which to evaluate what was happening. It is well to remember, then, that regardless of the nature of the fieldwork, during the entry stage more than at any other time, the observer is also the observed.

What You Say and What You Do

Fieldworkers’ actions speak louder than their words. Researchers necessarily plan strategies to present themselves and their function, but participant reactions to statements about the researcher’s role are quickly superseded by judgments based on how the person actually behaves.

The relative importance of words versus deeds in establishing credibility is partly a function of the length of time the observer expects to be in a setting. For some direct onlooker observations, the field-worker may be present in a particular program for only a few hours or a day. The entry problem in such cases is quite different from the situation where the observer expects to be participating in the program over some longer period of time.

All field workers are concerned about explaining their presence and their work to a host of people. “How shall I introduce myself?” they wonder, or, “what shall I say I am doing?” If the field worker plans to do a very rapid and efficient survey, questions like these are extremely important. The manner in which an interviewer introduces himself, the precise words he uses, may mean the difference between a first-rate job and a failure. . . . But if the field worker expects to engage in some variety of participant observation, to develop and maintain long-term relationships, to do a study that involves the enlargement of his own understanding, the best thing he can do is relax and remember that most sensible people do not believe what a stranger tells them. In the long run, his host will judge and trust him, not because of what he says about himself or about his research, but by the style in which he lives and acts, by the way in which he treats them. In a somewhat shorter run, they will accept or tolerate him because some relative, friend, or person they respect has recommended him to them. (Wax, 1971, p. 365)

Qualitative sociology pioneer William Foote Whyte (1984) extracted and summarized entry strategies used in a number of groundbreaking sociological studies, including the Lynds’ study of Middletown, W. Lloyd Warner’s study of Yankee City, Burleigh Gardner’s fieldwork in the Deep South, Elliot Liebow’s hanging around Tally’s Corner, Elijah Anderson’s fieldwork in a black neighborhood, Ruth Horowitz’s study of a Chicano neighborhood, Robert Cole’s work in Japan, and Whyte’s own experiences in Cornerville (pp. 37–63). All of these researchers, who produced classic qualitative studies from in-depth participant observation, had to adapt their entry strategy to the local setting, and they all ended up changing what they had planned to do as they learned from the initial responses to their efforts to gain acceptance. Exhibit 6.11 presents reflections of a part- time observer, a role that is quite common in evaluation, but rarely discussed in the qualitative literature.

EXHIBIT 6.11 Entry-to-the-Field Reflections From a Part-Time Observer

Introductory note: In the reflections that follow, a senior staff member of the University of Minnesota Center for Social Research describes her entry into fieldwork as a part-time observer for a program evaluation. Because limitations of time and resources are common in evaluation, many situations call for a part-time observer. Her reflections capture some of the special entry problems associated with this “Now you’re here, now you’re gone” role. This is a common situation in evaluation studies but one that is seldom documented.

One word can describe my role, at least initially, in a recent evaluation assignment: ambiguous. I was to be neither a participant-observer nor an outsider coming in for a brief but intensive stint. I was to allocate approximately six hours a week for seven months to observing the team development of a group of 23 professionals in an educational setting. At first, the ambiguity was solely on my side: What, really, was I to do? The team members, too busy in the beginning with defining their own roles, had little time to consider mine. Later on, as I became accustomed to my task, the team’s curiosity about my function began to grow.

When I met with the group for the first time, I directed most of my energies to matching names and faces. I would be taking notes at most of the sessions, and it was essential that I could record not only what was said but who said it. At the first session, everyone, including me, wore a name tag. But within a few days, they were all well acquainted and had discarded their name tags; I was the only one still fumbling for names. While being able to greet each member by name was important, so was knowing something about each one’s background. Coffee breaks allowed me to circulate among the group and carry on short conversations with as many as possible to try to fix in my mind who they were and where they came from, which provided insights into why they behaved in the group as they did.

But I could tell that in their eyes, I served no useful purpose that they could see. I was in the way a great deal of the time, inhibiting their private conversations. On the other hand, they appeared to be concerned about what I was thinking. Some of them—most of them—began to be friendly, to greet me as I came in, to comment when I missed a team meeting. They came to see me as I saw myself: neither really part of the group nor a separate, removed force.

Observing their interaction perhaps 6 hours a week out of their 40-hour work week obviously meant that I missed a great deal. I needed to develop a sense of when to be present, to choose among group meetings, subgroup meetings, and activities when all the members were to come together. At the same time, I was working on other contracts which limited the amount of adjustable time available. “Flexible” was the way I came to define my weekly schedule; others, not as charitable, would probably have defined it as “shifty.”

A hazard that I encountered as I filled my ambiguous, flexible role was that I soon discovered I was not high on the priority list to be notified in the event of schedule changes. I would have firmly in mind that a subgroup was to meet on Tuesday at 10:00 a.m. in a certain place. I would arrive to find no one there. Later, I would discover that on Monday the meeting had been changed to Wednesday afternoon, and no one had been delegated to tell me. At no time did I seriously feel that the changes were planned to exclude me; on the contrary, the members’ contrition about their oversight seemed quite genuine. They had simply forgotten me.

Another area of sudden change that caused me difficulty was in policy and procedure. What had seemed to be firm commitments on ways to proceed or tasks to be tackled were being ignored. I came to realize that while a certain amount of this instability was inherent in the program itself, other shifts in direction were outgrowths of planning sessions I had not attended or had not heard the results from after they had occurred. Therefore, keeping current became for me a high-priority activity.

With my observer role to continue over many months, I realized that I must maintain the difficult position of being impartial. I could not be thought of by the team members as being closely aligned with their leaders, nor could I expect the leaders to talk candidly and openly with me if they believed that I would repeat their confidences to the group members. Reluctantly, for I discovered several team members with whom friendships could easily have developed, I declined invitations to social activities outside of working hours.

Team members at first expressed a certain amount of enthusiasm for minutes to be taken of their meetings. This enthusiasm was short-lived, for willing volunteers to serve as secretary did not emerge. I noted (and ignored) a few passing suggestions that since I was obviously taking notes maybe I could. . . .

I took copious notes before I began to develop a sense of what was or was not important to record. Over time, my understanding of the group increased. I had to realize that, as a part-time observer, it was impossible for me to understand all of what was said. Often, I would have to jot down a reminder to myself to ask clarifying questions later.

Side-stepping sensitive questions from both leaders and team members became a fine art. I was frequently queried as to my perceptions of a particular individual or situation. On one occasion, I found a team member jumping into an elevator to ride two floors with me in a direction he didn’t want to go so that he could ask me privately what I thought of another team member. My response was “I think she’s a very interesting person,” or something equally innocuous, and received from him a highly raised eyebrow, since the woman in question had just behaved in a very peculiar manner at the meeting we had both just attended.

In-depth interviews with each team member began in the fourth month of my observations. These filled in many of the gaps in my understanding. The timing was perfect: I had gained enough familiarity with both personnel and project by that time so that I was knowledgeable, they had come to trust me, and they still cared deeply about the project. (This caring diminished for some as the project year drew to a close without any real hopes of refunding for a second year.) My interview design was intentionally simple and open-ended to balance my observations with their observations.

The amount of new information diminished throughout the six weeks or so that was required to interview all team members. My own performance unquestionably diminished too as the weeks went on. It was difficult to be animated and interesting as I asked the same questions over and over, devised strategies with which to probe, and recorded perceptions and incidents which I had heard many times before. Bit by bit, team members filled in holes in my information, and their repetitious descriptions of particular incidents helped me understand what was important and why. The

interviews also helped me become aware of misconceptions on my part caused by seeing only part of the picture due to time constraints.

The experience was a new one for me, that of part-time observer. Quite frankly, this mode of evaluation probably will never be a favorite one. On the other hand, it provided a picture that no “snap-shot” evaluation method could have accomplished as interactions changed over time and in a situation where the full participant-observer role was clearly not appropriate.

MODULE

53 Routinization of Fieldwork: The Dynamics of the Second Stage

“What did you learn in your readings today?” asked Master Halcolm.

“We learned that a journey of a thousand miles begins with the first step,” replied the learners.

“Ah, yes, the importance of beginnings,” smiled Halcolm.

“Yet I am puzzled,” said a learner. “Yesterday I read that there are a thousand beginnings for every ending.”

“Ah, yes, the importance of seeing a thing through to the end,” affirmed Halcolm.

“But which is more important, to begin or end?”

“Two great self-deceptions are asserted by the world’s self-congratulators: that the hardest and most important step is the first, or that the greatest and most resplendent step is the last.”

“While every journey must have a first and last step, what ultimately determines the nature and enduring value of the journey are the steps in-between. Each step has its own value and importance. Be present for the whole journey, learners that you are. Be present for the whole journey.”

—From Halcolm’s Journey Into the Present

During the second stage, after the fieldworker has established a role and purpose, made initial contacts, and entered into the field, the focus moves to high-quality data gathering and opportunistic investigation following emergent possibilities and building on what is observed and learned each step along the way. The observer, no longer caught up in adjustments to the newness of the field setting, begins to really see what is going on instead of just looking around. As Florence Nightingale said, “Merely looking at the sick is not observing.”

Describing the second stage as “routinization of fieldwork” probably overstates the case. In emergent designs and ever-deepening inquiry, the human tendency toward routines yields to the ups and downs of new discoveries, fresh insights, sudden doubts, and ever-present questioning of others—and often of self. Discipline is needed to maintain high-quality, up-to-date field notes. Openness and perseverance are needed to keep exploring, looking deeper, diverging broader, and focusing narrower, always going where the inquiry and data take you. Fieldwork is intellectually challenging at times, mind-numbingly dull at times, and, for many, an emotional roller coaster.

Managing Relationships and Feelings During Fieldwork

One of the things that can happen in the course of fieldwork is the emergence of a strong feeling of connection with the people being studied. As you come to understand the behaviors, ideals, anxieties, and feelings of other people, you may find yourself identifying with their lives, their hopes, and their pain. This sense of identification and connection can be a natural and logical consequence of having established relationships of rapport, trust, and mutuality. For me, that awakening identification involves some realization of how much I have in common with these people whose world I have been permitted to enter. At times during fieldwork, I feel a great separation from the people I’m observing, then at other times, I feel a strong sense of our common humanity. For a fieldworker to identify, however briefly, with the people in a setting or for an evaluator to identify with the clients in a program can be a startling experience because social science observers are often quite separated from those they study by education, experience, confidence, and income. Such differences sometimes make the world of programs as exotic to evaluators as nonliterate cultures are exotic to anthropologists.

There come times, then, when a fieldworker must deal with his or her own feelings about and perspectives on the people being observed. Part of the sorting-out process of fieldwork is establishing an understanding of the relationship between the observed and the observer. When that happens, and as it happens, the person involved in fieldwork may be no less startled than Joseph Conrad’s infamous character Marlowe in Heart of Darkness. Marlowe had followed Kurtz, the European ivory trader, up the deep river into the Congo, where Kurtz established himself as a man- god to the tribal people there. He used his position to acquire ivory. To maintain his position, he had to perform the indigenous rituals of human sacrifice and cannibalism. Marlowe, deeply enmeshed in the racism of his culture and time, was initially horrified by the darkness of the jungle and its peoples, but as he watched the rituals of those seeming savages, he found an emergent identification with them and even entertained the suspicion that they were not inhuman. He became aware of a linkage between himself and them:

They howled and leaped and spun, and made horrid faces; but what thrilled you was the thought of their humanity—like ours—the thought of your remote kinship with this wild and passionate uproar. Ugly. Yes, it was ugly enough; but if you were man enough you would admit to yourself that there was in you just the faintest trace of a response to the terrible frankness of that noise, a dim suspicion of there being a meaning in it which you—you so remote from the night of the first ages—comprehend. And why not? (Conrad, 1960, p. 70)

In this passage, Conrad chronicles the possibility of awakening to unexpected realizations and intense emotions in the course of encounters with the unknown and those who are different from us. In many ways, it is our common humanity, whether we are fully aware of it at any given moment or not, that makes fieldwork possible. As human beings, we have the amazing capability to become part of other people’s experiences, and through watching and reflecting, we can come to understand something about those experiences.

Finding Balance

As fieldwork progresses, the intricate web of human relationships can entangle the participant- observer in ways that will create tension between the desire to become more enmeshed in the setting so as to learn more versus the need to preserve some distance and perspective. Participant- observers carry no immunity to the political dynamics of the settings being observed. Virtually any setting is likely to include subgroups of people who may be in conflict with other subgroups. These factions or cliques may either woo or reject the participant-observer, but they are seldom neutral. During her fieldwork interviewing young women in India, Parameswaran (2001) reports efforts by

parents and teachers to get her to inform on the women she interviewed or to influence them in a desired direction. She found herself in the middle of deep generational divisions between mothers and their daughters, teachers and students, bookstore owners and their clients. She could not risk either deeply alienating or completely acquiescing to any of these important and competing groups, for they all affected her access and the ultimate success of her fieldwork.

In evaluations, the evaluator can be caught between competing groups and conflicting perspectives. For example, where divisions exist among the staff and/or the participants in a program, and such divisions are common, the evaluator will be invited, often subtly, to align with one subgroup or the other. Indeed, the evaluator may want to become part of a particular subgroup in order to gain further insight into and understanding of that subgroup. How such an alliance occurs, and how it is interpreted by others, can greatly affect the course of the evaluation.

My experience suggests that it is impractical to expect to have the same kind of relationship—close or distant—with every group or faction. Fieldworkers—human beings with their own personalities and interests—will be naturally attracted to some people more than others. Indeed, to resist those attractions may hinder the observer from acting naturally and becoming more thoroughly integrated into the setting or program. Recognizing this, the observer will be faced with ongoing decisions about personal relationships, group involvement, and how to manage differential associations without losing perspective on what the experience is like for those with whom the fieldworker is less directly involved.

Capturing Varying Perspectives

Perhaps the most basic division that will always be experienced in program evaluation is the separation of staff and participants. While the rhetoric of many programs attempts to reduce the distinction between staff and participants, there is almost always a distinction between those who are paid for their responsibilities in the program (staff) and those who are primarily recipients of what the program has to offer (participants). Sociologically, it makes sense that staff and participants would be differentiated, creating a distance that can evolve into conflict or distrust. Participants will often view the evaluator as no different from the staff or administration, or even the funding sources—virtually any group except the participants. If the evaluator-observer is attempting to experience the program as a participant, special effort will be required to make participation real and meaningful, and to become accepted, even trusted, by the other participants. On the other hand, staff and administrators may be suspicious of the evaluator’s relationships with funders or board members.

The point is this: Do not be naive about the tangled web of relationships the participant-observer will experience and the diverse perceptions that will be encountered. Be prepared. And be thoughtful about how fieldwork, data quality, and the overall inquiry are affected by interpersonal interactions and relationships, all of which have to be negotiated and attended to on an ongoing basis.

Alignments

Lofland (1971) has suggested that participant-observers can reduce suspicion and fear about a study by becoming openly aligned with a single broad grouping within a setting while remaining aloof from that grouping’s own internal disputes.

Thus, known observers of medical schools have aligned themselves only with the medical students, rather than attempting to participate extensively with both faculty and students. In mental hospitals, known observers have confined themselves largely to mental patients and restricted their participation with staff. To attempt to participate with both, extensively and simultaneously, would probably have generated suspicion about the observers among people on both sides of those fences. (pp. 96–97)

In contrast to Lofland’s advice, in evaluating the wilderness education program, I found myself moving back and forth between a full participant role, where I was identified primarily as a participant, and a full staff role, where I was identified primarily with those who carried responsibility for directing the program. During the first field conference, I took on the role of full participant and made as visible as possible my allegiance to fellow participants while maintaining distance from the staff. Over time, however, as my personal relationships with the staff increased, I became more and more aligned with the staff. This coincided with a change of emphasis in the evaluation itself, with the earlier part of the fieldwork being directed at describing the participant experience and the latter part of the fieldwork being aimed at describing the workings of the staff and providing formative feedback.

However, I was always aware of a tension, both within myself and within the group at large, about the extent to which I was a participant or a staff member. I found that as my observational skills became increasingly valued by the program staff, I had to more consciously and actively resist their desire to have me take on a more active and explicit staff role. They also made occasional attempts to use me as an informer, trying to seduce me into conversations about particular participants. The ambiguities of my role were never fully resolved. I suspect that such ambiguities were inherent in the situation and are to be expected in many evaluation fieldwork experiences.

Participatory Inquiry Relationships

Managing field relationships involves a different set of dynamics when the inquiry is collaborative or participatory. Under such designs, where the researcher involves others in the setting in fieldwork, a great deal of the work consists of facilitating the interactions with co-inquirers, supporting their data collection efforts, ongoing training in observation and interviewing, managing integration of field notes among different participant researchers, and monitoring data quality and consistency. These collaborative management responsibilities will reduce the primary researcher’s own time for fieldwork and will affect how others in the settings, those who aren’t participatory or collaborative researchers, view the inquiry and the fieldwork director, if that is the role taken on. In some cases, management of the collaborative inquiry effort is done by one of the participants and the trained field-worker serves primarily as a skills and process trainer and consultant to the group. Clarity about these roles and divisions of labor can make or break collaborative forms of inquiry. Having shared values about collaboration does not guarantee actually pulling it off. Collaborative inquiry is challenging work, often frustrating, but when it works, the findings will carry the additional credibility of collaborative triangulation, and the results tend to be rewarding for all involved, with enduring insights and new inquiry skills for those involved.

Key Informants and Key Knowledgeables

One of the mainstays of much fieldwork is the use of key informants or key knowledgeables as sources of information about what the observer has not or cannot experience, as well as a source of explanation for events the observer has actually witnessed. Key informants are people who are particularly knowledgeable about the inquiry setting and articulate about their knowledge—people

whose insights can prove especially useful in helping an observer understand what is happening and why. Because the term informants is often associated with spies, key knowledgeables may be a more appropriate designation. The term informants emphasizes their role in the inquiry. The term knowledgeables emphasizes why they are being engaged in the inquiry.

Selecting key informants must be done carefully to avoid arousing hostility or antagonisms among those who may resent or distrust the special relationships between the fieldworker and the key informant. Indeed, how—and how much—to make visible this relationship involves considering how others will react and how their reactions will affect the inquiry. There’s no formal announcement that the “position” of key informant is open or that it’s been filled; the key informant is simply that person or those persons with whom the researcher or evaluator is likely to spend considerable time.

Key informants/key knowledgeables (I’ll use the terms interchangeably) must be trained or developed in their role, not in a formal sense, but they will be more valuable if they understand the purpose and focus of the inquiry, the issues and questions under investigation, and the kinds of information that are needed and most valuable. Anthropologists Pelto and Pelto (1978) made this point in reflecting on their own fieldwork.

We noticed that humans differ in their willingness as well as their capabilities for verbally expressing cultural information. Consequently, the anthropologist usually finds that only a small number of individuals in any community are good key informants. Some of the capabilities of key informants are systematically developed by the field workers, as they train the informants to conceptualize cultural data in the frame of reference employed by anthropologists. . . . The key informant gradually learns the rules of behavior in a role vis-à-vis the interviewer-anthropologist. (p. 72)

The danger in cultivating and using key knowledgeables is that the researcher comes to rely on them too much and loses sight of the fact that their perspectives are necessarily limited, selective, and biased. Data from informants represent perceptions, not truths. Information obtained from key informants should be clearly specified as such in the field notes so that the researcher’s observations and those of the informants do not become confounded. This may seem like an obvious point, and it is, but over weeks and months of fieldwork, it can be become difficult to decipher what information came from what sources unless the fieldworker has a routine system for documenting sources and uses that system with great discipline, thoroughness and care.

SIDEBAR

KEY INFORMANTS/KEY KNOWLEDGEABLES

Key informants or key knowledgeables are people with in-depth understanding of the setting being studied and willing and able to share what they know as part of the inquiry. Because the term informants is often associated with spies, knowledgeables may be a more appropriate designation. The term informants emphasizes their role in the inquiry. The term knowledgeables emphasizes why they are being engaged in the inquiry.

Here is an excerpt on key informants from the Encyclopedia of Qualitative Research Methods (Given, 2008), which adds the designation “key actors” to the lexicon:

Key informants, or key actors, are individuals who are articulate and knowledgeable about their community. They are often cultural brokers straddling two cultures. This role gives them a special vantage point in describing their culture. Key actors play a pivotal role in the theater of qualitative

research, providing an understanding of cultural norms and responsibilities. Key informants represent an efficient source of invaluable cultural information. It is impossible to interview everyone and observe everything in a community and, logistically, it is easier to work with one or two reliable key informants than it is to assemble a series of focus groups.

Key informants help to establish a link between the researcher and the community. They may provide detailed historical data, photographs, manuscripts, knowledge about interpersonal relationships, a contextual framework in which to observe and interpret behavior, and a wealth of information about the nuances of everyday life. . . . In research on communities and large organizational studies where there is a paucity of relevant archival documentation, particularly concerning vested interests and power dynamics, key informants are especially valuable. . . .

Key informants provide not only personal feelings or opinions, but reflect on larger social patterns as well. They are considered “teachers” by some ethnographic researchers because they impart information, insight, and understanding. However, their insights are rarely accepted blindly. Their views are compared and combined with interviews, observations, and survey data in order to make a complete study. More to the point, key informant and qualitative researchers are collaborators, using questions, answers, and probes to better understand how and why things work. (Fetterman, 2008b, p. 477)

Key informants can be particularly helpful in learning about subgroups to which the observer does not or cannot have direct access. During the first year of the wilderness education program evaluation, as noted previously, two groups emerged on the first day’s hike and gave themselves names—turtles and truckers—to denote their differing styles of hiking. During the second year, with new participants, this history carried forward but took on a different connotation. One informal group, mostly women, adopted the name “turtles” to set themselves apart from participants, mostly men, who wanted to hike at a fast pace, climb the highest peaks, or otherwise demonstrate their prowess—a group they called somewhat disparagingly the “truckers” (trucks being unwelcome in the wilderness). Having had a full year of wilderness experiences the first year of the program and being male, I wasn’t particularly welcome to become an intimate member of the “turtles.” I therefore established an informant relationship with one of the “turtles,” who willingly kept me informed about the details of what went on in that group with the knowledge and permission of the group. Without that key informant relationship, I would have missed some very important information about the kinds of experiences the “turtle” participants were having and the significance of the project to them.

While being part of any setting necessarily involves personal choices about social relationships and political choices about group alliances, the emphasis on making strategic decisions in the field should not be interpreted as suggesting that the conduct of qualitative research in naturalistic settings is an ever-exciting game of chess in which players and pieces are manipulated to accomplish some ultimate goal. Fieldwork certainly involves times of both exhilaration and frustration, but the dominant motifs in fieldwork are hard work, long hours to do both observations and keep up-to-date with field notes, enormous discipline, attention to details, and concentration on mundane and day-to-day work. The routinization of fieldwork is a time of concentrated effort and immersion in gathering data. Alas, let the truth be told: The gathering of field data involves very little glory and an abundance of nose-to-the-grindstone drudgery.

©2002 Michael Quinn Patton and Michael Cochran

Constructivist Rashomon Heaven: Multiple and Diverse Perspectives

MODULE

54 Bringing Fieldwork to a Close

Before you came to live with us, our lives were as always, and we were happy.

We worked, we ate, and then we slept. When you came we were glad, for you brought us many fine gifts. And every night, instead of going to sleep, we sat with you, drank coffee, smoked your tobacco, and listened to your radio.

But now you go, and we are sorry, for all of these things go with you.

We now know pleasures to which we were unaccustomed, and we shall be unhappy. —The headman of Cabrua in 1953

to anthropologists Robert and Yolanda Murphy Placard on display at Harvard’s Peabody Museum (2010)

In traditional scholarly fieldwork within anthropology and sociology, it can be difficult to predict how long fieldwork will last. The major determinant of the length of the fieldwork is the investigator’s own resources, interests, and needs. Evaluation and action research typically have quite specific reporting deadlines, stated in a contract, that affect the duration of and resources available for fieldwork and the intended uses of evaluative findings.

In the previous section, we looked at the many complex relationships that get formed during fieldwork, relationships with key informants, hosts, and “sponsors” in the setting who helped with entry and may have supported ongoing fieldwork, helping solve problems and smoothing over difficulties. In collaborative research, relationships with coresearchers will have deepened. In any extended involvement within a setting, friendships and alliances are formed. As fieldwork comes to an end, an exit or disengagement strategy is needed. While a great deal of attention has traditionally been paid to entering the field, much less attention has been given to the disengagement process, what Snow (1980) has called the “neglected problem in participant observation research.”

One side of the coin is disengagement. The other side is reentry back to one’s life after extended fieldwork or an all-consuming project. When I went to do graduate research in Tanzania, our team received a lot of support and preparation for entry, much of it aimed at avoiding “culture shock.” But when we returned home, we were given no preparation for what it would be like to return to America’s highly commercial, materialistic, and fast-moving culture after months in an agrarian, community-oriented, slower-moving environment. The culture shock hit coming home, not going to Africa.

SIDEBAR

ETHICAL DIMENSIONS OF LEAVING THE FIELD

It is an issue of research ethics how field relationships are terminated. . . .

Power relations between field researcher and those researched may become evident in the expectations of these parties when fieldwork ends. Researchers may have greater access to resources (physical, intellectual, cultural) than those they research. Also, during fieldwork, a number of formal or informal research bargains may have been struck between researcher and participants, such as an agreement not to undermine hierarchies or other commitments concerning behavior or confidentiality. Such research bargains with informants may meet psychosocial needs; for example, to have an interested, unengaged, or neutral ear to listen. Participants may be willing to talk or be observed, so long as their anonymity is respected. Sometimes there will be a more tangible bargain; for instance, to assist in some project or struggle or to provide feedback to support participants’ objectives. When leaving the field, the researcher’s commitment to these bargains needs to be confirmed explicitly so that participants are reassured that their trust and participation in the research are respected and rewarded.

Informal relations may also have developed during fieldwork, including emotional or psychosocial engagements with researchers. These affective relations must be addressed before a researcher leaves a field, and culturally appropriate forms of valediction must be undertaken. Researchers may need to use informants to check what kinds of expectations are held by participants concerning departure. (Fox, 2008, p. 483)

Interpersonal, cross-cultural, disengagement, and reentry issues all deserve attention as fieldwork comes to a close. Relationships with people change and evolve from entry, through the middle days, and up to the end of fieldwork. So does the fieldworker’s relationship with the data and engagement in the inquiry process. That changed engagement in the inquiry process is what I want to focus on here.

Capturing Interpretive Insights While Still in the Field

As you near completion of data gathering, having hopefully become fairly knowledgeable about the setting being observed, more and more attention can be shifted to fine-tuning and confirming the observed patterns. Possible interpretations of and explanations for what was observed show up more in the field notes. Some of these explanations have been offered by others; some occur directly to the observer. In short, analysis and interpretation will have begun even before the observer has left the field.

Chapter 8 discusses analysis strategies at length. At this point, I simply want to recognize the fact that data gathering and analysis flow together in fieldwork, for there is usually no definite, fully anticipated point at which data collection stops and analysis begins. One process flows into the other. As the observer gains confidence in the quality and meaningfulness of the data, becoming more sophisticated about the setting under study and aware that the end draws near, additional data collection becomes increasingly selective and strategic.

Concluding Data Collection Rigorously

As fieldwork draws to a close, the researcher is increasingly concerned with verification of already collected data and less concerned with generating new inquiry leads. While in naturalistic inquiry one avoids imposing preconceived analytical categories on the data, as fieldwork comes to an end, experience with the setting will usually have led to thinking about prominent themes and dimensions that organize what has been experienced and observed. These emergent ideas, themes, concepts, and dimensions—generated inductively through fieldwork—can also now be deepened,

further examined, and verified during the closure period in the field. Bringing closure to fieldwork includes, as a matter of rigor, the search for disconfirming cases as well as confirming cases. This is also when saturation sampling comes into play (see Chapter 5, pp. 300–301), the judgment that no new information is emerging or is likely to emerge from additional data collection.

Guba (1978) described fieldwork as moving back and forth between the discovery mode and the verification mode like a “wave.” The ebb and flow of research involves moving in and out of periods when the investigator is open to new inputs, generative data, and opportunistic sampling and periods when the investigator is testing out hunches, fine-tuning conceptualization, sifting ideas, and verifying explanations.

The Psychology of Ending Fieldwork

When fieldwork has gone well, the observer grows increasingly confident that things make sense and begins to believe in the data. Glaser and Strauss (1967), commenting on grounded theory as an outcome of fieldwork, have described the feelings that the traditional field observer has as fieldwork moves to a close, data-based patterns have emerged, and the whole takes shape.

The continual intermeshing of data collection and analysis has direct bearing on how the research is brought to a close. When the researcher is convinced that his conceptual framework forms a systematic theory, that it is a reasonably accurate statement of the matter studied, that it is couched in a form possible for others to use in studying a similar area, and that he can publish his results with confidence, then he has neared the end of his research. . . .

Why does the researcher trust what he knows? . . . They are his perceptions, his personal experiences, and his own hard-won analyses. A field worker knows that he knows, not only because he has been in the field and because he has carefully discovered and generated hypotheses, but also because “in his bones” he feels the worth of his final analysis. He has been living with partial analyses for many months, testing them each step of the way, until he has built this theory. What is more, if he has participated in the social life of his subject, then he has been living by his analyses, testing them not only by observation and interview but also by daily living. (pp. 224–225)

This representation of bringing a grounded theory inquiry to a close represents the scholarly inquiry ideal. In the contracted deliverables world of program evaluation, with limited time and resources, and reporting schedules that may not permit as much fieldwork as is desirable, the evaluator may have to bring the fieldwork to a close before that state of real confidence has fully emerged. Nevertheless, I find that there is a kind of Parkinson’s law in fieldwork: As time runs out, the investigator feels more and more the pressure of making sense out of things, and some form of order does indeed begin to emerge from the observations. This is a time to celebrate emergent understandings even while retaining the critical eye of the skeptic, especially useful in questioning one’s own confident conclusions.

Feedback to Program Staff as Evaluation Fieldwork and Site Visits End

In doing fieldwork for program evaluation, in contrast to theory-oriented scholarly field research, the evaluator-observer must be concerned about providing feedback, making judgments, and generating recommendations. Thus, as the fieldwork draws to a close, the evaluator must consider what feedback is to be given to whom and how.

Giving feedback to program staff and/or participants can be part of the verification process in fieldwork. My own preference is to provide the participants and staff with descriptions and

analysis, verbally and informally, and to include their reactions as part of the data. Part of the reciprocity of fieldwork can be an agreement to provide participants with descriptive information about what has been observed. I find that participants and staff are hungry for such information and fascinated by it. I also find that I learn a great deal from their reactions to my descriptions and analyses. Of course, it’s neither possible nor wise to report everything one has observed. Moreover, the informal feedback that occurs at or near the end of fieldwork will be different from the findings that are reported formally based on the more systematic and rigorous analysis that must go on once the evaluator leaves the field. But that formal, systematic analysis will take more time, so while one is still in the field, it is possible to share at least some findings and to learn from the reactions of those who hear those findings.

Timing feedback in formative evaluations can be challenging. When the purpose is to offer recommendations to improve the program, the program staff will usually be anxious to get that information “ASAP” (as soon as possible). The evaluator-observer may even feel pressured to report findings prematurely before having confidence in the patterns that seem to be emerging. I experienced this problem throughout the evaluation of the wilderness education program evaluation. During the first year, we met with the staff at the end of each field conference program (the three 10-day field conferences were spread out over a year) to discuss what we had observed and to share interpretations about those observations. At the very first feedback session, the staff reaction was “I wish you’d told us that in the middle of the week, when we could have done something about it. Why’d you hold back? We could have used what you’ve learned to change the program right then and there.”

I tried to explain that the implications of what I observed had only become clear to me an hour or two before our meeting, when my co-evaluator and I had sat down with our field notes, looked them over, and discussed their significance together. Despite this explanation, which struck me as altogether reasonable and persuasive and struck the staff as altogether disingenuous, from that moment forth, a lingering distrust hung over the evaluation as staff periodically joked about when we’d get around to telling them what we’d learned next time. Throughout the three years of the project, the issue of timing feedback surfaced several times a year. As they came increasingly to value our feedback, they wanted it to come earlier and earlier during each field conference. During the second field conference in the second year, when a number of factors had combined to make the program quite different from what the staff had hoped for (major logistical and weather problems, personality conflicts among participants), the end-of-the-conference evaluation feedback session generated an unusual amount of frustration from the staff because my analyses of what had happened and why had not been shared earlier. Again, I found some distrust of my insistence that those interpretations had emerged later rather than sooner as the patterns became clear to me.

Evaluators who provide formative feedback on an ongoing basis need to be conscientious in resisting pressures to share findings and interpretations before they have confidence about what they have observed and sorted out important patterns—not certainty but at least some degree of confidence. The evaluator is caught in a dilemma: Reporting patterns before they are clearly established may lead program staff to intervene inappropriately; withholding feedback too long may mean that dysfunctional patterns become so entrenched that they are difficult, if not impossible, to change.

No ideal balance has ever emerged for me between continuing observations versus providing feedback. Timing feedback is a matter of judgment and strategy, and it depends on the nature of the evaluator’s relationship with program staff and the nature of the feedback, especially the balance between what staff will perceive as negative and positive feedback. When in doubt, and where the relationship between the evaluator and program staff has not stabilized into one of long-term trust, I

counsel evaluator-observers to err on the side of less feedback rather than more. As often happens in social relationships, negative feedback that was wrong is long remembered and often recounted. On the other hand, it may be a measure of the success of the feedback that program staff so fully adopt it that they make it their own and cease to credit the insights of the evaluator.

Once feedback is given, the role of the evaluator changes. Those to whom the feedback was presented are likely to become much more conscious of how their behavior and language is being observed. Thus, added to the usual effect of the fieldworker on the setting being observed, this feedback dimension of fieldwork increases the impact of the evaluator-observer on the setting in which he or she is involved.

Summary of Stages of Fieldwork

Fieldwork is dynamic and fluid. Each inquiry is unique. There are, however, patterns. One prominent pattern is that fieldwork must have a beginning and an ending, with ongoing fieldwork in the middle. Different issues surface at various stages of fieldwork. Exhibit 6.12 summarizes the issues we’ve been discussing at these three stages.

EXHIBIT 6.12 Overview of Stages of Fieldwork

MODULE

55 The Observer and What Is Observed: Unity, Separation, and Reactivity

The question of how the observer affects what is observed has natural as well as social science dimensions. The Heisenberg Uncertainty Principle in physics states that the instruments used to measure velocity and position of an electron alter the accuracy of measurement. When the scientist measures the position of an electron, its velocity is changed; and when velocity is measured, it becomes difficult to capture precisely the electron’s position. The process of observing affects what is observed. These are real effects, not just errors of perception or measurement. The physical world can be altered by the intrusion of the observer. How much more, then, are social worlds changed by the intrusion of fieldworkers?

The effects of observation vary depending on the nature of the observation, the type of setting being studied, the personality and procedures of the observer, and a host of unanticipated conditions. Nor is it simply in fieldwork involving naturalistic inquiry that scientific observers affect what is observed. Experimentalists, survey researchers, cost–benefit analysts, and psychologists who administer standardized tests all affect the situations into which they introduce data collection procedures. The issue is not whether or not such effects occur; rather, the issue is how to monitor those effects and take them into consideration when interpreting data.

A strength of naturalistic inquiry is that the observer is sufficiently a part of the situation to be able to understand personally what is happening. Fieldworkers are called on to inquire into and be reflective about how their inquiry intrudes and how those intrusions affect findings. But that’s not always easy. Consider the case of anthropologist Napoleon Chagnon, who did fieldwork for a quarter-century among the isolated and primitive Yanomami Indians, who lived deep in the rain forest at the borders of Venezuela and Brazil. He studied mortality rates by dispensing steel goods, including axes, as a way of persuading people to give him the names of their dead relatives in violation of tribal taboos. Brian Ferguson, another anthropologist knowledgeable about the Yanomami, believes that Chagnon’s fieldwork destabilized relationships among villages, promoted warfare, and introduced disease. Chagon denies these charges but acknowledges extracting tribal secrets by giving informants gifts like beads and fishhooks, capitalizing on animosities between individuals, and bribing children for information when their elders were not around. He gave away machetes in exchange for blood samples for his genealogical studies. The long-term effects of his fieldwork have become a matter of spirited debate and controversy within anthropology (Geertz, 2001; Tierney, 2000a, 2000b).

SIDEBAR

REACTIVITY: OBSERVER EFFECTS, OBSERVER AFFECTED

Reactivity, also known as the observer effect, takes place when the act of doing research changes the behavior of participants. . . .

A number of factors have been seen to influence the degree of reactivity. Conspicuous observers, or those who place themselves in the middle of the activities, are more intrusive than those who stand to the side. Characteristics of the observer (e.g., age, race, gender, dress) that differ substantially from those of participants are likely to cause more reactivity. Characteristics of participants may

also influence behavior. For example, children usually return to naturally occurring behavior more quickly than do adults. Reactive behavior usually decreases as time passes, a process known as habituation. It is postulated that this return to normality arises from the development of rapport and trust between participants and the researcher and the fact that it is difficult to sustain unnatural behavior for a long period. Participants’ understanding of the purpose of the research may cause reactivity. For example, if participants believe that the researcher is trying to document socially unacceptable or deviant activities, they may hide these behaviors.

In qualitative research, reactivity is usually seen as being inclusive of the researcher as well as the participants. The researcher keeps reflexive notes that document how his or her own behavior and understandings may have been affected by the research process. Reactivity is regarded as being inevitable in any research process that involves interaction among participants, the researcher, and a setting of interest. Reflexive analysis helps to uncover and respond to reactivity in appropriate ways. (McKechnie, 2008, pp. 729–730)

Reactivity in Collaborative Research and Evaluation

At the other end of the intrusion continuum, we find those qualitative designs where “intrusions” are intentionally designed because the qualitative inquiry is framed as an intended form of desired intervention. This is the case, for example, with collaborative and participatory forms of inquiry in which those people in the setting who become coresearchers are expected to learn from the inquiry. The processes of participation and collaboration can be designed and facilitated to have an impact on participants and collaborators quite beyond whatever findings they may generate by working together. In the process of participating, participants have the opportunity to learn the logic of research and data-based reasoning. Skills are acquired in problem identification, criteria specification, data collection, analysis, and interpretation. Acquisition of such research skills and ways of thinking can have a longer-term impact than the use of findings from a particular evaluation study. This learning from the process as an outcome of participatory and collaborative inquiry is called “process use,” in contrast to findings use (Patton, 2008, chap. 5; Patton, 2012a, pp. 140–167).

While it is not possible to know precisely how collaboration will affect coresearchers or to fully anticipate how an observer will affect the setting being observed, both cases illustrate the need to be thoughtful about the interconnections between the observers and the observed. When designing a study and making decisions about the observer’s degree of participation in the setting, the visibility and openness of fieldwork, and the duration of fieldwork (see Exhibit 6.4 earlier in this chapter), it is important to anticipate and be prepared for observation effects—and know how to handle them. For example, I have been involved as a participant-observer-evaluator in a number of professional development programs where participants were expected to exercise increasing control over the curriculum as the program evolved. Had I fully engaged in such participatory decision making, I could have influenced the direction of the program. Anticipating that problem and reviewing the implications with program staff, in each case, I decided not to participate actively in participant-led decision making, at least not to the full extent I might have had I not been involved in the role of evaluator-observer. I aimed my involvement at a level where I would not appear withdrawn from the process, yet at the same time, I attempted to minimize my influence, especially where the group was divided on priorities.

Another example comes from evaluation of a community leadership program mentioned previously in this chapter. As a three-person team of participant-observers, we participated fully in small-group leadership exercises. When the groups in which we participated were using concepts

inappropriately or doing the exercise wrong, we went along with what participants said and did so without offering corrections. Had we really been only participants—and not participant- evaluators—we would have offered corrections and solutions. Thus, our roles made us more passive than we tended naturally to be, so as not to dominate the small groups. We had anticipated this possibility in the design stage prior to fieldwork and had agreed on this strategy at that time.

The Dynamics of Participation

The role and impact of the evaluator-observer can change over the course of fieldwork. Early in the wilderness project, I kept a low profile during participant-led planning discussions. Later in the program, particularly during the final field conference of the second year, I became more engaged in discussions about the future direction of the project.

Methodological Reporting

Reporting on the relationship between the observer and the observed, then, and the ways in which the observer may have affected the phenomenon observed becomes part of the methodological discussion in published fieldwork reports and evaluation studies. In that methodological discussion (or the methods chapter of a dissertation), the observer presents both data about the effects of fieldwork on the setting and people therein as well as the observer’s perspective on what has occurred. As Patricia Carini (1975) has explained, such a discussion acknowledges that findings inevitably are influenced by the observer’s point of view during naturalistic inquiry:

The observer has a point of view that is central to the datum and it is in the articulation—in the revelation of his point of view—that the datum of inquiry is assumed to emerge. In effect the observer is here construed as one moment of the datum and as such the fabric of his thought is inextricably woven into the datum as he is assumed to be constituent of its meaning. From this assumption it is possible to consider the relationship of the observer to the phenomenon under inquiry. Relatedness can be stated in many ways: opposition, identity, proximity, interpenetration, isolation, to name only a few. All imply that the way in which a person construes his relationship to the phenomenal world is a function of his point of view about it. That is, relationship is not a given nor an absolute, but depends upon a personal perspective. It is also true that perspective can shift, the only necessity of a person’s humanity being that he takes some stance in relationship to the events about him. (pp. 8–9)

SIDEBAR

INSIDER VERSUS OUTSIDER PERSPECTIVES: “NOTHING ABOUT US, WITHOUT US”

As an example of reflexive methodological reporting, what follows are ruminations on a unique way of looking at insider (emic) versus outsider (etic) perspectives. In this case, the insider and outsider is the same person but at different times under different conditions. Understanding different perspectives from inside and outside a phenomenon goes to the core of qualitative inquiry. Experience affects perspective. Perspective shapes experience. In the next chapter, on interviewing, we shall continue to explore ways of capturing the experiences and getting deeply into the perspectives of those who have encountered whatever phenomenon interests us.

The reflections that follow look at the experience of mental illness from the inside and outside, and how being on the inside can dramatically change the view from the outside. Janice James

(not her real name) moved from the outside (PhD researcher and mental health professional) to the inside as an involuntary participant-observer (a patient in a locked mental health facility) and back again to the outside (as a professional program evaluator). She recounts those transitions from outside to inside, and inside to outside, and how they have shaped her perspective on research and evaluation.

Outside to Inside, Inside to Outside: Shifting Perspectives

I was a high school and junior high science teacher, a school psychologist and guidance counselor, and finally an educational researcher, before I completed my PhD. Since I had been a scientist of some sort for all of my career life, I was a “natural” for the field of evaluation. I thought like a scientist. I was familiar with the practice of research in biology, bacteriology, field botany and had a special interest in medicine. But through some quirks of fate and personality, I found myself working as a clinician, providing therapy and case management to people with severe mental disorders. Then, after having been gainfully employed my entire adult life, and successfully raising two children, who had now produced one grandchild each, I was forced into the locked, psychiatric ward of a hospital for the third time in my life.

Back at work, after nearly two months in the hospital, I found myself, for the first time, looking at my professional work and reading professional literature with the eyes of one from the other side of the locked doors and medical charts. The irony of my situation was obvious: I treated people like me! Thus began a shift of viewpoint that has radically altered my practice of evaluation in the field of mental health.

First, I had to throw out some grand assumptions. As a scientist, I trusted scientific method and worshipped at the same shrine of true experimental design and random assignment as everyone else. But now I was much more conscious that the “lab rats,” the subjects of our research, literally have minds of their own. Some of the most treasured assumptions of mental health research were looking awfully different from inside the maze. Probably, the most serious is that what I had assumed to be “treatment” from the viewpoint of the researcher now looked like mostly futile efforts that were most often experienced as punishment and threat from the viewpoint of “patient.” I was not asked if I would go into the hospital. I was told I had to. If I got angry at something, someone gave me powerful medications that made me feel like a zombie.

While some of what happened in my hospitalization helped, much of what I experienced made me feel much worse. I was incarcerated, and my jailors looked at me kindly, certain that I was being locked up, strapped to a bed, and injected with medications for my own good. Even though I actually entered the hospital “voluntarily,” the threat of involuntary treatment and permanent damage to my ability to earn a living was the driving force that got me there, and kept me there, and taught me to “make nice” for the staff, lest they refuse to certify me sane and let me free again.

Providers are taught, in all sincerity and good intentions, to act in a kind of parental role toward clients, a benign dictatorship. But their “subjects” are people who have already had their dignity as adults medically removed, their privacy invaded, and the job and relationship underpinnings of American self-esteem destroyed; who are told that they will be sick for life; and who are medicated so they cannot perform sexually or sometimes even read a good book. Is it any wonder that many of them (us) will accept survival as an adequate “quality of life”?

Evaluation designs that test the effects of treatment programs on the individual don’t address the problem of living in a neighborhood where life is stressful and taxis won’t take you home after dark and where the threatening voices might just as likely be real as hallucinated. The provider who is confined to the office and distance of a professional relationship will not know when there is abuse in the home the clients never speak of, because the abuser is someone they love or who controls their money as payee. Mental health workers are put in the role of defenders of the public purse, and then we wonder why clients feel their safety net of services threatened with every dollar they are given or earn and fail to trust their “providers.”

In short, I may apply many of the same ideas, theories, methods, and interpretations as I always did as a program evaluator. But now I always question not just the validity, reliability, and generalizability of the evaluation work itself but also the hidden assumptions that surround it. I will always be seeking to empower those disenfranchised by custom, poverty, and stigma. And I now know that if I want to be part of the solution, rather than part of the problem, I better be sure I know what the experience of different stakeholders really is, not constrained by the limited questions I may think to ask or guided too narrowly by work done in the past. Thus, I have adopted the motto of the people who do not claim the title of “consumer,” because they were not given true choice about treatment when they found themselves pinned with psychiatric labels: “Nothing about us, without us.“

Carini (1975) is here articulating the interdependence between the observer and what is observed. Prior to data collection, the fieldworker plans and strategizes about the hoped-for and expected nature of that interdependence. But things don’t always unfold as planned, so observers must make an effort to observe themselves observing and record the effects of their observations on the people observed and, no less important, reflect on changes they’ve experienced from having been in the setting. This means being able to balance observation with reflection and manage the tension between engagement and detachment.

Balancing Engagement and Detachment

Bruyn (1966), in his classic work on participant observation, articulated a basic premise of participant observation: that the “role of the participant observer requires both detachment and personal involvement” (p. 14). To be sure, there is both tension and ambiguity in this premise. How it plays out in any given situation will depend on both the observer and the phenomenon being observed.

Thus, we may observe at the outset that while the traditional role of the scientist is that of a neutral observer who remains unmoved, uncharged, and untouched in his examination of phenomena, the role of the participant observer requires sharing the sentiments of people in social situations; as a consequence he himself is changed as well as changing to some degree the situation in which he is a participant. . . . The effects are reciprocal for observer and observed. The participant observer seeks, on the one hand, to take advantage of the changes due to his presence in the group by recording these changes as part of his study, and on the other hand, to reduce the changes to a minimum by the manner in which he enters into the life of the group. (Bruyn, 1966, p. 14)

Whether one is engaged in participant observation or onlooker observation, what happens in the setting being observed will, to some extent, be dependent on the role assumed by the observer.

Likewise, the nature of the data collected will, to some extent, be dependent on the role and perspective of the observer. And just as the presence of the observer can affect the people observed, so too can the observer be affected.

The Personal Experience of Fieldwork

The intersection of social science procedures with individual capabilities and situational variation is what makes fieldwork a highly personal experience. At the end of her book on Doing Field Work, Wax (1971) reflected on how fieldwork had changed her.

SIDEBAR

CHANCE FAVORS THE PREPARED MIND AND OBSERVANT EYE

In 1949, an obscure Australian psychiatrist, John F. J. Cade, noticed that the urine of his manic patients was highly toxic to guinea pigs, and he began looking for the toxic chemical, which he suspected was uric acid. He began experimenting with lithium urate, not because of any psychiatric properties of lithium but because lithium urate was the most soluble salt of uric acid. To Cade’s surprise, far from being toxic, the salt protected the guinea pigs against the urine of manics, and it also sedated the animals, effects Cade found were due to the lithium. He immediately tried other lithium salts on himself and, when they proved safe, on 10 hospitalized manic patients, all of whom recovered, some almost miraculously.

Cade’s discovery is sometimes characterized as serendipitous, but the discovery of lithium as an antimanic agent resulted from one man’s deep commitment, diligent observation, and reasoned inference (Ironside, 1993).

A colleague has suggested that I reflect on the extent to which I was changed as a person by doing field work. I reflected and the result astonished me. For what I realized was that I had not been greatly changed by the things I suffered, enjoyed or endured; nor was I greatly changed by the things I did (though they strengthened my confidence in myself). What changed me irrevocably and beyond repair were the things learned. More specifically, these irrevocable changes involved replacing mythical or ideological assumptions with the correct (though often painful) facts of the situation. (p. 363)

Fieldwork is not for everyone. Some, like Henry James, will find that “innocent and infinite are the pleasures of observation.” Others will find observational research anything but pleasurable. Some students have described their experiences to me as tedious, frightening, boring, and “a waste of time,” while others have experienced challenge, exhilaration, personal learning, and intellectual insight. More than once, the same student has experienced both the tedium and the exhilaration, the fright and the growth, the boredom and the insight. Whatever the adjectives used to describe any particular individual’s fieldwork, of this much we are assured: The experience of observing provides the observer with experiences and observations, the interconnection being cemented by reflection. No less an authority than William Shakespeare gives us this assurance.

Armado. How hast thou purchased this experience?

Moth. By my penny of observation.

—William Shakespeare Love’s Labor Lost

Apart of and Apart From the World Observed

The personal, perspective-dependent nature of observations can be understood as both a strength and weakness, a strength in that personal involvement permits firsthand experience and understanding, and a weakness in that personal involvement introduces selective perception. In the deep engagement of naturalistic inquiry lies both its risks and its benefits. Reflection on that engagement, from inside and outside the phenomenon of interest, crowns fieldwork with reflexivity and makes the observer the observed—even if only by oneself. So we repeat Halcolm’s refrain that opened this chapter:

Go out into the world. Live among the peoples of the world as they live. Learn their language. Participate in their rituals and routines. Taste the world. Smell it. Watch and listen. Touch and be touched. Write down what you see and hear, how they think and how you feel.

Enter into the world. Observe and wonder. Experience and reflect. To understand a world you must become part of that world while at the same time remaining separate, a part of and apart from.

Go then, and return to tell what you see and hear, what you learn, and what you come to understand.