U8D1-68 - Data Analysis Strategies Reviewed by Peers - Please follow the instructions as outlined below. See attachments for readings.

Language Differences

The data from interviews are words. It is tricky enough to be sure what a person means when using a common language, but words can take on a very different meaning in other cultures. In Sweden, I participated in an international conference discussing policy evaluations. The conference was conducted in English, but I was there two days, much of the time confused, before I came to understand that their use of the term policy corresponded to my American use of the term program. I interpreted policies from an American context, to be fairly general directives, often very difficult to evaluate because of their vagueness. In Sweden, however, policies were articulated and even legislated at such a level of specificity that they resembled programmatic prescriptions more than the vague policies that typically emanate from the legislative process in the United States.

The situation becomes more precarious when a translator or interpreter must be used. Special and very precise training of translators is critical. Translators need to understand what, precisely, you want them to ask and that you will need full and complete translation of responses as verbatim as possible. Interpreters often want to be helpful by summarizing and explaining responses. This contaminates the interviewee’s actual response with the interpreter’s explanation to such an extent that you can no longer be sure whose perceptions you have—the interpreter’s or the interviewee’s.

Some words and ideas simply can’t be translated directly. People who regularly use the language come to know the unique cultural meaning of special terms. One of my favorites from the Caribbean is liming, meaning something like hanging out, just being, doing nothing—guilt free. In interviews for a Caribbean program evaluation, a number of participants said that they were just “liming” in the program. That was not meant as criticism, however, for liming is a highly desirable state of being, at least to the participants. Funders viewed the situation somewhat differently.

Rheingold (2000) published a whole book on “untranslatable words and phrases” with special meanings in other cultures. Below are four examples that are especially relevant to researchers and evaluators.

1. Schlimmbesserung (German): A so-called improvement that makes things worse

2. bigapeula (Kiriwina, New Guinea): Potentially disruptive, unredeemable true statements

3. animater (French): A word of respect for a person who can communicate difficult concepts to general audiences

4. ta (Chinese): To understand things and thus take them lightly

In addition to the possibility of misunderstandings, there may be the danger of contracting a culturally specific disease, including for some what the Chinese call koro—“the hysterical belief that one’s penis is shrinking” (Rheingold, 1988, p. 59).

Attention to language differences cross-nationally can, hopefully, make us more sensitive to barriers to understanding that can arise even among those who speak the same language. Joyce Walker undertook a collaborative study with 18 women who had written to each other annually for 25 years, from 1968 to 1993. She involved them actively in the study, including having them confirm the authenticity of her findings. In reviewing the study’s findings, one participant reacted to the research language used: “Why call us a cohort? There must be something better—a group, maybe?” (Walker, 1996, p. 10).

Differing Norms and Values

The high esteem in which science is held has made it culturally acceptable in Western countries to conduct interviews on virtually any subject in the name of scholarly inquiry and the common good. Such is not the case worldwide. Researchers cannot simply presume that they have the right to ask intrusive questions. Many topics discussed freely in Western societies are taboo in other parts of the world. I have experienced cultures where it was simply inappropriate to ask questions to a subordinate about a superordinate. Any number of topics may be insensitive to ask or indelicate if brought up by strangers, for example, family matters, political views, who owns what, how people came to be in certain positions, and sources of income.

Interviewing farmers for an agricultural extension in Central America became nearly impossible to do because for many their primary source of income came from growing illegal crops. In an African dictatorship, we found that we could not ask about “local” leadership because the country could have only one leader. Anyone taking on or being given the designation “local leader” would have been endangered. Interviewees can be endangered by insensitive and inappropriate questions; so can naive interviewers. I know of a case where an American female student was raped following an evening interview in a foreign country because the young man interpreted her questions about local sexual customs and his own dating experiences as an invitation to have sex.

EXHIBIT 7.15 Ten Examples of Variations in Cross-Cultural Norms That Can Affect Interviewing and Qualitative Fieldwork

As noted in the previous section on group interviews, different norms govern cross-cultural interactions. I remember going to an African village to interview the chief and finding the whole village assembled. Following a brief welcoming ceremony, I asked if we could begin the interview. I expected a private, one-on-one interview. He expected to perform in front of and involve the whole village. It took me a while to understand this, during which time I kept asking to go somewhere else so we could begin the interview. He did not share my concern about and preference for privacy. What I expected to be an individual interview soon became a whole village group dialogue.

In many cultures, it is a breach of etiquette for an unknown man to ask to meet alone with a woman. Even a female interviewer may need the permission of a husband, brother, or parent to interview a village woman. A female colleague created a great commotion, and placed a woman in jeopardy, by pursuing a personal interview without permission from the male headman. Exhibit 7.15 lists some of the variations in cultural behaviors that can affect cross-cultural interviewing and qualitative fieldwork.

The value of Cross-Cultural Interviewing

As difficult as cross-cultural interviewing may be, it is still far superior to standardized questionnaires for collecting data from nonliterate villagers. Salmen (1987) described a major water project undertaken by The World Bank based on a needs assessment survey. The project was a failure because the local people ended up opposing the approach used. His reflection on the project’s failure includes a comparison of survey and qualitative methods.

Although it is difficult to reconstruct the events and motivation that led to the rejection there is little question that a failure of adequate communication between project officials and potential beneficiaries was at least partly responsible. The municipality’s project preparation team had conducted a house-to- house survey in Guasmo Norte before the outset of the project, primarily to gather basic socioeconomic data such as family size, employment and income. The project itself, however, was not mentioned at this early stage. On the basis of this survey, World Bank and local officials had decided that standpipes would be more affordable to the people than household connections. It now appears, from hindsight, that the questionnaire survey method failed to elicit the people’s negative attitude toward standpipes, their own criterion of affordability, or the opposition of their leaders who may have played on the negative feelings of the people to undermine acceptance of the project. Qualitative interviews and open discussions would very likely have revealed people’s preferences and the political climate far better than did the preconstructed questionnaire. (Salmen, 1987, pp. 37–38)

Appropriately, Salmen’s book is called Listen to the People and advocates qualitative methods for international project evaluation of development efforts.

Interviewers are not in the field to judge or change values and norms. Researchers are there to understand the perspectives of others. Getting valid, reliable, meaningful, and usable information in cross-cultural environments requires special sensitivity to and respect for differences, including concerns about decolonizing research in cross-cultural contexts (Denzin & Lincoln, 2011, pp. 92 –93; L. T. Smith, 2012; Mutua & Swadener, 2011) and cultural competence (American Evaluation Association, 2013). Moreover, specific challenges emerge in specific countries, as Bubaker, Balakrishnan, and Bernadine (2013) demonstrated when conducting quality studies in Libya and Malaysia. (For additional discussion and examples of cross-cultural research and evaluation, see Ember & Ember, 2009; Lonner & Berry, 1986; Patton, 1985a.)

One final observation on international and cross-cultural evaluations may help emphasize the value of such experiences. Connor (1985) found that doing international evaluations made him

more sensitive and effective in his domestic evaluation work. The heightened sensitivity we expect to need in exotic, cross-cultural settings can serve us well in our own cultures. Sensitivity to and respect for other people’s values, norms, and worldviews is as much needed at home as abroad.

SIDEBAR

CROSS-CULTURAL PERSPECTIVES ON DEVELOPMENT

The titles of these books communicate the challenges and importance of generating cross- cultural understanding and appropriate action around the world. These books also illustrate the global scale at which mixed-methods inquiries are now being done.

• Can Anyone Hear Us? Voices of the Poor

The voices of more than 40,000 poor women and men in 50 countries from the World Bank’s participatory poverty assessments (Narayan, Patel, Schafft, Rademacher, & Koch- Schulte, 2000)

• Crying Out for Change: Voices of the Poor

Fieldwork on poverty conducted in 23 countries (Narayan, Chambers, Shah, & Petesch, 2000)

• Voices of the Poor: From Many Lands

Regional patterns of poverty and country case studies (Narayan & Petesch, 2002)

MODULE

62 Creative Modes of Qualitative Inquiry

Creativity is just connecting things. When you ask creative people how they did something, they feel a little guilty because they didn’t really do it, they just saw something. It seemed obvious to them after a while. That’s because they were able to connect experiences they’ve had and synthesize new things.

—Steve Jobs (1955–2011) Apple Computer founder

We have looked in some depth at various kinds of standard qualitative interviewing approaches: conversational interviews during fieldwork; in-depth, open-ended, one-on-one formal interviews; interactive, relationship-based interviews; focus group interviews; and cross-cultural interviews. These represent mainstream qualitative interviewing approaches. Now, we move beyond these well-known and widely used interviewing approaches to consider innovative, creative, and pioneering approaches to qualitative interviewing.

Photo Elicitation Interviewing

Photo elicitation involves using photographs to stimulate reflections, support memory recall, and elicit stories as part of interviewing. Photo elicitation methods are proving powerful supplements in in-depth interviewing. For example, to study the meaning of change in dairy farming in northern New York, sociologist Douglas Harper (2008) showed elderly farmers photographs from the 1940s (a period when they had been teenage or young adult farmers) and asked them to remember events, stories, or commonplace activities that the photos brought to mind. He used archived documentary photographs from the era that the elderly farmers had experienced at the beginnings of their careers. The photographs inspired “detailed and often deep memories.”

The farmers described the mundane aspects of farming, including the social life of shared work. But more important, they explained what it meant to have participated in agriculture that had been neighbor based, environmentally friendly, and oriented toward animals more as partners than as exploitable resources.

In this and other photo elicitation studies, photographs proved to be able to stimulate memories that word-based interviewing did not. The result was discussions that went beyond “what happened when and how” to themes such as “this was what this had meant to us as farmers.” (pp. 198–199)

Visual data generally are becoming increasingly important in qualitative research and evaluation (Azzam & Evergreen, 2013a, 2013b; Banks, 2007; Prosser, 2011). A classic example by Hedy Bach (1998) involved documenting the daily life of four schoolgirls who, in addition to regular schoolwork, engaged in art, drama, ballet, and music programs in and out of school. Using disposable cameras, the students visually documented their lives both inside and outside classrooms. Their photos served as the basis for one-on-one interviews with them about their

images. The analysis of the images and interviews revealed an “evaded curriculum” within adolescent life, exposing pain, pleasure, and the intensity of joy in making and creating schoolgirl culture.

Using photos as part of interviewing can reduce

the awkwardness that an interviewee may feel from being put on the spot and grilled by the interviewer . . . ; direct eye contact need not be maintained, but instead interviewee and interviewer can both turn to the photographs as a kind of neutral third party. Awkward silences can be covered as both look at the photographs, and in situations where the status difference between interviewer and interviewee is great (such as between an adult and a child) or where the interviewee feels they are involved in some kind of test, the photographic content always provides something to talk about. (Banks, 2007, pp. 65–66)

Lorenz (2011) used photo elicitation as one way to generate empathy in research with acquired brain injury survivors. Brain injury patients can experience

a lack of empathy that leads to feelings of being disrespected and powerless. . . . Practicing empathy by using photos to create discursive spaces in research relationships may help us to learn about ourselves as we learn with patients. (p. 259)

As powerful as photography is, a picture being legendarily worth a thousand words, visual methods have limitations, as do all methods. Qualitative methodologists debate “the myth of photographic truth,” recognizing that “photographs represent a highly selected sample of the ‘real’ world” (Prosser, 2011, p. 480). Seeing is not necessarily believing, as photos can be manipulated, distorted, and removed from context (Morris, 2011). In this regard, visual methods do not stand alone but are most credible and useful when used in conjunction with other data: interviews, direct observations, and supporting documents.

Video Elicitation Interviewing

Teachnological developments have increased access to video and lowered costs making video elicitation interviewing and storytelling practical as well as innovative—the frontier of visual qualitatiuve inquiry. During video elicitation interviews, researchers interview using video to stimulate responses. An example is using video elicitation interviews for investigating physician –patient interactions (Henry & Fetters, 2012). Patients or physicians were interviewed about a recent clinical interaction using a video recording of that interaction as an elicitation tool.

Video elicitation is useful because it allows researchers to integrate data about the content of physician–patient interactions gained from video recordings with data about participants’ associated thoughts, beliefs, and emotions gained from elicitation interviews. This method also facilitates investigation of specific events or moments during interactions. Video elicitation interviews are logistically demanding and time-consuming, and they should be reserved for research questions that cannot be fully addressed using either standard interviews or video recordings in isolation. As many components of primary care fall into this category, high-quality video elicitation interviews can be an important method for understanding and improving physician –patient interactions in primary care (Henry & Fetters, 2012, p. 118).

Nick Agafanoff (2014) calls the use of video real ethnography—using documentary film techniques to capture and communicate stories and narratives. An example of the kind of experimentation going on in video interviewing is the approach of award-winning documentary

filmmaker Errol Morris (The Thin Blue Line). He invented the “interrotron” to increase rapport and deepen eye contact when videotaping interviews. Morris believes that Americans are so comfortable with television sets that doing his interviews through a television enhances rapport. Morris asks his questions through a specially designed video camera in the same room with the interview subject. The interviewee sees and hears him on a television and responds by talking into the television. Morris, in turn, watches the interview live on TV. The whole interaction takes place face-to-face through televisions placed at right angles to each other in the same room. For the results of this approach, see Morris (2000, 2003, 2012).

Writing as Preparation for or as Part of the Interview

The subject–object interview methodology illustrates another basis for interviewing—including writing as part of the interview. Prior to interviewing the research participants about the 10 ideas, they are given 15 to 20 minutes to jot down things on the index cards. They subsequently choose which cards to talk about and can use their jottings to facilitate their verbal responses. Such an approach gives interviewees a chance to think through some things before responding verbally.

In reflective practice group interviews (Patton, 2011, pp. 266–269), participants are asked to come with stories written that address the focused question for the session:

• Tell about an effective collaboration you have experienced.

• Tell about a time you felt excluded from a group.

• Tell about a time when you overcame a huge obstacle.

Writing the stories in advance facilitates documenting the participants’ experiences and helps the group move more quickly to identifying patterns and themes across their stories. The process of asking and clarifying questions about one another’s stories deepens and enriches the stories.

Critical incidents are a revealing focus for reflective practice with groups. Interviewees can be invited to identify and write about incidents they consider “critical” to their own development, their family’s history, or the work of their organization. Dialogue ensues to understand what made the incident “critical.”

Projection Elicitation Techniques

Projection techniques are widely used in psychological assessment to gather information from people. The best-known projective test is probably the Rorschach Test. The general principle involved is to have people react to something other than a question—an inkblot, a picture, a drawing, a photo, an abstract painting, a film, a story, a cartoon, or whatever is relevant. This approach is especially effective in interviewing children, but it can be helpful with people of any age. I found, for example, when doing follow-up interviews two years after completion of a program, that some photographs of the program site and a few group activities greatly enhanced recall.

Students can be interviewed about work they have produced. In the wilderness program evaluation, we interviewed participants about the entries they shared from their journals. Walker (1996) used letters exchanged between friends as the basis for her study of a generation of

American women. Holbrook (1996) contrasted official welfare case records with a welfare recipient’s journals to display two completely different constructions of reality. Hamon (1996) used proverbs, stories, and tales as a starting point for her inquiry into Bahamian family life. Rettig, Tam, and Magistad (1996) extracted quotes from transcripts of public hearings on child support guidelines as a basis for their fieldwork. Laura Palmer (1988) used objects left in memory of loved ones and friends at the Vietnam War Memorial in Washington, D.C., as the basis for her inquiry and later interviews. An ethnomusicologist will interview people as they listen and react to recorded music. The options for creative interviewing stretch out before us like an ocean teeming with myriad possibilities, some already known, many more waiting to be discovered or created.

Robert Kegan (1982) and colleagues (Helsing, Howell, Kegan, & Lahey, 2008) have had success basing interviews on reactions to 10 words in what they call the subject–object interview. To understand how the interviewee organizes interpersonal and intrapersonal experience, real-life situations are elicited from a series of 10 uniform probes. The interviewee responds to 10 index cards, each listing an idea, concept, or emotion: Angry, Success, Sad, Moved/Touched, Change, Anxious/Nervous, Strong Stand/Conviction, Torn, Lost Something, and Important To Me.

Reactions to these words provide data for the interviewer to explore the interviewee’s underlying epistemology or “principle of meaning-coherence” based on Kegan’s work The Evolving Self (1982). The subject–object interview is a complex and sophisticated method that requires extensive training for proper application and theoretical interpretation. For my purposes, the point is that a lengthy and comprehensive interview interaction can be based on reaction to 10 deceptively simple ideas presented on index cards rather than fully framed questions.

Think-Aloud Protocol Interviewing

“Protocol analysis” or, more literally, the “think-aloud protocol” approach, aims to elicit the inner thoughts or cognitive processes that illuminate what’s going on in a person’s head during the performance of a task, for example, painting or solving a problem. The point is to undertake interviewing as close to the action as possible. While someone engages in an activity, the interviewer asks questions and probes to get the person to talk about what he or she is thinking as he or she does the task. In “teaching rounds” at hospitals, senior physicians do a version of this when they talk aloud about how they’re engaging in a diagnosis while medical students listen, presumably learning the experts’ thinking processes by hearing them in action. (For details of the think-aloud protocol method, see Ericsson & Simon, 1993; Krahmer & Ummelen, 2004; Pressley & Afflerbach, 1995.)

Wilson (2000) used a protocol research design in a doctoral dissertation that investigated student understanding and problem solving in college physics. Twenty students in individual 45-minute sessions were videotaped and asked to talk aloud as they tried to solve three introductory physics problems of moderate difficulty involving Newton’s second law. Wilson was able to pinpoint the cognitive challenges that confronted the students as they tried to derive the acceleration of a particle moving in various directions and angles with respect to a particular reference frame.

Real-Time Qualitative Data Collection

The experience sampling method was developed to collect information on people’s reported feelings in real time in natural settings during selected moments of the day. Participants in

experience sampling studies carry a handheld computer that prompts them several times during the course of the day (or days) to answer a set of questions immediately. They indicate their physical location, the activities in which they were engaged just before they were prompted, and the people with whom they were interacting. They also report their current subjective experience by indicating their feelings (Kahneman & Krueger, 2006, p. 9).While the experience sampling method as originally developed had participants respond on quantitative scales, the pervasive use of handheld devices in contemporary society means that qualitative data (e.g., texts, tweets) can be collected with this technique.

Creative Interviewing

As the preceding examples illustrate, qualitative inquiry need not be confined to traditional written interview protocols and taking field notes. Researchers and evaluators have considerable freedom to creatively adapt qualitative methods to specific situations and purposes using anything that comes to mind—and works—as a way to enter into the world and worldview of others. Not only are there many variations in what stimuli to use and how to elicit responses, but creative variations also exist for who conducts interviews. We turn now to those options.

SIDEBAR

PERFORMING THE INTERVIEW

Interviews are performance texts. A performative social science uses the reflexive, active interview as a vehicle for producing moments of performance theater, a theater that is sensitive to the moral and ethical issues of our time. This interview form is gendered and dialogic. In it, gendered subjects are created through their speech acts. Speech is performative. It is action. The act of speech, the act of being interviewed, becomes a performance itself. The reflexive interview, as a dialogic conversation, is the site and occasion for such performances; that is, the interview is turned into a dramatic, poetic text. In turn, these texts are performed, given dramatic readings.

—Norman K. Denzin (2003, p. 84)

Performance Ethnography

Collaborative and Participatory Interviewing

Chapter 4, “Practical and Actionable Qualitative Applications,” discussed conducting research and evaluation in a collaborative mode in which professionals and nonprofessionals become co- inquirers. Participatory approaches are widely used in community research, health research, educational evaluation, and organizational development. For example, participatory action research encourages collaboration within a mutually respectful inquiry relationship to understand and/or solve organizational or community problems. Participatory evaluation involves program staff and participants in all aspects of the evaluation process to increase evaluation understanding and use while also building capacity for future inquiries. Empowerment evaluation aims to foster self- determination among those who participate in the inquiry process. This can involve forming empowerment collaborations between researchers and participants and teaching participants to do research themselves. Feminist methods are often highly participatory (Ackerly & True, 2010;

Bamberger & Podems, 2002; Brisolara, Seigart, & SenGupta, 2014, Hesse-Biber, 2011; Hesse- Biber & Leavy, 2006b; Hughes, 2002; Mertens, 2005; Olesen, 2011; Podems, 2014b).

In-depth interviewing is especially useful for supporting collaborative inquiry because the methods are accessible to and understandable by nonresearchers and community-based people without much technical expertise. Exhibit 4.13 (p. 222) presents Principles of Fully Participatory and Genuinely Collaborative Inquiry. The following sections provide some specific examples of collaborative and participatory interviewing approaches.

SIDEBAR

PARTICIPATORY ACTION RESEARCH

Participatory action research is the sum of its individual terms, which have had and continue to have multiple combinations and meanings, as well as a particular set of assumptions and processes. . . . Participation is a major characteristic of this work, not only in the sense of collaboration, but in the claim that all people in a particular context (for both epistemological and, with it, political reasons) need to be involved in the whole of the project undertaken. Action is interwoven into the process because change, from a situation of injustice toward envisioning and enacting a “better” life (as understood from those in the situation) is a primary goal of the work. Research as a social process of gathering knowledge and asserting wisdom belongs to all people, and has always been part of the struggle toward greater social and economic justice locally and globally.

—Brydon-Miller, Kral, Maguire, Noffke, and Sabhlok (2011, p. 388)

Jazz and the Banyan Tree: Roots and Riffs on Participatory Action Research

Participant Interview Chain

As a participant-observer in the wilderness training program for adult educators, I was involved in (a) documenting the kinds of experiences program participants were having and (b) collecting information about the effects of those experiences on the participants and their work situations. In short, the purpose of the evaluation was to provide formative insights that could be used to help understand the personal, professional, and institutional outcomes of intense wilderness experiences for these adult educators. But the two of us doing the evaluation didn’t have sufficient time and resources to track everyone—40 people—in depth. Therefore, we began discussing with the program staff ways in which the participants might become involved in the data collection effort to meet both program and evaluation needs. The staff liked the idea of involving participants, thereby introducing them to observation and interviewing as ways of expanding their own horizons and deepening their perceptions.

The participants’ backpacking field experience was organized in two groups. We used this fact to design a data collection approach that would fit with the programmatic needs of sharing information between the two groups. Participants were paired to interview each other. At the very beginning of the first trip, before people knew each other, all of the participants were given a short, open-ended interview of 10 questions. They were told that each of them, as part of their project participation, was to have responsibility for documenting the experiences of their pairmate throughout the year. They were given a little bit of interview training and a lot of encouragement

about probing and were told to record responses fully, thereby taking responsibility for helping build this community record of individual experiences. They were then sent off in pairs and given two hours to complete the interviews with each other, recording the responses by hand.

At the end of the 10-day experience, when the separate groups came back together, the same pairs of participants, consisting of one person from each group, were again given an interview outline and sent off to interview each other about their respective experiences. This served the program need of sharing of information and an evaluation need of collection of information. The trade-off, of course, was that with the minimal interview training given to the participants and the impossibility of carefully supervising, controlling, and standardizing the data collection, the results were of variable quality. This mode of data collection also meant that confidentiality was minimal and certain kinds of information might not be shared. But we gathered a great deal more data than we could have obtained if we had had to do all the interviews ourselves.

Exhibit 7.16 presents an example of participatory inquiry in a study of a program aimed at helping women engaged in prostitution transition into a new life. The program participants, women who had themselves been prostitutes, were trained to conduct the evaluation interviews.

EXHIBIT 7.16 Training Nonresearchers as Focus Group Interviewers: Women Leaving Prostitution

Rainbow Research studied the feasibility of developing a transitional housing program for prostituted women. To assist us we recruited 5 women who had been prostituted, trained them in focus group facilitation and had them do our interviews with women leaving prostitution. For them the experience was empowering and transformational. They were excited about learning a new skill, pleased to be paid for this work and thought it rewarding that it might benefit prostituted women. Especially thrilling for them during the interviews was the validation and encouragement they received from their peers for the work they were doing. Our work together also had its light moments. During a group simulation, our interviewers loudly and provocatively bantered with one another as they might have on the street.

Our interviewers were proud of their contribution. At project’s end they requested certificates acknowledging the training they had received and the interviews successfully performed. And, because they had performed well, we were pleased to oblige. In the simulations they critiqued our interview guide, leading us to edit the language, content, order and length, introduce new questions and drop others. Clearly they had rapport with their peers based on shared discourse and experience, allowing them to gather information others without the experience of prostitution would have been hard pressed to secure. This was apparent in the reliability of our data. Comparing across the interviews, responses to the same items were highly consistent. For all concerned it was a positive experience, with findings that most definitely shaped our final recommendations.

—Barry B. Cohen

Executive Director, Rainbow Research

Minneapolis, Minnesota

Limitations certainly exist as to how far one can push participant involvement in data collection and analysis without interfering in the program or burdening participants. But before those limits are reached, a considerable amount of useful information can be collected by involving program participants in the actual data collection process. I have since used similar participant interview

pairs in a number of program evaluations with good results. The trick is to integrate the data collection into the program. Such cooperative and interactive interviewing can deepen the program experience for those involved by making qualitative data collection and reflection part of the change process (Cousins, Whitmore, & Shulha, 2014; Fetterman, Rodríguez-Campos, Wandersman, & O’Sullivan, 2014; King & Stevahn, 2013; Patton, 2011, 2012a).

Photovoice

Earlier, I discussed photo elicitation approaches to enhance interviewing. Incorporating photography into data collection can also be a highly participatory process. The best-known and most widely used participatory photo elicitation approach is Photovoice, which combines photography and qualitative inquiry with grassroots social action (see sidebar). The Minnesota Department of Health (2013) sponsored a Photovoice project in which youth from 13 communities were trained in Photovoice and given cameras to answer questions about their everyday lives through photography.

SIDEBAR

PHOTOVOICE

Photovoice combines photography and qualitative inquiry with grassroots social action. Community participants present their experiences and perspectives by taking photographs, developing narratives to go with their photos, analyzing them together, and using the results to advocate for change with policymakers and philanthropic funders. Photovoice, as a form of qualitative participatory photography, was developed by Caroline C. Wang and Mary Ann Burris while working in Beijing, China, in 1992. They sought a way for rural women of Yunnan Province, China, to tell their stories and influence the policies and programs that affected them (Wang & Burris, 1994). It has evolved into a global movement to combine visual evidence, qualitative narratives, and social change (http://www.photovoice.org/).

Data Collection by Program Staff

Program staff constitute another resource for data collection that is often overlooked. Involving program staff in data collection raises objections about staff subjectivity, data contamination, loss of confidentiality, the vested interests of staff in particular kinds of outcomes, and the threat that staff can pose to clients or students from whom they are collecting the data. Balancing these objections are the things that can be gained from staff involvement in data collection: (a) greater staff commitment to the evaluation, (b) increased staff reflectivity, (c) enhanced understanding of the data collection process that comes from training staff in data collection procedures, (d) increased understanding by staff of program participants’ perceptions, (e) increased data validity because of staff rapport with participants, and (f) cost savings in data collection.

One of my first evaluation experiences involved studying a program to train teachers in open education at the University of North Dakota. The faculty were interested in evaluating that program, but there were almost no resources available for a formal evaluation. Certainly not

enough funds existed to bring in an external evaluation team to design the study, collect data, and analyze the results. The main means of data collection consisted of in-depth interviews with student teachers in 24 different schools and classrooms throughout North Dakota and structured interviews with 300 parents who had children in those classrooms. The only evaluation monies available would barely pay for the transportation and the actual mechanical costs of data collection. Staff and students at the university agreed to do the interviews as an educational experience. I developed structured interview forms for both teacher and parent interviews and trained all of the interviewers in a full-day session. Interviewers were assigned to geographical areas, making sure that no staff collected data from their own student teachers. The interviews were recorded and transcribed. I did follow-up interviews with a 5% sample as a check on the validity and reliability of the student and staff data.

After data collection, seminars were organized for staff and students to share their personal perceptions based on their interview experiences. Their stories had considerable impact on both staff and students. One outcome was the increased respect both staff and students had for the parents. They found the parents to be perceptive, knowledgeable, caring, and deeply interested in the education of their children. Prior to the interviewing, many of the interviewers had held fairly negative and derogatory images of North Dakota parents. The systematic interviewing had put them in a situation where they were forced to listen to what parents had to say, rather than telling parents what they (as educators) thought about things, and in learning to listen, they had learned a great deal. The formal analysis of the data yielded some interesting findings that were used to make some changes in the program, and the data provided a source of case materials that were adapted for us in training future program participants, but it is likely that the major and most lasting impact of the evaluation came from the learnings of students and staff who participated in the data collection. That experiential impact was more powerful than the formal findings of the study—an example of “evaluation process use” (Patton, 2008, 2012a), using an evaluation process for participant and organizational learning.

By the way, had the interviewers been paid at the going commercial rate, the data collection could have cost at least $30,000 in personnel expenses. As it was, there were no personnel costs, and a considerable human contribution was made to the university program by both students and staff.

Such participatory action research remains controversial. As Kemmis and McTaggart (2000) noted in their extensive review of participatory approaches,

In most action research, including participatory action research, the researchers make sacrifices in methodological and technical rigor in exchange for more immediate gains in face validity: whether the evidence they collect makes sense to them in their context. For this reason, we sometimes characterize participatory action research as “low-tech” research: it sacrifices in methodological sophistication in order to generate timely evidence that can be used and further developed in a real-time process of transformation (of practices, practitioners, and practice settings). (p. 591)

Whether some loss of methodological sophistication is merited depends on the primary purpose of the inquiry and the primary intended users of the results. Participatory research will have lower credibility among external audiences, especially among scholars who make rigor their primary criterion for judging quality. Participants involved in improving their work or lives, however, lean toward pragmatism where what is useful determines what is true. As Kemmis and McTaggart (2000) conclude,

The inevitability—for participants—of having to live with the consequences of transformation provides a very concrete “reality check” on the quality of their transformative work, in terms of whether their

practices are more efficacious, their understandings are clearer, and the settings in which they practice are more rational, just, and productive of the kinds of consequences they are intended to achieve. For participants, the point of collecting compelling evidence is to achieve these goals, or, more precisely, to avoid subverting them intentionally or unintentionally by their action. Evidence sufficient for this kind of “reality checking” can often be low-tech (in terms of research methods and techniques) or impressionistic (from the perspective of an outsider who lacks the contextual knowledge that the insider draws on in interpreting this evidence). But it may still be “high-fidelity” evidence from the perspective of understanding the nature and consequences of particular interventions in transformations made by participants, in their context—where they are privileged observers. (p. 592)

Interactive Group Interviewing and Dialogues

The involvement of program staff or clients as colleagues or coresearchers in action research and program evaluation changes the relationship between evaluators and staff, making it interactive and cooperative rather than one-sided and antagonistic. William Tikunoff (1980), a pioneer in “interactive research” in education projects, found that putting teachers, researchers and trainer/developers together as a team increased both the meaningfulness and the validity of the findings because teacher cooperation with and understanding of the research made the research less intrusive, thus reducing rather than increasing reactivity. Their discussions were a form of group interviews in which they all asked each other questions.

The problem of how research subjects or program clients will react to staff involvement in an evaluation, particularly involvement in data collection, needs careful scrutiny and consideration in each situation in which it is attempted. Reactivity is a potential problem in both conventional and nonconventional designs. Breaches of confidence and/or reactivity-biased data cannot be justified in the name of creativity. On the other hand, as Tikunoff’s experiences indicate, interactive designs may increase the validity of data and reduce reactivity by making evaluation more visible and open, thereby making participants or clients less resistant or suspicious (King & Stevahn, 2013).

These approaches can reframe inquiry from a duality (interviewer–interviewee) to a dialogue in which all are co-inquirers. Miller and Crabtree (2000) advocate such a collaborative approach even in the usually closed and hierarchical world of medical and clinical research:

We propose that clinical researchers investigate questions emerging from the clinical experience with the clinical participants, pay attention to and reveal any underlying values and assumptions, and direct results toward clinical participants and policy makers. This refocuses the gaze of clinical research onto the clinical experience and redefines its boundaries so as to answer three questions: Whose question is it? Are hidden assumptions of the clinical world revealed? For whom are the research results intended? . . . Patients and clinicians are invited to explore their own and/or each other’s questions and concerns with whatever methods are necessary. Clinical researchers share ownership of the research with clinical participants, thus undermining the patriarchal bias of the dominant paradigm and opening its assumptions to investigation. This is the situated knowledge . . . where space is created to find a larger, more inclusive vision of clinical research. (p. 616)

SIDEBAR

MEDIATED CONVERSATIONS

Mediated conversations are an innovative method for the generation of rich qualitative data on complex issues such as those that researchers often face in education contexts. The method

draws from the sociocultural paradigm (Wertsch, 1991), which highlights the role of artifacts and audience in mediating participants’ actions and thoughts. Mediated conversations were developed during the Curriculum Implementation Exploratory Studies (in New Zealand), when we invited school leaders and teachers to a series of one-day data-generating “workshops.” The conversations that took place were mediated by two kinds of artifacts: one that participants had been asked to bring and discuss and the other the 10 contract research questions. The artifacts served as a practical situated exemplification of the participants’ descriptions of their work in relation to curriculum implementation. They served as a locus and prompt for questions and discussion. We convened the mediated conversations for our research purposes, but the participants experienced them as rich professional learning. Mediated conversations appear to provide a participatory method for data generation that has immediate benefits for researchers and research participants.

SOURCE: Cowie et al. (2009) and Hipkins, Cowie, Boyd, Keown, and McGee (2011).

Creativity and Data Quality: Qualitative Bricolage

I realized quite early in this adventure that interviews, conventionally conducted, were meaningless. Conditioned cliches were certain to come. The question-and-answer technique may be of some value in determining favored detergents, toothpaste and deodorants, but not in the discovery of men and women.

—Studs Terkel (Quoted by Douglas, 1985, p. 7)

No definitive list of creative interviewing or inquiry approaches can or should be constructed. Such a list would be a contradiction in terms. Creative approaches are those that are situationally responsive and appropriate, credible to primary intended users, and effective in opening up new understandings. The approaches just reviewed are deviations from traditional research practice. Each idea is subject to misuse and abuse if applied without regard for the ways in which the quality of the data collected can be affected. I have not discussed such threats and possible errors in depth because I believe it is impossible to identify in the abstract and in advance all the trade-offs involved in balancing concerns for accuracy, utility, feasibility, and propriety. For example, having program staff do client interviews in an outcomes evaluation could (a) seriously reduce the validity and reliability of the data, (b) substantially increase the validity and reliability of the data, or (c) have no measurable effect on data quality. The nature and degree of effect would depend on staff relationships with clients, how staff were assigned to clients for interviewing, the kinds of questions asked, the training of the interviewers, the attitudes of clients toward the program, the purposes to be served by the evaluation, the environmental turbulence of the program, and so on and so forth. Program staff might make better or worse interviewers than external evaluation researchers depending on these and other factors. An evaluator must grapple with these kinds of data quality questions for all designs, particularly nontraditional approaches.

Practical, but creative, data collection consists of using whatever resources are available to do the best job possible. Constraints always exist and do what constraints do—constrain. Our ability to think of alternatives is limited. Resources are always limited. This means data collection will be

imperfect, so dissenters from research and evaluation findings who want to attack a study’s methods can always find some grounds for doing so. A major reason for actively involving intended evaluation users in methods decisions is to deal with weaknesses and consider trade-off threats to data quality before data are collected. By strategically calculating threats to utility, as well as threats to validity and authenticity, it is possible to make practical decisions about the strengths of creative and nonconventional data collection procedures (Patton, 2008, 2012a).

The creative, adaptive inquirer using diverse techniques may be thought of as a “bricoleur.” The term comes from Levi-Strauss (1966), who defined a bricoleur as a “jack of all trades or a kind of professional do-it-yourself person” (p. 17).

The qualitative researcher as bricoleur or maker of quilts uses the aesthetic and material tools of his or her craft, deploying whatever strategies, methods, or empirical materials are at hand. If new tools or techniques have to be invented, or pieced together, then the researcher will do this. (Denzin & Lincoln, 2000, p. 4)

Creativity begins with being open to new possibilities, the bricolage of combining old things in new ways, including alternative and emergent forms of data collection, transformed observer –observed relations, and reframed interviewer–interviewee interconnections. Naturalistic inquiry calls for ongoing openness to whatever emerges in the field and during interviews. This openness means avoiding forcing new possibilities into old molds. The admonition to remain open and creative applies throughout naturalistic inquiry, from design through data collection and into analysis. Failure to remain open and creative can lead to the error made by a traveler who came across a peacock for the first time, a story told by Halcolm.

A traveler to a new land came across a peacock. Having never seen this kind of bird before, he took it for a genetic freak. Taking pity on the poor bird, which he was sure could not survive for long in such deviant form, he set about to correct nature’s error. He trimmed the long, colorful feathers; cut back the beak; and dyed the bird black. “There now,” he said, with pride in a job well done, “you now look more like a standard guinea hen.”

Adapting Interviewing for Particular Interviewees

This chapter has reviewed general principles of and approaches to in-depth qualitative interviewing. However, most researchers, evaluators, and practitioners specialize in working with and studying specific target populations using finely honed interviewing techniques. Interviewing “elites” or “experts” often requires an interactive style.

Elites respond well to inquiries about broad topics and to intelligent, provocative, open-ended questions that allow them the freedom to use their knowledge and imagination. In working with elites, great demands are placed on the ability of the interviewer, who must establish competence by displaying a thorough knowledge of the topic or, lacking such knowledge, by projecting an accurate conceptualization of the problem through shrewd questioning. (Rossman & Rallis, 1998, p. 134)

Robert Coles (1990) became adept at interviewing children, as have Guerrero-Manalo (1999), Graue and Walsh (1998), Holmes (1998), and Greig and Taylor (1998). Rita Arditti (1999) developed special culturally and politically sensitive approaches for gaining access to and interviewing grandmothers whose children were among “the disappeared” during Argentina’s military regime. Guerrero (1999a, 1999b) developed special participatory approaches for interviewing women in developing countries. Judith Arcana (1981, 1983) drew on her own

experiences as a mother to become expert at interviewing 180 mothers for two books about the experiences of mothering (one about mothers and daughters, one about mothers and sons).

SIDEBAR

ON INTERVIEWING POLITICIANS

Political people could always be more frank. . . . There is a truth that can’t be spoken. You can come to a shorthand understanding of certain realities—I know what we can’t talk about.

—Mark Leibovich (2013)

New York Times journalist

At the other end of the continuum from Cole’s delightful stories of childhood innocence are Jane Gilgun’s haunting interviews with male sexual offenders and Angela Browne’s intensive interviews with women incarcerated in a maximum security prison. Gilgun (1991, 1994, 1995, 1996, 1999) conducted hundreds of hours of interviews with men who have perpetrated violent sex offenses against women and children, most of them having multiple victims. She learned to establish relationships with these men through repeated life history interviews but did so without pretending to condone their actions, and sometimes challenging their portrayals. Two examples offer a sense of the challenges for someone undertaking such work through long hours and horrific details.

One man, engaged to be married at the time of his arrest, confessed to seven rapes; Gilgun interviewed him for a total of 14 hours over 12 different interviews, including detailed descriptions of his sexual violence. Another man molested more than 20 boys; at the time of his arrest, he was married, sexually active with his wife, and a stepfather of two boys; she obtained 20 hours of tape over 11 interviews. These cases were particularly intriguing as purposeful samples because both men were white college graduates in their early 30s who were employed as managers with major supervisory responsibilities and who came from upper middle-class, two-parent, never divorced families where the fathers held executive positions and mothers were professionals. Gilgun worked with an associate in transcribing and interpreting the interviews.

The data were so emotionally evocative that we spent a great deal of time working through our personal responses. Almost two years went by before we found we had any facility in articulating the meanings of the discourses we identified in the informants’ accounts. Their way of thinking was for the most part outside our frames of reference. As we struggled through these interpretive processes, we made notes of our responses. . . . Most compelling to us [was the men believing they were] entitled to take what they wanted and of defining persons and situations as they wished . . . , to suit themselves. Overall, the discourses they invoked served oppressive hegemonic ends. We also found that the men experienced chills, thrills, and intense emotional gratification as they imposed their wills on smaller, physically weaker persons. (Gilgun & McLeod, 1999, p. 175)

The life work of Angela Browne illustrates a similar commitment to in-depth, life history, interviewing with people who are isolated from the mainstream and whose experiences are little understood by the general culture. After her groundbreaking study of women who kill violent partners in self-defense (When Battered Women Kill, 1987), Browne began gathering life history narratives from women incarcerated in a maximum security prison. These interviews, conducted in a small room off a tunnel in the middle of the facility and six hours in length, included women with lifetime histories of trauma, much of it at the hands of family members in early childhood. Some

had witnessed brutal homicides; others were serving time for crimes of violence they had committed. Their stories were painful to tell and to hear. Interviews often were so emotionally draining that Browne came away exhausted, sometimes needing to debrief on both the impact of what she had heard and the dynamics of the interviewing process. On several occasions, we had lengthy phone conversations immediately following the interviews, while she was still within the prison walls. This kind of extreme interviewing takes unusual skill, dedication, and self-knowledge, coupled with a keen interest in the dynamics of human interaction. Although Browne returned home drained after each full week of conducting these day-long interviews, her enthusiasm for the task and her appreciation of the respondents’ strength and lucidity never dimmed.

The works of Gilgun and Browne illustrate the intensity, commitment, long hours, and hard work involved with certain in-depth and life history approaches to interviewing.

Adapting Interviewing to Particular Target Populations

Applications and methods of qualitative inquiry, especially interviewing techniques for specially targeted populations and specialized disciplinary approaches, continue to evolve as interest in qualitative methods grows exponentially (a metaphoric rather than a statistical estimation). General interview skills include asking open-ended questions, listening carefully to ask follow-up questions, effective and sensitive probing, distinguishing different kinds of questions, and pacing the interview. But additional challenges arise in interviewing particular target populations. General interview skills are necessary as a foundation for in-depth qualitative interviewing, but those skills must then be adapted to particular target groups and situations. Exhibit 7.17 on the next page highlights some specialized interviewing approaches and accompanying challenges, including online interviewing. As applications and techniques have proliferated, so have concerns about the ethical challenges of qualitative inquiry, our next topic.

EXHIBIT 7.17 Special Interviewing Challenges for Particular Target Populations: Five Examples

This table highlights some of the specific challenges that arise in interviewing particular target populations. This is by no means an exhaustive list of target populations or challenges. It is meant to illustrate how general interview skills are necessary as a foundation for in-depth qualitative interviewing but those skills must then be adapted to particular target groups and situations.

MODULE

63 Ethical Issues and Challenges in Qualitative Interviewing

Interviews are interventions. They affect people. A good interview evokes thoughts, feelings, knowledge, and experience not only to the interviewer but also to the interviewee. The process of being taken through a directed, reflective process affects the persons being interviewed and leaves them knowing things about themselves that they didn’t know—or least were not fully aware of—before the interview. Two hours or more of thoughtfully reflecting on an experience, a program, or one’s life can be change inducing; 10, 15, or 20 hours of life history interviewing can be transformative—or not. Therein lies the rub. Neither you nor the interviewee can know, in advance, and sometimes even after the fact, what impact an interviewing experience will have or has had.

The purpose of a research interview is first and foremost to gather data, not change people. Earlier, in the section on neutrality, I asserted that an interviewer is not a judge. Neither is a research interviewer a therapist. Staying focused on the purpose of the interview is critical to gathering high-quality data. Still, there will be many temptations to stray from that purpose. It is common for interviewees to ask for advice, approval, or confirmation. Yielding to these temptations, the interviewer may become the interviewee—answering more questions than are asked.

On the other hand, the interviewer, in establishing rapport, is not a cold slab of granite—unresponsive to learning about great suffering and pain that may be reported and even reexperienced during an interview. In a major farming systems needs assessment project to develop agricultural extension programs for distressed farm families, I was part of a team of 10 interviewers (working in pairs) who interviewed 50 farm families. Many of these families were in great pain. They were losing their farms. Their children had left for the city. Their marriages were under stress. The two-hour interviews traced their family history, their farm situation, their community relationships, and their hopes for the future. Sometimes questions would lead to husband–wife conflict. The interviews would open old wounds, lead to second-guessing decisions made long ago, or bring forth painful memories of dreams never fulfilled. People often asked for advice—what to do about their finances, their children, government subsidy programs, even their marriages. But we were not there to give advice. Our task was to get information about needs that might, or might not, lead to new programs of assistance. Could we do more than just ask our questions and leave? Yet, as researchers, could we justify in any way intervening? Yet again, our interviews were already an intervention. Such are the ethical dilemmas that derive from the power of interviews.

What we decided to do in the farm family interviews was leave each family a packet of information about resources and programs of assistance, everything from agricultural referrals to financial and family counseling. To avoid having to decide which families really needed such assistance, we left the information with all families—separate and identical packages for both husband and wife. When interviewees asked for advice during the interview, we could tell them that we would leave them referral information at the end of the interview.

While interviews may be intrusive in reopening old wounds, they can also be healing. In doing follow-up interviews with families who had experienced child sexual abuse, we found that most of the mothers appreciated the opportunity to tell their stories, vent their rage against the system, and share their feelings with a neutral, but interested, listener. Our interviews with elderly residents

participating in a program to help them stay in their homes and avoid nursing home institutionalization typically lasted much longer than planned because the elderly interviewees longed to have company and talk. When interviewees are open and willing to talk, the power of interviewing poses new risks. People will tell you things they never intended to tell you. This can be true even with reluctant or hostile interviewees—a fact depended on by journalists. Indeed, it seems at times that the very thing someone is determined not to say is the first thing he or she tells, just to release the psychological pressure of secrecy or deceit.

I repeat, people in interviews will tell you things they never intended to tell. Interviews can become confessions, particularly under the promise of confidentiality. But beware that promise. Social scientists can be summoned to testify in court. We do not have the legal protection of clergy and lawyers. In addition, some information must be reported to the police, for example, evidence of child abuse. Thus, the power of interviewing can put the interviewees at risk. The interviewer needs to have an ethical framework for dealing with such issues.

There are also direct impacts on interviewers. The previous section described the wrenching interviews conducted by Jane Gilgun with male sex offenders and Angela Browne with incarcerated women, and the physical and emotional toll of those interviews on them as interviewers, exposed for hours on end to horrendous details of violence and abuse. In a family sex abuse project (Patton, 1991), we found that the interviewers needed to be extensively debriefed, sometimes in support groups together, to help them process and deal with the things they heard. They could only take in so much without having some release, some safety valve for their own building anger and grief. Middle-class interviewers going into poor areas may be shocked and depressed by what they hear and see. It is not enough to do preparatory training before such interviewing. Interviewers may need debriefing—and their observations and feelings can become part of the data on team projects.

These examples are meant to illustrate the power of interviewing and why it is important to anticipate and deal with the ethical dimensions of qualitative inquiry. Because qualitative methods are highly personal and interpersonal, because naturalistic inquiry takes the researcher into the real world where people live and work, and because in-depth interviewing opens up what is inside people, qualitative inquiry may be more intrusive and involve greater reactivity than surveys, tests, and other quantitative approaches.

Exhibit 7.18 presents a checklist of 12 common ethical issues as a starting point in thinking through ethical issues in design, data collection, analysis, and reporting. The next sections elaborate some common issues of special concern.

• Informed consent and confidentiality

• Confidentiality versus people owning their own stories

• How much of an interview must be approved in advance?

• Reciprocity: Should interviewees be compensated? If so, how?

• How hard to push for sensitive information?

• Being careful in the face of danger in the field

EXHIBIT 7.18 Ethical Issues Checklist

1. Explaining purpose. How will you explain the purpose of the inquiry and the methods to be used in ways that are accurate and understandable?

• What language will make sense to the participants in the study? • What details are critical to share? What can be left out? • What’s the expected value of your work to society and to the greater good?

Guiding principle: Be clear, honest, and transparent about purpose.

2. Reciprocity. What’s in it for the interviewee?

• Why should the interviewee participate in the interview? • What is appropriate compensation?

Guiding principle: Honor the gift of an interviewee’s time in a meaningful and tangible way.

3. Promises. Don’t make promises lightly, for example, promising a copy of the recording or the report.

Guiding principle: If you make promises, keep them.

4. Risk assessment. In what ways, if any, will conducting the interview put people at risk?

• Psychological stress • Legal liabilities • In evaluation studies, continued program participation (if certain things become known) • Ostracism by peers, program staff, or others for talking • Political repercussions

How will you describe these potential risks to interviewees?

How will you handle them if they arise?

Guiding principle: First, do no harm.

5. Confidentiality. What are reasonable promises of confidentiality that can be fully honored? Know the difference between confidentiality and anonymity. (Confidentiality means you know but won’t tell. Anonymity means you don’t know, as in a survey returned anonymously.)

• What things can you not promise confidentiality about, for example, illegal activities, evidence of child abuse, or neglect?

• Will names, locations, and other details be changed? Or do participants have the option of being identified? (See discussion of this in the text.)

• Where will data be stored? • How long will data be maintained?

Guiding principle: Know the ethical and legal dimensions of confidentiality.

6. Informed consent. What kind of informed consent, if any, is necessary for mutual protection?

• What are your local institutional review board (IRB) guidelines and/or requirements, or those of an equivalent committee for protection of human subjects in research?

• What has to be submitted, under what timelines, for IRB approval, if applicable?

Guiding principle: Know and follow the standards of your discipline or field.

7. Data access and ownership. Who will have access to the data? For what purposes?

• Who owns the data in an evaluation? (Be clear about this in the contract.) • Who has right of review before publication? (For example, of case studies, by the person

or organization depicted in the case, or of the whole report, by a funding or sponsoring organization)

Guiding principle: Don’t wait until publication to deal with data ownership issues; anticipate data access and ownership issues from the beginning.

8. Interviewer mental health. How will you and other interviewers likely be affected by conducting the interviews?

• What might be heard, seen, or learned that may merit debriefing and processing? • Who can you talk to about what you experience without breaching confidentiality?

Guiding principle: Fieldwork is engaging, intellectually and emotionally. Take care of yourself and your coresearchers.

9. Ethical advice. Who will be the researcher’s confidant and counselor on matters of ethics during a study? Not all issues can be anticipated in advance. Knowing who you will go to in the event of difficulties can save precious time in a crisis and bring much needed comfort.

Guiding principle: Plan ahead, and know who you will consult on emergent ethical issues.

10. Data collection boundaries. How hard will you push for responses from interviewees?

• What lengths will you go to in trying to gain access to data you want? What won’t you do?

• How hard will you push interviewees to respond to questions about which they show some discomfort?

Guiding principles: Know yourself. Err on the side of caution. Don’t let the ends justify the means in overstepping boundaries.

11. Intersection of ethical and methodological choices.

• Methods and ethics are intertwined. Understand the intersection.

• As a qualitative methodologist, based on your own work, contribute to the field by reporting the ethical challenges you face.

Guiding principle: Include ethical dilemmas faced and handled in your methods discussion.

12. Ethical versus legal. What ethical framework and philosophy informs your work and ensures respect and sensitivity for those you study beyond whatever may be required by law?

• What disciplinary or professional code of ethical conduct will guide you?

Guiding principles: Don’t make up ethical responses along the way. Know your profession’s ethical standards. Know what the law in your jurisdiction requires.

For additional guidance on ethics in qualitative inquiry, see Christians (2000), Rubin and Rubin (2012, pp. 85–93), and Silverman and Marvasti (2008).

Informed Consent and Confidentiality

Informed consent protocols and opening statements in interviews typically cover the following issues:

What is the purpose of collecting the information?

Who is the information for? How will it be used?

What will be asked in the interview?

How will responses be handled, including confidentiality?

What risks and/or benefits are involved for the person being interviewed?

The interviewer often provides this information in advance of the interview and then again at the beginning of the interview. Providing such information does not, however, require making long and elaborate speeches. Statements of purpose should be simple, straightforward, transparent, and understandable. Long statements about what the interview is going to be like, and how it will be used, when such statements are made at the beginning of the interview, are usually either boring or anxiety producing. The interviewee will find out soon enough what kinds of questions are going to be asked and, from the nature of the questions, will make judgments about the likely use of such information. The basic messages to be communicated in the opening statement are (a) that the information is important, (b) the reasons for that importance, and (c) the willingness of the interviewer to explain the purpose of the interview out of respect for the interviewee. Here’s an example of an opening interview statement from an evaluation study.

I’m a program evaluator brought in to help improve this program. As someone who has been in the program, you are in a unique position to describe what the program does and how it affects people. And that’s what the interview is about: your experiences with the program and your thoughts about your experiences.

The answers from all the people we interview, and we’re interviewing about 25 people, will be combined for our report. Nothing you say will ever be identified with you personally. As we go through the interview, if you have any questions about why I’m asking something, please feel free to ask. Or if there’s anything you don’t want to answer, just say so. The purpose of the interview is to get your insights into how the program operates and how it affects people.

Any questions before we begin?

This may seem straightforward enough, but dealing with real people in the real world, all kinds of complications can arise. Moreover, some genuine ethical quandaries have arisen in recent years around the ethics of research in general and qualitative inquiry in particular.

How Much of an Interview Must Be Approved in Advance?

In the chapter on observation and fieldwork, I discussed the problems posed by approval protocols aimed at protecting human subjects given the emergent and flexible designs of naturalistic inquiry. IRBs for the protection of human subjects often insist on approving actual interview questions, which can be done when using the standardized interview format discussed in this chapter, but it works less well when using an interview guide and doesn’t work at all for conversational interviewing, where the questions emerge at the moment in context. A compromise is to specify those questions one can anticipate and list other possible topics while treating the conversational component as probes, which are not typically specified in advance. Adopt a strategy of “planned flexibility” insofar as possible, and be prepared to educate your IRB about why this is appropriate both methodologically and ethically (Silverman & Marvasti, 2008, p. 48). In a fully naturalistic inquiry design, interview questions can and should change as the interviewer understands the situation more fully and discovers new pathways for questioning. The tension between specifying questions for approval in advance versus allowing questions to emerge in context in the field led Elliott Eisner (1991) to ask, “Can qualitative studies be informed . . . [since] we have such a hard time predicting what we need to get consent about?” (p. 215). An alternative to specifying precise questions for approval in advance is to specify areas of inquiry that will be avoided—that is, to anticipate ways in which respondents might be put at risk and affirm that the interviewer will avoid such areas.

SIDEBAR

WHAT DO YOU DO WITH DANGEROUS REVELATIONS? THE ETHICAL CHALLENGE OF CONFIDENTIALITY

Robert Weiss (1994) reported the problem posed by a woman respondent who was HIV positive.

She said that all her life, from the time she was a child, she had been treated brutally by men. Contracting HIV from a boyfriend was only the most recent instance. Now she wanted to get even with the whole male sex. She visited barrooms every evening to pick up men with whom she could have intercourse, in the hope that she would infect them. The woman’s sister had already reported her to a public health agency, mostly because she wanted the woman stopped before she was hurt by some man she had tried to infect. The public health agency did nothing.

In our final interview I learned the woman was no longer seeking revenge through sex. She had met a man who had become her steady boyfriend and who remained with her even after he was told—by that same sister—that she was HIV positive. (His first reaction was to yell at the woman and, I think, push her around.) If in our final interview the woman had reported continuing her campaign to spread HIV among men, I would have told her to stop. I can’t believe that would have done much good, but I would have told her anyway. I also would have discussed her report with the head of the clinic where she was being treated, with the thought of devising some way to interrupt her behavior. (p. 132)

©2002 Michael Quinn Patton and Michael Cochran

Uninformed Consent Seeker

Conversational Interviewing Poses Special Informed Consent Problems:

Whipping out an informed consent statement and asking for a signature can be awkward at best. To the extent that interviews are an extension of a conversation and part of a relationship, the legality and formality of a consent form may be puzzling to your conversational partner or disruptive to the research. On the one hand, you may be offering conversational partners anonymity and confidentiality, and on the other asking them to sign a legal form saying they are participating in the study. How can they later deny they spoke to you—which they may need to do to protect themselves—if you possess a signed form saying they were willing to participate in the study? (Rubin & Rubin, 1995, p. 95)

Qualitative methodologists find that many IRBs are not competent to judge qualitative research. When engaging with an IRB, distinguish between legal compliance with human subjects protection requirements versus conscientious ethical behavior:

You cannot achieve ethical research by following a set of preestablished procedures that will always be correct. Yet, the requirement to behave ethically is just as strong in qualitative interviewing as in other types of research on humans—maybe even stronger. You must build ethical routines into your work. You should carefully study code of ethics and cases of unethical behavior to sensitize yourself to situations in which ethical commitments become particularly salient. Throughout your research, keep thinking and judging what are your ethical obligations. (Rubin & Rubin, 1995, p. 96)

New Directions in Informed Consent: Confidentiality Versus People Owning Their Own Stories

Confidentiality norms are also being challenged by new directions in qualitative inquiry. Traditionally, researchers have been advised to disguise the locations of their fieldwork and change the names of respondents, usually giving them pseudonyms, as a way of protecting their identities. The presumption has been that the privacy of research subjects should always be protected. This remains the dominant presumption, as well it should. It is being challenged, however, by participants in research who insist on “owning their own stories.” Some politically active groups take pride in their identities and refuse to be involved in research that disguises who they are. Some programs that aim at empowering participants emphasize that participants “own” their stories and should insist on using their real names. In a program helping them overcome a history of violence and abuse, I encountered women who were combating the stigma of their past by telling their stories and attaching their real names to their stories as part of healing, empowerment, and pride. Does the researcher, in such cases, have the right to impose confidentiality against the wishes of those involved? Is it patronizing and disempowering for a university-based human subjects committee to insist that these women are incapable of understanding the risks involved if they choose to turn down an offer of confidentiality? On the other hand, by identifying themselves, they give up not only their own privacy but also perhaps that of their children, other family members, and current or former partners.

A doctoral student studying a local church worked out an elaborate consent form in which the entire congregation decided whether to let itself be identified in his dissertation. Individual church members also had the option of using their real names or choosing pseudonyms. Another student studying alternative health practitioners offered them the option of confidentiality and pseudonyms or using their real identities in their case studies. Some chose to be identified, while some didn’t. A study of organizational leaders offered the same option. In all of these cases, the research participants also had the right to review and approve the final versions of their case studies and transcripts before they were made public. In cases of collaborative inquiry, where the researcher works with “coresearchers” and data collection involves more of a dialogue than an interview, the coresearchers may also become coauthors as they choose to identify themselves and share in publication.

These are examples of how the norms about confidentiality are changing and being challenged as tension has emerged between the important ethic of protecting people’s privacy and, in some cases, their desire to own their stories. Informed consent, in this regard, does not automatically mean confidentiality. Informed consent can mean that participants understand the risks and benefits of having their real names reported and choose to do so. Protection of human subjects properly insists

on informed consent. That does not now automatically mean confidentiality, as these examples illustrate.

Reciprocity: Should Interviewees Be Compensated? If So, How?

Quid pro quo: “something for something” (Ancient Roman principle)

The issues of whether and how to compensate interviewees involve questions of both ethics and data quality. Will payment, even of small amounts, affect people’s responses, increasing acquiescence or, alternatively, enhancing the incentive to respond thoughtfully and honestly? Is it somehow better to appeal to people on the basis of the contribution they can make to knowledge or, in the case of evaluation, improving the program, instead of appealing to their pecuniary interest? Can modest payments in surveys increase response rates to ensure an adequate sample size? Does the same apply to indepth interviewing and focus groups? The interviewer is usually getting paid. Shouldn’t the time of interviewees be respected, especially the time of low-income people, by offering compensation? What alternatives are there to cash for compensating interviewees? In Western capitalist societies, issues of compensation are arising more and more often both because people in economically disadvantaged communities are reacting to being overstudied and undervalued and because private sector marketing firms routinely compensate focus group participants; so this practice has spread to the public and nonprofit sectors.

Professionals in various fields differ about compensation for interviewees. Below are some comments from a lively discussion of these issues that took place on EvalTalk, the American Evaluation Association Internet Listserv.

• I believe in paying people, particularly in areas of human services. I am thinking of parenting and teen programs where it can be very difficult to get participation in interviews. If their input is valuable, I believe you should put your money where your mouth is. However, I would always make it very clear to the respondents that, although they are being paid for their time, they are not being paid for their responses and should be as candid and forthright as possible.

• One inner-city project offered vouchers to parent participants in a focus group to buy books for their kids, which for some low-income parents proved to be their first experience owning books rather than always borrowing them.

• Cash payment for participation in interviews is considered income and is therefore taxable. This can create problems if the payment comes from a public agency, as our county attorney has pointed out in the past. Consequently, when we “pay” for participation, we use incentives other than cash, for example, vouchers or gift certificates donated by local commercial vendors, such as a discount store. These seem to be as effective.

• If you are detaining a person with a face-to-face interview, and it isn’t a friendly conversation, rather it is a business exercise, it is only appropriate to offer to pay the prospective respondent for his or her time and effort. This should not preclude, and it would certainly help to explain, the importance of his or her contribution.

• We have paid and not paid incentives for focus groups for low-income folks as well as professionals and corporate CEOs. The bottom line is that in most cases the incentive doesn’t make a lot of difference in terms of participation rates, especially if you have well-trained interviewers and well-designed data collection procedures.

One of my concerns is that we are moving in a direction in which it is assumed (with very little substantive foundation) that people will only respond if given incentives. My plea here is that colleagues not fall into the trap of using incentives as a crutch but that they constantly examine and reexamine the whole issue of incentives and not simply assume that they are either needed or effective.

Alternatives to cash can instill a deeper sense of reciprocity. In doing family history interviews, I found that giving families a copy of the interview was much appreciated and increased the depth of responses because they were speaking not just to me, the interviewer, but to their grandchildren and great-grandchildren in telling the family’s story. In one project in rural areas, we carried a tape duplicator in the truck and made copies for them instantly at the end of the interview. Providing complete transcripts of interviews can also be attractive to participants. In an early-childhood parenting program where data collection included videotaping parents playing with their children, copies of the videotapes were prized by the parents. The basic principle informing these exchanges is reciprocity. Participants in research provide us with something of great value—their stories and their perspectives on their world. We show that we value what they give us by offering something in exchange.

How Hard to Push for Sensitive Information?

Skillful interviewers can get people to talk about things they may later regret having revealed. Or sharing revelations in an interview may unburden a person, letting one get something off one’s chest. Since one can’t know for sure, interviewers are often faced with an ethical challenge concerning how hard to push for sensitive information, a matter in which the interviewer has a conflict of interest since the interviewer’s predilection is likely to be to push for as much as possible. Herb and Irene Rubin tell of interviewing an administrator in Thailand and learning that two months after their fieldwork, he committed suicide, “leaving us wondering if our encouraging him to talk about his problems may have made them more salient to him” (Rubin & Rubin, 1995, p. 98).

In deciding how hard to push for information, the interviewer must balance the value of a potential response against the potential distress for the respondent. This requires sensitivity, but it is not a burden the interviewer need to take on alone. When I see that someone is struggling for an answer, seems hesitant or unsure, or I simply know that an area of inquiry may be painful or uncomfortable, I prefer to make the interviewee a partner in the decision about how deeply to pursue the matter. I say something like this:

I realize this is a difficult thing to talk about. Sometimes people feel better talking about something like this and, of course, sometimes they don’t. You decide how much is comfortable for you to share. If you do tell me what happened and how you feel and later you wish you hadn’t, I promise to delete it from the interview. Okay? Obviously, I’m very interested in what happened, so please tell me what you’re comfortable telling me.

Be Careful. It’s Dangerous Out There.

In our teaching and publications we tend to sell students a smooth, almost idealized, model of the research process as neat, tidy, and unproblematic. . . . Perhaps we should be more open and honest about the actual pains and perils of conducting research in order to prepare and forewarn aspiring researchers.

—Maurice Punch (1986, pp. 13–14)

In an old police investigation television show, Hill Street Blues, the precinct duty sergeant ended each daily briefing of police officers by saying, “Let’s be careful out there.” The same warning applies to qualitative researchers doing fieldwork and interviewing: “Be careful. It’s dangerous out there.” It’s important to protect those who honor us with their stories by participating in our studies. It’s also important to protect yourself.

I was once interviewing a young man at a coffee shop for a recidivism study when another man showed up, an exchange took place, and I realized I had been used as a cover for a drug purchase. In doing straightforward outcomes evaluation studies, I have discovered illegal and unethical activities that I would have preferred not to have stumbled across. When we did the needs assessment of distressed farm families in rural Minnesota, we took the precaution of alerting the sheriffs’ offices in the counties where we would be interviewing in case any problems arose. One sheriff called back and said that a scam had been detected in the county that involved a couple in a pickup truck soliciting home improvement work and then absconding with the down payment. Since we were interviewing in couple teams and driving pickup trucks, the sheriff, after assuring himself of the legitimacy of our work, offered to provide us with a letter of introduction, an offer we gratefully accepted.

I supervised a dissertation that involved interviews with young male prostitutes. We made sure to clear that study with the local police and public prosecutors and to get their agreement that promises of confidentiality would be respected given the potential contribution of the findings to reducing both prostitution and the spread of AIDS. This, by the way, was a clear case where it would have been inappropriate to pay the interviewees. Instead of cash, the reciprocity incentive the student offered was the result of a personality instrument he administered.

One of the more famous cases of what seemed like straightforward fieldwork that became dangerous involved dissertation research on the culture of a bistro in New York City. Through in- depth interviews, graduate student Mario Brajuha gathered detailed information from people who worked and ate at the restaurant—information about their lives and their views about others involved with the restaurant. He made the usual promise of confidentiality. In the midst of his fieldwork, the restaurant was burned, and the police suspected arson. Learning of his fieldwork, they subpoenaed his interview notes. He decided to honor his promises of confidentiality and ended up going to jail rather than turning over his notes. This case, which dragged on for years, disrupting his graduate studies and his life, reaffirmed that researchers lack the protection of clergy and lawyers when subpoenas are involved, promises of confidentiality notwithstanding. (For details, see Brajuha & Hallowell, 1986.)

It helps to think about potential risks and dangers prior to gathering data, but Brajuha could not have anticipated the arson. Anticipation, planning, and ethical reflection in advance only take you so far. As Maurice Punch (1986) has observed, sounding very much like he is talking from experience,

How to cope with a loaded revolver dropped in your lap is something you have to resolve on the spot, however much you may have anticipated it in prior training. (p. 13)

So be careful. It’s dangerous out there.

MODULE

64 Personal Reflections on Interviewing, and ChapterSummary and Conclusion

The word interview has roots in Old French and meant something like “to see one another.”

—Narayan and George (2003, p. 449)

Personal Reflections on Interviewing

Though there are dangers, there are also rewards.

I find interviewing people invigorating and stimulating—the opportunity for a short period of time to enter another person’s world. If participant observation means “walk a mile in my shoes,” in-depth interviewing means “walk a mile in my head.” New worlds are opened up to the interviewer on these journeys.

I’m personally convinced that to be a good interviewer you must like doing it. This means being interested in what people have to say. You must yourself believe that the thoughts and experiences of the people being interviewed are worth knowing. In short, you must have the utmost respect for these persons who are willing to share with you some of their time to help you understand their world. There is a Sufi story that describes what happens when the interviewer loses this basic sensitivity to and respect for the person being interviewed.

An Interview With the King of the Monkeys

A man once spent years learning the language of monkeys so that he could personally interview the king of monkeys. Having completed his studies, he set out on his interviewing adventure. In the course of searching for the king, he talked with a number of monkey underlings. He found that the monkeys he spoke to were generally, to his mind, neither very interesting nor very clever. He began to doubt whether he could learn very much from the king of the monkeys after all.

Finally, he located the king and arranged for an interview. Because of his doubts, however, he decided to begin with a few basic questions before moving on to the deeper meaning-of-life questions that had become his obsession.

“What is a tree?” he asked.

“It is what it is,” replied the king of the monkeys. “We swing through trees to move through the jungle.”

“And what is the purpose of the banana?”

SIDEBAR

THE PERSONAL EXPERIENCE OF QUALITATIVE INQUIRY

Question. What’s the most memorable or meaningful evaluation that you have been a part of—and why?

I worked briefly on an evaluation of a rail safety intervention program aimed at reducing safety- related incident rates through peer-to-peer observation (Zuschlag, Ranney, Coplen, & Harnar, 2012). To collect some qualitative insights, I was sent to San Antonio where I got a full tour, a ride aboard a locomotive, and the opportunity to talk with some engineers and conductors about changes they saw in safety related to the program. One of their stories, in particular, touches me every time I tell it.

A trained safety observer noticed an older engineer not wearing any hearing protection, even though he was right alongside a locomotive. The observer said something to the order of “Don’t you want to hear your granddaughter’s voice when she talks to you?” This relatively casual but purposeful effort had a tremendous impact. After that, every time the “old-timer” saw our observer he would point to his hearing protection to say, “Look, I’m wearing them!”

In my youth, I spent long days playing in the dead yards of the Reading Railroad (yes, the one from the Monopoly game, pronounced Redding), and I never dreamed of someday riding the tracks as a researcher and learning about that world through the eyes of its practitioners. Every time I recall the story it takes me back to that day in the locomotive and the hours I spent talking with those engineers and conductors, and it reminds me that one conversation can change a person’s world . . . and maybe let him hear his granddaughter’s voice in his old age.

—Michael A. Harnar, Mosaic Network, Inc.

Senior Associate, Research and Evaluation

American Evaluation Association Newsletter, November, 2012

“Purpose? Why, to eat.”

“How do animals find pleasure?”

“By doing things they enjoy.”

At this point, the man decided that the king’s responses were rather shallow and uninteresting, and he went on his way, crushed and cynical. Soon afterward, an owl flew into the tree next to the king of the monkeys. “What was that man doing here?” the owl asked.

“Oh, he was only another silly human,” said the king of the monkeys. “He asked a bunch of simple and meaningless questions, so I gave him simple and meaningless answers.”

Not all interviews are interesting, and not all interviews go well. Certainly, there are uncooperative respondents, people who are paranoid, respondents who seem overly sensitive and easily embarrassed, aggressive and hostile interviewees, timid people, and the endlessly verbose, who go on at great length about very little. When an interview is going badly, it is easy to call forth one of these stereotypes to explain how the interviewee is ruining the interview. Such blaming of the victim (the interviewee), however, does little to improve the quality of the data. Nor does it improve interviewing skills.

I prefer to believe that there is a way to unlock the internal perspective of every interviewee. My challenge and responsibility as an interviewer involve finding the appropriate and effective interviewing style and question format for a particular respondent. It is my responsibility as the interviewer to establish an interview climate that facilitates open responses. When an interview goes badly, as it sometimes does, even after all these years, I look first at my own shortcomings and miscalculations, not the shortcomings of the interviewee. That’s how, over the years, I’ve gotten better and come to value reflexivity, not just as an intellectual concept but also as a personal and professional commitment to learning and engaging people with respect.

Overview and Conclusion

Interviewing has become central in our modern knowledge age Interview Society. This chapter began by discussing different approaches to interviewing for different purposes: journalism, therapeutic interviews, forensic investigations, celebrity interviews, and personnel interviewing, among others (see Exhibit 7.1, pp. 423–424). We examined differences in how interviews are used to guide diverse theoretical and methodological traditions: ethnography, phenomenology, social constructionism, hermeneutics, and other frameworks for inquiry (see Exhibit 7.3, pp. 432–436).

A major focus of this chapter has been that obtaining high-quality data from interviews requires skilled interviewing. Exhibit 7.2 (p. 428) presented 10 core interviewing skills. We then moved to different types of interview formats: standardized questions, interview guides, and conversational interviewing (see Exhibit 7.4, pp.437–438).We looked at how to phrase questions and probe responses. A point of emphasis was the importance of anticipating analysis and reporting to organize, sequence, and format interviews (see Exhibit 7.6, p. 443).

Every interview is also an observation, a two-way interaction, and, therefore, a relationship. Exhibit 7.12 (pp.462–463) summarized six distinct approaches to undertaking interviewing as a relationship. We also compared interviews with individuals with group interviews, like focus groups and 11 other types of group interviews (see Exhibit 7.14, pp. 475–476).

Cross-cultural interviewing and qualitative fieldwork presents special challenges (see Exhibit 7.15, p. 482). Special target populations pose particular challenges in qualitative inquiries: interviewing children, older people, elites, and marginalized people or conducting online interviews (see Exhibit 7.17, pp. 493–494).

The interpersonal nature of in-depth qualitative interviewing raises special ethical dilemmas and concerns (see Exhibit 7.18, pp. 496–497). The people interviewed can be affected by the inquiry, but so can the person conducting the interviews. Nora Murphy (2014) interviewed homeless youth for her dissertation.

This also took a personal toll. At times, the harsh reality of the experiences the youths have gone through had me in tears at home. In fact, in some interviews, the youth and I cried together. The stories are so powerful because they are real and true, and sharing these truths means opening wounds. I felt guilty that I could not help them more and feared exploiting them. Sometimes I emailed advisors just to vent to him about how difficult it was; playing the role of an objective researcher who could not step in to help felt unnatural and uncomfortable. It helped when the youth thanked me for listening and glowed when I read their stories back to them. They often said things like, “You made connections about me that I didn’t realize . . . No one else in my life knows all of this . . . I want to share this with my brother. It will help me explain things to him that I’ve never been able to say before . . . I want to make this into a book.” I left each interview with the feeling that the youths appreciated being listened to, that they felt gratitude toward me and the process. I still harbor guilt that I could not do more for each of them, but I

also believe the process, overall, has done more good than harm. I still carry some of the sadness I experienced as I sat with the youth and listened to their tales of their difficult journeys, but I also carry a hope inspired by their optimism and their strength. (p. 428)

While this chapter has emphasized skill and technique as ways of enhancing the quality of interview data, no less important is a genuine interest in and caring about the perspectives of other people. If what people have to say about their world is generally boring to you, then you will never be a great interviewer. Unless you are fascinated by the rich variation in human experience, qualitative interviewing will become drudgery. On the other hand, a deep and genuine interest in learning about people is insufficient without disciplined and rigorous inquiry based on skill, technique, and a deep capacity for empathic understanding.

Conclusion: Halcolm on Interviewing

Ask.

Listen and record.

Ask.

Listen and record.

Asking involves a grave responsibility.

Listening is a privilege.

Researchers, listen and observe. Remember that your questions will be studied by those you study. Evaluators, listen and observe. Remember that you shall be evaluated by your questions.

To ask is to seek entry into another’s world. Therefore, ask respectfully and with sincerity. Do not waste questions on trivia and tricks, for the value of the answering gift you receive will be a reflection of the value of your question.

Blessed are the skilled questioners, for they shall be given mountains of words to ascend.

Blessed are the wise questioners, for they shall unlock hidden corridors of knowledge.

Blessed are the listening questioners, for they shall gain perspective. —From Halcolm’s Beatitudes

Looking Forward

Part I of this book presented the niche of qualitative methods in studying the world (Chapter 1), presented 12 fundamental qualitative strategies (Chapter 2), reviewed major theoretical traditions that inform diverse approaches to qualitative inquiry (Chapter 3), and examined practical applications for qualitative methods (Chapter 4). Part II presented qualitative design options (Chapter 5), fieldwork approaches (Chapter 6), and interview methods (Chapter 7). We now turn to Part III: analyzing and reporting qualitative findings (Chapter 8) and ways of enhancing the

credibility and utility of qualitative results (Chapter 9). As we make the transition from Part II to Part III, from ways of gathering data to ways of making sense of the data gathered, the Halcolm story that ends this chapter reminds us that things are not always what they seem. The story also reminds us of the centrality of Thomas’s theorem for interpretation of qualitative data: What is perceived as real is real in its consequences.

EXHIBIT 7.19 Examples of Standardized Open-Ended Interviews

The edited interviews below were used in evaluation of an Outward Bound program for the disabled. Outward Bound is an organization that uses the wilderness as an experiential education medium. This particular program consisted of a 10-day experience in the Boundary Waters Canoe Area of Minnesota. The group consisted of half able-bodied participants and half disabled participants, including paraplegics; persons with cerebral palsy, epilepsy, or other developmental disabilities; blind and deaf participants; and, on one occasion, a quadriplegic. The first interview was conducted at the beginning of the program; the second interview was used at the end of the 10-day experience; and the third interview took place six months later. To save space, many of the probes and elaboration questions have been deleted, and space for

writing notes has been eliminated. The overall thrust and format of the interviews have, however, been retained.

Precourse Interview: Minnesota Outward Bound School Course for the Able-Bodied and the Disabled

This interview is being conducted before the course as part of an evaluation process to help us plan future courses. You have received a consent form to sign, which indicates your consent to this interview. The interview will be recorded.

1. First, we’d be interested in knowing how you became involved in the course. How did you find out about it?

a. What about the course appealed to you? b. What previous experiences have you had in the outdoors?

2. Some people have difficulty deciding to participate in an Outward Bound course, and others decide fairly easily. What kind of decision process did you go through in thinking about whether or not to participate?

a. What particular things were you concerned about? b. What is happening in your life right now that stimulated your decision to take the

course?

3. Now that you’ve made the decision to go on the course, how do you feel about it?

a. How would you describe your feelings right now? b. What lingering doubts or concerns do you have?

4. What are your expectations about how the course will affect you personally?

a. What changes in yourself do you hope will result from the experience? b. What do you hope to get out of the experience?

5. During the course you’ll be with the same group of people for an extended period of time. What feelings do you have about being part of a group like that for nine full days?

a. Based on your past experience with groups, how do you see yourself fitting into your group at Outward Bound?

For the Disabled

6. One of the things we’re interested in understanding better as a result of these courses is the everyday experience of disabled people. Some of the things we are interested in are as follows:

a. How does your disability affect the types of activities you engage in? b. What are the things that you don’t do that you wish you could do? c. How does your disability affect the kinds of people you associate with? Clarification:

Some people find that their disability means that they associate mainly with other disabled persons. Others find that their disability does not affect their contacts with people. What has your experience been along these lines?

d. Sometimes people with disabilities find that their participation in groups is limited. What has been your experience in this regard?

For the Able-Bodied

7. One of the things we’re interested in understanding better as a result of these courses is feelings that able-bodied people have about being with disabled folks. What kinds of experiences with disabled people have you had in the past?

a. What do you personally feel you get out of working with disabled people? b. In what ways do you find yourself being different from your usual self when you’re

with disabled people? c. What role do you expect to play with disabled people on the Outward Bound course?

Clarification: Are there any particular things you expect to have to do? d. As you think about your participation in this course, what particular feelings do you

have about being part of an outdoor course with disabled people?

8. About half of the participants on the course are disabled people, and about half are people without disabilities. How would you expect your relationship with the disabled people to be different from your relationship with course participants who are not disabled?

9. We’d like to know something about how you typically face new situations. Some people kind of like to jump into new situations, whether or not some risk is be involved. Other people are more cautious about entering situations until they know more about them. Between these two, how would you describe yourself?

10. Okay, you’ve been very helpful. Are there other thoughts or feelings you’d like to share with us to help us understand how you’re seeing the course right now. Anything at all you’d like to add?

Postcourse Interview

We’re conducting this interview right at the end of your course with Minnesota Outward Bound. We hope this will help us better understand what you’ve experienced so that we can improve future courses. You have signed a form giving your consent for material from this interview to be used in a written evaluation of the course. This interview is being tape- recorded.

1. To what extent was the course what you expected it to be?

a. How was it different from what you expected? b. To what extent did the things you were concerned about before the course come true?

b-1. Which things came true?

b-2. Which didn’t come true?

2. How did the course affect you personally?

a. What changes in yourself do you see or feel as a result of the course? b. What would you say you got out of the experience?

3. During the past nine days, you’ve been with the same group of people constantly. What kind of feelings do you have about having been a part of the same group for that time?

a. What feelings do you have about the group? b. What role do you feel you played in the group? c. How was your experience with this group different from your experiences with other

groups? d. How did the group affect you? e. How did you affect the group? f. In what ways did you relate differently to the able-bodied and disabled people in your

group?

4. What is it about the course that makes it have the effects it has? What happens on the course that makes a difference?

a. What do you see as the important parts of the course that make an Outward Bound course what it is?

b. What was the high point of the course for you? c. What was the low point?

5. How do you think this course will affect you when you return to your home?

a. Which of the things you experienced this week will carry over to your normal life? b. What plans do you have to change anything or do anything differently as a result of

this course?

For the Disabled

6. We asked you before the course about your experience of being disabled. What are your feelings about what it’s like to be disabled now?

a. How did your disability affect the type of activities you engaged in on the course? Clarification: What things didn’t you do because of your disability?

b. How was your participation in the group affected by your disability?

For the Able-Bodied

7. We asked you before the course your feelings about being with disabled people. As a result of the experiences of the past nine days, how have your feelings about disabled people changed?

a. How have your feelings about yourself in relation to disabled persons changed? b. What did you personally get out of being/working with disabled people on this course? c. What role did you play with the disabled people? d. How was this role different from the role you usually play with disabled people?

8. Before the course, we asked you how you typically faced a variety of new situations. During the past nine days, you have faced a variety of new situations. How would you describe yourself in terms of how you approached these new experiences?

a. How was this different from the way you usually approach things? b. How do you think this experience will affect how you approach new situations in the

future?

9. Suppose you were being asked by a government agency whether or not they should sponsor a course like this. What would you say?

a. What arguments would you give to support your opinion?

10. Okay, you’ve been very helpful. We’d be very interested in any other feelings and thoughts you’d like to share with us to help us understand your experience of the course and how it affected you.

Six-Month Follow-Up Interview

This interview is being conducted about six months after your Outward Bound course to help us better understand what participants experience so that we can improve future courses.

1. Looking back on your Outward Bound experience, I’d like to ask you to begin by describing for me what you see as the main components of the course? What makes an Outward Bound course what it is?

a. What do you remember as the highlight of the course for you?

b. What was the low point?

2. How did the course affect you personally?

a. What kinds of changes in yourself do you see or feel as a result of your participation in the course?

b. What would you say you got out of the experience?

3. For nine days, you were with the same group of people, how has your experience with the Outward Bound group affected your involvement with groups since then?

For the Disabled

(*Check previous responses before the interview. If the person’s attitude appears to have changed, ask if he or she perceives a change in attitude.)

4. We asked you before the course to tell us what it’s like to be disabled. What are your feelings about what it’s like to be disabled now?

a. How does your disability affect the types of activities you engage in? Clarification: What are some of the things you don’t do because you’re disabled?

b. How does your disability affect the kinds of people you associate with? Clarification: Some people find that their disability means that they associate mainly with other disabled persons. Other people with disabilities find that their disability in no way limits their contacts with people. What has been your experience?

c. As a result of your participation in Outward Bound, how do you believe you’ve changed the way you handle your disability?

For the Able-Bodied

5. We asked you before the course to tell us what it’s like to work with the disabled. What are your feelings about what it’s like to work with the disabled now?

a. What do you personally feel you get out of working with disabled persons? b. In what ways do you find yourself being different from your usual self when you are

with disabled people? c. As you think about your participation in the course, what particular feelings do you

have about having been part of a course with disabled people?

6. About half of the people on the course were disabled people, and about half were people without disabilities. To what extent did you find yourself acting differently with disabled people compared with the way you acted with able-bodied participants?

7. Before this course, we asked you how you typically face new situations. For example, some people kind of like to jump into new situations even if some risks are involved. Other people are more cautious, and so on. How would you describe yourself along these lines right now?