U8A1-68 - TUTOR PLEASE FOLLOW THE INSTRUCTIONS AS OUTLINED TO COMPLETE THIS HOMEWORK....Data Analysis Strategies...SEE ATTACHED FILES
6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice
https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 1/13
During the latter part of the school year, Mike worked on several projects at once. He worked on a project on basic electricity and took a course on “Beginning Guitar” for project credit.
To improve his communication skills, Mike also worked on an intergroup relations project. This project grew out of an awareness by the staff that Mike liked other students but seemed to lack social interaction with his peers and the staff. Reports at the beginning of the year indicated that he appeared dependent and submissive and was an immature conversationalist. In response to these observations, Mike’s learning manager negotiated project objectives and activities with him that would help improve his communication skills and help him solve some of his interpersonal problems. At the end of the year, Mike noted a positive change related to his communication skills. “I can now speak up in groups,” he said.
Mike’s unfinished project related to his own experience and interests. He had moved to the Portland area from Canada 10 years previously and frequently returns to see relatives. The project was on immigration laws and regulations in the functional citizenship area. At the same time, it would help Mike improve his grammar and spelling. Since students have the option of completing a project started during their junior year when they are seniors, Mike had a chance to finish the project this year. Of the year, Mike said, “It turned out even better than I thought.” Things he liked best about the new experience in EBCE were working at his own speed, going to a job, and having more freedom.
At the end of the year, Mike’s tests showed significant increases in both reading and language skills. In the math and study skill areas, where he was already above average, only slight increases were indicated.
Tests on attitudes, given both at the beginning and at the end of the year, indicated positive gains in self- reliance, understanding of roles in society, tolerance for people with differences in background and ideas from his, and openness to change.
Aspirations did not change for Mike. He still wants to go into computer programming after finishing college. “When I started the year, I really didn’t know too much about computers. I feel now that I know a lot and want even more to make it my career.”
The description of Mike’s second year in EBCE is omitted. We pick up the case study after his second-year experience.
Mike’s Views of EBCE. Mike reported that his EBCE experiences, especially the learning levels, had improved all of his basic skills. He felt he had the freedom to do the kinds of things he wanted to do while at employer sites. These experiences, according to Mike, have strengthened his vocational choice in the field he wanted to enter and have caused him to look at educational and training requirements plus some other alternatives. For instance, Mike tried to enter the military, figuring it would be a good source of training in the field of computers, but was unable to because of a medical problem.
By going directly to job sites, Mike has gotten a feel for the “real world” of work. He said his work at computer repair–oriented sites furthered his conception of the patience necessary when dealing with customers and the fine degree of precision needed in the repair of equipment. He also discovered how a customer engineer takes a problem, evaluates it, and solves it.
When asked about his work values, Mike replied, “I figure if I get the right job, I’d work at it and try to do my best. . . . In fact, I’m sure that even though I didn’t like the job I’d still do more than I was asked to. . . . I’d work as hard as I could.” Although he has always been a responsible person, he feels that his experiences in EBCE have made him more trustworthy. Mike also feels that he is now treated more like an adult because of his own attitudes. In fact, he feels he understands himself a lot more now.
Mike’s future plans concern trying to get a job in computer programming at an automobile dealership or computer services company. He had previously done some computer work at the automobile dealership in relationship to a project in Explorer Scouts. He also wants more training in computer programming and has
6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice
https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 2/13
discussed these plans with the student coordinator and an EBCE secretary. His attitude toward learning is that it may not be fun but it is important.
When asked in which areas he made less growth than he had hoped to, Mike responded, “I really made a lot of growth in all areas.” He credits the EBCE program for this, finding it more helpful than high school. It gives you the opportunity to “get out and meet more people and get to be able to communicate better with people out in the community.”
Most of Mike’s experiences at the high school were not too personally rewarding. He did start a geometry class there this year but had to drop it as he had started late and could not catch up. Although he got along all right with the staff at the high school, in the past he felt the teachers there had a “barrier between them and the students.” The EBCE staff “treat you on a more individual type circumstance. . . . [They] have the time to talk to you.” In EBCE, you can “work at your own speed. . . . [You] don’t have to be in the classroom.”
Mike recommends the program to most of his friends, although some of his friends had already dropped out of school. He stated, “I would have paid to come into EBCE, I think it’s really that good a program. . . . In fact, I’ve learned more in these two years in EBCE than I have in the last four years at the high school.” He did not even ask for reimbursement for travel expenses because he said he liked the program so much.
Other Perspectives and Data
The Views of His Parents. When Mike first told his parents about the program, they were concerned about what was going to be involved and whether it was a good program and educational. When interviewed in March, they said that EBCE had helped Mike to become more mature and know where he is going.
Mike’s parents said that they were well informed by the EBCE staff in all areas. Mike tended to talk to them about his activities in EBCE, while the only thing he ever talked about at the high school was photography. Mike’s career plans have not really changed since he entered EBCE, and his parents have not tried to influence him, but EBCE has helped him to rule out mechanic and truck driving as possible careers.
Since beginning the EBCE program, his parents have found Mike to be more mature, dependable, and enthusiastic. He also became more reflective and concerned about the future. His writing improved, and he read more.
There are no areas where his parents felt that EBCE did not help him, and they rated the EBCE program highly in all areas.
Test Progress Measures on Mike. Although Mike showed great improvement in almost all areas of the Comprehensive Test of Basic Skills during the first year of participation, his scores declined considerably during the second year. Especially significant were the declines in Mike’s arithmetic applications and study skills scores.
Mike’s attitudinal scores all showed a positive gain over the total two-year period, but they also tended to decline during the second year of participation. On the semantic differential, Mike scored significantly below the EBCE mean at FY 75 posttest on the community resources, adults, learning, and work scales.
Mike showed continued growth over the two-year period on the work, self-reliance, communication, role, and trust scales of the Psychosocial Maturity Scale. He was significantly above the EBCE posttest means on the work, role, and social commitment scales and below average on only the openness to change scale. The openness to change score also showed a significant decline over the year.
The staff rated Mike on seven student behaviors. At the beginning of the year, he was significantly above the EBCE mean on “applies knowledge of his/her own aptitudes, interests, and abilities to potential career interests” and below the mean on “understands another person’s message and feelings.” At posttest time, he was still below the EBCE mean on the latter behavior, as well as on “demonstrates willingness to apply basic skills to work tasks and to vocational interests.”
6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice
https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 3/13
Over the course of the two years in the EBCE program, Mike’s scores on the Self-Directed Search showed little change in pattern, although the number of interests and competencies did expand. Overall, realistic (R) occupations decreased and enterprising (E) occupations increased as his code changed from RCI (where C is conventional and I is investigative occupations) at pretest FY 74 to ICR at pretest FY 75 (a classification that includes computer operators and equipment repairers) to CEI at posttest FY 75. However, the I was only one point stronger than the R, and the CER classification includes data processing workers. Thus, Mike’s Self- Directed Search codes appeared very representative of his desired occupational future.
Evaluators’ Reflections. Mike’s dramatic declines in attitudes and basic skills scores reflect behavior changes that occurred during the second half of his second year on the program and were detected by a number of people. In February, at a student staffing meeting, his learning manager reported of Mike that “no progress is seen in this zone with projects . . . still elusive . . . coasting right now . . . may end up in trouble.” The prescription was to “watch him—make him produce . . . find out where he is.” However, at the end of the next to last zone in mid-May, the report was still “the elusive butterfly! (Mike) needs to get himself in high gear to get everything completed on time!!!” Since the posttesting was completed before this time, Mike probably coasted through the posttesting as well.
Other data suggesting his lack of concern and involvement during the second half of his senior year was attendance. Although he missed only two days the first half of the year, he missed 13 days during the second half.
Mike showed a definite change in some of his personality characteristics over the two years he spent in the EBCE program. In the beginning of the program, he was totally lacking in social skills and self-confidence. By the time he graduated, he had made great strides in his social skills (although there was still much room for improvement). However, his self-confidence had grown to the point of overconfidence. Indeed, the employer- instructor on his last learning level spent a good deal of time trying to get Mike to make a realistic appraisal of his own capabilities.
When interviewed after graduation, Mike was working six evenings a week at a restaurant where he worked part-time for the last year. He hopes to work there for about a year, working his way up to cook, and then go to a business college for a year to study computers. SOURCE: Fehrenbacher, Owens, and Haenn (1976, pp. 1, 17–21). Used by permission of Education Northwest.
EXHIBIT 8.34 Excerpts From Codebook for Use by Multiple Coders of Interviews With Decision Makers and Evaluators About Their Utilization of Evaluation Research
This codebook was developed from four sources: (1) the standardized open-ended questions used in interviewing; (2) review of the utilization literature for ideas to be examined and hypotheses to be reviewed; (3) our initial inventory review of the interviews, in which two of us read all the data and added categories for coding; and (4) a few additional categories added during coding when passages didn’t fit well into the available categories.
Every interview was coded twice by two independent coders. Each individual code, including redundancies, was entered into our qualitative analysis database, so that we could retrieve all passages (data) on any subject included in the classification scheme, with brief descriptions of the content of those passages. The analyst could then go directly to the full passages and complete interviews from which these passages were extracted to keep quotations in context. In addition, the computer analysis permitted easy cross-classification and cross-comparison of passages for more complex analyses across interviews.
Characteristics of Program Evaluated
0101 Nature or kind of program
6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice
https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 4/13
0102 Program relationship to government hierarchy 0103 Funding (source, amount, determination of, etc.) 0104 Purpose of program 0105 History of program (duration, changes, termination, etc.) 0106 Program effectiveness
Evaluator Role in Specific Study
0201 Evaluator’s role in initiation and planning stage 0203 Evaluator’s role in data collection stage 0204 Evaluator’s role in final report and dissemination 0205 Relationship of evaluator to program (internal/external) 0206 Evaluator’s organization (type, size, staff, etc.) 0207 Opinions/feelings about role in specific study 0208 Evaluator’s background 0209 Comments on evaluator and evaluator process
Decision Maker’s Role in Specific Study
0301 Decision maker’s role in initiation and planning stage 0302 Decision maker’s role in data collection stage 0303 Decision maker’s role in final report and dissemination 0304 Relationship of decision maker to program 0305 Relationship of decision maker to other people or units in government 0306 Comments on decision maker and decision-making process (opinions, feelings, facts, knowledge, etc.)
Stakeholder Interactions
0501 Stakeholder characteristics 0502 Interactions during or about initiation of study 0503 Interactions during or about design of study 0504 Interactions during or about data collection 0505 Interactions during or about final report/findings 0506 Interactions during or about dissemination
Planning and Initiation Process of This Study (How and Who started)
0601 Initiator 0602 Interested groups or individuals 0603 Circumstances surrounding initiation
Purpose of Study (Why)
6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice
https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 5/13
0701 Description of purpose 0702 Changes in purpose
Political Context
0801 Description of political context 0802 Effects on study
Expectations for Utilization
0901 Description of expectations 0902 Holders of expectations 0903 Effect of expectations on study 0904 Relationship of expectations to specific decisions 0905 Reasons for lack of expectations 0906 People mentioned as not having expectations 0907 Effect of lack of expectations on study
Data Collection, Analysis, Methodology
1001 Methodological quality 1002 Methodological appropriateness 1003 Factors affecting data collection and methodology
Findings, Final Report
1101 Description of findings/recommendations 1102 Reception of findings/recommendations 1103 Comments on final report (forms, problems, quality, etc.) 1104 Comments and description of dissemination
Impact of Specific Study
1201 Description of impacts on program 1202 Description of nonprogram impacts 1203 Impact of specific recommendations
Factors and Effects on Utilization
1301 Lateness 1302 Methodological quality 1303 Methodological appropriateness 1304 Positive/negative findings 1305 Surprise findings
6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice
https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 6/13
1306 Central/peripheral objectives 1307 Point in life of program 1308 Presence/absence of other studies 1309 Political factors 1310 Interaction with evaluators 1311 Resources 1312 Most important factor
EXHIBIT 8.35 Excerpts From an Illustrative Interview Analysis: Reflections on Outcomes From Participants in a Wilderness Education Program
—Jeanne Campbell and Michael Patton
Experiences affect people in different ways. This experiential education truism means that the individual outcomes, impacts, and changes that result from participation in some set of activities are seldom predictable with any certainty. Moreover, the meaning and meaningfulness of such changes as do occur are likely to be highly specific to particular people in particular circumstances. While the individualized nature of learning is a fundamental tenet of experiential education, it is still important to stand back from those individual experiences in order to look at the patterns of change that cut across the specifics of person and circumstances. One of the purposes of the evaluation of the Learninghouse Southwest Field Training Project was to do just that—to document the experiences of individuals and then to look for the patterns that help provide an overview of the project and its impacts.
A major method for accomplishing this kind of reflective evaluation was the conduct of follow-up interviews with the 11 project participants. The first interviews were conducted at the end of October 1977, three weeks following the first field conference in the Gila Wilderness of New Mexico. The second interviews were conducted during the third week of February, three weeks after the wilderness experience in the Kofa Mountains of Arizona. The third and final interviews were conducted in early May, following the San Juan River conference in southern Utah. All interviews were conducted by telephone. An average interview took 20 minutes, with a range from 15 to 35 minutes. Interviews were tape-recorded and transcribed for analysis.
The interviews focused on three central issues: (1) How has your participation in the Learninghouse Project affected you personally? (2) How has your participation in the project affected you professionally? (3) How has your participation in the project affected your institution?
In the pages that follow, participant responses to these questions are presented and analyzed. The major purpose of the analysis was to organize participant responses in such a way that overall patterns would become clear. The emphasis throughout was on letting the participants speak for themselves. The challenge for the evaluators was to present participant responses in a cogent fashion that integrates the great variety of experiences and impacts recorded during the interviews.
Personal Change
“How has your participation in the Learninghouse Project affected you personally? What has been the impact of the project on you as a person?”
Questions about personal change generated more reactions from participants than subsequent questions about professional and institutional change. There is an intensity to these responses about individual change that makes it clear just how significant these experiences were in stimulating personal growth and
6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice
https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 7/13
development. Participants attempted throughout the interviews to indicate that they felt differently about themselves as persons because of their Learninghouse experiences. While such personal changes are often difficult to articulate, the interviews reflect a variety of personal impacts.
Confidence: A Sense of Self
During the three weeks in the wilderness, participants encountered a number of opportunities to test themselves. Can I carry a full pack day after day, uphill and downhill? Can I make it up that mountain? Do I have anything to contribute to the group? As participants encountered and managed stress, they learned things about themselves. The result was often an increase in personal confidence and a greater sense of self.
It’s really hard to say that LH did one thing or another. I think increased self-confidence has helped me do some things that I was thinking about doing. And I think that came, self-confidence came about largely because of the field experiences. I, right after we got back, I had my annual merit evaluation meeting with my boss, and at that I requested that I get a, have a change in title or a different title, and another title really is what it amounts to, and that I be given the chance for some other responsibilities that are outside the area that I work in. I want to get some individual counseling experience, and up to this point I have been kind of hesitant to ask for that, but I feel like I have a better sense of what I need to do for myself and that I have a right to ask for it least (Cliff, post-Kofas).
I guess something that has been important to me in the last couple of trips and will be important in the next one is just the outdoor piece of it. Doing things that perhaps I’d not been willing to attempt before whatever reason. And finding I’m better at it than expected. Before I was afraid (Charlene, post-Kofas).
The interviews indicate that increased confidence came not only from physical accomplishments but also —and especially—from interpersonal accomplishments.
After the Kofas I achieved several things that I’ve been working on for two years. Basically, the central struggle of the last two years of my life has been to no longer try to please people. No matter what my own feelings and needs are I try to please you. And in the past I had done whatever another person wanted me to do in spite of my own feelings and needs. And to have arrived at a point where I could tend to my own feelings and take care of what I needed to do for me is by far the most important victory I’ve won . . . a major one.
In the Kofas, I amazed myself that I didn’t more than temporarily buy into how . . . I was being described . . . when I didn’t recognize myself yet. And that’s new for me. In the past I’d accept others’ criticisms of me as if they were indeed describing me . . . and get sucked into that. And I felt that was an achievement for me to hold onto my sense of myself in the face of criticisms has long been one of my monsters I’ve been struggling with, so to hold onto me is, especially as I did, was definitely an achievement (Billie, post- Kofas).
I’ve been paying a lot of attention to not looking for validation from other people. Just sticking with whatever kinds of feelings I have and not trying to go outside of myself . . . and lay myself on a platter for approval. I think the project did have a lot to do with that, especially this second trip in the Kofas (Greg, post-Kofas).
I would say the most important thing that happened to me was being able to talk to other people quite honestly about, I think really about their problems more than mine. That’s very interesting in that I think that I had, I think I had an effect upon Billie and Charlene both. As a result of that it gave me a lot more confidence and positive feelings. Do you follow that? Where rather than saying I had this problem and I talked to somebody and they solved it for me, it was more my helping other people to feel good about themselves that made me feel more adequate and better than myself (Rod, post-Gila).
Another element of confidence concerns the extent to which one believes in one’s own ideas—a kind of intellectual confidence.
I think if I take the whole project into consideration. I think that I’ve gained a lot of confidence myself in some of the ideas that I have tried to use, both personally and let’s say professionally. Especially in my teaching
6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice
https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 8/13
aspects, especially teaching at a woman’s college where I think one of our roles is not only to teach women subject matter, but also to teach them to be more assertive. I think that’s a greater component of our mission than normally would have it at most colleges. I think that a lot of the ideas that I had about personal growth and about my own interactions with people were maybe reinforced by the LH experience, so that I felt more confident about them, and as a result they have come out more in my dealings with people. I would say specifically in respect to a sort of a more humanistic approach to things (Rod, post-Kofas).
Increased confidence for participants was often an outcome of learning that they could do something new and difficult. At other times, however, increased confidence emerged as a result of finding new ways to handle old and difficult situations, for example, learning how to recognize and manage stress.
A change I’ve noticed most recently and most strongly is the ability to recognize stress. And also the ability to recognize that I can do a task without needing to make it stressful which is something I didn’t know I did. So what I find I wind up doing, for example, is when I’ve had a number of things happen during the day and I begin to feel myself keying up I find myself very willing to say both to close friends and to people I don’t know very well, I can’t deal with this that you’re bringing me. Can we talk about it tomorrow? This is an issue that really needs a lot of time and a lot of attention. I don’t want to deal with it today, can we talk later . . . etc. So I’m finding myself really able to do that. And I’m absolutely delighted about it.
(Whereas before you just piled it on?)
Exactly. I’d pile it and pile it until I wouldn’t understand why I was going in circles (Charlene, post-Kofas).
Personal Change—Overview
The personal outcomes cited by Learninghouse participants are all difficult to measure. What we have in the interviews are personal perceptions about personal change. The evidence, in total, indicates that participants felt differently and, in many cases, behaved differently as a result of their project participation. Different participants were affected in different ways and to varying extent. One participant reported virtually no personal effects from the experiences.
And as far as the effect it had on me personally, which was the original question, okay, to be honest with you, to a large degree it had very little effect, and that’s not a dig on the program, because at some point in people’s lives I think things start to have smaller effect, but they still have effect. So I think that for me, what it did have an effect on was tolerance. Because there were as lot of things that occurred on the trip that I didn’t agree with. And still don’t agree, but I don’t find myself to be viciously in disagreement any longer, just plainly in disagreement. So it was kind of like before, I didn’t want to listen to the disagreement, or I wanted to listen to it but resolve it. Now, you know, there’s a third option, that I can listen to it, continue to disagree with it and not mind continuing to listen to it (Cory, post–San Juan).
The more common reaction, however, was surprise at just how much personal change occurred.
My expected outcome was increase the number of contacts in the Southwest, and everyone of my expected outcomes were professional. That, you know, much more talk about potential innovations in education and directions to go, and you know, field-based education, what that’s about, and I didn’t expect at all, which may not be realistic on my part, but at least I didn’t expect at all—the personal impact (Charlene, post-Gila).
For others, the year’s participation in Learninghouse was among the most important learning experiences of a lifetime, precisely because the project embraced personal as well as professional growth.
I’ve been involved in institutions and in projects as an educator, let’s say, for 20 years I started out teaching in high school, going to the NSF institutions during the summertime and I’ve gone to a lot of Chautauqua things and a lot of conferences, you know, of various natures. And I really think that this project has by far the greatest. . . . has had by far the greatest impact on me. And I think that the reason is that in all the projects that I’ve had in the past . . . they’ve been all very specifically oriented towards one subject or toward one . . . more of a, I guess, more of a science, more of a subject matter orientation to them. Whereas this having a process
6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice
https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 9/13
orientation has a longer effect. I mean a lot of the things I learn in these instances is out of date by now and you keep up with the literature, for example, and all that and maybe that stimulates you to keep up . . . but in reality as far as a growth thing on my part, I think on the part of other participants, I think that this has been phenomenal. And I just think that this is the kind of thing that we should be looking towards funding on any level, federal, or any level (Rod, post–San Juan).
We come now to a transition point in this report. Having reported participants’ perceptions about personal change, we want to report the professional outcomes of the Learninghouse project. The problem is that in the context of a holistic experience like the Southwest Field Training Project, the personal–professional distinction becomes arbitrary. A major theme running throughout discussions during the conferences was the importance of reducing the personal–professional schism—the desirability of living an integrated life and being an integrated self. This theme is reflected in the interviews, as many participants had difficulty responding separately to questions about personal versus professional change.
Personal–Professional Change
Analytically, there is at least a connotative difference between personal and professional change. For evaluation purposes, we tried to distinguish one from the other as follows: personal changes concern the thoughts, feelings, behaviors, intentions, and knowledge people have about themselves; professional changes concern the skills, competences, ideas, techniques and processes people use in their work. There is, however, a middle ground. How does one categorize changes in thoughts, feelings, and intentions about competences, skills, and processes? There are changes in the person that affect that person’s work. This section is a tribute to the complexity of human beings in defying the neat categories of social scientists and evaluators. This section reports changes that, for lack of a better nomenclature, we have called simply personal/professional impacts.
The most central and most common impact in this regard concerned changes in personal perspective that affected fundamental notions about and approaches to the world of work. The wilderness experiences and accompanying group processes permitted and/or forced many participants to stand back and take a look at themselves in relation to their work. The result was a changed perspective. The following four quotations are from interviews conducted after the first field conference in the Gila, a time when the contrasts provided by the first wilderness experience seemed to be felt most intensely.
The trip came at a real opportune time. I’ve been on this new job about 4–5 weeks and was really getting pretty thoroughly mired in it, kind of overwhelmed by it, and so it came after a particularly hellish week, so in that sense it was just a critical, really helpful time to get away. To feel that I had, to remember that I had some choices, both in terms of whether I stayed here or went elsewhere, get some perspective of what it was I actually wanted to accomplish in higher education rather than just surviving to keep my sanity. And it gave me some, it renewed some of my ability to think of doing what I wanted to do here at the University, or trying to, that there were things that were important for me to do rather than just handling the stuff that poured across my desk (Henry, post-Gila).
I think it’s helped make me become more creative, and just, and that’s kind of tied in with the whole idea of the theory of experiential education. And the way we approached it on these trips. And so for instance I’m talking with my wife the other night, after I got Laura’s paper that she’d given in Colorado, and I said you oughta read this because you can go out and teach history and you know, experientially. Then I gave her an idea of how I would teach frontier history for instance, and I don’t know beans about frontier history. But it was an idea which, then she told another friend about it, and this friend says oh, you can get a grant for that. You know. So that was just a real vivid example, and I feel like, it’s, I’ve been able to apply, or be creative in a number of different situations, I think just because I give myself a certain freedom, I don’t know, I can’t quite pinpoint what brought it about, but I just feel more creative in my work (Cliff, post–San Juan).
You know my biggest problem is I’ve been trying to save the world, and what I’m doing is pulling back. Because, perhaps the way I’ve been going about it has been wrong or whatever, but at least my motives are clearer and I know much more directly what I need and what I don’t need and so I’m more open but less, yeah,
6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice
https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 10/13
as I said, I’ve been in a let’s save the world kind of thing, now I feel more realistic and honest (Charlene, post- Gila).
I’ve been thinking about myself and my relationship to men and my boss, and especially to ideas about fear and risk. . . . I decided that I needed to become a little more visible at the department. After the October experience, I just said I was a bit more ready to become visible at the department level. And I volunteered then to work on developing a department training policy and develop the plan and went down to the department and talked to the assistant about it and put myself in a consulting role while another person was assigned the actual job of doing it. And I think that I was ready to make that decision and act on it after I first of all got clear that I was working on male–female relationships. My department has a man, again, not a terribly easy one to know, so it’s a risk for me to go talk with him and yet I did it. I was relatively comfortable and felt very good and very pleased with myself that I had done that and I think that’s also connected (Billie, post-Kofas).
The connection between personal changes and professional activities was an important theme throughout the Learninghouse Project. The passages reported in this section illustrate how that connection took hold in the minds and lives of project participants. As we turn now to more explicit professional impacts, it is helpful to keep in mind the somewhat artificial and arbitrary nature of the personal–professional distinction.
(Omitted are sections on changed professional knowledge about experiential education, use of journals, group facilitation skills, individual professional skills, personal insights regarding work and professional life, and the specific projects the participants undertook professionally. Also omitted are sections on institutional impacts. We pick up the report in the concluding section.)
Final Reflections
Personal change . . . professional change . . . institutional change. . . . Evaluation categories aim at making sense out of an enormously complex reality. The reflections by participants throughout the interviews make it clear that most of them came away from the Learninghouse program feeling changes in themselves. Something had touched them. Sometimes it meant a change in perspective that would show up in completely unexpected ways.
For one thing, I just finished the purchase of my house. First of all, that’s a new experience for me. I’ve never done it before. I’ve never owned a home and never even wanted to. It seemed odd to me that my desire to “settle down” or make this type of commitment to a place occurred just right after the Gila trip. Just sort of one of those things that I woke up and went, “Wow, I want to stay here. I like this place. I want to buy it.” And I had never in my life lived in a house or a place that I felt that way about. I thought that was kind of strange. And I do see that as a function of personal growth and stability. At least some kind of stability.
Other areas of personal growth: one has been, and this kind of crosses over I think into the professional areas, and that would be an ability to gain perspective. Certainly the trips I think . . . incredibly valuable for gaining perspective on what’s happening in my home situation, my personal life, my professional life . . . the whole thing. And it has allowed me to focus on some priority types of things for me. And deal with some issues that I’ve been kind of dragging on for years and years and not really wanting to face up with them or deal with them. And I have been able to move on and move through those kinds of things in the last 6 or 9 months or so to a much greater extent than ever before (Tom, post–San Juan).
Other participants came away from the wilderness experiences with a more concrete orientation that they could apply to work, play, and life.
The thing that I realized as I was trying to make some connections between the river and raft trip, was that in some ways I can see the parallels of my life being kind of like our raft trip was, and the rapids, or the thrill ride, and they’re a lot of fun, but it’s nice to get out of them for a while and dry off. It’s nice sometimes to be able to just drift along and not worry about things. But a lot of it also is just hard work. A lot of times I wish I could get out of it and go a different way, and that’s been kind of a nice thing for me to think about and kind of a viewpoint to have whenever I see things in a lull or in a real high speed pace, that I can say, “Okay, I’m going
6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice
https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 11/13
to be in this for a while, but I’m going to come out of it and go into something else.” And so that’s kind of a metaphor that I use as somewhat of a philosophy or point of view that’s helpful as I go from day to day (Cliff, post–San Juan).
A common theme that emerged as participants reflected on their year’s involvement with Learninghouse was a new awareness of options, alternatives, and possibilities.
I would say that if I have one overall comment, the effect of the first week overall, is to renew my sense of the broader possibilities in my job and in my life. Opens things to me. I realize that I have a choice to be here and be myself. And since I have a choice, there are responsibilities. Which is a good feeling (Henry, post-Gila).
I guess to me what sticks out overall is that the experience was an opportunity for me to step out of the rest of my life and focus on it and evaluate it, both my personal life and my work, professional life aspect (Michael, post–San Juan).
As participants stood back and examined themselves and their work, they seemed to discover a clarity that had previously been missing—perspective, awareness, clarity—stuff of which personal/professional/institutional change is made.
I think I had a real opportunity to explore some issues of my own worth with a group of people who were willing to allow me to explore those. And it may have come later, but it happened then. On the Learninghouse, through the Learninghouse . . . and I think it speeded up the process of growing for me in that way, accepting my own worth, my own ideas about education, about what I was doing, and in terms of being a teacher it really aided my discussions of people and my interactions. It really gave me a lot of focus on what I was doing. I think I would’ve muddled around a long time with some issues that I was able to, I think, gain some clarity on pretty quickly by talking to people who were sharing their experience and were working towards the same goals, self-directed learning, and experiential education (Greg, post–San Juan).
I think what happened is that for me it served as a catalyst for some personal changes, you know, the personal, institutional, they’re all wound up, bound up together. I think I was really wrestling with jobs and career and so on. For me the whole project was a catalyst, a kind of permission to look at things that I hadn’t looked at before. One of the realizations, one of the insights that I had in the process was, kind of neat on my part, to become concrete, specific in my actions in my life, no matter whether that was writing that I was doing, or if it was in my job, or whatever it was. But to really pay attention to that. I think that’s one of the things that happened to me (Peter, post–San Juan).
These statements from interviews do not represent a final assessment of the impacts of the Learninghouse Southwest Field Training Project. Several participants resisted the request to make summary statements about the effects and outcomes of their participation in the program because they didn’t want to force premature closure.
(Can you summarize the overall significance of participation in the project?)
I do want to make a summary, and I don’t again. . . . It feels like the words aren’t easy and for me being very much a words person, that’s unusual. It’s not necessarily that the impact hasn’t been in the cognitive areas. There have been some. But what they’ve been, where the impact has been absolutely overwhelming is in the affective areas. Appreciation of other people, appreciation of this kind of education. Though I work in it, I haven’t done it before! A real valuing of people, the profession, of my colleagues in a sense that I never had before. . . .
The impact feels like it’s been dramatic, and I’m not sure that I can say exactly how. I’m my whole . . . it all can be summarized perhaps by saying I’m much more in control. In a good kind of sense. In accepting risk and being willing to take it; accepting challenge and being willing to push myself on that; accepting and understanding more about working at the edge of my capabilities . . . what that means to me. Recognizing very comfortably what I can do and feeling good about that confidence, and recognizing that what I haven’t yet
6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice
https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 12/13
done, and feeling okay about trying it. The whole perception of confidence has changed (Charlene, post–San Juan).
The Learninghouse program was many things—the wilderness, a model of experiential education, stress, professional development—but most of all, the project was the people who participated. In response after response participants talked about the importance of the people to everything that happened. Because of the dominance of that motif throughout the interviews, we want to end this report with that highly personal emphasis.
I said before I think that to know some people, that meant a lot to me, people who were also caring. And people who were also involved, very involved in some issues, philosophical and educational, that were pretty basic not only to education, but to living. Knowing these people has been really important to me. It’s given me a kind of continuity and something to hold onto in the midst of a really frustrating, really difficult situation where I didn’t have people where I could get much feedback from, or that I could share much thinking about, talking about, and working with. It’s just kind of basic issues. That kind of continuity is real important to just my feelings, important to myself. Feeling like I have someplace to go. . . . Sometimes I feel funny about placing so much emphasis on the people. . . . But the people have really meant a lot to me as far as putting things together for myself. Being able to have my hands in something that might that really offers me a way to go (Greg, post– San Juan).
SOURCE: Patton (1978a, pp. 7–9).
APPLICATION EXERCISES
1. Practicing cross-case analysis: Three case studies are presented in different chapters of this book. The story of Li presents a Vietnamese woman’s experience in an employment training program (Exhibit 4.3, pp. 182–183). Thmaris is a case study of a homeless youth and his journey to a more stable life (Exhibit 7.20, pp. 511–516). Mike’s story, in this chapter, describes his experiences in a career education program (Exhibit 8.33, pp. 638–542). Conduct a cross-case analysis of these three cases. Identify patterns (descriptive similarities across cases) and themes (the labels you give to what the patterns mean). See page 541 for differentiation of patterns and themes.
2. Conducting a causal analysis: The module on causal analysis ends with a suggested assignment. Here, it is included again as a formal application exercise. In the Sufi story that follows, analyze the data represented by the story in two ways. (1) First, try to isolate specific variables that are important in the story, deciding which are the independent variables and which is the dependent variable, and then write a statement of the form “These things caused this thing.” (2) Read the story again. For the second analysis, try to distinguish among and label the different meanings of the situation expressed by the characters observed in the story; then write a statement of the form “These things and these things came together to create ______.” (3) Discuss the implications of these different interpretative and explanatory approaches. You aren’t being asked to decide that one approach is right and the other is wrong. You are being asked to analyze the implications of each approach for interpreting qualitative case data. Here’s the case data, otherwise known as a story.
Walking one evening along a deserted road, Mulla Nasrudin saw a troop of horsemen coming toward him. His imagination started to work; he imagined himself captured and sold as a slave, robbed by the oncoming horsemen, or conscripted into the army. Fearing for his safety, Nasrudin bolted, climbed a wall into a graveyard, and lay down in an open tomb.
6/2/2018 Bookshelf Online: Qualitative Research & Evaluation Methods: Integrating Theory and Practice
https://online.vitalsource.com/#/books/9781483314815/cfi/6/42!/4/2/4/2@0:0 13/13
Puzzled at this strange behavior, the men—honest travelers—pursued Nasrudin to see if they could help him. They found him stretched out in the grave, tense and quivering.
“What are you doing in that grave? We saw you run away and see that you are in a state of great anxiety and fear. Can we help you?”
Seeing the men up close, Nasrudin realized that they were honest travelers who were genuinely interested in his welfare. He didn’t want to offend them or embarrass himself by telling them how he had misperceived them, so Nasrudin simply sat up in the grave and said, “You ask what I’m doing in this grave. If you must know, I can tell you only this: I am here because of you, and you are here because of me.” (Adapted from Shah, 1972, p. 16)
3. Exercise on diverse approaches to causal explanation: The Halcolm graphic comic on causality (pp. 635–637) retells an African fable about causality. Exhibit 8.19 (pp. 600–601) presents 12 approaches to qualitative causal analysis. (a) Select three different approaches in Exhibit 8.19, and discuss those approaches as applied to the Halcolm story about why dogs bark. (b) Identify which approaches in Exhibit 8.19 cannot be used with a single case story and why. What is needed to engage in certain kinds of causal explanatory analysis?
4. Exercise on substantive significance: Exhibit 8.35 (pp. 643–649) presents excerpts from an evaluation report. Add your own new section to the end of the report in which you discuss what you believe is substantively significant in the findings. Qualitative inquiry does not have statistical significance tests, so qualitative judgments must be rendered about what is substantively significant. Demonstrate that you can do this and understand what it means by identifying what is substantively significant in the report presented in Exhibit 8.35. Justify your judgments. (See pp. 643–649 for a discussion of substantive significance.)
5. Exercise on reflexivity: Select a topic, issue, or question on which you either have done qualitative inquiry or would like to do a qualitative inquiry. Present a brief outline of your methods, either actual or proposed. Now, write a reflexive analysis section. How does who you are come into play in this inquiry? Discuss yourself as the instrument of qualitative inquiry and analysis. What are your strengths and weaknesses that come into play for this particular actual or proposed inquiry? (See pp. 70–74 for a discussion of reflexivity.)
6. The role of numbers in analysis: MQP Rumination #8 argues that numbers (how many interviewees said what) should not be reported in a qualitative analysis; essentially, qualitative analysis should stay qualitative (see pp. 557–560). Make the opposite argument. Write your own rumination making the case that reporting how many people said what is altogether appropriate and useful.
7. Evaluating qualitative analysis: Locate a qualitative study on a topic of interest to you. How are the analysis process and approach described in the report? What questions do you have about the analysis that are left unanswered, if any? Is reflexivity addressed? Are descriptions clearly separated from interpretations? To what extent are causal explanations offered in the analysis? Are strengths and weaknesses in the data and the analytical process discussed? How is qualitative analysis software discussed, if at all? Use Exhibit 8.32 (pp. 631–632), the qualitative analysis checklist, to review and evaluate a qualitative study of your own choosing.