Problem of Practice Based on Systems Thinking
STUDYING THE COMPLICATED
MATTER OF WHAT WORKS: EVIDENCE-BASED RESEARCH AND
THE PROBLEM OF PRACTICE
JOHN M. DIRKX Michigan State University
The demand for evidence-based research (EBR) in education has evoked considerable debate regarding the nature of knowledge practitioners hold, how they come to know, and the sociopolitical contexts in which that knowledge is generated. Proponents of EBR such as Michael Feuer stress the need for research that validly identifies solutions to important prob- lems of educational practice. Critics such as Elizabeth St.Pierre decry such approaches to research on practice as epistemotogicatly inappropriate and oblivious to their political and moral implications. Both positions illuminate important dimensions of improving practice, but what works seems to get lost in the rhetoric. In this article, the author suggests that we in adult education take seriously the question of what works in practice by developing a knowledge base grounded in research methods and strategies that give voice to the particularities of prac- tice contexts, what he refers to as the "insiderperspective."
Keywords: adult education; research methods: practitioner-based research: evidence-based research
In a recent workshop designed to help university and community college teachers in Vietnam work more effectively with adult learners, the group and facilitators were discussing the importance of using small groups in the teaching of adult learners. A teacher who works in a Vietnamese community college spoke up and said that he was struggling with assessing the work of small groups of adult learners in his teaching. What, he asked, could he do to assess student performance in small, adult learning groups?
The simple elegance of this question and how it came up speaks to the heart of the fiery debate that evidence-based research (EBR) has kindled within the field of
JOHN M. DIRKX is a professor of higher, adult, and lifelong education and director of the Michigan Center for Career and Technical Education at Michigan State University. His research interests focus on teaching and learning in adult, higher, and work-based education and transformative learning (e-mail: dirkx@msu.edu).
ADULT EDUCATION QUARTERLY, Vol. 56 No. 4, August 2006 273-290 DOI: 10.1177/0741713606289358 © 2006 American Association for Adult and Continuing Education
273
274 ADULT EDUCATION QUARTERLY / August 2006
education. On the surface, the question seems to ask for "what works," reflecting a practical concem with fairly instrumental matters. Yet, it represents the kind of question that practitioners are asking all the time in professional development programs—"Give me what I can use on Monday morning." In attempting to authen- tically address the question, the thoughtful respondent finds himself or herself caught up within a maelstrom of epistemologicai and moral issues. In considering the articles by Elizabeth St.Pierre (2006 [this issue]) and Michael Feuer (2006 [this issue]), we might ask how they would respond to this teacher's question. What would be the nature of knowledge they would rely on to offer this struggling teacher hope in improving his practice? How would we respond and on what basis?
Proponents of EBR, such as Michael Feuer, argue that practitioners are look- ing to the research community for strategies that effectively address their linger- ing practice problems. According to EBR proponents, the kind of knowledge they need is derived from research demonstrating particular characteristics. Chief among these characteristics are "rigorous, systematic, and objective methods" (Redfield, 2004, p. 24), preferably involving evidence derived from conducting randomized experiments, the "gold standard" of research. Acceptable alternatives include quasi-experimental designs using equating procedures or studies that employ regression discontinuity design. The intent of EBR and "what works" is to provide the educational consumer with confidence in being able to identify forms of practice that have been demonstrated, through application of this rigor- ous methodological approach, to produce intended, desired effects on student leaming. To this end, the U.S. Department of Education's Institute of Education has funded the What Works Clearinghouse (WWC; www.whatworks.edu.gov) to help evaluate and synthesize studies that meet these research criteria.
Lest we think that this argument is restricted only to the K-12 system, St.Pierre warns us that the government's focus on EBR and what works is moving through all levels of education, including adult education, a development that she and others vigorously oppose. Critics of EBR, such as St.Pierre, argue that the stress on EBR has more to do with political and ideological agendas than it does with improving the quality of educational experiences for students. They question the epistemologicai assumptions reflected in this approach to research and knowl- edge and suggest that an obsession with experimental research methodologies for demonstrating particular strategies that work in practice oversimplifies the com- plexity of practitioner knowledge. Such approaches also implicitly contribute to the control of education by politicians and corporate leaders (Olson, 2003), a claim that finds support in Alan Schoenfeld's (2006) description of his brief tenure with the WWC. St.Pierre charges that EBR is "retrograde science," indicative of the old process-product paradigm of 40 years ago. It bmshes aside qualitative methodologies and alternative epistemologies that have emerged in educational research over the past 30 years as "unscientific." Proponents of EBR largely ignore the ways in which knowledge and processes of coming to know are deeply embed- ded within the social, cultural, political, and economic relationships that constitute the various contexts of educational research.
Dirkx / STUDYING THE COMPLICATED MATTER OF WHAT WORKS 275
The writings of St.Pierre and Feuer provide for the adult education research community an opportunity to reflect on the meaning of research and scholarship within the field and its overall contribution to practice. Feuer and St.Pierre seem to have staked out positions that represent dramatically different epistemological and methodological stances for adult education researchers. In calling for adher- ence to rigorous methodological requirements, Feuer and EBR proponents mini- mize the moral, political, and epistemological complexity we know to constitute the nature of adult education practice. By stressing the political and ideological nature of research and charging EBR with being the stepchild of conservative and neo- conservative agendas, St.Pierre fails to adequately address an equally obvious issue, that of helping adult educators develop more effective ways of addressing the needs of their learners, be they individuals, groups, organizations, or communities.
Because I am not one of the original authors, my response to these articles is not in the form of a traditional "rejoinder." Rather, I use these articles to redirect our attention as adult education researchers to what I consider to be a key issue in this debate: the importance of the nature of adult education practice and our relationship, as researchers, to practice. I suggest the core of this debate is about the nature of professional practice itself (Anderson & Herr, 1999). In adult edu- cation, our research should reflect both the call for what works and the episte- mological, moral, and political complexity that such a call unwittingly surfaces. A concem for what works calls for "research on practice" (Richardson, 1994). Such a focus is insufficient, however, because it can stress formal, academic knowledge created by university researchers and can marginalize local knowl- edge generated and constructed by or with practitioners.
In this article, I encourage us to take this further and argue for the importance of practice-based or "insider research" (Anderson & Herr, 1999) as a means of both addressing my colleague's question about assessing learning in small groups and contributing to a broader knowledge base about adult education practice. Practice-based research refers to a general perspective on the relationship of research and practice. I use it in ways similar to what Richardson (1994) refers to as "practical inquiry" (p. 5). She defines practical inquiry as a form of research on practice that focuses on how practitioners understand their contexts, practices, and learners. For Richardson, practical inquiry is one form of research on prac- tice, but I suggest that it can and should be a major way we think about research in adult education.
In developing this position, I resonate with aspects of the arguments of both St.Pierre and Feuer. My epistemological and methodological positions on this question of what works, however, are closer to that of St.Pierre. But I write this response neither as a postmodern or poststructural scholar nor as an advocate of "scientific" studies of educational practice. Rather, I write as a person with an orientation toward practice-based research and with an abiding interest in how practitioners "transform" research-based information into knowledge they can use to guide their practice and at the same time potentially contribute to a broader knowledge base. My continuing interests in and experiences with adult education
276 ADULT EDUCATION QUARTERLY / August 2006
reflect a focus on curriculum, teaching, and learning, and it is through this set of experiential lenses that I develop this response.
I want to attend to the question of researching what works as well as the epistemologicai, moral, and political complexity that such a question evokes. I argue for an approach to practice that is guided by an overarching aim of giving voice to the particularities of specific practice settings, the insider perspective of practice. This view stands in sharp contrast to the outsider perspective reflected in EBR. Insider research can involve a focus on technical issues and use tradi- tional quantitative methodologies to investigate these issues. But such moves are understood within a broader narrative that seeks to give voice to the world of practice as perceived, understood, and stmggled with from the inside. In this view, what works is always seen in relationship to what is desired or valued. As Cherryholmes (1992) points out with regard to pragmatism, "Not everything that works is desirable, not every belief that is true is to be acted upon" (p. 14).
I develop this argument around the following points: (a) Helping practitioners and the adult education community identify what works should be a major goal of our effort in adult education research. Such an effort, however, requires us as researchers to remain in close contact with, if not immersed in, the world of prac- tice; (b) what works is troubled by the nature of practice; and (c) developing an understanding of what works is enhanced by insider research that gives voice to the epistemologicai, moral, and political complexities of practice. First, however, I provide a brief context for this discussion.
PROVIDING A CONTEXT FOR STUDYING WHAT WORKS
In a former life, I was a microbiologist who first worked for several years in a clinical laboratory and later became involved in the education and training of medical technologists and medical students. Similar to many practicing adult educators and university professors, I was introduced with very little formal train- ing in education to the challenges of curriculum planning, teaching, and assess- ing student leaming. Simplistic and highly instmmental notions of knowledge and what it means to come to know guided this early work. I had a job to do. These students knew little, if anything, about how to isolate, culture, and identify microorganisms from various clinical material. Each of these dimensions of the work was associated with specific tasks, requiring particular knowledge and skill. With respect to what I wanted them to learn, they were empty vessels. My task was to provide them with the knowledge and skills necessary to adequately per- form these tasks, when necessary, within a clinical situation. With no formal preparation for this work, I initially relied on how I was taught and trained. For me, what worked was informed by what worked in my training, a view shared by many adult educators and university teachers.
Admittedly shaped by a naive understanding of teaching and learning, these early experiences awakened my interests in curriculum, learning, and what it means to come to know. In graduate school, my work as a curriculum specialist
Dirkx / STUDYING THE COMPLICATED MATTER OF WHAT WORKS 277
within a department of medicine focused on the practical matter of developing a physician-training curriculum for the emerging field of primary care internal medicine. This experience further fueled my interest in how practitioners come to know, how they represent and act on the theoretical knowledge they were acquir- ing within their training programs, and educational methods that foster their emerging clinical knowledge. For reasons that I will describe later, it was the beginning of growing doubt in the use of what I would later learn was referred to as "technical rationality" in planning adult learning experiences (Schon, 1983).
As I entered academic life, my interests focused on education for the academi- cally underprepared adult leamers. Influenced by ethnographic training in gradu- ate school, my attention turned from a technical approach to defining what learn- ers need to know to a more interpretive perspective on how teachers made sense of their work with these leamers. At the time, I did not have the language for this approach to the adult education classroom, but I now understand it to represent a form of research on practice (Richardson, 1994). In this work, I sought to develop more theoretically grounded understandings of such problems as how teachers' beliefs and understandings of their practice shaped the kinds of leaming experi- ences they created for adult leamers (Dirkx, Fonfara, & Flaska, 1993; Dirkx & Spurgin, 1992), how curricula in these programs came to be represented (Dirkx, Amey & Haston, 1998; Dirkx & Prenger, 1994, 1997), what might be contributing to student persistence and attrition in adult literacy and General Equivalency Diploma (GED) programs (Dirkx & Jha, 1994), and how teachers might better understand and foster online collaborative group work (Dirkx & Smith, 2003).
My own journey as an educational practitioner and later also as a researcher implicitly reflects the epistemological complexity inherent in adult education practice with the process of struggling to make a difference and helping to define what works. It is a joumey first grounded in technical rationality that then gives way under a deepening appreciation for the complexity of practice. Concerned with what I saw as the epistemological limitations of technical rationality, I adopted a more interpretive stance on practice, reflected in a kind of practical inquiry that is now being informed by a growing awareness and understanding of the moral and political dimensions of our work. I now tum to a discussion of these ways of understanding practice and how they relate to helping define what works within all this complexity.
WHAT WORKS: PRACTICE AS A KIND
OF TECHNICAL RATIONALITY
At the core of EBR is a technical-rational understanding of the nature of edu- cational practice (Schon, 1983), a view reflected in my early work as an educa- tional practitioner. According to technical rationality, adult education practice involves the instrumental application of "proven" knowledge to particular prob- lems and issues that arise in helping individuals, groups, organizations, and com- munities leam and grow. This technical-rational view of practice undergirds EBR.
278 ADULT EDUCATION QUARTERLY / August 2006
According to Schon (1995), technical-rational approaches to educational research are characterized by two fundamental assumptions: (a) Practice is instmmental and (b) practice problems can be addressed and practice improved through the application of research done by outsiders. From this perspective, researchers identify the knowledge needed to address such problems and questions and trans- form this knowledge into technical principles that can then be used to guide the practitioner's work. For example, we should be able to look to the research liter- ature to derive specific principles that we can apply to the problem of assessing group performance. In this view, the value of educational research rests with its capacity to generate valid, generalizable, lawlike statements that can be the basis for the formulation of technical principles of practice. Institutions such as WWC are charged with evaluating and synthesizing those educational studies that adhere to the rigorous requirements of "scientifically based research."
This technical-rational view of practice is clearly evident in much of the research in adult education and education for the professions. During the late 1970s and early 1980s, medical education research was a fledging enterprise. Scholarly joumals, however, were rapidly reporting "scientific" studies of peda- gogical and curricular innovations related to this and other medical disciplines. Curriculum development reflected the influence of a behavioral paradigm (Jensen & Dirkx, 1989) illustrative of the kind of medical education reform that Abraham Flexner (1910) called for many years ago, the principles of scientific curriculum making reflected in the work of John Franklin Bobbitt (1918), and the behavioral approach to curriculum making advocated by Ralph Tyler (1950) and Robert Mager (1975). In attempting to better understand and improve patient care, algo- rithmic models of the clinical interview and decision-making processes were developed. Behavioral approaches were utilized to help improve doctor-patient relationships. Educational researchers conducted "scientific" studies of teaching approaches in medical schools and clinics and their effectiveness in bringing about certain educational outcomes.
This approach to developing a scientific approach to medical curricula, edu- cation, and training reflected, in many respects, the "process-product" model of teaching, a point St.Pierre clearly establishes with her reference to EBR as "ret- rograde science." In short, this paradigm suggested that effective curriculum planning and teaching strategies and methods could be determined through the application of careful, controlled, randomized, quantitative, quasi-experimental designs, testing one particular approach with another or with a control group. If randomization was not possible, a myriad of other quantitative strategies and creative research designs were offered to help control the threats to validity posed by lack of randomization (D. T. Campbell & Stanley, 1963).
This version of process-product thinking continues to manifest itself in the study of work-related leaming and adult literacy education. Over the past 15 years, the scholarship of workplace leaming within the United States has surged, due in large part to the formation of the Academy of Human Resource
Dirkx / STUDYING THE COMPLICATED MATTER OF WHAT WORKS 279
Development (AHRD). Altemative epistemologies and ways of knowing are clearly in evidence at their annual conferences and resulting publications (e.g., Daley, 2001; Dirkx, 2005; Fenwick, 2000). Yet, a brief pemsal of the conference's annual proceedings from any given year will bear witness to the pervasiveness in work-related educational research of a performance-oriented, positivist episte- mology reflected in the EBR movement. Although research in adult literacy edu- cation is characterized by more studies using altemative epistemologies, the call for practice guided by EBR is clearly evident in this field as well (Comings, Beder, Bingman, Reder, & Smith, 2003).
In professional development activities, adult education practitioners continue to seek what they refer to as practical solutions to the immediate needs of practice. For anyone who has spent time with adult education practitioners, this observa- tion is undoubtedly not surprising. But it also hints at the underlying difficulties of outsider research and the technical-rational view of educational practice. In addressing the problems they encounter in practice, adult educators tend to look to one another, rather than theory or the research community, for ideas, tips, and techniques that work. Many educational practitioners and policy makers have not even heard of many of our most valued research joumals. At a recent meeting of adult basic education, English as a second language (ESL), and GED practition- ers, Thomas Sticht, an intemationally known scholar of adult literacy education, asked how many in the audience of about 400 practitioners had heard of the National Adult Literacy Survey (Kirsch, Jungeblut, Jenkins, & Kolstad, 1993) or the National Assessment of Adult Literacy (2003). In response to either question, no more than 6 to 8 persons in his entire audience raised their hands. Clearly, we continue to experience a profound disconnect in adult education between outsider research and theory and insider needs and knowledge of practice.
T H E TROUBLING NATURE OF PRACTICE
IN ADULT EDUCATION RESEARCH
Anderson and Herr (1999) suggest that colleges of education, in which most of our preparation programs and research faculty in adult education are located, face a difficult problem. On one hand, they need to legitimate themselves to an environment that includes both "a university culture that values basic research and theoretical knowledge and a professional culture of schooling that values applied research and narrative knowledge" (p. 12). Schon (1995) suggests that the price of admission for these professional schools to the university research culture is technical-rationality.
Herein lies the problem for adult education and perhaps for other professional fields within the university as well. Adult education is, fundamentally, a field of practice. Unlike sociology, anthropology, psychology, and other social sciences that inform its research, practice provides the core focus of our collective con- cem. As educators, our focus is on fostering leaming and change among our
280 ADULT EDUCATION QUARTERLY / August 2006
Students, be they individuals, groups, communities, organizations, or whole societies. As Cervero and Wilson (1996) suggest, educational practice aims to "change the way people think and act" (p. 92), to act in the world to somehow make it differ- ent. Although obviously much more complex than this simplistic statement sug- gests, at its core that is what adult education is about. As researchers, our concern for the improvement of practice, of finding what works, animates much of our scholarship.
This simplistic statement also invites us to consider its underlying complexity. From one perspective, EBR draws our attention to the question of what works and seeks to contribute to a body of knowledge that practitioners can use to address such questions within their practice. As St.Pierre suggests, positivist approaches to the relationship of research and practice have been tried in adult education, with relatively little illumination on the complex and difficult curricular and pedagogi- cal problems with which practitioners struggle. In its commitment to the gold stan- dard of research and its quest for certainty, EBR and technical rationality gloss over difficult and troubling questions within the nature of practice. For example, Wenger's (1998) work on communities of practice stresses the importance of meaning within practice. For Wenger, "Practice is, first and foremost, a process by which we can experience the world and our engagement with it as meaningful.... But a focus on practice is not merely a functional perspective on human activities . . . it is not a mechanical perspective" (p. 51). In negotiating the meaning of practice and what works, practitioners go beyond mechanistic conceptions of getting things done, rendering meaning to the activities engaged. Wenger argues, "In the end, it is the meanings that we produce that matter" (p. 51).
In the conception of research to practice reflected in EBR, practitioners receive the wisdom of the researchers and apply it technically to particular problems of practice. From Wenger's (1998) perspective, however, through their interactions with one another and the institutional demands in which they operate, practition- ers construct the knowledge they use to address these problems. This knowledge and the meaning of what they do are bound up within the particular situations in which they work and the relationships of which they are a part within these situ- ations. According to Wenger,
All that we do and say may refer to what has been done and said in the past, and yet we produce again a new situation, an impression, an experience: we produce mean- ings that extend, redirect, dismiss, reinterpret, modify, or confirm—in a word, nego- tiate anew—the histories of meanings of which they are a part. (p. 52)
As Schon (1983) suggests, practice involves a significant interpretive dimension in which we interact and negotiate with the setting in which we find ourselves. Practice involves crafting what we need to know to respond to the unique demands and challenges of the particular context we are facing. This idea of practice goes well beyond that implied by technical-rational conceptions of what adult educators do.
Dirkx / STUDYING THE COMPLICATED MATTER OF WHAT WORKS 281
To further illustrate how the nature of practice troubles this technical-rational view, I retum briefly to my experiences in curriculum work with the health pro- fessions. In the mid-1980s, we spent more than 5 years carefully delineating and describing, from a positivist, technical-rational perspective, the particular knowl- edge and skills required for the effective practice of primary care internal medi- cine (Jensen & Dirkx, 1989). Yet, when we were "finished," it seemed as though we had not fully captured the essence of this form of practice. Despite powerful formative experiences in the biological and clinical sciences, this early curricu- lum work caused me to seriously question the efficacy of technical rationality and a positivist epistemology to curriculum making in education for the professions and, more broadly, adult education (Dirkx, 1986). Although I believe such speci- fication makes a modest contribution to our understanding of skilled or expert performance within a given domain, much in the world of medical (and educa- tional) practice seems unspecified, a kind of "tacit" knowledge (Polanyi, 1998) that remains refractory to our bold attempts to lay it bare for all the world to see. Practitioner knowledge seemed much more complex than simply specifying com- petent behaviors needed to be effective. We as practitioners rely heavily on "craft- like" knowledge (Schon, 1983) to be effective in our work.
The process of negotiating meaning and interpreting our settings in adult edu- cation practice also takes place within complex moral and political contexts (Cervero, 1988; Cervero & Wilson, 1994, 1996; Cervero, Wilson, & Associates, 2001; Forester, 1989; Nowlen, 1988). Writing from the context of program plan- ning practice, Wilson and Cervero (1996) note that "we construct educational programs that change the way people think and act" (p. 92), and in so doing we use our power to negotiate interests that ultimately shape the nature of the edu- cational practice. This characteristic of practice illuminates its social, moral, and political nature, about which EBR is essentially silent. How practitioners perceive and make sense of the problems they face in practice is fundamentally connected to what ultimately gets done. In other words, what works reflects not only the mechanical demands of the institution or the context but also what is desired and the ways in which the practice problem is construed and the processes that are used to negotiate that construal.
Scholars working from critical theory (Brookfield, 2005; Freire, 1990), post- modernist and poststructuralist (Butler, 1990; Cunningham, 2000; Tisdell, 1998; Usher, Bryant, & Johnson, 1997), feminist (English, 2006; Hayes & Flannery, 2000; hooks, 2000; Ryan, 2001), Black feminist (Johnson-Bailey, 2006), and Africentrist (Colin & Guy, 1998; Haymes, 1995; Sheared, 1994) perspectives also contribute to a critique of the essentialist, causal, and instrumental language reflected in approaches to research illustrated in EBR. These issues of meaning, ethics, and power trouble the rather static technical and mechanical nature of prac- tice conveyed by EBR and complicate overly simplistic notions of what works in adult education practice. Many of these objections are forcefully articulated in the St.Pierre article. Both their objections and my own budding relationship with
282 ADULT EDUCATION QUARTERLY / August 2006
epistemologicai issues reflect a concern with the distorted view of practice that emerges from an approach to educational research grounded in EBR.
In summary, my reservations with the EBR perspective rest with its underly- ing assumptions about the nature of educational practice and the ways it claims to represent knowledge useful to practitioners. By stressing a need to clearly delineate and decontextualize through behavioral specification the particular behaviors, knowledge, and attitudes we seek to foster, EBR severely understates the nature of practical knowledge. The knowledge that experienced practitioners use to guide and improve their practice does not reflect a kind of "key principles" or "best practices," the focus of much of the EBR perspective and governmental policy makers. Although such sets of statements are useful as orienting frame- works, they simply fail to capture the complexity that characterizes "theories- in-use" (Argyis, 1974) or the ways in which race, gender, and social class struc- ture relate to what we know, our access to knowledge, and our ability to use it effectively in our daily lives.
Yet, a thread within some of this scholarship troubles me. Chalk it up to con- versations during the mid-1970s with my Marxist friend Maury, who, in the early years of my own career, would sit with us in bars and deride the system and its evils, with no clue as to its altemative. Some scholars advocating alternative epis- temologies reflect a similar preoccupation with what is wrong with positivist understandings of research and practice but offer us little in the way of alterna- tive ways of addressing the problems with which practice presents us. Other the- orists stay close to troubles of practice and attempt to articulate implications for practice that address issues of social justice (e.g., Cunningham, 2000; Sheared, 1994). How they would address the questions many practitioners present to us, however, such as the Vietnamese community college instructor, remains unclear.
St.Pierre would suggest, as she does of the proponents of EBR, that I am embedded in a way of knowing that contributes to my current way of being, and because of this position, I am unable to fully see the world through these altema- tive epistemologies. Perhaps, but what concerns me more about such epistemo- logicai approaches to research and practice is reflected in the overall tone of St.Pierre's article itself. Virtually the entire article focuses on challenging the ascendancy of EBR and the positivist epistemology emblematic of this approach to knowing. Consistent with her deconstructionist perspective, St.Pierre appro- priately identifies the implicitly insidious nature of this worldview on educational policy, practice, and research. Yet, we are left with little sense of how might we, as adult education researchers, address the kinds of problems described in the opening vignette of this article or the perennial imperative that practitioners call for in professional development workshops—just show me what t can use on Monday moming. She forcefully presents an articulate if somewhat rambling account of what is wrong with the EBR perspective, but she offers little in the way of an altemative frame for addressing the educational problems EBR seeks to address. In this regard, I have to agree with Feuer, who argues that many of her
Dirkx / STUDYING THE COMPLICATED MATTER OF WHAT WORKS 283
positions lack support or evidence. Her position is riddled with claims that are left undeveloped and unsubstantiated.
Although I remain sympathetic to her argument, I am not sure railing at the system is the most effective way to encourage an alternative approach to the prob- lem EBR purports to address. If what works does not work, what will? How can we position our work and our research so that we are helpful to my teacher friend from Vietnam and others like him? I recognize this perspective may well be beyond the scope of what she intended with this article, and perhaps she has artic- ulated this position elsewhere. Yet, the polemical nature of her argument seems to suggest more of a rallying call to resist the insipid creep of an implicit conspir- acy within govemmental policy makers to increasingly insert their particular political agenda into educational research and practices dedicated to helping schoolchildren and adult learners. Although this resistance may be needed, we also must attend to the problem of what works.
Having said all that, how do we come to understand and practice a kind of research or scholarship that focuses on addressing and improving the troubles of practice?
PRACTICE-BASED, INSIDER RESEARCH
IN ADULT EDUCATION
In this final section, I want to argue for more of a role in adult education of practice-based insider research. A Vietnamese colleague with whom I worked on a program-planning project in Vietnam recently described an article that he and four other colleagues were publishing. The research was based on the concept of reflective teaching and involved each of the teachers studying and reflecting on their teaching practice. The article focuses on not only the process of reflective teaching but also how this work contributes to improvement of their practice. This project illustrates a model of research and scholarship for the field of adult and continuing education. Known by other names, such as the practitioner as researcher (A. Campbell, McNamara, & Gilroy, 2004; Robinson & Lai, 2006) and the reflective practitioner (Schon, 1983), this approach recognizes practice as the central focus for our research and the critical role of observation and reflec- tion on developing new understandings and ways through practice problems. It is by no means a new idea but perhaps an idea that, in the wake of a stinging assault from EBR advocates, merits revisiting.
Although insider research does not necessarily imply qualitative versus quan- titative approaches, it nonetheless stresses problems and questions that arise within and among practitioners as they engage their work in the field. This per- spective is intended to address the limitations of an outsider, technical-rational approach to research on practice in adult education. As Clandinin and Connelly (1995) suggest, practitioners often experience outsider knowledge as a stream of conclusions that reach them through various informational sources and that trans- mit prepositional and theoretical knowledge with little or no awareness of
284 ADULT EDUCATION QUARTERLY / August 2006
contextual nuances of their practice landscape. Insider research seeks to honor and give voice to complexity and the multilayered nature of understanding that adult educators hold about the various dimensions of their practice, such as their students, the curriculum, teaching strategies, and issues of gender, race, class, and other social issues (Anderson & Herr, 1999). For example, a math teacher recently explained to me his experience of moving from the university to the com- munity college. Despite being charged with teaching similar courses, he changed his teaching strategies, texts, materials, and ways of interacting with the students, based on his growing awareness of the complex nature of their lives as well as the organizational contexts of the community college.
Practice-based, insider research grounds itself within the narrative of practice (Anderson & Herr, 1999). This idea of the power of story, rather than randomized control group designs favored by EBR proponents, as a method for generating useful knowledge about and for practice characterizes the work of many researchers in adult education. At the center of such approaches is the action of practitioners reflecting on and narrating the stories of their work (Schon, 1983). There is much from which to choose as examples, but I would like to focus briefly on a few of these works that reflect differing commitments to research and prac- tice hut nonetheless advance our understanding through the use of the story. The first is illustrated in Jane Vella's (2002) text, Learrting to Listert, Learning to Teach. In this text, Vella describes 12 principles for helping adults learn. In pre- senting and elaborating these principles, however, Vella uses stories from her experience worldwide in planning and teaching programs for adult learners and their communities. Each principle is couched within a story that reveals its par- ticular context, demands, uncertainties, complexities, and ambiguities. Rather than sitting apart and decontextualized from the messy fray of educational practice, the principle itself seems to organically arise from within the story she narrates. Students and practitioners respond quite favorably to this presentation of educa- tional "theory" because they are able to immediately grasp and relate the principle to the particular aspects of practice for which it is intended. They do not leave her stories with lists of best practices, but they do hold an image in their minds of what this principle in action might look like. In teaching, we do similar things when we effectively use examples from our own experiences as educators.
But there is more going on here than simply couching theoretical principles or knowledge in interesting examples. Vella's (2002) method is not unlike that used by scholar practitioners in psychotherapy. Like most educational contexts, ran- domization, control groups, and large sample sizes are virtually impossible. How then does the field of psychotherapy deyelop new knowledge? With therapists dating back to the use of self-analysis by Freud (1955) and Jung (1963) and per- haps earlier, scholars in psychotherapy study their own practices, one patient at a time. Their method involves disciplined observation and critical self-reflection. As they seek to identify, explore, elaborate, and convey theoretical ideas about therapy, they rely on vignettes from their patients' lives and their interactions with
Dirkx / STUDYING THE COMPLICATED MATTER OF WHAT WORKS 285
the patients. As with Vella, their ideas are often couched within individual case descriptions or particular problems with which they are wrestling. As they observe, study, and reflect on their patient cases, these scholar practitioners are able to per- ceive patterns that serve to verify or refute initial hunches they may have. Through the descriptive stories provided, other practitioners using a similar ther- apeutic approach are able to examine the proposed theoretical ideas within the context of their own experiences.
In medical education, this reflective narrative process has been also used to study and improve the interactions and relationships that physicians establish with their medical patients. The overall quality of medical practice often turns on the quality of relationships and interactions that physicians have with their patients (Mizrahi, 1986). In particular, the medical interview itself represents a powerful diagnostic tool. Studies of the doctor-patient relationship indicate that the success of this interview rests with the complex psychosocial aspects of the interaction. If practitioners are unaware of these powerful emotional dynamics and how they shape and influence their interactions with patients, they are likely to engage in behaviors that may compromise the overall quality of the interview process as a diagnostic or therapeutic intervention.
Informed largely by the work of Michael Balint (1964), the use of doctor- patient relationship groups essentially relies on practitioners telling and elaborating clinical stories around difficult problems they are experiencing in their clinical practice. Balint's method on doctor-patient relationships helps physicians become more aware of these dynamics and how they shape their clinical actions. This process occurs by engaging the practitioner, within a small group of his or her peers, to relate the details of a case that is troubling or problematic. In so doing, the presenting doctor unconsciously becomes the patient and the group the doc- tor. In this fascinating role reversal process, the group mirrors back to the pre- senting doctor issues of which he or she may not have been aware. In so doing, the practitioner becomes more aware of the psychodynamic complexity of his or her relationships with patients and how interactions may be improved and made more effective.
West (2001) applies this theoretical approach to understanding the lifelong leaming practices of general practitioners in inner-city London. In so doing. West broadens the framing of the problem beyond the particulars of the interpersonal relationship and demonstrates how broader social, cultural, and political issues often reveal themselves within the emotional lives of physicians seeking to leam and improve under difficult circumstances. West relies on the power of narrative to help practitioners understand and make sense of difficult dimensions of their work and to help them see their way through to a more effective stance and rela- tionship with these aspects of their practice.
Many more examples of the power of reflection and narrative in addressing problems of practice within educadon could be cited and described, such as reflective practice (Schon, 1983), action research (Carr & Kemmis, 1986), and
286 ADULT EDUCATION QUARTERLY / August 2006
action inquiry approaches (Brooks & Watkins, t994; Watkins & Marsick, t993). Although conducted by academic researchers, the study of program planning by Cervero and Wilson (1994) illustrates aspects of this practice-based, insider research perspective as well. My point here is that the use of a reflective, narra- tive insider approach in the study of adult education practice or adult leaming rep- resents an important approach to generating deeper understanding of practice. In so doing, the practitioner becomes involved in the research, observing and reflecting on his or her practice. Practitioners themselves learn to become the researchers of their own practice. When framed as a scholarly enterprise, this practitioner-based research can potentially build and contribute to a body of knowledge in the way that psychotherapists and other professions build a knowledge base within their fields (Anderson & Herr, 1999; Richardson, 1994).
Narrative approaches to research and practice offer an altemative to the narrow and limited application to adult and continuing education of experimental and quasi-experimental research. Narrative researchers seek understanding and inter- pretation of the world. By engaging practitioners in dialogue with the researcher and, by extension, themselves and their practices, narrative researchers create opportunities for practitioners to perceive and address problems in a situated and contextual manner. The EBR folks, of course, have all kinds of difficulties with this approach: too small an n, too few controls over bias and threats to validity, lack of objective and generalizable results, and so forth. What narrative research recog- nizes, however, is that "the knower and the known interact and shape one another" (Denzin & Lincoln, 2003, p. 33). Such an assumption requires our research to use interpretive and naturalistic methods that recognize the multiple ways in which real- ity can and is constructed within practice settings. To reiterate an earlier point, this does not always mean, however, exclusively qualitative methods.
This practice-based, insider approach to research in adult education calls for what Schon (1995) refers to as a new form of scholarship, one that conflicts with the norms and values of technical rationality embedded in EBR. It is a form of scholarship that recognizes practitioners as legitimate producers of knowledge that is both local and potentially applicable to other contexts as well. It also involves academics in collaboration with practitioners in practice-based research, and this is a role to which many of us in research universities might gravitate. But I would also encourage us, as exemplified by the field of psychotherapy, to consider researching our own practice. Understanding why so many of us write so many failed grant applications for outside funding to pursue the study of practice settings miles from our institutions is a task beyond my scope here. But I would suggest the field would be better off and our graduates would be better off if we took sed- ously our own teaching contexts as locations of insider, practice-based research.
I am not suggesting that the practice-based, insider research model represents the only way we should do research in adult and continuing education. An EBR perspective is useful and can play an important role in helping to inform and shape policy. If, however, our primary interests are to understand and improve
Dirkx / STUDYING THE COMPLICATED MATTER OF WHAT WORKS 287
practice (however that is defined), particular practice questions that involve matters of the curriculum or of teaching and leaming, EBR methods will neces- sarily play a more limited role.
I am not suggesting that simply telling stories about our practice is a satisfac- tory formulation of an alternative to EBR, although it might represent a very good beginning. I am suggesting, however, more widespread use of systematic approaches to the use of reflection and narrative with one's own practice, to encourage and train more practitioners to study their own practices, to suggest that we as academics begin by taking seriously our own teaching as a context for scholarly research, and to mentor our graduate students in such an approach to the study of the field. In short, I am suggesting that we consider featuring more prominently case methodologies that stress the practitioner as researcher, not unlike that which characterizes much of the field of psychotherapy and other pro- fessional disciplines. Combined with the alternative epistemologies suggested by St.Pierre and by many adult education researchers, we are provided with a pow- erful conceptual and methodological framework for developing ethical and just approaches to identifying what works. In doing so, we both address the troubling problems of practice and inform our theoretical understanding of that practice.
Like medical education practitioners and their counterparts in primary, sec- ondary, and higher education, practitioners in adult and continuing education seek to bring about leaming and change, whether in individual understandings and behaviors or institutional and social policies. On this point, with some reluctance I resonate with Feuer's stance as one interested in the practice of policy. As schol- ars within this field, we should be committed to the understanding and improve- ment of practice. Far too many educational researchers fancy education as a kind of social science and themselves as social science scholars, perhaps even scien- tists. At times, their writing and scholarship seems to become increasingly obtuse and inaccessible to other interested scholars, let alone scholarly practitioners. This perspective is corrosive for adult education, both as a field of practice and as a field of scholarship. Our work as scholars needs to be and should be anchored in practice. Without practice, we as researchers have little reason to justify our existence. These ends are also implicit in the EBR movement in education and, more broadly, in other professional practice fields, such as medicine. The articles of Feuer and St. Pierre, when taken together, implicitly suggest a new scholarship for adult education, one that honors both the question of what works and the epis- temologicai, moral, and political nature of adult education practice.
REFERENCES
Anderson, G. L., & Herr, K. (1999). The new paradigm wars: Is there room for rigorous practitioner knowledge in schools and universities? Educational Researcher, 28, 12-21, 40.
Argyris, C. (1974). Theory in practice: Increasing pmfessionat effectiveness. San Francisco: Jossey-Bass.
288 ADULT EDUCATION QUARTERLY / August 2006
Balint, M. (1964). The doctor, his patient, and the illness. New York: International Universities Press. Bobbitt, J. F. (1918). The curriculum. Boston: Houghton Mifflin. Brookfield, S. D. (2005). The power of critical theory: Liberating adult learning and teaching. San
Francisco: Jossey-Bass. Brooks, A., & Watkins, K. E. (Eds.). (1994). The emerging power of action inquiry technologies. San
Francisco: Jossey-Bass. Butler, J. (1990). Gender trouble: Feminism and the subversion of identity. New York: Routledge. Campbell. A., McNamara, O., & Gilroy, P. (2004). Practitioner research and professional devetop-
ment in education. London: Paul Chapman. Campbell. D. T, & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research.
Chicago: Rand McNally. Can-, W., & Kemmis, S. (1986). Becoming critical: Education, knowledge, and action research.
London: Falmer. Cervero, R. M. (1988). Effective continuing education for the professions. San Francisco: Jossey-Bass. Cervero, R. M., & Wilson, A. L. (1994). Planning responsibly for adult education: A guide to nego-
tiating power and interests. San Francisco: Jossey-Bass. Cervero, R. M., & Wilson, A. L. (Eds.). (1996). What really matters in adult education program plan-
ning: Lessons in negotiating power and interests (New Directions for Adult and Continuing Education, No. 69). San Francisco: Jossey-Bass.
Cervero, R. M., Wilson, A. L., & Associates. (200Ij. Power in practice: Adult education and the struggle for knowledge and power in society. San Francisco: Jossey-Bass.
Cherryholmes, C. H. (1992). Notes on pragmatism and scientific realism. Educational Researcher, 21, 13-17.
Clandinin, J., & Connelly, M. (1995). Teachers' professional knowledge landscapes. New York: Teachers College Press.
Colin, S. A., Ill, & Guy, T. A. (1998). An Africentric interpretive model of curriculum orientations for course development in graduate programs in adult education. PAACE Journal of Lifelong Learning, 7, 43-55.
Comings, J. P., Beder, H., Bingman, B., Reder, S., & Smith, C. (2003). Establishing an evidence-based adult education system (Occasional paper 2003-09-00). Boston: National Center for the Study of Adult Learning and Literacy. (ERIC Documentation Reproduction Service No. ED 481 828)
Cunningham, P. M. (2000). A sociology of adult education. In A. Wilson & E. R. Hayes (Eds.), Handbook of adult and continuing education (new ed., pp. 573-591). San Francisco: Jossey-Bass.
Daley, B. (2001). Leaming and professional practice: A study of four professions. Adult Education Quarterly, 52, 39-54.
Denzin, N. K., & Lincoln, Y. S. (2003). Introduction: The discipline and practice of qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), The landscape of qualitative research: Theories and issues (2nd ed., pp. 1-35). Thousand Oaks, CA: Sage.
Dirkx, J. M. (1986). The tacit dimension of practical knowledge and the utilization of research in adult education. Proceedings of Midwest research-to-practice conference in adult, community and con- tinuing education. Muncie, IN: Ball State University.
Dirkx, J. M. (2005). "To develop a firm persuasion": Workplace learning and the problem of meaning. In C. Elliott & S. Turnbill (Eds.), Critical thinking in human resource development (pp. 155-174). London: Routledge.
Dirkx, J. M., Amey, M., & Haston, L. (1998). Boundaries and beliefs among teachers of adults: A case study of curricular transformation. In J. Fleming (Ed.), Proceedings ofthe 1998 Midwest research- to-practice conference in adult, continuing, and community education (pp. 69-74). Muncie, IN: Ball State University.
Dirkx, J. M., Fonfara, T, & Flaska, K. (1993). Self and not-self in the practitioner-learner relationship: The problem of boundary awareness in ABE practice. Adult Basic Education: An Interdisciplinary Journat for Adult Literacy Educators,J, 51-68.
Dirkx / STUDYING THE COMPLICATED MATTER OF WHAT WORKS 289
Dirkx, J. M., & Jha, L. R. (1994). Predicting rapid completion, persistence, and attrition in adult basic education. Adult Education Quarterly, 45, 269-285.
Dirkx, J. M., & Prenger, S. (1994). A living amoeba or a textbook topic? Conceptions of integrated, theme-based instruction in adult education. In L. Martin (Ed.), Proceedings of the thirteenth Midwest annual research-to-practice conference in adult, continuing, and community education (pp. 42-47). Milwaukee: University of Wisconsin-Milwaukee.
Dirkx, J. M., & Prenger, S. M. (1997| A guide to planning and implementing instruction for adults: A theme based approach. San Francisco: Jossey-Bass.
Dirkx, J. M., & Smith, R. O. (2003). Thinking out of a bowl of spaghetti: Learning to leam in online collaborative groups. In T S. Roberts (Ed.), Online collaborative learning: Theory and practice (pp. 132-159). Hershey, PA: Idea Group Publishing.
Dirkx, J. M., & Spurgin, M. (1992). Implicit theories of adult basic education teachers: How they think about their students. Adult Basic Education: An Interdisciplinary Journal for Adult Literacy Educators, 2, 20-4\.
English, L. M. (2006). A Foucauldian reading of feminist leaming in feminist, nonprofit organiza- tions. Adult Education Quarterly, 56, 85-101.
Fenwick, T. J. (2000). Putting meaning into workplace learning. In A. Wilson & E. R. Hayes (Eds.), Handbook of adult and continuing education (new ed., pp. 294-311). San Francisco: Jossey-Bass.
Feuer, M. J. (2006). Response to Bettie St.Pierre's "Scientifically based research in education: Epistemology and ethics." Adult Education Quarterly, 56(4), 267-272.
Flexner, A. (1910). Medical education in the United States and Canada. Carnegie Foundation for the Advancement of Teaching. Boston: Merrymount Press.
Forester, J. (1989). Planning in the face of power. Berkeley: University of California Press. Freire, P. (1990). Pedagogy ofthe oppressed. New York: Continuum. Freud, S. (1955). The interpretation of dreams. New York: Basic Books. Hayes, E., & Flannery, D. (with Brooke, A. K., Tisdell, E. J., & Hugo, J. M.). (2000). Women as learn-
ers: The significance of gender in adult learning. San Francisco: Jossey-Bass. Haymes, S. (1995). Race, culture and the city: A pedagogy for Black urban struggle. Albany: State
University of New York Press. hooks, b. (2000). Feminism is for everybody: Passionate politics. Cambridge, MA: South End Press. Jensen, N., & Dirkx, J. M. (1989). A curriculum for primary care internal medicine: The Wisconsin
experience. Philadelphia: American College of Physicians. Johnson-Bailey, J. (2006). African Americans in adult education: The Harlem Renaissance revisited.
Adult Education Quarterly, 56, 102-118. Jung, C. J. (1963). Memories, dreams, reflections. New York: Vintage. Kirsch, I. S., Jungeblut, A., Jenkins, L., & Kolstad, A. (1993). Adult literacy in America: A first look
at the results of the National Adult Literacy Survey. Washington, DC: U.S. Department of Education, National Center for Education Statistics.
Mager, R. F. (1975). Preparing instructional objectives. Belmont, CA: Fearon. Mizrahi, T. (1986). Getting rid of patients. New Brunswick, NJ: Rutgers University Press. National Assessment of Adult Literacy. (2003). A first look at the literacy of American adults in the 21st
century. Washington, DC: U.S. Department of Education, National Center for Education Statistics. Nowlen, P. M. (1988). A new approach to continuing education for business and the professions: A
performance model. New York: American Council on Education and Macmillan. Olson, D. L. (2003). Principles, impracticality, and passion. Phi Delta Kappan, 85, 307-309. Polanyi, M. (1998). Personal knowledge: Towards a post-critical philosophy. London: Routledge. Redfield, D. (2004). A closer look at scientifically based research: How to evaluate educational
research. T.H.E. Journal, 31, 24-25. Richardson. V. (1994). Conducting research on practice. Educational Researcher, 23, 5-10. Robinson, V. M., & Lai, M. K. (2006). Practitioner research for educators: A guide to improving
classrooms and schools. Thousand Oaks, CA: Sage.
290 ADULT EDUCATION QUARTERLY / August 2006
Ryan, A. B. (2001). Feminist ways of knowing: Towards theorizing the person for radical adult edu- cation. Leicester, UK: National Institute for Adult Education.
St.Pierre, E. A. (2006). Scientifically based research in education: Epistemology and ethics. Adult Education Quarterly, 56(4), 239-266.
Schoenfeld, A. H. (2006). What doesn't work: The challenge and the failure of the What Works Clearinghouse to conduct meaningful reviews of studies of mathematics curricula. Educational Researcher, 35, 13-21.
Schon, D. (1983). The reflective practitioner. New York: Basic Books. Schon, D. (1995). The new scholarship requires a new epistemology. Change: The Magazine of
Higher Leaming, 27(6), 27-34. Sheared, V. (1994). Giving voice: An inclusive model of instruction—A womanist perspective. In
E. Hayes & S. A. J. Colin, III (Eds.), Confronting racism and sexism in adult education (pp. 27-37). San Francisco: Jossey-Bass.
Tisdell, E. J. (1998). Poststructural feminist pedagogies: The possibilities and limitations of a femi- nist emancipatory adult learning theory and practice. Adult Education Quarterly, 48, 139-156.
Tyler, R. W. (1950). Basic principles of curriculum and instruction. Chicago: University of Chicago Press.
Usher, R., Bryant, 1, & Johnson, R. (1997). Adult education and the postmodern challenge: Learning beyond the limits. London: Routledge.
Vella, J. (2002). Learning to listen, learning to teach: The power of dialogue in educating adults. San Francisco: Jossey-Bass.
Watkins, K. E., & Marsick, V. J. (1993). Sculpting the learning organization: Lessons in the art and science of systemic change. San Francisco: Jossey-Bass.
Wenger, E. (1998). Communities of practice: Learning, meaning, and identity. Cambridge, UK: Cambridge University Press.
West, L. (2001). Doctors on the edge: General practitioners, health and learning in the inner city. London: Free Association Books.
Wilson, A. L., & Cervero, R. M. (1996). Learning from practice: Learning to see what matters in program planning. In R. M. Cervero & A. L. Wilson (Eds.), What really matters in adult educa- tion program planning: Lessons in negotiating power and interests (pp. 91-99, New Directions for Adult and Continuing Education, No. 69). San Francisco: Jossey-Bass.