DQ

R E S E A R C H A R T I C L E

Evaluating environmental conflict resolution: Practitioners, projects, and the movement

Micah Fisher1,2 | Tina Sablan2,3

1Geography Department, University of Hawai‘i at M�anoa, Honolulu, Hawaii 2Urban and Regional Planning, University of Hawai‘i at M�anoa, Honolulu, Hawaii 3Spark M. Matsunaga Institute for Peace and Conflict Resolution, University of Hawai‘i at M�anoa, Honolulu, Hawaii

Correspondence Micah R. Fisher, Geography Department, University of Hawai‘i at M�anoa, 2424 Maile Way, Saunders Hall 445, Honolulu, HI 96822. Email: micahrf@hawaii.edu

Environmental conflict is complex and variable, and over time, a concerted field has developed to study processes for collaboration and resolution. This article examines the evaluations of multistakeholder collaborative processes underpinning the field of Environmental Conflict Resolu- tion (ECR). Specifically, we analyze ECR evaluations from over four decades, across different approaches, geog- raphies, and scales. We also corroborate our findings through interviews and discussions with key scholars and practitioners in the field. We highlight the valuable empir- ical data from evaluations and point to a three-pronged approach for reinvigorating evaluations that support prac- titioners and projects and promote broader ideals of ECR collaboration.

1 | INTRODUCTION

Disputes over natural resources are often complicated, politically and emotionally charged, and seem- ingly intractable. They may cross territorial boundaries, involve multiple parties and scales of gover- nance, and raise difficult questions not only of science but also of social justice and equity. Global challenges like climate change are a case in point, compelling nuanced approaches and layered solu- tions at varying scales to address complex human–environment interactions. The dramatic environ- mental changes of our time demand innovations in both technology and governance. Much is discussed in the realm of technological fixes. For example, catalytic converters for car exhausts cleared smoggy cities, and affordable renewable energy promise pathways to reduce dependence on fossil fuels. Process innovations, on the other hand, are more difficult to identify and are less visible than technological ones.

By “process,” we mean how communities at different scales approach environmental change, make decisions about resource management, and resolve conflict, which can profoundly impact both environmental and social goals. A concerted field has emerged across multidisciplinary social sci- ences that provide innovations for addressing pressing environmental challenges that range from

Received: 27 September 2017 Revised: 10 April 2018 Accepted: 12 April 2018

DOI: 10.1002/crq.21222

© 2018 Association for Conflict Resolution and Wiley Periodicals, Inc.

Conflict Resolution Quarterly. 2018;36:7–19. wileyonlinelibrary.com/journal/crq 7

multistakeholder collaborative process, public policy dispute resolution, environmental collaboration and conflict resolution, and environmental justice (Dukes, 2004; Emerson, Nabatchi, & O'Leary, 2017; Stephens & Berner, 2011; Walker, 2010). Here, we examine these process innovations through evaluations under the broader umbrella of Environmental Conflict Resolution (ECR).

Like Orr, Emerson, and Keyes (2008) and Bingham (1986), we envision ECR as more than just an alternative to more conventional methods of addressing conflict, such as litigation and arbitration. The most viable option—or range of options—for conflict resolution depends on the circumstances and nature of the conflict and the context in which it occurs. An ECR approach may be more effec- tive than litigation in some cases but not others. When environmental issues become, or have the potential to become, rancorous, dividing people into camps that are sharply delineated along values and positions, ECR can offer new ways to engage, to restore trust and relationships, and identify common and higher ground upon which to build innovative, collaborative solutions and decision- making processes.

ECR as a practice arose out of the social and environmental movements of the 1960s and 1970s, during the early years of National Environmental Policy Act (NEPA) implementation and through global sustainability summits of the early 1990s (Emerson et al., 2017). ECR has become increas- ingly relevant in certain forums since then, also becoming a concerted programmatic focus at research institutions. As we discuss in greater detail in this paper, early studies on ECR focused on analyzing whether and how agreements were reached, while subsequent reviews assessed process costs, time- lines, and participant satisfaction. Today, major players at the federal level are instituting formalized evaluation systems that focus on mandates, projects, and programs. Meanwhile, the “theorists” have diverged from the “practitioners,” thus creating a void for broader ECR learning across the communi- ties of practice. Our findings show that a continued commitment to examining ECR has waned, thus reducing the capacity of process innovations to address pressing environmental issues.

Reinvigorating the field in particular ways and presenting findings through more deliberate evalu- ative processes can help to provide an important set of tools for intervening in, and even preventing, the difficult and often polarizing disputes that affect our communities and natural resources. This study, therefore, focuses on the very studies that have sought to examine process innovations under the broader field of ECR. Through our examination of ECR evaluations, we offer a three-pronged approach to revisit ECR evaluation—for ECR practitioners, projects, and the broader movement going forward.

2 | ECR AND EVALUATION

Evaluation is not easy, particularly in a field as diverse and complex as ECR. Evaluation, in and of itself, is a contested field of study. As an academic and professional field, evaluation evolved from the more rigid approaches that measure objectives with outcomes to more adaptive developmental approaches (Patton, 2010; Rauschmayer & Wittmer, 2006). For ECR, simply settling on agreed ter- minology presents an additional challenge. Dukes (2004) offers an in-depth discussion of different terminologies used (i.e., from “Environmental Dispute Resolution” to “Multistakeholder Collabora- tive Process”). Stephens and Berner (2011) also make a distinction between “Public Participation” and “Public Policy Dispute Resolution”—the former values participation and social justice ends, while the latter focuses on resolving particular disputes. Stephens and Berner further observe that the longer-term process in the former can help to enhance outcomes in the latter.

Because we are interested in engaging with a broad group of practitioners, government agencies, and scholars who see multistakeholder collaborative process as a way to address environmental

8 FISHER AND SABLAN

challenges with deliberative and inclusive approaches, we use ECR in this paper to refer to a range of consensus-building approaches in which parties voluntarily participate to resolve environmental issues that are or could potentially be in dispute (Dukes, 2004; Emerson et al., 2017; Stephens & Ber- ner, 2011; Susskind & Ozawa, 1983). ECR emerged as the most commonly used terminology in the literature, but the breadth of the term “Environmental Conflict Resolution” requires still further articulation.1

First, there are different ways to understand the term environment. The environment frequently comes into focus for communities when there has been a disruption in a resource, especially in terms of quality or access, or when there is a threat of interference (Robbins, 2012; Walker, 2010). We follow the tradition of ECR scholars and practitioners, which views ECR as a process of engaging stakeholders in resolving or preventing the disputes that arise out of such resource disruptions. As a process of engagement, ECR can be applied in a wide range of contexts (Dukes, 2004; O’Leary & Bingham, 2003)—for example, when a water conservation plan is being negotiated (d’Estree & Colby, 2004), when community groups rally to avoid development projects in their “backyards,” or when government agencies propose more stringent environmental regulations (O'Leary & Raines, 2001). Our analysis incorporates evaluation studies that encompass this diversity of ECR applications.

Second, there are many variations and notions of conflict (cf. Emerson, Nabatchi, O'Leary, & Ste- phens, 2003). Does conflict merely relate to the particular terms and circumstances of the immediate dispute, or does the term cover a much broader scope that includes long-standing historical and struc- tural socioeconomic problems? To provide clarity on this point, scholars and practitioners have also cre- ated the heuristic of up-, mid-, and downstream conflicts to articulate the more preventative versus responsive elements of conflict resolution (Bean, Fisher, & Eng, 2007). Furthermore, is ECR as a field bounded to conflicts, or does ECR also involve broader efforts at elevating and enhancing collaborative governance (Stephens & Berner, 2011)? Other research has also highlighted that conflict presents a pos- itive indicator of challenge to power yielding improved resource access for the marginalized (Yasmi, Guernier, & Colfer, 2009), while others focus on the improved postconflict relationships from process elements toward more collaborative decision making (Dukes, 2004). The breadth of interpretation around conflict, therefore, poses a challenge to evaluate how success is defined across a broad spectrum of ECR cases (Straube, 2013). Attention to the way evaluations are approached help to illuminate efforts by practitioners and scholars interested in fostering more deliberative and collaborative approaches to addressing environmental conflict, which situates the broader objectives of our research.

It is worth noting that scholarly works have criticized ECR evaluations and the often-positive claims made about outcomes. Early on, Sipe and Stiftel (1995; Sipe, 1998) provided an extensive review of these critiques. According to their critical perspective, ECR evaluations have frequently overstated success because the available information is overwhelmingly provided by the practitioners themselves and amounted to little more than navel gazing. Such criticisms raise questions about con- flicts of interest and gatekeeping in the field of practice. More recently, Emerson et al. (2017, pp. 276–277) categorized three broad areas of ECR critique (what they term ECCR, in which they broaden ECR to include the word “collaboration”). In their categorization of ECR criticism, the first involves ad-hoc replacement or undermining of existing formal processes. The second relates to the exclusion of some stakeholders and the privileging of others through the process. Third is the more cynical perception that ECR is explicitly used to achieve certain goals at the expense of others, abus- ing trust and disarming dissent through the use of mediators. A more coherent, explicitly independent approach to evaluation can address these legitimate concerns about integrity and accountability and bring attention to ECR processes that work and what place ECR can have in contentious public debates.

FISHER AND SABLAN 9

3 | METHODOLOGICAL APPROACH

For this research, we began with an interest in discovering the different types and approaches to ECR evaluations. Approached as a meta-analysis, we collected, compiled, and compared as many literature evaluations as we could find. We began by collecting available research in academic libraries, using qualitative grounded theory to review their approaches. We also examined available project evalua- tions (e.g., federal ECR evaluations) but relied more on the existing research that studied them (Emerson, Orr, Keyes, & McKnight, 2009; Nabatchi, 2007; O'Leary & Raines, 2001). We thereafter purposively selected from an array of ECR evaluations, categorizing and identifying trends and simi- larities and selecting from among the more diverse approaches.

Subsequently, we found it helpful to inventory the shifting priorities over time, which we describe as our empirical material in the section entitled a genealogy. We offer a narrative of how evaluations have evolved over the past four decades, highlighting how priorities changed and diverged over time. To complement listing temporal changes in ECR evaluations, we also separated them by type, scale, approach, and findings. Overall, our research is culled from 36 total studies on different approaches to ECR evaluation, which are listed in the reference section below. We conclude our genealogy with the observation of the increasing distance between “theorists” and “practitioners.”

We recognized that confining our approach to a desk-study analysis had limitations, especially as ECR is heavily shaped by practitioners with fewer incentives to publish in academic journals. To gather perspectives beyond those in the literature, we also made a deliberate effort to crosscheck our findings through interviews with key informants. We identified and interviewed influential actors (i.e., scholars, practitioners, and policymakers) in the field to examine how ECR, and different evalu- ation priorities, changed over time. Finally, we also had the opportunity to present our findings at the Association for Conflict Resolution's (ACR) 2014 annual meeting, during a session of the Environ- mental Public Policy (EPP) national conference. During this session, we were able to collect addi- tional feedback from participants, further refining our analysis from these discussions. As a result of this reflexive approach of examining studies and engaging with key actors, we present our study in the following ways.

Based on our analysis of ECR evaluations and discussions with practitioners and scholars, we stratified our findings into a three-pronged framework that reflects the different types of ECR evalua- tions and priorities among those that identify the central role of evaluation in environmental process innovations. The categories of practitioner, project, and movement suggest how we can learn from the broader body of knowledge to guide the future, explore opportunities, and strengthen the field of practice. We conclude with some thoughts on a path forward.

4 | RESEARCHING ECR EVALUATIONS: A GENEALOGY

In this section, we review the history, evolution, and range of ECR evaluation, beginning in the 1980s with the early studies that focused on descriptive cases, process outcomes, and participant sat- isfaction and moving into the 2000s when federal government–led, institutionalized systems of evalu- ation dominated the field.

4.1 | Early evaluations—The 1980s

In the United States, the ECR movement arose in the 1970s out of the challenges of NEPA implementa- tion and the rigidity of legal mechanisms that acted as barriers to consensus and mutually agreeable

10 FISHER AND SABLAN

resolutions. Creative solutions began to emerge based on the ability of mediators and the interests of dif- ferent stakeholders to arrive at a consensus based on common principles. Early evaluations focused on descriptive case studies, participant satisfaction, whether agreements were reached, and process outcomes such as improved relationships. Bingham (1986) undertook one of the first reviews of ECR processes, documenting and examining a diverse body of 161 cases involving disputes over different resources, site- specific projects, and policies across the United States, in an effort to evaluate the first 10 years of prac- tice. Her study identifies measures of success that include how often agreements were reached in these cases and how well those agreements addressed issues in dispute. Even in cases when agreements were not reached, Bingham (1986) also observed that success might be affirmed by participants in dispute reso- lution if the process improved aspects of communication.

4.2 | ECR evaluation in the 1990s

In the 1990s, ECR evaluation studies shifted focus to compare and contrast the empirical work and outcomes of ECR processes with non-ECR approaches (Sipe & Stiftel, 1995). Work in this area is still limited, however, due to the difficulty of comparing ECR cases (d'Estree and Colby, 2004; Dukes, 2004). Evaluations also began to consider ECR in terms of the types of conflicts involved— that is, “upstream” (related to public policymaking or planning), “midstream” (such as environmental permitting), and “downstream” disputes (on compliance and enforcement with respect to issues such as land use or pollution) (Bingham & Anderson, 1997; Emerson et al., 2003).

Professional associations like The Society of Professionals in Dispute Resolution (SPIDR) and Alternative Dispute Resolution and the Association for Conflict Resolution’s Environment and Public Policy Section committee sponsored a range of discussions to identify what constituted best practices in the field. d'Estree and Colby (2004) highlight efforts to bring groups together to address criteria for suc- cess and impact. For example, a roundtable coordinated to explore such criteria by RESOLVE and the National Institute for Dispute Resolution (NIDR) articulated these values for success:

including all relevant parties, promoting problem-solving, de-escalating the conflict, building empathy among participants, empowering and giving voice to less powerful interests, generating a decision (which may or may not constitute a consensus agree- ment) that is well informed, building institutions for future policy decisions, setting a good public policy precedent, contributing to a good track record for ADR, and posi- tively influencing future interactions. (d'Estree & Colby, 2004, p. 19)

Evaluation of ECR in the 1990s also drew attention to long-term social outcomes. For example, Pruitt, Peirce, McGillicuddy, Welton, and Castrianno (1993) suggested outcomes that Stephens and Berner (2011) would describe as falling into the “Public Participation” schools of thought, as opposed to “Public Policy Dispute Resolution.” According to Stephens and Berner (2011), each can learn from and enhance the other, as the former focuses on social justice outcomes, while the other looks at how to arrive at settlement and resolution. Innes and Booher (1999) suggest that the bench- mark for success should discard the rigid binary of “success versus failure.” Instead evaluations should opt for criteria that reflect whether ECR processes resulted in building trust. In the work at the Consensus Building Institute, Susskind (1999) also applies considerations of trust outcomes in the work on land use decisions and conflicts. D’Estree and Colby’s (2004) review of studies from the University of Virginia also identifies perspectives that go beyond mediated agreements, indeed exam- ining what new arrangements they create. Further, such broader contextualization of an agreement calls for incorporating a political economy approach to evaluating ECR.

FISHER AND SABLAN 11

4.3 | ECR evaluation in the 2000s

In 1999, the U.S. Institute of Environmental Conflict Resolution (USIECR) was created to institu- tionalize federal support for ECR. The USIECR developed a roster of professional mediators and played a convening role in connecting practitioners with federal agencies and programs. Dukes's (2004) comprehensive review provides helpful context and insight on the work occurring at that time. Who defines success is an important consideration, and according to Dukes (2004), one USIECR evaluation reported that 75% of respondents believed that ECR processes work. In the same evalua- tion, evidence of an increasing pool of skilled practitioners was reported, as well as improved rela- tionships among ECR participants, though empirical evidence for the latter was limited. In addition, the evaluation suggested that nonpractitioners, such as environmental lawyers, saw value in ECR pro- cesses as well (see also O’Leary & Husar, 2002).

As the USIECR began to act as a bridge for practitioners and federal agencies, these agencies also assumed additional leadership roles in mobilizing ECR processes to address their own disputes (O’Leary, Nabatchi, & Bingham, 2005). Numerous programs and partnerships developed among the US Department of the Interior,2 the USDA Forest Service, the US Environmental Protection Agency,3 and others that continue to support collaborative projects involving diverse stakeholders and complex environmental conflicts. As of 2018, an Interagency Alternative Dispute Resolution Working Group continues to convene broader federal ADR conversations, but newsletter communi- cations only occasionally raise ECR-specific topics. Increased utilization of ECR by large public agencies meant that evaluation was essential to justify the costs and effort afforded to these processes. A series of studies evaluating ECR processes at the USEPA initiatives found that there were enhanced relationship outcomes among parties (O'Leary & Raines, 2001). Meanwhile, Leach and Sabatier (2003) noted that facilitators were being asked to prove their marginal costs, suggesting a pushback on the allocation of funds for ECR.

Numerous sectoral studies also began to emerge. For example, Andrew (2001) examined ECR in waste management. d'Estree and Colby (2004), and more recent studies by Scott (2015) and Ulibarri (2017), addressed water conflicts in the western United States. They analyzed these diverse water conflict cases to theorize potential evaluation criteria that could be broadened to apply to other kinds of cases (see criteria in Table 1). They also expanded the scope of evaluation to consider different stages of analysis, variables for setting baselines, and what happens during and after the ECR pro- cess, as well as short-term and long-term outcomes. d'Estree and Colby (2004) acknowledge that, in some ways, evaluation in ECR amounts to comparing apples to oranges, particularly in reviewing cases across different fields. That said, evaluation provides important opportunities for reflection and conversations about how to improve ECR.

Since the mid-2000s, however, ECR evaluation studies seem to have waned for three reasons. First, the field has been dominated by privileging a specific type of program with limited entry (fed- eral programs). We arrive at this conclusion because the only available studies during this time were of highly formalized federal programs, such as the Systematic Evaluation of Environmental and Eco- nomic Results (SEEER) and the Measurement, Assessment, and Evaluation Section II (MAES II). These evaluations are specifically targeted at reviewing federally mandated ECR processes. We were unable to find similar studies evaluating and comparing ECR at state and local levels of government or by other institutions (O’Leary & Yandle, 2000).4 This does not suggest that such evaluations are not taking place but rather that there are declining opportunities to access and communicate such studies under the broad umbrella of ECR.

Second, what we believe is a key factor in the drop-off in the evaluation literature is the diver- gence between “theorists” and “practitioners,” which was already expressed by Birkhoff and Lowry

12 FISHER AND SABLAN

(2003). Theorists, such as O'Leary and Vij (2012), came to describe the field as a “low paradigm field”:

It is safe to say that the study and practice of collaborative public management is gener- ally fragmented with a low level of consensus. From a research perspective, it is a low- paradigm field. Some of the specific reasons behind this slow knowledge development include the lack of agreed-upon definitions of commonly used terms and variables, little consensus on significant research topics, a piecemeal approach to theory building, reli- ance on anecdotal description, differing and incomplete units of analysis, and a failure to keep up with global and IT developments. From a practice perspective, there is a poor connection between practice and theory, a lack of agreed-upon definitions of com- monly used terms, paradoxes and complexities in collaborative practices, challenges in balancing autonomy with interdependence, difficulties that arise when leading when one is not in charge, and limited transfer of knowledge and lessons learned from one manager to another. (O'Leary and Vij, 2012, p. 518)

Those described as ECR “theorists” were fragmented into their interests on more detailed aspects, some focused on specializing in public administration innovations and others more focused on dis- cursive or societal effects. Practitioners, on the other hand, were more interested on other specific aspects of reflection and inquiry. Mediators were especially interested in gauging individual capabili- ties and progress and focused their efforts on guild-building aspects of ECR to access skills training and, for some, to align with professional certification networks. Furthermore, practitioners were eager to understand the extent that projects and interventions had succeeded, how an issue was satisfacto- rily resolved, at what cost, and to what extent agreements were sustained. These project level

TABLE 1 Examples of evaluation criteria

Study (year) Criteria

Brunet (1987) Whether the process provided justice

Ury, Brett, and Goldberg (1988) Communication; satisfaction with the outcome; durability of solution; effect on relationships

Bush (1989) Six clusters: Individual satisfaction; individual autonomy; social control; social justice; social solidarity; personal transformation

Moore (1996) Political; interests; relationships; responsibility

Emerson et al. (2003) Scales and stages of evaluation: Upstream, midstream, and downstream approaches

Beierle and Cayford (2003) Incorporating public values; improving decision quality; resolving conflict; building institutional trust; educating the public

d'Estree and Colby (2004) a. Conceptual strategies: Outcome achieved; process quality; outcome quality; relationship of parties to outcomes; relationship quality among parties; social capital

b. Follows guidelines of accessibility, reliability, validity, timing for measurement as a success criteria

c. Stages of analysis: Baselines, during process, upon completion, implementation (short term/long term)

Orr et al. (2008); also compare with Todd (2001)

Measuring intangible outcomes (i.e., improved relationships); moving beyond “reaching agreement” to characterizing the attributes of agreements and identifying anticipated longer-term impacts; capturing participants' perspectives on the relative merits of ECR compared to other approaches of addressing environmental problems and conflicts; evaluating in a way that is consistent across cases, contexts, and organizations

Emerson et al. (2009) Structural components: Inputs; activities; outcomes; impacts

For ECR, these components are: Desired process conditions; expected process dynamics; outcomes; and impacts

FISHER AND SABLAN 13

evaluations, however, had become more rigid within the project mandate. Evaluations appeared more interested in responding to administrative requirements and less so on improving process or dissemi- nating learning outcomes. As a result of this divergence between theorists and practitioners, the field became more diffuse and specialized, losing its convening center around ECR, weakening its com- municative potential, creating barriers to entry, and overall becoming less visible as a field.

The third reason ECR evaluation studies have waned relates to the broader structural changes tak- ing place in careers related to environmental management. A recent Mongabay study published in the Guardian highlighted the irony between growing interests in environmental issues and the declining opportunity for conservation careers (Hance, 2017). In short, although project-level evaluations may be required institutionally, there are fewer avenues for engaging in project evaluations that take a deliberative and reflective approach.

5 | DISCUSSION: REENGAGING ECR EVALUATION IN A THREE- PRONGED APPROACH

The limited availability of ECR evaluation studies and the decline that began in the late 2000s does not necessarily mean evaluations are not continuing to take place. It does suggest, however, that these studies, where they do exist, are not easily accessible to researchers who are looking to ECR as a field that offers solutions to complex, contentious environmental conflicts. This observation calls attention to the need for a repository or clearinghouse of ECR studies that researchers and practitioners can access in order to learn from past initiatives and findings and advance the field of practice as a whole. One respondent suggested that university conflict resolution programs should develop a network for such a repository, sharing curriculum and syllabi, consolidating research, and actively responding to issues involving ECR. Although Emerson et al. (2017) suggest an increasing number of university programs on conflict and collaboration, the programs fall under a broad umbrella of fields ranging from labor relations to international peacemaking. Furthermore, respondents from university pro- grams also suggested declining budgets, enrollment, and purpose within the academic community.

In our interviews and engagement with scholars and practitioners, we centered our discussions around the evaluation imperative. Why should ECR evaluations take place? Or put another way, why are evaluations not more commonplace, more accessible? These questions yielded answers that eval- uation is too costly, too time-consuming, exposes sensitive and contentious information, not useful, or otherwise not a priority. To be sure, practitioners and administrators among the leadership of the ECR community noted that evaluations take time and cost money. But these respondents also noted that evaluations could potentially save time and money. Evaluations, as one respondent noted, forces those in the field to ask difficult, reflective, and critical questions and can also encourage innovation and expand the reach and impact of ECR in public deliberations and decision making. On a practical level, another respondent expressed that evaluation can help practitioners learn how to do their jobs better and help build the case for ECR (i.e., in terms of cost and time savings, or improvements to the environment and communities). Furthermore, evaluation can foster accountability for practitioners and the many stakeholders involved in ECR and facilitate the entry and learning of new practitioners into the field.

Our findings in examining a genealogy of ECR evaluations, combined with our engagement with scholars and practitioners, yielded a stratification of priorities as a guiding framework for revisiting and enhancing ECR evaluation in order to highlight process-level innovations in addressing environ- mental conflict. These three aspects and the kinds of evaluative questions that might be asked are cat- egorized as follows: (a) Practitioner, (b) Project, and (c) Movement.

14 FISHER AND SABLAN

Practitioner-level evaluations might pose questions such as “How am I helping [or hindering] in my role supporting parties to find pathways to dispute resolution? How do I know I'm being effec- tive? Do I have the skills and guiding ethical compass to do this well?” Project-level evaluations might look at questions like “Did this process save time or money? Did it improve environmental outcomes? Does it represent the will and interests of communities?” Examples of questions about the ECR movement might include higher-order questions such as “What is the visibility and impact of the field in public discourse? Are ECR principles and processes being accepted and incorporated into the policy lexicon?”

Further to these three levels of evaluation that we have identified from our study of evaluations, and corroborated through interviews and discussions with scholars and practitioners, another key point was raised about accountability. In each of these three levels, we also noted a significant ele- ment and interest on the follow-up “How do we know?” questions. This sheds light on the need to ensure that robust, meaningful measures of success are developed and applied in evaluation and that the evaluation tools themselves are also carefully considered. Empirical, descriptive, and comparative studies all have an important role in offering useful insight. In the subsections below, we expand on the three-pronged approach to developing a more robust, holistic, and iterative system of evaluation in ECR.

5.1 | The (new) reflective practitioner

This first prong involves evaluations around a rigorous skills and capabilities approach for the indi- vidual practitioner. We call this the reflective practitioner, taking the reference from Donald Schön (1983) to encourage evaluation processes that incorporate reflection (and learning) in action. Today, reflective practitioners continue to incorporate innovative technologies and practices that are designed to enhance group interactions, extend outreach opportunities, and engage in more collaborative processes.

Practitioners, however, expressed that they are often extremely busy and move from project to project, with limited time or ability to document, assess, synthesize, and compare what they are learn- ing. Indeed, it is not “new” for practitioners to incorporate reflection, learning, and innovation in their work. Recent research also highlights the emotional benefits of mediation work as an important dimension to reflect upon, and that insights into the practice can also extend to broader elements of overall community well-being (Malizia & Jameson, 2018; Raines, 2018). What we suggest based on this research, therefore, is a new reflective practitioner that relies on more networks to resources, with greater opportunities to adopt regularized and systematic approaches to reflection, learning, and inno- vation. Our discussions with practitioners highlighted the interest in more meaningfully reflecting on individual roles and impact. Practitioners also noted a perceived decline in the periodic opportunities to convene, noting the valuable learning potential from sharing their work with one another and hear- ing reflections on challenges and innovations. In addition, developing more reflective practitioners should include evaluating skills-building programs.

5.2 | The project

Here, we identify opportunities to revitalize and enhance project-level evaluations. Our review sug- gests that ECR is only systematically tracked at the federal level through MAES II, CADR, and SEEER, and we identified the opportunities to build upon that body of evaluative work to define and measure success in ECR at state and local levels, as well as among nongovernmental institutions and other entities. In addition, we see a need to better understand the impacts of ECR on communities

FISHER AND SABLAN 15

and the environment. For example, respondents noted the timeliness of evaluations that ask the fol- lowing: Does ECR actually improve relationships and facilitate healing in our communities? Does it build better places to live and improve capacity to address difficult environmental challenges? How do we know if we are achieving our goals? These are the types of questions that stakeholders, includ- ing ECR practitioners (e.g., mediators and facilitators), activists, and project administrators, are eager to learn of every ECR project.

5.3 | The movement

This third prong involves examining whether ECR is gaining traction in public awareness and deci- sion making and how the language of ECR is being accepted, applied, and trusted by the public. We think of a “movement” as a sustained, collective effort toward lasting change, descriptions that those involved with ECR often ascribe to it, namely, that multistakeholder collaborative processes present pathways for addressing complex environmental problems. Recent research also highlights the role that ECR practitioners play in encouraging broader community healing and reconciliation (Dukes, 2009; Raines, 2018). Here, we look to Michele Straube (2013), who developed a “report card” for ECR in Utah, to highlight broader impacts of the “movement.” The report card evaluates ECR in terms that include stakeholder awareness, understanding, and application of ECR in the public policy arena. Addressing such questions regularly and rigorously of ECR as a movement reifies the field and speaks to its future viability and visibility.

6 | CONCLUSION

Over the years, many breakthroughs have been made from evaluating ECR. The three-pronged approach provides a more systematic division of themes that highlight the core areas for continued collaboration that ask: “How are my skills as a practitioner and how am I influencing the process? Are the processes working? And finally, is the field gaining relevance and awareness in our commu- nities as one possible and potentially powerful set of tools to address complex public challenges?”

As a practical matter, given the finite project funds and limited resources, practitioners will priori- tize project preparation and implementation over evaluation by necessity. Second, we recognize that there continues to be a discussion about what ECR is and whether it is even a “field.” Adler (2009) has expressed frustration about guild-building through his predictions of the “end of mediation” and criticisms about the cooptation of practice by self-promoters and his insistence that mediation should not be considered a distinct field.

If, however, there is indeed an imperative to nurture more deliberative and collaborative processes in public debates, there must be an explicit effort to increase accountability and develop a shared ethic of practice in ECR. Our analysis does not advocate for certification programs—another area of contentious debate we observed among practitioners—but we are raising points that were raised throughout our numerous discussions with scholars and practitioners, namely, that the field has lost sight of itself, its goals fragmented into consultancies influenced by shifting institutional priorities. Perhaps these trends are part of larger influences in the political economy of contracting, trends that extend beyond the initial scope of this research. Emerson et al. (2017, p. 280) provide some initial insight on this matter, citing a 2014 study of the American Bar Association, which noted the declin- ing budgets and tracking systems on ADR and ECR. Their study, however, also suggested that ADR practices are being subsumed and incorporated within agencies and institutions, and there was evi- dence of greater “upstream collaboration.” Whichever the case, we observed collective interest in the

16 FISHER AND SABLAN

broader ECR community to sustain an explicitly ECR movement and galvanize a collective memory about what good practice, successful projects, and an expanded awareness and appreciation for col- laboration can mean. We see systematic, independent, and critical approaches to evaluation as being central to these goals.

The contexts, complexities, and urgencies in environmental conflicts are ever-shifting, and collab- orative processes offer a promising set of tools to clear pathways and forge solutions among multiple parties with diverse interests. Without a concerted effort to continue to build upon the lessons learned over the past four decades of ECR, and to transfer and translate that knowledge to upcoming genera- tions of practitioners, many vital elements of innovative ECR practice may fall by the wayside. As one practitioner noted: What we do not value, we lose. Moreover, another respondent noted: In fail- ing to evaluate or to make transparent and accessible the results of our work, we create barriers to entry and good practice.

The environmental challenges of our day require more than just technological innovations. Pro- cess innovations must also be a salient part of the conversation, and ECR provides a range of options premised on collaborative ideals that can help address tough, multifaceted public policy debates and the relationships underpinning human–environment interactions in new and exciting ways. The dearth of evaluative attention on ECR in recent years is a concern, and also an opportunity. Practi- tioners have to keep up with the changes and demands of the time, and the body of knowledge in the field has to continually evolve to stay relevant. Periodic, critically reflective evaluation helps those committed to ECR research and practice remember where we came from, be aware of what has been tried, know what has worked and what has failed and why, and how we can improve. Learning is an integral component of the collaborative processes that practitioners promote among stakeholders and must be an integral component of their own practice as well. It is our hope that this study helps bring the evaluation imperative to the fore—to reinvigorate the habits of the reflective practitioner, to care- fully scrutinize projects and elevate the lessons they offer, and—in learning from the past—to thoughtfully consider the directions that ECR as a movement might head into the future.

ACKNOWLEDGMENTS

This paper is a result of a partnership with the New Directions Committee (NDC) of the Environmental Public Policy (EPP) Section of the Association for Conflict Resolution (ACR). We worked with the NDC and its cochairs Marina Piscolish and Caitlin Ellsworth over the course of about a year. An early version of the content in this paper was presented at the 14th Annual ACR Conference in Cincinnati, Ohio, in October 2014 as part of a panel on evaluation in EPP that was moderated by Piscolish.

The conversations at the conference led us to agree to further develop our research and collaborate on a larger project that would integrate our examination of ECR evaluation with the data and reflections of practitioners drawn from a pilot study conducted by the NDC in the summer of 2014. The broad goal was to assess the state of the field of ECR and identify opportunities for future growth and development, including new partnerships, increased visibility and accountability, and expanded impacts.

We thank the following individuals for their invaluable advice and guidance throughout the course of this study: Dolores Foley of the University of Hawai‘i's Department of Urban and Regional Planning and the Matsunaga Institute who served as our capstone advisor; the members of the NDC, especially cochair Marina Piscolish; Anne Smoke of the Matsunaga Institute; Kem Lowry, Professor Emeritus of the University of Hawai‘i's Department of Urban and Regional Planning; and Michele Straube of the University of Utah's College of Law, Environmental Dispute Resolution Program. Any errors in this paper are ours alone.

FISHER AND SABLAN 17

NOTES

1We also recognize that ECR has slight variations on terminology, and in our review, we have kept a broad definition to ensure that we did not overlook any evaluation studies. 2The Office of Collaborative Action and Dispute Resolution. 3The Conflict Prevention and Resolution Center. 4We did, however, review state websites on alternative dispute resolution, such as the Maryland Mediation and Conflict Reso- lution Office, the ADR center in Hawaii, and others. Very rarely do these programs highlight environmental issues and multi- stakeholder processes.

ORCID

Micah Fisher http://orcid.org/0000-0002-8246-2318

REFERENCES

Adler, P. (2009). The end of mediation: An unhurried ramble on why the field will fail and mediators will thrive over the next two decades. Mediate.com. Retrieved from http://www.mediate.com/articles/adlerTheEnd.cfm

Andrew, J. S. (2001). Examining the claims of environmental ADR evidence from waste management conflicts in Ontario and Massa- chusetts. Journal of Planning Education and Research, 21(2), 166–183.

Bean, M., Fisher, L., & Eng, M. (2007). Assessment in environmental and public policy conflict resolution: Emerging theory, patterns of practice, and a conceptual framework. Conflict Resolution Quarterly, 24(4), 447–468.

Beierle, T. C., & Cayford, J. (2003). Dispute resolution as a method of public participation. In R. O'Leary & L. B. Bingham (Eds.), The promise and performance of environmental conflict resolution (pp. 53–68). Washington, DC: Resources for the Future Press.

Bingham, G. (1986). Resolving environmental disputes: A decade of experience. Washington, DC: Conservation Foundation. Bingham, L., & Anderson, J. (1997). Upstream effects from mediation of workplace disputes: Some preliminary evidence from the

USPS. Labor Law Journal, 48, 601–615. Birkhoff, J. E., & Lowry, K. (2003). Whose reality counts. In R. O'Leary & L. B. Bingham (Eds.), The promise and performance of

environmental conflict resolution (pp. 27–52). Washington, DC: Resources for the Future Press. Brunet, E. (1987). Questioning the quality of alternate dispute resolution. Tulane Law Review, 62, 1. Bush, R. A. B. (1989). Efficiency and protection, or empowerment and recognition: The mediator's role and ethical standards in media-

tion. Florida Law Review, 41, 253. d’Estree, T. P., & Colby, B. G. (2004). Braving the currents: Evaluating environmental conflict resolution in the river basins of the

American West (Vol. 26). Boston, MA: Springer Science & Business Media. Dukes, E. F. (2004). What we know about environmental conflict resolution: An analysis based on research. Conflict Resolution Quar-

terly, 22(1–2), 191–220. Dukes, E. F. (2009). Truth, understanding, and repair. Law and Contemporary Problems, 72, 57. Emerson, K., Nabatchi, T., & O'Leary, R. (2017). Environmental collaboration and conflict resolution. In R. F. Durant & D. J. Fiorino

(Eds.), Environmental governance reconsidered: Challenges, choices, and opportunities (pp. 263–296). Cambridge, MA: MIT Press.

Emerson, K., Nabatchi, T., O’Leary, R., & Stephens, J. (2003). The challenges of environmental conflict resolution. In R. O’Leary & L. Bingham (Eds.), The promise and performance of environmental conflict resolution (pp. 3–26). Washington, DC: Resources for the Future.

Emerson, K., Orr, P. J., Keyes, D. L., & McKnight, K. M. (2009). Environmental conflict resolution: Evaluating performance outcomes and contributing factors. Conflict Resolution Quarterly, 27(1), 27–64.

Hance, J. (2017, August 17). All work, no pay: The plight of young conservationists. The Guardian, section on Environment. Retrieved from http://www.theguardian.com/environment/2017/aug/17/all-work-no-pay-the-plight-of-young-conservationists

Innes, J. E., & Booher, D. E. (1999). Consensus building and complex adaptive systems: A framework for evaluating collaborative planning. Journal of the American Planning Association, 65(4), 412–423.

Leach, W., & Sabatier, P. (2003). Facilitators, coordinators, and outcomes. In R. O'Leary & L. B. Bingham (Eds.), The promise and performance of environmental conflict resolution (pp. 148–174). Washington, DC: Resources for the Future Press.

Malizia, D. A., & Jameson, J. K. (2018). Hidden in plain view: The impact of mediation on the mediator and implications for conflict resolution education. Conflict Resolution Quarterly, 35(3), 301–318.

Moore, S. A. (1996). Defining ‘successful’ environmental dispute resolution: Case studies from public land planning in the United States and Australia. Environmental Impact Assessment Review, 16(3), 151–169.

Nabatchi, T. (2007). The institutionalization of alternative dispute resolution in the federal government. Public Administration Review, 67(4), 646–661.

18 FISHER AND SABLAN

O'Leary, R., & Bingham, L. B. (2003). The promise and performance of environmental conflict resolution. Washington, DC: Resources for the Future Press.

O'Leary, R., & Husar, M. (2002). What environmental and natural resource attorneys really think about ADR: A national survey. Natu- ral Resources & Environment, 16(4), 262–264.

O'Leary, R., Nabatchi, T., & Bingham, L. (2005). Assessing and improving conflict resolution in multiparty environmental negotia- tions. International Journal of Organization Theory and Behavior, 8(2), 181–209.

O'Leary, R., & Raines, S. S. (2001). Lessons learned from two decades of alternative dispute resolution programs and processes at the US Environmental Protection Agency. Public Administration Review, 61(6), 682–692.

O’Leary, R., & Vij, N. (2012). Collaborative public management: Where have we been and where are we going? The American Review of Public Administration, 42(5), 507–522.

O'Leary, R., & Yandle, T. (2000). Environmental management at the millennium: The use of environmental dispute resolution by state governments. Journal of Public Administration Research and Theory, 10(1), 137–155.

Orr, P. J., Emerson, K., & Keyes, D. L. (2008). Environmental conflict resolution practice and performance: An evaluation framework. Conflict Resolution Quarterly, 25(3), 283–301.

Patton, M. Q. (2010). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York, NY: Guil- ford Press.

Pruitt, D. G., Peirce, R. S., McGillicuddy, N. B., Welton, G. L., & Castrianno, L. M. (1993). Long-term success in mediation. Law and Human Behavior, 17(3), 313–330.

Raines, S. S. (2018). Becoming the change we wish to see: The unexpected benefits of conflict resolution work. Conflict Resolution Quarterly, 35, 319–327.

Rauschmayer, F., & Wittmer, H. (2006). Evaluating deliberative and analytical methods for the resolution of environmental conflicts. Land Use Policy, 23(1), 108–122.

Robbins, P. (2012). Political ecology: A critical introduction (Vol. 2). Malden, MA: John Wiley & Sons. Schön, D. A. (1983). The reflective practitioner: How professionals think in action (Vol. 5126). New York, NY: Basic Books. Scott, T. (2015). Does collaboration make any difference? Linking collaborative governance to environmental outcomes. Journal of

Policy Analysis and Management, 34(3), 537–566. Sipe, N. G. (1998). An empirical analysis of environmental mediation. Journal of the American Planning Association, 64(3), 275–285. Sipe, N. G., & Stiftel, B. (1995). Mediating environmental enforcement disputes: How well does it work? Environmental Impact

Assessment Review, 15(2), 139–156. Stephens, J. B., & Berner, M. (2011). Learning from your neighbor: The value of public participation evaluation for public policy dis-

pute resolution. Journal of Public Deliberation, 7(1), 10. Straube, M. (2013). Report card on environmental dispute resolution in Utah – grade: Incomplete but showing progress. Journal of

Environmental Law and Litigation, 28(2), 227–261. Susskind, L. (1999). Using assisted negotiation to settle land use disputes. Cambridge, MA: Lincoln Institute for Land Policy. Susskind, L., & Ozawa, C. (1983). Mediated negotiation in the public sector mediator accountability and the public interest problem.

American Behavioral Scientist, 27(2), 255–279. Todd, S. (2001). Measuring the effectiveness of environmental dispute settlement efforts. Environmental Impact Assessment Review,

21(1), 97–110. Ulibarri, N. (2017). Does collaboration affect the duration of environmental permitting processes? Journal of Environmental Planning

and Management, 61(4), 617–634. Ury, W. L., Brett, J. M., & Goldberg, S. B. (1988). Getting disputes resolved: Designing systems to cut the costs of conflict. San Fran-

cisco, CA: Jossey-Bass. Walker, G. (2010). Environmental justice: Concepts, evidence and politics. London, England: Routledge. Yasmi, Y., Guernier, J., & Colfer, C. J. P. (2009). Positive and negative aspects of forestry conflict: Lessons from a decentralized forest

management in Indonesia. International Forestry Review, 11(1), 98–110.

How to cite this article: Fisher M, Sablan T. Evaluating environmental conflict resolution: Practitioners, projects, and the movement. Conflict Resolution Quarterly. 2018;36:7–19. https://doi.org/10.1002/crq.21222

FISHER AND SABLAN 19

Copyright of Conflict Resolution Quarterly is the property of John Wiley & Sons, Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.