PADM530Wk1
Rawono1
siddiki-goel-2015-assessing-collaborative-policymaking-outcomes-an-analysis-of-u-s-marine-aquaculture-partnerships.pdf
American Review of Public Administration 2017, Vol. 47(2) 253 –271
© The Author(s) 2015 Reprints and permissions:
sagepub.com/journalsPermissions.nav DOI: 10.1177/0275074015599603
journals.sagepub.com/home/arp
Article
Assessing Collaborative Policymaking Outcomes: An Analysis of U.S. Marine Aquaculture Partnerships
Saba Siddiki1 and Shilpi Goel1
Abstract Administrators and policymakers increasingly rely on collaborative policymaking groups to inform policy development. While this trend is observed in a wide array of policy domains, it is particularly common in the regulation of natural resource-based industries which requires the simultaneous consideration of an interrelated set of economic, technical, and social factors. In this article, we examine outcomes associated with collaborative policymaking groups involved in informing state aquaculture policy, referred to herein as aquaculture partnerships. We define outcomes here as consequences on relevant contextual conditions (social, political, and environmental) that follow from the work or design of collaborative processes. Using data collected through an online survey of partnership participants (n = 123), we examine individual and procedural factors that significantly associate with partnerships’ positive or negative influence on a set of policy and social outcomes, as perceived by their participants. Overall, we find that participants’ ability to mobilize scientific and technical resources to achieve group objectives, perceptions of procedural fairness, and individual-level learning are all positively associated with partnership influence on policy and/or social outcomes. We conclude our article by highlighting the value of this research for both scholars and practitioners interested in better understanding collaborative group dynamics and outcomes relating thereto.
Keywords outcomes, technical expertise, process, learning
Introduction
Part of the complexity of governance stems from the need to devise context-specific solutions that reflect the unique set of social, political, and environmental considerations tied to individual policy domains. Within each policy context, administrators and policymakers must carefully consider the relevant positions and resources of vested policy stakeholders, public sensitivities, institutional landscapes, and administrative capacities and motivations to further their long-term
1Indiana University–Purdue University Indianapolis, IN, USA
Corresponding Author: Saba Siddiki, School of Public and Environmental Affairs, Indiana University–Purdue University Indianapolis, 801 W. Michigan Street, BS 4072, Indianapolis, IN 46202, USA. Email: [email protected]
599603 ARPXXX10.1177/0275074015599603American Review of Public AdministrationSiddiki and Goel research-article2015
254 American Review of Public Administration 47(2)
policy objectives. To help them navigate complex policy contexts, policymakers and administra- tors increasingly rely on collaborative stakeholder groups comprised of various parties who have a vested interest in a given policy domain (Heikkila & Gerlak, 2014). Engaging diverse stake- holders in collaborative settings is expected to help decision makers craft contextually appropri- ate policy solutions, harness expert knowledge, reduce the potential for policy-related conflict, increase policy receptivity, and facilitate shared understandings of policy problems and solutions (Beierle & Cayford, 2002; Maggioni, Nelson, & Mazmanian, 2012; Ostrom, 2005; Weible & Sabatier, 2009). A question that naturally emerges in collaborative contexts is how diverse stake- holders with presumably divergent interests, capacity, norms, and policy views can overcome their individual-level differences to achieve mutually desired outcomes (Heikkila & Gerlak, 2014)? And, more importantly, what factors facilitate the attainment of such outcomes given stakeholder differences?
Relative to other dimensions of collaborative arrangements, such as antecedents or pro- cesses, assessments of their products have been limited. In part, this represents definitional challenges, whereby products can be variably operationalized in terms of outputs, outcomes, and impacts. Definitions of these three concepts are inconsistent in the literature. Consistent with Koontz and Thomas (2006), we define outputs in the context of collaborative governance as “tangible items generated through collaborative processes” (p. 113). Examples of outputs in the context of this research include specific policy recommendations relating to aquaculture licensing. Our definitions of outcomes and impacts in the collaborative governance context are analogous to Heinrich’s (2002) definitions of these in the context of federal job training pro- grams. We define outcomes as consequences on relevant contextual conditions (social, political, and environmental) that follow from the work or design of collaborative processes and impacts as the consequences on relevant conditions (i.e., outcomes) directly resulting from specific out- puts produced through a collaborative process or specific attributes of a collaborative process. An outcome building from the output example above would be increases in aquaculture industry entrants. The corresponding impact would be an increase in the number of industry entrants based on changes in the application process promulgated in the collaborative group’s policy recommendations (i.e., outputs).
Inconsistent definitions of outputs, outcomes, and impacts limit the ability to summarize find- ings across studies on the products of collaborative governance, but measurement challenges associated with accurately capturing products have also undoubtedly contributed to a relative dearth on literature relating to such. A prominent measurement challenge is the long time scale often required to observe either tangible (e.g., policy recommendations) or nontangible (e.g., stakeholder relations) impacts (Koontz & Thomas, 2006; Leach, Pelkey, & Sabatier, 2002). A second difficulty arises from the inability to identify objective measures of certain types of out- comes and impacts. Scholars can rely on subjective measures in such cases (see Leach et al., 2002), though this approach has notable bias limitations. A third difficulty arises from the preva- lence of confounding variables that challenge the linking of certain types of outcomes directly to collaborative processes (Biddle & Koontz, 2014).
The measurement challenges listed above naturally lead scholars to focus on tangible or demonstrable outputs, which is both logical and warranted. This article, however, seeks to build off the work of scholars who simultaneously study social and political outcomes of collabora- tive arrangements (Leach et al., 2002) and factors relating to such in an increasingly salient policy context. A key motivation guiding this work is the desire to evaluate the outcomes of collaborative groups in such a way that connects to the very logic undergirding the use of this approach more broadly. For example, if collaborative policymaking is valued for its ability to harness relevant expertise, foster social capital among diverse stakeholders, and provide contex- tually appropriate policy solutions, among other benefits, the commensurate outcome analysis should focus on the extent to which collaborative governance arrangements improve the science
Siddiki and Goel 255
or information used in policy decision making, feelings of trust among policy stakeholders engaged in the policy process, and appropriateness of policy solutions. With this in mind, in this article, we contribute to the literature by responding to the following two research questions:
Research Question 1: To what extent do participants in collaborative policymaking groups perceive them as positively or negatively contributing to policy, technical, and social outcomes? Research Question 2: What procedural and individual-level factors facilitate positive group influence on policy, technical, and social outcomes?
We explore these research questions in the context of 10 collaborative policymaking groups involved in the development of aquaculture policy, referred to herein as aquaculture partnerships. Aquaculture partnerships include some mix of government and nongovernment stakeholders1 that are convened organically or through an institutional mandate (e.g., legislation) to participate in the aquaculture policymaking process. Similar to other policy settings in which the use of col- laborative policymaking techniques are increasingly observed, the aquaculture policy context is one in which policymakers are forced to grapple with an interrelated set of economic, technical, and social factors (Firestone, Kempton, Krueger, & Loper, 2004). Aquaculture partnerships con- vene diverse sets of stakeholders that variably represent issues relating to each of these factors in an effort to capture their diverse expertise and inform the many facets of aquaculture policy. Data for this research were collected through an online survey (n = 123) of participants in the 10 aqua- culture partnerships.
This article proceeds as follows. First, we provide an introduction to the literature on collab- orative policymaking as relevant to our research questions. We then provide a more detailed background about aquaculture partnerships and the research methods used therein. We then pres- ent findings from our analyses of survey data which highlight the salience of technical expertise, procedural fairness, and individual-level learning as predictors of partnership influence on aqua- culture issues. We conclude our article by highlighting the value of this research for both scholars and practitioners interested in better understanding collaborative group design and outcomes.
Conceptual Background
Reflecting its increasing prevalence in the administrative and policymaking process, a rich litera- ture on collaborative governance and its various dimensions (i.e., antecedents, process, and prod- ucts) has emerged in recent decades. Scholars of public administration and related fields have provided rich insights into factors that drive the creation of collaborative arrangements (Bryson, Crosby, & Stone, 2006) as well as their structural, functional, and procedural characteristics (Heikkila & Gerlak, 2014). A portion of the literature focuses specifically on the outputs, out- comes, or impacts of collaborative processes and factors linking to each (Innes & Booher, 1999; Lubell, 2003), a further subset of which focuses on these concepts specifically in the context of collaborative policymaking.
Key Conceptual Definitions
Collaborative policymaking is a generic term that refers to the engagement of nongovernmental actors in decision making relating to some part of the policy process and is inclusive of different forms of participatory policymaking including stakeholder partnerships, advisory committees, public hearings, and negotiated rulemaking (see Leach et al., 2002, for defining characteristics of each). Outputs are generally commonly defined in recent scholarship as products or services resulting from a particular process (Thomas, 2008); examples provided by Koontz and Thomas
256 American Review of Public Administration 47(2)
(2006) include “plans, projects, and other tangible items” (p. 113). Definitions, and related opera- tionalizations, of outcomes and impacts, however, are more inconsistent in the literature. Provided here are some of the various definitions that are offered (though definitions of outputs, outcomes, and impacts used in this article are based on Heinrich, 2002, as explained in more detail below). Koontz and Thomas (2006) define outcomes as “effects of outputs on environmental or social conditions” (p. 113). Rogers and Weber (2010) define outcomes more broadly and decoupled from outputs. Their conceptualization of outcomes encompasses general improvements in public agency and/or collaborative processes and capacity. Emerson (2009) elevates outcomes to impacts where the former “contribute(s) to more effective problem solving, conflict manage- ment, and governance” (p. 222). In a more recent publication, Emerson, Nabatchi, and Balogh (2012) define impacts as “intentional (and unintentional) changes of state within the system context . . . alterations in a preexisting or projected condition that has been deemed undesirable or in need of change” that are “spurred by collaborative dynamics” (p. 18).
Looking beyond studies of collaborative governance to studies of performance measurement more broadly can help offer definitional clarity on the distinctions between outputs, outcomes, and impacts. Heinrich (2002), in particular, offers and advances through an empirical application of performance measurement in the context of federal job training programs, a precise distinction between outputs, outcomes, and impacts. She characterizes outputs as the tangible activities, products, services that commonly are the bases of government appraisal of programmatic perfor- mance. These are often assessed in relation to programmatic inputs or costs. She characterizes outcomes in terms of the programmatic products that correspond to the realization of program- matic goal attainment. Finally, Heinrich links outputs and outcomes in her definition of impacts, by characterizing impacts as outcomes resulting from outputs. Because Heinrich’s conceptual- izations are clear and empirically robust, we define outputs, outcomes, and impacts in the context of collaborative policymaking in this article as the following:
1. Outputs are tangible items generated through policy or programmatic efforts. 2. Outcomes are consequences on relevant contextual conditions (social, political, and envi-
ronmental) that follow from the work or design of collaborative processes. 3. Impacts are consequences on relevant conditions (i.e., outcomes) directly resulting from
specific outputs produced through a collaborative process or specific attributes of a col- laborative process.
Consistent with Heinrich, in our definition, the attribution of outcomes to specific outputs is a critical conceptual dimension of impacts. We also allow in our definition of impacts the attribu- tion of outcomes to specific characteristics of a collaborative process. For example, increased social capital among policy stakeholders in a particular subsystem as a result of a collaborative process would be characterized as an outcome. Increased social capital among policy stakehold- ers in a particular system linked to products of a collaborative process (e.g., joint research pro- posals) or deliberative decision-making processes used within a collaborative process would both be examples of impact. In our study, as we explain further in the “Method” section, we do not attribute consequences of interest to particular characteristics of collaborative policymaking groups in U.S. aquaculture and thus typify our study as one of collaborative policymaking out- comes, not impacts.
Linking Collaborative Products to Process and Participants
While public administration and policy scholars vary in their analytical interests and conceptual- izations of collaborative group outputs, outcomes, and/or impacts, a common thread in this schol- arship is identifying the factors that contribute to such. Factors in two categories—procedural and
Siddiki and Goel 257
individual—emerge as being particularly salient in facilitating collective outcomes. Example fac- tors in the procedural category explored in this article include procedural fairness and the presence of a respected mediator and all critical stakeholders in the collaborative process. At the individual level, we consider the following factors based on existing scholarship: individual resources, per- ception of individuals that the venue in which they are a participant is the only one in which they can advance their goals, duration of participation, and participant learning.
Across a wide array of collective action settings, perceptions that the collective deliberation process treats parties fairly and consistently have been found to be associated with positive out- comes (Herian, Hamm, Tomkins, & Zillig, 2012; Leach & Sabatier, 2005; Lubell, 2003; Tyler, Degoey, & Smith, 1996). Similarly, borrowing from the alternative dispute resolution literature, scholars have shown that perceptions of the existence of a respected mediator within a group will facilitate positive outcomes (Sabatier & Weible, 2007). In addition to being held in high esteem by fellow group members, a respected mediator is someone that actively engages in conflict miti- gation. The role of such an actor is expected to be particularly crucial in settings that engage stakeholders with divergent political and policy views. Also relating to stakeholder characteris- tics and consistent with the foundational assumptions of collaborative governance, scholars have confirmed that the inclusion of all vested parties relating to the work of the collaborative process to be a critical factor influencing its performance (Weber, 1998).
A slightly different facet of the stakeholder environment that can have notable bearing on collaborative group outcomes is stakeholders’ ability to harness financial and scientific/tech- nical resources to further the work of a collaborative (Bidwell & Ryan, 2006). Financial resources are necessary for completing projects, tasks, and other operational functions of the collaborative process. Scientific and technical resources, however, may directly influence the decision-making process by altering the availability of certain types of information (Weible, 2008). Furthermore, insofar as collaborative policymaking groups are considered more struc- tured variants of issue networks (Heclo, 1978), technical expertise is a critical asset, the exchange of which can potentially alter stakeholder dynamics. Finally, Leach and Sabatier (2005) demonstrate the influence of a mutual stalemate in affecting outcomes. A mutual stale- mate is the perception by stakeholders participating in a collaborative process that no alterna- tive venue exists in which they can pursue their policy goals. Presumably, stakeholders will be more engaged in a process if it is viewed as being the only venue in which they can address their concerns.
At the individual level, this research also focuses on assessing the salience of time and learn- ing among collaborative group participants as determinants of outcome perceptions. Time, con- ceptualized here in terms of continuity in participation, is required for maintaining sustained negotiation with other stakeholders toward the attainment of policy objectives as well as observ- ing the impacts of collaborative processes (Sabatier & Weible, 2007). Because incrementalism is favored in the policy process (Baumgartner & Jones, 2010), noticeable policy change is unlikely to be perceived in short time horizons. Similarly, changes in social norms are unlikely observable in short time scales (Ostrom, 1990).
Numerous scholars have examined learning in the context of collaborative policymaking (Leach, Weible, Vince, Siddiki, & Calanni, 2013; Resh, Siddiki, & McConnell, 2014). In their study of learning, Leach et al. (2013) differentiate between simple learning (i.e., knowledge acquisition) and deep learning (i.e., changes in one’s opinions on science or policy issues) and identify factors that contribute to each. Resh et al. (2014) examine the role of government actors in facilitating learning in collaborative policymaking venues. In this article, we are interested in an alternative assessment, wherein learning, operationalized as knowledge acquisition, is treated as a predictor of outcome perceptions. Presumably, individuals participating in collaborative groups who are learning more will perceive greater influence on outcomes thereof because part- nerships that are successful at the individual level are also likely to be successful at the collective
258 American Review of Public Administration 47(2)
level or because individual-level learning correlates with more positive perceptions of the group process overall.
Based on the existing scholarship discussed here, it is expected that participants in collabora- tive policymaking groups will perceive their respective partnerships to have greater positive influence on policy, technical (e.g., science or technology), and social outcomes, when:
Process-related propositions
1. the group process is perceived as being fair and consistent for all involved parties; 2. all critical stakeholders are engaged in the process; and 3. the group has at least one participant who mediates conflict and is held in high esteem;
Individual-related propositions
4. they perceive that the collaborative group is the only venue in which they can pursue their policy objectives;
5. they are able to mobilize financial resources to achieve the group’s objectives; 6. they are able to mobilize scientific and technical resources to achieve the group’s
objectives; 7. they personally experience learning on various dimensions of the issues that are addressed
by their group; and 8. they have been participating in the collaborative process for an extended period of time.
In the following sections, more detail is provided about aquaculture partnerships and the methods used to assess the above propositions therein.
Collaborative Policymaking in U.S. Marine Aquaculture
Aquaculture—also known as fish and shellfish farming—is formally defined by the National Oceanic and Atmospheric Administration (NOAA) as “the breeding, rearing, and harvesting of plants and animals in all types of water environments including ponds, rivers, lakes, and the ocean” (NOAA, 2014, para. 1). Aquaculture development in the United States faces a number of barriers, including an uncertain regulatory landscape (Firestone et al., 2004; Wirth and Luzar, 1999); complex interdependencies among ecological, economic, technical, and social factors (Firestone et al., 2004); resistance from the general public regarding farmed seafood (Amberg & Hall, 2010; Mazur & Curtis, 2006); conflict about aquaculture development (Kaiser & Stead, 2002); and numerous concerns about the industry from disease control to degradation of marine ecosystems (Mazur & Curtis, 2006; Naylor et al., 2000; Treece, 2002). Over the past several years, numerous partnerships have materialized across the United States with the intent of over- coming these barriers by bringing together various, and often diverse, interests associated with the development of the aquaculture industry. Because aquaculture has largely been devolved to the state level, due in part to the geographic and species sensitive nature of aquaculture produc- tion, most of the partnerships are housed at the state level.
In most cases, aquaculture partnerships are formally convened, either through a legislative or regulatory mandate. In other cases, partnerships are formed based on the impetus of diverse aquaculture stakeholders who desire a shared venue in which to deliberate and develop recom- mendations relating to aquaculture industry. Whether formally or organically developed, aqua- culture partnerships typically function to advise on industry policies, develop and/or implement research priorities as it relates to aquaculture policy, implement programs, advise on adminis- trative budgetary decision relating to the industry, address the industry’s marketing issues, and/ or evaluate the effectiveness of aquaculture policies. Given the salience of the responsibilities
Siddiki and Goel 259
assigned to aquaculture partnerships, they provide an appropriate context within which to assess their influence on policy, social, and technical issues. More broadly, they are character- istically similar to diversely composed advisory committees and citizen participation efforts increasingly observed within and outside of natural resource industry–based policy contexts, lending opportunities to generalize theoretically and practically oriented research findings across policy fields.
Method
Aquaculture Partnerships
For research purposes, aquaculture partnerships were defined as local, state, or regional organiza- tions, comprising some mix of aquaculture industry, government agency, and nongovernment organization representatives that collaborate on policy or research, or both, relating to marine aquaculture in the United States. At the time the research was conducted, a total of 10 partner- ships were identified nationwide that met our definition and all were included in this study.2 Partnerships were identified based on extensive web research, consultation with a project advi- sory committee3 comprised of expert stakeholders representing different dimensions of aquacul- ture production, and other aquaculture experts. Coordinators of the 10 identified partnerships were contacted, and each group agreed to participate in this study.
Together, the 10 partnerships included in this research represent each major coastal region of the United States, with 3 located along the Pacific Coast, 5 located along the north-eastern Atlantic Coast, and 2 located on the south-eastern Atlantic Coast.4 The 10 partnerships are char- acteristically similar in terms of structure and function. Reflecting the geographically sensitive nature of aquaculture production, most of the selected partnerships focus on state-level issues, though 1 was created to address aquaculture issues pertaining to an entire region. All of the groups advise on policy issues: 7 as formal advisory committees established through legislative mandates and 3 as collaborative groups initiated or sponsored by government. The partnerships variably focus on variety of aquaculture issues, such as, fish health, public safety, and/or other issues relating to the expansion of the aquaculture industry. All of the partnerships are comprised of vested stakeholders representing diverse aspects of the aquaculture industry. The groups differ in terms of the number of recommendations or policy-related research projects they generate. All of the groups were created within the 25 years, with the oldest being created in the late 1980s and the newest being created in 2007.
Data Collection
Data were collected using an online survey of participants in the sample of 10 aquaculture part- nerships. Participants were defined as anyone currently participating in one of the partnerships as well as anyone who closely followed the work of the partnership. Lists of active partnership members at the time of this research and their contact information were obtained from online membership lists or directories or were provided by partnership coordinators. Information regarding participants who were not officially members but closely followed the work of the partnership, and their contact information, was obtained from interviews (described below) and/ or partnership coordinators. Across the 10 partnerships, the number of participants ranged from 7 to 43. The construction of the online survey was informed by data obtained through interviews with 43 participants across the 10 partnerships, 32 in person and 11 by telephone.5 Interview responses, for example, were used to help researchers identify key dimensions of concepts explored through survey questions from theoretical (as consistent with existing scholarship) and practical perspectives. The interviews and survey explored similar questions, and thus
260 American Review of Public Administration 47(2)
interviews yielded useful information given their intended purpose (i.e., to inform survey mea- sure construction).
The online survey was sent to 198 participants identified across the 10 partnerships, of which 123 or 62% responded.6 The online survey contained a variety of questions relating to aquacul- ture policy generally, the functioning of partnerships, and attitudes and experiences of the partici- pants therein. Furthermore, relevant to the analyses in this article, respondents were provided a list of problems typically associated with marine aquaculture and asked to indicate their percep- tions of severity of each problem. They were then provided this same list and asked to indicate whether their respective partnership made the problem better or worse (i.e., partnership influence on issues represented in problem statements). The list of seven aquaculture problems analyzed is provided in Table 1. Also provided in Table 1 are the survey prompts used to elicit responses on problem severity and partnership influence7 as related to each problem listed, response scales, and variable characterizations for subsequently performed analyses.
Dependent Variable
Perceptions of partnership influence on the following issues (i.e., partnership outcomes), again issues as represented in problem statements, were the main dependent variables in our analyses: “appropriateness” and “sufficiency” of aquaculture regulations, public perceptions of aquaculture, availability of “good” science for aquaculture decision making, level of trust among aquaculture policy stakeholders, socioeconomic health of fishers and tribes, and dissemination of misinforma- tion. In other words, we were interested in determining whether partnerships contributed to more or less appropriate or sufficient aquaculture regulations, improved or worse public perceptions of aquaculture, more or less science deemed “good” for policy decision making, increased or decreased levels of trust among aquaculture policy stakeholders, improved or worsened socioeco- nomic health of fishers and tribes, and more or less misinformation dissemination.
In the case of the last issue here, it is not partnership participants who are disseminating mis- information. Rather, they are involved in addressing the spread of misinformation relating to the industry by various parties either through efforts of the collaborative process or through the convening of diverse stakeholders with different perspectives, and possibly information, in the same venue. This is not characterized as an impact because influence on this issue, as well as all others in the above list, is not directly associated with specific partnership outputs or process attributes in the data collection methods used in this research. But while we do not link partner- ship influence on the problems to specific activities, reports, and so on (and thus we characterize our dependent variable in terms of outcomes), it is useful to identify the general types of activities that partnerships engage in that may support their influence on aquaculture-related problems: Inappropriate or insufficient aquaculture regulations and socioeconomic health of fishers are tribes (e.g., development of policy recommendations), negative public perceptions of aquacul- ture and dissemination of misinformation (e.g., education/outreach), lack of good science in aquaculture decision making (e.g., development of technical reports or research on aquaculture issues), and distrust among stakeholders (e.g., collaborative meetings involving diverse aquacul- ture stakeholders).
Using perceptions of partnerships’ positive or negative influence on a predefined set of prob- lems as a proxy for actual influence is appropriate given the nature of the issues upon which we seek to determine partnerships’ positive or negative influence. With the exception of the socio- economic health of fishers and tribes and misinformation dissemination, no clear objective mea- sures exist for analytical purposes and thus subjective measures of such are considered appropriate. Even still, it is important to recognize the limitations of using subjective measures of outcomes and impacts. Common source bias and the halo effect are related phenomena that are relevant to consider in the context of this research (Meier & O’Toole, 2013; Nisbett & Wilson, 1977). Both
261
T ab
le 1
. D
at a
D es
cr ip
tio n.
V ar
ia bl
e na
m e
Su rv
ey it
em R
es po
ns e
sc al
e
D ep
en de
nt v
ar ia
bl es
: Pr
ob le
m S
ev er
ity : “
In y
ou r
op in
io n,
pl
ea se
in di
ca te
t he
s er
io us
ne ss
o f
th e
fo llo
w in
g pe
rc ei
ve d
pr ob
le m
s as
so ci
at ed
w ith
t he
e xp
an si
on o
f t he
m
ar in
e aq
ua cu
ltu re
in du
st ry
in y
ou r
re gi
on .”
In flu
en ce
o n
Pr ob
le m
: “ Fo
r ea
ch o
f t he
fo
llo w
in g
is su
es , p
le as
e in
di ca
te w
he th
er t
he
[p ar
tn er
sh ip
n am
e] h
as h
ad a
ny a
ct ua
l i m
pa ct
” so
fa r
us in
g a
sc al
e ra
ng in
g fr
om “
m ad
e th
e is
su e
m uc
h w
or se
” to
“ m
ad e
th e
is su
e m
uc h
be tt
er .”
Pr ob
le m
S ev
er ity
: N ot
a p
ro bl
em a
t a ll
(1 )
to v
er y
se rio
us p
ro bl
em (
5)
Pe rc
ei ve
d Pr
ob le
m S
ev er
ity In
flu en
ce o
n Pr
ob le
m : M
ad e
th e
iss ue
m
uc h
w or
se (
− 2)
t o
m ad
e th
e iss
ue
m uc
h be
tte r
(2 )
Pa
rt ne
rs hi
p In
flu en
ce o
n Pr
ob le
m s
In ap
pr op
ri at
e re
gu la
tio ns
g ov
er ni
ng
aq ua
cu ltu
re In
ap pr
op ri
at e
re gu
la tio
ns g
ov er
ni ng
U .S
. m ar
in e
aq ua
cu ltu
re
In
su ffi
ci en
t po
lic ie
s fo
r fa
ci lit
y pe
rm its
a nd
li ce
ns es
In su
ffi ci
en t
po lic
ie s
fo r
gr an
tin g
lo ng
-t er
m le
as es
, l ic
en se
s, a
nd /o
r pe
rm its
fo r
aq ua
cu ltu
re
fa ci
lit ie
s
N
eg at
iv e
pu bl
ic p
er ce
pt io
ns N
eg at
iv e
pu bl
ic p
er ce
pt io
ns a
bo ut
m ar
in e
aq ua
cu ltu
re
La
ck o
f g oo
d sc
ie nc
e fo
r de
ci si
on
m ak
in g
La ck
o f g
oo d
sc ie
nc e
to m
ak e
so un
d de
ci si
on s
re ga
rd in
g th
e si
tin g
an d
op er
at io
n of
aq
ua cu
ltu re
fa ci
lit ie
s
D
is tr
us t
am on
g st
ak eh
ol de
rs D
is tr
us t
am on
g st
ak eh
ol de
rs in
vo lv
ed in
m ar
in e
aq ua
cu ltu
re
So
ci oe
co no
m ic
h ea
lth o
f f is
he rs
a nd
tr
ib es
N eg
at iv
e im
pa ct
s fr
om m
ar in
e aq
ua cu
ltu re
o n
th e
so ci
oe co
no m
ic h
ea lth
o f l
oc al
fi sh
in g-
de pe
nd en
t co
m m
un iti
es o
r tr
ib es
D
is se
m in
at io
n of
m is
in fo
rm at
io n
D is
se m
in at
io n
of m
is in
fo rm
at io
n ab
ou t
m ar
in e
aq ua
cu ltu
re
In de
pe nd
en t
va ri
ab le
s: F
ac to
rs A
ffe ct
in g
Pa rt
ne rs
hi p
In flu
en ce
Fa
ir p
ro ce
ss T
he [
pa rt
ne rs
hi p]
p ro
ce ss
t re
at s
al l p
ar tie
s fa
ir ly
a nd
c on
si st
en tly
St ro
ng ly
di sa
gr ee
( −
2) t
o st
ro ng
ly
ag re
e (2
)
C ri
tic al
p ar
tie s
ab se
nt So
m e
cr iti
ca l i
nt er
es ts
a re
n ot
e ffe
ct iv
el y
re pr
es en
te d
in t
he [
pa rt
ne rs
hi p
na m
e] St
ro ng
ly di
sa gr
ee (
− 2)
t o
st ro
ng ly
ag
re e
(2 )
A
lte rn
at iv
e ve
nu e
op tio
n If
th e
[p ar
tn er
sh ip
] fa
ils t
o ad
op t
w or
ka bl
e so
lu tio
ns , m
y co
nc er
ns c
ou ld
p ro
ba bl
y be
s at
is fie
d by
a pp
ea lin
g di
re ct
ly t
o th
e le
gi sl
at ur
e, c
ou rt
s, o
r in
di vi
du al
a ge
nc ie
s St
ro ng
ly di
sa gr
ee (
− 2)
t o
st ro
ng ly
ag
re e
(2 )
R
es pe
ct ed
m ed
ia to
r T
he re
is a
t le
as t
on e
pa rt
ic ip
an t
w ho
m ed
ia te
s co
nf lic
t an
d is
r es
pe ct
ed b
y th
e ot
he r
[p ar
tn er
sh ip
] m
em be
rs St
ro ng
ly di
sa gr
ee (
− 2)
t o
st ro
ng ly
ag
re e
(2 )
Fi
na nc
ia l c
ap ac
ity C
ap ac
ity y
ou r
or ga
ni za
tio n
ha s
to m
ob ili
ze fi
na nc
ia l r
es ou
rc es
t o
he lp
a ch
ie ve
[ pa
rt ne
rs hi
p]
ob je
ct iv
es N
o ca
pa cit
y (1
) to
c om
pl et
e ca
pa cit
y (5
)
Sc
ie nt
ifi c
an d
te ch
ni ca
l c ap
ac ity
C ap
ac ity
y ou
r or
ga ni
za tio
n ha
s to
m ob
ili ze
s ci
en tif
ic a
nd t
ec hn
ic al
r es
ou rc
es t
o ac
hi ev
e [p
ar tn
er sh
ip ]
ob je
ct iv
es N
o ca
pa cit
y (1
) to
c om
pl et
e ca
pa cit
y (5
)
Pa
rt ic
ip an
t le
ar ni
ng M
ul ti-
ite m
m ea
su re
: T he
[p ar
tn er
sh ip
] h as
g iv
en m
e a
be tt
er u
nd er
st an
di ng
o f (
a) o
th er
st
ak eh
ol de
rs ’ p
er sp
ec tiv
es ; (
b) a
qu ac
ul tu
re s
ci en
ce ; (
c) a
qu ac
ul tu
re p
ol ic
y, la
w , o
r re
gu la
tio ns
; a nd
(d
) a qu
ac ul
tu re
e co
no m
ic s
or b
us in
es s.
C ro
nb ac
h’ s α
= .8
7
In di
vi du
al it
em s:
S tr
on gl
y di
sa gr
ee (
− 2)
to
s tr
on gl
y ag
re e
(2 ).
M ul
ti- ite
m
m ea
su re
r an
ge (
− 8)
t o
(+ 8)
Ex
te nd
ed t
im e
in p
ar tn
er sh
ip In
w ha
t ye
ar a
nd m
on th
d id
y ou
fi rs
t st
ar t
pa rt
ic ip
at in
g in
t he
[ pa
rt ne
rs hi
p] 1
= 1
0 ye
ar s
or m
or e;
0 =
le ss
t ha
n 10
y ea
rs
262 American Review of Public Administration 47(2)
of these phenomenon point to tendencies of individuals to conflate perceptions of different vari- ables that relate to a shared experience, process, and so on. In the case of this research, the most likely manifestation of either phenomenon is that partnership participants would evaluate group’s influence on the above issues based on their perceptions of other partnership attributes—for example, process or fellow participants.
Independent Variables
In the survey, respondents were also asked specific questions operationalizing variables relating to each of the propositions offered in this article. These variables include the following:
Process variables
1. fair process 2. critical parties absent 3. respected mediator
Individual variables
4. alternative venue option 5. financial capacity 6. scientific and technical capacity 7. participant learning 8. time in partnership
These questions and their associated variable characterization and measurement scales are also provided in Table 1. Each of these was treated as an independent variable predicting impact perceptions in our analyses.
Data Analysis
Analyses of survey data were performed in two steps. In the first step, partnership participants’ problem perceptions and perceptions of partnerships’ influence on policy, technical, and social issues were analyzed to (a) descriptively assess the perceived severity of aquaculture problems by participants in the different partnerships and their partnerships’ influence on such (i.e., calcu- lation of group means relating to each problem examined) and (b) assess variance across the partnerships in problem and associated influence perceptions. Significant differences between group means on individual problems were assessed using an independent sample, Kruskal–Wallis test. The Kruskal–Wallis test was determined to capture potentially relevant differences among groups. Aquaculture production and associated policy, technical, and social issues have the potential to differ substantially by state and region, and thus the researchers were interested in ascertaining intergroup heterogeneity.
In the second step of the data analysis, seven multiple regressions were conducted to deter- mine whether individual participants’ perceptions of a fair partnership process, absence of critical parties in the collaborative process, presence of a respected mediator, alternative venue option, financial capacity among partnership members, scientific and technical capacity among partner- ship members, participant leaning, and extended time in the partnership significantly associate with perceptions of positive or negative partnership influence on the seven problem issues listed in Table 1. This regression method is appropriate given the analytical objective of identifying significant relationships between a set of independent variables and dependent variable of inter- est. In the regression models, standard errors were clustered on a variable identifying each
Siddiki and Goel 263
respondent’s partnership to account for the possibility that residuals are correlated for respon- dents within a partnership.8 One regression model was conducted for each issue; the set of predic- tor variables (above) were common to all of the models.
Results
Results from the survey data analyses are presented in this section as they relate to each of our posited research questions.
Research Question 1: To what extent do participants in collaborative policymaking groups perceive them as positively or negatively contributing to policy, technical, and social outcomes?
Table 2 reports the partnership means for perceived problems and partnership influence on such. Across all the partnerships, the problems perceived as being most severe were dissemina- tion of misinformation (summed mean9: 39.78), negative public perceptions (summed mean: 35.01), inappropriate regulations governing aquaculture (summed mean: 33.4), and insufficient policies for facility permits and licenses (summed mean: 32.4). The problems perceived as being least severe were socioeconomic health of fishers and tribes (summed mean: 19.97) and lack of good science for decision making (summed mean: 24.93).
Across all partnerships, participants perceived the greatest influence of their respective groups on the availability of good science for decision making (summed mean: 7.58), dissemination of misinformation (summed mean: 7.56), permitting and licensing policy sufficiency (summed mean: 7.52), and level of trust among aquaculture policy stakeholders (summed mean: 7.42). Less influence was perceived on socioeconomic health of fishers and tribes (summed mean: 3.13). Results from this analysis therefore suggest that partnerships are indeed having notable influence on at least some of the problems regarded as the most severe, as perceived by important policy stakeholders in the aquaculture subsystem. They are also having an influence on issues that are not necessarily perceived as being problems for the industry; for example, availability of good science in decision making. This is also important to know as it can signal group capacity, priorities, and/or other important group dynamics.
In terms of problem perceptions, the Kruskal–Wallis test of variance revealed significant differences among partnerships on the following, where p < .01: insufficient policies for facility permits and licenses and negative public perceptions. In terms of influence on prob- lems, significant differences were observed among partnerships on the following: appropri- ateness of regulations for governing aquaculture, licensing and permitting policy sufficiency, negative public perceptions, and level of trust among stakeholders, where p < .01, and avail- ability of good science for decision making and socioeconomic health of fishers and tribes, where p < .10. For both problem and related influence perceptions, no structural or geo- graphical characteristics were common to partnerships exhibiting similarly high, moderate, or low means.
Research Question 2: What procedural and individual-level factors facilitate positive group influence on policy, technical, and social outcomes?
Descriptive statistics for variables included in the regression models are provided in Table 3. Results from the regressions are provided in Table 4. All models are highly significant and explain a respectable amount of variance (R2 = .20-.40) given that the data come from an online survey that often have measurement error (Lubell & Fulton, 2008). Results relating to each of the specific variables explored in the models are provided below.
264
T ab
le 2
. M
ea n
Pe rc
ep tio
ns o
f P ar
tn er
sh ip
In flu
en ce
o n
Pr ob
le m
s (In
f.) a
nd P
ro bl
em S
ev er
ity (
Se v.
).
Is su
e Pa
rt ne
rs hi
p 1
Pa rt
ne rs
hi p
2 Pa
rt ne
rs hi
p 3
Pa rt
ne rs
hi p
4 Pa
rt ne
rs hi
p 5
Pa rt
ne rs
hi p
6 Pa
rt ne
rs hi
p 7
Pa rt
ne rs
hi p
8 Pa
rt ne
rs hi
p 9
Pa rt
ne rs
hi p
10
In
f. Se
v. In
f. Se
v. In
f. Se
v. In
f. Se
v. In
f. Se
v. In
f. Se
v. In
f. Se
v. In
f. Se
v. In
f. Se
v. In
f. Se
v.
In ap
pr op
ri at
e re
gu la
tio ns
go
ve rn
in g
aq ua
cu ltu
re 0.
17 2.
81 0.
90 3.
30 0.
10 3.
62 0.
89 3.
43 1.
17 3.
14 1.
00 3.
33 0.
83 2.
92 0.
76 3.
52 0.
50 3.
53 1.
00 3.
80
In su
ffi ci
en t
po lic
ie s
fo r
fa ci
lit y
pe rm
its a
nd li
ce ns
es 0
3. 45
0. 70
3. 10
0. 50
4. 25
1. 55
3. 14
0. 67
2. 14
1. 00
2. 50
0. 83
3. 08
0. 52
4. 00
0. 55
3. 94
1. 20
2. 80
N eg
at iv
e pu
bl ic
p er
ce pt
io ns
0. 33
3. 45
0. 40
3. 10
0. 50
3. 00
0. 89
3. 43
0. 50
3. 86
0 3.
83 0.
58 2.
77 0.
90 4.
21 0.
61 3.
76 1.
40 3.
60 La
ck o
f g oo
d sc
ie nc
e fo
r de
ci si
on m
ak in
g 0.
33 2.
73 0.
30 2.
30 0.
50 2.
38 0.
89 3.
29 1.
33 2.
71 0.
80 1.
83 0.
83 1.
92 0.
85 2.
43 0.
55 2.
94 1.
20 2.
40
D is
tr us
t am
on g
st ak
eh ol
de rs
− 0.
5 3.
55 0.
70 2.
70 0.
80 3.
00 1.
33 2.
71 1
3. 29
0. 60
3. 00
1. 00
3. 00
0. 71
3. 30
0. 78
3. 06
1. 00
2. 60
So ci
oe co
no m
ic h
ea lth
o f
fis he
rs a
nd t
ri be
s −
0. 17
2. 09
0. 20
2. 50
0. 30
1. 38
0. 56
2. 57
0. 17
2. 29
0 1.
17 0.
42 1.
92 0.
52 2.
13 0.
33 2.
12 0.
80 1.
80
D is
se m
in at
io n
of
m is
in fo
rm at
io n
0. 08
3. 73
0. 90
3. 30
0. 70
3. 75
1. 22
4. 14
1. 17
4. 14
0. 60
4. 50
0. 33
3. 92
0. 95
4. 48
0. 61
3. 82
1. 00
4. 00
N ot
e. V
al ue
s pe
r pa
rt ne
rs hi
p re
pr es
en t
th e
m ea
n re
sp on
se s
of p
ar tn
er sh
ip p
ar tic
ip an
ts t
o qu
es tio
ns a
bo ut
t he
in flu
en ce
o f r
es po
nd en
ts ’ p
ar tn
er sh
ip o
n th
e pr
ob le
m s
lis te
d as
w el
l a s
th e
pe rc
ei ve
d se
ve ri
ty o
f th
es e
pr ob
le m
s. T
he s
ur ve
y pr
om pt
fo r
th e
pr ob
le m
s ev
er ity
q ue
st io
n w
as a
s fo
llo w
s: “
In y
ou r
op in
io n,
p le
as e
in di
ca te
t he
s er
io us
ne ss
o f t
he fo
llo w
in g
pe rc
ei ve
d pr
ob le
m s
as so
ci at
ed w
ith t
he e
xp an
si on
o f t
he
m ar
in e
aq ua
cu ltu
re in
du st
ry in
y ou
r re
gi on
.” T
he c
or re
sp on
di ng
r es
po ns
e sc
al e
w as
1 =
n ot
a p
ro bl
em a
t a ll
to 5
= v
er y
se rio
us p
ro bl
em . T
he s
ur ve
y pr
om pt
fo r
th e
pa rt
ne rs
hi p
in flu
en ce
v ar
ia bl
e w
as a
s fo
llo w
s:
“F or
e ac
h of
t he
fo llo
w in
g is
su es
, p le
as e
in di
ca te
w he
th er
t he
[ pa
rt ne
rs hi
p na
m e]
h as
h ad
a ny
a ct
ua l i
m pa
ct s
o fa
r us
in g
a sc
al e
ra ng
in g
fr om
‘m ad
e th
e is
su e
m uc
h w
or se
’ ( −
2) t
o ‘m
ad e
th e
is su
e m
uc h
be tt
er ’
(+ 2)
.”
Siddiki and Goel 265
Table 3. Descriptive Statistics for Regression Model.
Variable name Number of
observationsa M SD Minimum Maximum
Fair process 115 0.96 0.85 −2 2 Critical parties absent 113 0.01 0.98 −2 2 Respected mediator 112 0.72 0.73 −1 2 Alternative venue option 113 0 0.88 −2 2 Financial capacity 103 2.14 0.92 1 5 Scientific and technical capacity 110 3.2 1.15 1 5 Participant learning 114 3.38 3.08 −5 8 Extended time in partnership 123 0.25 0.46 0 1
aUnits of analysis are partnership participants.
Procedural fairness was significant in five of the seven models (p < .10 to p < .01). As pre- dicted, perceptions that the partnership process treats all parties fairly and consistently are associ- ated with greater perceived positive influence on policy and social issues. The only two problems in relation to which procedural fairness was not associated with positive group influence upon were the availability of good science for policy decision making and socioeconomic health of fishers and tribes. In terms of input variables, while financial capacity of partnership participants was not found significant in any of the models, scientific and technical capacity was significant and positive across all but two (p < .05 to p < .01)—level of trust among stakeholders and socio- economic health of fishers and tribes. Similarly, participant learning was found to be signifi- cantly associated (p < .01) with positive group influence on all but two problems—negative public perceptions and dissemination of misinformation. Extended time in a partnership was significant in the models for appropriateness of regulations governing aquaculture (p < .10), negative public perceptions (p < .10), level of trust among policy stakeholders (p < .05), and socioeconomic health of fishers and tribes (p < .05).
The absence of critical parties in the partnership process was found to be significantly associ- ated with group influence on only one issue—level of trust among stakeholders—whereby an increase in agreement that critical parties were absent is associated with less positive influence on this issue (p < .05). Having an alternative venue (e.g., legislature, courts, individual agencies) to which participants can appeal should their partnership fail to adopt workable solutions was not found to be significant in any of the models. The presence of a person in the collaborative process who mediates conflict and is respected by the partnership members was found to be positively associated with group influence on the appropriateness of regulations governing aquaculture.
Discussion
Assessing the effectiveness of collaborative governance arrangements is both logical and neces- sary. Public administration and policy scholars have variably sought to do this by studying out- comes, outputs, and/or impacts that result from collaborative processes. There has been particular emphasis on studying tangible outputs in light of measurement challenges relating to longtime horizons required for observing noticeable changes in either policy or social norms and difficulty isolating effects of collaborative process amid a vast array of potentially confounding factors (Biddle & Koontz, 2014). Still, given the importance of evaluating less tangible policy, technical, and social outcomes, this article adds to recent scholarship oriented at better understanding what factors contribute to positive collaborative group influence relating to such (Leach & Sabatier, 2005; Lubell, 2003; Rogers & Weber, 2010). We see distinct benefit in analyzing outcomes that
266
T ab
le 4
. R
eg re
ss io
n M
od el
R es
ul ts
. R eg
ul at
or y
ap pr
op ri
at en
es s
Pe rm
itt in
g an
d lic
en si
ng p
ol ic
y su
ffi ci
en cy
N eg
at iv
e pu
bl ic
pe
rc ep
tio ns
A va
ila bi
lit y
of g
oo d
sc ie
nc e
fo r
po lic
y de
ci si
on m
ak in
g
Le ve
l o f t
ru st
be
tw ee
n po
lic y
st ak
eh ol
de rs
So ci
oe co
no m
ic
he al
th o
f f is
he rs
an
d tr
ib es
D is
se m
in at
io n
of
m is
in fo
rm at
io n
Fa ir
p ro
ce ss
.2 7*
** .2
4* *
.3 4*
** .0
4 .2
8* **
.1 6
.1 4*
C ri
tic al
p ar
tie s
ab se
nt −
.0 7
− .0
9 .1
4 −
.0 4
− .2
2* *
.0 4
− .0
9 R
es pe
ct ed
m ed
ia to
r .1
5* .1
1 .0
0 .0
3 .2
2 .0
4 .1
6 A
lte rn
at iv
e ve
nu e
op tio
n −
.1 0
− .0
9 .0
1 −
.0 3
.0 6
− .0
2 .0
8 Fi
na nc
ia l c
ap ac
ity −
.0 5
− .0
4 −
.1 0
.0 2
− .1
1 −
.0 8
− .1
2 Sc
ie nt
ifi c
an d
te ch
ni ca
l c ap
ac ity
.1 3*
* .1
5* *
.2 0*
** .1
7* **
.1 1
.1 1
.1 8*
** Pa
rt ic
ip an
t le
ar ni
ng .0
6* **
.0 5*
** .0
4 .1
2* **
.0 7*
** .0
4* **
.0 5
Ex te
nd ed
t im
e in
p ar
tn er
sh ip
.2 7*
.2 5
.2 8*
.1 4
.4 2*
* .2
8* *
.2 8
N um
be r
of o
bs er
va tio
ns 96
97 97
97 97
97 97
R2 .3
0 .2
2 .2
6 .3
7 .4
0 .2
4 .2
0 M
od el
s ig
ni fic
an ce
.0 0
.0 0
.0 0
.0 0
.0 0
.0 0
.0 0
*p <
.1 0.
* *p
< .0
5. *
** p
< .0
1.
Siddiki and Goel 267
relate to the very rationales of collaborative governance. As such, we examine which procedural and individual-level variables are positively associated with perceived partnership influence on the following issues: (a) the appropriateness of aquaculture regulations, (b) sufficiency of permit- ting and licensing policies, (c) negative public perceptions regarding aquaculture, (d) availability of good science for policy decision making, (e) level of trust among stakeholders in the aquacul- ture policy arena, (f) socioeconomic health of fishers and tribes, and (g) dissemination of misin- formation about aquaculture.
Overall, we find that participants’ ability to mobilize scientific and technical resources to achieve group objectives, perceptions of procedural fairness, and individual-level learning are all positively associated with group influence on the above issues. The positive association between scientific and technical resource capacity of participants and perceptions of group influence is evident in relation to policy-related problems, negative public perceptions, lack of good science for decision making, and dissemination of misinformation. In contrast, capacity to mobilize financial resources was not found to be a significant predictor of group influence in relation to any of the problems. These findings confirm expectations that financial and scientific and techni- cal resources, which are sometimes conceptualized analogously under resources generally, are in fact conceptually distinct and have differing potential in affecting collaborative group dynamics. Whereas financial resources are important for conducting operational tasks of groups and sup- porting the development and implementation of tangible policy outputs, scientific and technical resources can alter the decision context of the group by bringing new or otherwise critical infor- mation to deliberations. Furthermore, when viewed as legitimate and unbiased, scientific and technical resources can introduce neutrality to the decision process that oversteps existing rifts between opposing stakeholders and possibly restructure relations between them. Indeed, as Weible (2008) argues, uncertainty, presumably linked to the absence of scientific and technical information, creates opportunities for stakeholders to rely on their beliefs for decision making and can thus contribute to divisive relations among deliberating parties.
The salience of scientific and technical resources may also be punctuated in the context of aquaculture partnerships as aquaculture production requires a high degree of scientific and tech- nical competence relating to the unique physiological characteristics of farmed species, fish health, natural resource management, and aquaculture marketing and business. However, because aquaculture is akin in many ways to other natural resource-based industries, we argue that the results found in this research are generalizable to other contexts that involve complex interdepen- dencies between social, environmental, policy, and economic realms.
Our findings also suggest a positive association between individual learning and perceptions of partnership influence on policy-related problems, lack of good science for decision making, distrust among aquaculture stakeholders, and the socioeconomic health of fishers and tribes. Our learning measure captures what Leach et al. (2013) refer to as simple learning or knowledge acquisition. The specific measure we use is a multi-item measure reflecting partnership participants’ enhanced understanding of other stakeholders’ perspectives, aquaculture science, aquaculture policy, and aquaculture economics/business. Our findings regarding learning suggest one of two possibilities. One possibility is that partnerships that are successful in fostering individual-level learning on policy, technical, and social dimensions are also effective in producing positive collective outcomes on issues relating to these dimensions more broadly. Another possibility is that individuals who report intragroup learning are more likely to view the collective process as being more effective.
This latter interpretation could reflect a halo effect, or in this case particularly, could lend support to recent scholarship that emphasizes conceptual pathways relating to learning in collaborative pol- icy processes. For example, this finding offers empirical support for Gerlak and Heikkila’s (2011) learning framework that posits a distinction between learning processes and learning products and general relationships between them (Ostrom, 2005), whereby the process of knowledge acquisition in a policy context can translate to policy or institutional changes. They explicitly acknowledge
268 American Review of Public Administration 47(2)
learning products in evaluation terms as encompassing, “immediate outputs, such as the develop- ment of a new project, as well as longer term outcomes or impacts, such as the effects of a project on society or the environment” (Gerlak & Heikkila, 2011, p. 622). Although learning is treated as an individual-level outcome rather than process variable in this research per se, the learning measure used is generally consistent with their conceptualization of learning processes. Future research could more actively integrate Gerlak and Heikkila’s learning framework for a more nuanced assessment of how learning processes contribute to the types of learning products analyzed in this article.
No study is without limitations. First, even though we suggest our findings are relevant in the context of recent collaborative governance research (i.e., learning findings), we must consider the possibility that individuals’ perceptions of partnerships impact are biased by their perceptions of other attributes of the groups (Meier & O’Toole, 2013; Nisbett & Wilson, 1977). In addition, this research is constrained by limitations endemic to all survey research. Inevitably it is impos- sible to completely guard against the possibility of measurement error. However, we argue that the survey approach was appropriate given the types of variables we were interested in exploring in this article. Another limitation of this research is that we capture perceptions of partnerships impact using a cross-sectional study. Indeed, given the nature of our interests, a longitudinal design would be preferable. Incorporating an additional wave of data collection among partner- ship participants is a logical next step for this research. Despite these limitations, however, we argue that the research presented herein offers valuable insights about the key concepts in the collaborative policymaking literature.
Conclusion
Understanding outcomes relating to different modes of governance is a critical pursuit of schol- ars of public administration and related fields. As alternative forms of governance continue to develop to accommodate shifting actor, institutional, and public contexts, the need to examine their relative performance (in terms of outputs, outcomes, and impacts) is even more pressing. This research seeks to help illuminate procedural and individual-level factors that relate to col- laborative policymaking groups’ influence in the aquaculture policy domain on important pol- icy, technical, and social issues. This research contributes to the literature by highlighting the conceptual differences between different types of resources often studied in the context of col- laborative governance arrangements as well as reinforcing recent scholarships that links learn- ing through participation in collaborative settings and learning products such as changes in policy, environmental, or social conditions. For public administrators, this research offers guid- ance on how to structure or design collaborative processes to achieve desired policy objectives. Whereas their capacity to directly shape individual-level learning may be somewhat constrained, certainly they can encourage inclusivity, particularly by scientific and technical experts, proce- dural consistency, and low turn-around among group participants through various administra- tive strategies. A logical next step in this research is to conduct additional waves of data collection among partnership participants to assess changes in perceived group influence on policy, technical, and social problems relevant to their respective groups’ work over time.
Acknowledgements
This research was conducted as part of the Aquaculture Partnerships Project. Members of the research team included Chris Weible, Bill Leach, John Calanni, Scott Vince, and Saba Siddiki. The lead author thanks all members of the research team for their guidance, support, and colleagiality. Also, the authors wish to thank the anonymous reviewers for their constructive feedback on previous versions of the manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Siddiki and Goel 269
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publi- cation of this article: This study was supported by National Science Foundation Grant 0721067.
Notes
1. Individuals or entities that have a vested professional and/or personal interest in the development and/ or regulation of the aquaculture industry. Example stakeholders include aquaculturists, environmen- talists, industry regulators, and commercial fishermen. While many such stakeholders can be present within a state’s aquaculture industry context, partnerships typically include a limited number of repre- sentatives from different stakeholder groups.
2. Due to budgetary constraints, we excluded partnerships in Hawaii and Alaska. We also excluded nationwide partnerships that did not have a particular geographic focus.
3. Based on preliminary research conducted in the early stages of this study, a research advisory commit- tee was convened to help identify all partnerships currently or recently active at the time the data were collected, guide development of data collection instruments, and pilot test the survey instrument.
4. Individual partnership names are kept confidential to protect the anonymity of study participants. 5. To recruit interviewees, an interview request was first sent to each member of the 10 partnerships.
Anyone who responded to this call was interviewed. During the interviews, research team members requested names of individuals who were not currently members but who closely followed the work of interviewees’ partnerships that could be recruited for participation in an interview. The interviews typically lasted 60 to 90 min. Given that the purpose of the interviews was to inform the construction of survey questions and associated response items, normal concerns about interview sample bias are attenuated.
6. The number of participants per partnership ranged from 7 to 43. Individual response rates for the partnerships were as follows: Partnership 1 = 13/22 or 59%; Partnership 2 = 10/14 or 71%; Partnership 3 = 12/23 or 52%; Partnership 4 = 10/17 or 59%; Partnership 5 = 8/10 or 80%; Partnership 6 = 6/7 or 86%; Partnership 7 = 16/25 or 64%; Partnership 8 = 25/43 or 58%; Partnership 9 = 18/28 or 64%; and Partnership 10 = 5/9 or 56%.
7. The survey prompt for partnership influence on listed problems uses the term “impact” in it (“For each of the following issues, please indicate whether the [partnership name] has had any actual impact so far using a scale ranging from ‘made the issue much worse’ to ‘made the issue much better’”), though we characterize partnership influence as outcomes in this study because of the inability to tie partnership influence to specific partnership outputs or attributes.
8. Unit of analysis is individual partnership participants. 9. For each partnership, we took the mean of participants’ responses to questions about problem severity
and influence of respondents’ partnership on listed problems (listed in Table 2). The response scale for problem severity was 1 = not a problem at all to 5 = very serious problem. The response scale for perceived influence was −2 = made the issue much worse to +2 = made the issue much better. Next, we added partnership level mean responses to the severity and influence questions across all partnerships to get a “summed mean” for problem severity and a “summed mean” for problem influence. Given the differing scales for the severity and influence questions, the possible range of values for the problem severity question across all 10 partnerships is 10 = not a serious problem at all to 50 = very serious problem. The possible range of values for the problem influence question across all partnerships is −20 = made the issue much worse to +20 = made the issue much better.
References
Amberg, S., & Hall, T. E. (2010). Precision and rhetoric in media reporting about contamination in farmed salmon. Science Communication, 32, 489-513.
Baumgartner, F. R., & Jones, B. D. (2010). Agendas and instability in American politics. Chicago, IL: University of Chicago Press.
Beierle, T. C., & Cayford, J. (2002). Democracy in practice: Public participation in environmental deci- sions. Washington, DC: Resources for the Future.
270 American Review of Public Administration 47(2)
Biddle, J. C., & Koontz, T. M. (2014). Goal specificity: A proxy measure for improvements in envi- ronmental outcomes in collaborative governance. Journal of Environmental Management, 145, 268-276.
Bidwell, R. D., & Ryan, C. M. (2006). Collaborative partnership design: The implications of organizational affiliation for watershed partnerships. Society & Natural Resources, 19, 827-843.
Bryson, J. M., Crosby, B. C., & Stone, M. M. (2006). The design and implementation of cross-sector col- laborations: Propositions from the literature. Public Administration Review, 66(Suppl. 1), 44-55.
Emerson, K. (2009). Synthesizing practice and performance in the field of environmental conflict resolu- tion. In R. O’Leary & L. Blomgren Bingham (Eds.), The collaborative public manager (pp. 215-232). Washington, D.C.: Georgetown University Press.
Emerson, K., Nabatchi, T., & Balogh, S. (2012). An integrative framework for collaborative governance. Journal of Public Administration Research and Theory, 22, 1-29.
Firestone, J., Kempton, W., Krueger, A., & Loper, C. E. (2004). Regulating offshore wind power and aqua- culture: Messages from land and sea. Cornell Journal of Law and Public Policy, 14, 71-111.
Gerlak, A. K., & Heikkila, T. (2011). Building a theory of learning in collaboratives: Evidence from the Everglades Restoration Program. Journal of Public Administration Research and Theory, 21, 619-644.
Heclo, H. (1978). Issue networks and the executive establishment. In A. King (Ed.), The new American political system (pp. 87-124). Washington, D.C.: American Enterprise Institute for Public Policy Research.
Heikkila, T., & Gerlak, A. K. (2014). Investigating collaborative processes over time: A 10-year study of the South Florida ecosystem restoration task force. The American Review of Public Administration. Advance online publication. doi:10.1177/0275074014544196
Heinrich, C. J. (2002). Outcomes-based performance management in the public sector: Implications for government accountability and effectiveness. Public Administration Review, 62, 712-725.
Herian, M. N., Hamm, J. A., Tomkins, A. J., & Zillig, L. M. P. (2012). Public participation, procedural fairness, and evaluations of local governance: The moderating role of uncertainty. Journal of Public Administration Research and Theory, 22, 815-840.
Innes, J. E., & Booher, D. E. (1999). Consensus building and complex adaptive systems: A framework for evaluating collaborative planning. Journal of the American Planning Association, 65, 412-423.
Kaiser, M., & Stead, S. M. (2002). Uncertainties and values in European aquaculture: Communication, management and policy issues in times of “changing public perceptions.” Aquaculture International, 10, 469-490.
Koontz, T. M., & Thomas, C. W. (2006). What do we know and need to know about the environmental outcomes of collaborative management? Public Administration Review, 66(Suppl. 1), 111-121.
Leach, W. D., Pelkey, N. W., & Sabatier, P. A. (2002). Stakeholder partnerships as collaborative policy- making: Evaluation criteria applied to watershed management in California and Washington. Journal of Policy Analysis and Management, 21, 645-670.
Leach, W. D., & Sabatier, P. A. (2005). To trust an adversary: Integrating rational and psychological models of collaborative policymaking. American Political Science Review, 99, 491-503.
Leach, W. D., Weible, C. M., Vince, S. R., Siddiki, S. N., & Calanni, J. C. (2013). Fostering learning through collaboration: Knowledge acquisition and belief change in marine aquaculture partnerships. Journal of Public Administration Research and Theory, 24, 591-622.
Lubell, M. (2003). Collaborative institutions, belief-systems, and perceived policy effectiveness. Political Research Quarterly, 56, 309-323.
Lubell, M., & Fulton, A. (2008). Local policy networks and agricultural watershed management. Journal of Public Administration Research and Theory, 18, 673-696.
Maggioni, E., Nelson, H., & Mazmanian, D. A. (2012). Industry influence in stakeholder-driven state cli- mate change planning efforts. Policy Studies Journal, 40, 234-255.
Mazur, N. A., & Curtis, A. L. (2006). Risk perceptions, aquaculture, and issues of trust: Lessons from Australia. Society & Natural Resources, 19, 791-808.
Meier, K. J., & O’Toole, L. J. (2013). Subjective organizational performance and measurement error: Common source bias and spurious relationships. Journal of Public Administration Research and Theory, 23, 429-456.
National Oceanic and Atmospheric Administration. (2014). What is aquaculture? Retrieved from http:// www.nmfs.noaa.gov/aquaculture/what_is_aquaculture.html
Siddiki and Goel 271
Naylor, R. L., Goldburg, R. J., Primavera, J. H., Kautsky, N., Beveridge, M. C., Clay, J., . . .Troell, M. (2000). Effect of aquaculture on world fish supplies. Nature, 405, 1017-1024.
Nisbett, R. E., & Wilson, T. D. (1977). The halo effect: Evidence for unconscious alteration of judgments. Journal of Personality and Social Psychology, 35, 250-256.
Ostrom, E. (1990). Governing the commons: The evolution of institutions for collective action. Cambridge, UK: Cambridge University Press.
Ostrom, E. (2005). Understanding institutional diversity. Princeton, NJ: Princeton University Press. Resh, W., Siddiki, S., & McConnell, W. R. (2014). Does the network centrality of government actors mat-
ter? Examining the role of government organizations in aquaculture partnerships. Review of Policy Research, 31, 584-609.
Rogers, E., & Weber, E. P. (2010). Thinking harder about outcomes for collaborative governance arrange- ments. The American Review of Public Administration, 40, 546-567.
Sabatier, P. A., & Weible, C. M. (2007). The advocacy coalition framework: Innovations and clarifica- tions. In Theories of policy process. In P. Sabatier (Ed.), Theories of the policy process (pp. 189-220). Boulder, CO: Westview.
Thomas, C. W. (2008, April). Evaluating the performance of collaborative environmental governance. Consortium on Collaborative Governance Mini-Conference, Santa Monica, CA.
Treece, G. D. (2002). Shrimp farm effluents. In R. Stickney & J. McVey (Eds.), Responsible marine aqua- culture (pp. 297-309). New York, NY: CABI Publishing.
Tyler, T., Degoey, P., & Smith, H. (1996). Understanding why the justice of group procedures matters: A test of the psychological dynamics of the group-value model. Journal of Personality and Social Psychology, 70, 913-930.
Weber, E. P. (1998). Pluralism by the rules: Conflict and cooperation in environmental regulation. Washington, D.C.: Georgetown University Press.
Weible, C. M. (2008). Expert-based information and policy subsystems: A review and synthesis. Policy Studies Journal, 36, 615-635.
Weible, C. M., & Sabatier, P. A. (2009). Coalitions, science, and belief change: Comparing adversarial and collaborative policy subsystems. Policy Studies Journal, 37, 195-212.
Wirth, F. F., & Luzar, E. J. (1999). Environmental management of the U.S. aquaculture industry: Insights from a national survey. Society & Natural Resources, 12, 659-672.
Author Biographies
Saba Siddiki is an Assistant Professor in the School of Public and Environmental Affairs, Indiana University-Purdue University Indianapolis.
Shilpi Goel is a Master of Public Affairs student in the School of Public and Environmental Affairs, Indiana University-Purdue University Indianapolis.
