Week 4 - Assignment: Assess Distributive Policy Administration at the Agency Level and Week 5 - Assignment: Evaluate the Political Backlash of Failed Policies

Policy Failure and Policy Learning: Examining the Conditions of Learning after Disaster

Kristin O’Donovan Wayne State University

Abstract

The policy learning literature indicates that governments can and do learn after a policy failure but not always. The purpose of this study is to examine the conditions under which policy failure leads to policy learning. It asks two questions. First, is policy failure associated with policy learning? Second, if policy is associated with policy learning, does the failure initiate different types of learning? Using the policy failure and learning literatures as an organizing framework, this paper analyzes three comparative cases of policy failure revealed by tornados in Greensburg, Kansas (2007); Joplin, Missouri (2011); and Moore, Oklahoma (2013). It finds that failure of the policy process in agenda setting is associated with instrumental policy learning. It also finds that there are two types of failure of decision making: failure to make a timely decision and failure to make any decision. The two types of decision-making failure are associated with different types of policy learning. In addition, the contextual factors underpinning the link between policy failure and learning are prior experience with the policy problem and capacity to learn.

KEY WORDS: policy failure, policy learning, governance, disaster and risk management

Overview

The policy learning literature indicates that governments can and do learn after a policy failure but not always. Policy failures are not always easy to identify because

they are contested constructs, both in practical policy terms and in scholarly investi-

gation (Bovens & ‘t Hart, 2016). Policy failure is not a complete or absolute phe-

nomenon, meaning that a policy can fail in some areas but succeed in others

(McConnell, 2015, 2016). Moreover, policy failure is a multidimensional phenome-

non, meaning that policy failures vary in extent, duration, visibility, intensity, and

avoidability, among other factors (Howlett, 2012).

The purpose of this study is to examine the conditions under which a perceived

policy failure leads to policy learning. It seeks to advance the discussion concerning

the relationship between policy failure and learning by systematically analyzing

three comparative cases. It asks two questions. First, is policy failure associated with

policy learning? Second, if policy is associated with policy learning, does the failure

initiate different types of learning? Using the policy failure and learning literatures

as an organizing framework, this paper analyzes three cases of policy failure

revealed by tornados in Greensburg, Kansas (2007); Joplin, Missouri (2011); and

Moore, Oklahoma (2013).

Examining the three cases of Greensburg, Joplin, and Moore allows for the com-

parison of how different communities learn after the same triggering mechanism,

natural disasters. The damage caused to each of these communities creates the per-

ception that the policies in place to mitigate and lessen the effects of tornadoes

failed. Comparison of the cases allows opportunities to examine the extent to which

local government officials engaged in policy learning about hazard mitigation.

Review of Policy Research, Volume 34, Number 4 (2017) 10.1111/ropr.12239 VC 2017 Policy Studies Organization. All rights reserved.

537

bs_bs_banner

This study has two central findings. First, it develops an analytic framework that

links policy failure with policy learning. The framework examines the connection

through the lens of types of policy failure, specifically failures of policy programs and

failures of the policy process. This is an important contribution because it assumes

that policy failures can create the different conditions for learning. Second, the

findings from the three comparative case studies support the link between failures

of agenda setting and learning about policy instruments and the link between fail-

ures of decision making with social and political learning. It makes an important

distinction between failure to make a timely decision and failure to make any

decision.

The remainder of this paper is divided into four sections. The next section dis-

cusses policy failure and the connection to policy learning. The third section

presents the case study and the cases. The fourth section presents that comparative

case analysis and discusses the meaning and implications of the results.

A Framework for Assessing Policy Failure, Policy Learning, and Types of Policy Learning

This section presents an analytical framework for assessing the link between policy

failure and learning and determining whether policy failure leads to specific types

of policy learning. Policy failure is an important aspect of policy studies because it

signals possible shortcomings of policy design, the inability of a policy to meet its

intended goals, potential problems associated with implementation, and the pros-

pect of policy reformulation and revision. However, it is not without conceptual,

theoretical, and practical challenges. Policy failure as a theoretical idea to define

and test is as Bovens and ‘t Hart (2016) point out a “contested construct” for schol-

arly inquiry. An early challenge of specifying policy failure was in conceiving of it as

a problem of social construction (Bovens & ‘t Hart, 1995). Furthermore, policy fail-

ure tends to occur on a continuum and policy makers do not neatly make bound-

edly rational decisions (Hindmoor & McConnell, 2013). The discussion in the

literature has moved toward examining and understanding the root causes for the

persistence of policy failures, including the political aspects of the policy process

(Howlett, Ramesh, & Wu, 2015).

Furthermore, policy learning has been conceived of occurring based on how

knowledge and information is used. It can happen reflexively where problems

are manageable and preferences of actors are malleable; epistemic learning

occurs where problems are manageable but the knowledge and preferences of

the actors are fixed; and learning through bargaining where the problems are

somewhat intractable but the preferences of decision makers are malleable

(Dunlop & Radaelli, 2013). Among these mechanisms for how decision makers

balance issues and preferences, there is some outcome that occurs. Even in the

most fundamental sense, policy learning leads to an updating of the most basic

beliefs (Dunlop & Radaelli, 2013). However, policy learning is inherently politi-

cal. In spite of the work in epistemic communities to produce policy-relevant

knowledge, the depths of learning can be limited in the assessment of policy

alternatives (Dunlop, 2009). It can take the form of bargaining about

538 Kristin O’Donovan

competition for scarce resources rather than learning reflexively about the con-

ceptual nature of the policy problem in a way that relies upon deliberation and

cooperation (Dunlop & Radaelli, 2016).

How Is Policy Failure Associated with Policy Learning?

Policy failure is associated with policy learning in two important ways. First, policy

failure can shape or influence the policy options available for officials to learn from

or learn about. The link between policy failure and policy learning can be contin-

gent upon the availability of policy solutions to decision makers (Walsh, 2006). This

link is consistent with an explanation of policy failure which suggests that failure is

caused by an apparent disconnect between the rhetoric about what a policy is

promised to achieve and the implementation of the policy (May, 2015). Walsh

(2006) draws on the availability of policy ideas from the Multiple Streams

Approach, which indicates that one stream captures and assesses the policy problem, a second contains multiple policy ideas or solutions, and a third represents the polit- ical context and actors shaping the policy subsystem (Kingdon, 2003; Zahariadis, 2003, 2014).

Second, failure of a public policy in how a policy is formulated, decided upon,

and enacted can create opportunities for learning about how well the policy

addresses a problem and its implementation. For a policy to be successful, the poli-

cy process in which the policy is formulated, considered, and adopted must be

sound; the politics of the issue must be sound; and the resulting program must be

effective (Marsh & McConnell, 2010; McConnell, 2010). In his interpretation of

policy successes as they related to policy failures, Howlett (2012) argued that poli-

cies fail because of programmatic issues, process issues, and political issues. The

failures of policy programs and process are important because they are more

focused on policy problems and the tools associated with addressing such prob-

lems. The failures of national politics specifically, the coalitional, legislative, elector-

al, and executive ability to govern seem less relevant at the local level of

government. However, in keeping with the sentiment of “all politics are local,”

there is a likelihood that local political concerns in a community would be relevant

in the concerns for how policy solutions are considered, debated, and ultimately

passed (see Table 1).

In this framework, policy failures that stem from the policy tools or techni-

ques related to a specific program can be addressed by gathering information,

assessing, and learning about the specific techniques (Howlett, 2012). In the

context of hazard mitigation after a disaster, programmatic failures have two

common sets of choices to make about how to reduce disaster risk. Structural

mitigation techniques like building safe rooms or storm shelters are one set of

policy ideas available in light of a policy failure revealed by a tornado. Nonstruc-

tural mitigation policies include enforcement of building codes, raising the mini-

mum standards for building codes for wind hazards, or limiting development

through land use planning to avoid residential or commercial construction in

hazardous areas of communities. Failure to understand the differences between

structural and nonstructural types of mitigation is one way to conceive of pro-

grammatic failure.

Policy Failure and Policy Learning 539

Failures of the policy process are more varied. Failures in agenda setting can be

caused by government overreach in trying to set an unattainable policy goal (Howl-

ett, 2012). Failure in policy formulation can be caused by a lack of research into the

cause of the policy problem and its alternative solutions (Howlett, 2012). Failures

can be shortcomings in decision making, meaning failing to make a timely decision

or distorting the intent of a policy through deal making (Howlett, 2012). Failures

in policy implementation are caused by an inability to deal with a lack of resources,

and principal–agent problems, among others (Howlett, 2012). Finally, failures of

policy evaluation imply a lack of learning because of ineffective policy evaluation

and feedback (Howlett, 2012).

Natural hazard mitigation takes place in a “patchwork system” that is inherently

problematic because the federal government sets priorities for national standards

and objectives while state and local governments have traditionally had the respon-

sibility and authority to reduce risks caused by natural hazards (May & Deyle,

1998). This integrated approach to policy making is prone to policy failures

because of the challenges of alternative selection and policy design, among other

factors (Vince, 2015).

In sum, the literature indicates that policy failure is associated with policy learn-

ing in two ways. The manner in which policy solutions or tools are considered to

address the problem is one way to link failure with learning. The other is the ways

in which policy is formulated, decided upon, and implemented.

Table 1. Types of Policy Failure Applied to Tornado Risk Mitigation

Type of Policy Failure Definition

Application to Tornado Risk

Mitigation

Failure of policy program Failure of a policy program to reach

its stated goals. Failure to gather information about specific tools and

techniques

Failure to seek out existing informa-

tion about how to reduce tornado risks

Failure of policy process

Agenda setting Overreaching government establishing or agreeing to establish

overburdened or unattainable

policy agendas

Overly ambitious or overly modest policies to eliminate, rather than

reduce the risk of tornado damage

Policy formulation Attempting to deal with problems without investigating or researching

problem causes and identifying the

effects on policy alternatives

Reflexive policy making to address tornado risk based without

consideration for local needs

Decision making Failing to decide on a policy within a reasonable period of time or

distorting its intent through

bargaining and logrolling

Failure to make a decision to adopt a tornado mitigation policy. Watering

down preparedness standards to

gain community support

Policy implementation Failing to deal with implementation problems including lack of

resources, principal–agent

problems, oversight failures, and

others

Lack of adequate human, capital, or political resources to implement

existing building codes, planning

ordinances, maintain shelters, drill

warning systems, and shelters Policy evaluation Lack of learning due to lack of or

ineffective or inappropriate policy

monitoring and/or feedback

processes and structures

Absence of policy evaluation, either

formal or informal, after a tornado

that would include damage assess-

ments, review of existing warning, sheltering, and construction policies

Source: Adapted from Howlett (2012).

540 Kristin O’Donovan

Does Policy Failure Initiate Different Types of Learning?

Policy failures occur when “decision makers responsible for initiating the consider-

ation of and approving new policies conclude that current policy is no longer

achieving the political and program goals they prefer” (Walsh, 2006). Policy failure

is not absolute, rather in the event that a policy may not achieve the intended goals,

some portions of the policy can still be considered successful (McConnell, 2016).

This paper argues that treating all policy failures the same is too general of an

approach to analyzing the connection between failure and learning. It is necessary

to consider why policy fails and the different types of failure that occur discern the

connection to policy learning and the differences in learning types.

The extent to which policy failure occurs can range from tolerable failures that

include a small shortcoming that does not interfere with achieving goals, to conflict-

ed failure where shortcomings and goal attainment are evenly balanced, and out-

right failure where the policy does not achieve its goals (McConnell, 2015).

Policy failure has been conceived of as inherently a subjective concept that leaves

open interpretation of what success might be and what it means to different stake-

holders if a policy falls short. In addition, policy failure or success is a concept that

is argued by proponents or detractors of a policy and framing; framing and label-

ing a policy as a failure is an inherently political endeavor that can be highly vari-

able based on different views from advocates and stakeholders (Bovens & ‘t Hart,

2016).

Different sets of criteria to objectively evaluate claims of policy success or failure

have been developed in response to challenges of subjectivity and social construc-

tivism in the policy failure literature. An early set focused on the basis of the claim

of success or failure, including whether the original objectives were met, the impact

on the target population, the results of the policy, the importance, the sources of

support from stake holders, jurisdiction, costs and benefits, innovativeness, and

normative concerns (McConnell, 2010). However, this approach was critiqued for

presenting failure as the opposite of success (Howlett, 2012).

A refined set of criteria focused on the dimensions of policy failure, arguing poli-

cy failure could be assessed based on the extent or size, the avoidability, the visibili-

ty, the intentionality, the duration, and the intensity (Howlett, 2012; Howlett et al.,

2015).

Policy can fail from facets of the policy process, although policy can also fail from

shortcomings in the program itself or in the politics of the policy (Bovens, 2010;

Howlett, 2012; Howlett et al., 2015). When policy failure occurs as an issue arising

from the policy process, it can stem from agenda setting, policy formulation, deci-

sion making, implementation, or evaluation (Howlett, 2012). Policy failure, or the

perception by decision makers that policies failed, provides the opportunity for

policy change. Even though policy failures should be a mechanism for policy learn-

ing, and ultimately policy change, this is not always the case (May, 1992). However,

policy failure alone is not enough to explain policy change (Walsh, 2006).

The challenge of considering natural disasters, in this case tornadoes, as policy

failures is that the relative suddenness of the events and the revelation of a failure

of preparedness or mitigation policy is immediately evident. By contrast, most poli-

cy failures, or evidence of policy failure, take years to appear and become evident.

Policy Failure and Policy Learning 541

And once they do become apparent, the data and indicators of the failure will be

interpreted, argued, and framed (Bovens & t’Hart, 1995, 2016). Policy failures are

often revealed through potential focusing events, sudden and rare events that cre-

ate the opportunity for policy change by revealing harm or the potential for harm,

and the public and policy makers become aware of nearly simultaneously

(Birkland, 1998, 2006).

A key challenge with considering natural disasters as revealing policy failure is

that in the immediate aftermath of a tornado, the damage is undeniable and public

efforts are focused on returning to normal life as quickly as possible (Mileti, 1999).

Indeed, most behavior after a disaster is prosocial in nature, meaning that people

want to help their neighbors and communities bounce back (Tierney, Bevc, &

Kuligowski, 2006). This creates an opportunity for policy learning where finger

pointing for the policy failure and blame avoidance by policy makers is less critical

than it is for most policy failures (Howlett et al., 2015).

Types of Learning

Policy learning can generally be conceived of as “thin” learning about policy tools

and techniques and “thick” learning that experiments in understanding the root

causes of problems or strategically political tactics (Howlett, 2012). One way of

examining the outcome of policy learning has been to look for evidence of out-

comes in policy change. There is a basic typology for policy learning indicating that

it can occur in three ways: instrumental policy learning, social policy learning, or

political learning (May, 1992).

Instrumental policy learning is defined as a new understanding about how a policy tool works, is designed, or is implemented; learning occurs by observation or trial

and error and is not necessarily a purposive act (May, 1992). In the context of haz-

ard mitigation, local governments have typically shown little interest in implement-

ing policies that will protect people and property before a disaster. For example,

building codes are often well written but poorly enforced (Kunreuther, 1996). This

study anticipates that the same type of learning would be evident for local govern-

ments. However, in the context of hazard mitigation, incomplete or haphazard

implementation may constrain instrumental policy learning by limiting the under-

standing about how a policy tool works.

What kind of failure might initiate instrumental policy learning? Based on the

analytic framework, it is anticipated that failures of policy programs and specific

failures of the policy process, namely agenda setting, would initiate thin policy

learning about instruments and tools.

Social policy learning is defined as a new understanding of a policy by a change in the social construction of the problem and the causal reasoning underlying the def-

inition of the problem (May, 1992). The agenda-setting literature indicates that

“focusing events” (Kingdon, 2003) are defined as large-scale sudden events that

simultaneously focus public and elite attention, revealing harm or the potential for

harm caused by a policy problem (Birkland, 1997). Focusing events create the

opportunity for change in how a problem is socially constructed. For example,

research on aviation security indicates that social policy learning occurred after

September 11 when the issue of airport security was redefined from preventing

542 Kristin O’Donovan

hijackings to protecting people from terrorists (Birkland, 2004a, 2004b). It is possi-

ble that a disaster, even a highly localized event, could similarly create the opportu-

nity for social policy learning about hazard mitigation (see Table 2).

It is anticipated that failures of the policy process, specifically failures of policy

formulation, decision making, implementation, and evaluation would initiate social

policy learning. A policy that is formulated with a flawed causal model or theory of

action might indicate that decision makers do not understand the fundamental

causes of a policy problem. Or alternatively the failure of decision making—either

the inability to make a decision or a partial or brokered decision—would initiate

social policy learning after a disaster.

Political learning is defined as changes in knowledge about the effectiveness of strategies that policy advocates use to advance policy ideas or draw attention to

problems (May, 1992). Political learning about mitigation can generally be con-

strained by other problems in the community, like crime, or because of pres-

sures for development in areas that are attractive and prone to natural hazards

(Prater & Lindell, 2000). Political learning has the potential to underscore a

conundrum regarding planning and mitigation. While mitigation would protect

communities in the long term, it sometimes occurs at the expense of short-term

economic development concerns. It is anticipated that specific kinds of failures

will not exclusively lead to political learning. However, failures of the policy pro-

cess may initiate political learning in addition to other types of policy learning.

As learning solely about tactics to press policy change would be insufficient,

learning about tools and underlying causes might also be insufficient to create

favorable conditions for learning that results in policy change. Simply put, poli-

cy failure of any kind is not very likely to lead to political learning alone, rather

political learning is likely to be coupled with another form of learning after a

failure.

Table 2. Policy Learning Types

Learning Type Definition Requisite Conditions Prima Facia Indicators

Instrumental learning New understanding about

how a policy tool works, is

designed, or imple- mented. Learning occurs

by trial and error and is

not necessarily purposive

Improved understanding of

policy instruments or

implementation based on experience or formal

evaluation

Policy redesign entailing

change in instruments for

carrying out the policy (inducements, penalties,

assistance, funding, timing

of implementation, and

organizational structures) Social policy learning New understanding of a

policy by a change in the

social construction of the

problem and the causal reasoning underlying the

definition of the problem

Improved understanding or

alteration of dominant

causal beliefs about a

policy problem or solution within the relevant policy

domain

Policy definition entailing

change in policy goals or

scope (policy direction,

target groups, rights)

Political learning Changes in knowledge about

the effectiveness of strategies that policy

advocates use to advance

policy ideas or draw

attention to problems

Awareness of political

prospects and factors that affect them

Change in political strategy,

changing venues, new arguments, and new

tactics to call attention to

a problem

Source: Adapted from May (1992).

Policy Failure and Policy Learning 543

What Can Be Learned after a Tornado?

The process of gathering new information or knowledge about ways to address a

policy problem often begins with “lesson drawing” from other places that have

encountered the same problem (Rose, 1991). The process of “lesson drawing”

begins from dissatisfaction with the status quo, rather than the desire to seek out

new ways to address policy problems that is supported by political leaders (Rose,

1991). For tornado risk reduction, the status quo of unmitigated tornado damage

risk is signaled by the disaster. Dissatisfaction with the status quo might be indicated

by the need to change policies to reduce the risk of tornado damage in the future.

Lesson drawing takes place by looking “across time or across space; the choice

depends upon a subjective definition of proximity, upon epistemic communities

linking experts together, functional interdependence between governments and

the authority of intergovernmental institutions” (Rose, 1991, pp. 5–6). For tornado

risk reduction, the process of lesson drawing could take place across time, spanning

from one major disaster to another, in which the epistemic communities of engi-

neers, planners, insurance, and building officials would be linked, and the func-

tional interdependence of local, state, and federal agencies and institutions would

possess a set of lessons to be drawn.

Moreover, Rose’s (1991) lesson drawing concept very specifically defines lessons

to be “an action oriented conclusion about a program or programs in operation

elsewhere” (p. 7). There are five specific lessons that can be drawn as policy

responses after a tornado.1

The first instrument is encouraging or requiring the use of safety rooms. One

common approach is including safety rooms in new construction of schools, hospi-

tals, daycares, and nursing homes. Another approach is to encourage builders to

include safety rooms in new residential construction. A third approach is to retrofit

existing residences with safety rooms or storm shelters that can be installed

underground.

The second instrument available is to require or encourage building practices

that promote wind resistance. For new residential construction, this technique

means including structural bracing for walls and roofs, hurricane straps and clips

for roofs, using anchor bolts to secure the house to the foundation, using laminated

or impact-resistant windows, and using reinforced exterior doors and garage

doors, and installing shutters for windows. For commercial construction, this

means pouring concrete walls on-site, rather than using prefabricated concrete

wall panels. This instrument saves property and lives.

For both safe rooms and changes in building practices, the extent to which com-

munities choose to implement these techniques can range from changes in the out-

right requirements, manifested as changes in the building codes, to incentives and

encouragement for building to safer standards or installing a shelter. Generally,

FEMA and state-level emergency management agencies provide funding for indi-

viduals to install underground storm shelters for existing residences.

The third instrument available is public education concerning tornado risks and

is intended to save lives. Public education concerning tornado risks generally aims

to remind people about the difference between a tornado watch and warning. It

also includes public awareness campaigns at the start of tornado season to remind

544 Kristin O’Donovan

people when, where, and how to take shelter. Public awareness and education cam-

paigns are often supported by FEMA and state-level emergency management

agencies.

The fourth instrument is use of tornado warning systems and is also intended

to save lives. Testing the sirens, improving the warning system, increasing the vol-

ume of the sirens, or adding more sirens is another instrument intended to

reduce the impact of tornadoes. Tornado sirens are very effective at warning the

public to get out of harm’s way and take shelter. However, the sirens have to be

heard by the public in order to be effective. The fifth instrument is maintenance

of landscaping to trim bushes and cut down trees that might become projectiles

during a tornado.

Comparative Case Methodology

This paper relies on three case studies to explore the analytical framework pre-

sented above. A comparative case study approach is appropriate when the study is

examining a real-life phenomenon where the real-life context and phenomenon

are not easily separated and multiple sources of evidence are used (Agranoff &

Radin, 1991; Yin, 2013). As the goal of this study is to contribute to the scholarly

discussion on policy failure and learning, a comparative case study approach is

appropriate for one important reason. Using comparative cases allows the research

to selectively focus on a set of concepts that can only be examined within a real-life

context and the comparison of concepts, like policy failure and learning, allow for

greater generalization (Agranoff & Radin, 1991).

The cases selected for this study were chosen for three reasons. First, the dam-

age caused by the tornadoes was similar in scale, thus making impact of the disas-

ters as comparable as possible. The cases represent three of the most recent and

most damaging tornados in the United States. Second, by selecting tornadoes as

cases, all contain the same triggering event for policy failure. By comparing three

events with potentially three different instances of policy learning, there should be

some degree of generalization of the conditions under which policy failure is associ-

ated with learning and the type of learning that it triggers. Third, these three cases

allow us to assess policy learning by focusing on a common set of policy responses

available to decision makers.

For the case studies, information was gathered from a Lexis Nexus search of the

local newspaper of record in each community using the search terms “tornado,”

“tornado damage,” “disaster recovery,” and “mitigation.” A Google Scholar search

was conducted using the same search terms to establish what the academic and

trade literature had investigated on each disaster. Separate searches were con-

ducted for government documents. First, at the local level, disaster preparedness

and planning documents, press releases, city council meeting minutes, and recov-

ery plans were gathered from Moore, Joplin, and Greensburg. At the state level,

state emergency management agency records were searched for information about

the tornados. For federal-level information, the online FEMA Resource and Docu-

ment library was searched for disaster-specific information. The time span for each

search began on the day after the tornado and ended one year later.

Policy Failure and Policy Learning 545

Each document retrieved as a result of the searches was saved as a .pdf file and

stored in the appropriate cloud-based directory for Moore, Joplin, and Greens-

burg. Using the newspaper reports, timelines of the recovery were developed as a

foundation for each case. Details were added in steps beginning with local, state,

and federal resources and concluding with any academic or trade sources. Using

the analytic framework outline above, all documents were examined for evidence

of failure and then of learning.

In some instances, it was not possible to document every aspect of the event. For

example, there were no public records that documented deliberations within the

City of Moore local government in what was important in how they considered

information about structural failure and damage when considering building code

changes. In this case, the relative importance of the information was inferred from

a host of sources, including press releases, technical reports, and meeting agendas.

And, for example, in the case of Greensburg, there were not adequate sources

from the municipal government to assess policy learning directly from disaster

plans or meeting minutes. However, failure and learning were assessed based on

ample state and federal sources which indicated that the City of Greensburg was

primarily concerned with a recovery strategy centered on green building practices

and not mitigating the risk of future tornado damage.

Case Studies

Moore, Oklahoma

Moore, Oklahoma, is a city located south of Oklahoma City and north of Norman.

A large and powerful tornado struck Moore during the afternoon on May 20,

2013. The strength of the tornado was rated EF-5, the highest rating on the

Enhanced-Fujitsa scale. While there had been some limited warning, the tornado

killed 24 people, including 7 children. The tornado destroyed hundreds of homes

and, most notably, leveled the Plaza Towers Elementary School. Plaza Towers did

not have a tornado safe room nor did it have a fortified shelter area within the

school. The lack of tornado shelters is not uncommon in schools in Oklahoma,

where 62% of schools do not have fortified shelters.

The absence of a storm shelter or fortified shelter within Plaza Towers Elementa-

ry presents a policy failure, or at least a perceived policy failure in the lack of torna-

do hazard mitigation in the schools. Moore had experienced a damaging tornado

in 1999 that was similar in strength and followed the same path (Horning, 2013).

However, there is no evidence of policy learning following the 1999 tornado. Sur-

veys of tornado damage to residential structures revealed that local building offi-

cials were enforcing the minimum requirements of the building code, even though

construction that followed more stringent standards—like the use of “hurricane

straps” on roofs instead of nails or bolting metal plates to slab foundations to secure

walls—resulted in less damage (Marshall, 2002).

There is prima facie evidence to suggest that the City of Moore engaged in policy

learning. There were no public shelters in Moore prior to the 2013 tornado and

the policy to shelter-in-place remains unchanged (City of Moore, 2014). This is not

546 Kristin O’Donovan

uncommon in tornado-prone areas where people are often instructed to shelter-in-

place rather than venture out during a tornado in search of a public shelter. How-

ever, the City of Moore has changed residential building codes for future construc-

tion. The code is designed to make homes withstand 135-mph winds by requiring

hurricane straps on roofs, wall anchors, and materials requirements for roof and

structural sheathing (Ordinance No. 768 {14}).

The ordinance, adopted on March 17, 2014, is an amendment to the standard

building code written by the International Code Commission, which, like in many

communities, is incorporated, in whole or in part, by Moore’s building code.

Moore’s amendment to the code by ordinance reflects the first-ever adoption of

building code requirements specifically designed for tornado activity (City of

Moore, 2014). In fact, this was the first ever change in the standard building code

by any municipality that specifically addressed tornado mitigation in the United

States (City of Moore, 2014). This particular code change stemmed from a report

from a team of researchers that evaluated damage after the tornado. The

researchers also made the argument that residents of Moore should expect simi-

larly strong tornadoes every two or three years (Ramseyer & Holliday, 2014).

This change in the building code only applies to new construction; there are no

requirements in place to retrofit existing residential structures. It is important to

point out that retrofits for wind hazard mitigation are very rare. In addition,

there are still no local ordinances in place that require homes, schools, or busi-

nesses to have safe rooms or storm shelters (Schwartz, 2013). One common expla-

nation is the cost of installing safe rooms is prohibitive without major renovations

or grants from the city or state.

However, there is evidence to suggest that the City of Moore has taken steps to

encourage private storm shelters under the ShelterMoore Storm Shelter Rebate

Program that provides $2,500 rebates for storm shelters that meet specific safety

requirements. This is in spite of problems with the availability of mitigation funding

from FEMA. Moore is in Cleveland County, and the county’s hazard mitigation

plan expired in 2011 and so it was not eligible to receive hazard mitigation grant

funding from FEMA (Adcock, 2013). When the presidential disaster declaration

was granted, hazard mitigation projects in communities like Stillwater, Oklahoma

that had existing plans approved and in place received the majority of the federal

mitigation funding (Adcock, 2013).

There is no evidence to suggest that the building code change spread among

neighboring cities. There have not been similar changes or amendments to the

building code for the neighboring City of Norman, which still retains the 95-mph

building standards that it had in place before the May 2013 tornado (Ordinance

No. O-0910-7). Nor have there been similar changes or amendment to the building

code for Oklahoma City.

The Moore Public School district borders the following school districts: Crooked

Oak, Mid-Del City, Robin Hill, McCloud, Little Axe, Norman, New Castle,

Mustang, and Western Heights. Moore Public Schools has built new schools with

storm shelters, replacing damaged or destroyed schools. In addition, it has reviewed

the weather and sheltering policies. Based on a review of the meeting minutes from

each school district’s board meetings for the year after the May 2013 tornado, there

is some evidence of policy learning among the neighboring school districts.

Policy Failure and Policy Learning 547

In general, building tornado shelters in schools or retrofitting buildings to be

storm resistant has occurred on a piecemeal basis because of the limited availability

of funding (Schwartz, 2013). Of the nine school districts in the area, five have no

evidence of school shelters and no evidence of consideration of shelters a year after

the storm. Among the four remaining, two districts—Mid-Del City and Western

Heights—have shelters in place, although the timing of when the shelters were

installed or constructed is unknown. Much like Moore Public Schools, Norman

Public Schools reviewed its sheltering procedures and weather policies. However, it

is unknown the extent to which Norman has sought to retrofit existing schools for

safe room or build shelters. Finally, Mustang Public Schools sought funding

through a school bond to fortify or add a storm shelter onto the construction of a

new gymnasium. The bond was passed in 2014 and the first storm shelter was put

in place in 2016.

At the state level, two competing pieces of legislation emerged that would sup-

port building shelters in schools. One bill would support a $500 million bond for

all schools, and the other bill would extend the ability of school districts to issue

bonds to raise funds to build the school shelters themselves (Norman Transcript,

2014). The bill introduced a state-wide bond and failed because state legislators

were reluctant to take on any more debt. In addition, a petition drive to amend the

Oklahoma constitution to grant the bond to fund school shelters failed to receive

enough signatures (Norman Transcript, 2013).

While the policy debate about whether and how the state should mandate school

districts to construct safe rooms or shelters was occurring, private fundraising

efforts were under way to support the safe rooms in schools. Shelter Oklahoma

Schools has reportedly raised over $1,000,000 to support the construction of safe

rooms in schools across Oklahoma. Indeed, the role of the private sector or private

groups in funding for public school shelters appears to be a preferred mechanism

because of perceptions about the ability of private groups to cut through bureau-

cratic red tape and build shelters more efficiently (Schwartz, 2013).

Joplin, Missouri

Joplin, Missouri is a city located in the southwest corner of the state, along the

Ozark Mountain region, and borders Arkansas, Kansas, and Oklahoma. On May

22, 2011, an EF-5 level tornado that was part of a series of late-spring storms

touched down in Joplin. The tornado traveled a 13-mile path through the western

portion of the city, leaving a 0.75-mile wide trough of damage (Onstot, 2015). The

Joplin tornado was the deadliest tornado in the United States since record keeping

began in 1950 (National Oceanic and Atmospheric Administration, 2012). It

resulted in 161 deaths and approximately 1,371 injured (Joplin City Health

Department, 2011) and resulted in an estimated $3 billion of damage. The May

2011 tornado was the third major tornado that had struck Joplin since 1971. Gen-

erally though, Joplin is a community that is familiar and experienced with torna-

does. Joplin experiences frequent tornadoes of very minor magnitude (Paul &

Stimers, 2012).

While most of the damage from the storm occurred in rural areas of Jasper and

Newton Counties, the effect of the tornado was still significant. It caused a fire at

548 Kristin O’Donovan

the St. John Regional Medical Center, and destroyed Joplin High School and a

technology education center. Commercial businesses were also affected by the tor-

nado. A Home Depot store was destroyed, resulting in 28 deaths. The deaths were

attributed to the construction method of the big box store, which met minimum

building code standards. This construction method is known as the “tilt-up” meth-

od where concrete wall panels are poured on the construction site and lifted into

place (McGraw, 2011).

There is prima facie evidence to suggest that the City of Joplin engaged in policy learning after the May 2011 tornado. The evidence indicates that most of the policy

learning was centered on warning systems, storm shelters, and building standards.

First, with respect to tornado warning systems, there was an existing tornado

warning system in place for Joplin at the time of the storm. The nature of the tor-

nado that struck Joplin made prediction and warning difficult because the funnel

cloud was surrounded in rain, which is unusual (Cohen, 2011). Following the

storm, the fire chief in Joplin requested a new warning system from the Joplin City

Council to improve the existing 25 sirens in the city. The city council ultimately

approved $270,000 to replace two sirens, upgrade 16 sirens, and increase back-up

activation capacity and solar power capabilities (City of Joplin, Joplin City Council,

2012).

Increasing the capabilities of the tornado warning system suggests that the City

of Joplin was adjusting the tools that they used to prepare for a tornado. System

upgrades, rather than replacements, suggest some degree of learning about the

effectiveness of the warning system. The quality of tornado warnings and the

extent to which people comply with the warnings are a critical way to reduce the

risk of casualties in a tornado (Daley et al., 2005). Moreover, the more lead-time

that a tornado warning can provide, the better able people will be to seek shelter

(Balluz, Schieve, Holmes, Kiezak, & Malilay, 2000; Paul & Stimers, 2012; Simmons

& Sutter, 2008). On the face, upgrades to the warning system suggest that the 2011

tornado revealed some degree of failure or weakness in the system and the ways in

which the City of Joplin could address it. Research indicates that 10% of people in

Joplin did not hear the tornado sirens and 23% of people failed to heed the warn-

ings (Paul & Stimers, 2012).

Second, with respect to storm shelters, after the tornado the City of Joplin and

its citizens were active in considering the shelter issue. Before the storm there were

no known community storm shelters or community safe rooms. Much like Moore,

this is not uncommon or unusual in tornado-prone areas. It is considered safer for

people to shelter-in-place than it is to venture out during a tornado warning where

conditions can be uncertain to seek shelter. In a survey where people took shelter

in Joplin during the tornado 45% of respondents used interior rooms or closets,

30% used a bathtub or bathroom, 13% used a basement, 6% used a storm shelter,

and 3% used a safe room (Paul & Stimers, 2012). In Joplin, 78% of houses lack a

basement because the basic geology and the risk of land subsidence from past min-

ing in the area (Paul & Stimers, 2012).

There is evidence that the City of Joplin engaged in policy learning with respect

to storm shelters. Two years after the storm, the City received 16 storm shelters

from FEMA by way of the State of Missouri for public use and another 16 that the

city would give to private citizens (Insurance Journal, 2013). Furthermore, the City

Policy Failure and Policy Learning 549

of Joplin created the Citizens Advisory Recovery Team (CART) by recommenda-

tion of FEMA to create a forum where “ideas could be brought to the table, consen-

sus formed and ideas and/or projects can be taken to the city council or other

appropriate organizations for consideration” (Citizens Advisory Recovery Team,

2011). One of the recommendations from CART’s report was to build safe rooms

in local schools as a part of disaster recovery. While the City of Joplin did not adopt

all of CART’s recommendations, there is evidence of an understanding of how the

implementation of Joplin’s sheltering policy needed to change. CART recom-

mended installing or building safe rooms for schools and other community spaces.

City officials stated they were not likely to require storm shelters be built in new

homes or businesses because they did not want to place added expenses on an eco-

nomically vulnerable population.

The Joplin High School and Franklin Technological Center were rebuilt to

include four staff and student safe rooms and one community safe room that could

be used during a tornado (Joplin Schools, 2011). Thirteen community safe rooms

have also opened inside Joplin schools since 2011. They are all designed to with-

stand 250-mph winds, as outlined in FEMA guidelines, and not only accommodate

students, faculty, and staff of each school but also citizens who live within a 5-min

walk and a half-mile drive. They each have an average capacity of 1,000–1,500 peo-

ple (Younker, 2014).

Third, there is some prima facia evidence to suggest that the City of Joplin engaged in some policy learning related to building codes after the May 2011 tor-

nado. Building codes in Missouri vary, but the structures in the so-called “Tornado

Alley” states are typically required to withstand 90-mph winds. While the City of

Joplin rejected requiring storm shelters to be built (Wooden, 2011), ultimately it

did adopt formal changes to the building code that required hurricane straps for

roofs and metal reinforcing clasps to make construction more storm resistant.2

Greensburg, Kansas

Greensburg is located in south central Kansas, approximately 110 miles west of

Wichita and had a population of approximately 1,400 when the tornado hit. The

tornado started at approximately 9:00 PM CDT on May 4, 2007, in Comanche

County and ended at 10:05 PM CDT outside Greensburg, reaching EF-5 status. It

was the strongest tornado of a larger outbreak of 25 total storms that affected vari-

ous areas across Oklahoma, Colorado, Kansas, and South Dakota. The National

Weather Service in Dodge City confirmed twelve tornado tracks by the Greensburg

supercell in Kansas alone on May 4.

The tornado was devastating, destroying 95% of Greensburg (U.S. Federal

Emergency Management Agency, 2007). The tornado’s path went through the

southern part of the city destroying or leveling residential areas and the entire

downtown. The tornado continued north of Greensburg and dissipated past city

limits. The size of the tornado was 28.2 miles long and 1.7 miles wide (U.S. Federal

Emergency Management Agency, 2007). Not only was the effect of the tornado

damaging to residential and commercial structures, but it also created a hazardous

chemical spill when a tanker car of a train was overturned and ten deaths resulted

from the May 4 tornado (U.S. Federal Emergency Management Agency, 2007).

550 Kristin O’Donovan

The death toll may have been higher if the National Weather Service had not sent

out advanced warnings 20 min before the storm hit (U.S. Federal Emergency Man-

agement Agency, 2007). Before the 2007 storm, the City of Greensburg did not

have a formal hazard mitigation plan in place and there were no formal mutual aid

agreements with neighboring communities to provide a coordinated response.

The City of Greensburg was highly committed to rebuilding after the tornado in a

sustainable and environmentally friendly way. The city council pledged to rebuild all

public buildings to LEED Platinum standards (Gertz, 2009). It gained commitments

from citizens and local government to rebuild as a “green” community (U.S. Federal

Emergency Management Agency, 2007). However, the extent to which the building

standards reflect learning about reducing tornado risk is not entirely clear.

In terms of the building code and its enforcement, the Greensburg City Council

voted to create a code enforcement official position within city government (Greens-

burg City Council, 2008). And while enforcement is an important aspect to building

codes that promote tornado risk mitigation, there is no prima facia evidence that

Greensburg made substantial changes to the building code. For example, the Greens-

burg City Council discussed rewriting ICB 2003 Section 104.9.1 (states that used

materials can be reused only if the building official approves) to be more specific on

types of materials that could be used to establish an objective standard (Greensburg

City Council, 2007a). This proposed change was discussed to make enforcement of

the building code easier but ultimately no action was taken (Greensburg City Council,

2007b). Federal officials told city administrators that they could only get funding to

rebuild at the same level as before the storm and any extra costs would not be covered.

This is not unusual for disaster recovery assistance where the dominant paradigm is to

return communities to their prestorm levels, not improve them. Nevertheless, it

appears the City of Greensburg had political support from the governor’s office, the

Kansas Senate, and the Kansas House of Representatives.

Analysis

At the outset, this study asked whether policy failure was a mechanism for policy

learning and whether policy failures initiate different types of learning. It argued

that treating all policy failures the same was too general of a tool to analyze that

question and put forth a typology of failures on which to focus the analysis. It out-

lined a theoretical framework of policy failure types and policy learning types for

tornado risk reduction. The remainder of this section analyzes the cases through

these two lenses with the goal of identifying important contextual factors that help

facilitate policy learning after a disaster.

How Is Policy Failure Associated with Policy Learning?

In the analytical framework, this paper theorized that policy failure is associated with

learning in two ways. First, that failures influence the policy tools available for deci-

sion makers to learn about or learn from. Second, policy failures create opportunities

for policy makers to learn by reassessing how well existing policies address the prob-

lems. The following analysis examines each case in the context of the framework.

Policy Failure and Policy Learning 551

The case of Moore presents the strongest example supporting the connection

between policy failure and policy learning. The failure of existing policies to reduce

risk and damage in Moore initiated learning about new tools to address the prob-

lem of tornado mitigation—specifically the building practices for commercial and

residential structures and sheltering policies. The Moore case highlights an exam-

ple of policy failure that resulted in very robust policy learning.

The failure also seemed to create opportunities for officials in Moore to reassess

how well tornado mitigation policies were working. As previously noted, Moore

had experienced similar disasters and received assessments from engineers and

academics about the nature of the risk and how to reduce risk in the past. The poli-

cy failures revealed by the 2014 tornado appear to be associated with learning

about policy tools and about the relative effectiveness of existing policies.

The case of Joplin supports the connection between policy failure and learning

about tools but not necessarily learning that by reassessment. Officials in Joplin

learned a significant amount about sheltering tools and policy options for building

and retrofitting public spaces for shelter. However, there is no evidence to support

that Joplin reevaluated their risk or policy tools to address the problem. In addi-

tion, the Joplin case provides an example of positive learning after a failure. Officials in Joplin learned that there were aspects of their preparation that worked reason-

ably well, like sirens. As a result, they chose to improve their warning systems.

The case of Greensburg presents a challenge for the connection between policy

failure and learning. As previously noted, policy failure occurs when a policy fails

to meet the desired outcomes of policy makers.

A policy failure in the form of decision making occurred in Greensburg. Based

on the definition presented earlier such a failure would be indicated by “failing to

decide on a policy within a reasonable period of time.” But as demonstrated by the

case study, there was no hazard mitigation plan in place in Greensburg before the

2007 tornado. The absence of mitigation planning implies that there was no prima facia evidence of a policy to address the risk of tornado damage in place. This sug- gests that the policy failure occurred because no decision was made. Public policy is

what government does or chooses not to do in response to a policy problem

(Birkland, 2014).

To answer the question posed at the outset, does failure initiate learning: the

answer is yes, but not necessarily policy learning that the theory suggests it would.

Moore and Joplin demonstrate support for a policy failure to initiate policy learning.

Furthermore, experience with a problem is an important characteristic in connecting

policy failure leading to subsequent policy learning. Both Moore and Joplin had expe-

rience with similar tornadoes and with less intense, more frequent tornadoes. Over

time, each community had considered taking steps to mitigate the risks of tornado

impact and damage. The experience connection implies that there is an implicit

threshold of experience with a policy problem and knowledge of existing ways to

address it connecting failure of a public policy and subsequent policy learning.

By contrast, because Greensburg had little experience with similar tornadoes

and had no tangible mitigation plan in place it presents a challenge to the classic

idea of a connection between failure and learning. It is clear that there was a policy

failure and some form of learning occurred. However, the learning was misdir-

ected toward policy ideas, tools, and instruments that were not intended to reduce

552 Kristin O’Donovan

risk. So, yes, learning occurred but not necessarily in a way that produced knowl-

edge about how to address the problem revealed by the failure.

In addition, the Greensburg case presents a challenge of comparability. While

the tornado and its damage was similar to that experienced by Moore and Joplin,

the governmental capacity and resources were very different. For policy learning

that is focused on the problem revealed by a failure, there appears to be some mini-

mum level of governmental capacity and resources. For example, before the torna-

do, the City of Greensburg did not have an employee position in the government

for a building inspector in contrast with Moore and Joplin that had dedicated staffs

for buildings, development, and city management (see Table 3).

Does Policy Failure Initiate Different Types of Learning?

In the analytical framework, this paper theorized that different types of policy failure

would initiate different kinds of policy learning. First, programmatic failures, failures

associated with agenda setting and formulation, would initiate instrumental policy learn-

ing. Second, failure to make policy decisions, a failure of implementation or evaluation,

would lead to social policy learning. Third, it theorized that specific kinds of failure would

not be associated with political learning alone. Instead programmatic or policy process

failures might couple political learning with other types of policy learning.

There were two different types of policy failures that were present in case stud-

ies: a failure in agenda setting and a failure in policy decision making. A policy fail-

ure of agenda setting as applied to tornado risk mitigation, as previously noted,

would be either an overly ambitious policy to eliminate the risk of tornado damage

or overly modest policies to only marginally reduce the risk of tornado damage.

Joplin had experience preparing, responding, and recovering from tornadoes in a

range of scales from nuisance tornadoes to larger scale disasters. However, the impor-

tant difference is that unlike Moore, Joplin was not presented with information about

how to reduce the risk in the future for a similar storm. In this case, the policy failure in

Joplin is a failure of agenda setting. The absence of tornado risk mitigation on the local

policy agenda represents a policy failure, particularly in light of the ongoing experience

Table 3. Types of Policy Failure by Case

Moore Joplin Greensburg

Failure of policy

program

No failure No failure No failure

Failure of policy process

Agenda setting No failure Frequent experience with

tornadoes but failure to

place mitigation on the agenda

No failure

Policy formulation No failure No failure No failure

Decision making Lack of a decision to

mitigate or reduce tornado risk after 1999

tornado when presented

with evidence

No evidence Lack of a decision to

engage in mitigation planning before

the tornado

Policy implementation No failure No failure No failure Policy evaluation No failure No failure No failure

Policy Failure and Policy Learning 553

with tornadoes in that community. Otherwise, Joplin is similar to Moore in that it has

approximately the same population and is part of a larger metropolitan area, St. Louis.

The subsequent policy learning that occurred in Joplin was the “thinner” form

of learning that resulted in reevaluation of tools that were already available. In addi-

tion, the learning that took place in Joplin was grounded in confirmatory informa-

tion of what worked well—sirens and shelters. The significance of the association of

failures in agenda setting leading to instrumental policy learning confirms what

scholars have long argued about the connection of governmental reaction to a poli-

cy failure. There is an overwhelming desire to learn something, do something in

response to the failure or crisis and policy makers who have been reluctant to give

mitigation space or attention on the agenda are compelled to take action to address

the problem. Instrumental policy learning provides such an opportunity to quickly

address the problem in ways that are either easily accessible or have some confirma-

tory evidence of their relative effectiveness. The important contribution from this

analysis is that failure to pay attention to a problem, to place it on the agenda or to

do the bare minimum means that the learning that occurs is of expedience. The

problem will continue to garner limited policy attention and agenda space once the

time of failure and crisis have passed and the tools from instrumental learning have

been put in place. So, the failure may repeat itself in the future (see Table 4).

The other type of policy failure in the case analysis is the failure of decision mak-

ing. However, the types of learning that occurred as a failure of decision making

varied dramatically. Both Moore and Greensburg failed to make a policy decision

to act within a reasonable amount of time. In 1999, Moore had experienced a tor-

nado similar to the 2013 storm and had been presented with similar evidence about

how to reduce risk in the future. Officials in the City of Moore failed to make a

decision in a timely way after the 1999 storm.

As a result of the 2013 tornado Moore’s failure of policy decision making

resulted in robust learning about policy instruments, about the underlying causes

of the failure, and the political tactics to translate social policy learning into

Table 4. Evidence of Learning by Policy Learning Type

Case

Instrumental Policy

Learning Social Policy Learning Political Learning

Misdirected

Learning

Moore Review of the sheltering

procedures in place in school

districts. Reassessment

of the policy tools

for sheltering, warning, and sirens

Building code

changes. Underlying

rationale that

storms of similar

severity will be increasing

State-level learning.

Public opinion support for state action with

unsuccessful tactics

—

Joplin Learning about changes in

building codes, storm

shelters, and warning systems

No learning No direct evidence.

CART mission sug-

gests stakeholder engagement for polit-

ical support

—

Greensburg No learning No learning No learning Learning about

sustainable and green building standards

for recovery

554 Kristin O’Donovan

significant policy change. The failure of decision making after previous tornados

created the circumstances for deeper learning about the fundamental cause of the

problems that caused the damage, including the perceptions about risk. But the

coupling with political learning was necessary to push through policy change. This

is important because it signals that certain types of policy failure are linked to learn-

ing about tactics and approaches to transform learning into policy change.

In Greensburg, the policy failure occurred when the city opted not to make a deci-

sion to reduce the risk of tornado damage; the community did not have a hazard

mitigation plan, the most elementary effort of managing risk. The policy failure of

Greensburg is evident in its failure to make any decision about tornado risk reduc-

tion. The failure was to make a decision to have a policy and to seriously consider

how to handle the risk. It raises the question—even if only a nominal policy existed,

can it fail? Much of the foundational public policy literature asserts that public policy

is also made when government fails to act. Can we call it a policy failure or is it a gov-

ernance failure? The case of Greensburg suggests the possibility of broader contextu-

al factors, such as the lack of capacity and experience in the city government.

Greensburg did learn after the tornado, but it did not learn about what might be

expected. Local officials learned a great deal about sustainable building standards.

However, they did not learn about how to reduce the impact of a tornado or lessen

damage caused by tornadoes. Much of the policy learning literature normatively

assumes that when there is evidence that policies fail, there will be some attempt to

improve the policy. That does not seem to be the case in Greensburg. As such, this

study introduces a type of learning, which could be referred to as misdirected

learning. Misdirected learning is policy learning that occurs but does not address

the policy problem. One case in an empirical analysis is insufficient to make gener-

alizations about the phenomenon about misdirected learning and its connection to

failures of decision making. However, asserting that no learning occurred in after

failure in Greensburg would ignore evidence that points in an unexpected

direction.

Conclusions

At the outset, this study asked two questions. First, is policy failure associated with

policy learning? The answer, based on this analysis is yes, but not necessarily always

in ways that address the policy problem that the failure revealed. The important

characteristics for linking policy failure with learning in this study are experience

with a problem and some degree of governmental capacity to engage in classic

types of policy learning. In the absence of experience and capacity, policy failure is

associated with misdirected policy learning, or learning that does not address the

problem exposed by the failure. The importance of experience and capacity imply

that there is an implicit threshold for a policy failure to initiate instrumental learn-

ing and social policy learning.

Second, if policy is associated with policy learning, does the failure initiate differ-

ent types of learning? On this point, there are two types of failures associated with

the types of learning. First, failures of agenda setting—specifically in carving out too

modest policy goals—is associated with instrumental policy learning. Because of the

Policy Failure and Policy Learning 555

limited agenda space and attention, the learning that occurs after failure would be

characterized by the urge to “do something” as quickly as possible and a search of

policy instruments resulting in the most expedient choice.

Second, failure of decision making is associated with different kinds of learning.

This study finds that failures of decision making that stem from differences in the

failure to make a timely decision appear to be associated with different type of learning compared to failure to make any policy decision. Failure to make a timely decision is associated with robust learning about the underlying cause of the problem, the

tools needed to address the problem, and the political tactics to change policy.

However, failure to make any policy decision is associated with learning that is

misdirected.

Future study is necessary to investigate the threshold of experience and capacity

that links policy failure with learning. In addition, future research should aim to

help us gain a better understanding of how other policy failure types, such as fail-

ures of implementation and evaluation, may be associated with policy learning.

Notes

1 The list of policy responses available to reduce the damage caused by a tornado has been adapted

from the following: “Tornado Protection: Selecting Refuge Areas in Buildings,” FEMA P-431, Sec-

ond Edition/October 2009. U.S. Department of Homeland Security, Federal Emergency Manage-

ment Agency; “Protect Your Property from High Winds,” U.S. Department of Homeland Security,

Federal Emergency Management Agency; and “Thunderstorms, Tornadoes, and Lightning:

Nature’s Most Violent Storms: A Preparedness Guide,” U.S. Department of Commerce, National

Oceanic and Atmospheric Administration, National Weather Service.

2 Sections R403.1.6 & R404.1.1 of the Joplin, Missouri building code.

About the Author

Kristin O’Donovan is an Assistant Professor in the Department of Political Science at Wayne State University. Her research focuses on the policy process, policy learning after disaster and crisis, and the politics of disasters.

References

Adcock, C. (2013). Moore shut out of prevention aid. Oklahoma Watch. Retrieved from http://oklahomawatch.

org/2014/07/20/moore-other-damaged-areas-fail-to-tap-disaster-prevention-aid/

Agranoff, R., & Radin, B. A. (1991). The comparative case study approach in public administration. Research

in Public Administration, 1(1), 203–231.

Balluz, L., Schieve, L., Holmes, T., Kiezak, S., & Malilay, J. (2000). Predictors for people’s response to a tor-

nado warning, Arkansas, 01 March, 1997. Prehospital and Disaster Medicine, 15(S2), S89–S89.

Birkland, T. A. (1997). After disaster: Agenda setting, public policy, and focusing events. Washington, DC: George-

town University Press.

Birkland, T. A. (1998). Focusing events, mobilization, and agenda setting. Journal of Public Policy, 18(1),

53–74.

Birkland, T. A. (2004a). Learning and policy improvement after disaster: The case of aviation security.

American Behavioral Scientist, 48(3), 341–364.

Birkland, T. A. (2004b). The world changed today: Agenda-setting and policy change in the wake of the

September 11 terrorist attacks. Review of Policy Research, 21(2), 179–200.

Birkland, T. A. (2006). Lessons of disaster: Policy change after catastrophic events. Washington, DC: Georgetown

University Press.

Birkland, T. A. (2014). An introduction to the policy process: Theories, concepts and models of public policy making.

London: Routledge.

556 Kristin O’Donovan

Bovens, M. (2010). A comment on Marsh and McConnell: Towards a framework for establishing policy suc-

cess. Public Administration, 88(2), 584–585.

Bovens, M., & ‘t Hart, P. (1995). Frame multiplicity and policy fiascoes: Limits to explanation. Knowledge and

Policy, 8(4), 61–82.

Bovens, M., & ‘t Hart, P. (2016). Revisiting the study of policy failures. Journal of European Public Policy,

23(5), 653–666.

Citizens Advisory Recovery Team. (2011). Report of the Citizens Adviosry Recovery Team: Listening to Jop-

lin. Retrieved from http://joplinareacart.com/wp-content/uploads/sites/26/2011/11/Listening-to-Joplin-

Report-to-City-Council-FINAL.pdf

City of Joplin, Joplin City Council. (2012). Joplin City Council meeting agenda. Retrieved from http://www.

joplinmo.org/AgendaCenter/ViewFile/Agenda/170

City of Moore. (2014). City adopts new building codes, first in the nation. Retrieved from http://www.cityof-

moore.com/node/2111

Cohen, N. (2011). Tornado hits Missouri City, killing many. The New York Times. Retrieved from http://nyti.

ms/1C4CssY.

Daley, W. R., Brown, S., Archer, P., Kruger, E., Jordan, F., Batts, D., & Mallonee, S. (2005). Risk of tornado-

related death and injury in Oklahoma, May 3, 1999. American Journal of Epidemiology, 161(12), 1144–

1150.

Dunlop, C. A. (2009). Policy transfer as learning: Capturing variation in what decision-makers learn from

epistemic communities. Policy Studies, 30(3), 289–311.

Dunlop, C. A., & Radaelli, C. M. (2013). Systematising policy learning: From monolith to dimensions. Politi-

cal Studies, 61(3), 599–619.

Dunlop, C. A., & Radaelli, C. M. (2016). Policy learning in the Eurozone crisis: Modes, power and function-

ality. Policy Sciences, 49(2), 107–124.

Gertz, E. (2009, June 12). Putting the green in Greensburg: Tornado-ravaged town reinvents itself. Scientific

American. Retrieved from http://www.scientificamerican.com/article/greensburg-green-revolution/

Greensburg City Council. (2007a, December 17). Greensburg City Council meeting minutes. Retrieved from https://

www.greensburgks.org/government/agendas-minutes/2007-agendas-minutes/Dec.%2017%2C%202007.doc/view

Greensburg City Council. (2007b, October 15). Greensburg City Council meeting minutes. Retrieved from

https://www.greensburgks.org/government/agendas-minutes/2007-agendas-minutes/Oct.%2015%2C%202007%

20Minutes.doc/view

Greensburg City Council. (2008, February 18). Greensburg City Council meeting minutes. Retrieved from

https://www.greensburgks.org/government/agendas-minutes/2008-agendas-minutes/Feb.%2018%2C%

202008%20Minutes.doc/view

Hindmoor, A., & McConnell, A. (2013). Why didn’t they see it coming? Warning signs, acceptable risks and

the global financial crisis. Political Studies, 61(3), 543–560.

Horning, C. (2013). Deadly tornado nearly follows path of ’99 storm. The Norman Transcript. Retrieved from

http://www.mooreamerican.com/news/local_news/deadly-tornado-nearly-follows-path-of-storm/article_

87051c7e-b6df-5a94-937a-415760cd17c0.html

Howlett, M. (2012). The lessons of failure: Learning and blame avoidance in public policy-making. Interna-

tional Political Science Review, 33(5), 539–555.

Howlett, M., Ramesh, M., & Wu, X. (2015). Understanding the persistence of policy failures: The role of

politics, governance and uncertainty. Public Policy and Administration, 30(3-4), 209–220.

Insurance Journal. (2013, August 8). Joplin, Missouri to receive 32 storm shelters. Retrieved from http://

www.insurancejournal.com/news/midwest/2013/08/08/301218.htm

Joplin City Health Department. (2011). Annual report. Retrieved from http://joplinmo.org/documentcenter/

view/3102

Joplin Schools. (2011, May 22). Tornado recovery: Information and helpful resources. Retrieved from

http://www.joplinschools.org/Domain/60

Kingdon, J. W. (2003). Agendas, alternatives, and public policies (2nd ed.). New York: Longman.

Kunreuther, H. (1996). Mitigating disaster losses through insurance. Journal of Risk and Uncertainty, 12(2-3),

171–187.

Marsh, D., & McConnell, A. (2010). Towards a framework for establishing policy success: A reply to Bovens.

Public Administration, 88(2), 586–587.

Marshall, T. P. (2002). Tornado damage survey at Moore, Oklahoma. Weather and Forecasting, 17(3), 582–598.

May, P. J. (1992). Policy learning and failure. Journal of Public Policy, 12(4), 331–354.

May, P. J. (2015). Implementation failures revisited: Policy regime perspectives. Public Policy and Administra-

tion, 30(3–4), 277–299.

May, P. J., & Deyle, R. E. (1998). Governing land use in hazardous areas with a patchwork system. Washington DC:

Joseph Henry Press.

Policy Failure and Policy Learning 557

McConnell, A. (2010). Understanding policy success: Rethinking public policy. New York: Palgrave Macmillan.

McConnell, A. (2015). What is policy failure? A primer to help navigate the maze. Public Policy and Adminis- tration, 30(3–4), 221–242.

McConnell, A. (2016). A public policy approach to understanding the nature and causes of foreign policy failure. Journal of European Public Policy, 23(5), 667–684.

Mileti, D. (1999). Disasters by design: A reassessment of natural hazards in the United States. Washington, DC: Joseph Henry Press.

National Oceanic and Atmospheric Administration. (2012, March 20). 2011 Tornado information. Retrieved from http://www.noaanews.noaa.gov/2011_tornado_information.html

Norman Transcript. (2013, August 13). School shelter drive looking for more signatures. Retrieved from http://www.normantranscript.com/opinion/school-shelter-drive-looking-for-signatures/article_1b8eb1c3- 814c-5b9d-8d46-07a0ca1e8214.html

Norman Transcript. (2014, February 1). Competing plans could dominate session. Retrieved from http://www. normantranscript.com/opinion/competing-shelter-plans-could-dominate-session/article_fb462477-3f6d-532a- 8900-1b13e33109c6.html

Onstot, L. I. (2015). Joplin, Missouri hit by EF-5 Tornado on May 22, 2011. Joplin, MO: Public Information Office.

Paul, B. K., & Stimers, M. (2012). Exploring probable reasons for record fatalities: The case of 2011 Joplin, Missouri, tornado. Natural Hazards, 64(2), 1511–1526.

Prater, C. S., & Lindell, M. K. (2000). Politics of hazard mitigation. Natural Hazards Review, 1(2), 73–82.

Ramseyer, C., & Holliday, L. (2014). City of Moore: New building code for tornado reslience. Retrieved from http://www.cityofmoore.com/sites/default/files/main-site/City%20of%20Moore%20Code%20Adop- tion%20Feb%202014%20-%20Web%20reduced.pdf

Rose, R. (1991). What is lesson-drawing? Journal of Public Policy, 11(1), 3–30.

Schwartz, J. (2013, March 21). Why no safe room to run to? Cost and plains culture. The New York Times. Retrieved from http://www.nytimes.com/2013/05/22/us/shelter-requirements-resisted-in-tornado-alley. html?pagewanted5all

Simmons, K. M., & Sutter, D. (2008). Tornado warnings, lead times, and tornado casualties: An empirical investigation. Weather and Forecasting, 23(2), 246–258.

Tierney, K., Bevc, C., & Kuligowski, E. (2006). Metaphors matter: Disaster myths, media frames, and their consequences in Hurricane Katrina. The Annals of the American Academy of Political and Social Science, 604(1), 57–81.

U.S. Federal Emergency Management Agency (2007, October 27). Final report: Tornado damage investiga- tion Greensburg, Kanasa. 1699 DR-KS. Retrieved from http://www.fema.gov/media-library-data/ 20130726-1646-20490-3544/greenburg_ks_tornado_damage.pdf

Vince, J. (2015). Integrated policy approaches and policy failure: The case of Australia’s Oceans Policy. Policy Sciences, 48(2), 159–180.

Walsh, J. (2006). Policy failure and policy change: British security policy after the Cold War. Comparative Political Studies, 39(4), 490–518.

Wooden, D. (2011, June 7). Council endorses building code changes. The Joplin Globe. Retrieved from http:// www.joplinglobe.com/news/local_news/council-endorses-building-code-changes/article_5b2dfb1f-f034- 5835-a4ac-c6d470f02784.html

Yin, R. K. (2013). Case study research: Design and methods. Los Angeles: Sage Publications.

Younker, E. (2014, April 17). Joplin school district readies community safe rooms for storm season. The Jop- lin Globe. Retrieved from http://www.joplinglobe.com/news/local_news/joplin-school-district-readies-com- munity-safe-rooms-for-storm-season/article_fc6511df-283a-57db-bb9f-9b89d40f987a.html

Zahariadis, N. (2003). Ambiguity and choice in public policy: Political decision making in modern democracies. Wash- ington, DC: Georgetown University Press.

Zahariadis, N. (2014). Ambiguity and multiple streams. In P. Sabatier & C. Weible (Eds.), Theories of the policy process (3rd ed., pp. 25–58). Boulder, CO: Westview Press.

558 Kristin O’Donovan

Copyright of Review of Policy Research is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.