Reading I NEED HELP ON MY HOMEWORK

How the Public Defines Terrorism

Author(s): Connor Huff and Joshua D. Kertzer

Source: American Journal of Political Science , JANUARY 2018, Vol. 62, No. 1 (JANUARY 2018), pp. 55-71

Published by: Midwest Political Science Association

Stable URL: https://www.jstor.org/stable/26598750

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at https://about.jstor.org/terms

Midwest Political Science Association is collaborating with JSTOR to digitize, preserve and extend access to American Journal of Political Science

This content downloaded from �������������24.254.82.142 on Mon, 20 Dec 2021 21:26:56 UTC�������������

All use subject to https://about.jstor.org/terms

How the Public Defines Terrorism 00

Connor Huff Harvard University Joshua D. Kertzef Harvard University

Abstract: Every time a major violent act takes place in the United States, a public debate erupts as to whether it should be

considered terrorism. Political scientists have offered a variety of conceptual frameworks, but have neglected to explore how

ordinary citizens understand terrorism, despite the central role the public plays in our understanding of the relationship

between terrorism and government action in the wake of violence. We synthesize components of both scholarly definitions

and public debates to formulate predictions for how various attributes of incidents affect the likelihood they are perceived as

terrorism. Combining a conjoint experiment with machine learning techniques and automated content analysis of media

coverage, we show the importance not only of the type and severity of violence, but also the attributed motivation for the

incident and social categorization of the actor. The findings demonstrate how the language used to describe violent incidents,

for which the media has considerable latitude, affects the likelihood the public classifies incidents as terrorism.

Replication Materials: The data, code, and any additional materials required to replicate all analyses in this arti cle are available on the American Journal of Political Science Dataverse within the Harvard Dataverse Network, at

https://doi.org/! 0.7910/DVN/LD6NL8.

Following the mass shooting in Charleston, South Carolina, there was widespread debate throughout

the United States about whether to classify the vi

olent incident as terrorism. FBI Director James Comey offered a negative assessment, noting that "terrorism is

[an] act of violence ... to try to influence a public body or citizenry, so it's more of a political act. ... [B]ased on

what I know so far I don't see [Charleston] as a political act" (Hattem 2015). His assessment was condemned from

across the political spectrum, from critics on both the left (including then Democratic presidential candidate Hillary Clinton) and right (including then GOP presi dential candidate Rick Santorum). Similar public debates

erupted following other violent incidents, including the

bombing at the Boston Marathon and shooting in Or lando, Florida. These debates highlight not only the con

tentiousness of classifying terrorism, but also the stakes

involved in doing so, for policy makers, academics, and members of the public alike.

In this article, we turn to experimental methods to

explain the tenor of these public debates. We investigate

terrorism in a public opinion context not because we believe that the mass public's intuitions can necessarily resolve normative debates about what should or should

not be considered terrorism, but rather because of the

central role that public opinion plays in our understand

ing of how terrorism works. In a vast array of prior re search, terrorism is understood as a form of violence that

functions by attracting public attention. It is because ter

rorism hinges on public reaction that Margaret Thatcher

suggested terrorists depend on "the oxygen of publicity,"

that Carlo Pisacane declared terrorism to be "propaganda

by deed," and that Ayman al-Zawahiri suggested that for

al-Qaeda, media coverage is "more than half' the battle

(Smith and Walsh 2013,312). If the responses of ordinary

citizens constitute a central causal mechanism through which terrorism operates, it logically follows that un derstanding what ordinary citizens think terrorism is is a

crucial prerequisite to understanding how they react to it.

Connor Huff is Ph.D. Candidate, Department of Government, Harvard University, 1737 Cambridge Street, Cambridge, MA 02138 ([email protected]). Joshua D. Kertzer is Assistant Professor, Department of Government, Harvard University, 1737 Cambridge Street, K206, Cambridge, MA 02138 ([email protected]).

We thank Bob Bates, Mia Bloom, Alex Braithwaite, Dara Kay Cohen, Naoki Egami, Marcus Holmes, Jeff Kaplow, In Song Kim, Melia Pfannenstiel, Peter Vining, Ariel White, Thomas Zeitzoff, and audiences at the Political Violence Workshop at Harvard University, ISA 2016, MPSA 2016, Trinity College, Dublin, and William and Mary for helpful comments, suggestions, and assistance. Kertzer acknowledges the support of the Weatherhead Center for International Affairs, Institute for Quantitative Social Science, and Niehaus Center for Globalization and Governance; thanks to Perry Abdulkadir, Michael Chen, and Aaron Miller for fantastic research assistance. Authors' names are in alphabetical order.

American Journal of Political Science, Vol. 62, No. 1, January 2018, Pp. 55-71

©2017, Midwest Political Science Association DOI: 10.111 l/ajps.12329

55

This content downloaded from �������������24.254.82.142 on Mon, 20 Dec 2021 21:26:56 UTC�������������

All use subject to https://about.jstor.org/terms

56

Employing a conjoint experiment on 1,400 American

adults, we show how ordinary citizens classify terrorism

based not only on relatively objective facts on the ground,

but also on fairly subjective considerations about the per

petrator. On the one hand, considerations about the type

and severity of violence matter—though interestingly, not

the distinction between civilian and military targets that

plays a central role in contemporary legal definitions. On

the other hand, our respondents are also heavily influ enced by descriptions about the perpetrator's identity and motivations, considerations about which the media has

considerable latitude in the language it uses to cover inci

dents. In this sense, because actions do not always speak

louder than words, the media has considerable power based on how it chooses to frame incidents: using predic

tive models derived from our experimental results shows

how the likelihood Americans will classify incidents with the characteristics of the San Bernadino attacks, for ex

ample, can vary from around 31% to 82%, depending on

the narrative constructed about the perpetrators' identity

and motivations. In so doing, we empirically demonstrate

the ways in which terrorism can be socially constructed.

The discussion that follows has four parts. First, we

discuss the important role that public opinion plays in our

understanding of how terrorism works, and we present

a simple typology of factors that either loom large in formal terrorism classification schemes or in popular de

bates. Second, we discuss the design of our experiment. Third, we present our main results, whose implications we highlight using machine learning techniques to con struct counterfactual simulations of real-world incidents,

and automated content analyses of media coverage. We conclude by discussing the implications of our findings for the study of terrorism more generally.

The Stakes of Defining Terrorism

Whenever a major violent action takes place in the United

States, a public debate erupts as to whether it should be classified as terrorism or not, mirroring a similarly

contentious debate in the study of political violence as to

how terrorism should be defined in the first place. The contours of these debates matter for three rea

sons. First, classifying actions as terrorism has direct pol

icy implications for how the perpetrators are prosecuted: in the United States, for example, terrorism is a federal

charge, prosecuted in federal courts, whereas most violent

crime is handled by the states. Similarly, transnational

organizations deemed to be terrorist entities are sub jected to harsh financial sanctions, foreign nationals on

CONNOR HUFFAND JOSHUA D. KERTZER

terrorist watch lists are not allowed to enter the country,

and Americans are prohibited from dealing with them. The consequences of acts being classified as terrorism are

real and immediate. More generally, defining terrorism is

crucial to fighting it: the United Nations has been unable

to move forward with a comprehensive treaty against ter

rorism precisely because of the inability of its member

states to come to a mutually acceptable definition of what

terrorism is in the first place.

Second are important normative implications. As a discursive category, terrorism is understood as qualita tively different from other types of acts of violence: an ex

tranormal or exceptional act, mandating an exceptional response (Agamben 2005). In the 2004 election cam paign, when John Kerry pronounced that terrorism was

ultimately a law enforcement issue, the George W. Bush

campaign lampooned him in television advertisements for failing to take the threat seriously (Kertzer 2007,964

65). Public debates following the shooting in Orlando in June 2016 displayed the same tendency. To categorize

something as terrorism is to delegitimate its goals; ter rorism is not merely a problem to be managed, but one

to be destroyed; terrorists are to be hunted, rather than

negotiated with (Hodges 2011). These normative ques tions are further exacerbated by complaints about the double standards with which the terrorist label is applied.

When a group of armed white ranchers seized the head quarters of a U.S. Fish and Wildlife Service complex in Oregon in January 2016 to protest the federal govern ment's policies on grazing and land rights, left-leaning commentators condemned the refusal of the media to la

bel the group as terrorists, and bemoaned the seeming leniency the group was being given by law enforcement:

"If Muslims had seized a federal building, they'd all be dead by now," the commentator Cenk Uygur wrote, and

hashtags like #YallQaeda, #YokelHaram, and #VanillaISIS

circulated on Twitter (Uygur 2016). Scholars have sim ilarly noticed a double standard, in which the media is more likely to adopt an Islamic terror frame when the

perpetrator is Muslim, and more likely to explore the at

tacker's personal life and mental health if the perpetrator

is not (Powell 2011).

Third, these debates also have important stakes for

political scientists. Almost every book or review arti cle on terrorism we are aware of includes a compulsory

line acknowledging the contentiousness of its definitional

politics, and there are a variety of scholarly classifica tion schemes that political scientists regularly employ.1

'See, for example, Crenshaw (2000), Pape (2003), Horgan (2004), Goodwin (2006), Hoffman (2006), Kydd and Walter (2006), Merolla and Zechmeister (2009), Enders and Sandler (2011),

This content downloaded from �������������24.254.82.142 on Mon, 20 Dec 2021 21:26:56 UTC�������������

All use subject to https://about.jstor.org/terms

HOW THE PUBLIC DEFINES TERRORISM

Indeed, a number of political scientists have used these typologies to constructively wade into public debates (Phillips 2015; Young 2012). Just as our understanding of the dynamics of civil wars changes when we adopt dif

ferent operational definitions of the construct, our rules of thumb for how we know terrorism when we see it have

important implications for which actions get turned into rows in our data sets and, thus, our fundamental under

standing of how terrorism works.

In this article, we shift the focus from how gov ernments, the media, and academics classify terrorism to instead explore how members of the mass public understand the term, as part of a growing body of re search throughout the social sciences, using experimental

methods to unpack our folk intuitions about political concepts.2 We investigate terrorism in a public opinion context not because we believe that folk intuitions can

resolve normative debates about what should and should

not be considered terrorism—although disconnects be tween the judgments of citizens and the laws that govern

them are indeed worthy of study (Audi 2009)—but rather

because of the central role that public opinion plays in our understanding of how terrorism works. Most of our

academic models of terrorism emphasize "victim-target differentiation," or the notion that terrorism is a commu

nicative act ("propaganda by deed") directed at broader audiences (Asal and Hoffman 2015; Braithwaite 2013;

Fortna 2015). In Bueno de Mesquita and Dickson (2007), terrorists carry out attacks in order to mobilize public support among the population they claim to represent; in

Enders and Sandler (2011), terrorists seek to change gov

ernment policy through public pressure; and in Kydd and

Walter's (2006) model of terrorism as costly signaling, three out of the five terrorist strategies they describe are

directed at persuading the public. The public's reactions are frequently posited to explain why terrorists adopt tactics like suicide bombing that tend to receive more publicity (Bloom 2005; O'Rourke 2009), and why democ racies are more likely to be targeted by terrorist attacks

(Eubank and Weinberg 1994; Pape 2003; Stanton 2013). Unlike many foreign policy issues, terrorism is highly

salient, capturing the public imagination—and produc ing downstream political consequences—to an extent that

more ubiquitous forms of violence do not.3 It receives ex

Phillips (2014), Fortna (2015), Horowitz (2015), Huff and Kruszewska (2016).

2See, for example, Malle and Knobe (1997), Kertzer and McGraw (2012), and Kertzer, Renshon, and Yarhi-Milo (2016).

3For example, a December 2015 Public Religion Research Institute poll found that 84% of Republicans and 70% of Democrats saw terrorism as a "critical issue" facing the country, as opposed to

57

tensive media coverage (Nacos, Bloch-Elkon, and Shapiro

2011) and fuels powerful emotional responses that not only affect the public's attitudes toward foreign policy is sues (Albertson and Gadarian 2015; Huddy et al. 2005), but also how citizens act towards each other (Merolla and Zechmeister 2009). Politicians are aware of terror

ism's sway on the popular imagination, which is one rea

son why Democratic lawmakers use terrorism frames to

pressure their Republican counterparts into supporting

gun control measures (Cowan and Cornwell 2016). Out side the United States, political scientists have found that

terrorist attacks increase support for right-wing parties

in the locale where the attacks took place (Berrebi and Klor 2008), diminish willingness to negotiate (Huff and Kruszewska 2016), and sway elections (Bali 2007). Even the mere threat of terrorism is enough to change vot ing behavior (Getmansky and Zeitzoff 2014). Given this

wealth of prior research in which public reactions play an important role as a causal mechanism linking violent

incidents to downstream political consequences, it is cru

cial for political scientists to understand what the public

thinks it is reacting to.

A Typology for Classifying Terrorism

In this section, we present a typology for classifying terror

ism, integrating common components of formal classifi

cation schemes with the contentious elements of public

debates that erupt in the wake of violent incidents. The

typology is composed of two broad components. The first consists of relatively objective facts on the ground:

information about the type and severity of the violence

employed, and the incident's target and location. This information is often available immediately after the inci dent occurs. In contrast, the second consists of informa

tion pertaining not to "what" questions, but "who" and "why," concerning the identity of the perpetrators and

motivation attributed to them. This category thus con sists of considerations that are often relatively subjective,

are unavailable until days or weeks after an incident oc

curs, and give the media more leeway in how they frame events. We discuss each element in turn.

Objective Criteria: The Facts on the Ground

The first component focuses on the fundamental descrip

tive characteristics of the action. Scholarly typologies and

merely "one among many important issues." In contrast, the same poll found that although 74% of Democrats called mass shootings a critical issue, only 60% of Republicans did (Cox and Jones 2015).

This content downloaded from �������������24.254.82.142 on Mon, 20 Dec 2021 21:26:56 UTC�������������

All use subject to https://about.jstor.org/terms

58

definitions of terrorism consistently emphasize two types

of factors when discussing terrorist incidents. The first is

violence, mentions of which appear in almost every legal

and academic definition alike (Weinberg, Pedahzur, and Hirsch-Hoefler 2004). In the discussion below, we sug gest violent incidents can be understood along two key dimensions: the type of tactic used, and the severity of

the consequences as measured by casualties. The second component is information about whom the violence was

against, conveyed through either the type of target or loca

tion of the incident. We discuss each component in turn.

Type and Severity of Violence. We argue that the violence observed in incidents can be understood along two dimensions. First, we can think of different "tech

nologies" (Kalyvas and Balcells 2010), or "repertoires of contention" (Tilly 1986), characterized by the particular

tactics employed in an incident. For example, bombing is

a different type of violence than shooting or hostage tak

ing. Following the Boston Marathon Bombing in 2013, President Barack Obama stated that "any time bombs are used to target innocent civilians, it is an act of terror"

(Landler 2013). This statement is emblematic of the close

mapping in both academic and policy circles between the

type of violence used and the likelihood the incident will

be classified as terrorism, buttressed by recent work in

political science on the role that different types of violent

tactics play in structuring perceptions of actors ( Abrahms

2013; Huff and Kruszewska 2016). The empirical impli cations of this prior research are twofold. First, violent

tactics should be more likely to be classified as terrorism

than nonviolent tactics. Second, bombings are a unique form of violence often associated with terrorism (Bloom

2005), such that we expect them to be more likely to be classified as terrorism than other forms of violence.4

In addition to characterizing violence by its type, it can also be characterized by its severity. Indeed, the

magnitude of the carnage imposed by an attack is often the central focus of the media following violent incidents,

with headlines typically bringing the number of casualties

to the fore. This leads to a second empirical implication:

the higher the number of casualties, the more likely the incident will be classified as terrorism.

The Target and Location. We can characterize the target of violent incidents in two ways. First is whether

the target is a civilian or noncombatant (Coady 2004; Rodin 2004). Consistent with the principle of distinction

4This could either be due to the frequency with which the pub lic views bombings as being conducted by organizations associ ated with terrorism, or due to their often indiscriminate nature

(Goodwin 2006).

CONNOR HUFFAND JOSHUA D. KERTZER

between combatants and noncombatants in jus in bello, the logic is that attacks upon the government or state apparatus might be undesirable, but they are nonetheless

legitimate in a way that targeting civilians is not. The clear empirical implication is that attacks on targets closely associated with formal state institutions are more

likely to be considered legitimate, and thus should be less

likely to be considered terrorism.

Second is the broader location in which the target is

situated. This distinction was salient following the terror ist attacks in Paris and Lebanon in the fall of 2015, when

a number of pundits explicitly criticized Western publics

for the selective empathy revealed by their willingness

to change their Facebook profile pictures in solidarity with Paris while ignoring the attacks that had occurred

in Lebanon the previous day. Consistent with research on

ingroup favoritism (Brewer 1999) and affective responses

decreasing with social and spatial distance (Liberman, Trope, and Stephan 2007,373), the empirical implication is that Americans should be more likely to classify events

as terrorism when the incident takes place in the United

States, compared to in foreign countries, particularly for

eign countries unlike the United States.

Subjective Criteria: The Categorization of the Actor and Motivation

If the previous definitional components focus on the act

carried out, another set emphasizes the actors themselves and the motivation attributed to their behavior. This idea

is most commonly embedded in formal definitions not

ing that actors must have engaged in the act for "political

aims and motives" (Hoffman 2006), "achieving a political

goal" (Keller 2005), or "the advancement of some polit ical, ideological, social, economic, religious, or military agenda" (Shanahan 2010).

Three points are worth noting here. First, determin

ing motivations is difficult, since it requires overcoming

"the problem of other minds" and accessing an interior attribute of other actors, who may either have strategic

reasons to misrepresent, or who themselves may not be

aware of their underlying reasons for action. It is for this reason that classic international relations (IR) schol

ars like Morgenthau (1985, 5) famously discouraged po litical scientists from studying motivation altogether. At

the same time, however, as political scientists (Herrmann

1988) and psychologists (Malle and Knobe 1997) note, we intuitively rely on assumptions about motivations both when explaining actions and evaluating the actors who carried them out.5 In other words, the difficulty of

5 See, for example, work in IR on the security dilemma.

This content downloaded from �������������24.254.82.142 on Mon, 20 Dec 2021 21:26:56 UTC�������������

All use subject to https://about.jstor.org/terms

HOW THE PUBLIC DEFINES TERRORISM 59

assessing motivations does not deter us from relying on them.

Second, as a result, we often infer motivations for

action using information about the actors themselves; we

tend to provide answers to questions about "why" using

answers to questions about "who." It is thus worthwhile

to parse out which particular components of descriptions

of actors lead individuals to classify acts as terrorism.

Third, precisely because of the subjectivity inherent in

interpreting actions, the media has a considerable amount

of leeway in the narrative it constructs about the actor, from the characteristics it mentions to the rationales it

attributes. We discuss each of these considerations in turn.

Type of the Actor. Following violent incidents, govern ments, the media, and the public search for who is respon sible, whether individuals or collectives. The differences are embodied in the distinction between "lone wolves"

and "terrorist organizations." We argue that we should be

more likely to expect incidents perpetrated by organiza

tions to be classified as terrorism than those by individu als. Because we tend to think of terrorism as "a collective,

organized activity" (Spaaij 2010, 855) and expect indi viduals to infer that formal organizations act strategically

in pursuit of some broader political or social agenda, it follows that incidents carried out by lone individuals are

less likely to be seen as terrorism than incidents carried

out by groups, particularly more formal organizations. This means that when a bombing or hostage taking is carried out by an organization, for example, we tend to

be more likely to assume it was carried out in pursuit of

a broader political agenda than if the same incident were

carried out by a "lone wolf."

By contrast, in the wake of violent incidents perpe

trated by individuals, there is far more ambiguity about

whether the action was taken purposively in pursuit of a

broader agenda. Given the kaleidoscope of violence un dertaken by individuals in the United States, knowing that

an action was undertaken by an individual provides less information about whether an incident was undertaken

for a broader political purpose than whether an inci dent was perpetrated by an organization. In this sense,

providing information about the type of actor provides

information about why the event occurred.

We can use similar logic to consider incidents per petrated by individuals with a history of mental illness.

These types of incidents stir some of the most contro versial public debates on when and how events should

be defined as terrorism. For example, following recent shootings in Fort Hood, Charleston, and Orlando, pun dits debated whether the perpetrators' history of mental

illness should affect how we judge their actions. Just as

we expect incidents perpetrated by individuals to be less

likely to be classified as terrorism than those carried out by

more formal organizations, we can similarly expect inci

dents perpetrated by individuals with a history of mental

illness to be less likely to be attributed to a purposive political agenda, and thus less likely to be considered ter rorism.

Actor Categorization and Motivation. The broader motivations for incidents are often understood in two

ways. The first is through the identity categories used to

describe the type of actor. Because of the pervasiveness of

social categorization processes (Brewer 1999), these de scriptions both provide information about the actor and

enable observers to draw inferences about broader polit

ical agendas. Some have argued that what matters is not

only that the actor is seen as fighting for a broader po

litical purpose, but also the particular agenda for which

the actor is mobilized, hence the flurry of debates about

the presence of "double standards" in whom we define as terrorists. For example, it might matter if the actor is

described as Muslim, rather than Christian, or left-wing,

rather than right-wing. Most broadly, we hypothesize that

incidents perpetrated by an actor described as Muslim are

more likely to be classified as terrorism than other types

of descriptions (D'Orazio and Salehyan 2017; Lemieux and Karampelas 2016). The second is through the explicit

motivation attributed to the action. For example, was the

actor motivated by hatred toward the target, a personal

agenda, or a desire to change government policy? Each of these types of motivations provides varying levels of

information about the purposiveness of the action and, thus, the likelihood we should expect an incident will be

perceived as terrorism.

Method

In this article, we seek to directly assess the attributes of

incidents ordinary citizens use to define incidents as ter

rorism. We do so using a conjoint experimental design (Hainmueller, Hopkins, and Yamamoto 2014; Kertzer,

Renshon, and Yarhi-Milo 2016) in which we present our

participants with a series of incidents with randomly gen

erated features and then ask whether they would classify

each incident as terrorism. The use of a ratings-based conjoint design offers two main advantages. First, we can

hold fixed a range of attributes of incidents that could

confound observational studies on the topic. Even if there

were an abundance of high-quality public opinion data on

how the public defines terrorism, it would be extremely

This content downloaded from �������������24.254.82.142 on Mon, 20 Dec 2021 21:26:56 UTC�������������

All use subject to https://about.jstor.org/terms

6o

difficult to disentangle the effects of our attributes of

interest, particularly given the presence of both incident

specific and time-varying characteristics of the political

environment surrounding the event, as well as our in terest in the different types of media frames that prior

research has shown vary with the type of actor. Given

these problems, experiments give researchers vital con trol. Second, the use of a conjoint experimental design allows us to study the effect of each attribute, using a relatively large number of experimental treatments, in a

manner that would not be possible with more traditional

factorial experiments due to statistical power constraints.

While prior experimental research has sought to explore the effects of some of the factors we discuss in this article,

a conjoint design allows us to explore how they operate in tandem.6

The Survey Instrument

In the experiment, we manipulate seven attributes of an incident. As noted above, each of the attributes of the incident chosen, and the treatments within them, were

included to reflect long-standing debates within both aca

demic and policy worlds about how we should define ter

rorism. Each of these attributes is manipulated within a text block where the structure is intended to mirror what

respondents would observe in the first few sentences of

a newspaper article describing a recent incident. After respondents read about the details of the incident, they

were then asked whether they would classify it as terror

ism. Respondents went through this process seven times

in total. Each of these attributes is presented in Table 1 and discussed in the text below.

First, we manipulated the tactics: whether the actor

carried out a protest, hostage taking, shooting, or bomb

ing. Protests were chosen as a baseline category of nonvi

olent political action to test how the use of more violent

tactics affects whether individuals perceive an incident to

be terrorism. Shooting and hostage taking were chosen

to represent what we generally think of as violent tactics

that may or may not be associated with formal organi zations or terrorists. Finally, bombing represents one of the most emblematic tactics associated with terrorism.

Each of these tactics also has the benefit of plausibly be

ing adopted by a range of actors with varying ideologies,

targets, and motivations. Second, we manipulated the severity of the casualties,

which was set to vary from either none, one, two, or

6See Appendix §1.2 in the supporting information (SI) for further discussion of ratings-based conjoints.

CONNOR HUFFAND JOSHUA D. KERTZER

10. We specified these casualty levels with three goals in

mind. First, they serve as a way of unpacking the marginal

effect of individuals dying as the result of an incident, to

capture the notion that there is something systematically different about incidents that result in fatalities from those

that do not. Second, they explore the functional form of

the effect: is a single casualty enough for an incident to be seen as terrorism, or does the likelihood increase

with severity? Finally, the casualty levels are calibrated to

reflect the real-world distribution of casualty levels for

incidents commonly classified as terrorism, while also ensuring sufficient variation to be able to reflect higher

casualty incidents.7

Third, we manipulated the target of the incident, which was either a military facility, a police station, a school, a Christian community center, a Muslim com munity center, a Jewish community center, a church, a

mosque, or a synagogue. The use of a military facility allows us to have a target not associated with civilians, and police stations reflect targets that are affiliated with

the government but generally not perceived to be engaged

in combat. The remaining seven categories represent dif

ferent types of civilian targets, with the use of a school

capturing a type of target that is affiliated directly with

neither combat activities nor religion. Finally, we distin

guish between religious places of worship and religiously

affiliated community centers, due to the heightened sym

bolism of attacking the former.

Fourth, we manipulated the location of the incident,

such that the act took place in either the United States,

a foreign democracy, a foreign democracy with a his tory of human rights violations, a foreign dictatorship,

or a foreign dictatorship with a history of human rights violations. We varied the location to explicitly test how differences in the attributes of the target population and

government affect whether incidents should be perceived

to be terrorism. By manipulating the location of the in

cident, we are able to explicitly test whether and how incidents more closely linked to Western institutions af

fect whether respondents are more likely to classify an event to be terrorism.

Fifth, we manipulated the actor who carried out the

incident, varying whether the perpetrator was an organi

zation, an organization with ties to the United States, an

organization with ties to a foreign government, a group, an individual, or an individual with a history of mental

illness. Employing these different types of perpetrators

7These figures reflect the low number of individuals typically killed in these incidents. In the Global Terrorism Database (GTD), for

example, the median bombing has no fatalities, a bombing that kills 10 people is in the 96th percentile, and a hostage taking that kills 10 people is in the 98th percentile.

This content downloaded from �������������24.254.82.142 on Mon, 20 Dec 2021 21:26:56 UTC�������������

All use subject to https://about.jstor.org/terms

HOW THE PUBLIC DEFINES TERRORISM 6l

Table 1 Conjoint Treatments

(A) Tactic

(B) Target

(C) Location

(D) Casualties

(E) Actor Description

(F) Actor Type

(G) Actor Motivation

The ...

(1) ... protest (2) ... hostage taking (3) ... shooting (4) ... bombing ... occurred at a ...

(1) ... military facility

(2) ... police station (3) ... school (4) ... Christian community center

(5) ... Muslim community center (6) ... Jewish community center (7) ... church

(8) ... mosque (9) ... synagogue

the United States.

a foreign democracy.

a foreign democracy with a history of human rights violations,

a foreign dictatorship.

a foreign dictatorship with a history of human rights violations.

... in

(1) (2) (3) (4) (5)

There ...

(1) ... were no individuals (2) ... was one individual (3) ... were two individuals (4) ... were ten individuals

... killed in the [tactic].

The [tactic] was carried out by ... (1) ... an (2) ... a Christian (3) ... a Muslim

(4) ... a left-wing (5) ... a right-wing

(1) (2) (3) (4) (5) (6)

organization.

organization with ties to the United States,

organization with ties to a foreign government,

group. individual.

individual with a history of mental illness.

News reports suggest ... (1) (2) (3) (4) (5)

that there was no clear motivation for the incident.

the incident was motivated by the goal of overthrowing the government.

the incident was motivated by the goal of changing government policy.

the incident was motivated by hatred towards the target.

the individual had been in an ongoing personal dispute with one of the targets.

[ States.

lemocracy.

lemocracy with a history oi

lictatorship.

[ictatorship with a history c

dividuals

dividual

ndividuals

idividuals

actic].

irried out by ...

This content downloaded from �������������24.254.82.142 on Mon, 20 Dec 2021 21:26:56 UTC�������������

All use subject to https://about.jstor.org/terms

62 CONNOR HUFFAND JOSHUA D. KERTZER

allows us to test the ways in which the perceived "purpo siveness" of the actor affects the likelihood an incident is

perceived to be terrorism.

Sixth, we manipulated the social categorization of the

perpetrator, either describing the actor as Christian, Mus

lim, left-wing, or right-wing, or assigning the actor to a

control condition in which no categorization was given,

acting as a baseline from which to interpret the effects of

the other descriptions. We chose these categories as they

loomed the largest in recent prominent debates about classifying incidents as terrorism.

Finally, we manipulated the motivation attributed to

the perpetrator: government overthrow, policy change,

hatred toward the target, a personal dispute with the tar

get, or no clear motivation. By including a category where

the motivation is unclear, we can mirror the general un

certainty and absence of public information immediately

following a violent incident, approximating the actual dissemination of information as an incident unfolds. The

government overthrow and policy change motivations al

low us to capture some of the most prominent goals of terrorist organizations as studied by political scientists (Kydd and Walter 2006), whereas attributing the incident

to hatred toward the target allows us to capture some of

the more contentious incidents subject to recent debates.

Finally, the use of a personal dispute as a baseline cate gory allows us to study the effects of political motivations

versus apolitical ones. To be clear, our claim is not that the seven sets of fac

tors discussed above and summarized in Table 1 constitute

a complete set of characteristics that could be relevant to

debates about terrorism, but rather that they mirror a wide set of characteristics from either public debates or

the academic literature. Indeed, despite not being an ex haustive list, we note that a fully crossed factorial featuring

all permutations of the factors we manipulate here would

contain 108,000 combinations.8 An example of how the

vignette appeared to respondents is presented below in Figure 1.

Fielding the Experiment

The experiment was embedded in a survey fielded on 1,400 adults recruited using Amazon's Mechanical Turk

(MTurk) in August 2015.9 One commonly voiced concern

with MTurk samples is that although they are relatively

diverse, they are not representative of the American pop

ulation as a whole (Huff and Tingley 2015). To mitigate against biased estimates of our treatment effects, for the

main analyses presented below we employ entropy bal ancing to reweight the data to population parameters (Hainmueller 2012; for a similar empirical application, see Kertzer et al. 2014), though in Appendix § 1.3 in the SI, we show that the substantive conclusions remain the same

regardless of whether weights are used. Because we have

1,400 participants who classify seven incidents each, the

analyses that follow are of 9,800 different randomly gen

erated scenarios, analyzed with clustered, bootstrapped standard errors at the participant level.

Results

In this section, we present the experimental results.10 We

do so in three parts. First, we demonstrate the impor tance of the type of tactic and number of casualties. In

contrast, the target and location of the incident have no

significant effect. Second, we show that the perceived po

litical purposiveness of the actor, inferred either indirectly

through their social categorization or directly through their posited motivation, has a significant effect on the

likelihood an incident is classified as terrorism. In gen eral, the perception that an actor is acting purposively for some broader political agenda matters more than the

content of the agenda itself.

The Type of Tactic and Severity of the Violence

The results suggest that the type of violence plays a cen

tral role in determining whether an incident is classified

as terrorism, whereas the severity of the violence seems

to be much less central. The left-hand panel of Figure 2

shows the relative importance of the tactics chosen by

the actor regarding whether an incident is classified to

be terrorism. As is standard in conjoint experiments, our

quantities of interest here are average marginal compo nent effects (AMCEs), depicted in a probability scale on

the x-axis, such that point estimates further to the right

8Thankfully, as noted above, conjoint experiments leverage between- and within-subject variation across participants to achieve higher levels of statistical power. Moreover, following best practices, we included randomization constraints to prevent im plausible or problematic combinations of incident attributes from skewing the results. See Appendix §1.2 in the SI.

'For a more detailed discussion of the timing of the experiment and how it might substantively affect results, see Appendix §1.1 in the SI. See Appendix §1.4 for a discussion of MTurk.

10See Appendices §1-4 in the SI for a range of additional method ological information and analyses.

This content downloaded from �������������24.254.82.142 on Mon, 20 Dec 2021 21:26:56 UTC�������������

All use subject to https://about.jstor.org/terms

HOW THE PUBLIC DEFINES TERRORISM 63

Figure 1 An Example Vignette

The incident: a shooting. The shooting occurred at a church in a foreign democracy with a history of human rights violations.

There were two individuals killed in the shooting. The shooting was carried out by a Muslim individual with a history of mental illness. News reports suggest the individual had been in an ongoing personal dispute with one of the targets.

The incident: a shooting. The shooting occurred at a church in a foreign democracy with a history of human rights violations.

There were two individuals killed in the shooting. The shooting was carried out by a Muslim individual with a history of mental illness. News reports suggest the individual had been in an ongoing personal dispute with one of the targets.

Figure 2 The Effect of Tactics and Casualties

(a) Extremity of tactics

Protest

Hostage Taking

Shooting

Bombing —I—

0.4

~r

-0.2 0.0 0.2 0.4 0.6

Average marginal component effect (AMCE)

indicate a greater probability that an event will be classi fied as terrorism; as in the figures that follow, the point estimates are depicted with 95% confidence intervals de rived from B = 1,500 clustered bootstraps. Two patterns are evident. First, the use of violence overwhelmingly increases the likelihood an incident will be classified as

terrorism, consistent with the prominent role violence plays throughout scholarly definitions. Second, the type of violence matters as well, as bombings are significantly

more likely than either shootings or hostage taking to be classified as terrorism. This finding is consistent with the

statement made by President Obama in the wake of the Boston Marathon Bombing, which implied a uniqueness about bombing as a tactic overwhelmingly associated with terrorist organizations. The results presented here suggest

this perception is shared by ordinary citizens.

The right-hand panel of Figure 2 shows the effect of increasing casualties on whether an incident is considered

(b) Severity of violence

No Casualties i >

One Casualty #

Two Casualties —

Ten Casualties —•

I I 1 1 1

-0.2 0.0 0.2 0.4 0.6 0.8

Average marginal component effect (AMCE)

to be terrorism. Although higher-casualty incidents are more likely considered to be terrorism, the size of the effect is small relative to the increase associated with violent

tactics. In supplementary analyses in Appendix §2.2 in the SI, we reiterate the point by showing that a bombing with no fatalities is statistically indistinguishable from a

shooting that kills 10 people.

The Target and Location

Figure 3 demonstrates that there is no significant relation

ship between either the target or location of an incident and the likelihood it is classified as terrorism. Whether

the incident occurred at home or abroad, or in countries

with different regime types or human rights records ap pears to be immaterial. We therefore find no evidence that

respondents employ differential definitions in contexts

One Casualty

Two Casualties

This content downloaded from �������������24.254.82.142 on Mon, 20 Dec 2021 21:26:56 UTC�������������

All use subject to https://about.jstor.org/terms

64 CONNOR HUFF AND JOSHUA D. KERTZER

Figure 3 Location and Target of Incident

(a) Location

United States

Foreign Democracy

Foreign Democracy HR Violations

Foreign Dictatorship

Foreign Dictatorship HR Violations

I 1 1 1 1

-0.4 -0.2 0.0 0.2 0.4

Average marginal component effect (AMCE)

(b) Target

Military Facility

Police Station

School

Christian Center

Church

Muslim Center

Mosque

Jewish Center

Synagogue I I

-0.4 -0.2 0.0 0.2

Average marginal component effect (AMCE)

0.4

where the attack might be "justified" by prior government

repression.

We observe similarly null effects for the target of the

incident: there is no statistically significant difference be

tween how respondents view incidents targeting military

facilities and other government, civilian, or religious cen

ters. This absence is striking given the frequency with which terrorism definitions often distinguish between civilian and military targets; ordinary citizens seem not to make this distinction.11

Attributes of the Actor: Type, Descriptions, and Motivation

Thus far, we have shown that both the type and severity of

the violent tactics employed strongly shape the likelihood

individuals will classify an event as terrorism, whereas the

target and location of the attack do not. The above factors

are relatively objective "facts on the ground," informa

tion available relatively shortly after an incident has taken

place. In contrast, we now turn to relatively subjective characteristics about the actor itself about which the me

dia has much greater latitude when calibrating coverage.

In Figure 4, we present the results from our actor treatment, which reveal two main patterns. Acts carried

"Though see Young and Findley (2011,299-300).

Figure 4 The Political Purposiveness of the Perpetrator

Individual

Individual Mental

Group

Organization

Org. U.S. Ties

Org. Foreign Ties I r

-0.4 -0.2 0.0 0.2 0.4

Average marginal component effect (AMCE)

out by collectives are around 15% more likely to be un derstood as terrorism than acts carried out by individuals.

The effect is particularly strong for organizations with ties

to foreign governments, showcasing the extent to which

Figure 4 The Political Purposiveness of the Perpetrator

Group

This content downloaded from �������������24.254.82.142 on Mon, 20 Dec 2021 21:26:56 UTC�������������

All use subject to https://about.jstor.org/terms

HOW THE PUBLIC DEFINES TERRORISM

Figure 5 Social Categorization Effects

65

Figure 6 Motive Attribution Effects

No Ideology •

Christian

Muslim

Left-Wing

Right-Wing I 1 1 1 1 -0.4 -0.2 0.0 0.2 0.4

Average marginal component effect (AMCE)

terrorism is thought of as politically purposive. Consistent

with this emphasis on purposive action, incidents perpe

trated by an individual with a history of mental illness are

significantly less likely to be classified as terrorism.

What is especially interesting about these findings is

that some of these considerations are relatively subjec tive, thereby highlighting the way in which variation in

the presentation of information about the actor affects

whether it is perceived as terrorism. This means that the

media have the ability to shape perceptions of violent in

cidents, and subsequently whether they are classified as

terrorism, in the way that they choose to frame events, a

point we explore in detail below.

We see a less dramatic pattern in Figure 5, which de

picts how the way in which the actor was described shapes

perceptions of whether an incident should be classified as

terrorism. The left-hand panel shows how providing a social categorization (particularly Muslim or right-wing)

significantly increases the likelihood an incident will be

classified as terrorism. The effect is strongest for perpe

trators described as Muslim, though even then, the effect

size remains relatively small compared to the other factors

presented above. Interestingly, incidents where the perpe

trator is described as Christian are not significantly more

likely to be classified as terrorism. The fact that we find

these effects in spite of potential social desirability biases

suggests we should think of the magnitude of our Muslim effect as an underestimate, rather than an overestimate,

Personal Dispute •

Unclear Motivation

Hatred

Policy Change

Government Overthrow »

I 1 1 1

-0.4 -0.2 0.0 0.2 0.4

Average marginal component effect (AMCE)

although supplementary analyses reported in Appendix §2.3 in the SI suggest the magnitude of the bias is small.

Finally, Figure 6 shows how the motivation attributed

to the incident affects whether an incident is perceived to

be terrorism. The main findings are threefold. First, inci

dents motivated by a personal dispute with the target are

the least likely to be understood as terrorism. This mat

ters given recent debates over whether, for example, the

shooting of three Muslim university students in Chapel

Hill, North Carolina, in 2015 (ostensibly over a park ing dispute) was terrorism or not. Second, compared to personal disputes, we find that an incident where the motivation is unclear increases the likelihood it is under

stood as terrorism: when in doubt, our respondents are more likely to assume an incident is political rather than

personal. Third, we find that from a baseline where the

motivation is unclear, the motivations of hatred, policy

change, and government overthrow increase the likeli

hood an incident is classified as terrorism. The powerful

role of hatred here (an effect not statistically significant

from that of policy change) is especially interesting since

it does not exert a similarly large presence in legal defini

tions of terrorism, which tend to emphasize formal pol

icy goals rather than animosities. Nonetheless, the strong

effects of policy-oriented goals here reinforce the extent

to which our participants associate terrorism with acting

purposively for a broader political agenda.

Figure 5 Social Categorization Effects Figure 6 Motive Attribution Effects

Change

This content downloaded from �������������24.254.82.142 on Mon, 20 Dec 2021 21:26:56 UTC�������������

All use subject to https://about.jstor.org/terms

66

Extensions and Applications

The strength of the conjoint experiment is that it lets us

examine the relative contributions of a large number of

factors in shaping how ordinary citizens think about ter rorism. However, it also has another virtue: we can invert

our experimental findings to model how the public thinks

about real-world incidents. Doing so has three benefits. First, it allows us to code a range of incidents that have

occurred in the real world using the typology presented

above in order to predict the probability that participants

would classify each event as terrorism. Second, we exploit

the subjectivity in many of these coding decisions to show

how we can construct a number of different, equally plau

sible narratives about the same event, leading to sizable differences in the proportions of the public that should understand the event as terrorism or not. In this sense,

we show empirically how terrorism can be socially con structed through media and elite framing. Finally, we can

validate our findings using automated content analyses of

media coverage. We sketch out all three, in turn.

Application I: Mapping onto Real-World Events

First, we code 39 recent or otherwise prominent inci dents that can be subsumed by the typology presented above. Because of the potential nonadditive effects of the

attributes we explore here, we employ a machine learn

ing method proposed by Imai and Ratkovic (2013) and Egami and Imai (2015), who apply a variable selection problem approach to the study of high-dimensional treat

ment interactions by adapting a support vector machine

classifier with LASSO constraints, thereby estimating a

high-dimensional, interactive model without overfitting the data.12 We then calculate fitted values to estimate the

predicted probability that each incident would be under

stood as terrorism, presented in Table 2.

Several patterns in Table 2 are striking, including the

extent to which bombings perpetrated by formal organi

zations (e.g., Hamas, Hezbollah, ETA, and the Ku Klux Klan) are the types of incidents most likely to be per ceived as terrorism, as well as the proportion of incidents

with predicted probabilities clustering around 50%, il lustrating the contentiousness and difficulty of defining terrorism in the wake of violent incidents.

12Consistent with the main results presented throughout this arti cle, we present weighted estimates here; see Appendix §4 in the SI for alternative modeling specifications.

CONNOR HUFFAND JOSHUA D. KERTZER

Application II: Demonstrating Framing Effects

Especially important for our purposes, though, is that the

ranking of many of these incidents is in part a function

of relatively subjective coding decisions regarding the de

scription of the actor and motivation. In our view, this is

one of the most striking and important implications of

this article, since the media make similar coding decisions

when choosing whether to highlight a suspect's religion,

speculate on foreign ties, attribute a political motivation

behind the incident, or psychoanalyze the perpetrator from a distance.

To make this discussion concrete, consider the shoot

ing that took place in December 2015 in San Bernardino,

California. In the aftermath of the shooting, a number of

facts emerged about the two perpetrators: Tashfeen Ma

lik had pledged allegiance to ISIS in a Facebook post, and

Syed Rizwan Farook's father was described as being both

abusive and mentally ill. There are at least three subjective decisions one can make about how to code the event. First,

was the act carried out by lone individuals, or were they

actors with foreign ties? Is pledging allegiance on social

media sufficient to constitute a foreign tie? Second, was

the attack motivated by policy goals, or was the motiva

tion behind the shooting unclear? Third, is shooting up an

office party indicative of mental illness, or merely malign

intentions? Our model suggests these coding decisions have important implications for how people understand

the attack. If the perpetrators are coded as having foreign

ties and the goal of changing policy, our model suggests

an 82% likelihood of its being terrorism. If, instead, the

perpetrators are simply seen as individuals with no clear

motivation, the probability of classifying the incident as

terrorism drops to 50%. If the relevant narrative instead

raises the specter of mental illness and does not mention

the attackers' religion, the probability drops to 31%. In this sense, there is over a 50% difference in the likelihood

the attacks will be classified as terrorism, based solely on

how the story is framed.

Application III: Analyzing Media Coverage

The results in Table 2 also suggest two additional impli cations. First, if our classifications are relatively accurate,

they should correlate with the rate at which the incidents are described as terrorism in the real world: an event

with the characteristics that our participants see as highly

likely to be terrorism should generally be described as such in the press, for example. Second, the extremity of

the probability estimates should be negatively correlated

This content downloaded from �������������24.254.82.142 on Mon, 20 Dec 2021 21:26:56 UTC�������������

All use subject to https://about.jstor.org/terms

HOW THE PUBLIC DEFINES TERRORISM 67

Table 2 The Predicted Probability a Range of Incidents Are Classified as Terrorism Using a Weighted Support Vector Machine Classifier with Lasso Constraints

Incident

East Selma Church Shooting

Dallas Police HQ Shooting UCLA Black Lives Matter Protest

University of California Tuition Hike Protests

Zvornik Police Station Shooting

Marysville Pilchuck High School Shooting

Rocori High School Shooting

Shooting of Police in Brooklyn

Islamic Community Center of Phoenix Demonstrations

Shooting of Police in Oakland

Pentagon Metro Shooting

St. Columbanus Church Shooting

Poe Elementary School Bombing

Newport Church Hostage Situation

Copenhagen Synagogue Shooting

University of Alabama Huntsville Shooting

Charleston Church Shooting

Camp Shelby Shootings

Overland Park Jewish Community Center Shooting

Knoxville Unitarian Universalist Church Shooting

Rosemary Anderson High School Shooting

Shooting of George Tiller

Bombing of Shiraa Village Mosque

Kehilat Bnei Torah Synagogue Attack

KKK Selma Bombing

Seatde Jewish Federation Shooting

Nag Hammadi Massacre Contra Attack in Quilali

ETA Sanguesa Car Bombing

Lombard Islamic School Bombing Hamas Attack on IDF in Khan Yunis

Shebaa Farms Incident

Camp Integrity Suicide Bombing

Porte de Vincennes Hostage Situation

Pakistan Army General HQ Hostage Situation

Zif School Bombing

Aksu Bombing

Chattanooga Shootings

Fort Hood Shootings

Date Predicted Probability

09/20/15 0.23

06/13/15 0.25

10/08/15 0.28

03/18/15 0.31

04/27/15 0.33

10/24/14 0.33

09/24/03 0.33

12/20/14 0.35

10/10/15 0.35

03/21/09 0.35

03/04/10 0.36

11/26/12 0.39

09/15/59 0.39

07/30/06 0.39

02/14/15 0.39

02/12/10 0.43

06/17/15 0.46

08/05/15 0.47

04/13/14 0.47

07/27/08 0.48

12/13/14 0.49

05/31/09 0.55

12/30/14 0.55

11/18/14 0.59

09/15/63 0.61

07/28/06 0.62

01/07/10 0.62

11/11/87 0.62

05/30/03 0.64

08/12/12 0.71

12/24/14 0.71

01/28/15 0.72

08/07/15 0.74

01/09/15 0.74

10/10/09 0.78

09/17/02 0.84

08/19/10 0.85

07/16/15 0.85

11/05/09 0.85

with the magnitude of public debate: events clustered at

around 50% on our probability scale should be the ones that produce the greatest dissensus.

Automated content analyses of newspaper coverage thus serve as a helpful means of validating our find ings. As a plausibility probe, we selected two events from

Table 2: an incident with a high classification rate (the

shooting at Fort Hood, with a terrorism probability of 85%), and an incident with a classification rate close to

50% (the shooting at Charleston, with a terrorism prob

ability of 46%). We then used LexisNexis to collect every

newspaper article about the incidents published in the

This content downloaded from �������������24.254.82.142 on Mon, 20 Dec 2021 21:26:5 1976 12:34:56 UTC

All use subject to https://about.jstor.org/terms

68 CONNOR HUFF AND JOSHUA D. KERTZER

Figure 7 Automated Content Analysis of Media Coverage

(a) Proportion of articles mentioning terrorism or terrorist

Proportion of Articles

(b) Proportion of articles mentioning debate

Proportion of Articles

Charleston Fort Hood

!

1 week

Weeks after Incident

2 weeks

Charleston — - Fort Hood

!

1 week

Weeks after Incident

2 weeks

Note. Automated content analysis of daily newspaper coverage of the attacks in Charleston and Fort Hood (which our models classify as 86% and 46% likely to be classified as terrorism, respectively) provides results consistent with our theory. Panel (a) shows that newspaper coverage of Fort Hood is consistently more likely to refer to terrorism than articles covering Charleston. Panel (b) shows that newspaper coverage of Charleston was significantly more likely to discuss the presence of "debate" than newspaper coverage of Fort Hood.

two weeks following each attack.13 There are several im

portant similarities between these incidents: for example,

both shootings received high levels of media coverage, were perpetrated by an individual, and took place in the

United States. Yet there are also a number of important

differences that map directly onto the typology presented

in this article: for example, the perpetrator in Charleston

had a more pronounced history of mental illness, was associated with right-wing extremists, and was described

as being motivated by hatred rather than formal policy

change. For each event, we calculated two quantities of in

terest. The first is the proportion of articles published

making reference to the term terrorism or terrorist if our

model is accurate, we should expect that news cover

age of the incident with a high predicted classification rate should more frequently invoke terrorism than the

coverage of the event with a moderate predicted clas sification rate. The second is the proportion of articles

published making reference to "debate": if our model is

reliable, we should expect that the coverage of the event

13 See Appendix §6 in the SI for details.

with a high predicted classification rate should feature less

debate than the coverage of the event with a predicted clas

sification rate that hovers near a coin toss. Sure enough,

when we plot the daily counts of both quantities in Figure 7, we find that articles about the shooting at Fort

Hood were more likely to mention the word terrorism or

terrorist than articles about the shooting in Charleston. Indeed, in the two-week window around each of the re

spective incidents, articles about Charleston were approx

imately 13% less likely to mention the word terrorism or terrorist than articles about Fort Hood (p < .01), consis

tent with the idea that incidents we predict as more likely to be classified as terrorism are indeed discussed as such

in the news media.

Similarly, the right panel of Figure 7 demonstrates

that articles about the shooting at Charleston were more

likely to mention the word debate than articles about the

shooting at Fort Hood. In the two-week window around the incidents, articles about Charleston were 15% more

likely to mention the word debate than articles about Fort Hood. Although more of a plausibility probe than a full test of our theory, these results show how media

coverage offers another domain in which our theoretical

framework can be explored.

This content downloaded from �������������24.254.82.142 on Mon, 20 Dec 2021 21:26:56 UTC�������������

All use subject to https://about.jstor.org/terms

HOW THE PUBLIC DEFINES TERRORISM

Conclusion

This article contributes to broader research on the rela

tionship between public opinion and government action

in the wake of violent events by exploring the character

istics that shape how ordinary citizens classify incidents

as terrorism. A rich literature in IR explores the distinc

tion between terrorism and "big" forms of violence like

civil wars (e.g., Findley and Young 2012; Stanton 2013);

our interest here is in differentiating between terrorism

and "small" forms of violence. Combining a conjoint ex periment with machine learning techniques, our main findings are threefold.

First, the likelihood that ordinary citizens classify an

event as terrorism is heavily dependent on relatively ob

jective facts on the ground, such as the extremity and severity of violence employed. And yet, the public is also

heavily influenced by the descriptions offered of who car

ried out the incident and why: acts are more likely to be

seen as terrorism if they are carried out by organizations,

less likely if they are carried out by individuals with his

tories of mental illness, more likely if they are carried out

by Muslims, more likely if they are carried out in order to

achieve political goals, and so on. Second, due to the sub

jective nature of some of these considerations, the media

has significant agency in shaping how the public comes

to classify violent events; given the extent to which terror

ism increases news ratings, market-based theories of the

media (e.g., Baum and Groeling 2010) would expect the media to have an incentive to frame ambiguous violent

events in certain ways. Since violent incidents do not speak

for themselves, our findings thus mirror other critiques

of strictly "event-driven" theories of public opinion in foreign policy (e.g., Berinsky 2007). Third, although the public generally classifies events similarly to the formal

classification schemes employed by different government

agencies, it also deviates in some ways: violent incidents do not need to target civilians in order to be understood

as terrorism, for example, and the public thinks that inci

dents motivated by hatred are just as likely to be terrorism

as those motivated by more formal policy goals. In this sense, there seems to be a disconnect between formal le

gal definitions and our folk intuitions (Audi 2009) worth

exploring.14

Future work should build on this analysis in four ways. First, although we show the powerful role that

alternate frames can play in how and whether people clas

sify events as terrorism, one of the interesting dynamics

from a political science perspective is the extent to which

14As we show in Appendix §3 in the SI, a similar disconnect exists with scholarly definitions as well.

69

people are often presented with competing frames simul

taneously (Druckman 2004): the New York Post covered the San Bernardino attacks very differently than MSNBC.

Understanding how people classify events as terrorism in

the face of competing frames is thus worthy of study. Sec

ond, although an automated content analysis of media coverage finds results consistent with our experimental

findings, our theory also raises further questions about

strategic media behavior: for example, if the media has

ratings-based incentives to frame events consistent with

the terrorist tropes discussed above, why emphasize white

perpetrators' histories of mental illness (Powell 2011), even though that lowers the likelihood of perceiving an

event as terrorism? Additional analysis of media coverage

will thus further enrich our understanding of how people come to understand events as terrorism.

Third, our results also suggest politicians can po tentially manipulate perceptions of terrorism by fram

ing violent incidents in certain ways. For example, if de

cision makers are in favor of pursuing more assertive foreign policies to combat a particular terrorist organi zation, we might expect them to highlight the poten tial for "foreign ties" in order to increase the likelihood

the public perceives an incident to be terrorism and de

mand retribution. Finally, although our interest here is in how Americans think about terrorism, a final area of

future research should explore whether foreign publics

espouse similar judgments. The rise of knife attacks in

Israel in 2015, for example, suggests a different portfolio

of tactics would resonate with the public there; citizens

of European countries with strict gun control laws might

similarly be more likely to view shootings as terrorism than Americans do. We view each of these areas as an

exciting part of future research that will continue to con

tribute to a deeper understanding of the important rela

tionship between political violence and public opinion.

References

Abrahms, Max. 2013. "The Credibility Paradox: Violence as a Double-Edged Sword in International Politics." Interna tional Studies Quarterly 57(4): 660-71.

Agamben, Giorgio. 2005. State of Exception. Chicago: University of Chicago Press.

Albertson, Bethany, and Shana Kushner Gadarian. 2015. Anx ious Politics: Democratic Citizenship in a Threatening World. New York: Cambridge University Press.

Asal, Victor, and Aaron M. Hoffman. 2015. "Media Effects: Do

Terrorist Organizations Launch Foreign Attacks in Response to Levels of Press Freedom or Press Attention?" Conflict Man agement and Peace Science 33(4): 381-99.

This content downloaded from �������������24.254.82.142 on Mon, 20 Dec 2021 21:26:56 UTC�������������

All use subject to https://about.jstor.org/terms

70 CONNOR HUFFAND JOSHUA D. KERTZER

Audi, Robert. 2009. The Good in the Right: A Theory of Intu ition and Intrinsic Value. Princeton, NJ: Princeton University Press.

Bali, Valentina A. 2007. "Terror and Elections: Lessons from Spain." Electoral Studies 26(3): 669-87.

Baum, Matthew A., and Tim J. Groeling. 2010. War Stories: The Causes and Consequences of Public Views of War. Princeton, NJ: Princeton University Press.

Berinsky, Adam J. 2007. "Assuming the Costs of War: Events, Elites, and American Public Support for Military Conflict." Journal of Politics 69(4): 975-97.

Berrebi, Claude, and Esteban F. Klor. 2008. "Are Voters Sensitive to Terrorism? Direct Evidence from the Israeli Electorate."

American Political Science Review 102(3): 279-301.

Bloom, Mia. 2005. Dying to Kill: The Allure of Suicide Terror. New York: Columbia University Press.

Braithwaite, Alex. 2013. "The Logic of Public Fear in Terror ism and Counter-terrorism." Journal of Police and Criminal Psychology 28(2): 95-101.

Brewer, Marilynn B. 1999. "The Psychology of Prejudice: In group Love or Outgroup Hate?" Journal of Social Issues 55(3): 429-44.

Bueno de Mesquita, Ethan, and Eric S. Dickson. 2007. "The Propaganda of the Deed: Terrorism, Counterterrorism, and Mobilization." American Journal of Political Science 51(2): 364-81.

Coady, Cecil A.J. 2004. "Terrorism and Innocence." Journal of Ethics 8(1): 37-58.

Cowan, Richard, and Susan Cornwell. 2016. "Democrats Link Guns to Terrorism, Turn to Gun Control After Or lando." Reuters, http://www.reuters.com/article/us-florida shooting-guns-idUSKCN0YZ2DV

Cox, Daniel and Robert P. Jones. 2015. "Nearly Half of Ameri cans Worried That They or Their Family Will Be a Victim of Terrorism." PRRI. https://www.prri.org/research/survey nearly-half-of-americans-worried-that-they-or-their family-will-be-a-victim-of-terrorism/

Crenshaw, Martha. 2000. "The Psychology of Terrorism: An Agenda for the 21st Century." Political Psychology 21(2): 405-20.

D'Orazio, Vito, and Idean Salehyan. 2017. "Who Is a Terrorist? Ethnicity, Group Affiliation, and Understandings of Political Violence." Paper presented at the annual meeting of the International Studies Association.

Druckman, James N. 2004. "Political Preference Formation: Competition, Deliberation, and the (Ir)relevance of Fram ing Effects." American Political Science Review 98(4): 671— 86.

Egami, Naoki, and Kosuke Imai. 2015. "Causal Interaction in High-Dimension." Working paper. http://imai.princeton. edu/research/files/int.pdf.

Enders, Walter, and Todd Sandler. 2011. The Political Economy

of Terrorism. New York: Cambridge University Press.

Eubank, William Lee, and Leonard Weinberg. 1994. "Does Democracy Encourage Terrorism?" Terrorism and Political Violence 6(4): 417-35.

Findley, Michael G., and Joseph K. Young. 2012. "Terrorism and Civil War: A Spatial and Temporal Approach to a Conceptual Problem." Perspectives on Politics 10(2): 285-305.

Fortna, Virginia Page. 2015. "Do Terrorists Win? Rebels' Use of Terrorism and Civil War Outcomes." International Organi zation 69(3): 519-56.

Getmansky, Anna, and Thomas Zeitzoff. 2014. "Terrorism and Voting: The Effect of Rocket Threat on Voting in Israeli Elections." American Political Science Review 108(3): 588— 604.

Goodwin, Jeff. 2006. "A Theory of Categorical Terrorism." So cial Forces 84(4): 2027-46.

Hainmueller, Jens. 2012. "Entropy Balancing for Causal Effects: A Multivariate Reweighting Method to Produce Balanced Samples in Observational Studies." Political Analysis 20(1): 25-46.

Hainmueller, Jens, Daniel J. Hopkins, and Teppei Yamamoto. 2014. "Causal Inference in Conjoint Analysis: Understand ing Multidimensional Choices via Stated Preference Experi ments." Political Analysis 22(1): 1-30.

Hattem, Julian. 2015. "Clinton Calls Charleston Shooting 'Racist Terrorism.'" The Hill, http://thehill.com/policy/ national-security/245943-clinton-calls-charleston shooting-racist-terrorism

Herrmann, Richard. 1988. "The Empirical Challenge of the Cognitive Revolution: A Strategy for Drawing Inferences about Perceptions." International Studies Quarterly 32(2): 175-203.

Hodges, Adam. 2011. The "War on Terror" Narrative: Discourse and Intertextuality in the Construction and Contestation of Sociopolitical Reality. Oxford: Oxford University Press.

Hoffman, Bruce. 2006. Inside Terrorism. New York: Columbia

University Press.

Horgan, John. 2004. The Psychology of Terrorism. New York: Routledge.

Horowitz, Michael C. 2015. "The Rise and Spread of Sui cide Bombing." Annual Review of Political Science 18: 69 84.

Huddy, Leonie, Stanley Feldman, Charles Taber, and Gallya Lahav. 2005. "Threat, Anxiety, and Support of Antiterrorism Policies." American Journal of Political Science 49(3): 593 608.

Huff, Connor, and Dominika Kruszewska. 2016. "Banners, Bar ricades, and Bombs: The Tactical Choices of Social Move ments and Public Opinion." Comparative Political Studies 49(13): 1774-1808.

Huff, Connor, and Dustin Tingley. 2015. '"Who Are These Peo ple?' Evaluating the Demographic Characteristics and Polit ical Preferences of MTurk Survey Respondents." Research & Politics 2(3): 1-12.

Imai, Kosuke, and Marc Ratkovic. 2013. "Estimating Treatment Effect Heterogeneity in Randomized Program Evaluation." Annals of Applied Statistics 7(1): 443-70.

Kalyvas, Stathis N„ and Laia Balcells. 2010. "International Sys tem and Technologies of Rebellion: How the End of the Cold War Shaped Internal Conflict." American Political Science Review 104(3): 415-29.

Keller, Simon. 2005. Philosophy 9/11: Thinking about the War on Terrorism. Chicago: Open Court.

Kertzer, Joshua D. 2007. "Seriousness, Grand Strategy, and Paradigm Shifts in the 'War on Terror.'" International Journal 62(4): 961-79.

This content downloaded from �������������24.254.82.142 on Mon, 20 Dec 2021 21:26:56 UTC�������������

All use subject to https://about.jstor.org/terms

HOW THE PUBLIC DEFINES TERRORISM 71

Kertzer, Joshua D„ and Kathleen M. McGraw. 2012. "Folk Realism: Testing the Microfoundations of Realism in Or dinary Citizens." International Studies Quarterly 56(2): 245-58.

Kertzer, Joshua D„ Kathleen Powers, Brian C. Rathbun, and Ravi Iyer. 2014. "Do Moral Values Shape Foreign Policy Preferences?" Journal of Politics 76(3): 825—40.

Kertzer, Joshua D„ Jonathan Renshon, and Keren Yarhi-Milo.

2016. "How Do Observers Assess Resolve?" Working pa per. http://people.fas.harvard.edu/ ~ jkertzer/Research_files/ KertzerRenshonYarhi-Milo021415.pdf

Kydd, Andrew H., and Barbara F. Walter. 2006. "The Strategies of Terrorism." International Security 31(1): 49-80.

Landler, Mark. 2013. "Obama Calls Blasts an Act of Ter rorism.'" New York Times, http://www.nytimes.com/ 2013/04/17/us/politics/obama-calls-marathon-bombings an-act-of-terrorism.html

Lemieux, Anthony, and Elizabeth Karampelas. 2016. "'Heil Hitler' versus Allahu Akbar': An Experimental Approach to How Terrorism Is Differentially Perceived and Labeled." Paper presented at the annual meeting of the International Studies Association.

Liberman, Nira, Yaacov Trope, and Elena Stephan. 2007. "Psy chological Distance." In Social Psychology: Handbook of Basic Principles, eds. Arie W. Kruglanski and E. Tory Higgins. New York, NY: The Guilford Press, 353-83.

Malle, Bertram F., and Joshua Knobe. 1997. "The Folk Concept of Intentionality." Journal of Experimental Social Psychology 33(2): 101-21.

Merolla, Jennifer L., and Elizabeth J. Zechmeister. 2009. Democ

racy at Risk: How Terrorist Threats Affect the Public. Chicago: University of Chicago Press.

Morgenthau, Hans J. 1985. Politics among Nations: The Struggle for Power and Peace. Brief ed. Boston: McGraw-Hill.

Nacos, Brigitte L„ Yaeli Bloch-Elkon, and Robert Y. Shapiro. 2011. Selling Fear: Counterterrorism, the Media, and Public Opinion. Chicago: University of Chicago Press.

O'Rourke, Lindsey. 2009. "What's Special about Female Suicide Terrorism?" Security Studies 18(4): 681-718.

Pape, Robert A. 2003. "The Strategic Logic of Suicide Ter rorism." American Political Science Review 97(3): 343 61.

Phillips, Brian J. 2014. "Terrorist Group Cooperation and Longevity." International Studies Quarterly 58(2): 336— 47.

Phillips, Brian J. 2015. "Was What Happened in Charleston Ter rorism?" The Monkey Cage. https://www.washingtonpost. com/news/monkey- cage/wp/2015/06/18/was-what happened-in-charleston-terrorism/.

Powell, Kimberly A. 2011. "Framing Islam: An Analysis of U.S. Media Coverage of Terrorism since 9/11." Communication Studies 62(1): 90-112.

Rodin, David. 2004. "Terrorism without Intention." Ethics 114(4): 752-71.

Shanahan, Timothy. 2010. "Betraying a Certain Corruption of Mind: How (and How Not) to Define 'Terrorism.'" Critical Studies on Terrorism 3(2): 173-90.

Smith, Megan, and James Igoe Walsh. 2013. "Do Drone Strikes Degrade AI Qaeda? Evidence from Propaganda Output." Terrorism and Political Violence 25(2): 311-27.

Spaaij, Ramon. 2010. "The Enigma of Lone Wolf Terrorism: An Assessment." Studies in Conflict & Terrorism 33(9): 854-70.

Stanton, Jessica A. 2013. "Terrorism in the Context of Civil War." Journal of Politics 75(4): 1009-22.

Tilly, Charles. 1986. The Contentious French. Cambridge, MA: Harvard University Press.

Uygur, Cenk. 2016. "Let's be Clear: If Muslims had Seized a Federal Building, they'd all be Dead by now. #whitepriv ilege #OregonUnderAttack." Tweet, https://twitter.com/ cenkuygur/status/683770365009514497?lang=en

Weinberg, Leonard, Ami Pedahzur, and Sivan Hirsch-Hoefler. 2004. "The Challenges of Conceptualizing Terrorism." Ter rorism and Political Violence 16(4): 777-94.

Young, Joseph K. 2012. "What's in a Name? How to Classify Recent Violent Events." Political Violence at a Glance.

https://politicalviolenceataglance.org/2012/08/17/three reasons-why-the-recent-attack-in-aurora-co-was-not terrorism/.

Young, Joseph K., and Michael G. Findley. 2011. "Promise and Pitfalls of Terrorism Research." International Studies Review

13(3): 411-31.

Supporting Information

Additional Supporting Information may be found in the

online version of this article at the publisher's website:

• Survey information

• Higher-order quantities of interest

• Scholarly classifications of terrorism

• Model diagnostics • Alternative predicted probabilities • Additional information about the media content

analysis

This content downloaded from �������������24.254.82.142 on Mon, 20 Dec 2021 21:26:56 UTC�������������

All use subject to https://about.jstor.org/terms