Assignment: Criminal Organization Legislation

O R I G I N A L P A P E R

Deterring Gang-Involved Gun Violence: Measuring the Impact of Boston’s Operation Ceasefire on Street Gang Behavior

Anthony A. Braga • David M. Hureau • Andrew V. Papachristos

Published online: 20 March 2013 � Springer Science+Business Media New York 2013

Abstract Objectives The relatively weak quasi-experimental evaluation design of the original Boston Operation Ceasefire left some uncertainty about the size of the program’s effect on

Boston gang violence in the 1990s and did not provide any direct evidence that Boston

gangs subjected to the Ceasefire intervention actually changed their offending behaviors.

Given the policy influence of the Boston Ceasefire experience, a closer examination of the

intervention’s direct effects on street gang violence is needed.

Methods A more rigorous quasi-experimental evaluation of a reconstituted Boston Ceasefire program used propensity score matching techniques to develop matched treat-

ment gangs and comparison gangs. Growth-curve regression models were then used to

estimate the impact of Ceasefire on gun violence trends for the treatment gangs relative to

comparisons gangs.

Results This quasi-experimental evaluation revealed that total shootings involving Boston gangs subjected to the Operation Ceasefire treatment were reduced by a statisti-

cally-significant 31 % when compared to total shootings involving matched comparison

Boston gangs. Supplementary analyses found that the timing of gun violence reductions for

treatment gangs followed the application of the Ceasefire treatment.

Conclusions This evaluation provides some much needed evidence on street gang behavioral change that was lacking in the original Ceasefire evaluation. A growing body of

scientific evidence suggests that jurisdictions should adopt focused deterrence strategies to

control street gang violence problems.

Keywords Gang violence � Guns � Deterrence � Problem-oriented policing

A. A. Braga Rutgers University, Newark, NJ, USA

A. A. Braga (&) � D. M. Hureau � A. V. Papachristos John F. Kennedy School of Government, Harvard University, 79 John F. Kennedy Street, Cambridge, MA 02138, USA e-mail: Anthony_Braga@harvard.edu

A. V. Papachristos Yale University, New Haven, CT, USA

123

J Quant Criminol (2014) 30:113–139 DOI 10.1007/s10940-013-9198-x

Introduction

Boston received national acclaim for its innovative approach to preventing youth violence

in the 1990s (see, e.g. Butterfield 1996; Witkin 1997). The well-known Operation Ceasefire

initiative was an interagency violence prevention program that focused enforcement and

social service resources on a small number of gang-involved offenders at the heart of the

city’s youth violence problem (Kennedy et al. 1996). The Ceasefire ‘‘pulling levers’’

focused deterrence strategy was associated with a near two-thirds drop in youth homicide

in the late 1990s (Braga et al. 2001; Piehl et al. 2003) and was soon embraced by the U.S.

Department of Justice as an effective approach to crime prevention. In his address to the

American Society of Criminology, former National Institute of Justice Director Jeremy

Travis (1998) announced ‘‘[the] pulling levers hypothesis has made enormous theoretical

and practical contributions to our thinking about deterrence and the role of the criminal

justice system in producing safety.’’ Subsequently, the basic elements of the Boston

Ceasefire framework has been applied in many American cities through federally spon-

sored violence prevention programs such as the Strategic Alternatives to Community

Safety Initiative and Project Safe Neighborhoods (Dalton 2002).

The evaluation of Boston’s Operation Ceasefire, however, has been greeted with both a

healthy dose of skepticism (Fagan 2002; Rosenfeld et al. 2005) and some support (Cook

and Ludwig 2006; Morgan and Winship 2007). The relatively weak quasi-experimental

evaluation design of the original implementation leaves some uncertainty about the size of

Ceasefire’s effect on gang violence in Boston and does not provide any direct evidence that

Boston gangs subjected to the Ceasefire intervention actually changed their offending

behaviors (Ludwig 2005; Wellford et al. 2005). Given the influence of the Operation

Ceasefire experience on policing and violence prevention policy, a more rigorous exam-

ination of the intervention’s effects on street gang behavior in Boston is sorely needed.

In this paper, we take advantage of unique data on gangs and gang-involved gun

violence in Boston in a quasi-experimental evaluation of the group-level violence pre-

vention effects of a reconstituted Operation Ceasefire strategy implemented in 2007. As

compared to previous evaluations of Operation Ceasefire that focused solely on aggregate

rates of violence, our quasi-experimental evaluation focuses squarely on the gangs that

were targeted for treatment. Propensity score matching techniques were used to develop

matched Ceasefire treatment gangs and comparison gangs. Growth-curve regression

models were then used to estimate the impact of Ceasefire on gun violence trends for the

treatment gangs relative to comparisons gangs. We find that the Ceasefire intervention was

associated with statistically significant reductions in gun violence trends for treatment

gangs relative to gun violence trends for the comparison gangs. A supplementary analysis

examined the specific timing of the Ceasefire intervention as applied to each matched

treatment gang and found that sharp reductions in gun violence immediately followed the

intervention.

Literature Review

The Boston Gun Project and Operation Ceasefire

The Boston Gun Project was a problem-oriented policing enterprise expressly aimed at

taking on a serious, large-scale crime problem—homicide victimization among young

people in Boston. Like many large cities in the United States, Boston experienced a large

114 J Quant Criminol (2014) 30:113–139

123

sudden increase in youth homicide between the late 1980s and early 1990s. The Project

began in early 1995 and implemented what is now known as the ‘‘Operation Ceasefire’’

intervention, which started in the late spring of 1996 (Kennedy et al. 1996). Led by the

Boston Police Department (BPD), a working group of law enforcement personnel, youth

workers, and Harvard University researchers diagnosed the youth violence problem in

Boston as one of patterned, largely vendetta-like (‘‘beef’’) hostility amongst a small

population of chronic offenders, and particularly among those involved in loose, informal,

mostly neighborhood-based gangs. These gangs represented less than 1 % of the city’s

youth between the ages of 14 and 24, but were responsible for more than 60 % of youth

homicide in Boston.

The focused deterrence strategy behind Operation Ceasefire was designed to prevent

violence by reaching out directly to gangs, saying explicitly that violence would no longer

be tolerated, and backing up that message by ‘‘pulling every lever’’ legally available when

violence occurred (Kennedy 1997, 2011). The chronic involvement of gang members in a

wide variety of offenses made them—and their groups—vulnerable to a coordinated

criminal justice response. The authorities could disrupt street drug activity, focus police

attention on low-level street crimes such as trespassing and public drinking, serve out-

standing warrants, cultivate confidential informants for medium- and long-term investi-

gations of gang activities, deliver strict probation and parole enforcement, seize drug

proceeds and other assets, ensure stiffer plea bargains and sterner prosecutorial attention,

request stronger bail terms (and enforce them), and bring potentially severe federal

investigative and prosecutorial attention to gang-related drug and gun activity. Rather than

simply dealing with individual offending, groups were held accountable for outbreaks of

serious gun violence.

Simultaneously, youth workers, probation and parole officers, and later churches and

other community groups offered gang members services and other kinds of help. These

partners also delivered an explicit message that violence was unacceptable to the com-

munity and that ‘‘street’’ justifications for violence were mistaken. The Ceasefire Working

Group delivered this message in formal meetings with gang members (known as ‘‘forums’’

or ‘‘call-ins’’), through individual police and probation contacts with gang members,

through meetings with inmates at secure juvenile facilities in the city, and through gang

outreach workers. The deterrence message was not a deal with gang members to stop

violence. Rather, it was a promise to gang members that violent behavior would evoke an

immediate and intense response. If gangs committed other crimes but refrained from

violence, the normal workings of police, prosecutors, and the rest of the criminal justice

system dealt with these matters. But if gang members persisted in their violent behaviors,

the Working Group concentrated its enforcement actions on their gangs.

The idea of the Ceasefire ‘‘crackdowns’’ specifically but the focused deterrence model

more generally was not to eliminate gangs or stop every aspect of gang activity, but rather

to control and deter serious violence among specified groups (Kennedy 1997). To do this,

the Working Group explained its actions against targeted gangs to other gangs, as in ‘‘this

gang did violence, we responded with the following actions, and here is how to prevent

anything similar from happening to you.’’ The ongoing Working Group process regularly

watched the city for outbreaks of gang violence and framed any necessary responses in

accord with the Ceasefire strategy. As the strategy unfolded, the Working Group continued

communication with gangs and gang members to convey its determination to stop violence,

to explain its actions to the target population, and to maximize both voluntary compliance

and the strategy’s deterrent power.

J Quant Criminol (2014) 30:113–139 115

123

Operation Ceasefire Deterrence Mechanisms

Deterrence theory posits that crimes can be prevented when the costs of committing the

crime are perceived by the offender to outweigh the benefits (Gibbs 1975; Zimring and

Hawkins 1973). Most discussions of the deterrence mechanism distinguish between

‘‘general’’ and ‘‘special’’ deterrence (Cook 1980). General deterrence is the idea that the

general population is dissuaded from committing crime when it sees that punishment

necessarily follows the commission of a crime. Special deterrence involves punishment

administered to criminals with the intent to discourage them from committing crimes in the

future. Much of the literature evaluating deterrence focuses on the effect of changing

certainty, swiftness, and severity of punishment associated with certain acts on the prev-

alence of those crimes (see, e.g. Apel and Nagin 2011; Blumstein et al. 1978; Cook 1980;

Nagin 1998; Paternoster 1987).

In addition to any increases in certainty, severity, and swiftness of sanctions associated

with gun violence, the Operation Ceasefire strategy sought to gain deterrence through the

advertising of the law enforcement strategy, and the personalized nature of its application.

The effective operation of general deterrence is dependent on the communication of

punishment threats to the public. As Zimring and Hawkins (1973) observe, ‘‘the deterrence

threat may best be viewed as a form of advertising’’ (p. 142). A key element of the strategy

was the delivery of a direct and explicit ‘‘retail deterrence’’ message to a relatively small

target audience regarding what kind of behavior would provoke a special response and

what that response would be. 1

The available research suggests that deterrent effects are ultimately determined by

offender perceptions of sanction risk and certainty (Nagin 1998). As described above,

Operation Ceasefire was targeted on very specific behaviors by a relatively small number

of chronic offenders who were highly vulnerable to criminal justice sanctions. The

approach directly confronted violent gang members and informed them that continued

offending will not be tolerated and how the system will respond to violations of these new

behavior standards. Face-to-face meetings with offenders are an important first step in

altering their perceptions about sanction risk (Horney and Marshall 1992; Nagin 1998). As

McGarrell et al. (2006) suggest, direct communications and affirmative follow-up

responses are the types of new information that may cause offenders to reassess the risks of

committing crimes.

In their recent essay on the limits of lengthy prison stays to deter crime, Durlauf and

Nagin (2011, p. 40) suggest that ‘‘strategies that result in large and visible shifts in

apprehension risk are most likely to have deterrent effects that are large enough not only to

reduce crime but also apprehensions.’’ Focused deterrence strategies, such as Boston’s

Operation Ceasefire, are identified by Durlauf and Nagin (2011) as having this charac-

teristic. Moreover, they suggest that these ‘‘carrot and stick approaches’’ to crime pre-

vention creatively use positive incentives, such as social services and job opportunities, to

reward compliance and facilitate nonviolent behavior. Durlauf and Nagin (2011) conclude

their discussion of the promise of focused deterrence strategies with a call for additional

research and evaluation on the crime reduction benefits of these new approaches.

1 Wright et al. (2004, p. 184) offer a clever metaphor of this perspective: ‘‘A restaurant owner can sell more

prime rib by lowering its price, but not to vegetarian patrons. The price of prime rib here represents the situational inducement toward ordering meat, but vegetarianism represents a predisposition away from it, and thus the effect of meat pricing significantly varies by level of meat eating.’’

116 J Quant Criminol (2014) 30:113–139

123

Evaluation Evidence

A large reduction in the yearly number of Boston youth homicides followed immediately

after Operation Ceasefire was implemented in mid-1996. A U.S. Department of Justice

(DOJ)-sponsored evaluation of Operation Ceasefire revealed that the intervention was

associated with a 63 % decrease in the monthly number of Boston youth homicides, a

32 % decrease in the monthly number of shots-fired calls, a 25 % decrease in the monthly

number of gun assaults, and, in one high-risk police district given special attention in the

evaluation, a 44 % decrease in the monthly number of youth gun assault incidents (Braga

et al. 2001). The evaluation also suggested that Boston’s significant youth homicide

reduction associated with Operation Ceasefire was distinct when compared to youth

homicide trends in most major U.S. and New England cities (Braga et al. 2001). In a

companion paper to the main impact evaluation, Piehl et al. (2003) developed an econo-

metric model that evaluated all possible monthly break points in the time series to identify

the maximal monthly break point associated with a significant structural change in the

trajectory of the time series. Controlling for trends and seasonal variations, the timing of

the ‘‘optimal break’’ in the monthly counts of youth homicides time series was in the

summer months after Ceasefire was implemented in 1996.

Given the high profile of the Boston experience, the Ceasefire evaluation has been

reviewed by a number of researchers and the relationship between the implementation of

Ceasefire and the trajectory of youth homicide in Boston during the 1990s has been closely

scrutinized. Fagan (2002) suggested that some of the decrease in homicide may have

occurred without the Ceasefire intervention in place as violence was decreasing in most

major U.S. cities. In support of this perspective, Fagan (2002) presented a simple time-

series graph on youth gun homicide in Boston and in other Massachusetts cities that

suggested a general downward trend in gun violence may have existed before Ceasefire

was implemented. Using growth-curve analysis to examine predicted homicide trend data

for the 95 largest U.S. cities during the 1990s, Rosenfeld et al. (2005) found some evidence

of a sharper youth homicide drop in Boston than elsewhere but suggest that the small

number of youth homicide incidents precludes strong conclusions about program effec-

tiveness based on their statistical models. However, in his review of their analysis, Berk

(2005) raised a number of statistical and methodological concerns with the analysis

developed by Rosenfeld and his colleagues.

Other reviewers, however, have been more supportive of a program effect in their

reviews of the Ceasefire impact evaluation (see Cook and Ludwig 2006). Ludwig (2005)

suggested that Ceasefire was associated with a large drop in youth homicide but, given the

complexities of analyzing city-level homicide trend data, there remained some uncertainty

about the extent of Ceasefire’s effect on youth violence in Boston. Morgan and Winship’s

(2007) review of the Ceasefire evaluation concluded that the analysis was a ‘‘very high-

quality example’’ of how to conduct an interrupted time series analysis of program impact

and further noted ‘‘they offer four types of supplemental analysis … which can be used to strengthen the warrant for causal assertion’’ (p. 252).

The National Academies’ Panel on Improving Information and Data on Firearms

(Wellford et al. 2005) concluded that the Ceasefire evaluation was compelling in associ-

ating the intervention with the subsequent decline in youth homicide. However, the Panel

also suggested that many complex factors affect youth homicide trends and it was difficult

to specify the exact relationship between the Ceasefire intervention and subsequent

changes in youth offending behaviors. The Panel further observed that the Ceasefire

evaluation examined aggregate citywide data and did not provide any empirical evidence

J Quant Criminol (2014) 30:113–139 117

123

that treated gangs modified their violent behaviors after being exposed to the intervention.

In a recent article in The New Yorker (Seabrook 2009, p. 37), well-respected deterrence

scholar Professor Franklin Zimring echoed the concerns raised by the Panel by stating:

Ceasefire is more of a theory of treatment rather than a proven strategy … It’s odd that no one has ever said, O.K., here are the youths who were not part of the

Ceasefire program in Boston, let’s compare them to the youths who were. And no

one has ever followed up any long range studies of the criminal behavior of the group

that was in the program, either. We just don’t have the evidence, and until we do we

can’t evaluate how effective Ceasefire really is.

The Current Study

While the existing evidence is strong enough to suggest an association between the

implementation of Ceasefire and the subsequent drop in Boston youth homicides, we agree

with the concerns raised by the National Academies’ Panel and Professor Zimring that it is

difficult to determine whether Ceasefire actually changed violent gang behaviors in Boston

based on the analysis of aggregate citywide trend data during the 1990s—a period known

for sudden and surprising decreases in violent crime in the United States (see, e.g. Cook

and Laub 2002). A more rigorous test of Ceasefire would compare pre-test and post-test

trends in gun violence outcomes by treated Boston gangs to pre-test and post-test trends in

gun violence outcomes for an equivalent group of untreated Boston gangs. In this study, we

take advantage of new data on gangs and gang-involved gun violence in Boston to conduct

a stronger quasi-experimental evaluation of a reinvigorated version of Operation Ceasefire

implemented during the late 2000s.

Despite the national acclaim, the BPD discontinued the Ceasefire strategy as its primary

response to outbreaks of gang violence in January 2000 (see Braga and Winship 2006).

Yearly counts of gang homicides, unfortunately, increased linearly after Ceasefire was

halted in Boston (Braga et al. 2008a). In 1999, the last full year of Ceasefire intervention,

there were only 5 gang-motivated homicides in Boston. By 2006, this number had

increased more than seven-fold to 37 gang-motivated homicides in Boston. During this

time period, the BPD experimented with alternative approaches to violence prevention by

adapting certain Ceasefire tactics to a broader range of problems such as investigating

unsolved shootings, facilitating the re-entry of incarcerated violent offenders back into

high-risk Boston neighborhoods, and addressing criminogenic families in hot spot areas

(Braga and Winship 2006). Unfortunately, the slate of new approaches seemed to diffuse

the ability of the City of Boston to deal with gang violence as no one group was focused

exclusively on addressing ongoing conflicts among street gangs (Braga et al. 2008a).

At the beginning of December 2006, Edward F. Davis III, former Chief of the Lowell,

Massachusetts, Police Department, was sworn in by Mayor Thomas M. Menino as the new

Commissioner of the BPD and was immediately charged with reducing gun violence in the

city. Drawing on his past experience with a pulling levers strategy to control gang violence

in Lowell (Braga et al. 2008b), Davis announced that Operation Ceasefire would once

again be the BPD’s main response to outbreaks of serious gang violence. He promoted

Gary French, who led many of the BPD’s Ceasefire efforts during the 1990s, to Deputy

Superintendent with oversight of the Youth Violence Strike Force (YVSF, known infor-

mally as the ‘‘gang unit’’), school police unit, and the tactical bicycle unit. With the support

of Davis and his command staff, French reinstated the Ceasefire approach as a citywide,

118 J Quant Criminol (2014) 30:113–139

123

interagency effort to disrupt ongoing cycles of gang violence. Between January 2007 and

December 2010, 19 Boston gangs were subjected to the Ceasefire pulling levers focused

deterrence strategy.

Analytical Approach

We used a non-randomized quasi-experimental design to compare serious gun violence

trends for Boston gangs subjected to the Ceasefire intervention to serious gun violence

trends for a matched comparison group of Boston gangs that did not receive the Ceasefire

intervention (Shadish et al. 2002; Rossi et al. 2006). This section describes the develop-

ment of the data and units of analysis in our quasi-experiment, the identification of

comparison gangs, and the specification of appropriate statistical models to estimate the

effect of the Ceasefire intervention on serious gun violence trends for treated gangs relative

to serious gun violence trends for comparison gangs.

Data and Units of Analysis

In this study, we measured serious gun violence by using computerized records of BPD

official reports of Homicide by Firearm and Assault and Battery by Means of a Deadly

Weapon—Firearm (ABDW—Firearm) incidents between January 1, 2006 and December

31, 2010. Incident reports are generated in the BPD by detectives or police officers after an

initial response to a request for police service. In the State of Massachusetts, ABDW—

Firearm incidents essentially represent shooting events where guns were fired and victims

were physically wounded by the fired bullets. 2

The availability of non-fatal incident data

has the significant advantage of allowing us to include a wider range of gang-involved gun

violence. More importantly, the difference between a gun homicide and a non-fatal

shooting event, as one police officer related to us, ‘‘is often only a matter of inches and

luck—a lot of times a non-fatal shooting is just a failed homicide.’’ The officer’s sentiment

suggests that whether or not an event becomes lethal is contingent on several uncontrol-

lable factors—the aim of the shooter, the distance to the target, a rapid call to the police,

the response time of medical assistance, and so on. In fact, Zimring’s (1968, 1972) studies

of wounds inflicted in gun and knife assaults demonstrate considerable overlap between

fatal and non-fatal attacks and suggest that the difference between life and death is just a

matter of chance. In the text that follows, we use ‘‘shooting’’ as a term of convenience to

represent both fatal and non-fatal shooting incidents.

It is well known that police incident data, such as the Federal Bureau of Investigation’s

Uniform Crime Reports, have shortcomings. For instance, crime incident data are biased

by the absence of crimes not reported by citizens to the police and by police decisions not

to record all crimes reported by citizens (see Black 1970). Although incident reports have

flaws, careful analyses of these data can yield useful insights on crime (Schneider and

Wiersema 1990). Moreover, official police incident data are widely used for assessing

trends and patterns of gun crime (Blumstein 1995; Cook and Laub 2002) and the evalu-

ation of gun violence reduction programs (see, e.g. Sherman and Rogan 1995; McGarrell

et al. 2001; Cohen and Ludwig 2003).

2 See Massachusetts General Laws, Chapter 265, Section 15A.

J Quant Criminol (2014) 30:113–139 119

123

To determine whether a shooting event involved a gang member as a suspect, victim, or

both, the ‘‘crime incident review’’ process was used (see Klofas and Hipple 2006).

Between 2006 and 2010, the BPD’s Boston Regional Intelligence Center (BRIC) convened

separate quarterly shooting review meetings for the four policing districts (B-2, B-3, C-11,

and D-4) that experience the bulk of gun violence in Boston and one quarterly shooting

review meeting for the remaining policing districts. For each district meeting, detectives

and officers with detailed knowledge on gangs and gang violence problems were required

to attend; this included district detectives, plainclothes Anti-Crime district officers, Drug

Control Unit detectives and officers, Homicide Unit detectives, Special Investigations Unit

detectives, and YVSF detectives and officers. In each quarterly shooting review meeting,

BRIC detectives and civilian analysts presented the objective characteristics of each

shooting event (date, location, victim information, and, if arrested, offender information)

and the available gang intelligence on the event based on their computerized data systems.

The meeting participants shared their working knowledge on circumstances of the shooting

event, the relationships between victims and suspects, and, if the event involved gang

members, details on the gangs involved in the shooting.

Researchers attended the quarterly shooting review meetings and partnered with the

BRIC in collecting, coding, entering, and analyzing the qualitative insights on the nature of

each shooting event. Figure 1 presents the yearly counts of gang-involved shootings in

Boston between 2006 and 2010. Gang-involved shootings were relatively stable between

2006 (N = 263) and 2007 (N = 253), decreased over the course of 2008 (N = 232) and

2009 (N = 172), and, despite a small increase over the previous year, remained relatively

low in 2010 (N = 183). Between 2006 and 2010, gang-involved shootings in Boston

decreased by 30.4 %.

The units of analysis in this evaluation are quarterly counts of shootings by and against

specific Boston gangs between 2006 and 2010. Since shootings by and against any par-

ticular gang were relatively rare events, we aggregated specific shootings into quarterly

counts to provide more stable estimates of any measurable impacts of Ceasefire on gang

shooting behaviors. There were N = 123 gangs in Boston involved in at least one shooting

between 2006 and 2010. We analyzed three quarterly outcomes for each gang included in

the evaluation: victim gang-involved shootings, suspect gang-involved shootings, and total

gang-involved shootings (victim and suspect summed).

263 253

232

172 183

0

50

100

150

200

250

300

20102009200820072006

N u

m b

e r

o f

V ic

ti m

s

Fig. 1 Gang-involved shootings in Boston, 2006–2010

120 J Quant Criminol (2014) 30:113–139

123

Matching Treatment Gangs with Comparable Control Gangs

It is important to note here that evaluating Ceasefire is a particularly difficult task. The

Ceasefire intervention was explicitly designed to deter continued gun violence by gangs

not subjected to the treatment. As Kennedy et al. (1996, p. 181) describe in their discussion

of evaluating Ceasefire in Boston during the 1990s:

…rather than trying to protect certain areas or groups from the intervention, as in the traditional experimental design, the working group went to considerable effort to

design an intervention that would create ‘‘spillover’’ effects onto other gangs and

neighborhoods – through the communications strategy, interfering in active or nas-

cent gang vendettas, fear reduction, and the like. Thus, a traditional evaluation would

find no impact—youth homicide would fall in the targeted areas … and in all other areas of the city…

Kennedy et al. (1997, p. 240) describe how social network analysis concepts were used to

assist the diffusion of the deterrence message across Boston’s gang landscape:

We used structural network analysis in pursuit of support for an effective commu-

nications strategy. Here, [social network analysis software] was employed to identify

naturally existing subgroups, or ‘‘cliques,’’ such that talking to one member would

effectively be talking to all members [of that clique] … for clique identification, conflict and alliance networks were combined and analyzed.

The post-2007 version of Boston Ceasefire attempted to create these spillover effects

onto other gangs that were socially connected to targeted gangs through rivalries and

alliances. As Ceasefire interventions were completed on targeted gangs, the Ceasefire

Working Group directly communicated to their rivals and allies that ‘‘they would be next’’

if these groups decided to retaliate against treated rival gangs or continue shootings in

support of treated allied gangs. These messages were delivered to members of socially-

connected gangs via individual meetings with gang members under probation supervision

and through direct ‘‘street conversations’’ with gang members by BPD officers and gang

outreach workers.

One key assumption that underlies all controlled program evaluations is the ‘‘stable unit

treatment value assumption’’ (SUTVA). This assumption requires that the treatment or

control condition to which a unit is assigned has no impact on the response of another unit

(Rubin 1990). Including untreated Boston street gangs that were socially connected to

Ceasefire gangs as comparison groups in our impact evaluation would violate SUTVA. The

Ceasefire program was explicitly designed to ensure that knowledge of Ceasefire actions

taken against their immediate rivals and allies would diffuse into these untreated groups

and influence their subsequent gun violence behaviors. To minimize SUTVA violations,

we excluded all untreated Boston street gangs that were known to have a rivalry or alliance

with Ceasefire gangs from consideration as comparison groups in our quasi-experimental

evaluation. This process resulted in N = 82 gangs that were not socially connected to the

N = 19 Ceasefire gangs as possible comparison groups. 3

3 We used data from a recent social network analysis of the rivalries and alliances among Boston gangs to

identify the untreated gangs that were socially connected to the N = 19 Ceasefire gangs. Rivalries and alliances between gangs were determined through focus groups with police officers, probation officers, and streetworkers (city-employed gang outreach workers) based on their working knowledge of past and ongoing gang violence. Some gangs connected in rivalries and alliances to Ceasefire gangs also directly received treatment. For instance, the Lucerne Street Doggz had rivalries with eight other gangs and alliances

J Quant Criminol (2014) 30:113–139 121

123

We recognize that our strategy to address possible SUTVA violations is limited. The

gun violence behaviors of untreated gangs with second- and third-order social connections

to treated gangs may have been impacted through the indirect transmission of knowledge

on the consequences experienced by treated gangs. The Ceasefire intervention could have

also affected the gun violence behaviors of untreated gangs located in proximate neigh-

borhoods that were not socially connected to treatment gangs through local non-gang

social networks. In essence, these social dynamics introduce a potential bias against

establishing a statistically-significant Ceasefire treatment effect. Our analyses would then

represent a very conservative test of program impacts.

Using Stata 12.1 statistical software, we executed PSMATCH2 propensity score

matching routines (Leuven and Sianesi 2003) to develop matched comparison and treat-

ment groups from the untreated gangs and the Ceasefire gangs. Propensity score matching

techniques attempt to create equivalent treatment and comparison groups by summarizing

relevant pre-treatment characteristics of each subject into a single-index variable (the

propensity score) and then matching subjects in the untreated comparison pool to subjects

in the treatment group based on values of the single-index variable (Rosenbaum and Rubin

1983, 1985). As such, we drew upon detailed information on the characteristics of Boston

gangs from a recent investigation of the relative importance of prior conflicts and the

proximity of gang turf on gun violence outcomes. The propensity score matching routine

included the following nine characteristics:

1. Number of total shootings committed by each gang in 2006 (pre-Ceasefire). Gun

violence among Boston gangs has been previously described as perpetuated by

vendettas and ongoing series of retaliations (Kennedy et al. 1996). Gangs with higher

levels of gun violence have an increased risk of persisting in their shooting behaviors

over time.

2. Gang membership size. Gangs with larger memberships have an increased number of

members who can commit or be victimized by shootings. 4

3. Adjacency to another gang’s turf. Research suggests that gang violence is more likely

to erupt at the boundaries where gangs’ turf meet (Papachristos 2009; Tita and

Greenbaum 2009; Tita and Radil 2011). Boston gangs with turf adjacent to the turf of

another gang are more likely to be involved in serious gun violence. 5

4. Gang longevity. Gangs that have been in existence since the 1990s will have a more

stable set of rivalries and a longer history of death and injury at the hands of their

Footnote 3 continued with four other gangs. During the study period, three of their rivals (Castlegate, Morse, and Norfolk) and three of their allies (Favre, Kaos, and Orchard Park) also experienced Ceasefire interventions. N = 22 untreated gangs directly connected to Ceasefire gangs via rivalries or alliances were excluded from con- sideration for inclusion in our quasi-experimental design. The exercise resulted N = 82 gangs as possible comparison groups (123 total gangs—19 treated gangs—22 untreated gangs that were socially connected to treated gangs = 82 possible comparison gangs). 4

Gang membership size was calculated from the roster of members of each gang in the BPD BRIC’s gang intelligence database. 5

We used ArcGIS 10.0 mapping software to map the turf of Boston gangs as polygons that occupied a circumscribed amount of space. We created a matrix of turf adjacency where a tie occurs if any side of a gang polygon touches at least one side of another gang polygon.

122 J Quant Criminol (2014) 30:113–139

123

rivals; the longevity of these gangs and their ongoing disputes with rivals may increase

the likelihood of a violent dispute during the study period. 6

5. Number of rivalries with other gangs. Gangs with larger numbers of rivalries with

other gangs have an increased risk that one or more of these rivalries could turn into an

active violent dispute that would generate a string of retaliatory shootings. Retaliation

and retribution are perhaps the most frequently cited mechanisms of gang violence

(Decker 1996; Hughes and Short 2005; Papachristos 2009).

6. Number of alliances with other gangs. Similar to alliance systems in international

relations, some gangs form alliances for the benefit of mutual protection. For instance,

in a unique study on gang finances and strategy in Chicago, Levitt and Venkatesh

(2000) describe how one gang parlayed and negotiated such alliances to rally other

groups to their aid during a gang war.

7. Gang located in housing project. Research has found that housing project areas are

associated with increased levels of gang homicide relative to other city areas without

housing projects (Smith 2012).

8. The concentration of social disadvantage in each gang turf area. We included an

index that measured concentrated social disadvantage 7

in the 2000 US Census block

group(s) surrounding gang turfs to make certain that comparison gangs were selected

from neighborhoods that were similar to the neighborhoods in which the Ceasefire

gangs were located. Research reveals that the degree of concentrated social

disadvantage in a neighborhood is strongly correlated with the concentration of

violent crime (Morenoff et al. 2001; Sampson and Wilson 1995) and gang crime in

these areas (Papachristos and Kirk 2006; Rosenfeld et al. 1999).

9. Number of gang members arrested in 2006 (pre-Ceasefire). Finally, local police

departments traditionally use arrest-based enforcement strategies to suppress gang

violence (Klein 1993). Arrests of gang members could plausibly impact the likelihood

that a particular gang engages in gun violence through the removal of likely

‘‘shooters’’ from the street.

We recognize that it would have been ideal to include a greater number of covariates in

our final propensity score matching model. Indeed, the ability to balance treatment and

comparison groups on as many covariates as possible is the main strength of propensity

score methods. Unfortunately, these nine covariates represented the only group-level

descriptors for Boston gangs available at the time of this analysis. Nevertheless, we believe

6 Longevity was determined by comparing the roster of N = 123 gangs with at least one shooting during

the 2006–2010 study time period to the roster of active Boston gangs in 1995 identified by Kennedy et al. (1997). 7

The concentrated disadvantage index is a standardized index composed of the percentage of residents who are black, the percentage of residents receiving public assistance, the percentage of families living below the poverty line, the percentage of female-headed households with children under the age of 18, and the percentage of unemployed residents (as measured by the percentage of men over the age 16 who did not work in the previous year) (see Morenoff et al. 2001; Sampson et al. 1997). Because of the high correlation of these variables, we conducted principal components factor analysis, which revealed that variables load on a single factor (which was retained as a standardized index variable). For example, a Boston block group featuring a disadvantage index score of 1.5 would be 1.5 SD more disadvantaged than the mean Boston block group. As such, the disadvantage index is adjusted specifically for the city of Boston using 2000 Census variables, even while the components used to construct the index remain constant across much neighborhood research and remain robust predictors of crime across a variety of city types and spatial aggregations. For those gangs whose turf spanned more than one census block group, we used a spatially- weighted mean of the connected block groups to calculate the disadvantaged index for the neighborhood surrounding each gang’s turf.

J Quant Criminol (2014) 30:113–139 123

123

our parsimonious propensity score model captures the gang-level covariates most directly

associated with gun violence behaviors that would influence the selection of particular

gangs for Ceasefire treatment. As we describe in detail below, our impact analysis was not

affected by unobserved variables that could simultaneously affect gang assignment to the

Ceasefire treatment and gun violence outcomes.

The broader propensity score matching literature identifies a wide variety of matching

algorithms with different choices that need to be made when each approach is used (see,

e.g. Apel and Sweeten 2010; Heckman et al. 1997; Imbens 2004; Smith and Todd 2005).

We selected radius matching with a caliper = 0.01 as our primary propensity score

matching algorithm. According to Dehejia and Wahba (2002), the basic idea of this variant

is to use not only the nearest neighbor within each caliper but all of the comparison

members within the caliper. A benefit of radius matching is that the approach uses only as

many comparison units as are available within the caliper and therefore allows for usage of

additional units when good matches are available or fewer units when good matches are

not available (Caliendo and Kopeinig 2005). As such, the approach minimizes the risk of

bad matches.

Table 1 reports the results of the propensity score radius matching with a cali-

per = 0.01. The table presents the pre- and post-matching t tests and the standardized bias

statistics which represents the mean difference as a percentage of the average standard

deviation between the groups (Rosenbaum and Rubin 1985). In the matched sample, all

p values are higher than 0.05, and all bias statistics are less than 20.0 (a general ‘rule of

thumb’ for balanced groups; see also Austin et al. 2007). 8

This confirmed that we achieved

balanced treatment and comparison groups. PSMATCH2 radius matching (caliper = 0.01)

routine revealed that the 16 matched Ceasefire treatment gangs and 37 matched compar-

ison gangs were in the common support region. This ensures that gangs with the same

X values have a positive probability of being both treated and untreated (Heckman et al.

1999).

Growth-Curve Regression Model Specification

We use a variation of a multi-level negative binomial regression model in order to analyze

the quarterly change in gang-involved shootings for treatment and comparison gangs over

a 5-year observation period (2006–2010, N = 20 quarters). 9

More specifically, we

8 For balancing properties to be satisfied in the propensity score matching analysis, certain pre-treatment

characteristics needed to be entered as dummy variables into the Stata 12.0 PSMATCH2 routine. The total number of shootings in 2006 and the number of members of each gang were entered as interval-level measures. Adjacent gang turf was coded ‘‘0’’ for gangs that did not have turf adjacent to another gang’s turf and ‘‘1’’ for gangs that did have turf adjacent to another gang’s turf. Longevity was coded ‘‘0’’ for gangs that did not exist in 1995 and ‘‘1’’ for gangs that did exist in 1995. The number of rivalries was coded as ‘‘0’’ for gangs that had 2 or fewer rivalries and ‘‘1’’ for gangs that had 3 or more rivalries. The number of alliances was coded as ‘‘0’’ for gangs that had no alliances and ‘‘1’’ for gangs that had alliances with at least one other gang. Housing project gang was coded as ‘‘0’’ for gangs not located in a housing project and ‘‘1’’ for gangs were located in a housing project. The concentration of disadvantage in the surrounding Census block group(s) was coded as ‘‘0’’ for gang turf located in block groups below the 75th percentile and ‘‘1’’ for gang turf located in block groups at the 75th percentile or greater. The number of gang arrests in 2006 was coded as ‘‘0’’ for gangs with 14 or fewer arrests in 2006 and ‘‘1’’ for gangs with 15 or more arrests in 2006. 9

The quarterly total gang-involved shootings for the N = 53 treatment and comparison gangs used in these analyses were distributed as overdispersed count data. The distribution had a mean = 1.39, standard deviation = 1.89, and variance = 3.57. One sample Kolmogorov–Smirnov nonparametric tests rejected the null hypotheses that the observed distribution was not different from a normal distribution (p \ 0.0001) and not different from a Poisson distribution (p \ 0.0001).

124 J Quant Criminol (2014) 30:113–139

123

developed individual growth curve models to estimate street gang changes in violent index

crime incidents over the observation period (Gelman 2005; Singer and Willet 2003). Here

we used a longitudinal negative binomial model where we predict within unit variation at

level 1 and between unit variation at level 2 using level 1 intercepts and slopes as out-

comes. In non-technical terms, we are interested in accurately analyzing the overall

shooting trend of each of the street gangs during the observation period. Each street gang is

also allowed to have its own slope and intercept in order to model different starting levels

of shootings as well as different rates of change. This is consistent with the variation

observed in shootings by gangs—some groups are highly active and others are less active.

Formally, the model is specified as follows where yit is the count for the tth observation

in the ith group. The model begins with yitj�Poisson citð Þ where citj�gamma kit; dið Þ with

Table 1 Balancing treatment and comparison gangs through propensity score matching

Characteristics Treated Untreated % Bias % Bias reduction t test p [ |t|

Total shootings in 2006

Unmatched 8.368 3.683 53.8 2.73** 0.007

Matched 9.562 8.374 13.6 74.7 0.33 0.746

Gang size

Unmatched 41.375 31.920 31.7 1.17 0.247

Matched 42.267 39.365 9.7 69.3 0.26 0.794

Adjacent gang turf

Unmatched 0.526 0.471 10.9 0.44 0.661

Matched 0.667 0.644 4.4 59.1 0.13 0.901

Longevity

Unmatched 0.625 0.540 16.9 0.61 0.543

Matched 0.600 0.586 2.8 83.6 0.07 0.941

More than 2 rivalries

Unmatched 0.577 0.251 69.6 2.95** 0.004

Matched 0.751 0.699 11.0 84.2 0.27 0.787

1 or more gang alliances

Unmatched 0.684 0.416 55.0 2.20* 0.030

Matched 0.733 0.804 -14.7 73.3 -0.45 0.656

Housing project gang

Unmatched 0.211 0.173 9.4 0.39 0.698

Matched 0.250 0.269 -4.8 48.6 -0.12 0.905

High disadvantage in census block group

Unmatched 0.571 0.254 66.4 2.35* 0.021

Matched 0.500 0.524 -5.1 92.4 -0.11 0.911

15 or more gang arrests in 2006

Unmatched 0.578 0.291 59.5 2.51* 0.013

Matched 0.667 0.675 -1.8 96.9 -0.05 0.960

N = 53 (16 treated gangs, 37 comparison gangs)

Radius matching propensity score model (caliper = 0.01)

* p \ 0.05 ** p \ 0.01

J Quant Criminol (2014) 30:113–139 125

123

kit ¼ expðxitb þ offsetitÞ and di represents the dispersion parameter. This produces the following equation:

Pr Yit ¼ yitjxit; dið Þ¼ C kit þ yitð Þ

C kitð ÞC yit þ 1ð Þ 1

1 þ di

� �kit di 1 þ di

� �yit ð1Þ

Following Gelman (2005) and others (Long and Freese 2006; Singer and Willet 2003),

this specification yields a negative binomial model for the ith group with dispersion equal

to 1 ? d, in other words, a constant dispersion within groups. Thus, we feel that such a specification fits the observed distribution of our data.

For a random-effects over-dispersion model, d varies randomly across observational

units. We therefore assume that 1 1þd

i

� � �Betaðr; sÞ. Accordingly, the joint probability of

the counts for the ith group is:

Pr Yi1 ¼ yi1;. . .;Yini ¼ yinijXið Þ¼ Z 1

0

Yni t¼1

Pr Yit ¼ yitjxit;dið Þf dið Þddi

¼ C rþsð ÞCðrþ

P ni t¼1kitÞCðsþ

Pni t¼1 yitÞ

CðrÞCðsÞCðr þsþ Pni

t¼1 kit þ Pni

t¼1 yitÞ Yni t¼1

Cðkit þyitÞ CðkitÞCðyit þ1Þ

ð2Þ For Xi ¼ðxi1; . . .; xinÞ and where f is the probability density for di. This yields the

following log likelihood:

ln L ¼ Xn i¼1

wi ln Cðr þ sÞþ ln C r þ Xni k¼1

kik

! þ ln C s þ

Xni k¼1

yik

! � ln CðrÞ¼ ln CðsÞ

"

� ln C r þ s þ Xni k¼1

kik þ Xni k¼1

yik

! þ Xni t¼1

ln C kit þ yitð Þ� ln C yit þ 1ð Þf g #

ð3Þ Following these equations, our final model is as follows:

Yij ¼ ai þ b1iðCeasefireÞþ b2iðperiodÞþ b3iðimpactÞþ b4iðtrendÞþ b5iðtrend2Þ þ b6iðquarter2Þþ b7iðquarter3Þþ b8iðquarter4Þþ b9iðiptwÞ ð4Þ

where the quarterly counts of total gang-involved shooting incidents over the 5-year study

time period was our primary outcome measure (Yij). However, in addition to our simple

effect size analyses, we also analyzed changes in the quarterly counts of victim gang-

involved shootings and the quarterly counts of suspect gang-involved shootings. To esti-

mate the effect of the Ceasefire treatment, we created a dichotomous dummy variables

indicating whether a street gang was in the treatment group (1) or in the comparison group

(0) (Ceasefire) and whether the quarter was pre-intervention (0) or during the intervention

period (1) (period). We then created a differences-in-differences (DID) estimator by inter-

acting these two dummy variables (impact).

To account for secular linear and nonlinear quarterly trends in the dependent variable,

we included a variable that was measured as the simple linear additive progression for each

quarter over the course of the 5-year observation period (trend) and a variable that squared

this simple linear additive progression for each quarter (trend2). We also controlled for

seasonal variations in the quarterly counts of shootings by including a polychotomous

126 J Quant Criminol (2014) 30:113–139

123

dummy variable (quarter2, quarter3, and quarter4). 10

We estimated the growth curve

regression models with the inverse-weighted propensity score value (1/p) for each of the

treatment and comparison gangs (represented in the above equation by iptw). The inclusion

of this covariate controlled for observable differences between the gangs in the treatment

and comparison groups given the covariates used to calculate the propensity score (Imbens

and Wooldredge 2009).

The XTNBREG command in Stata 12.1 statistical software was used to calculate the

maximum likelihood estimate of the parameters for the DID estimator and to compute the

associated probability values; this provided estimates of the effects of the Ceasefire

intervention on the treatment gangs as relative to the comparison gangs. The parameter

estimates were expressed as incidence rate ratios (i.e., exponentiated coefficients). Inci-

dence rate ratios are interpreted as the rate at which things occur; for example, an incidence

rate ratio of 0.90 would suggest that, controlling for other independent variables, a one unit

increase in the selected independent variable was associated with a 10 % decrease in the

rate at which the dependent variable occurs. Following social science convention, the two-

tailed 0.05 level of significance was selected as the benchmark to reject the null hypothesis

of ‘‘no difference.’’

Results

Simple Pre-Post Analysis of Matched Ceasefire Gangs and Matched Comparison

Gangs

Figure 2 presents the yearly mean total gang-involved shootings between 2006 and 2010

for the 16 matched Ceasefire gangs and the 37 matched comparison gangs. During the

study time period, the yearly mean total gang-involved shootings per Ceasefire gang

decreased by 57.3 % from 9.6 shootings in 2006 to 4.1 shootings in 2010. In contrast, the

yearly mean total gang-involved shootings per comparison gang decreased by only 20.2 %

from 8.4 shootings in 2006 to 6.7 shootings in 2010. Consistent with the trends in yearly

mean total gang-involved shootings, the Ceasefire gangs experienced larger decreases in

both yearly mean suspect and victim gang-involved shootings relative to the comparison

gangs. Between 2006 and 2010, yearly mean suspect gang-involved shootings per

Ceasefire gang decreased by 60.7 % (from 5.6 to 2.2) and yearly mean victim gang-

involved shootings per Ceasefire gang decreased by 52.5 % (from 4.0 to 1.9); in contrast,

yearly mean suspect gang-involved shootings per comparison gang decreased by 23.3 %

(from 4.3 to 3.3) and yearly mean victim gang-involved shootings per comparison gang

decreased by 17.1 % (from 4.1 to 3.4).

Standardized mean difference effect size statistics were used to determine whether the

shooting reductions observed for the treated Ceasefire gangs were significantly larger than

the shooting reductions observed for the comparison gangs. The standardized mean-dif-

ference effect size (d) is designed for contrasting two groups on a continuous dependent

variable (Lipsey and Wilson 2001). For this simple analysis, we calculated the mean Time

10 Quarter 1 served as the reference category for this polychotomous dummy variable. Quarter 1 represented

whether the outcome included the sum of January, February, and March shootings (1 = Yes, 0 = No). Quarter 2 represented whether the outcome included the sum of April, May, and June shootings (1 = Yes, 0 = No). Quarter 3 represented whether the outcome included the sum of July, August, and September shootings (1 = Yes, 0 = No). Quarter 4 represented whether the outcome included the sum of October, November, and December shootings (1 = Yes, 0 = No).

J Quant Criminol (2014) 30:113–139 127

123

2 (year 2010) minus Time 1 (year 2006) gain score, the SD of the gain score, and the

correlation between the Time 1 and Time 2 scores for the matched 16 Ceasefire gangs and

the 37 matched comparison gangs. These statistics were entered into David B. Wilson’s

Practical Meta-Analysis Effect Size Calculator to estimate the standard mean difference

effect sizes. 11

For total gang-involved shootings, the Ceasefire intervention was associated

with a large, statistically-significant standardized mean difference effect size favoring

treatment conditions over control conditions (d = -0.7678; 95 % CI = -1.4221,

-0.1136; v = 0.1114). For suspect gang-involved shootings, the Ceasefire intervention

was associated with a larger statistically-significant standardized mean difference effect

size favoring treatment conditions over control conditions (d = -0.869; 95 % CI =

-1.6022, -0.1358; v = 0.1339). While the statistic suggested a beneficial impact on

victim gang-involved shootings, the standardized mean difference effect size was modest

and not statistically significant (d = -0.4799; 95 % CI = -1.1807, 0.2209; v = 0.1278).

Growth Curve Regression Model and Sensitivity Analysis Results

Table 2 presents the results of the growth curve regression models. Controlling for the

other covariates, the Ceasefire intervention was associated with a statistically-significant

30.8 % reduction (p \ 0.05) in quarterly total gang-involved shootings, a statistically- significant 34.7 % reduction (p \ 0.05) in quarterly suspect gang-involved shootings, and a statistically-significant 26.9 % reduction (p \ 0.05) in quarterly victim gang-involved shootings for the treatment gangs relative to the comparison gangs. The Ceasefire dummy

variable was not statistically significant (p \ 0.05) for all three outcome variables, con- firming that the matched groups were comparable on the gun violence outcome measures

controlling for the other covariates. For all three outcome variables, the growth curve

regression models revealed that Boston gang-involved shootings had statistically-sig-

nificant seasonal variations; relative to January through March quarterly gang-involved

shooting counts (Quarter 1), April through June (Quarter 2) and July through September

(Quarter 3) experienced higher counts of gang-involved shootings (p \ 0.01). As expected, the inverse propensity score had a statistically-significant negative association with the

9.1

6.8

4.1

5

9.6

6.2

7.48 8.4

6.7

0

2

4

6

8

10

12

20102009200820072006M e a n

T o

ta l S

h o

o ti

n g

s P

e r

G a n

g

Ceasefire

Matched Ceasefire Gangs (N=16) Matched Comparison Gangs (N=37)

Fig. 2 Mean gang-involved shootings for matched ceasefire gangs and matched comparison gangs

11 http://gemini.gmu.edu/cebcp/EffectSizeCalculator/d/mean-gains-scores-and-gain-score.html.

128 J Quant Criminol (2014) 30:113–139

123

three gang-involved shooting outcome variables (p \ 0.01).12 This suggests that Boston gangs with higher levels of shootings were more likely to be included in the quasi-

experimental analysis.

While the propensity score matching process ensures balance on observed confounders,

unobserved variables could simultaneously affect assignment into treatment and the out-

come (Rosenbaum 2002). This hidden bias would alter our inferences about Ceasefire

treatment effects. For instance, our propensity score model did not include information

about the organizational structure of Boston gangs. A recent study by Decker et al. (2008)

demonstrates that even modest increases in organizational structure are correlated with

increases in patterns of victimization and offending. To examine the robustness of our

results against possible hidden bias, we used the bounding approach proposed by

Rosenbaum (2002) via the RBOUNDS user-written routine in Stata 12.1 (DiPrete and

Gangl 2004). The Rosenbaum bounds techniques allows researchers to determine how

strongly an unobserved variable must influence the selection process to alter inference

about treatment effects. No hidden bias is represented when bound estimate C = 1.

Table 2 Ceasefire impacts on gang-involved shooting incidents: growth curve regression models

Shooting suspect Shooting victim Total shooting

Ceasefire impact (interaction) 0.653 (0.117)* 0.731 (0.101)* 0.692 (0.108)*

Ceasefire gang (1 = treated) 1.167 (0.139) 1.031 (0.098) 1.099 (0.109)

Period (1 = intervention) 0.681 (0.173) 0.816 (0.211) 0.756 (0.152)

Trend 0.917 (0.055) 0.993 (0.063) 0.940 (0.045)

Trend-squared 1.004 (0.002) 0.999 (0.003) 1.002 (0.002)

Quarter 2 1.504 (0.181)** 1.364 (0.177)* 1.463 (0.143)**

Quarter 3 1.389 (0.177)** 1.376 (0.187)* 1.401 (0.145)**

Quarter 4 0.763 (0.115) 0.982 (0.149) 0.859 (0.102)

Inverse propensity score 0.984 (0.007)** 0.976 (0.007)** 0.978 (0.006)**

Constant 3.495 (1.001)** 2.644 (0.787)* 3.275 (0.722)**

Log likelihood -1,123.031 -1,069.628 -1,552.587

Wald v2 108.12** 58.82** 123.36**

Wald df 9 9 9

Observations (gangs 9 quarters) 1,060 1,060 1,060

Number of gangs 53 53 53

Coefficients expressed as incidence rate ratios. SE are in parentheses. Quarter 1 is the reference category for the seasonal dummy variable

* p \ 0.05 ** p \ 0.01

12 Since the selection of a matching algorithm and its particular specification can be a subjective process

(Apel and Sweeten 2010), we conducted a supplementary analysis to ensure that any program impacts were robust across a variety of matching algorithms and caliper/bandwidth selections. This exercise was not intended to be an exhaustive examination of all possible propensity score methods. As such, we included a representative selection of approaches: radius matching (calipers = 0.1, 0.01, 0.001), Gaussian kernel matching (bandwidth = 0.1, 0.01, 0.001), Epanechnikov kernel matching (bandwidth = 0.1, 0.01, 0.001), stratification matching (10 strata), and simple nearest neighbor matching. While the estimates differed somewhat across the varying propensity score matching methods, the Ceasefire treatment effect remained robust. The Ceasefire impact estimates ranged from a statistically significant 28 % reduction (p \ 0.05) to a statistically significant 35 % reduction (p \ 0.05).

J Quant Criminol (2014) 30:113–139 129

123

Underestimated or overestimated treatment effects that may be due to unobserved con-

founding are represented by C bound estimates higher than 1. A scenario of C = 1.50 suggests that hidden bias would increase the odds of receiving Ceasefire treatment for

gangs actually receiving Ceasefire treatment by 50 % relative to gangs that did not receive

Ceasefire treatment.

Table 3 presents the results of our Rosenbaum bounds sensitivity analysis. Given the

direction of the estimated Ceasefire treatment effect, our analysis focused on negative self-

selection of gangs into the treatment. Positive self-selection of gangs would simply cause

our findings to be conservative. The p-critical values represent the bound of the signifi-

cance level of the treatment effect in the case of endogenous selection into treatment status

(DiPrete and Gangl 2004). The results show that the critical level of C at which the estimated Ceasefire treatment effect would no longer be statistically significant at the 5 %

level is 1.45 for total gang shootings, 1.55 for suspect gang shootings, and 1.40 for victim

gang shootings. 13

Our conclusion that gun violence involving gangs that received the

Ceasefire treatment was significantly lower than gun violence involving gangs that did not

receive Ceasefire treatment would be challenged if an unobserved variable increased the

odds that Ceasefire gangs received the Ceasefire treatment by 45 % for total shootings, by

55 % for suspect shootings, and by 40 % for victim shootings.

Table 3 also presents the magnitude of the hidden bias that would cause us to revise our

findings of the causal effects of Ceasefire on gang shootings. Hidden bias equivalents were

calculated at the mean of the covariates for 2010 gang shootings. For total gang shootings

in 2010, the critical level of C = 1.45 is attained at a difference of 4.57 shootings per gang. The Ceasefire average treatment effect on treated (ATT) is -5.21 (SE = 2.41, p \ 0.05) for 2010 total gang shootings.

14 The unobserved variable would have to produce a dif-

ference of similar magnitude to the Ceasefire treatment effect in order to alter our con-

clusions. While these results convey important information about the level of uncertainty

contained in matching estimators by showing how large a confounding variable must be to

undermine the conclusions of our matching analysis, it is important to note that Rosenbaum

bounds represent a ‘‘worst case’’ scenario (DiPrete and Gangl 2004). As such, these

Table 3 Sensitivity analysis: Rosenbaum bounds for Ceasefire treatment effect

Hidden bias equivalents were calculated at the mean of the observed covariates for 2010 gang shootings

Ceasefire treatment effect C p-critical Hidden bias equivalent

Total gang-involved shootings 1.40 0.050 -4.41

1.45 0.055 -4.57

Gang suspect shootings 1.50 0.048 -2.52

1.55 0.054 -2.68

Gang victim shootings 1.35 0.046 -1.73

1.40 0.053 -1.88

13 A value of C = 1.45 for total gang shootings indicates that the confidence interval for the Ceasefire

treatment effect would include zero if an unobserved variable caused the chance of treatment assignment to differ between treatment and control groups by 1.45 and if this variable’s effect on total shootings was so strong as to almost perfectly determine whether total shootings would be bigger for the treatment or the control gang in each pair of matched gangs in the data (see DiPrete and Gangl 2004). 14

Similar conclusions can be drawn by comparing the Rosenbaum bounds results to the ATT models for 2010 suspect shootings (ATT = -3.54, SE = 1.73, p \ 0.05) and 2010 victim shootings (ATT = -2.11, SE = 1.24, p \ 0.10). For all three ATT models (radius matching, caliper = 0.01), bootstrapped standard errors with 100 replications are provided.

130 J Quant Criminol (2014) 30:113–139

123

analyses suggest that our propensity score matching estimators are robust to hidden bias

caused by an unobserved confounder.

Supplementary Analysis of the Timing of Treatment and

Observed Reductions in Gang Shootings

We selected January 2007 as the start date of the reinvigorated Operation Ceasefire

strategy because it represented the first full month of a regime change in the BPD that

delivered a fully-implemented program. Given the complex and intensive work required to

implement a focused deterrence intervention on an individual gang, it was simply not

possible for the Ceasefire Working Group to address the persistent violent behavior gen-

erated by all treated gangs at the same point in time. The Ceasefire intervention was

applied to 9 gangs in 2007, 6 gangs in 2008, and 1 gang in 2009. As such, the actual

delivery of the intervention to treated gangs occurred in a staggered manner during the

post-intervention time period. The overall dosage of Ceasefire intervention to Boston gangs

increased during the post-intervention period as suggested by the linear decrease in yearly

total shootings by treated gangs in Fig. 2.

To make a direct link between the application of the Ceasefire treatment and subsequent

changes in violent gang behavior, we conducted an exploratory analysis to identify abrupt

statistically-significant reductions, known as structural breakpoints, in quarterly total gang-

involved shootings for each of the 16 matched Ceasefire gangs. Using the NBREG com-

mand in Stata 12.1, we ran a series of 18 negative binomial regressions for each Ceasefire

gang with a varying quarterly intervention point between Quarter 2 and Quarter 19 that

included controls for secular trends and quarterly seasonal variations. Dummy variables

(0 = pre-intervention, 1 = intervention) were used to estimate the adjusted pre-post mean

difference in total shootings by and against each Ceasefire gang for each of the 18 quarters

between Quarter 2 and Quarter 19. 15

A sharp and sustained break in the quarterly shooting

time series will lead to significant before and after differences for several time periods

around the intervention. This is because these structural breakpoint analyses involve, in

essence, comparisons of two means adjusted for other factors (see Piehl et al. 2003).

However, if Ceasefire did produce the desired impact, the maximal structural breakpoint in

each time series should coincide with the quarter when treatment was applied or in the

quarter immediately following the treatment application.

We reviewed official records maintained by the BPD on Ceasefire actions during the

study time period to determine the specific quarter that the treatment was fully imple-

mented. Ceasefire was considered fully implemented for a targeted gang when three

components were present: (1) direct communications with the gang had occurred, (2) social

services and opportunities were available to gang members who wanted them, and (3) a

customized law enforcement response was delivered. We illustrate our structural break-

point analyses by presenting the details of this exercise for the first gang to receive the full

Ceasefire treatment under the new regime.

The Lucerne Street Doggz was the first group selected for Ceasefire intervention

because it was the most violent gang in Boston at the beginning of the study time period.

The Doggz were a loosely-organized gang based in the disadvantaged Lucerne Street area

of the Mattapan section of Boston (District B-3). In 2006, the Lucerne gang had roughly 50

members and was involved in violent disputes with eight rival gangs—Big Head Boys,

15 We excluded Quarter 1 and Quarter 20 to ensure that our quarterly impact estimates were based on at

least two quarters (6 months) of shooting data for each Ceasefire gang.

J Quant Criminol (2014) 30:113–139 131

123

Morse Street, Norfolk, Greenwood, Heath Street, Orchard Park, H-Block, and Winston

Road. Lucerne was the suspect group in 30 gang-involved shootings and the victim group

in 7 gang-involved shootings in 2006. BRIC intelligence suggested that most of the

Lucerne shootings, which accounted for nearly 10 % of all Boston shootings in 2006, were

carried out by no more than 6 or 7 members of the gang.

In late 2006, BPD District B-3 detectives and officers decided to implement a Ceasefire

intervention to address the persistent shootings generated by Lucerne. They partnered with

the U.S. Attorney’s Office, Suffolk County District Attorney’s Office, Boston School

Police, Massachusetts Department of Youth Services, Massachusetts Department of Pro-

bation, Boston Ten Point Coalition, Boston Centers for Youth and Families streetworkers,

Youth Service Providers Network (social work program) and Youth Opportunities Boston

(non-profit employment development agency) on a ‘‘call-in’’ to deliver the Ceasefire anti-

violence message. On November 14, 2006, 22 members of the Lucerne Street Doggz

attended the call-in; 11 members made appointments with Youth Opportunities Boston to

explore job placement options and 7 members requested follow-up meetings with Youth

Service Providers Network counselors. Unfortunately, since the BPD was not fully

invested in the Ceasefire approach, Lucerne did not face any enhanced enforcement

response to their continued violent behavior after the call-in. BPD participation in the

Lucerne Street effort was limited to a handful of B-3 detectives and officers; the citywide

YVSF and the Drug Control Unit were not involved in this initiative. After a relatively

quiet winter period, Lucerne continued its torrid involvement in shootings and, by the end

of May 2007, was the suspect group in another 21 gang-involved shootings and the victim

group in another 6 gang-involved shootings.

As described earlier, in December 2006, newly-appointed Commissioner Davis man-

dated that Ceasefire needed to be the BPD’s marquee response to ongoing gang violence.

In January 2007, then-Deputy Superintendent Gary French, who was charged by Davis to

coordinate the citywide implementation of Ceasefire, started regular meetings of the

interagency Operation Ceasefire working group. It was critical to establish the credibility

of the Ceasefire anti-violence message on the streets of Boston again. Since Lucerne had

been subjected to a call-in and continued on its violent path, the Ceasefire working group

needed to make good on the promise that a strong enforcement response would soon

follow. With the support of the Drug Control Unit and District B-3 personnel, the YVSF

worked with the U.S. Attorney’s Office, Suffolk County District Attorney’s Office, Drug

Enforcement Administration and Bureau of Alcohol, Tobacco, Firearms, and Explosives in

a focused investigation of the Lucerne Street Doggz. On May 24, 2007, 25 Lucerne Street

gang members were taken into custody and charged with federal and state drug and

firearms offenses (Ellement 2007). As Fig. 3 reveals, the impact of the Ceasefire inter-

vention on their gun violence behavior was noteworthy. In 2006 and 2007, Lucerne gang

averaged 33.5 total shootings per year. Their yearly average plummeted by 87.2 % to 4.3

per year between 2008 and 2010.

Table 4 presents a summary assessment of the timing of Ceasefire interventions and

maximum quarterly total shooting reductions for the 16 matched treatment gangs. Since

this was an exploratory analysis of only 20 quarterly observations for each gang, we

relaxed our benchmark to reject the null hypothesis of ‘‘no difference’’ to the less

restrictive p \ 0.10 level. The key components of Ceasefire intervention on the Lucerne Street Doggz—direct communications with the gang, offers of services and opportunities,

and the delivery of an enhanced enforcement response—were in place in Quarter 6 (April–

June 2007). The table shows that the maximum statistically-significant reduction

132 J Quant Criminol (2014) 30:113–139

123

30

22

3 4 3

7

8

1 1

1

0

5

10

15

20

25

30

35

40

20102009200820072006

N u

m b

e r

o f

S h

o o

ti n

g s

Suspect Victim

Lucerne Operation Completed May 24, 2007

Fig. 3 Total shootings involving Lucerne Street Doggz, 2006–2010

Table 4 The timing of ceasefire interventions and maximum shooting reductions for 16 matched treatment gangs

Treatment gangs Ceasefire quarter Max. reduction quarter Ceasefire coef. (SE) Effect?

Lucerne Apr–Jun 07 Jul–Sep 07 -0.65 (0.30)* Yes

Morse Apr–Jun 07 Jul–Sep 07 -0.71 (0.48) ?

Yes

Favre Apr–Jun 07 Jul–Sep 07 -1.29 (0.58)* Yes

Norfolk Jul–Sep 07 Jul–Sep 07 -0.73 (0.29)** Yes

Kaos Jul–Sep 07 Jan–Mar 08 -0.53 (0.98) No

Castlegate Jul–Sep 07 Oct–Dec 07 -3.00 (1.20)** Yes

Everton/Geneva Jul–Sep 07 Oct–Dec 07 -2.39 (0.82)** Yes

Greenfield Jul–Sep 07 Jul–Sep 07 -1.94 (0.85)** Yes

Heath Jul–Sep 07 Jul–Sep 07 -0.83 (0.42)* Yes

St. James Jan–Mar 08 Apr–Jun 08 -1.04 (0.47)* Yes

H-Block Jan–Mar 08 Apr–Jun 08 -0.54 (0.31) ?

Yes

Wood Ave Jan–Mar 08 Apr–Jun 08 -0.44 (0.28) No

Orchard Park Apr–Jun 08 Jul–Sep 08 -0.75 (0.41) ?

Yes

Forest Hills Jul–Sep 08 Oct–Dec 08 -0.51 (0.30) ?

Yes

Wainwright Park Oct–Dec 08 Apr–Jun 09 -0.87 (0.59) No

Annunciation/mission Apr–Jun 09 Jul–Sep 09 -3.19 (1.03)** Yes

N = 20 quarters per gang

Negative binomial regression models controlling for simple linear trends and seasonal variations were used to identify the maximal break point in each of the 16 time series. The models suggested a statistically- significant Ceasefire impact in 13 of the 16 gang-involved total shooting time series (binomial sign test proportion = 0.8125, two-tailed p = 0.0213) ?

p \ 0.10 * p \ 0.05 ** p \ 0.01

J Quant Criminol (2014) 30:113–139 133

123

(p \ 0.05) in the quarterly counts of total shootings for Lucerne occurred in Quarter 7 (Jul–Sep 07) of the time series.

As Table 4 reveals, 13 of the 16 matched treatment gangs experienced their largest

statistically-significant reduction in total shootings in the same quarter as or the quarter

immediately following the full implementation of Ceasefire. To test whether this distri-

bution of ‘‘successes’’ relative to ‘‘failures’’ was significantly different than what would be

expected by chance, we used an application of the binomial distribution known as the sign

test (Blalock 1979). This test examines the probabilities of getting an observed proportion

of successes from a population of equal proportions of successes and failures. The

observed distribution binomial sign test proportion = 0.8125 (13/16) with a two-tailed

p = 0.0213. This suggests that the observed relationship between the implementation of

Ceasefire and the timing of the largest statistically-significant reductions was not generated

by a random process. In other words, Ceasefire generated noteworthy changes in the gun

violence behaviors of targeted gangs during the post-intervention time period.

Conclusions

There is a growing body of evidence that focused deterrence strategies, such as the pulling

levers approach pioneered by Operation Ceasefire in Boston, generate significant crime

reduction benefits. A recently completed Campbell Collaboration review of 11 controlled

evaluations found that focused deterrence strategies were associated an overall statistically

significant, medium-sized crime reduction effect (Braga and Weisburd 2012). This review

considered replications of the Boston Ceasefire program in five other jurisdictions,

including Cincinnati (Engel et al. 2011), Indianapolis (Corsaro and McGarrell 2009;

McGarrell et al. 2006), and Los Angeles (Tita et al. 2004). Indeed, the available scientific

evidence suggests that cities suffering from gang and criminally-active group violence

should experiment with pulling levers focused deterrence strategies.

Our quasi-experimental evaluation estimated that the reconstituted Boston Ceasefire

intervention generated a 31 % reduction in total shootings for treated gangs relative to total

shootings for matched comparison gangs. Relative to matched comparison gangs, matched

treatment gangs committed significantly fewer shootings and experienced significantly

lower levels of violent gun victimization. However, it is important note that this evaluation

yielded a much more conservative violence reduction estimate when compared to the two-

thirds reductions in youth homicides reported in the original Ceasefire quasi-experimental

evaluation (Braga et al. 2001; Piehl et al. 2003). While the biases in quasi-experimental

research are not clear (e.g. Campbell and Boruch 1975; Wilkinson and Task Force on

Statistical Inference 1999), recent reviews in crime and justice suggest that weaker

research designs often lead to more positive outcomes (e.g. see Weisburd et al. 2001;

Welsh et al. 2011). 16

16 Using the Maryland Scientific Methods Scale (Sherman et al. 1997) as a standard, the original Ceasefire

impact evaluation would be considered a ‘‘Level 3’’ evaluation and also regarded as the minimum design that is adequate for drawing conclusions about program effectiveness. This design rules out many threats to internal validity such as history, maturation/trends, instrumentation, testing, and mortality. However, as Farrington et al. (2002) observe, the main problems of Level 3 evaluations center on selection effects and regression to the mean due to the non-equivalence of treatment and control conditions. This evaluation of Ceasefire would be considered a ‘‘Level 4’’ evaluation as it measures outcomes before and after the program in multiple treatment and control condition units. These types of designs have better statistical control of extraneous influences on the outcome and, relative to lower level evaluations, deals with selection and regression threats more adequately.

134 J Quant Criminol (2014) 30:113–139

123

Importantly, this study also provides some much needed evidence to address some of

the well-thought out concerns over the original Boston Ceasefire evaluation raised by the

National Academies’ Panel on Improving Information and Data on Firearms (Wellford

et al. 2005) and by Professor Zimring. Our analyses showed that Boston gangs subjected to

the post-2007 Ceasefire treatment did indeed change their gun violence behaviors relative

to Boston gangs that did not receive Ceasefire treatment. Our study also represents an

important advance over other focused deterrence evaluations that examined aggregate

citywide changes in group behavior. In Indianapolis (Corsaro and McGarrell 2009) and

Cincinnati (Engel et al. 2011), evaluators compared citywide gang and criminally-active

group homicide trends, respectively, to citywide non-gang and non-criminally-active group

homicide trends, respectively. These evaluations did not distinguish post-intervention

homicide trends for treated groups relative to post-intervention homicide trends for

untreated groups.

Some readers may wonder whether this evaluation can comment on the ‘‘true’’ impact

of Ceasefire on serious gun violence in Boston. Indeed, this evaluation focused on

addressing a key question posed by the National Academies’ Panel and Zimring—whether

treated Ceasefire gangs actually changed their violent behavior. Kennedy (1997), however,

suggests that the Ceasefire focused deterrence strategy was intentionally designed to deter

the violent behavior of gangs not directly exposed to the intervention. In essence, our

statistical models estimated the effect of treatment on the ‘‘directly’’ treated gangs but not

on the ‘‘indirectly’’ treated gangs. A full accounting of Ceasefire violence reduction effects

in Boston would also examine these second-order impacts. An important avenue of future

research would be to determine whether focused deterrence strategies created ‘‘spillover’’

violence reduction effects onto other gangs and neighborhoods. Indeed, building upon this

study, we are pursuing analyses to examine whether untreated gangs changed their gun

violence behaviors after their rivals and/or allies were subjected to the Ceasefire

intervention.

The available research on Ceasefire and its replications has thus far provided scant

empirical evidence on the ways individuals nested within targeted groups and social net-

works may change their criminal decision making processes. The Ceasefire mechanism of

putting gangs ‘‘on notice’’ is designed to increase the certainty of punishment for the group

as a whole, but it does so through (a) the diffusion of the message among individual group

members and (b) reliance on the group members, as a collective, to modify behavior

accordingly. Unfortunately, our study was not able to analyze data on individual behavior.

However, the next generation of research on focused deterrence strategies should take

advantage of an important opportunity to understand how changing the certainty of pun-

ishment for group-level criminal activity may affect individual as well as group behavior.

A recent study by Loughran et al. (2011b) offers evidence of a ‘‘tipping’’ effect, whereby

perceived risk deters only when it reaches a certain threshold, and a substantially accel-

erated deterrent effect for individuals at the high end of the risk continuum (Loughran et al.

2011b). Yet, another study found diminishing ambiguity of certainty had no observed

deterrent effect for crimes involving contact between offender and victim (Loughran et al.

2011a). It is possible that Ceasefire’s unambiguous face-to-face meetings with gang

members, coupled with demonstrated increases in the swiftness, certainty, and severity of

punishment for gun violence, exceeds the threshold for a tipping effect and substantially

accelerates the deterrent effect of the intervention for high-risk gang members. Or, it is

possible that some such tipping points operate in unexplored ways when collectivities such

as gangs are involved. Future research on Ceasefire-like interventions would do well to

J Quant Criminol (2014) 30:113–139 135

123

consider how individual decision making processes operate in the context of group

accountability and interventions.

While determining whether a program generates the desired outcomes remains an

important task, we strongly believe that the next wave of research on focused deterrence

strategies needs to understand why these strategies seem to work and how these strategies

can be sustained over time. A growing number of scholars suggest that that there seems to

be additional crime control mechanisms at work in these strategies beyond straight-up

deterrence (Braga 2012; Corsaro et al. 2012; Papachristos et al. 2007). Other prevention

frameworks, such as community social control and procedural fairness, might help explain

the observed impacts of focused deterrence programs on crime. There is also a growing

body of literature suggesting that it is very difficult in practice to sustain these initiatives

over an extended time period. Beyond the cessation of Ceasefire in Boston noted earlier,

replication programs in Baltimore and Minneapolis unraveled rapidly after some encour-

aging initial crime control success stories (see Kennedy 2011). The Cincinnati Initiative to

Reduce Violence, however, has been able to institutionalize and sustain its focused

deterrence interventions through the establishment of a comprehensive organizational

structure and a governing board (Engel et al. 2011). Clearly, jurisdictions interested in

implementing focused deterrence strategies need to understand how to keep these programs

on track for the long-term.

References

Apel RJ, Nagin D (2011) General deterrence: a review of recent evidence. In: Wilson JQ, Petersilia J (eds) Crime and public policy. Oxford University Press, New York, pp 411–436

Apel RJ, Sweeten G (2010) Propensity score matching in criminology and criminal justice. In: Piquero A, Weisburd DL (eds) Handbook of quantitative criminology. Springer, New York, pp 543–562

Austin P, Grootendorst P, Anderson G (2007) A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study. Stat Med 26:734–753

Berk R (2005) Knowing when to fold ‘em: an essay on evaluating the impact of Ceasefire, Compstat, and Exile. Criminol Public Policy 4:451–466

Black D (1970) The production of crime rates. Am Sociol Rev 35:733–748 Blalock H (1979) Social statistics, 2nd edn. McGraw-Hill, New York Blumstein A (1995) Youth violence, guns, and the illicit-drug industry. J Crim Law Criminol 86:10–36 Blumstein A, Cohen J, Nagin D (eds) (1978) Deterrence and incapacitation: estimating the effects of

criminal sanctions on crime rates. National Academy of Sciences, Washington, DC Braga AA (2012) Getting deterrence right? Evaluation evidence and complementary crime control mech-

anisms. Criminol Public Policy 11:201–210 Braga AA, Weisburd DL (2012) The effects of focused deterrence strategies on crime: a systematic review

and meta-analysis of the empirical evidence. J Res Crime Delinq 49:323–358 Braga AA, Winship C (2006) Partnership, accountability, and innovation: clarifying Boston’s experience

with pulling levers. In: Weisburd DL, Braga AA (eds) Police innovation: contrasting perspectives. Cambridge University Press, New York, pp 171–190

Braga AA, Kennedy DM, Waring E, Piehl AM (2001) Problem-oriented policing, deterrence, and youth violence: an evaluation of Boston’s Operation Ceasefire. J Res Crime Delinq 38:195–225

Braga AA, Hureau DM, Winship C (2008a) Losing faith? Police, black churches, and the resurgence of youth violence in Boston. Ohio State J Crim Law 6:141–172

Braga AA, Pierce G, McDevitt J, Bond BJ, Cronin S (2008b) The strategic prevention of gun violence among gang-involved offenders. Justice Q 25:132–162

Butterfield F (1996) In Boston, nothing is something. The New York Times, November 21: A20 Caliendo M, Kopeinig S (2005) Some practical guidance for the implementation of propensity score

matching (discussion paper 1588). Institute for the Study of Labor, Bonn

136 J Quant Criminol (2014) 30:113–139

123

Campbell DT, Boruch RF (1975) Making the case for randomized assignment to treatment by considering the alternatives. In: Bennett C, Lumsdaine A (eds) Evaluation and experiments: some critical issues in assessing social programs. Academic Press, New York, pp 195–296

Cohen J, Ludwig J (2003) Policing crime guns. In: Ludwig J, Cook PJ (eds) Evaluating gun policy: effects on crime and violence. Brookings Institution Press, Washington, DC, pp 217–239

Cook PJ (1980) Research in criminal deterrence: laying the groundwork for the second decade. In: Morris N, Tonry M (eds) Crime and justice: an annual review of research, vol 2. University of Chicago Press, Chicago, pp 211–268

Cook P, Laub J (2002) After the epidemic: recent trends in youth violence in the United States. In: Tonry M (ed) Crime and justice: a review of research, vol 29. University of Chicago Press, Chicago, pp 1–38

Cook PJ, Ludwig J (2006) Aiming for evidence-based gun policy. J Policy Anal Manage 48:691–735 Corsaro N, McGarrell EF (2009) Testing a promising homicide reduction strategy: reassessing the impact of

the Indianapolis ‘‘pulling levers’’ intervention. J Exp Criminol 5:63–82 Corsaro N, Hunt ED, Hipple NK, McGarrell EF (2012) The impact of drug market pulling levers policing on

neighborhood violence: an evaluation of the High Point drug market intervention. Criminol Public Policy 11:167–200

Dalton E (2002) Targeted crime reduction efforts in ten communities: lessons for the project safe neigh- borhoods initiative. US Attorney’s Bull 50:16–25

Decker S (1996) Collective and normative features of gang violence. Justice Q 13:243–264 Decker S, Katz C, Webb V (2008) Understanding the black box of gang organization: implications for

involvement in violent crime, drug sales, and violent victimization. Crime Delinq 54:153–172 Dehejia RH, Wahba S (2002) Propensity score matching methods for nonexperimental causal studies. Rev

Econ Stat 84:151–161 DiPrete T, Gangl M (2004) Assessing bias in the estimation of causal effects: Rosenbaum bounds on

matching estimators and instrumental variables estimation with imperfect instruments. Sociol Meth- odol 34:271–310

Durlauf S, Nagin D (2011) Imprisonment and crime: can both be reduced? Criminol Public Policy 10:13–54 Ellement JR (2007) 25 alleged Boston gang members charged with gun, drug offenses. The Boston Globe,

May 24, p A1 Engel RS, Skubak Tillyer M, Corsaro N (2011) Reducing gang violence using focused deterrence: evalu-

ating the cincinnati initiative to reduce violence (CIRV). Justice Q. doi:10.1080/07418825. 2011.619559

Fagan J (2002) Policing guns and youth violence. Future Child 12:133–151 Farrington D, Gottfredson D, Sherman L, Welsh B (2002) The Maryland scientific methods scale. In:

Sherman L, Farrington D, Welsh B, MacKenzie D (eds) Evidence-based crime prevention. Routledge, London, pp 13–21

Gelman A (2005) Analysis of variance: why it is more important than ever. Ann Stat 33:1–53 Gibbs JP (1975) Crime, punishment, and deterrence. Elsevier, New York Heckman J, Ichimura H, Todd P (1997) Matching as an econometric evaluation estimator: evidence from

evaluating a job training programme. Rev Econ Stud 64:605–654 Heckman J, LaLonde R, Smith J (1999) The economics and econometrics of active labor market programs.

In: Ashenfelter O, Card D (eds) Handbook of labor economics, vol 3. Elsevier, Amsterdam, pp 1865–2097

Horney J, Marshall IH (1992) Risk perceptions among serious offenders: the role of crime and punishment. Criminology 30:575–594

Hughes L, Short J (2005) Disputes involving gang members: micro-social contexts. Criminology 43:43–76 Imbens GW (2004) Nonparametric estimation of average treatment effects under exogeneity: a review. Rev

Econ Stat 86:4–29 Imbens GW, Wooldredge J (2009) Some recent developments in the econometrics of program evaluation.

J Econ Lit 47:5–86 Kennedy DM (1997) Pulling levers: chronic offenders, high-crime settings, and a theory of prevention.

Valparaiso Univ Law Rev 31:449–484 Kennedy DM (2011) Don’t shoot. Bloomsbury, New York Kennedy DM, Piehl AM, Braga AA (1996) Youth violence in Boston: gun markets, serious youth offenders,

and a use-reduction strategy. Law Contemp Probl 59:147–196 Kennedy DM, Braga AA, Piehl AM (1997) The (un)known universe: mapping gangs and gang violence in

Boston. In: Weisburd D, McEwen JT (eds) Crime mapping and crime prevention. Criminal Justice Press, Monsey, pp 219–262

Klein M (1993) Attempting gang control by suppression: the misuse of deterrence principles. Stud Crime Crime Prev 2:88–111

J Quant Criminol (2014) 30:113–139 137

123

Klofas J, Hipple NK (2006) Crime incident reviews. Project safe neighborhoods: strategic interventions case study 3. US Department of Justice, Washington, DC

Leuven E, Sianesi B (2003) PSMATCH2: Stata module to perform full Mahalanobis and propensity score matching, common support graphing, and covariate imbalance testing. Available online: http:// ideas.repec.org/c/boc/bocode/s432001.html

Levitt S, Venkatesh S (2000) An economic analysis of a drug-selling gang’s finances. Q J Econ 115:755–789

Lipsey M, Wilson DB (2001) Practical meta-analysis. Applied social research methods series, vol 49. Sage, Thousand Oaks

Long JS, Freese J (2006) Regression models for categorical dependent variables using Stata. StataCorp, LP, College Station

Loughran T, Paternoster R, Piquero A, Pogarsky G (2011a) On ambiguity in perceptions of risk: implica- tions for criminal decision making and deterrence. Criminology 49:1029–1061

Loughran T, Pogarsky G, Piquero A, Paternoster R (2011b) Re-examining the functional form of the certainty effect in deterrence theory. Justice Q 29(5):712–741

Ludwig J (2005) Better gun enforcement, less crime. Criminol Public Policy 4:677–716 McGarrell EF, Chermak S, Weiss A, Wilson J (2001) Reducing firearms violence through directed police

patrol. Criminol Public Policy 1:119–148 McGarrell EF, Chermak S, Wilson J, Corsaro N (2006) Reducing homicide through a ‘lever-pulling’

strategy. Justice Q 23:214–229 Morenoff JD, Sampson RJ, Raudenbush SW (2001) Neighborhood inequality, collective efficacy, and the

spatial dynamics of urban violence. Criminology 39:517–559 Morgan SL, Winship C (2007) Counterfactuals and causal inference: methods and principals for social

research. Cambridge University Press, New York Nagin D (1998) Criminal deterrence research at the outset of the twenty-first century. In: Tonry M (ed)

Crime and justice: a review of research, vol 23. University of Chicago Press, Chicago, pp 1–42 Papachristos A (2009) Murder by structure: dominance relations and the social structure of gang homicide.

Am J Soc 115:74–128 Papachristos A, Kirk D (2006) Neighborhood effects and street gang behavior. In: Short J (ed) Studying

youth gangs. Alta Mira, Landham, pp 63–84 Papachristos A, Meares T, Fagan J (2007) Attention felons: evaluating project safe neighborhoods in

Chicago. J Emp Legal Stud 4:223–272 Paternoster R (1987) The deterrent effect of the perceived certainty and severity of punishment: a review of

the evidence and issues. Justice Q 4:173–217 Piehl AM, Cooper SJ, Braga AA, Kennedy DM (2003) Testing for structural breaks in the evaluation of

programs. Rev Econ Stat 85:550–558 Rosenbaum P (2002) Observational studies, 2nd edn. Springer, New York Rosenbaum P, Rubin D (1983) The central role of the propensity score in observational studies for causal

effects. Biometrika 70:41–55 Rosenbaum P, Rubin D (1985) Constructing a control group using multivariate matched sampling methods

that incorporate the propensity score. Am Stat 39:33–38 Rosenfeld R, Bray TM, Egley A (1999) Facilitating violence: a comparison of gang-motivated, gang-

affiliated, and nongang youth homicides. J Quant Criminol 15:495–516 Rosenfeld R, Fornango R, Baumer E (2005) Did Ceasefire, Compstat, and Exile reduce homicide? Criminol

Public Policy 4:419–450 Rossi PH, Lipsey M, Freeman H (2006) Evaluation: a systematic approach, 7th edn. Sage, Newbury Park Rubin DB (1990) Formal modes of statistical inferences for causal effects. J Stat Plan Inference 25:279–292 Sampson RJ, Wilson WJ (1995) Toward a theory of race, crime, and urban inequality. In: Hagan J, Peterson

R (eds) Crime and inequality. Stanford University Press, Stanford, pp 37–56 Sampson RJ, Raudenbush SW, Earls F (1997) Neighborhoods and violent crime: a multilevel study of

collective efficacy. Science 277:918–924 Schneider VW, Wiersema B (1990) Limits and use of uniform crime reports. In: MacKenzie DL, Baunach

PJ, Roberg RR (eds) Measuring crime. State University of New York Press, Albany, pp 21–48 Seabrook J (2009) Don’t shoot: a radical approach to the problem of gang violence. The New Yorker, June

22, pp 32–39 Shadish W, Cook T, Campbell D (2002) Experimental and quasi-experimental designs for generalized

causal inference. Houghton Mifflin, Boston Sherman LW, Rogan D (1995) Effects of gun seizures on gun violence: ‘hot spots’ patrol in Kansas City.

Justice Q 12:755–782

138 J Quant Criminol (2014) 30:113–139

123

Sherman LW, Gottfredson D, MacKenzie DL, Eck JE, Reuter P, Bushway S (1997) Preventing crime: what works, what doesn’t, what’s promising. U.S. Department of Justice, National Institute of Justice, Washington, DC

Singer JD, Willet JB (2003) Applied longitudinal data analysis: modeling change and event occurrence. Oxford University Press, New York

Smith C (2012) The influence of gentrification on gang homicides in Chicago neighborhoods, 1994 to 2005. Crime Delinq. doi:10.1177/0011128712446052

Smith J, Todd P (2005) Does matching overcome LaLonde’s critique of nonexperimental estimators? J Econom 125:303–353

Tita G, Greenbaum R (2009) Crime, neighborhoods, and units of analysis: putting space in its place. In: Weisburd D, Bernasco W, Bruinsma G (eds) Putting crime in its place. Springer, New York, pp 145–170

Tita G, Radil S (2011) Spatializing the social networks of gangs to explore patterns of violence. J Quant Criminol 27:521–545

Tita G, Riley J, Ridgeway G, Grammich C, Abrahamse A, Greenwood P (2004) Reducing gun violence: results from an intervention in East Los Angeles. RAND Corporation, Santa Monica

Travis J (1998) Crime, justice, and public policy. Plenary presentation to the American Society of Crimi- nology, (http://www.ojp.usdoj.gov/nij/speeches/asc.htm), November 1, Washington, DC

Weisburd D, Lum C, Petrosino A (2001) Does research design affect study outcomes in criminal justice? Annals 578:50–70

Wellford CF, Pepper JV, Petrie CV (eds) (2005) Firearms and violence: a critical review. Committee to improve research information and data on firearms. The National Academies Press, Washington, DC

Welsh BC, Peel ME, Farrington DP, Elffers H, Braga AA (2011) Research design influence on study outcomes in crime and justice: a partial replication with public area surveillance. J Exp Criminol 7:183–198

Wilkinson L, Task Force on Statistical Inference (1999) Statistical methods in psychology journals: guidelines and expectations. Am Psychol 54:594–604

Witkin G (1997) Sixteen silver bullets: smart ideas to fix the world. US News and World Report, December 29, p 67

Wright B, Caspi A, Moffitt T, Paternoster R (2004) Does the perceived risk of punishment deter criminally prone individuals? Rational choice, self-control, and crime. J Res Crime Delinq 41:180–213

Zimring F (1968) Is gun control likely to reduce violent killings? Univ Chic Law Rev 35:21–37 Zimring F (1972) The medium is the message: Firearm caliber as a determinant of death from assault.

J Legal Stud 1:97–124 Zimring F, Hawkins G (1973) Deterrence: the legal threat in crime control. University of Chicago Press,

Chicago

J Quant Criminol (2014) 30:113–139 139

123

Copyright of Journal of Quantitative Criminology is the property of Springer Science & Business Media B.V. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.