Discussion 12: Cost-Efficiency Evaluations, Part I

8

Cost-Efficiency Evaluations

W HEN ALL IS SAID AND DONE, WE NOT ONLY WANT CRIMINAL JUSTICE

policies to produce intended outcomes, like less crime; we also want the most “bang for the buck.” That is, we want policies that provide the most benefit for the least cost.

Most of us implicitly apply this logic in our day-to-day decision making. For example, we typically want a mode of transportation, such as a car, that reliably gets us to our destination, but we also want to pay as little as possible for it. Although the decision-making calculus may be complicated, the formula that we apply is not: among the available cars from which we can choose, we want the one that maximizes the benefit (e.g., reliable transportation) and minimizes costs (e.g., purchase price, repairs, fuel) over some set period of time.1

The formula applies as well when we have competing goals, but the cal- culations can be a bit more challenging. Perhaps we have a car that will no longer run. Then we learn that we have a dental problem that, if untreated, will lead to substantial pain. We need the car to get to work, but we also need surgery to live without pain. If we have the funds and no other competing demands, we could pay for both. Problem solved. However, many people face a real dilemma: buy a car or pay for surgery?2 Without a common met- ric for making a comparison between the qualitatively different outcomes (employment versus pain), we face a somewhat greater challenge in deter- mining which decision will yield the most benefit for the least cost. Even so, the focus on comparing benefits with costs remains the same.

Cost-efficiency evaluation involves this same type of logic and it comes in two varieties, one for each of the two situations just mentioned. Cost- effectiveness analysis is used to determine which of several approaches is best for achieving a given outcome, and cost-benefit analysis is used to determine which of several approaches that target qualitatively different outcomes

207

208 AMERICAN CRIMINAL JUSTICE POLICY

creates the most benefit relative to costs. To illustrate, law enforcement agencies must make decisions about which strategies most efficiently pre- vent crime. Cost-effectiveness analyses can be helpful in comparing such strategies. However, these agencies also must make decisions about other strategies that target other outcomes, such as efforts to reduce fear of crime and to provide emergency services that may save lives.3 Here, the outcomes (crime, fear of crime, lives saved) differ, and so we need a common metric for comparing the strategies that target them so that, in the end, we can compare which one produces the most benefit for the least cost.

Efficiency is of paramount importance to society and most organizations, especially when the risks are great (e.g., increased crime, fear, and injustice) and resources are scarce. That situation aptly characterizes criminal justice. For these reasons, cost-efficiency evaluation stands at the top of the evalu- ation hierarchy and is central to creating a more accountable and effective criminal justice system.

This chapter describes efficiency evaluation and the differences between cost-effectiveness and cost-benefit analyses. It focuses particular attention on the logic of such analyses and the basic steps involved in conducting them. It also highlights the central limitations and benefits of efficiency evaluation for criminal justice policy. The chapter then provides two case studies, one on community policing and the other on private prisons, to illustrate the idea of such evaluations. Paralleling the approach taken in previous chapters, it concludes by exploring the research foundation for assessing the efficiency of prominent U.S. criminal justice policies.

What Is a Cost-Efficiency Evaluation?

Cost-Efficiency Evaluation: Step 5 in the Hierarchy

THE EFFICIENCY MANDATE. The fifth and final step in the evaluation hier- archy involves cost-efficiency evaluation. Efficiency stands as perhaps the central guiding mandate of government – the aim is to provide services and achieve goals at the least cost and, more generally, to provide the maximum benefits to society. Achieving that mandate can be difficult when so many competing goals exist (e.g., education, public safety, roads) and when, in each instance, different approaches exist for trying to achieve them.

Regardless, the mandate exists and is embodied in virtually any taxpayer- funded effort. For example, when states allocate a certain amount of funds to corrections and not to education, they implicitly assume that the benefits of doing so exceed those associated with allocating the funds to education.

COST-EFFICIENCY EVALUATIONS 209

They may not conduct a study to test that assumption, but they nonetheless follow the logic. Similarly, when a department of corrections decides to build and operate a supermax prison, they assume that doing so produces benefits, relative to the costs, that exceed what could be obtained through other investments.

Despite policy makers’ calls for greater accountability, evaluations of the efficiency of public policies occur infrequently. Certainly, and as will be discussed later in this chapter, they occur rarely in criminal justice. The end result? Policy makers and criminal justice administrators perforce must make assumptions – ones that could be incorrect – about the relative benefits and costs of various policies, programs, and practices.4 This situation in turn creates substantial doubts about whether the government operates as efficiently as it could.

In the meantime, crime clearly creates substantial costs for society. As noted earlier, criminal justice expenditures in 2006 were $215 billion, a fig- ure that does not factor in the costs of victimization or other adverse effects of crime.5 In 1993, the last year for which a comprehensive assessment was conducted, the costs were estimated to be $450 billion.6 Many schol- ars dispute the validity of this estimate, noting, among other things, that they build on cost estimates associated with extreme rather than run-of-the- mill crimes.7 Even so, the expenditures alone and the dramatic increase in them over the past two decades highlight the fact that society has invested heavily in criminal justice and presumably expects some return on that investment.

COST-EFFECTIVENESS ANALYSIS VERSUS COST-BENEFIT ANALYSIS. Effi- ciency evaluations can consist of either cost-effectiveness analysis or cost- benefit analysis. Cost-effectiveness analyses identify the cost per outcome (e.g., for every dollar spent, one crime was averted). They are useful when compar- ing policies that target the same outcome. In the beginning of this chapter, the comparison of vehicles illustrated the logic of a cost-effectiveness analy- sis – the consumer aims to identify which vehicle will, over some period of time, provide the most reliable transportation for the least cost.

Cost-benefit analyses – sometimes referred to as benefit-cost analyses – identify, in monetary terms, policy costs and benefits (e.g., for every dollar spent, two dollars were saved). They are useful when comparing policies that target different outcomes (e.g., one aiming to prevent teen pregnancies and another aiming to reduce crime). The vehicle-versus-dental-care example at the beginning of the chapter illustrated the logic. In the example, the person attempts to identify whether achieving one goal (buying a car) will produce

210 AMERICAN CRIMINAL JUSTICE POLICY

more benefits than achieving another (obtaining dental care), relative to the costs of producing these benefits.

To illustrate further the distinction between the two types of analyses, consider the following two scenarios. First, you are the mayor a small town of 30,000 people, one that includes many of your best friends and family members. You have only $1 million to spend on efforts other than the bare bones minimum of paying for such services as roads, schools, and a medi- cal clinic. In the course of the past year, the number of aggravated assaults increased by 10 percent, from 100 to 110. You want to address the problem and task a committee with presenting several options. They convene and rec- ommend that you consider one of the following policies: longer prison terms for individuals who commit assaults (assume here that incarceration costs are paid for by the town), an employment training and placement program for at-risk populations, or a gang-reduction initiative. In each instance, the policy goal is the same – reduce assaults. Which policy do you select? Cost- effectiveness analyses would develop cost estimates for each policy and use information from impact evaluations of the policies. They then would create policy-specific estimated costs for, say, avoiding some number of assaults. The policy that costs the least to produce this result would be the most cost-effective.

Now, imagine a second scenario. Here, again, you are the mayor of a small town of 30,000 people and you have $1 million to spend on social problems. When you look over the past year, you see that assaults have increased 10 percent. However, you also see that teen pregnancies and the number of homeless people have both increased by 10 percent as well. To simplify matters, assume that in each case, a 10 percent increase amounts to ten additional cases (e.g., ten additional assaults, ten additional teen pregnancies, ten additional homeless persons). For the sake of argument, assume that impact evaluations have been conducted and show that, for $1 million, (a) a policy exists that will result in five fewer assaults in the coming year, (b) another policy will result in five fewer teen pregnancies, and, (c) a third policy will result in five fewer homeless persons. The investment cost in each instance is the same, but the outcomes differ. And therein lies the rub: an assault differs from a teen pregnancy and both differ from homelessness. What we need is a way to establish a basis of comparison. Monetizing – putting a dollar value on – a given unit of outcome enables us to establish comparability among the three outcomes. In turn, we then can create policy-specific estimates of costs versus benefits. The policy with the greatest return, as expressed in dollars, would be the most cost beneficial, all else being equal.

COST-EFFICIENCY EVALUATIONS 211

In some cases, cost-effectiveness analyses may suffice and have the virtue of sparing evaluators the difficult challenge of arriving at defensible mon- etized estimates of impact. The valuation of a life, for example, is a con- tentious issue.8 The problem lies in the fact that many criminal justice poli- cies and programs have multiple goals. Unless we factor all of them into an efficiency analysis, we end up with an unbalanced comparison. Enter cost-benefit analyses. They enable us to use a common metric – money – to compare a diverse set of outcomes. It might seem that cost-effectiveness studies would be suitable for a large swath of criminal justice initiatives aimed exclusively at reducing recidivism. However, different crimes have different costs. A rape, for example, typically would be presumed to produce more harm than theft of a bicycle.9 For this reason, cost-benefit analysis is use- ful, and indeed, necessary, for recidivism studies if we are to develop valid comparisons among policies.

How to Conduct a Cost-Benefit Analysis

Here, we will examine cost-benefit analysis and a number of conceptual elements central to undertaking one. The focus on cost-benefit analysis stems from the fact that the logic and steps involved parallel those for a cost-effectiveness analysis; the main difference is that cost-benefit analysis requires that we monetize all outcomes.

The focus on conceptual considerations stems from several considera- tions. First, it accords with a theme running throughout this book – namely, clear conceptualization is critical to undertaking and understanding policy evaluation, and, in many respects, it matters far more than the use of sophis- ticated research designs. That idea holds true with cost-benefit analysis as much as it does for needs, theory, implementation, and outcome and impact evaluations. Indeed, cost-benefit analysis, as Mark Cohen has emphasized, “is often more of an art than a science.”10 Second, an emphasis on con- ceptual considerations provides a more direct link to the discussions in the earlier chapters. For example, the veracity of a cost-benefit analysis rests heavily on a credible estimate of policy impact. Indeed, it can reasonably be said that a cost-benefit analysis is only as credible as that of the esti- mated impacts on which it rests.11 However, such estimates frequently may not exist, as Chapter 7 highlighted. By implication, any cost-benefit anal- ysis premised on an assumed impact should be viewed with considerable caution. Third, many excellent detailed accounts of the concrete nuts and bolts of cost-benefit analysis already exist and so do not need to be covered here.12

212 AMERICAN CRIMINAL JUSTICE POLICY

Many distinct tasks go into a cost-benefit analysis. Broadly, however, six steps need to be taken: (1) state the policy question, (2) identify the perspec- tive of analysis, (3) identify all relevant costs and benefits, (4) assign values to costs and benefits, (5) compare the costs and benefits of one or more poli- cies, and (6) assess the sensitivity of the results to critical assumptions and detail all relevant assumptions and limitations. Each step is described next.

1. STATE THE POLICY QUESTION. The relevant policy question and per- spective of analysis may seem obvious, but frequently they are not. Identi- fying them can require a great deal of conceptual footwork. For example, we might be interested, as a general matter, in whether the benefits of a particular policy, say, a supermax, exceed the costs. But such a question lacks specificity. In reality, we likely want to know if building a new super- max facility would be cost beneficial relative to some other alternative. Or we likely want to know if an existing supermax constitutes a cost-beneficial investment as compared to the costs and benefits of tearing it down and investing in a different type of facility or perhaps some other strategy or set of strategies for improving order and safety in a prison system. In short, the central challenge, at bottom, is similar to what we face when conducting or critiquing impact evaluations – identifying the relevant basis of comparison.

To make a useful determination about appropriate comparisons, we typ- ically need information about relative efficiency, that is, the efficiency of one approach as compared to another. To this end, it can be helpful to have a thorough needs evaluation from which to work. Such an evaluation can provide important context for describing the specific problems and the possible solutions to them. Perhaps, for example, a jurisdiction has experi- enced a jail-overcrowding problem. A needs evaluation might show that a large number of low-risk first-time offenders are being jailed prior to trial. If so, an efficiency analysis for a proposed jail might want to include not only the returns expected from constructing a new jail but also those that might accrue from a distinctly different policy option, namely, using jail space primarily for medium- and high-risk offenders.

2. IDENTIFY THE PERSPECTIVE OF ANALYSIS. Different perspectives exist – programs, government agencies, particular communities, cities and counties, states, and society, to name but a few – and each possesses a unique view that might well shape the specific questions posed in an efficiency evaluation. The societal perspective appears to be the one most frequently used. Government programs, for example, exist to help the public. Thus, their costs and benefits should be assessed with respect to society at large,

COST-EFFICIENCY EVALUATIONS 213

TABLE 8.1. The influence of the perspective of analysis on the classification of costs and benefits: An illustration using supermax prisons

Perspective of analysis

Department of Local Corrections Society community

a. Improved prison management Benefit – – b. Fewer in-prison assaults Benefit – – c. Improved postrelease success for

general population prisoners Benefit Benefit –

d. Weakened supermax inmate family relationships

– Cost –

e. Increased recidivism of supermax prisoners

Cost Cost –

f. Increased mental health problems among supermax prisoners

Cost Cost –

g. Increased domestic dispute incidents among prison staff and families of supermax prisoners

Cost Cost Cost

h. Additional property taxes Cost Cost Benefit

Source: Adapted from Lawrence and Mears (2004:10).

not some particular agency or group. However, cost-efficiency scholars dis- agree about the best or most appropriate perspective.13 Consider, for exam- ple, that a government agency charged with public health might reasonably ignore the crime-reduction benefits of drug treatment, while a criminal jus- tice agency might ignore the public-health benefits of reduced drug abuse.14

Why? When a department of corrections hires more staff, that investment comes from its budget not that of other state agencies. Accordingly, the department would likely include the additional staff as part of a cost-benefit analysis aimed at determining whether this investment or some other (e.g., purchase of equipment or technology) produced the most returns for some outcome of interest of direct relevance for the department (e.g., systemwide order and safety). In turn, any resulting cost-efficiency estimates of this investment might look appreciably different from those produced from dif- ferent perspectives of analysis (e.g., those of a department of public health).

To illustrate this point, consider, again, supermax prisons and the under- taking of cost-benefit analyses from three distinct perspectives – a depart- ment of corrections, society, and a local community.15 Table 8.1 presents

214 AMERICAN CRIMINAL JUSTICE POLICY

those perspectives and shows how some outcomes might be treated by each. Consider the first two outcomes, improved prison management (row a) and reduced in-prison assaults (row b). From a corrections perspective, both changes, potentially resulting from supermax incarceration, would constitute a benefit and so should be included in a cost-benefit analysis. However, from the perspective of society or a local community, neither outcome constitutes a benefit or cost. Neither outcome, for example, necessarily contributes to greater public safety.

When we turn to improved post-release success for general population inmates (row c), the picture changes slightly. Success might consist of lower recidivism rates, which a department of corrections would want to include in a cost-benefit analysis. From a societal perspective, lower recidivism clearly would stand as a benefit as well, and so from this perspective should be included in such an analysis. A benefit to the local community would be unlikely given that state prison inmates typically come from places other than the area where a given prison is located. However, should many of the released inmates return to that area, then a cost-benefit analysis from a local community perspective would want to include information about lower rates of recidivism.

Supermax housing might weaken inmates’ ties to their families and weaken or disrupt family relationships (row d). An effect on inmates’ fami- lies would not constitute a cost to the department of corrections. However, it might well constitute a cost to society, as depicted in the table.

If supermax incarceration were to increase rather than decrease recidivism (row e), then it would be a cost rather than a benefit, one that would be relevant from the perspectives of both the department of corrections and society. It also is possible that supermaxes increase mental health problems (row f), or so critics contend. Any increase would also count as a cost from these same perspectives.

The possibility exists that working in supermax facilities increases stress and domestic disputes among staff who work in them and the inmates placed there (row g). If such effects indeed exist, they likely would count as costs from each of the three perspectives of analysis.

Finally, we turn to an instance in which an outcome may be a cost or benefit. Increased property taxes (row h) may result when a department of corrections, and, by extension, society, invests in a supermax prison. These thus would be counted as costs from department of corrections and societal perspectives. However, for a local community, such taxes might well be treated as a benefit because of the increased revenues.

This example illustrates that no single or correct perspective of analysis exists. It also illustrates how the perspective of analysis can determine the

COST-EFFICIENCY EVALUATIONS 215

classification of outcomes as costs or benefits. In turn, it demonstrates the critical importance, when presenting or interpreting cost-efficiency results, of clarifying the perspective of analysis used and how the results might differ if other perspectives were used.

3. IDENTIFY COSTS AND BENEFITS. The third step in conducting a cost- benefit analysis entails identifying all relevant costs and benefits. As the discussion of step 2 indicates, the perspective of analysis will determine what is counted as a cost or a benefit and, more generally, what costs and benefits should be considered. A systematic and detailed theory evaluation can aid in this effort by highlighting a policy’s critical features, processes, and outcomes. Once we have selected the perspective and consulted a theory evaluation, the next step involves cataloging any cost or benefit related to the policy of interest.

In a cost-benefit analysis, a cost consists of anything that entails a loss and a benefit costs of anything that entails a gain, as expressed monetarily. To illustrate, expenditures – such as capital and operating costs associated with building and running a supermax prison – would be counted as costs, while improvements in outcomes (e.g., recidivism) would be counted as benefits. Whether a given item is classified as a cost or benefit ultimately depends, however, on whether it produces a (monetized) loss or a gain. For example, if a supermax prison increases recidivism, it creates a loss; accordingly, the increased recidivism would be classified as a cost. If, however, it decreases recidivism, it creates a gain or improvement; accordingly, the decreased recidivism would be classified as a benefit.

From an accounting perspective, several distinct types of costs and ben- efits should be cataloged. Numerous sources discuss these in detail, and Cohen and others have done so within a criminal justice framework.16 Con- sequently, here we will review only some of them to illustrate the conceptual and analytic work involved in undertaking and interpreting a cost-benefit analysis.

Costs and benefits can be classified in many ways, but one of the most important distinctions is between costs and benefits that are direct versus indirect. Direct costs and benefits “are those that are closely related to the primary objective of the [policy].”17 Direct costs of supermaxes, for exam- ple, include construction, personnel, technology, and other inputs, whereas direct benefits might include reduced violence and disorder throughout the prison system. Indirect costs and benefits “are by-products, multipliers, spillovers, or investment effects of the [policy],” what sometimes are referred to as externalities.18 Indirect costs can be intended or unintended. An in- tended indirect cost might include overhead associated with operating a

216 AMERICAN CRIMINAL JUSTICE POLICY

supermax and an unintended indirect cost might include increased sys- temwide violence or disorder that results from the use of supermax housing. An indirect benefit of a supermax might be reduced recidivism; it would be classified as indirect in states that expected no recidivism-reducing effect of supermax prisons.

Costs and benefits can also be classified as tangible and intangible. Tan- gible costs and benefits are those that we can easily identify and monetize, whereas intangible costs and benefits may be difficult to identify or mone- tize.19 Tangible costs might include, for example, victimization costs, such as medical expenses and lost wages. Tangible benefits might include reduced crime. Among released offenders, it might include improved physical or mental health, and, in turn, a reduced need for and use of social services or health care.20 Intangible costs might include increased pain and suffering or fear of crime and reduced quality of life, while intangible benefits might include decreased pain and suffering and increased quality of life.

Not least, costs and benefits can be classified as fixed and marginal. Fixed costs and benefits are constant, regardless of the size of or change in a policy. For example, if we purchase a video surveillance system, that cost remains the same whether we use it or not. By contrast, marginal costs vary depending on the size of or change in a policy or some feature of it. Should more drug- addicted offenders enter prison, more expenses may be incurred to provide treatment and thus would constitute a variable cost. Many criminal justice cost-benefit analyses use marginal capital and operating costs.21 Ultimately, the decision about what to include depends heavily on the specific focus of the cost-benefit analysis. For example, if a jurisdiction were deciding whether to expand drug treatment offered by an existing program, we would focus on marginal costs; most fixed costs would be irrelevant because the program already exists.22

Several observations about costs and benefits bear mention. First, dif- ferent types can be combined to help guide our efforts to identify all rele- vant costs and benefits. For example, there may be direct intangible ben- efits (e.g., a community policing program makes residents feel safer) and indirect intangible benefits (e.g., a community policing program helps pre- serve neighborhoods or facilitates efforts at revitalizing them).23 It may not always be easy to identify or classify them, but all such possibilities should be explored to ensure as comprehensive a listing of costs and benefits as possible.

Second, opportunity costs are an important consideration in cost-benefit analyses. “The opportunity cost of using a resource is the value of its next most valuable alternative use.”24 For example, an office might be needed to

COST-EFFICIENCY EVALUATIONS 217

support a particular crime-prevention initiative. If used in this manner, it no longer can be used in other ways. We cost the office, therefore, by measuring the value of whatever its next best use would be (e.g., its rental value).25

Unfortunately, it may not always be clear what the next-best use is or what value to include, which can create uncertainty in cost-benefit estimates.26

Also, what an opportunity cost is may depend on the perspective of analysis. How a prison system would have used funds for a supermax may vary, for example, from how state legislators would have used them.

Third, in developing a list of costs and benefits, we invariably will run into situations where we are not sure what to include.27 Here, an important rule of thumb can guide us: in general, we want to include only those costs and benefits related to the policy under consideration and that we would not otherwise occur.28 For example, if a supermax wing is added to an existing prison facility, we would include only the personnel costs associated with operating that wing, not the entire prison facility. The reason parallels the logic of impact evaluations – we want a clear assessment of a policy’s costs and benefits relative to what would have happened without that policy.

Fourth, the credibility of a cost-benefit analysis depends heavily on accu- rate cost and benefit estimates. For this reason, we typically want to proceed with an efficiency analysis only (a) if we have a credible estimate of policy impact or (b) if we want to explore how large an impact would have to be – and whether the magnitude of impact is reasonable to expect – to break even with our investment. If no impact evaluations have been conducted or those that exist rest upon a weak research design, then, by extension, we should have less trust in any efficiency estimates.

4. ASSIGN VALUES. Once all costs and benefits have been identified, we then want to assign monetary values to them. This step can be aided by recourse to data from agency records.29 However, it can be highly contro- versial because there may be little agreement about how to monetize some costs and benefits, especially intangible ones. Tangible costs and benefits “typically pass through a market system and have a price, such as articles of clothing or computer equipment.”30 In these cases, we can readily compile monetary values for each cost and benefit.

Intangible costs and benefits, however, typically have no market and thus no price. For example, no market exists for fear, pain, or suffering. Our option in this case is to proceed with a cost-benefit analysis that excludes such items. In so doing, however, we understate or overstate the true relationship between total costs and benefits. Our other option is to rely on imperfect estimates of intangible costs and benefits. The risk, again,

218 AMERICAN CRIMINAL JUSTICE POLICY

however, is that we understate or overstate the total costs relative to the benefits. No simple solution to this quandary exists, especially given that “the largest component of crime costs is quality-of-life or intangible costs.”31

However, one safe approach involves conducting the analyses in different ways and reporting how the results vary.

In recent years, a number of methods have emerged for developing mon- etary estimates of the cost of crime.32 For example, one can combine infor- mation about a range of dimensions – including victims’ lost productivity, medical care, mental health care, police and fire services, social and victim services, and property loss and damage – to develop crime-specific cost esti- mates.33 It also is possible to survey the public and ask them how much they would be willing to pay to avoid victimization of family members or themselves. In this way, we can obtain an indirect estimate of crime costs and arguably closer estimate of true market-derived crime costs, were such available.34

For criminal justice policies aimed expressly at reducing crime, accurate information about impacts on trajectories of offending is especially impor- tant. Many recidivism studies track offenders for only one to three years. However, if positive effects wash out shortly after this time span, or, alter- natively, if they continue for many years after, we would want to revise our estimates of the crime-reducing effects of the policy and thus our cost- benefit analyses.35 Criminal justice policies necessarily involve a focus on retribution, and so ideally cost-benefit analyses would also take this fact into account. However, few studies assess the economic returns that retribution produces and how those returns vary across policies.36

Finally, relatively little attention has been given to developing monetary estimates associated with noncrime outcomes (e.g., increased employment and education, decreased drug abuse).37 If no such outcomes existed, this situation would not be problematic. However, many criminal justice policies create benefits that go well beyond crime and so ideally should be included in efficiency analyses.38

5. COMPARE COSTS VERSUS BENEFITS OF ONE OR MORE POLICIES.

Once all costs and benefits have been identified and monetized, it is a simple matter to sum them and then create a bottom-line assessment. Before doing so, however, a process called discounting must be undertaken. Any multiyear endeavor, which characterizes most criminal justice policies, accrues costs and benefits over time, with some costs occurring at different times than some benefits. This fact introduces a complication: the value of money today,

COST-EFFICIENCY EVALUATIONS 219

the present value, is more valuable than money in the future, regardless of inflation. Consider, for example, that a bank might pay 4 percent interest on a $100 investment in a savings account. After one year, we would have $104. From an economist’s perspective, the $104 a year from now has a present value of $100 in current dollars and so the discount rate would be 4 percent.

Cost-benefit analysis addresses this fact – the relatively greater value of money in the present than in the future – through discounting. Specifically, “the estimated costs and benefits are arranged over time (usually a num- ber of years) and then those two annual flows of dollars are discounted to present value.”39 (Discounting is not an adjustment for inflation; the lat- ter typically is addressed using constant dollars in all analyses.40) The U.S. Office of Management and Budget publishes guidelines that recommend a 7 percent discount rate on public investments.41 However, the Washing- ton State Institute for Public Policy used a 3 percent discount rate in what many view as one of the more rigorous cost-benefit analyses of a wide range of criminal justice policies.42 Because selection of the discount rate can greatly influence cost-benefit estimates, it is important to assess the effects of using different rates and to identify a rate appropriate to a particular policy context.43

Selection of the policy time span used can also exert a strong effect on cost-benefit estimates. We want to select an appropriate time span that accurately reflects the duration or lifetime over which a policy will generate costs and benefits. In so doing, we need to calibrate our estimates of specific costs and benefits to adjust for changes in their timing and magnitude. To illustrate, crime rate reductions might not begin for a year or two until after a community-focused prevention program has begun and the effect may be more pronounced initially before then tapering off. Conversely, some costs, such as those associated with constructing a prison, are incurred in the first year and, in cost-benefit analyses, are amortized (spread out) over a number of years.

Once the process of discounting has been completed, the summed costs and benefits can be used to generate the net benefit (i.e., benefits – costs) – technically the net present value – or a benefit-cost ratio (i.e., benefits/costs) over the life course of a policy. For example, if a policy produces $1,000,000 in discounted benefits and has discounted costs of $700,000, the net benefit would be $300,000 and benefit-cost ratio would be 1.43. The benefit-cost ratio does not reflect issues of scale. For example, two policies could have identical benefit-cost ratios, but one could be much larger in scale and

220 AMERICAN CRIMINAL JUSTICE POLICY

thus produce, in aggregate, a greater benefit. For that reason, it typically is preferable to present, at a minimum, the net benefit.44

Table 8.2 provides an illustration from a report a colleague, Sarah La- wrence, and I produced on cost-benefit analysis of supermax prisons.45 To convey the basic logic better, the illustration focuses on the question of whether upgrading a building to a supermax facility is cost beneficial and it makes a number of simplifying assumptions. For example, it assumes a one-year time frame and does not incorporate all relevant costs and benefits. In a real cost-benefit analysis, these and other assumptions would need to be revised. As can be seen in the table, the total benefits add up to $640,000 and the total costs add up to $886,000. The net benefit thus is negative (−$246,600) and the benefit-cost ratio is less than 1.0 (.72). In this example, then, supermax housing is not cost beneficial.

A nice feature of cost-benefit analysis consists of the use of a standard- ized metric, money, which enables us to readily compare the net benefits of different policies. All else being equal, for example, we would want to select the policy that provided the largest net benefit. For example, if a dispersion strategy for managing violent or disruptive inmates produced a net benefit of $100,000, then, as compared with a supermax, it would clearly be the preferred choice. “All else being equal” constitutes a critical qualification, however. For example, cost-benefit analyses at best serve as one part of a general decision-making process. And some decisions, such as the perspec- tive of analysis, may entail political decisions that ultimately can only be resolved within the political arena.46

6. ASSESS SENSITIVITY AND ARTICULATE LIMITATIONS. Of course, it might be that estimated cost-benefit estimates are incorrect or are sensi- tive to minor changes in assumptions about critical costs and benefits. It also is quite possible that a number of important limitations exist. Perhaps, for example, an important intangible cost or benefit was not included or an estimated cost or benefit derived from the use of data of questionable reliability or validity. Perhaps, too, evaluators were unable to determine if the costs and benefits differentially accrued for one group or area as against another (e.g., perhaps crime reductions occurred primarily in one neighbor- hood but the costs were borne by all county residents). For these reasons, an absolutely essential part of any cost-benefit analysis involves (1) explicit mention of all relevant limitations and (2) assessment of the sensitivity of the results to changes in such dimensions as the magnitude of certain impacts, the monetized values in cases where debate exists about the appropriate

TA B

LE 8.

2. C

o st

-b en

efi ta

n al

ys is

o fa

su p

er m

ax p

ri so

n

N um

b er

o f

Pe r

un it

M ea

su re

un it

s va

lu e

($ )

To ta

l( $)

B en

efi ts

R ed

uc ed

vi ol

en ce

am on

g in

m at

es N

o. of

in m

at e-

on -i

nm at

e as

sa ul

ts 15

0 1,

20 0

pe r

as sa

ul t

18 0,

00 0

R ed

uc ed

vi ol

en ce

ag ai

ns ts

ta ff

N o.

of in

m at

e- on

-s ta

ff as

sa ul

ts 50

1, 20

0 pe

r as

sa ul

t 60

,0 00

Fe w

er vi

ol en

ti nc

id en

ts re

qu ir

in g

ho sp

ita liz

at io

n N

o. of

ho sp

ita liz

at io

ns 40

2, 50

0 pe

r in

ci de

nt 10

0, 00

0 Le

ss st

re ss

fu ll

iv in

g co

nd iti

on s

N o.

of in

m at

e gr

ie va

nc es

fil ed

20 0

50 0

pe r

gr ie

va nc

e 10

0, 00

0 Le

ss st

re ss

fu lw

or ki

ng co

nd iti

on s

N o.

of si

ck or

pe rs

on al

da ys

ta ke

n 25

0 80

0 pe

r da

y 20

0, 00

0 64

0, 00

0 C

os ts

C on

st ru

ct io

n co

st s

Su m

of ch

ar ge

s re

la te

d to

co ns

tr uc

tio n

– 50

0, 00

0 N

ew eq

ui pm

en ta

nd te

ch no

lo gy

C os

to fn

ew eq

ui pm

en ta

nd te

ch no

lo gy

– 25

,0 00

St af

fh ir

in g

an d

tr ai

ni ng

Pe rs

on ne

lc os

ts fo

r hi

ri ng

an d

tr ai

ni ng

80 20

pe r

ho ur

1, 60

0 Pe

rs on

ne lc

os ts

Pe rs

on ne

lc os

ts fo

r fiv

e ne

w of

fic er

s 10

,4 00

30 pe

r ho

ur 31

2, 00

0 U

til iti

es A

dd iti

on al

am ou

nt of

m on

th ly

ut ili

ty bi

lls 12

4, 00

0 pe

r m

on th

48 ,0

00 88

6, 60

0 N

et B

en efi

ts (B

en efi

ts −

C os

ts )

= −2

46 ,6

00 B

en efi

t-C os

tR at

io (B

en efi

ts /C

os ts

) =

.7 2

So ur

ce :

A da

pt ed

fr om

La w

re nc

e an

d M

ea rs

(2 00

4: 32

).

221

222 AMERICAN CRIMINAL JUSTICE POLICY

TABLE 8.3. Sensitivity analysis: An illustration using supermax prisons and assumptions about reductions in inmate-on-inmate assaults

Number of reduced Benefit-cost inmate-on-inmate assaults per year Net benefit ratio

75 (low impact) −$336,000 .62 150 (expected impact) −$246,600 .72 300 (high impact) −$66,000 .92 355 (break-even impact) No gain or loss 1.00 400 (best-case impact) $173,400 1.20

Source: Adapted from Lawrence and Mears (2004:33).

values to use, the discount rate or time horizon, and the specific treatment of certain cost and benefit items.

To demonstrate how cost-benefit analyses might easily change if we use different assumptions, let us return to the supermax illustration. Table 8.3 models several different scenarios. This exercise is termed sensitivity analysis and allows us to determine how much efficiency estimates will change based on different assumptions, such as the expected or estimated values of costs and benefits.

The first scenario assumes a low impact of supermax housing on inmate- on-inmate assaults; here, such housing only produces 75 fewer assaults. The second assumes the expected impact of 150 fewer such assaults. The third assumes a considerably larger impact of 300 fewer assaults. The fourth shows the number of reduced assaults (355) required to break even (i.e., for benefits and costs to equal one another). And the fifth assumes a best-case scenario impact of 400 fewer assaults.

As we can see, the low, expected, and high impact scenarios consistently result in a net loss. To break even, we would need a reduction in assaults that is halfway between the optimistic high-impact- and best-case-scenario impacts. Only in the best-case scenario would we achieve a net benefit ($173,400).

To reiterate, these estimates do not reflect the true impacts or costs and benefits of supermax prisons. The information for an accurate assessment remains lacking to date. Instead, the estimates, and how they vary with key assumptions, serve to underscore several themes of this chapter. First, credible estimates of impact are absolutely essential for deriving efficiency estimates that we can trust. If impact evaluations produced flawed assess- ments, efficiency evaluations will also be flawed.

COST-EFFICIENCY EVALUATIONS 223

Second, efficiency analyses can be useful for providing information about the potential returns on certain policy investments and also the uncertainty associated with those returns. In the present example, a cost-benefit analysis shows that supermax housing produces a net loss rather than a net bene- fit. That assessment is reinforced by the sensitivity analyses, which show that a net loss is relatively certain – indeed, only a best-case scenario would return a positive net benefit. However, we just as easily could have modeled a situation in which changes in the assumed impacts of supermax prisons consistently produced net benefits. The point in either case is that sensitiv- ity analyses, if conducted with reasonable estimates about likely costs and benefits, can provide important information about whether investing in a policy makes sense.

The importance of sensitivity analysis is difficult to overstate – even slight changes in assumptions can result in dramatically different results. For exam- ple, assumptions about such issues as tax credits or the assumed medical costs of inmates at different custody levels can profoundly alter an efficiency estimate of prison privatization, as has been detailed by Gerald Gaes and his colleagues.47 Similarly, altering the time span can easily reverse an effi- ciency assessment. Eugene Bardach, for example, has shown how changing the underlying time-span assumptions in an analysis of mandatory minimum sentencing laws for drug offenses can dramatically change an assessment of efficiency. In his analyses, the use of short time horizons make it appear that longer prison sentences are more cost-effective, whereas longer time hori- zons highlight that conventional enforcement and sentencing strategies, as well as treatment, may be more cost-effective.48

Limitations of Cost-Efficiency Evaluation

Many cost-efficiency evaluations are subject to what might be called the house-of-cards criticism. That is, they build on a wide range of implausi- ble or untested assumptions, unreasonable time frames or discount rates, or incorrect cost and benefit estimates (e.g., the magnitude of expected policy impact may be substantially off the mark). In such cases, any result- ing net benefit or cost-benefit ratio may include so much error as to be meaningless.

“Garbage in, garbage out” captures the problem in a more colorful way. Consider, for example, an efficiency evaluation in a context where an impact evaluation suffered from a number of critical flaws. Perhaps, for example, it failed to control adequately for selection biases. The result? An effi- ciency evaluation then proceeds to produce inflated cost-effectiveness or

224 AMERICAN CRIMINAL JUSTICE POLICY

cost-benefit estimates. Or perhaps the impact evaluation failed to iden- tify or quantify important negative unintended effects. Once, again, the efficiency estimates would be inflated. To illustrate, a cost-efficiency evalua- tion of supermax prisons that failed to include information about potentially adverse effects on inmate mental health would overstate the benefits of these prisons relative to their costs.

Another important limitation stems from the perspective of analysis. Quite simply, the perspective taken may influence the results because it “may require selectively taking different costs and outcomes into account, depending on the perspectives and values of sponsors, stakeholders, tar- gets, and evaluators themselves.”49 Consider the possibility that a crime- prevention program may reduce crime in one area but do so primarily by displacing it to a neighboring county or state. (Alternatively, it may reduce crime in both places.50) From the perspective of the county with the crime- prevention program, the program created a benefit (i.e., reduced crime). From a societal perspective, however, the program produced no benefit. It merely displaced crime and consumed resources in the process. The problem in such situations lies in the fact that only careful consumers of cost-benefit research may appreciate how different the results might be were a different perspective to be used, especially if descriptions of the results neglect to highlight the potential variability of the results.

One of the more prominent limitations of efficiency evaluations concerns intangible costs and benefits. What, for example, is the proper or accurate monetary value of fear, trust in and satisfaction with criminal courts, and any of a number of crimes? Estimates exist, but scholars disagree about their validity.51 Similarly, what value should be put on procedural injustice? Such injustice first has to be documented. Perhaps a prison system unfairly places some inmates in supermax housing or a local law enforcement agency disproportionately pulls over minority speeders. The next logical step might be to factor a reduction in such injustice as a benefit or an increase as a cost. However, few credible estimates exist for placing a monetary value on a “unit” of perceived or actual injustice.

These observations underscore the fact that cost-benefit analysis “is not a value-free concept but instead involves definitions and explicit boundaries to determine whose costs and benefits matter.”52 For example, a conventional cost-benefit analysis would not consider the pain and suffering that inmates experience while incarcerated, but most inmates presumably would hold a different view about that decision.

Although some of these limitations can be avoided, some cannot. That does not necessarily undermine the value of cost-efficiency evalua- tions. Instead, it simply highlights the importance of presenting important

COST-EFFICIENCY EVALUATIONS 225

assumptions and caveats alongside of efficiency estimates. For example, one can explicitly state that a particular efficiency evaluation did not include information about a policy’s effects on procedural injustice. Such steps can help to avoid what James Kee has referred to as the “black box syndrome,” which occurs when researchers are “tempted to hide the messiness of the analysis from the decision maker” and instead report concise synopses that omit or gloss over important limitations, even when the “messiness” might reveal contradictory results depending on the assumptions used in the anal- yses.53 Recall, again, that society and various agencies implicitly undertake cost-efficiency evaluations every time they invest in a particular policy and not another. In such cases, the same limitations apply but arguably are greater because a variety of assumptions go unarticulated and untested.

Why Are Cost-Efficiency Evaluations Important?

Cost-efficiency evaluations serve several important purposes. First, they pro- vide a general sensitizing function similar to that of needs and theory evalu- ations by clarifying a policy’s goals, the outcomes relevant to assessing those goals, and a policy’s costs. For example, an ex ante efficiency evaluation (i.e., one that occurs prior to funding a new initiative) can build on needs and theory evaluations by drawing attention to the importance of identifying the exact measures that will be used to assess policy impact.54

Second, they can provide guidance about how likely it is that a given pol- icy or set of policies will produce substantial returns. Impact evaluations do not provide such information, and precisely for that reason typically lead to different policy implications than do cost-efficiency evaluations.55 Here, again, ex ante efficiency evaluations can be helpful. Such evaluations neces- sarily must proceed with estimates of expected rather than actual benefits. When, however, reasonable grounds exist for anticipating a level of impact and for monetizing it, they can provide a reasonable foundation for produc- ing efficiency estimates that in turn can be credibly used to inform policy deliberations.

Third, ex post efficiency evaluations (i.e., those that occur after a policy has been implemented and its impacts have been assessed) provide critical information for deciding whether to continue, expand, or terminate a pol- icy. Cost-effectiveness analyses can inform such decisions, but cost-benefit analyses provide a far more useful platform for them. A cost-effectiveness analysis tells us which of several options may produce a given outcome (e.g., reduced gang-related crime) for the least amount of money. That information can be quite useful in deciding among these options. How- ever, it provides little foothold for comparing policies with different goals,

226 AMERICAN CRIMINAL JUSTICE POLICY

including policies that target different crimes. By contrast, cost-benefit anal- yses enable us to develop comparisons, using a standardized metric (i.e., dollars), of the policies’ relative costs and benefits.

Fourth, cost-efficiency evaluations – because of all the elements that go into them – force us to be explicit about the range of goals that inform our judgments about a policy’s worth, to identify potential unintended effects, to acknowledge empirically unverified assumptions that substantially influ- ence our assessments, and also to acknowledge the value judgments that contribute to support or opposition to a policy. A cost-efficiency evalua- tion might show, for example, that supermax prisons produce more benefits than costs, but only if we assume moderate to strong reductions in disor- der and violence. And it might highlight that it cannot address concerns about whether supermax confinement constitutes a humane form of incar- ceration, but that such concerns should be considered as part of a balanced deliberation about the merits of supermax prisons.

Fifth, in some cases, efficiency evaluations can be used to monitor per- formance. If, for example, the estimated efficiency of a community policing program declines over time, it might indicate a need to examine the program more closely. Perhaps smaller reductions in violence are occurring than in the early phases of the program and could be increased with more consistent or higher quality implementation. Clearly, we could monitor crime rates over time and arrive at a similar concern. Efficiency estimates, however, would enable us to put a dollar value on what the changes mean in bottom-line terms – that is, the returns on a continued investment in the policy.56

Case Studies: Community Policing and Private Prisons

Community policing and private prisons have both emerged as promi- nent policies in recent decades. Although community policing is more widespread, prison privatization has steadily increased, tapping in part into a broader societal shift toward privatizing government functions. Here, the two policies are presented as case studies to illustrate the importance of cost-efficiency evaluations and some of the ideas central to conducting and interpreting them. After discussing these case studies, the discussion turns to an exploration of the state of criminal justice policy as viewed through an efficiency lens.

Community Policing

THE POLICY. Policing models have changed over time.57 Until recent de- cades, the traditional policing approach centered on a reactive model in

COST-EFFICIENCY EVALUATIONS 227

which patrol officers would wait for calls for service.58 Under this model, the locus of control for identifying and addressing problems rested primarily with the police, not the community. Moreover, police supervisors dictated the focus and nature of police activity. The result? Officers tended to feel isolated from the community, and residents tended to resist or feel ambivalent toward the police.

Community policing emerged in part as a response to the perceived inef- fectiveness of this model in responding to crime. During the 1980s, when violent crime rates were on the rise, law enforcement agencies across the country revised their approach. They zeroed in on the idea that their offi- cers and the communities in which they worked held unique insight into ways in which crime might be better prevented and addressed. The result? Community policing. This new model aimed to develop community-police relationships that would be collaborative in identifying crime and how best to solve it.59

The end goal of community policing is to reduce crime. However, other goals exist as well. Successful community policing ventures should also lead to greater community order and to residents feeling safer. Residents also should have greater trust in the police and feel more satisfied with them. Even if these other goals did not reduce crime, they would be considered important ends in and of themselves. That fact means that efforts to evaluate the impacts of community policing should take the different goals, not just crime, into account.60

COST-EFFICIENCY OF COMMUNITY POLICING. Almost no research exists on the cost efficiency of community policing despite the popularity of this approach to law enforcement and the substantial federal and state funding of it.61 Regardless, any efficiency evaluation would be problematic because of the limited evidence of community policing impacts on fear of crime; trust in and satisfaction with the police; disorder; and, not least, crime.62

To be certain, studies exist that suggest that community policing can yield improvements along all of these dimensions.63 However, few such studies rely on rigorous methodologies, and many of them provide no such evidence.

At the same time, considerable and consistent evidence exists that docu- ments the fact that the implementation of community policing initiatives is highly variable. This fact alone suggests that any potential impacts – and thus efficiency – may be highly variable. They may depend greatly, for example, on the characteristics of communities. They also may depend on the precise activities that constitute community policing (e.g., shorter res- ponse times, information sharing with the public, meetings with residents, problem-solving cooperative efforts with residents). Not least, they depend

228 AMERICAN CRIMINAL JUSTICE POLICY

on the intensity and quality with which communities and the police under- take such activities.

This situation characterizes another popular and widely prevalent law enforcement strategy that emerged during the 1980s and 1990s: problem- oriented policing. The main difference between the two approaches is that “community policing has a softer image and more preventive orientation . . . , relying more on engaging the public as partners in reducing crime and disorder than on aggressive law enforcement.”64 By contrast, the problem- oriented approach retains a fundamentally police-centered emphasis, one in which the police identify problems – in particular, the root causes of crime – and then develop and assess responses to them.65 As with the literature on community policing, few rigorous evaluations of this widely used strategy exist, but some studies suggest that it may modestly reduce crime.66 With problem-oriented policing, the primary outcome is crime, not fear of crime or public trust in or satisfaction with the police.

OBSERVATIONS AND IMPLICATIONS. Community policing has cost billions in taxpayer dollars. Between 1995 and 2000 alone, for example, $8.8 billion in federal funding went to support the hiring of more police officers and to improving community policing efforts nationwide.67 Yet basic questions remain about whether such investments have produced appreciable impacts on crime or other community policing goals. It thus is too soon to know whether any impacts, should they exist or be of any sizable magnitude, are cost efficient.

What, though, would cost efficiency look like? One approach would be to identify the extent to which the cost of preventing crime is lower than traditional policing or other new approaches to policing. Cost-effectiveness analyses would facilitate such comparisons. However, they would require treating all crime as more or less equal. One solution would be to weight some crimes more than others, which is precisely what a cost-benefit anal- ysis would allow. A cost-benefit analysis would be especially relevant for assessing the efficiency of community policing, given that it aims to improve several qualitatively distinct types of outcomes (e.g., crime, disorder, fear). Among other things, it would provide a basis for comparing community policing benefits with those of police strategies that aim exclusively to reduce crime.

Although reliable impact estimates of community policing do not yet exist, cost-efficiency evaluations nonetheless could prove useful in identify- ing whether expected impacts would, at a minimum, offset the costs. Per- haps, for example, small to modest reductions in crime alone would suffice

COST-EFFICIENCY EVALUATIONS 229

to break even. However, perhaps large reductions, especially in serious or violent crime, would be required.

The fact that community policing targets multiple outcomes means that comprehensive cost-benefit analyses would need not only valid impact esti- mates for each outcome but also valid estimates of the monetary value of them. For example, we would need defensible estimates of the value of specific types of crime and the fear of each type. Such estimates could be derived from a number of sources, including surveys that ask people how much they would be willing to pay to obtain reductions in crime.68

The perspective of an efficiency analysis would be especially critical to consider when interpreting any results. For example, if we focused exclu- sively on neighborhoods where community policing was implemented, we might find that this strategy was cost-efficient. However, consider the possi- bility that crime could have increased in other nearby neighborhoods where community policing was not implemented. If we focused on the perspective of the neighborhoods, we might find that the strategy was inefficient because of crime displacement effects. If we focused on the entire county, city, or town, we presumably would average out these estimates. Doing so, however, would obscure area-specific costs and benefits that might be important to recognize. Such possibilities illustrate the importance of exploring multiple perspectives of analysis and whether efficiency estimates vary or overlap when doing so.

Private Prisons

THE POLICY. Calls for privatizing government functions emerged as a prominent theme in the 1980s and 1990s, one that resonated strongly in the corrections arena. The embrace of privatization can be seen in national statistics. By 2000, there were 87,369 inmates held in private facilities, and by 2008, that number had increased by 47 percent to 128,524 inmates and accounted for 8 percent of the total state and federal prison population.69

The privatization efforts have been more successful at the federal level: as of 2008, 16.5 percent of federal inmates were housed in private facilities, more than double the 6.8 percent of state prisoners in such facilities.70

Why privatize prisons?71 The main argument is that private prisons can provide the same care and control and achieve the same or better outcomes for less cost. Some people feel that the government operates inefficiently, and so they assume that privately run prisons must be more efficient. Others feel that government agencies operate quite efficiently and that, from an ethical standpoint, certain functions – such as housing, managing, and treating

230 AMERICAN CRIMINAL JUSTICE POLICY

prisoners – should only be undertaken by government. From this perspective, the idea of private companies making a profit by performing a state function (i.e., punishment) seems inappropriate. A related argument suggests that a profit motive creates an incentive to cut services and treatment, regardless of the well-meaning intentions of private company administrators or staff.

COST EFFICIENCY OF PRIVATE PRISONS. An evidence-based approach to corrections cannot address the ethical critiques of prison privatization. It can, however, inform debates about the basic assumptions needed to support a cost-efficiency argument. What specific information would help?

Consider first the initial premise – that government-run prisons operate inefficiently. Do they? In fact, no standardized metric for prison efficiency exists. If it did, certain elements would be essential to include. For example, prisons typically must provide education and job training of some type and also physical and mental health services. They must maintain order and safety and do so in a humane and legally allowable manner. Not least, they typically must strive to minimize inmate recidivism.

If we reflect on these dimensions, it becomes evident that a cost-effective- ness study would be insufficient. Certainly, one could develop cost esti- mates and then juxtapose them against the number of inmates served who recidivate. Doing so would, however, ignore a number of other outcomes that would be relevant in any comparison. Perhaps, for example, the rate of inmate-on-inmate assaults in a privately run facility is twice that of a government-run prison. Such assaults would be critical to any comparison, as would such outcomes as the prevalence of postrelease mental illness, homelessness, and unemployment resulting from the incarceration experi- ence. Of course, we could line up all relevant outcomes and provide a cost- effectiveness assessment for each outcome (e.g., the cost per assault, the cost per recidivism event, the cost per mental illness). However, because the outcomes qualitatively differ from one another, we need a common metric, such as money, to create a uniform basis of comparison. In short, what we want is a cost-benefit analysis of public prisons to establish a baseline value of efficiency. This type of analysis, by and large, has not been conducted.

Next, consider a second premise implicit in the first; private prisons can provide the same services and the same or better outcomes as public prisons but at lower cost. The operational costs typically are relatively straightfor- ward to compare. If we could accept the ceteris paribus (all else equal) assumption, we could stop there. That assumption is, however, highly ques- tionable. What if implementation evaluations were to show that private prisons offer fewer services, that inmate abuse and drug use occurs more

COST-EFFICIENCY EVALUATIONS 231

frequently in them, and that inmate visitation and treatment for physical or mental health problems occur less frequently? And what if impact evalua- tions were to show that postrelease outcomes, such as recidivism, employ- ment, and homelessness, were worse as compared to public prisons? Alter- natively, what if private prisons fare better on all these dimensions? Any substantial deviations would greatly alter any efficiency analysis.72

So, what does the research say? Some studies suggest that private prisons are no more cost-efficient than public prisons, and a small handful suggest that they may be slightly more cost-efficient.73 The more relevant findings, however, are that (a) few cost-efficiency studies of private prisons exist, (b) those that do exist use widely varying and frequently inappropriate method- ologies, and (c) none systematically compare public and private prisons – much less ones that are comparable in all respects (e.g., types of inmates, custody levels) – with respect to the implementation of required services and activities or to a range of in-prison and postrelease outcomes.74 The result, then, is a situation in which only apples-to-oranges comparisons can be made.

OBSERVATIONS AND IMPLICATIONS. Public prisons may be inefficient and private prisons may be the solution to this problem. As with commu- nity policing, such claims remain largely undocumented. It thus remains unknown whether the privatization trend has been beneficial. What is known is that existing efficiency estimates produce widely varying esti- mates depending on the methodologies and assumptions used to produce them.75

This situation reflects a broader problem in the field of corrections – namely, few states or jurisdictions have performance-monitoring systems in place sufficient for creating valid comparisons of the performance among public prisons, much less the performance of public versus private facilities, along a range of dimensions (see, for example, Table 6.1).76 Such monitor- ing would facilitate implementation, impact, and efficiency evaluations. In addition, it would enable evaluators to identify facility-specific cost-benefit estimates. This specificity is important because facilities vary greatly in the types of inmates they house and in the type and quality of administration and staffing. Average estimates among public prisons or among private pris- ons obscures this variation and creates the incorrect impression that, say, all private prisons are cost-beneficial when perhaps only some of them are.77

The good news is that increased attention has been given to efficiency evaluations of private prisons and that concrete guidance for conducting them now exists. Gaes and his colleagues, for example, have provided a

232 AMERICAN CRIMINAL JUSTICE POLICY

clear set of guidelines to undertaking such evaluations.78 These guidelines may promote greater consistency and rigor in prison efficiency analyses, which in turn should allow a body of work to emerge that more definitively identifies when, and under what conditions and with what populations, prison privatization results in taxpayer cost savings.

Are Current Criminal Justice Policies Cost Efficient?

By and large, the answer to this question is we don’t know. The vast bulk of criminal justice policies have not been subject to cost-efficiency evaluations, and extant efficiency evaluations typically suffer from critical methodologi- cal shortcomings or rely on sufficiently different methodologies as to make comparisons from one efficiency analysis to another meaningless.79 Com- parison of existing cost-benefit analyses of various policies, for example, is problematic because they frequently use different costs and benefits and different approaches to computing and monetizing them.80

The problem is actually worse – credible efficiency estimates depend on credible assessments of impact, but such assessments are also in short supply. Thus, even were we to undertake efficiency evaluations of the major criminal justice policies in place today, any resulting estimates would be vulnerable to the legitimate criticism that they do not accurately reflect true costs and benefits. Consider, for example, the policies examined in the case stud- ies, including, from this chapter, community policing and private prisons, and, from previous chapters, mass incarceration, sex crime laws, supermax prisons, faith-based prison programs, transfer laws, mandatory domestic vio- lence arrest laws, drug courts, and gun laws. The precise impacts, intended or otherwise, of these efforts remain unclear.

Much the same can be said of many other criminal justice policies. Con- sider unemployment checks, which recently have been lauded as a way to protect employers and their staff and customers while others have criticized them as constituting a significant barrier to gainful employment among released prisoners. We know little, however, about how these checks affect ex-prisoner postrelease employment.81 Even if we had robust estimates of the impacts, a great deal more information would be needed for cost-benefit analyses. As Richard Freeman has noted, we would want reliable and valid estimates of such dimensions as “the impact of the reduced lower hiring on ex-offender criminal behavior; the costs of this additional criminal behav- ior; . . . the potential differential productivity between someone without a criminal record and someone with a record; and the possible deterrent effect of information on criminal behavior nonoffenders.”82

COST-EFFICIENCY EVALUATIONS 233

Consider, too, the many practices and decision-making points in criminal justice, which constitute, in their own right, policies. Most go unexamined and yet hold the potential to create considerable costs and benefits for soci- ety.83 For example, when law enforcement agencies mistakenly target low- crime rather than high-crime hot spots for more intensive attention or when offenders receive treatments, sanctions, or classifications that do not match their particular risk and needs profiles, more recidivism and crime is likely.84

Without reasonably precise estimates of the impacts of these decisions, as well as the range of policies and programs that populate the criminal justice landscape, credible efficiency estimates will remain ephemeral.

One bright spot is the fact that economic analyses have begun to filter into more criminal justice research and policy evaluation.85 Another is that policy evaluation appears likely to become more frequent and rigorous given the sustained calls for increased government accountability and evidence-based practice. Any such trend likely bodes well for the frequency and quality of efficiency evaluations.

Still another bright spot exists – a few rigorous efficiency evaluations sug- gest that some criminal justice policies are in fact cost efficient. Steve Aos and his colleagues, for example, have conducted some of the most com- prehensive criminal justice efficiency analyses to date. For each of a range of policies, they created per-participant “net present values of the long-run benefits of crime reduction minus the net up-front costs.”86 They found that, from crime victim and taxpayer perspectives, the net present values of many policies were positive.87 To illustrate, consider the per-participant net present values – in parentheses and 2006 dollars – of the following evidence-based, adult-focused policies that they identified: vocational education in prison ($13,738); intensive supervision, treatment-oriented programs ($11,563); general education programs in prison ($10,669); cognitive-behavioral ther- apy in the prison or community ($10,299); drug treatment in the commu- nity ($10,054); correctional industries in prison ($9,439); drug treatment in prison ($7,835); adult drug courts ($4,767); employment and job training in the community ($4,359); and electronic monitoring to offset jail time ($870).88

The net benefits for youth-focused interventions were, in many cases, greater. Policies with positive net values included: multidimensional treat- ment foster care ($77,798), diversion for low-risk offenders ($40,623), family integrated transitions ($40,545), functional family therapy for youths on probation ($31,821), multisystemic therapy ($18,213), aggression- management training ($14,660), teen courts ($9,208), juvenile boot camps ($8,077), juvenile sex offender treatment ($7,829), restorative justice for

234 AMERICAN CRIMINAL JUSTICE POLICY

low-risk offenders ($7,067), interagency coordination programs ($5,186), and juvenile drug courts ($4,622).89

These estimates hinge heavily on assumptions Aos and his colleagues made about the estimated impacts of the different policies. Although they employed a rigorous set of criteria for identifying well-evaluated policies and for estimating impacts, other scholars might disagree about some of the analyses and assumptions. Estimates of incarceration costs and benefits are, for example, heavily contested.90 In addition, estimated impacts stemmed in some cases from only a few studies, and the efficiencies of many policies could not be evaluated because too few or no credible impact studies existed. Regardless, the approach taken by Aos, and the fact that they conducted the efficiency analyses at the behest of the Washington State Legislature, illustrates both that cost-efficiency analyses can feasibly be undertaken in criminal justice and that they can be used to inform policy debates and discussions.

Finally, an important bright spot involves estimates of the costs of crime. Cohen and his colleagues, as well as other scholars, have devised increasingly sophisticated approaches to identifying the costs of crime to society. By one account, the “present value of saving a fourteen-year-old, high-risk juvenile from a life of crime [ranges] from $2.6 to $5.3 million.”91 Such information can be used to anchor efforts to provide realistic and defensible estimates of savings stemming from different criminal justice policies. At the same time, meta-analyses increasingly have provided more credible estimates of impacts that may be safely assumed for a variety of well-implemented policies.92

Such efforts hold the potential for increasing the use of efficiency eval- uations to inform policy debates and decisions. Among other things, they can be used to explore the assumptions underlying current policies. For example, for more than one hundred years, U.S. taxpayers have funded the operation of an entirely separate system of justice for juveniles, an adminis- trative encumbrance justified by the logic that the benefits necessarily offset the costs. A cost-efficiency analysis could be used to identify the implicit assumptions about the greater improvements that youth face when pro- cessed under this system of justice.93 Such assumptions could be juxtaposed against the best available evidence concerning adult and juvenile justice sys- tem policies and interventions. Clearly, an efficiency evaluation alone would not determine the fate of the juvenile justice system. Just as clearly, however, it could be used to clarify precisely what we expect in the form of returns from this system.

COST-EFFICIENCY EVALUATIONS 235

Conclusion

Cost-efficiency evaluations constitute the final step in the evaluation hierar- chy and enable us to assess the returns on criminal justice policy investments. Two types of such evaluations exist. Cost-effectiveness analyses identify pol- icy costs and the outcomes that they produce. They are useful primarily when comparing two or more policies that share the same goals. Cost- benefit analyses identify policy costs and the benefits, expressed in monetary terms. They are useful for comparing two or more policies that have differ- ent or nonoverlapping goals. Efficiency evaluations can highlight whether a policy’s impact justifies the expense required to produce it. More gener- ally, they can inform and complement deliberations about whether policies should be implemented, continued, expanded, or terminated.

Few efficiency evaluations occur in criminal justice, in part because economic analysis has not featured prominently in the training of crim- inologists and criminal justice scholars.94 Regardless, the end result con- sists of a situation in which the vast bulk of criminal justice policies pro- ceed based on hunches and assumptions about both the costs and the benefits of these policies. Such a situation clearly runs counter to the recent calls for increased government accountability and effectiveness. For- tunately, it may well change in coming years because of this trend and because of the increased infusion of economic analyses into criminological research.

The basic steps in efficiency evaluations are straightforward, although each step can entail considerable complexity. The steps include (1) clearly stating the policy question; (2) determining the perspective of analysis, (3) identifying all relevant costs and benefits; (4) assigning monetary values to the costs and benefits; (5) contrasting the total costs and benefits; and (6) conducting sensitivity analyses to determine the consistency of the results under different sets of assumptions and, in this same vein, articulating any and all relevant limitations that may affect the credibility or interpretation of the results.

A number of limitations may undermine the credibility of efficiency eval- uations. For example, they may proceed based on estimated impacts that themselves lack credibility. The case studies of community policing and pri- vate prisons, as well as the other examples discussed in the chapter, illus- trate the pervasiveness of this particular problem. The limited information about the impacts of various criminal justice policies – including the mag- nitude of the impacts and how much they can be expected to emerge in a

236 AMERICAN CRIMINAL JUSTICE POLICY

diverse set of contexts – severely hamstrings the validity of efficiency esti- mates. Other limitations, such as the challenge of monetizing certain out- comes (e.g., rapes, murders, fear of crime, satisfaction with law enforcement performance), exist as well.

Even so, it bears emphasizing that society implicitly conducts an efficiency evaluation every time it invests in a policy. The sole difference is that no empirical analyses inform the implicit evaluation and the limitations remain obscure. Society arguably is better off with explicit rather than implicit analyses. Why? When we fail to identify the assumptions on which our decisions rest, we risk selecting policies that rest on highly questionable or incorrect assumptions. When we identify these assumptions, we have a better chance of flagging those that lack merit and in turn create the opportunity for selecting a policy that we can better trust will provide the re- turns we hope for and expect. Such improvements are critical given the stakes involved (e.g., crime, victimization, injustice) and the scarce resources with which the criminal justice system works.

Criminal justice policies and the debates about them are influenced by many factors. Efficiency evaluations constitute but one of them. Nonethe- less, they can provide critical information to improve the quality of delib- erations about a diverse range of criminal justice policies. However, for them to be useful and to be used appropriately, it is critical that we be aware of their conceptual underpinnings and their limitations. Even the best efficiency evaluations, for example, require judgment calls about such issues as the appropriate perspective of analysis, which outcomes should be weighed more heavily, and the extent to which estimated policy impacts can be trusted.

Discussion Questions

What are the benefits of evaluating the cost efficiency of a criminal justice policy? What are the problems of not evaluating policy cost efficiency?

What is the difference between a cost-effectiveness analysis and a cost- benefit analysis? When do you use one versus the other?

How do you conduct a cost-benefit analysis? What are the main challenges in conducting cost-efficiency evaluations of

criminal justice policies? Because it is not possible to conduct cost-efficiency evaluations of every

criminal justice policy, what criteria should we use for selecting policies that we will evaluate?

COST-EFFICIENCY EVALUATIONS 237

Given that policy makers and the public frequently seem to place a high premium on the idea of getting the most “bang for the buck,” why are cost-efficiency evaluations of criminal justice policies so rare?

What steps can be taken to increase the number and quality of cost- efficiency evaluations of criminal justice policies?