Business Intelligence 5

304 Part II • Predictive Analytics/Machine Learning

Introduction and Motivation

Analytics has been used by many businesses, organi- zations, and government agencies to learn from past experiences to more effectively and efficiently use their limited resources to achieve their goals and objec- tives. Despite all the promises of analytics, however, its multidimensional and multidisciplinary nature can sometimes disserve its proper, full-fledged application. This is particularly true for the use of predictive analyt- ics in several social science disciplines because these domains are traditionally dominated by descriptive analytics (causal-explanatory statistical modeling) and might not have easy access to the set of skills required to build predictive analytics models. A review of the extant literature shows that drug court is one such area. While many researchers have studied this social phenomenon, its characteristics, its requirements, and its outcomes from a descriptive analytics perspective, there currently is a dearth of predictive analytics mod- els that can accurately and appropriately predict who would (or would not) graduate from intervention and treatment programs. To fill this gap and to help author- ities better manage the resources, and to improve the outcomes, this study sought to develop and compare several predictive analytics models (both single models and ensembles) to identify who would graduate from these treatment programs.

Ten years after President Richard Nixon first declared a “war on drugs,” President Ronald Reagan signed an executive order leading to stricter drug enforcement, stating, “We’re taking down the surren- der flag that has flown over so many drug efforts; we are running up a battle flag.” The reinforcement of the war on drugs resulted in an unprecedented 10-fold surge in the number of citizens incarcerated for drug offences during the following two decades. The sky- rocketing number of drug cases inundated court dockets, overloaded the criminal justice system, and overcrowded prisons. The abundance of drug-related caseloads, aggravated by a longer processing time than that for most other felonies, imposed tremen- dous costs on state and federal departments of justice. Regarding the increased demand, court systems started to look for innovative ways to accelerate the inquest of drug-related cases. Perhaps analytics-driven deci- sion support systems are the solution to the problem.

To support this claim, the current study’s goal was to build and compare several predictive models that use a large sample of data from drug courts across different locations to predict who is more likely to complete the treatment successfully. The researchers believed that this endeavor might reduce the costs to the criminal justice system and local communities.

Methodology

The methodology used in this research effort included a multi-step process that employed pre- dictive analytics methods in a social science con- text. The first step of this process, which focused on understanding the problem domain and the need to conduct this study, was presented in the previous section. For the steps of the process, the research- ers employed a structured and systematic approach to develop and evaluate a set of predictive models using a large and feature-rich real-world data set. The steps included data understanding, data pre- processing, model building, and model evaluation; they are reviewed in this section. The approach also involved multiple iterations of experimentations and numerous modifications to improve individual tasks and to optimize the modeling parameters to achieve the best possible outcomes. A pictorial depiction of the methodology is given in Figure 5.25.

The Results

A summary of the models’ performances based on accuracy, sensitivity, specificity, and AUC is pre- sented in Table 5.10. As the results show, RF has the best classification accuracy and the greatest AUC among the models. The heterogeneous ensemble_ (HE) model closely follows RF, and SVM, ANN, and LR rank third to last based on their classifica- tion performances. RF also has the highest specific- ity and the second highest sensitivity. Sensitivity in the context of this study is an indicator of a model’s ability in correctly predicting the outcome for suc- cessfully graduated participants. Specificity, on the other hand, determines how a model performs in predicting the end results for those who do not suc- cessfully complete the treatment. Consequently, it can be concluded that RF outperforms other models for the drug courts data set used in this study.

Application Case 5.6 To Imprison or Not to Imprison: A Predictive Analytics-Based Decision Support System for Drug Courts

Chapter 5 • Machine-Learning Techniques for Predictive Analytics 305

10-fold Cross-Validation

Data Preprocessing Merging Aggregating Cleaning Binning Selecting

True Positive Count (TP)

False Positive Count (FP)

True Negative Count (TN)

False Negative Count (FN)

True/Observed Class Positive Negative

P os

it iv

e N

e ga

ti ve

P re

di c te

d C

la s s

100 90

80 70

60 50

40 30

40 50

Variable Names

Im po

rt a nc

e

X1

X2

ANN

LR

SVM

RF

Pre-processed Data

SplittingData Preparation Modeling Assessment

Treat. DB Case DB Court DB

Domain Expert(s)

X1

X2

M ax

im um

-m ar gi n hy

pe rp lan

e

M argin

HE

Individual Models

Ensemble Models

10 %

10 %

10 %

10 % 10 %

10% 10%

10%

10%

10%

10%

10% 10%

10%

10%

Variable Importance

Prediction Accuracy 26 24 22 0

0.5

1

2 4 6

26 24 22 0

0.5

1

2 4 6

FIGURE 5.25 Research Methodology Depicted as a Workflow.

TABLE 5.10 Performance of Predictive Models Using 10-Fold Cross-Validation on the Balanced Data Set

Model Type

Confusion Matrix Accuracy (%)

Sensitivity (%)

Specificity (%) AUCG T

In d

iv id

u a l M

o d

e ls ANN

G 6,831 1,072 86.63 86.76 86.49 0.909

T 1,042 6,861

SVM G 6,911 992

88.67 89.63 87.75 0.917 T 799 7,104

LR G 6,321 1,582

85.13 86.16 81.85 0.859 T 768 7,135

E n

se m

b le

s

RF G 6,998 905

91.16 93.44 89.12 0.927 T 491 7,412

HE G 6,885 1,018

90.61 93.66 87.96 0.916 T 466 7,437

ANN: artificial neural networks; DT: decision trees; LR: logistic regression; RF: random forest; HE: heterogeneous ensemble; AUC: area under the curve; G: graduated; T: terminated

(Continued )

306 Part II • Predictive Analytics/Machine Learning

u SECTION 5.9 REVIEW QUESTIONS

1. What is a model ensemble, and where can it be used analytically? 2. What are the different types of model ensembles? 3. Why are ensembles gaining popularity over all other machine-learning trends? 4. What is the difference between bagging- and boosting-type ensemble models? 5. What are the advantages and disadvantages of ensemble models?

Although the RF model performs better than the other models in general, it falls second to the HE model in the number of false negative predictions. Similarly, the HE model has a slightly better performance in true negative predictions. False positive predictions represent participants who were terminated from the treatment, but the models mistakenly classified them as successful graduates. False negatives pertain to individuals who graduated, but the models predicted them as dropouts. False positive predictions are syn- onymous with increased costs and opportunity losses whereas false negatives carry social impacts. Spending resources on those offenders who would recidivate at some point in time during the treatment and, hence, be terminated from the program prevented a number of (potentially successful) prospective offenders from participating in the treatment. Conspicuously, depriv- ing potentially successful offenders from the treatment is against the initial objective of drug courts in reinte- grating nonviolent offenders into their communities.

In summary, traditional causal-explanatory sta- tistical modeling, or descriptive analytics, uses sta- tistical inference and significance levels to test and evaluate the explanatory power of hypothesized underlying models or to investigate the association

between variables retrospectively. Although a legiti- mate approach for understanding the relationships within the data used to build the model, descriptive analytics falls short in predicting outcomes for pro- spective observations. In other words, partial explana- tory power does not imply predictive power, and predictive analytics is a must for building empirical models that predict well. Therefore, relying on the findings of this study, application of predictive analyt- ics (rather than the sole use of descriptive analytics) to predict the outcomes of drug courts is well grounded.

Questions for Case 5.6

1. What are drug courts and what do they do for the society?

2. What are the commonalities and differences between traditional (theoretical) and modern (machine-learning) base methods in studying drug courts?

3. Can you think of other social situations and sys- tems for which predictive analytics can be used?

Source: Zolbanin, H., and Delen, D. (2018). To Imprison or Not to Imprison: An Analytics-Based Decision Support System for Drug Courts. The Journal of Business Analytics (forthcoming).

Chapter Highlights

• Neural computing involves a set of methods that emulates the way the human brain works. The basic processing unit is a neuron. Multiple neurons are grouped into layers and linked together.

• There are differences between biological and arti- ficial neural networks.

• In an artificial neural network, knowledge is stored in the weight associated with each connec- tion between two neurons.

• Neural network applications abound in almost all business disciplines as well as in virtually all other functional areas.

• Business applications of neural networks include finance, bankruptcy prediction, time-series fore- casting, and so on.

• There are various neural network architectures for different types of problems.

• Neural network architectures can be applied not only to prediction (classification or estimation)

Application Case 5.6 (Continued)