Criminal Justice Research Paper

Incremental Prediction of Intimate Partner Violence: An Examination of Three Risk Measures

Mark E. Olver University of Saskatchewan

Sandy Jung MacEwan University

Improvements in the risk prediction of domestic violence against intimate partners have the potential to inform policing practices in the prevention of further victimization. The present study examined the incremental predictive validity of 3 measures of risk for intimate partner violence (IPV)—Spousal Assault Risk Assessment (SARA), Ontario Domestic Assault Risk Assessment (ODARA), and the Family Violence Investigative Report (FVIR)—for IPV, general violence, and general recidivism outcomes. The sample featured 289 men and women who were reported to police for IPV and followed up approximately 3 years post release. Archival ratings of the 3 measures demonstrated that SARA scores showed incremental validity for IPV recidivism, ODARA scores incrementally predicted general vio- lence, and both tools incrementally predicted general recidivism. The FVIR did not incrementally predict any outcomes. Fine grained analyses demonstrated that the Psychosocial Adjustment domain of the SARA contributed most uniquely to the prediction of IPV. Survival analysis supported the use of the SARA and ODARA in tandem for appraising risk for IPV or general criminal recidivism. Calibration analyses using logistic regression modeling also demonstrated 3-year recidivism estimates for SARA and ODARA scores. Implications for the use of multiple tools in clinical practice are discussed, particularly for combining the SARA and ODARA measures to augment IPV risk assessment and management.

Public Significance Statement This study suggests that use of the Ontario Domestic Assault Risk Assessment (ODARA) and Spousal Assault Risk Assessment (SARA) tools in tandem can augment the assessment of risk and prediction of recidivism with intimate partner violence (IPV) offenders.

Keywords: intimate partner violence, SARA, ODARA, risk assessment, recidivism

Research on risk assessment of domestic violence, or more specifically violence against intimate partners (i.e., intimate part- ner violence or IPV), has progressed in the past two decades with the development and validation of several measures. The utility of a few of these instruments have been empirically examined with both convicted IPV offenders and preconvicted perpetrators of police-reported IPV, and there is consistent evidence indicating

empirically based risk assessments can meaningfully predict IPV recidivism. Moreover, since law enforcement is generally the first offender-focused service that will identify a suspect in a sexual assault case and in a unique position to coordinate prevention and early intervention responses, adoption of evidence-based ap- proaches by police services offers a practical and scientific solu- tion to ensure public safety and proper allocation of resources. The present study examines both the predictive accuracy of three tools and the viability of combining the tools to improve on their predictive accuracy. Specifically, the incremental improvement of risk assessment for IPV is examined for their potential to substan- tially improve the accuracy of current risk assessment methods.

A recent review of 39 published studies with 19 different intimate partner risk assessment measures (Nicholls, Pritchard, Reeves, & Hilterman, 2013) identified and described existing risk assessment measures and summarized the psychometric properties of these measures, including information on their predictive valid- ity. Two of these measures have received particular attention in the literature and include the Spousal Assault Risk Assessment Guide (SARA; Kropp, Hart, Webster, & Eaves, 1995) and the Ontario Domestic Assault Risk Assessment (ODARA; Hilton et al., 2004). The SARA was one of the first scales developed for use with domestically violent offenders. Version 2 of the SARA, which enjoyed widespread use for more than 20 years, is a 20-item

This article was published Online First June 26, 2017. Mark E. Olver, Department of Psychology, University of Saskatchewan;

Sandy Jung, Department of Psychology, MacEwan University. We express our deep appreciation to the Domestic Offender Crimes

Section and the Office of Strategy Management of Edmonton Police Services and specifically, Brian Roberts, Sean Armstrong, and Devin Laforce for their direction in this research. We also thank Wojciech Kujawa, Maxine Tremblay, and Melissa Russo for their research support, and Chad Tawfik and Carlos Cardoso for their ongoing support of this research. The points of view expressed in this article do not necessarily represent the views of the Edmonton Police Service.

Correspondence concerning this article should be addressed to Mark E. Olver, Department of Psychology, University of Saskatchewan, 9 Campus Drive, Arts 154, Saskatoon, Saskatchewan, Canada S7N 5A5. E-mail: [email protected]

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

Law and Human Behavior © 2017 American Psychological Association 2017, Vol. 41, No. 5, 440 – 453 0147-7307/17/$12.00 http://dx.doi.org/10.1037/lhb0000251

440

structured professional judgment (SPJ) tool in which item ratings (of 0, 1, or 2) are scrutinized and translated into risk ratings of low, medium, or high; total scores are not computed in clinical practice or linked to risk categories. Moreover, the SARA has several items, particularly those subsumed under the Psychosocial Adjust- ment section, that are potentially dynamic in nature and which dovetail with the Central Eight criminogenic needs within An- drews and Bonta’s (2010) risk-need-responsivity (RNR) frame- work. For instance, Item 4 “Recent relationship problems” may have an analogue with Family/Marital, Item 5 “Recent employ- ment problems” with Employment/Education, Item 7 “Recent sub- stance abuse dependence” with Drug/Alcohol problem, Items 8 and 9 “suicidal/homicidal ideation, psychotic and/or manic symp- toms” with Personal/Emotional concerns, and Item 10 “Personality disorder with anger, impulsivity, and behavioral instability” with Antisocial Personality Pattern. In addition, the Spousal Assault History section has item content tapping the attitudes and cogni- tions of perpetrators (e.g., Items 16 and 17 “extreme denial/ minimization and attitudes supportive of spousal assault”), which conceptually might be seen as IPV counterparts to Procriminal Attitudes in the Central Eight framework.

Although research has yet to formally examine whether poten- tially criminogenic item domains of the SARA are truly dynamic, an abundance of past research has demonstrated that the SARA has good predictive validity. These validation studies have shown the SARA to have adequate ability to predict future IPV with an average receiver operating characteristic (ROC) area under the curve (AUC) value of .63 (studies ranged from .59 to .77; see review by Helmus & Bourgon, 2011) with a weighted average AUC of .63 (Messing & Thaller, 2013). This prediction effect size would be considered approaching moderate in magnitude based on the effect size rubric of Rice and Harris (2005) in which AUC values of .56, 64, and .71 correspond to small, medium, and large effect sizes, respectively.

In rating the tools, many of these studies have used file infor- mation only (e.g., Hilton et al., 2004) and have employed a quantification of the measures’ total scores, despite recommenda- tions from some authors, particularly for SPJ measures, to eschew summing the items and report the accuracy of the risk judgment (Nicholls et al., 2013). In addition to its original use by mental health professionals, Belfrage et al. (2012) reported that the SARA can be used by police officers and has shown to have a moderate effect size (AUC � .63), although a spin-off of the SARA, the Brief Spousal Assault Form for the Evaluation of Risk (B-SAFER; Kropp, Hart, & Belfrage, 2005) was subsequently constructed for use by police and showed a moderate to high effect for predicting IPV recidivism (AUC � .70; Storey, Kropp, Hart, Belfrage, & Strand, 2014).

The ODARA was developed by its authors to be used by law enforcement as an actuarial risk measure for predicting spousal assault recidivism (Hilton et al., 2004; Hilton, Harris, & Rice, 2010). Based on the rubric from Rice and Harris (2005), the authors have demonstrated the ODARA to have moderate to large effect sizes with AUC’s ranging from .65 to .74 (Hilton, Harris, Rice, Houghton, & Eke, 2008; Hilton & Harris, 2008) and a weighted average AUC of .67 (Messing & Thaller, 2013). The ODARA is comprised of 13 binary items, most of which are static in nature given that they pertain to general and IPV offense history, features of the index offense, and details of the perpetrator’s

relationship with the victim. With total scores ranging from 0 –13, development and validation research indicates that scores of 7 or more on the ODARA represent the highest risk for intimate partner assault and its severity, and is associated with intimate partner homicide (Eke, Hilton, Harris, Rice, & Houghton, 2011). Since the development of the ODARA, an examination of the Violence Risk Appraisal Guide’s (VRAG; Harris, Rice, & Quinsey, 1993) ability to be domain specific in predicting IPV recidivism inspired the development of the Domestic Violence Risk Appraisal Guide (DVRAG; Hilton et al., 2008). The DVRAG, which includes the Hare Psychopathy Checklist-Revised (Hare, 1991, 2003; PCL-R) score as a fourteenth item added to the ODARA, yielded an AUC of .70 (Hilton et al., 2008).

Both the SARA and ODARA measures have shown that they have moderate to large effects for predicting IPV recidivism. Although Nicholls et al.’s (2013) systematic review identified that these tools are particularly well researched and widely used, they note that other measures have been developed for assessing risk for IPV and femicide, albeit these studies are limited. For instance, the Domestic Violence Screening Instrument (DVSI), developed in the Colorado Domestic Violence Risk Reduction Project, demon- strated an AUC of .60 for IPV recidivism and an AUC of .68 for any recidivism during an 18-month follow-up period (Williams & Houghton, 2004), and its revision, the DVSI-R, showed a small effect for predicting family violence recidivism (AUC � .62) and moderate effect for predicting any recidivism (AUC � .66; Williams, 2012). One of the oldest tools available that was de- signed to assess lethal IPV is the Danger Assessment (DA; Camp- bell, 1986). Although it has been examined in numerous studies, various versions of the DA and different outcome measures (e.g., attempted lethal IPV, any IPV, general violent recidivism) have been used (see Storey & Hart, 2014, for a review), and no pro- spective studies using attempted or completed femicide as the outcome criterion have been conducted (Nicholls et al., 2013). Measures that were not specifically developed to assess for IPV risk, such as the Level of Service measures (e.g., LSI-OR, LSI; Hendricks, Werner, Shipway, & Turinetti, 2006; Hilton et al., 2010) have been examined, although the available body of re- search is limited and has tended to provide support for the predic- tion of recidivism outcomes aside from intimate partner violence.

Specific measures in the domain of IPV risk that are well- supported in the empirical literature are generally limited to the SARA and ODARA; their degree of predicting postindex violence against the same or other intimate partners is comparable (e.g., Campbell, French, & Gendreau, 2009; Nicholls et al., 2013), but few direct comparisons have been conducted (e.g., Hilton et al., 2004). A recent study (Jung & Buro, 2017) compared the AUC prediction magnitudes for the SARA, ODARA, and a regionally used measure, called the Family Violence Investigative Report (FVIR; Alberta Justice & Solicitor General, 2013) using a police- reported sample of IPV men. The authors found that the ODARA and their modified 14-item version of the SARA had moderate to large effect sizes for predicting new IPV convictions (AUCs � .70 and .74; respectively) and charges (AUCs � .66 and .68, respec- tively). Similarly, both measures predicted general violent recidi- vism well with the ODARA producing AUCs of .71 and .67 for convictions and charges, and the SARA producing AUCs of .72 and .66, for these criteria, respectively. On the other hand, the regionally produced tool, the FVIR, did not significantly predict

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

441INCREMENTAL PREDICTION OF IPV

reconvictions or further charges for violent or IPV recidivism (all AUCs � .57, except for violent charges, AUC � .54). Prior to the Jung and Buro (2017) study, the FVIR had yet to be empirically examined as a risk assessment tool.

Although existing research has demonstrated that the SARA and the ODARA perform comparably in predicting IPV recidivism, these tools were developed for related but slightly different pur- poses. The ODARA was originally developed as an easy-to-use actuarial tool to assess risk of spousal assault by front-line per- sonnel (Hilton et al., 2004). It shows good predictive accuracy for this outcome and other outcomes (e.g., Hilton et al., 2004; Jung & Buro, 2017), and it is feasible that it could inform service intensity at the front end, as per the risk principle (e.g., high ODARA scorers would receive high intensity services; see Jung & Buro, 2017). However, more than three quarters of the items are static in nature and the ODARA may thus be limited when informing intervention services or assessing changes in risk; that is, it would do little to inform dynamic areas to target for IPV intervention, and the tool is not structured to assess possible reductions in risk from treatment or other change agents.

In contrast, aside from being an SPJ measure, the SARA is different from the ODARA in two clinically relevant ways. First, as noted previously, nearly half of the items on the SARA are ostensibly dynamic variables that are potentially changeable and which have conceptual overlap with the Central Eight (e.g., recent employment instability, recent relationship instability, substance abuse concerns, assault supportive attitudes). In this respect, it may help inform treatment and IPV risk management. Second, the SARA was developed from a comprehensive list of empirically supported and clinically useful risk items linked to IPV recidivism (Kropp et al., 1995). With 20 such items, the SARA arguably provides more comprehensive coverage of the content domain of IPV risk, or at least it samples from different regions of that content domain. As such, there will likely be risk constructs embedded within the SARA that are unique to the SARA and not contained in the ODARA. Therefore, it stands to reason that there may be value added by using both measures in the context of an IPV risk assessment to increase predictive power since they would appear to measure a common overarching construct of IPV risk, but in some importantly different ways, which is the focus of the present study. We also examine capacity of the FVIR to potentially improve upon risk assessment and recidivism prediction of these two well-established IPV instruments, particularly given that it is mandated for use within its jurisdiction for police reported domes- tic incidents.

Present Study

The SARA and ODARA are used internationally to assess risk for IPV, to inform sentencing, bail decisions, conditional release, and supervision. In the Canadian context, the SARA and ODARA instruments are used by a range of service providers within cor- rectional and forensic mental settings, provincially and federally, while use of the FVIR is limited to first responders within the province of Alberta. In an expanded analysis of the Jung and Buro (2017) data, the present study sought to build on this previous work in order to examine the incremental validity of these three IPV risk measures in the prediction of recidivism among male and female perpetrators. The potential value in using multiple instru-

ments has received considerable attention in the sex offender risk assessment literature, with some research groups finding incre- mental predictive validity in combining multiple empirical actu- arial scales, such as through averaging (Lehmann et al., 2013), while others have not (Seto, 2005). Still, routinely static and dynamic sex offender risk measures are found to be incrementally predictive of sexual recidivism (Hanson, Helmus, & Harris, 2015).

By contrast, quantitative reviews of the IPV risk literature (Helmus & Bourgon, 2011; Nicholls et al., 2013) reveals that most, if not all, studies have heretofore examined the instruments in isolation, as IPV “risk instrument silos”; it has been our observa- tion that efforts to examine the integration of any of these tools has been noticeably absent from the literature. Although there is both overlapping and unique content among the three aforementioned IPV measures, it has yet to be examined to what extent adminis- tering multiple measures has value added, that is, incremental validity in the prediction of IPV recidivism as well as other recidivism outcomes. The present study sought to redress this gap in the literature.

Method

Participants

Data was coded from a random selection of 300 cases of IPV that were reported to a local police service over a 4-year period from 2010 to 2013. Eligibility criteria included IPV cases that involved a perpetrator and a complainant who were, at one time, intimate partners (e.g., break-up prior to the occurrence, currently partners), the incident led to formal charges for direct violence or the threat of direct violence to an intimate partner, and it was clear who was the perpetrator (i.e., cases were excluded in scenarios where both parties were assaultive in the incident and both were charged or not charged). The overall sample comprised mostly male perpetrators (n � 255; 85%) with female perpetrators repre- senting 15% of the sample (n � 45). All cases involved opposite sex intimate partner relationships. The average age of the perpe- trators was 34.5 years (SD � 10.74; range from 17.5 to 70.1). Most perpetrators were Caucasian (55.3%), followed by Aboriginal or Métis (30.3%), and the remainder (14.3%) included Black, Asian, South Asian, Hispanic, and Middle Eastern groups. Most of the perpetrators were current intimate partners at the time of the index occurrence (77%) and married to their victims (64.7%). A majority of the victims were female (84.3%) and 39% of perpetrators shared children with their victims. Over half (55%) of the perpetrators were criminally charged with level 1 assault (i.e., assaults that cause little or no physical harm) with a third (34.2%) charged with level 2 (i.e., assault with a weapon or assault causing bodily harm) or level 3 (i.e., causing serious bodily injury or life-threatening, such as aggravated assault; Dauvergne, 2009).

Measures

ODARA. The ODARA is an actuarial tool that was designed for police use to assess the risk of domestic violence recidivism among men who have already been identified by the police as having committed at least one such act (Hilton et al., 2004, 2010). The 13 items on the ODARA include the offender’s criminal history (e.g., prior domestic incident, failure on prior conditional

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

442 OLVER AND JUNG

release), aspects of current or prior domestic assaults (e.g., threat to harm or kill, confinement, assault on victim when pregnant), children in the relationship (e.g., more than one child), and the victim’s circumstances (e.g., barriers to victim support). For this study, we omitted the pregnant spouse item for female perpetrators (i.e., assigning a score of 0 to female perpetrators). The ODARA is intended to be scored from police and criminal justice records without a victim interview, and several studies have shown that the ODARA can be scored reliably from archival data in Canada (Hilton et al., 2010); therefore, retrospective scoring in IPV cases does not rely on proxy interviews.

SARA. The SARA is a structured professional judgment tool that was designed for use by mental health forensic pro- fessionals with interviewing and psychological assessment skills. It contains 20 items scored on a 3-point scale that are grouped into four sections: Criminal History, Psychosocial Ad- justment, Spousal Assault History, and Current Offense (Kropp et al., 1995). In the formal clinical use of the SARA, an overall judgment of risk is based on the items where the evaluator determines whether the offender is at a low, medium, or high risk of causing imminent harm to their intimate partner, and the items on the SARA are scored based on an interview with the offender and the victim, the use of standardized measure of psychological and emotional abuse, and collateral sources, such as criminal records (Kropp et al., 1995). A past study has shown that the interrater reliability for the items was moderate, sub- stantial item variability with intraclass correlations (ICCs) ranging from .45 to .86 (Kropp & Hart, 2000). In the current study, a 17-item version was used, which excluded three items, all of which had insufficient information to score for 80% of the cases, making use of missing data estimation procedures unten- able. The AUCs reported for the 14-item version in Jung and Buro’s (2017) study compared to the 17-item SARA in this study were virtually identical; the 17-item version is employed to increase generalizability of findings and to help ascertain which construct domains of the tool may be most predictive.

FVIR. The FVIR was created and mandated for use in the region where the study was conducted and was intended to be an investigative and case management tool for police to gather critical information within 12 hours of the incident (Alberta Justice & Solicitor General, 2013); the FVIR is not currently used in other provinces or countries. The use of the FVIR was consistent with tools that are used to make high stakes decisions in the criminal justice system, such as addressing questions and information re- quired for bail hearings, providing information for sentencing considerations, and providing Crown prosecutors with a quick overview of cases. Eighteen items of the FVIR were totaled mechanically to produce a score (see Jung & Buro, 2017, for list of items).

Recidivism. Recidivism outcomes included any new convic- tions, any violent convictions, and any intimate partner violence convictions (i.e., defined as violence against a current or past intimate partner). Data was coded from criminal records received from three sources obtained from federal, provincial, and local police information. Recidivism data was interpretable for 289 cases; 11 cases were omitted from the analyses of the recidivism data (e.g., criminal records were unavailable, perpetrators was in prison for duration of follow-up, perpetrator died).

Procedure

Items from the three measures described above and the recidi- vism outcomes were coded from extensive review of police file documentation. As the present study is retrospective in nature, these instruments were coded prior to obtaining recidivism data and thus blind to outcome to prevent criterion contamination. Each of the measures were mechanically totaled to produce scores for the ODARA, SARA (modified to include 17 items), and FVIR. A subset of the data (n � 20) were coded by a second rater to examine the interrater reliability of the items on the ODARA and the SARA. For the ODARA, the ICC was 0.90 and percentage agreements ranged from 74% to 100% for ODARA items (kappa coefficients ranged from 0.36 to 1.00). Most items were consis- tently coded by the two raters (above 80% agreement), except for item 10 (nondomestic violence against other person) and item 13 (barriers to victim support), where there was 75% and 74% agree- ment, respectively. For the SARA, the ICC was 0.84 and percent- age agreements ranged from 40% to 95% for 17 SARA items (kappa coefficients ranged from 0.12 to 0.80). Nine items were consistently coded (above 80%), and eight items fell below, al- though only one fell at 40% agreement (item 5, recent employment problems). It is important to note that the second rater had com- plete access to provincial and federal records but only limited access to the electronic police records when coding the items on the risk measures. The FVIR items were completed by police officers; therefore, interrater reliability was not examined for the FVIR. However, 7.5% of the entries made by officers were cor- rected by the study’s primary investigator (second manuscript author), and most of the items required corrections on 10% or less of the entries, although three items required corrections on 10.3 to 13.3% of the officers’ entries.

This research was reviewed and approved by the MacEwan Research Ethics Board and the Strategic Planning, Evaluation and Research Unit Office of Strategy Management Division of the Edmonton Police Service. As per the 2014 Tri-Council Policy Statement-2 Article 5.5A (pp. 64 – 65), researchers may use sec- ondary or archival sources of identifiable information from partic- ipants without obtaining consent if they have satisfied an institu- tional review board (IRB) that: a) identifiable information was essential to the research; b) the use of such information was unlikely to adversely affect participants; c) researchers took ap- propriate measures to protect the privacy of individuals and safe- guard identifiable information; d) researchers will comply with any preferences expressed by participants about use of their infor- mation (if any); e) it is deemed unreasonable and impractical to obtain consent directly from the parties involved; and f) research- ers have obtained any other necessary permissions for use of the secondary data (e.g., organizational approval from an agency). As all these criteria were judged by the IRB as being met, consent was waived for the purpose of conducting this research.

Data Analytic Plan

The primary objective of the present study was to examine the incremental predictive validity of the three IPV risk measures. To permit greater generalizability of study findings to other samples and settings as well as to maximize power, we included both men and women in the present study. The first set of analyses were simple convergent validity correlations among total scores on the

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

443INCREMENTAL PREDICTION OF IPV

three study measures to examine the extent to which they mea- sured a common construct of IPV risk.

To build on Jung and Buro (2017), three sets of incremental validity analyses were conducted using Cox regression survival analysis to predict IPV recidivism, any violent recidivism, or general violent recidivism over time. The scale total of one of the three measures (e.g., ODARA) was entered in the first block for each of these analyses to assess its univariate association with outcome, with the time to recidivism (or total follow-up time) entered as a time variable. The hazard ratio (eB) generated repre- sents the relative increase in the hazard of a given recidivism outcome for each one-point increase in risk score; values above 1.0 indicate a positive association between predictor and criterion. In the second block, one of the two additional measures (e.g., SARA or FVIR) would be entered to evaluate their incremental contribu- tions to the prediction of one of the three recidivism outcomes, controlling for the initial measure. Although the measures can be entered simultaneously to assess their unique contributions to outcome, and hence incremental validity, the predictors were en- tered in separate blocks in this fashion to more explicitly illustrate the examination of incremental predictive validity controlling for scores on the ODARA, SARA, or FVIR.

The aforementioned analyses informed the third set of analyses, which sought to examine the incremental contributions of the SARA’s scale components in the prediction of the three recidivism outcomes via Cox regression survival analysis, given the hetero- geneity of the SARA’s item content. This was done through entering all four components simultaneously—Criminal History, Psychosocial Adjustment, Spousal Assault History, and Current Offense. The unique predictors in each of the three regression models (i.e., for each of the outcomes) were then examined in relation to the ODARA for each outcome to ascertain which components would be incremental, and thus account for the bulk of the risk variance with respect to a given outcome.

The fourth set of analyses examined the calibration of the SARA and ODARA scales, that is, what recidivism rates were associated with different individual scores and risk bands on the tools. The analyses were partly informed by the preceding analyses with these two tools being prioritized given their psychometric profiles. IPV recidivism and general recidivism were prioritized as the two main outcomes for space considerations and parsimony. The planned analyses are described briefly in this section and elabo- rated upon in the Results where they are positioned in context. Kaplan Meier survival analyses were conducted on each tool to examine rates of these recidivism outcomes over time at low, medium, and high risk levels. In turn, the risk categories for the tools were combined and trajectories of recidivism were examined across five risk bands to ascertain if the SARA and ODARA used in tandem corresponded to more nuanced and accurate estimates of risk and recidivism.

The final set of calibration analyses examined actual and esti- mated rates of intimate partner violence recidivism over a fixed 3-year follow-up, at the three risk bands (actual recidivism) and while using logistic regression to generate recidivism estimates at the range of observed scores on the tools in the present sample. The analyses demonstrate to what extent actual and estimated rates of intimate partner violence recidivism align for actuarial and SPJ risk measures, and heretofore have not been conducted with the SARA or ODARA. Given that actual rates of recidivism tend to be

unstable, especially at extreme scores with small ns, and can vary across jurisdictions and settings, logistic regression can be used to estimate rates of recidivism associated with a given risk score (or set of scores) over a defined follow-up period based on the recid- ivism base rate of the sample and magnitude of the predictor- criterion association.

Results

Convergent Validity and Descriptive Statistics

The three measures demonstrated strong convergent validity as each had large positive correlations with each other ranging from the .60s to .70s, indicating the measurement of a common under- lying construct of IPV risk (see Table 1). Scrutiny of the means and standard deviations is informative. The SARA scores were by and large fairly low, even for the truncated 17-item version of the instrument, with the sample averaging less than 1-point (partially present) on a given item. The ODARA scores, by contrast were quite high relative to the development sample in Hilton et al. (2004), which had a mean of 2.89 (SD � 2.14), in contrast to the current sample, whose mean score was more than a full standard deviation higher (d � 1.07). Nevertheless, the two sets of scores demonstrated strong linear association.

Incremental Predictive Validity of the ODARA, SARA, and FVIR

The sample was followed up an average of 3.30 years (SD � 1.16) in the community, during which time, 9.3% (n � 27) were convicted for a new intimate partner violent offense, 15.6% (n � 45) were convicted for any new violent offense, and 47.4 (n � 137) were convicted for any new offense. Compared to Jung and Buro’s (2017) findings with the male perpetrator subgroup, when female perpetrators were included in the present sample, scores on the tools yielded either the same or slightly higher predictive accuracy for IPV, violent, and general recidivism as follows: SARA (see Table 5; AUCs � .74 [.63, .86], .74 [.65, .83], and .78 [.72, .84], respectively, all p � .001), ODARA (AUCs � .72 [.63, .82], .73 [.65, .81], and .75 [.69, .81], respectively, all p � .001), and FVIR (AUCs � .60 [.48, .73], p � .112, .60 [.50, .70], p � .056, and .70 [.63, .77], p � .001, respectively). As there were no female IPV recidivists and only one violent recidivist, stratified AUC analyses on the female subsample could only be completed

Table 1 Intercorrelations of Intimate Partner Violence Risk Measures and Descriptive Statistics

Measure SARA ODARA FVIR

SARA (.74) [.69, .79] ODARA .74 [.68, .79] (.71) [.65, .75] FVIR .75 [.68, .81] .64 [.55, .71] (.78) [.73, .83]

Mean 12.00 5.42 8.93 SD 5.53 2.77 3.71

Note. SARA � Spousal Assault Risk Assessment Guide; ODARA � Ontario Domestic Assault Risk Assessment; FVIR � Family Violence Investigative Report. Correlations significant at p � .001. Cronbach’s alpha in parentheses on principal diagonal. 95% CIs in square parentheses.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

444 OLVER AND JUNG

for general recidivism (19.5%), of which the SARA evinced the strongest prediction (AUC � .86 [.67, 1.00], n � 33, p � .006), followed by the ODARA (AUC � .70 [.45, .94], n � 41, p � .090), and FVIR (AUC � .66 [.39, .93], n � 30, p � .233).

Cox regression survival analyses were conducted to examine the incremental predictive validity of the measures in their prediction of each of these three recidivism outcomes, while controlling for scores on the other measures and individual differences in follow-up time.

Controlling for the ODARA. Table 2 reports the results ex- amining incremental validity of the SARA and FVIR controlling for the ODARA. In the first block, ODARA scores were significantly associated with increased IPV, any violent, and general recidivism, with each one-point increase in ODARA score being associated with a 28% to 39% increase in the hazard of one of these recidivism outcomes. SARA scores were incrementally predictive of IPV recid- ivism at p � .063 (Model 1) after controlling for the ODARA, and significantly incrementally predicted general recidivism (Model 5); however, SARA scores did not incrementally predict general violent

recidivism above and beyond the ODARA (Model 3). FVIR scores did not incrementally predict any of the three recidivism outcomes controlling for the ODARA (Models 2, 4, and 6).

Controlling for the SARA. Table 3 reports the results of incremental validity analyses controlling for SARA scores entered in the first block, followed by either the ODARA or FVIR. The results in this table have overlap with the previous table, but are repeated for parsimony of presentation and interpretation in the ordering of predictors. When entered in the first block, SARA scores on their own were significantly associated with IPV, any violent, and general recidivism over time, with each one-point increase in SARA score being associated with a 12% to 16% increase in the hazard of a given recidivism outcome. ODARA scores, however, were not incrementally predictive of IPV recid- ivism when controlling for the SARA (Model 1) but did signifi- cantly incrementally predict any violent (Model 3) and general (Model 5) recidivism. As with previous analyses, the FVIR did not incrementally predict any recidivism outcomes when controlling for the SARA (Models 2, 4, and 6).

Table 2 Cox Regression Survival Analysis: Incremental Prediction of Recidivism Outcomes Controlling for ODARA Scores

Regression model (1– 6)

95% CI

B SE Wald p eB LL UL

Intimate partner violence recidivism 1. Block 1

ODARA .27 .10 7.67 .006 1.30 1.081 1.573 Block 2 ODARA .11 .13 .61 .433 1.11 .855 1.442 SARA .12 .06 3.45 .063 1.12 .994 1.267

2. Block 1 ODARA .28 .09 9.42 .002 1.32 1.105 1.570 Block 2 ODARA .32 .11 8.28 .004 1.37 1.106 1.702 FVIR �.06 .09 .42 .518 .94 .793 1.124

Any violent recidivism 3. Block 1

ODARA .330 .08 19.52 <.001 1.39 1.201 1.609 Block 2 ODARA .24 .10 5.80 .016 1.27 1.046 1.551 SARA .06 .05 1.78 .182 1.06 .972 1.163

4. Block 1 ODARA .28 .07 16.56 <.001 1.33 1.158 1.520 Block 2 ODARA .33 .09 14.65 <.001 1.39 1.174 1.643 FVIR �.06 .07 .84 .360 .94 .821 1.074

General recidivism 5. Block 1

ODARA .25 .04 46.02 <.001 1.28 1.191 1.373 Block 2 ODARA .14 .05 7.45 .006 1.15 1.041 1.277 SARA .07 .03 8.28 .004 1.08 1.023 1.128

6. Block 1 ODARA .22 .04 39.81 <.001 1.25 1.167 1.341 Block 2 ODARA .20 .04 21.02 <.001 1.22 1.122 1.334 FVIR .03 .04 .72 .396 1.03 .961 1.106

Note. ODARA � Ontario Domestic Assault Risk Assessment; SARA � Spousal Assault Risk Assessment Guide; FVIR � Family Violence Investigative Report. Significant p-values in bold font; p-values � .07 in italics. ODARA and SARA analyses n � 220; ODARA and FVIR analyses, n � 212.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

445INCREMENTAL PREDICTION OF IPV

Controlling for the FVIR. Table 4 reports incremental valid- ity analyses controlling for the FVIR, followed by either the ODARA or SARA. When entered in the first block, FVIR scores significantly predicted or approached significance in the prediction of the three recidivism outcomes, with the exception of IPV (Model 1) in the regression model with the ODARA (which included a larger number of cases than with the SARA). Regard- less, the FVIR’s contribution to the prediction of each outcome was reduced to nonsignificance in the second block of all six regression models. SARA scores, however, significantly incre- mentally predicted IPV (Model 2) and general recidivism (Model 6), but not any violence (Model 4) after controlling for the FVIR. ODARA scores significantly incrementally predicted all three re- cidivism outcomes (Models 1, 3, and 5).

Incremental Predictive Validity of SARA Component Scores

Finer grained analyses were conducted to identify components of the SARA that may be contributing to improved predictive

accuracy relative to the ODARA by way of a series of follow-up analyses employing the four scale components of the SARA: Criminal History, Psychosocial Adjustment, Spousal Assault His- tory, and Current Offense. As seen in Table 5, all SARA compo- nents but Current Offense significantly predicted each recidivism criterion in univariate predictive accuracy analyses. Spousal As- sault History evidenced AUC magnitudes that were in the lower end of the moderate range, while Criminal History and Psychos- ocial Adjustment were clearly the strongest predictors, demonstrat- ing moderate to large AUC magnitudes for the three outcomes. The modified 17-item SARA had large AUC magnitudes in its predictive accuracy for each outcome.

A series of Cox regression survival analyses were conducted to examine the incremental validity of the SARA components with and without controlling for the ODARA. As seen in Table 6 (Model 1) Criminal History and Psychosocial Adjustment uniquely predicted IPV recidivism; however, when these two predictors were combined in a regression model with the ODARA (Model 2), SARA Psychosocial Adjustment score was the only

Table 3 Cox Regression Survival Analysis: Incremental Prediction of Recidivism Outcomes Controlling for SARA Scores

Regression model (1– 6)

95% CI

B SE Wald p eB LL UL

Intimate partner violence recidivism 1. Block 1

SARA .15 .04 11.31 .001 1.16 1.064 1.266 Block 2 SARA .12 .06 3.45 .063 1.12 .994 1.267 ODARA .11 .13 .61 .433 1.11 .855 1.442

2. Block 1 SARA .14 .05 9.10 .003 1.15 1.051 1.266 Block 2 SARA .16 .07 4.92 .027 1.17 1.018 1.341 FVIR �.03 .11 .06 .805 .97 .780 1.212

Any violent recidivism 3. Block 1

SARA .14 .03 18.50 <.001 1.15 1.079 1.225 Block 2 SARA .06 .05 1.78 .182 1.06 .972 1.163 ODARA .24 .10 5.80 .016 1.27 1.046 1.551

4. Block 1 SARA .11 .04 10.06 .002 1.12 1.044 1.198 Block 2 SARA .09 .05 2.68 .101 1.09 .983 1.215 FVIR .05 .09 .31 .575 1.05 .887 1.242

General recidivism 5. Block 1

SARA .12 .02 50.38 <.001 1.13 1.092 1.167 Block 2 SARA .07 .03 8.28 .004 1.08 1.023 1.128 ODARA .14 .05 7.45 .006 1.15 1.041 1.277

6. Block 1 SARA .12 .02 39.99 <.001 1.12 1.084 1.165 Block 2 SARA .09 .03 10.40 .001 1.10 1.036 1.158 FVIR .06 .05 1.48 .224 1.06 .967 1.155

Note. SARA � Spousal Assault Risk Assessment Guide; ODARA � Ontario Domestic Assault Risk Assess- ment; FVIR � Family Violence Investigative Report. Significant p-values in bold font; p-values � .07 in italics. ODARA and SARA analyses n � 220; SARA and FVIR analyses, n � 182.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

446 OLVER AND JUNG

incremental predictor and thus had the most unique risk variance linked to this outcome. In the prediction of any violence, only the Criminal History subscale was incremental relative to the four other predictors (Model 3); however when entered into a subse-

quent regression model it was out predicted by the ODARA (Model 4), the only incremental predictor in the model. Finally, in the prediction of general recidivism, again, Criminal History and Psychosocial Adjustment were significant incremental predictors

Table 4 Cox Regression Survival Analysis: Incremental Prediction of Recidivism Outcomes Controlling for FVIR Scores

Regression model (1– 6)

95% CI

B SE Wald p eB LL UL

Intimate partner violence recidivism 1. Block 1

FVIR .09 .06 2.02 .155 1.09 .967 1.238 Block 2 FVIR �.06 .09 .42 .518 .94 .793 1.124 ODARA .32 .11 8.28 .004 1.37 1.106 1.702

2. Block 1 FVIR .15 .08 4.05 .044 1.16 1.004 1.345 Block 2 FVIR �.03 .11 .06 .805 .97 .780 1.212 SARA .16 .07 4.92 .027 1.17 1.018 1.341

Any violent recidivism 3. Block 1

FVIR .09 .05 3.49 .062 1.09 .996 1.202 Block 2 FVIR �.06 .07 .84 .360 .94 .821 1.074 ODARA .33 .09 14.65 <.001 1.39 1.174 1.643

4. Block 1 FVIR .15 .06 7.04 .008 1.16 1.040 1.301 Block 2 FVIR .05 .09 .31 .575 1.05 .887 1.242 SARA .09 .05 2.68 .101 1.09 .983 1.215

General recidivism 5. Block 1

FVIR .12 .03 20.22 <.001 1.13 1.071 1.192 Block 2 FVIR .03 .04 .72 .396 1.03 .961 1.106 ODARA .20 .04 21.02 <.001 1.22 1.122 1.334

6. Block 1 FVIR .16 .03 27.81 <.001 1.17 1.106 1.246 Block 2 FVIR .06 .05 1.48 .224 1.06 .967 1.155 SARA .09 .03 10.40 .001 1.10 1.036 1.158

Note. FVIR � Family Violence Investigative Report; ODARA � Ontario Domestic Assault Risk Assessment; SARA � Spousal Assault Risk Assessment Guide. Significant p-values in bold font; p-values � .07 approaching significance in italics. ODARA and FVIR analyses n � 212; SARA and FVIR analyses, n � 182.

Table 5 ROC Analysis: Predictive Accuracy of SARA Component and Total Scores for Recidivism

SARA Measure

Intimate partner violence recidivism

Any violent recidivism General recidivism

AUC 95% CI AUC 95% CI AUC 95% CI

Criminal history .72�� [.63, .82] .73��� [.64, .82] .76��� [.70, .83] Psychosocial adjustment .72�� [.61, .83] .67�� [.58, .76] .75��� [.69, .82] Spousal assault history .65� [.51, .79] .63� [.53, .74] .64��� [.57, .71] Current offense .54 [.42, .67] .60 [.50, .70] .61�� [.53, .68] SARA total .74��� [.63, .86] .74��� [.65, .83] .78��� [.72, .84]

Note. ROC � receiver operating characteristic; SARA � Spousal Assault Risk Assessment Guide; AUC � area under the curve. SARA total based on 17/20 items. � p � .05. �� p � .01. ��� p � .001.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

447INCREMENTAL PREDICTION OF IPV

(Model 5), relative to the other SARA components. When exam- ined relative to the ODARA, both of these SARA components incrementally predicted this outcome, while the ODARA no longer did (Model 6).

Calibration Analyses of SARA and ODARA Scores

The final set of analyses examined rates of recidivism at differ- ent score thresholds of the SARA, ODARA, or the use of the instruments in tandem as per incremental validity analyses. The analyses centered on these two instruments given that they each were incremental beyond the FVIR, or demonstrated more com- parable predictive accuracy to each other with respect to each recidivism outcome. These analyses are an extension of the Cox regression survival analyses and demonstrate through use of cat- egorical risk bands if the two instruments used in tandem are potentially more informative than either instrument used individ- ually with respect to the prediction of recidivism.

Kaplan-Meier survival analyses. Figures 1 and 2 present the results of Kaplan-Meier survival analysis at different score thresh- olds of the SARA, ODARA, and their combination with respect to the prediction of IPV recidivism and any recidivism, respectively, over time. As the SARA is an SPJ tool but it was not scored in an SPJ manner in the present study, low, medium, and high groups were created by identifying scores approximately one standard deviation (5.53) below the mean of 12.0 (low risk, 0 – 6, M � 4.57, n � 45), scores at one standard deviation above the mean (high

risk, 18 –28, M � 20.82, n � 34), and the remaining 152 cases scoring between 7 and 17 (medium risk, M � 12.04). As seen in Figure 1A, the high risk SARA group had significantly higher rates of IPV recidivism than the medium (log rank �2[n � 186] � 11.02, p � .001) or low (log rank �2[n � 79] � 9.13, p � .003) risk groups; the latter two groups were not significantly different. The ODARA risk categories were formed through following the distribution of ODARA scores from Hilton et al. (2004) in 7–13 (the top band) corresponded to high risk (M � 8.41, n � 98), 3 to 6 (M � 4.52, n � 124) corresponded to medium risk, and 0 to 2 (M � 1.47, n � 49) corresponded to low risk. As seen in Figure 1B, the high risk ODARA group had significantly higher rates of IPV recidivism than the medium (log rank �2[n � 222] � 6.63, p � .010) or low (log rank �2[n � 147] � 5.37, p � .020) risk groups; the latter two groups were not significantly different.

The SARA and ODARA were then combined to form five risk bands: high risk SARA and ODARA (n � 25), high risk on at least one of SARA and ODARA and medium risk on the other (n � 38), medium risk SARA and ODARA (n � 93), medium risk on at least one of SARA and ODARA and low risk on the other (n � 37), and low risk SARA and ODARA (n � 27). The two instruments were combined through integrating their risk levels in this manner, given that the ODARA is an actuarial instrument and the SARA is an SPJ measure, and doing so would serve as one means of systematically integrating the risk estimates from tools developed from different traditions (e.g., as opposed to summing the scores

Table 6 Cox Regression Survival Analysis: Incremental Validity of SARA Scale Component Scores in the Prediction of Recidivism Outcomes Controlling for the ODARA

Regression model (1– 6): Simultaneous entry of predictors

95% CI

B SE Wald p eB LL UL

Intimate partner violence recidivism 1. SARA Criminal history .30 .15 4.24 .040 1.35 1.014 1.789

SARA Psychosocial adjustment .37 .16 5.05 .025 1.44 1.048 1.986 SARA Spousal assault history .11 .13 .72 .395 1.11 .869 1.428 SARA Current offense �.28 .21 1.73 .189 .76 .501 1.146

2. ODARA �.01 .16 .01 .945 .99 .724 1.350 SARA Criminal history .33 .22 2.24 .134 1.39 .903 2.148 SARA Psychosocial adjustment .37 .16 5.57 .018 1.45 1.065 1.976

Any violent recidivism 3. SARA Criminal history .33 .10 9.87 .002 1.39 1.130 1.697

SARA Psychosocial adjustment .17 .11 2.12 .145 1.18 .944 1.476 SARA Spousal assault history .06 .09 .47 .493 1.07 .889 1.277 SARA Current offense .02 .15 .02 .899 1.02 .760 1.368

4. ODARA .27 .11 5.43 .020 1.31 1.043 1.633 SARA Criminal history .12 .16 .53 .468 1.12 .821 1.535

General recidivism 5. SARA Criminal history .26 .05 23.06 <.001 1.30 1.167 1.444

SARA Psychosocial adjustment .26 .06 18.38 <.001 1.30 1.154 1.467 SARA Spousal assault history �.03 .05 .46 .496 .97 .880 1.064 SARA Current offense .03 .09 .16 .693 1.04 .872 1.228

6. ODARA .06 .06 1.04 .308 1.07 .943 1.203 SARA Criminal history .19 .08 5.36 .021 1.21 1.029 1.415 SARA Psychosocial adjustment .23 .06 15.29 <.001 1.26 1.123 1.417

Note. SARA � Spousal Assault Risk Assessment Guide; ODARA � Ontario Domestic Assault Risk Assess- ment. Significant p-values in bold font. SARA component analyses (Models 1,3,5) n � 231; SARA and ODARA (Models 2, 4, 6) n � 220.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

448 OLVER AND JUNG

on both tools and generating risk cutoffs in a strictly actuarial manner). Results are presented in Figure 1C. The high risk SARA and ODARA group had higher rates of IPV recidivism than the medium (log rank �2[n � 118] � 5.43, p � .020), medium/low (log rank �2[n � 62] � 7.70, p � .006), and low (log rank �2[n � 52] � 3.59, p � .058) risk groups. The difference in rates of IPV recidivism between the medium/high risk (10.5%) and high risk (20%) groups was not significant; the large drop at the end of the survival curve for the medium/high group can be attributed to a late recidivist with proportionately few cases remaining. The me- dium/high SARA-ODARA group was different from the medium/

low group at (log rank �2[n � 75] � 3.63, p � .057). No other differences attained significance.

These analyses were repeated examining general recidivism as an outcome for the two measures and their combination (see Figure 2). As seen in Figure 2A, the high risk SARA group had significantly higher rates of general recidivism than the medium (log rank �2[n � 186] � 15.41, p � .001) or low (log rank �2[n � 79] � 49.24, p � .001) risk groups; the latter two groups were also significantly different (log rank �2[n � 197] � 14.58, p � .001). As seen in Figure 2B, the high risk ODARA group had signifi- cantly higher rates of IPV recidivism than the medium (log rank

Figure 1. Kaplan-Meier Survival Analysis: Rates of intimate partner violence (IPV) recidivism over time as a function of risk level: 1A) Spousal Assault Risk Assessment Guide (SARA): high risk group has significantly higher rates of recidivism than low and medium groups; 1B) Ontario Domestic Assault Risk Assessment (ODARA): high risk group has significantly higher rates of recidivism than low and medium groups; and 1C) combined SARA and ODARA risk levels: High SARA and ODARA group has significantly higher rates of recidivism than the Medium and one Low-one Medium groups, and p � .058 with respect to the Low group. One Medium-one High group is different from the one Low-one Medium group at p � .057.

Figure 2. Kaplan-Meier Survival Analysis: Rates of any criminal recidivism over time as a function of risk level: 2A) Spousal Assault Risk Assessment Guide (SARA): all three groups are significantly different; 2B) Ontario Domestic Assault Risk Assessment (ODARA): all three groups are significantly different; and 2C) combined SARA and ODARA risk levels: High SARA and ODARA group and one Medium-one High groups are each significantly different from the three remaining lower risk groups, Medium SARA and ODARA are significantly different from both lower risk groups. No other group differences are significant.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

449INCREMENTAL PREDICTION OF IPV

�2[n � 222] � 8.50, p � .004) or low (log rank �2[n � 147] � 32.76, p � .001) risk groups; the latter two groups were also significantly different (log rank �2[n � 173] � 20.28, p � .001). The combined SARA and ODARA risk groups demonstrated greater separation among the risk bands, and hence support for the incremental contributions of these two measures, in the prediction of general recidivism.

As seen in Figure 2C, the high risk SARA and ODARA group had significantly higher rates of general recidivism than the me- dium (log rank �2[n � 118] � 11.15, p � .001), medium/low (log

rank �2[n � 62] � 24.74, p � .001), and low (log rank �2[n � 52] � 30.61, p � .001) risk groups, although the difference from the medium/high group was not significant (log rank �2[n � 63] � 0.77, p � .380). The medium/high SARA-ODARA group simi- larly had significantly higher rates of general recidivism than the medium (log rank �2[n � 131] � 6.58, p � .010), medium/low (log rank �2[n � 75] � 17.26, p � .001), and low (log rank �2[n � 65] � 21.82, p � .001) risk groups. Finally, the medium risk SARA and ODARA group had significantly lower rates of general recidivism than the medium/low (log rank �2[n � 130] � 6.11,

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

3- msivi

dicer ec nel

oiv citse m

o d raey

SARA score

A

3 year IPV actual 3 year IPV logis�c regression

0

0.05

0.1

0.15

0.2

0.25

0.3

0 1 2 3 4 5 6 7 8 9 10 11 12 13

3- msivi

dicer ec nel

oiv cits e m

o d ra ey

ODARA score

B

3 year IPV actual 3 year IPV logis�c regression

Low

Medium

High

Low Medium

High

Figure 3. Spousal Assault Risk Assessment Guide (SARA; 3A) and Ontario Domestic Assault Risk Assess- ment (ODARA; 3B) Calibration Analyses. Actual (% reconviction) versus estimated (logistic regression derived) rates of 3-year intimate partner violence (IPV) recidivism as a function of risk level (actual rates of recidivism at low, medium, high categories) or individual score (logistic regression derived estimates at specific obtained scores). Low, medium, and high risk groups are represented by the mean score of each group, created by 1 SD � mean, the mean value, and 1 SD � mean, respectively.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

450 OLVER AND JUNG

p � .013) and low (log rank �2[n � 120] � 9.35, p � .002) risk groups. The difference between the medium/low and low risk groups was not significant (log rank �2[n � 64] � 0.98, p � .322).

Fixed follow-ups. Finally, chi square analysis and logistic regression modeling were employed to examine rates of actual and predicted IPV recidivism over fixed three year follow-ups (see Figure 3). The Hosmer-Lemeshow test was not significant for either the SARA or ODARA in predicting the recidivism out- comes, suggesting that the rates of IPV and general reoffending approximated a logistic distribution. SARA scores significantly predicted 3-year domestic violence recidivism B1 � .153, p � .012 (constant, B0 � �4.620) with the results in Figure 3A showing a progressive increase in projected rates of domestic violence recid- ivism with each successive increase in SARA score, up to a maximum observed score of 28 for the sample. When actual rates of IPV recidivism were obtained at low, medium, and high scores (corresponding to rounded scores of 5, 12, and 21, respectively), these very closely paralleled the estimated rates at the respective scores; chi square analysis demonstrated a significant association between the risk bands and actual IPV recidivism (�2[n � 151] � 9.98, p � .007, � � .26). ODARA scores predicted 3-year IPV recidivism at B1 � .205, p � .061 (constant, B0 � �3.783) with the results in Figure 3B showing a progressive increase in pro- jected rates of IPV recidivism with each successive increase in ODARA score, up to a maximum observed score of 13 for the sample. When actual rates of IPV recidivism were obtained at low, medium, and high scores (corresponding to rounded scores of 2, 5, and 8, respectively), these also very closely paralleled the esti- mated rates at the respective scores. The association between the risk bands and actual IPV recidivism was not significant (�2[n � 175] � 3.23, p � .199, � � .14), possibly owing to the large number of cases within the high risk ODARA band.

Discussion

The present study examined the incremental validity of three IPV risk measures—the SARA, ODARA, and FVIR—in a sample of police-charged IPV men and women. For the most part, each of the instruments significantly predicted their targeted criterion as in Jung and Buro (2017), within the first block of each analysis, which examined a given instrument as a whole without partition- ing off variance shared with one of the other measures. The correla- tional analyses, however, demonstrated clearly that there is an abun- dance of shared variance among the three measures, as one would reasonably expect given that each is designed to appraise risk for domestic violence against an intimate partner. The incremental valid- ity analyses were illuminating when examining the unique contribu- tions of each instrument in the prediction of recidivism, when vari- ance shared with the other instruments was removed in Cox regression survival analysis, which also controls for individual differ- ences in follow-up time.

First, the SARA seemed to outperform the ODARA and the FVIR in the prediction of IPV recidivism; it was the only measure to come close to the incremental prediction of recidivism when controlling for the ODARA, but not vice versa. By contrast, if there was a winner in the prediction of general violence, it would be the ODARA in this sample, as only it uniquely incrementally predicted this outcome irrespective of the instrument controlled; however, neither the SARA nor FVIR uniquely predicted general

violence when controlling for the ODARA. This may not be unexpected given that the ODARA was developed from an item pool and statistical method of scale development similar to the VRAG, SORAG, and VRAG-R, all of which predict general violence better than more specific forms of violence (e.g., sexual recidivism; Harris, Rice, Quinsey, & Cormier, 2015). Finally, both the SARA and ODARA demonstrated incremental predictive va- lidity in the prediction of general recidivism over time; as such, there was unique variance within each instrument that was infor- mative in the prediction of this broad outcome.

Follow-up analyses demonstrated that the SARA Criminal History and Psychosocial Adjustment subscales contained most of the risk variance in the prediction of the three outcomes. The results of Cox regression analyses utilizing the scale components demonstrated that it was the Psychosocial Adjustment subscale that drove the SARA’s incremental prediction of IPV recidivism, while this and the Criminal History subscale performed well relative to the ODARA in the pre- diction of any recidivism. (The SARA components seemed to be outperformed by the ODARA regardless in the prediction of any violent recidivism.) The Psychosocial Adjustment scale contains sev- eral items that tap lifestyle, mental health, and personality variables such as relationship problems, employment problems, substance abuse, personality pathology (which a symptom constellation resem- bling cluster B), active symptoms of severe mental illness, and sui- cidal/homicidal ideation—a collection of acute and stable dynamic risk variables, arguably with overlap of Andrews and Bonta’s (2010) Central Eight covariates of criminal conduct. The Criminal History subsection of the tool is arguably a static counterpart, covering a history of rule violations and assaultive behavior, which in many respects seems to be complementary to the more dynamic compo- nents of the scale.

The FVIR struggled the most of the three instruments and did not have incremental value in the prediction of any of the three recidivism outcomes after controlling for two well-established IPV risk assess- ment measures. The performance may reflect important differences in approaches to instrument development and selection of item content. The FVIR, for instance, is a regionally developed and used measure, that was constructed from static predictors through procedures that were less psychometrically formalized. The ODARA, by contrast employs empirically validated risk markers that predicted an outcome of interest, while the SARA was developed by a network of forensic clinical psychologists who identified a collection of static and osten- sibly dynamic risk markers that demonstrated potential links to spou- sal assault through theory and research.

The calibration analyses of SARA and ODARA scores demon- strated that successively higher rates of recidivism were observed with successive increments in risk level for each tool. What is interesting is that for the groups with one instrument in the medium range, when it was paired with an instrument scored in the high range, the rates of recidivism were not discernably different than when both instruments were rated high. On the other hand, if one instrument is scored low while the other is scored medium, the corresponding observed rates of recidivism were not discernably different from individuals scoring low risk on both instruments. The implications are that use of a single instrument scoring in the medium range could result in an over estimate or under estimate of risk if a second instrument generated a different estimate. The results would seem to provide at least partial support for the incremental predictive validity of the tools at the categorical risk level for these two recidivism outcomes. Extending

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

451INCREMENTAL PREDICTION OF IPV

the calibration analyses through employing a 3-year fixed follow-up for IPV recidivism demonstrated close correspondence between ac- tual and estimated rates of recidivism for both tools. The results would seem to speak to the stability of risk estimates in the present sample. Although the SARA is an SPJ tool by design, the results indicate that it also performs well in the prediction of IPV when employed in an actuarial manner.

These findings would suggest that there is value added in using more than one tool in IPV risk assessment, such as the SARA and ODARA. We suggest there could be merit in this instrument combi- nation in a number of respects. First, from an purely empirical standpoint, on their own they each predict their targeted recidivism outcomes, and when taken together, they seem to have different strengths in prediction tasks, at least in the present sample. Second, clinical wisdom and guidelines from SPJ instrument manuals suggest that risk assessment is strengthened when multiple tools are used, to assess multiple domains of functioning, and including static and dynamic variables/measures (e.g., Boer, Hart, Kropp, & Webster, 1997; Kropp et al., 1995). The tools in tandem yield a more compre- hensive volume of information to inform risk assessment and man- agement. From an RNR standpoint, the ODARA is particularly adept at providing an appraisal of baseline risk, and the SARA, in addition to providing risk relevant information, may have the potential to identify where to intervene to prioritize treatment and risk manage- ment resources as per the need principle. Both tools would seem to have the potential to inform service intensity as per the risk principle. By analogy, in the sex offender risk assessment field, common static- dynamic instrument combinations are used such as the Static-99R and the STABLE 2007 (see Hanson et al., 2015), and in some instances, particularly with very large samples, multiple instruments within the same category can have incremental predictive validity (Lehmann et al., 2013).

The integration of multiple tools is a challenge that forensic eval- uators routinely face. Mills (2017) reminds us, however, that a risk instrument is not the same thing as a risk assessment. Elsewhere it has been argued that the clinical practice of risk assessment is essentially an SPJ process given that multiple tools developed from different traditions (e.g., actuarial, SPJ) are typically used (e.g., SARA, PCL-R, LSI-R), and not uncommonly, these generate disparate risk estimates which evaluators need to reconcile to generate global appraisals of recidivism risk (Olver, 2016). We illustrated one potential means of doing this through combining SARA and ODARA risk bands, both of which could be extrapolated to other IPV assessment contexts; in clinical use, however, SARA categories would be SPJ driven, as opposed to being actuarially driven, as in the current study. We did not examine combinations with the FVIR, given that there were no indications from the incremental validity analyses that it augmented prediction; however, such findings would underscore the need of supplementing the FVIR with additional measures in the context of a clinical IPV assessment.

There are some important strengths and limitations of the present study, with implications for research and practice. First, to our knowl- edge, this is the first study to examine the incremental predictive validity of the three measures, and particularly, the SARA and ODARA. The results suggest that pairing the latter two measures in IPV assessment may be fruitful and that tools may complement one another; while the tools independently or in combination can estimate risk for IPV, any violence, and general recidivism, the SARA has domains that may be targeted for intervention and risk management.

Moreover, as Nicholls et al. (2013) note, most measures are examined by networks of people linked to their development, and thus the present body of work provides an independent examination of impor- tant psychometric properties of the ODARA and the SARA. The inclusion of female perpetrators also builds on existing IPV risk assessment research with female perpetrators (Glass et al., 2008; Hilton, Popham, Lang, & Harris, 2014), although the relatively small female subsample precluded more extensive gender stratified analy- ses. Future research may profit from examinations of larger numbers of female perpetrators to replicate and extend these findings.

An additional limitation was that the risk ratings were completed at only a single time point, and the sample was not consistently treated or otherwise systematically exposed to change agents. The dynamic predictive validity of the SARA, and more specifically, its capacity to measure changes in IPV risk, is currently an unknown property; further research on a treated or supervised sample of IPV men and women may be fruitful in evaluating its capacity to assess dynamic IPV risk. Moreover, the SARA was also not examined in an SPJ manner; that is, items were summed to generate numeric totals that were subjected to analysis. Although this is not an uncommon prac- tice in psychometric research of SPJ tools, this may also be construed as a further study limitation and examination of the incremental validity of SARA summary judgments relative to ODARA risk scores may be informative in order to extend these findings to the SPJ properties of the tool. Moreover, the vagaries of file information, which can vary in its depth and completeness, precluded scoring all SARA items, notably some that seemed to be potentially dynamic in nature (e.g., attitudes that support or condone spousal assault). In our view this is also a minor shortcoming, as the vast majority (85%) of the items could be rated in order to generate a reasonable proxy of the full tool. We similarly anticipate the findings may extend to the newer SARA Version 3, although further research on the psychometric properties of the latest iteration, including its incremental validity, will obviously better establish this.

A final potential limitation is that the study examined only binary recidivism outcomes, as more specific information concern- ing offense severity in instances of IPV recidivism was not avail- able. Shrout and Yip-Bannicq (2017) have shown that basing inferences on binary outcomes reduces statistical power and is a potential source of false positive errors (see also Gendreau, 2002, on the interpretation of effect size magnitudes in the context of null hypothesis significance testing). Future research may benefit from examination of offense severity indicators to ascertain if multiple measures may be useful in appraising risk for such outcomes. In conclusion, risk assessment technology for IPV has made impor- tant inroads. The use of multiple tools may improve the accuracy of risk assessment, quality of decision making, and effectiveness of prevention efforts to manage IPV risk to promote public safety.

References

Alberta Justice and Solicitor General. (2013). Domestic violence police guidelines. Edmonton, Alberta: Author.

Andrews, D. A., & Bonta, J. (2010). The psychology of criminal conduct (5th ed.). New Providence, NJ: LexisNexus.

Belfrage, H., Strand, S., Storey, J. E., Gibas, A. L., Kropp, P. R., & Hart, S. D. (2012). Assessment and management of risk for intimate partner violence by police officers using the Spousal Assault Risk Assessment Guide. Law and Human Behavior, 36, 60 – 67. http://dx.doi.org/10.1037/ h0093948

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

452 OLVER AND JUNG

Boer, D. P., Hart, S. D., Kropp, P. R., & Webster, C. D. (1997). Manual for the Sexual Violence Risk-20: Professional guidelines for assessing risk of sexual violence. Vancouver, British Columbia: Institute against Fam- ily Violence and the Mental Health, Law, and Policy Institute, Simon Fraser University.

Campbell, J. C. (1986). Nursing assessment for risk of homicide with battered women. Advances in Nursing Science, 8, 36 –51. http://dx.doi .org/10.1097/00012272-198607000-00006

Campbell, M. A., French, S., & Gendreau, P. (2009). The prediction of violence in adult offenders: A meta-analytic comparison of instruments and methods of assessment. Criminal Justice and Behavior, 36, 567– 590. http://dx.doi.org/10.1177/0093854809333610

Dauvergne, M. (2009). Trends in police-reported serious assaults. Juristat Report No. 85– 002-X: Statistics Canada. Retrieved from http://www .statcan.gc.ca/pub/85-002-x/2009004/article/10930-eng.htm

Eke, A. W., Hilton, N. Z., Harris, G. T., Rice, M. E., & Houghton, R. E. (2011). Intimate partner homicide: Risk assessment and prospects for prediction. Journal of Family Violence, 26, 211–216. http://dx.doi.org/ 10.1007/s10896-010-9356-y

Gendreau, P. (2002). We must do a better job of cumulating knowledge. Canadian Psychology, 43, 205–210. http://dx.doi.org/10.1037/h0086917

Glass, N., Perrin, N., Hanson, G., Bloom, T., Gardner, E., & Campbell, J. C. (2008). Risk for reassault in abusive female same-sex relationships. American Journal of Public Health, 98, 1021–1027. http://dx.doi.org/10 .2105/AJPH.2007.117770

Hanson, R. K., Helmus, L. M., & Harris, A. J. R. (2015). Assessing the risk and needs of supervised sexual offenders: A prospective study using STABLE-2007, Static-99R, and Static-2002R. Criminal Justice and Behavior, 42, 1205–1224. http://dx.doi.org/10.1177/0093854815602094

Hare, R. D. (1991). Hare Psychopathy Checklist–Revised (PCL-R). Hare, R. D. (2003). Hare Psychopathy Checklist–Revised (PCL-R; 2nd

ed.). Harris, G. T., Rice, M. E., & Quinsey, V. L. (1993). Violent recidivism of

mentally disordered offenders: The development of a statistical predic- tion instrument. Criminal Justice and Behavior, 20, 315–335.

Harris, G. T., Rice, M. E., Quinsey, V. L., & Cormier, C. A. (2015). Violent offenders: Appraising and managing risk (3rd ed.). Washington, DC: American Psychological Association.

Helmus, L., & Bourgon, G. (2011). Taking stock of 15 years of research on the Spousal Assault Risk Assessment Guide (SARA): A critical review. The International Journal of Forensic Mental Health, 10, 64 –75. http:// dx.doi.org/10.1080/14999013.2010.551709

Hendricks, B., Werner, T., Shipway, L., & Turinetti, G. J. (2006). Recid- ivism among spousal abusers: Predictions and program evaluation. Jour- nal of Interpersonal Violence, 21, 703–716. http://dx.doi.org/10.1177/ 0886260506287310

Hilton, N. Z., & Harris, G. T. (2008). How nonrecidivism affects predictive accuracy: Evidence from a cross-validation of the Ontario Domestic Assault Risk Assessment (ODARA). Journal of Interpersonal Violence, 24, 326 –337. http://dx.doi.org/10.1177/0886260508316478

Hilton, N. Z., Harris, G. T., & Rice, M. E. (2010). Risk assessment for domestically violent men: Tools for criminal justice, offender interven- tion, and victim services. Washington, DC: American Psychological Association. http://dx.doi.org/10.1037/12066-000

Hilton, N. Z., Harris, G. T., Rice, M. E., Houghton, R. E., & Eke, A. W. (2008). An indepth actuarial assessment for wife assault recidivism: The Domestic Violence Risk Appraisal Guide. Law and Human Behavior, 32, 150 –163. http://dx.doi.org/10.1007/s10979-007-9088-6

Hilton, N. Z., Harris, G. T., Rice, M. E., Lang, C., Cormier, C. A., & Lines, K. J. (2004). A brief actuarial assessment for the prediction of wife assault recidivism: The Ontario domestic assault risk assessment. Psy- chological Assessment, 16, 267–275. http://dx.doi.org/10.1037/1040- 3590.16.3.267

Hilton, N. Z., Popham, S., Lang, C., & Harris, G. T. (2014). Preliminary validation of the ODARA for female intimate partner violence offenders. Partner Abuse, 5, 189 –203. http://dx.doi.org/10.1891/1946-6560.5.2.189

Jung, S., & Buro, K. (2017). Appraising risk for intimate partner violence in a police context. Criminal Justice and Behavior, 44, 240 –260. http:// dx.doi.org/10.1177/0093854816667974

Kropp, P. R., & Hart, S. D. (2000). The Spousal Assault Risk Assessment (SARA) Guide: Reliability and validity in adult male offenders. Law and Human Behavior, 24, 101–118. http://dx.doi.org/10.1023/A: 1005430904495

Kropp, P. R., Hart, S. D., & Belfrage, H. (2005). Brief spousal assault form for the evaluation of risk (B -SAFER): User manual. Vancouver, British Columbia: ProActive ReSolutions Inc.

Kropp, P. R., Hart, S. D., Webster, C. W., & Eaves, D. (1995). Manual for the Spousal Assault Risk Assessment Guide (2nd ed.). Vancouver, British Columbia: British Columbia Institute on Family Violence.

Lehmann, R. J. B., Hanson, R. K., Babchishin, K. M., Gallasch-Nemitz, F., Biedermann, J., & Dahle, K. P. (2013). Interpreting multiple risk scales for sex offenders: Evidence for averaging. Psychological Assessment, 25, 1019 –1024. http://dx.doi.org/10.1037/a0033098

Messing, J. T., & Thaller, J. (2013). The average predictive validity of intimate partner violence risk assessment instruments. Journal of Inter- personal Violence, 28, 1537–1558. http://dx.doi.org/10.1177/ 0886260512468250

Mills, J. F. (2017). Violence risk assessment: A brief review, current issues, and future directions. Canadian Psychology, 58, 40 – 49. http:// dx.doi.org/10.1037/cap0000100

Nicholls, T. L., Pritchard, M. M., Reeves, K. A., & Hilterman, E. (2013). Risk assessment in intimate partner violence: A systematic review of contemporary approaches. Partner Abuse, 4, 76 –168. http://dx.doi.org/ 10.1891/1946-6560.4.1.76

Olver, M. E. (2016). Some considerations on the use of actuarial and related forensic measures with diverse correctional populations. Journal of Threat Assessment and Management, 3, 107–121. http://dx.doi.org/ 10.1037/tam0000065

Rice, M. E., & Harris, G. T. (2005). Comparing effect sizes in follow-up studies: ROC Area, Cohen’s d, and r. Law and Human Behavior, 29, 615– 620. http://dx.doi.org/10.1007/s10979-005-6832-7

Seto, M. C. (2005). Is more better? Combining actuarial risk scales to predict recidivism among adult sex offenders. Psychological Assess- ment, 17, 156 –167. http://dx.doi.org/10.1037/1040-3590.17.2.156

Shrout, P. E., & Yip-Bannicq, M. (2017). Inferences about competing measures based on patterns of binary significance tests are questionable. Psychological Methods, 22, 84 –93.

Storey, J. E., & Hart, S. D. (2014). An examination of the Danger Assessment as a victim-based risk assessment instrument for lethal intimate partner violence. Journal of Threat Assessment and Manage- ment, 1, 56 – 66. http://dx.doi.org/10.1037/tam0000002

Storey, J. E., Kropp, P. R., Hart, S. D., Belfrage, H., & Strand, S. (2014). Assessment and management of risk for intimate partner violence by police officers using the Brief Spousal Assault Form for the Evaluation of Risk. Criminal Justice and Behavior, 41, 256 –271. http://dx.doi.org/ 10.1177/0093854813503960

Williams, K. R. (2012). Family violence risk assessment: A predictive cross-validation study of the Domestic Violence Screening Instrument- Revised (DVSI-R). Law and Human Behavior, 36, 120 –129. http://dx .doi.org/10.1037/h0093977

Williams, K. R., & Houghton, A. B. (2004). Assessing the risk of domestic violence reoffending: A validation study. Law and Human Behavior, 28, 437– 455. http://dx.doi.org/10.1023/B:LAHU.0000039334.59297.f0

Received November 10, 2016 Revision received April 29, 2017

Accepted May 2, 2017 �

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

453INCREMENTAL PREDICTION OF IPV