Inter-Rater Reliability NO PLAGERISM

Journal of Threat Assessment and Management The CTAP, a Threat Assessment Tool for the Initial Evaluation of Concerning or Threatening Communications: Development and Inter-Rater Reliability David V. James, Philip Allen, Andrew Wolfe Murray, Rachel D. MacKenzie, Junyi Yang, Alice De Silva, and Frank R. Farnham Online First Publication, January 31, 2022. http://dx.doi.org/10.1037/tam0000173

CITATION James, D. V., Allen, P., Wolfe Murray, A., MacKenzie, R. D., Yang, J., De Silva, A., & Farnham, F. R. (2022, January 31). The CTAP, a Threat Assessment Tool for the Initial Evaluation of Concerning or Threatening Communications: Development and Inter-Rater Reliability. Journal of Threat Assessment and Management. Advance online publication. http://dx.doi.org/10.1037/tam0000173

The CTAP, a Threat Assessment Tool for the Initial Evaluation of Concerning or Threatening Communications:

Development and Inter-Rater Reliability

David V. James, Philip Allen, Andrew Wolfe Murray, Rachel D. MacKenzie, Junyi Yang, Alice De Silva, and Frank R. Farnham Theseus Risk Management Ltd., Chippenham, United Kingdom

Best practice in threat assessment requires the use of standardised, evidence-based tools to assist in case formulation. Considerable numbers of such tools have been developed for use in risk assessment in criminal justice, mental health and community settings: instruments have been published for the threat assessment of cases in specific areas, such as workplace violence, school violence, stalking, and terrorism. However, there has been less attention to a more general and common problem faced regularly by politicians, public companies, those in the talent industry, and others in the public eye. That is how the recipients of problematic or threatening communications can judge which of these need referral to security departments or police for further investigation; and how agencies receiving such referrals can reach initial conclusions as to which cases require prioritisa- tion and which do not. Relevant risks in this area concern not simply violence, the focus of most threat assessment instruments, but also persistence, escalation, disruption of function, reputational damage, financial loss, and psychological harm. The Communica- tions Threat Assessment Protocol (CTAP) is a structured professional judgement tool for the initial threat assessment of unwanted communications, designed to fill this gap. This article details the background to the CTAP and its development into a manualised threat assessment instrument for use both in the private sphere and in policing contexts. It reports the first evaluation of the reliability of the CTAP. The results of this study are promising in terms of the use of CTAP in the initial assessment of problematic communications.

Public Significance Statement When people in public life receive bizarre or threatening communications, some form of system is needed to help them divide these into ones to bin and ones that need further examination by security departments or police. Further, such departments need a means to make an initial threat assessment in order to triage such referrals—and one that can be done quickly in response to limited information. The Communications Threat Assessment Protocol provides such a system, the background to which and reliability of which is illustrated in this paper.

Keywords: threat assessment, threatening communications, screening tools, structured professional judgement, policing

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

David V. James https://orcid.org/0000-0002-5046- 8045 Philip Allen https://orcid.org/0000-0003-3912-8071 Junyi Yang https://orcid.org/0000-0003-2322-3525 The group that designed and conducted this study

included the authors of the CTAP (David V. James, Rachel

D. MacKenzie, Frank R. Farnham). Two of the group (Philip Allen and Andrew Wolfe Murray) are directors of Theseus, which produces the CTAP manual. Correspondence concerning this article should be

addressed to David V. James, Theseus Risk Management Ltd., 6 Prince Maurice House, Chippenham SN14 6LH, United Kingdom. Email: [email protected]

1

Journal of Threat Assessment and Management © 2022 American Psychological Association ISSN: 2169-4842 https://doi.org/10.1037/tam0000173

In the field of threat assessment, the use of standardised tools by multidisciplinary teams de- fines best practice (Meloy et al.,2021,p. 16).Such tools provide a systematic, structured means of assessing risk, and comprise an aide-mémoire which assists the assessors in making sure that all relevant factors are considered in their evalua- tion (Skeem & Monahan, 2011), whilst allowing for the consideration of idiosyncratic factors in a given case. A considerable number of risk assess- ment instruments is available for examining risks in different spheres. Indeed, Singh et al. (2014) identified at least 400 structured risk assessment measures for violence alone that are used by professionals working in health, criminal jus- tice, and the general community. Specialist vo- lumes detail the role of risk assessment tools in a variety of settings, including the workplace, the public figure context, schools, stalking, and terror- ism, with a particular emphasis on risk of violence (e.g., Douglas & Otto, 2020; Meloy & Hoffmann, 2021; Wormith et al., 2020). In the design of these tools, a differentiation

can be made between risk assessment and threat assessment. Risk assessment generally involves consideration of a case in a review setting, where there is little time-pressure and a considerable amount of information is available about the case. Threat assessment, by contrast, concerns the making of decisions quickly, based on limited information, in order to determine the level of immediate response. As such, threat assessment tools incorporate a screening role. Examples of risk assessment tools in the area of harassment, intrusive behaviours and the making of threats would include the Guidelines for Stalking Assess- ment and Management (SAM; Kropp et al., 2008) and the Stalking Risk Profile (SRP; MacKenzie et al., 2009), whereas an example of a threat assess- ment tool would be the Screening Assessment for Stalking and Harassment (SASH; James & Sheridan, 2020; McEwan et al., 2017). The risk factors incorporated into these tools are

generally data items which have been shown to be statistically associated with the outcome in question in previous research studies. Rather than individual prediction, which is not possible, such instruments concern the allocation, based on group data, of an individual into a general risk band according to the number and/or nature of the risk factors which are showntobepresentintheircase.Inriskassessment, there tend to be three such groups, high, medium and low (Meloy et al., 2012). Some risk assessment

instruments incorporate “red flag” indicators, the presence of one of which is sufficient to determine that an individual should be considered high risk. With threat assessment instruments, by contrast, what is being assessed is not a comprehensive judgement as to risk, but a preliminary determina- tion as to the level of concern (and therefore priority for further assessment or intervention) which should be accorded to the individual under assessment (Scalora, Baumgartner, Zimmerman, et al., 2002). Risk assessment instruments, and some threat assessment tools, generally enable conclusions to be drawn, not only as to the level of problem, but as to how it can be managed or reduced. Whatever the type of tool, it is necessary to

ensure that its results are both reliable and valid— in other words, that it produces consistent and reproducible results between investigators and over time, and that it measures what it is supposed to measure. This involves exercises in the mea- surement of inter-rater and test–retest reliability, and of predictive validity. Also relevant is the concept of construct validity in terms of whether the content of the instrument items completely represents the construct, and whether scores cor- relate with those from other instrumentsassessing the same construct (Cook & Beckman, 2006). Eventually, it is relevant to investigate the utility of tools, in other words how useful and acceptable they are found in practice (Hart & Logan, 2011). New tools are usually based upon a systematic review of the relevant scientific and professional literatures, and refined through various forms of practical evaluation, before being subject to for- mal reliability and validity testing. The present study concerns the development and reliability of the Communications Threat Assessment Protocol (CTAP; James, MacKenzie, et al., 2014), a threat assessment tool for the initial evaluation of con- cerning or threatening communications.

Threatening and Concerning Communications

Those in the public eye, or whose jobs are public-facing, receive letters and electronic com- munications from members of the general public because of who they are, what they do, or who they work for. This applies amongst others to politicians, public officials, the CEOs and em- ployees of major companies, sports and media

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

2 JAMES ET AL.

personalities, and creative artists. The majority of such communications concern legitimate enquiry, reasonable complaint, requests for help, or fan mail. However, a significant minority will be strange, bizarre, angry or inappropriate in content. In a proportion, the writer may seem mentally unwell. Some cases will constitute persistent or querulent complaining, some amount to cam- paigns of harassment, and a few will be inappro- priately amorous. Sometimes, communications will be received which contain direct or indirect threats. These may concern reputational or finan- cial damage, persistent unwanted intrusion, or more rarely, physical violence. Even where no such threats are included, the communications may be psychologically upsetting or distressing tothose receiving them, they may cause disruption of function, and valuable time and resources may be consumed in dealing with them. The majority of such inappropriate, aggressive

or threatening communications are harmless, but some are not. The problem is how to separate the two. The dangers, should this not be done effectively, are either that warnings of future harm are missed, with potential adverse con- sequences, or alternatively that costly investi- gative and security resources are spent on cases where they are not required. In the operational situation, there are two levels at which practical guidance is needed as to which cases warrant further attention and which can safely be ignored. The first concerns those who open or receive correspondence, whether that be indi- vidual addressees, their correspondence staff, or public-facing departments such as customer relations or complaints handlers. Here, a brief and simple screening process or filter is neces- sary for deciding which cases to refer on to security departments or risk managers. The second level concerns guidance as to the initial threat assessment and management of cases by the risk managers, security staff or policing teams to which such cases are referred. Its purpose is to avoid missing risk, to prevent unnecessary distress from over-estimating risk, to avoid unnecessary expense, and to point towards a suitable investigative response. Rather than a nuanced risk assessment involving detailed information and an absence of time pressure, it concerns the making of quick decisions in response to limited information in a dynamic, real-time setting. The CTAP was evolved to fulfil these functions.

The Knowledge Base

Literature

There is a considerable background body of published research to assist in the construction of such tools. This has been expertly summarised elsewhere (Meloy, 2014; Meloy et al., 2004, 2011, 2021), and the account here will be limited to the broad brush. Formal study of the attacks on public figures began in the nineteenth century with the work of Régis in France and Laschi and Lombroso in Italy (Farnham & Busch, 2016; James, 2015; Laschi & Lombroso, 1886; Régis, 1890). Modern research into this area was greatly advanced by Dietz and Martell’s (1989) report “Mentally Disordered Offenders in Pursuit of Celebrities and Politicians,” an unpublished work of 619 pages plus appendices in which was buried a treasure-trove of information about the character- istics of concerning correspondence and asso- ciations withsubsequent approach. Unfortunately, this was robbed of context in the two associated publications (Dietz, Matthews, Martell, et al., 1991; Dietz, Matthews, Van Duyne, et al., 1991), as mental illness was excluded from the account for“securityreasons”(Dietz&Martell,2010).The U.S. Secret Service’s Exceptional Case Study Project on lethal and near-lethal attacks, which had considerable influence, examined an historical cohort of 83 cases and found that 61% of indivi- duals had a history of psychiatric problems, 43% of delusional ideas, and 10% of violent com- mand hallucinations (Fein & Vossekuil, 1998): this compares with a point prevalence of psychotic illness in the general community of around 0.4% (Kirkbride et al., 2012). A further historical exer- cise looking at judicial figures was contributed by Calhoun (1998). Scalora and colleagues then pro- duced a series of studies of problematic contacts towards members of the U.S. Congress and factors associated with subsequent approach (Marquez & Scalora, 2011; Scalora, Baumgartner, Hatch- Maillette, et al., 2002; Scalora, Baumgartner, Zimmerman, et al., 2002; Scalora et al., 2003; Schoeneman et al., 2011; Schoeneman-Morris et al., 2007), which form the foundation of recent work in this area and aided subsequent research in the United Kingdom. As regards sectors other than politicians, Meloy and colleagues advanced knowledge of problematic communications to celebrities (Meloy, Mohandie, et al., 2008), and there are the beginnings of work on inappropriate

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 3

approaches to corporate figures (Hoffman & Sheridan, 2008; Meloy, Amman, et al., 2021). In parallel, conceptual work has taken place on the subjects of warning behaviours (Meloy et al., 2012) and leakage (Meloy & O’Toole, 2011). Inthe U.K., the Fixated Research Group (FRG)

produced a 300-page report in 2006 and published nine research papers in peer-reviewed journals (James, 2010; James et al., 2007, 2008, 2009; James, McEwan, et al., 2010; James, Meloy, et al., 2010; James, Mullen, et al., 2010; Meloy et al., 2004; Mullen et al., 2009). These papers concerned concerning communications to the British royal family and to politicians, the role of mentalillness,andfactorsthatpredictedpersistence, approach and attack. The relation of the FRG’s findings to the U.S. literature is set out by Mullen et al. (2008). It is from the work of the FRG, both in termsofitsinvestigationsandresearchpublications, that the CTAP was initially developed.

Threat Assessment Services

A number of threat assessment services in the public and private sectors over the last 25 years has published broad detail of their threat assess- ment approach (for instance, Cawood & Corcaran, 2003; Calhoun & Weston, 2003, 2009; de Becker, 1996;Fein&Vossekuil,2000;McElhaney,2004). Accounts are available of the work of the Los Angeles Police Threat Assessment Unit (Bixler et al., 2021; Dunn, 2014); the U.S. Secret Service (Fein & Vossekuil, 1998); the U.S. Capitol Police Threat Assessment Section (Scalora et al., 2008); and a range of other threat assessment services and approaches (Meloy & Hoffmann, 2014, 2021; Meloy, Sheridan, et al., 2008a, 2008b). Perhaps the most has been written in recent years about the Fixated Threat Assessment Centres (FTACs), which developed out of the work of the Fixated Research Group. Based on its re- commendations, the U.K.’s Office of Security and Counter-Terrorism (OSCT) and the Department of Health financed the establishment of a unit to assess and manage the risk to politicians and the royal family from individuals who write con- cerning letters or make concerning approaches. This, the U.K. FTAC, is a police unit, within the Metropolitan Police Service’s protection com- mand, but one which is jointly staffed by police officers and forensic psychiatrists, psychologists andnursesfromtheU.K.’sNationalHealthService (NHS).The rationale and operating practices of the

FTACs have been extensively described (Barry- Walshet al., 2020; James et al., 2014; Wilson et al., 2018, 2019, 2021). It was within this unit that the foundations of the CTAP were developed. TheFTACmodelwassubsequentlyreplicatedin

the Netherlands (Nelen et al., 2012, 2013; Sizoo & vanNobelen,2021),Queensland(Pathéetal.,2015, 2017; Pathé, Haworth, et al., 2016; Pathé, Lowry, et al., 2016), in the Australian state of Victoria, in Western Australia, in New South Wales, in Canberra (Riddle et al., 2019), and in Wellington, New Zealand. The extent of the problem of threats and disturbing correspondence to members of parliament from lone individuals has meanwhile been established in a number of jurisdictions (Akhtar & Morrison, 2019; Bjelland & Bjørgo, 2014; Brottsförebygganderådet(CrimePrevention Council), 2012; De Groot et al., 2010; Every- Palmer et al., 2015; Hamers et al., 2010; James, Farnham, et al., 2016; James, Sukhwal, et al., 2016; Muller et al., 2010; Narud & Dahl, 2015; Nijdam et al., 2008; Wallin & Wallin, 2014). Outcome studies from the U.K. FTAC have been published (James & Farnham, 2016; James, Kerrigan, et al., 2010),andfurtherresearch onriskfactorshas been produced from the FTAC database (Clemmow et al., 2021; Gill et al., 2021). Other research involving the FTACs has demonstrated the close similarities between non-ex-intimate stalkers in the general population and those who harass or pursue public figures (James et al., 2014; James, McEwan, et al., 2010; McEwan et al., 2012; Pathé, 2017). This is important in that it enables research on risk factors in stalking to be translated across into the field of risk assessment in public figures.

Assessment Tools

Individual threat assessment units and depart- ments will have their own internal data collection and assessment procedures, though few will be subject to research scrutiny. However, a number of structured assessment approaches have been developed and published in recent years to aid in the assessment and management of risk in situa- tions where inappropriate communications and approaches occur. For instance, the aforemen- tioned SAM, SRP and SASH concern stalking: the Workplace Assessment of Violence Risk-21 (WAVR-21; White & Meloy, 2016) and the Cawood Assessment Grid (CAG; Cawood et al., 2020) concern workplace violence; the Terrorist Radicalization Assessment Protocol-18

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

4 JAMES ET AL.

(TRAP-18) concerns lone terrorist activity (Meloy, 2016). Exercises have been undertaken to establish their reliability and validity (Cawood et al., 2020; Meloy, 2018; Meloy et al., 2013). With the notable exception of the stalking instru- ments, most emphasis in this area is focussed on theriskofviolenceandseriousphysicalharm.This is understandable in terms of their development in environments where firearms can be acquired by members of the public. However, it leaves some- thing of a gap. As described above, those in the public eye are in need of some general means of differentiating between concerning communica- tions which they can safely ignore, and those which they need to take seriously and pass on to theirsecuritydepartments.Securitydepartmentsin major companies and other organisations need a means of threat (rather than risk) assessment to aid in their initial prioritisation of responses to worrying contacts, a form of “screening in” and “screeningout”(Shafferetal.,2020).Andwhereas violence is evidently important, it is rare and other forms of risk (e.g., persistence, escalation, dis- ruption of function, reputational damage, financial loss, and psychological harm) are often of more immediate day-to-day concern. Attending to these needsistherolethattheCTAPwasdesignedtofill.

Nature of the CTAP

The CTAP comprises two parts: a preliminary Quick Correspondence Screen (QCS), and the main CTAP itself. The QCS is designed to be completed by those to whom communications are addressed or their staff. It comprises 18 items concerning the content of the communication. If any item on the list is present, the letter should be referred to the security or threat assessment department for more detailed examination, using the CTAP. If none are present, the communication needsnofurtheraction.TheCTAPitselfisdesigned to be used by security managers, threat assessors or police who have undergone training in its use. It contains 25 items, each of which is associated with increased risk. The CTAP is contained in a 55-paged manual which includes: an introduction, instructions for use, descriptions of the 25 CTAP items, instructions for allocation of concern level, frameworkforinitialmanagementplans,andguide- linesforcaseswhichdonotproveamenabletorapid resolution. The nature of each CTAP item is described in

turn on a separate page, with illustrations where

necessary, and the specification for each possible coding set out. For most items, there are two possible codings—yes or no. For others, a third “?” coding is defined. Guidance is given as to how to allocate cases into low, medium or high concern categories. The CTAP is not totalled into a simple “score. ” Whilst a higher number of positives is likely to be associated with a higher level of concern, the items are not equal in value, nor in likelihood of occurrence. Some of the factors are rarely encountered: however, their significance when present is such that it is important that they are always considered. Five of the factors are “red flag” items associated with a high risk of violence, such that the presence of one item alone is sufficient to render the case of high concern until shown otherwise. Use of the CTAP requires prior attendance at an approved, standardised training session, in person or on-line, lasting 4 hr and incorporating discussion about the practical appli- cation of the CTAP that may be peculiar to the delegates’ organisation, most usually concerning issues of business structure, reporting processes, and company policy regarding communication with the general public. The CTAP manual has been translated into Dutch and Norwegian. The CTAP has been in use over the last 7 years

by major international corporations, financial institutions and non-governmental organisations, family offices and the management teams of a range of sporting, acting, presenting and musical talent, security departments of elected assemblies, by policing departments, including the Netherlands National Police, and by various public agencies and charities.

Aims of the Study

ThisarticlesetsoutthebackgroundtotheCTAP and its development, before presenting the find- ings of inter-rater and test–retest reliability studies, and considering ways of evaluating its ability effec- tively to accord casesan appropriatelevelofpriority. It is the first part of the evaluation project, the second part of which will look at the validity of the CTAP.

Development of the CTAP and QCS

This section describes the identification of the problem with which the CTAP was designed to deal, the sources used in its construction, its evo- lution, the factors determining its format, and its adaptation to the non-governmental sector.

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 5

The Nature of the Problem

The ur-version of what was subsequently devel- oped into the CTAP was a product of the U.K. FTAC. The first problems faced by the develop- ment team in establishing FTAC were twofold. How should referrals to the new unit (mainly concerning correspondence, but also people trying to gain access to Royal and government buildings) be selected (or limited)? And how could a stan- dardised system be set up for deciding which cases warranted further assessment and determining the priority that they were afforded within limited resources? The FRG had already identified the stark problems within the existing systems. The absence of a suitable system for filtering cases for referral had seriously hampered attempts on the part of Royalty Protection police to assess inappropriate and threatening communications to membersoftheroyalfamily.Theirhouseholdshad been forwarding all letters that seemed odd or concerning, these amounting to some 11,000 a year. The few officers who were attempting to deal with them were overwhelmed, and the back- log had stretched to 6-months, by which time there was littlethatcouldbedone withmostofthe letters other than logging them. By contrast, the cor- respondence office at 10 Downing Street was well-staffed and efficient, and correspondence handlers were adept at identifying at least a pro- portion of thosecommunicationswhich weremost concerning. However, the system fell down in the waythesewerehandled.Themostworryingletters were given to the Prime Minister’s personal pro- tection officers who, it transpired upon investiga- tion, placed them in a cardboard box under a desk. Such “out of sight, out of mind” approaches proved to be a relatively common phenomenon, not just in government departments, but in private companies, where worrying letters were often simply placed in the “mad drawer” without any further action. Given the low base-rate of vio- lence, inaction in the great majority of such cases will be without consequence. However, the consequences of inaction can be so severe that systems need to be in place to assess risk, even where the chances of an adverse outcome occurring are low. The task, then, was to design a screening or filter instrument for use by correspon- dence offices, and to create a risk assessment protocol for the initial assessment of concerning communications in the new FTAC.

Examining Best Practice Elsewhere

The development team visited relevant organi- sations in Europe and the U.S. engaged in similar forms of threat assessment in order to study their processes and schedules. Some proved more rele- vant to FTAC’s purposes than others. The U.S. Secret Service placed such value on their main protectee that their methods appeared over-inclu- sive. The methods of the Behavioral Analysis Unit in Virginia were focussed on their various func- tions in assisting U.S. law enforcement agencies in investigations of specific crimes and were over- concentrated for FTAC purposes on the mainte- nance of specialised databases and the use of relatively complicated methods, such as textual analysis and the employment of algorithms. The experience of the Swedish Security Police in Stockholm was directly relevant in that the country possessed both a parliament and a con- stitutional monarchy, and there had been devel- opments in threat assessment procedures in the aftermath of the assassination of Olaf Palme, including a collaboration in case assessment between the Security Police and forensic psy- chologists and psychiatrists from the Karolinska Institute. A similar, but less developed, service was operating in the Norwegian Police Security Service. However, the most useful source of procedural material was the Capitol Police Threat Assessment Section in Washington D.C. (Scalora etal.,2008),whereMarioScaloraoftheUniversity of Nebraska had devised and supervised a sched- ule with a prominent psychological component to evaluate threatening and harassing contacts towards members of Congress (Marquez & Scalora, 2011; Scalora, Baumgartner, Hatch- Maillette, et al., 2002; Scalora, Baumgartner, Zimmerman, et al., 2002; Scalora et al., 2003, 2008; Schoeneman et al., 2011). The trove of material collected on these trips comprised a series of protocols, lists of risk factors, risk grids, and of less practical relevance, lists of key words to identify in correspondence (including multiple synonyms for kill, bomb, shoot, and engagement in procreational activity).

Examining the Literature

Extensive searching of the literature on risk assessment, threat assessment, violence, violence and mental illness, and stalking and harassment

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

6 JAMES ET AL.

was conducted. Opinions and advice were also sought directly from experts in the field. The research publications by the FRG referred to above were directly relevant to the identification of statistical associations with adverse outcomes, these associations ineffect constituting riskfactors. An example is that of “end-of-tether” language which, since its inclusion in the FRG publications, has been recognised as an important form of warning behaviour (Meloy et al., 2012) and been adopted asan indicator for violence risk, notjust in theCTAP,butinotherriskassessmentinstruments suchas the Stalking RiskProfile(MacKenzie et al., 2009) and the TRAP-18 (Meloy, 2016; Meloy & Gill, 2016). Other than statistical analyses, his- torical case studies were also informative in the consideration of relevant warning behaviours, par- ticularly where the phenomena in question were rare. An illustration is the factor which eventually became item 23 of the CTAP, “Delusions involv- ing horrific or frightening beliefs, which threaten the individual’s or others’ existence or physical integrity.” Two of a series of examples were Adel- heid Streidel’s stabbing of politician, Oscar Lafontaine, to bring attention to the abduction of people to underground killing factories where new beings were constructed from their body parts, the remaining flesh being sold as sausages in butchers’ shops across Germany: and Dieter Kaufmann’s shootingoftheGermaninteriorminister,Wolfgang Schäuble, to try and stop the beaming into his body by state transmitters of alternating pain and intense homosexual lust. Both had taken matters into their own hands when their multiple com- plaints to the authorities had come to nothing (James et al., 2007). Table 1 sets out some links to supporting

literature for each of the 25 CTAP items, together with references to other risk assessment instru- ments in which similar items have been incor- porated. This list is intended as illustrative, not exhaustive. It should be noted that the QCS items are simplified versions of CTAP items. As such, the table is also relevant to the locating of the QCS items within the literature.

Parallel Development With Other Risk Assessment Instruments

Two other assessment instruments were devel- oped in parallel with the CTAP, the Stalking Risk Profile and the SASH. The three CTAP authors

include the lead author and a co-author of the SRP, and two of the four authors of the SASH. The work on the Stalking Risk Profile contributed three important ideas which assisted in the construction of the CTAP: that it is important to consider domains of risk other than violence, specifically the risks of persistence, escalation, disruption of function, reputational damage, financial loss, and psychological harm to those upon whom the unwanted behaviours are focussed; that risk factors will differ according to the domain of risk; and that risk in each domain is likely to vary according to underlying motivation. The SRP also provided a model in terms of the description and scoring of risk items and their manualisation, and it contains an addendum on the structured assessment of risk to public figures. The SRP and the SASH, like the CTAP, incorporate the idea of “red flag” items, the presence of any one of which automatically means that the case is rated as being of high concern. There is an overlap in risk items included in the three instruments, and the CTAP incorporates four of the five “red-flag” items for violence contained in the SRP. The SASH uses the low, moderate and high concern levels adopted by the CTAP, rather than risk levels. The SASH manual follows the same format as that of the CTAP.Reliabilityandpredictivevaliditystudiesof both the SRP and the SASH have been published (Hehemann et al., 2017; McEwan et al., 2018).

Evolution of the Fixated Threat Assessment Centres Correspondence Screen and Initial Threat Screen

Correspondence Screen

Initial experience at FTAC of dealing with correspondence offices demonstrated the futility of trying to impart an effective understanding of threat assessment to their staff, or of relying on individuals’ experience and personal judgement. The rapid turn-over of employees in these roles meant that it was unrealistic to expect them to engage in any evaluation process, and instead it was necessary to have a simple screen for them to follow which covered no more than one side of paper. The items had to be written in a way that was easy to comprehend. In order to flag up a case for referral, the presence of any one characteristic described—in other words, the scoring of a single item as present—had to be the determining factor, so that no further judgement was needed. FTAC

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 7

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

T ab

le 1

C T A P It em

s a n d L in ks

to th e L it er a tu re

C T A P it em

S o m e li n k s to

th e li te ra tu re

1 . T h re at s

T h re at s in

st al ki n g ar e as so ci at ed

w it h v io le n ce , p ar ti cu la rl y in

ex -i n ti m at es

(e .g ,. C h u rc he r &

N es ca , 2 0 1 3 ). T h re at s to

p u b li c fi g u re s h av e y et

to b e d is ag g re g at ed

ac co rd in g to

m o ti va ti o n ; in

u n d if fe re n ti at ed

sa m p le s, th re at s ar e

n eg at iv el y as so ci at ed

w it h v io le n ce . S er io u s v io le n ce

is co m m on ly

p re ce d ed

b y th re at s, ev en

if m o st th re at s ar e n o t

fo ll o w ed

b y v io le n ce . T h re at s sh ou ld

al w ay s b e ta k en

se ri o u sl y in

p u b li c fi g u re

ri sk

as se ss m en t (M

el o y et

al ., 2 0 1 1 ).

T h re at s ar e a ri sk

fa ct o r in

th e S A M

(K ro pp

et al ., 2 0 0 8)

an d th e S A S H

(M cE

w an , S tr an d , et

al ., 2 0 1 7) , as

ar e d ir ec t

th re at s in

th e T R A P -1 8 (M

el o y,

2 0 1 6) .

2 . D ec la ra ti o n o f in te nt io n

D ec la ra ti o n o f in te n t to

ap p ro ac h is a si gn ifi ca nt as so ci at io n w it h ap p ro ac h in

p u b li c fi g u re ca se s (S ch o en em

an et al ., 2 0 1 1 ),

an d a st ro n g as so ci at io n o f ap p ro ac h in

th e F T A C d at a (G

il l et

al ., 2 0 2 1 ). It is as so ci at ed

w it h ap p ro ac h ca se s in

D ie tz

an d M ar te ll ’s

(1 9 8 9 ) w o rk

o n p o li ti ci an s, as

is st at in g a p ar ti cu la r ti m e o r p la ce

w h en

so m et h in g w o u ld

o cc u r.

It is a ri sk

fa ct o r fo r d is ru p ti o n in

th e p u b li c fi g u re

se ct io n o f th e S R P (M

ac K en zi e et

al ., 2 0 0 9 ).

3 . E v id en ce

o f d is p la ce m en t

M en ti on

o f tr av el

is as so ci at ed

w it h ap p ro ac h to

p o li ti ci an s (D

ie tz

& M ar te ll , 1 98 9 ), as

ar e k no w n p re v io u s tr ip s

to th e v ic in it y o f th e p o li ti ci an . S ee k in g p ro xi m it y is a ri sk

fa ct o r in

th e S A S H

(M cE

w an , S tr an d , et

al ., 2 0 1 7 )

4 . E x tr em

es o f an g er

A n g er

is a d ri v er

o f v io le n t o ff en di n g in

g en er al

(N o v ac o , 2 0 1 1 , 2 0 1 3 ), a ri sk

fa ct o r fo r v io le n ce

in p sy ch o ti c p at ie n ts

(M o na h an

et al ., 2 0 0 1,

p . 5 8 ), an d a m ed ia to r b et w ee n v ar io u s fo rm

s o f d el us io n an d v io le n ce

(C o id

et al ., 2 0 1 3 ).

E le v at ed

an g er

is a ri sk

fa ct o r in

th e S R P (M

ac K en zi e et

al ., 2 0 0 9 ) an d in

th e S A M

(K ro pp

et al ., 2 0 0 8) . M o ra l

o u tr ag e is a ri sk

fa ct o r in

th e T R A P -1 8 (M

el o y,

2 01 6 ).

5 . E sc al at io n in

an g er

o r in cr ea si n g p re o cc u p at io n

E sc al at io n h as

b ee n fo u n d to

b e an

as so ci at io n o f ap p ro ac h in

ce le b ri ti es

(M el o y,

M o ha n d ie , et

al ., 2 0 0 8 , p . 4 1 ).

A n es ca la ti on

o f b eh av io u r is a ri sk

fa ct o r in th e S A M

(K ro p p et al ., 2 00 8 ) an d in th e S A S H (M

cE w an ,S

tr an d ,e ta l. ,2 0 1 7 ).

6 . H ig h ly

p er so n al

q u es t fo r ju st ic e

T h e m aj or

ro le

o f “ fi x at io n”

in th is fi el d h as

b ee n d es cr ib ed

(M u ll en

et al ., 2 0 0 9 ) to ge th er

w it h it s o v er la p

w it h p er si st en t co m p la in in g an d w it h q u er u la n ce , w h ic h is as so ci at ed

w it h es ca la ti o n an d so m et im

es v io le n t

o u tc o m es

(M u ll en

& L es te r, 2 0 0 6) . R ef er en ce

to in ju st ic e o r v io la te d ri g h ts ar e a st ro n g in d ic at io n o f ap p ro ac h

in p u b li c o ffi ci al

ca se s (S ch o en em

an et

al ., 2 01 1 ). T h o se

p u rs u in g an

ag en d a in

th e ro y al

fa m il y st u d ie s w er e

m o re

li k el y su cc es sf u ll y to

b re ac h se cu ri ty

b ar ri er s, to

ac h ie v e p ro x im

it y to

th ei r ta rg et , to

ca rr y a w ea p on

an d to

en g ag e in

in ti m id at in g b eh av io u r (F ix at ed

R es ea rc h G ro up , 2 0 0 6) .

7 . D em

an d s to

ch an ge

b eh av io u r

T h e im

po rt an ce

o f d em

an d s to

ch an ge

b eh av io ur

is n o t o nl y th at

th ey

ar e in ap p ro p ri at e an d in tr us iv e in

cr o ss in g a

b o u n d ar y b et w ee n p ub li c an d p er so n al

ro le s, b ut

th at

th ey

sh o w

ev id en ce

o f a se n se

o f en ti tl em

en t w h ic h im

p li es

a la ck

o f co n st ra in t b y so ci al

n o rm

s. S u ch

d em

an d s ar e o ft en

in fu se d w it h an g er . T h e as so ci at ed

se n se

o f en ti tl em

en t

is a ri sk

fa ct o r fo r p er si st en ce

in th e S R P (M

ac K en zi e et

al ., 2 0 0 9) .

8 . D em

an d fo r m on ey /a p o lo g y

T h e m ak in g o f d em

an d s an d cl ai m s o f en ti tl em

en ta re as so ci at ed

w it h ap p ro ac h in p o li ti ci an

ca se s (S ch o en em

an et al ., 2 0 1 1 ),

as ar e h el p -s ee k in g re q u es ts . M u lt ip le

ap p ro ac h es

ar e si g n ifi ca nt ly

m o re

li k el y th an

si ng le

ap p ro ac h es

in th os e

u si n g d em

an d la n gu ag e in

ro y al

fa m il y ca se s (F ix at ed

R es ea rc h G ro up , 2 00 6 ). D em

an d s fo r sp ec ifi c o u tc o m es

w h ic h ar e u n li ke ly

to b e m et

ar e m o re

li k el y to

en g en d er

an g er

an d b e as so ci at ed

w it h p er si st en ce

an d es ca la ti o n .

F ai lu re s to

sa ti sf y co m p la in ts ar e a ri sk

fa ct o r in

th e S R P (M

ac K en zi e et

al ., 2 0 0 9 ).

9 . P ro li fi c co rr es p o n d en ce

It is a co n si st en t fi n d in g in

th is fi el d th at

in te n si ty

o f co n ta ct

is as so ci at ed

w it h ap p ro ac h (M

el oy

et al ., 2 0 1 1) .

T h is ca n b e m an if es t in

v o lu m e o f co m m u n ic at io ns , n u m b er

o f p eo pl e co n ta ct ed , an d n u m b er

o f d if fe re n t m et h o ds

o f co n ta ct

u se d.

M u lt ip le

co m m u n ic at io n s ar e as so ci at ed

w it h p er si st en t ap p ro ac h to

m em

b er s o f th e R o y al

F am

il y

(J am

es et

al ., 2 0 0 9) . M u lt ip le

co m m u n ic at io n s ar e a ri sk

fa ct o r in

th e p ub li c fi g ur e se ct io n o f th e S R P

(M ac K en zi e et

al ., 2 00 9 ).

8 JAMES ET AL.

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

T ab

le 1 (c o n ti n u ed )

C T A P it em

S o m e li n k s to

th e li te ra tu re

1 0 . A w ar en es s o f p er so n al

d et ai ls

K n o w le d g e o f p ri v at e d et ai ls n o t in

th e p u b li c d o m ai n re p re se n ts a d eg re e o f d et er m in at io n an d ac ti v e in tr us io n .

A w ar en es s o f v ic ti m

lo ca ti on

is a p re re q u is it e to

ap p ro ac h . It is a ri sk

fa ct o r in

th e S R P (M

ac K en zi e et

al ., 2 0 0 9 ).

1 1 . H is to ry

o f in tr u si ve

b eh av io u rs

P re v io us

st al k in g h as

b ee n li n k ed

to in cr ea se d li k el ih o o d o f fu tu re

st al ki n g . A

h is to ry

o f in tr us iv e b eh av io u rs

is a ri sk

fa ct o r in

th e S R P (M

ac K en zi e et

al ., 2 0 0 9)

an d in

th e S A S H

(M cE

w an , S tr an d , et

al ., 2 0 1 7) .

1 2 . E n d- o f- te th er

la n gu ag e

T h e im

po rt an ce

o f la st re so rt th in ki n g w as

re co gn is ed

in ca se

m at er ia l fr o m

th e F ix at ed

R es ea rc h G ro u p (2 0 0 6 ),

an d w as

ad o p te d as

a re d -fl ag

ri sk

fa ct o r in

th e S R P (M

ac K en zi e et

al ., 2 0 0 9 ) an d in

th e S A S H

(M cE

w an ,

S tr an d , et

al ., 2 0 1 7) . It is a w ar n in g b eh av io u r in

th e cl as si fi ca ti o n o f M el oy

et al . (2 0 12 ), an d in co rp o ra te d in to

th e

T R A P -1 8 (M

el oy , 2 01 6 ).

1 3 . S u ic id al

id ea ti on

S ta lk er s ar e at in cr ea se d ri sk

o f su ic id e, an d u p to a th ir d re p or ts u ic id al id ea ti on

(M cE

w an

et al ., 2 0 1 0; M o ha n d ie et al ., 2 0 0 6 ).

C as es

o f at te m pt ed

o r co m p le te d h o m ic id e- su ic id e h av e b ee n re p or te d in

ce le b ri ty

ca se s (S ch le si n g er , 2 00 6 ) an d

w id el y in

th e p re ss , as

w el l as

b ei n g a re co g n is ed

p h en o m en on

in ex -i n ti m at e st al k in g ca se s. S u ic id al

id ea ti on

m ay

ac t as

a d is in h ib it in g fa ct o r, in

th at

th o se

n o t ex p ec ti n g to

li v e ar e u n d er

fe w er

co n st ra in ts . It ca n al so

b e

as so ci at ed

w it h id ea s o f d em

o n st ra ti n g v ic ti m

st at us

an d in du ci n g re g re t in

th os e w h o ar e p er ce iv ed

as h av in g

h ar m ed

o n e.

S u ic id al

id ea ti o n is a re d -fl ag

in d ic at o r in

th e S R P (M

ac K en zi e et

al ., 2 00 9 ).

1 4 . S ex u al ly

ag g re ss iv e la n g u ag e o r fa n ta si es

S ex u al ly in ap p ro p ri at e la n gu ag e is n ot as so ci at ed

w it h m o re in tr u si ve

b eh av io ur s o r ap p ro ac h ,b ut it ca us es

p ar ti cu la r d is tr es s

w h ic h re q u ir es

in te rv en ti o n . M o re

ag g re ss iv e fa n ta si es

ar e a ri sk

fa ct o r fo r as sa u lt in

p re d at o ry

st al ke r ca se s in

th e S R P (M

ac K en zi e et

al ., 2 0 0 9) .

1 5 . In te re st in

at ta ck er s o r v io le n t ex tr em

is m

In te re st in

at ta ck er s an d v io le n t ex tr em

is m

is re co g n is ed

as a ri sk

fa ct o r in

lo n e ac to r g ri ev an ce .I t is a w ar n in g b eh av io u r in

th e cl as si fi ca ti o n o f M el oy

et al . (2 0 12 ), an d a ri sk

it em

in th e T R A P -1 8 (M

el o y,

2 0 1 6) .

1 6 . R ef er en ce s to

w ea p on s

T h e re le v an ce

o f w ea p on s w as

n ot ed

b y M el o y,

S h er id an , et

al . (2 0 0 8 b,

p . 2 5) . P o ss es si on

o f fi re ar m s is as so ci at ed

w it h ap p ro ac h (S ch o en em

an et

al ., 2 01 1 ). A

h is to ry

o f w ea p o n p o ss es si o n w as

si g n ifi ca nt ly

as so ci at ed

w it h

at te m pt in g to

b ea ch

se cu ri ty

b ar ri er s am

o n gs t F T A C ap p ro ac h er s an d th er e w as

a si g n ifi ca nt

as so ci at io n b et w ee n

ap p ro ac h an d h is to ry

o f co n ce al in g w ea po n s (G

il l et

al ., 2 0 2 1) . A cc es s to

o r af fi n it y w it h w ea p o n s is a ri sk

fa ct o r in

th e S R P (M

ac K en zi e et

al ., 2 0 0 9) .

1 7 . K n o w n h is to ry

o f v io le n ce

P re v io us

v io le n ce

is a p re d ic to r o f fu tu re

v io le n ce

in b o th

m en ta ll y d is o rd er ed

an d n on -d is o rd er ed

o ff en d er s

(B o n ta et al ., 1 9 9 8 ). S ta lk er s w it h a h is to ry

o f v io le n ce

ar e m o re li k el y to b e v io le n t( se e m et a- an al y si s: C h ur ch er & N es ca ,

2 0 1 3 ). P ro b le m at ic

ap p ro ac h er s to

p u b li c fi g u re s ar e si g n ifi ca nt ly

m o re

li k el y to

h av e a h is to ry

o f v io le n ce

th an

co m m un ic at o rs (G

il le ta l. ,2 02 1 ;S

ca lo ra ,B

au m g ar tn er ,H

at ch -M

ai ll et te ,e ta l. ,2 0 0 2) .P

re v io us

v io le n ce

is a ri sk

fa ct o r in

th e T R A P -1 8 (M

el o y,

2 0 1 6)

an d in

th e S A S H

(M cE

w an , S tr an d , et

al ., 2 0 1 7) .

1 8 . H o m ic id al

id ea ti o n

E n q u ir y ab o u t h om

ic id al

id ea ti on

is a st an d ar d el em

en t o f v io le n ce

ri sk

as se ss m en t an d in co rp o ra te d in to

ri sk

as se ss m en t to o ls su ch

as th e C la ss ifi ca ti o n o f V io le n ce

R is k (C O V R ; M o n ah an

et al ., 2 00 6 ). H o m ic id al id ea ti o n is g re at ly

in cr ea se d in

se ri o u s p sy ch o ti c il ln es se s (C ar b on e et

al ., 2 0 2 0 ). It w as

fo u n d to

b e an

o m n ib u s ri sk

fa ct o r in

a p ri so n

sa m p le

(D eL

is i et

al ., 2 01 7 ). It is a re d fl ag

it em

fo r v io le n ce

ri sk

in th e S R P (M

ac K en zi e et

al ., 2 00 9 ). It is g en er al ly

co n ce p tu al is ed

al o ng

si m il ar

li n es

to su ic id al

id ea ti o n .

1 9 . D el u si o n s o f lo v in g re la ti o n sh ip

E ro to m an ia

is as so ci at ed

w it h in tr us iv en es s an d w it h a h ig h d eg re e o f p er si st en ce

in th e ab se n ce

o f in te rv en ti o n

(M cE

w an

et al ., 2 0 0 9 ; M cE

w an , S tr an d , et al ., 2 0 1 7) , as

is in ti m ac y -s ee ki n g m ot iv at io n (J am

es , M cE

w an , et al ., 2 0 1 0 ).

E ro to m an ia

is a ri sk

fa ct o r in

th e S R P (M

ac K en zi e et

al ., 2 0 0 9 ).

(t a b le

co n ti n u es )

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 9

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

T ab

le 1 (c o n ti n u ed )

C T A P it em

S o m e li n k s to

th e li te ra tu re

2 0 . D el u si o n s o f je al o u sy

T h is it em

in cl ud es

d el us io n al

je al ou sy

an d je al o u sy

in re la ti o n sh ip s w h ic h ar e d el us io n al . P at h o lo g ic al

je al o u sy

is as so ci at ed

w it h as sa u lt (M

u ll en

& M aa ck ,1 9 8 5 ;S

il v a et al ., 1 9 9 8) ,a nd

je al o u sy

w it hi n d el u si on al re la ti o n sh ip s (a lt h o u g h

u n co m m o n in

o th er

se tt in g s)

is p re su m ed

to b e si m il ar .

2 1 . B el ie f in

sh ar ed

p as t o r d es ti n y

N o n -e ro to m an ic

d el u si o n s o f sh ar ed

p as t o r d es ti n y ar e as so ci at ed

w it h in cr ea se d in tr us iv en es s an d p er si st en ce .

F o r in st an ce , th os e w it h d el us io n al

fa m il y ti es

w er e si gn ifi ca n tl y m o re

li k el y to

ap p ro ac h in

R o y al

F am

il y C as es

an d

si g n ifi ca nt ly

m o re

li k el y to

m ak e m u lt ip le

ra th er

th an

si ng le

ap p ro ac h es . (F ix at ed

R es ea rc h G ro up , 2 0 0 6) .

D el u si o n s o f a sp ec ia l re la ti o n sh ip

ar e in cl u d ed

u nd er

“ d el us io n s re la te d to

st al ki n g ” as

a ri sk

fa ct o r in

th e S R P

(M ac K en zi e et

al ., 2 00 9 ).

2 2 . B el ie f p eo pl e ar e im

p os te rs

o r p o ss es se d

T h e as so ci at io n s o f d el us io n al

m is id en ti fi ca ti o n sy n dr o m es

w it h v io le n ce

h av e b ee n n ot ed

in co ll ec ti o n s o f ca se

st ud ie s

(S il v a et

al ., 1 99 6 ) an d fo u nd

in co m m u n it y sa m p le s o n d el us io n s an d v io le n ce

(C o id

et al ., 2 0 1 3 ).

2 3 . T h re at

to p er so n al

in te gr it y

F in d in g s ab o u t th e as so ci at io n b et w ee n th re at -c o nt ro l- o ve rr id e sy m p to m s an d v io le n ce

(L in k et

al ., 1 9 9 8)

ap p ea r

to b e b or n o ut

in fu rt h er

w o rk

(e .g ., H o d gi n s et

al ., 2 0 0 3 ). S u ch

h ig h -r is k p sy ch ot ic

p he n o m en a ar e a re d -fl ag

in d ic at o r fo r v io le n ce

ri sk

in th e S R P (M

ac K en zi e et

al ., 2 0 0 9 ). T h re at -c o n tr o l- o ve rr id e sy m p to m s ar e as so ci at ed

w it h p ro b le m at ic

ap p ro ac h b eh av io u r to w ar d s p u b li c o ffi ci al s (S ch o en em

an et

al ., 2 0 1 1 ).

2 4 . B el ie f in

o w n d iv in it y o r o n a d iv in e m is si o n

S er io u s v io le n ce

h as

b ee n si gn ifi ca n tl y as so ci at ed

w it h g ra n d io si ty

in th o se

w it h sc h iz op h re n ia

(S w an so n et

al ., 2 00 6 ).

G ra n d io si ty

si gn ifi ca n tl y d if fe re n ti at ed

b et w ee n th o se

w h o ap p ro ac h ed

U .S . ce le b ri ti es

fr o m

th os e w h o d id

n o t

(D ie tz

& M ar te ll , 1 98 9 ). It is as so ci at ed

w it h ap p ro ac h in

p u b li c o ffi ci al

ca se s (S ch o en em

an et

al ., 2 01 1 ) an d in

p ro b le m at ic ap p ro ac h to w ar d a m em

b er o f th e B ri ti sh

R o y al F am

il y (J am

es ,M

cE w an ,e t al ., 2 0 1 0 ). T h o se

w it h d el us io n s

o f d iv in e p u rp o se

w er e id en ti fi ed

in th e F ix at ed

R es ea rc h G ro up

(2 0 06 ) re p or t as

a p ar ti cu la rl y d if fi cu lt g ro u p

u n tr am

m el le d b y in h ib it io n s o r b o u n d ar ie s. E x tr em

es o f g ra n d io si ty

ar e a ri sk

fa ct o r in

th e p u b li c fi g u re

se ct io n

o f th e S R P (M

ac K en zi e et

al ., 2 0 0 9 ).

2 5 . G u t re ac ti o n

G u t fe el in g s co n st it ut e as so ci at iv e p ro ce ss es

w h ic h ar e in tu it iv e an d n o t im

m ed ia te ly

ac ce ss ib le

to co n sc io us

aw ar en es s, b u t ar e v al ua b le , in

p ar ti cu la r in

th o se

w it h ex p er ie n ce

o f a g iv en

fi el d (S lo v ic

et al ., 2 0 0 4) . W h er ea s

th ey

ar e n o su b st it u te

fo r ev id en ce -b as ed

as se ss m en t, th ey

ar e an

ad ju n ct

w h ic h sh ou ld

n ot

b e ig no re d

(M cE

w an , S tr an d , et

al ., 2 0 1 7 ).

N o te .

C T A P = C o m m u n ic at io n s T h re at A ss es sm

en t P ro to co l; S A M

= S ta lk in g A ss es sm

en t an d M an ag em

en t; S A S H = S cr ee n in g A ss es sm

en t fo r S ta lk in g an d H ar as sm

en t; F T A C =

F ix at ed

T h re at

A ss es sm

en t C en tr e;

T R A P -1 8 =

T er ro ri st R ad ic al iz at io n A ss es sm

en t P ro to co l- 1 8 ; S R P =

S ta lk in g R is k P ro fi le .

10 JAMES ET AL.

adopted a liaison system, with a police officer and a community psychiatric nurse assigned to each of the main referring correspondence sections. This provided a feed-back mechanism as to which features or wording of the screen caused problems to any individuals. In addition, the referrals to FTAC each day, made using the screen, consti- tuted a daily exercise in examining its efficacy in correctly identifying cases suitable for referral—in other words, in the identification of false positives. Such cases were then discussed with the originat- ing correspondence offices to ascertain the reasons for the case being referred. The FTAC liaison staff also undertook regular examination during the development period of samples of letters from their linked correspondence offices which had been thought inappropriate in some way but had notbeen referred toFTACon the basis of using the screen. In other words, this was a regular exercise in ascertaining the presence of false negatives. The wording in the screen was adapted through

experience of its use on several thousand cases. In addition, the original list of some thirty items in the screen was reduced over time for two reasons: firstly, someitemswerenever scoredaspositive— largely because they necessitated knowledge which could rarely be obtained from the presenting material; secondly, some items almost always co-occurred becausethey wereineffectmeasuring the same thing. It proved possible to add in additional items which concerned a pattern of behaviour, because the royal households kept a computerised database of correspondents. The final version of the screen covered similar content to that of the CTAP, but in simplified form. As a specific exercise in searching for false

negatives, a sample of the Home Secretary’s mail was examined, the correspondence office having removed all that simply concerned core business. FTAC police staff used the correspondence screen onasampleof283inappropriatecommunications: those that were not flagged up by the screen were then examined by FTAC consultant forensic psychiatrists to ascertain whether cases had been “missed” by the screen which should have occa- sioned sufficient concern for referral. None was identified in the sample.

Initial Threat Screen

The initial FTAC threat assessment tool was applied to all new referrals, enabling a judgement as to whether the case was of low, moderate or

high concern, this determining the priority of intervention. Low concern cases were logged, but no other action taken. Moderate concern cases were taken on for further investigation and man- agement. High concern cases were dealt with as an urgent priority. The threat assessment tool started as a grid, but

then changed in form into a series of items con- cerning the subject’s characteristics or behaviour, grouped into thematic sections and incorporating “red-flag” items. The list of items was developed from the sources described above. It concerned not simply violence, the base rate of which is low, but also the areas of persistence, escalation, dis- ruption, impairment of function, psychological harm, and reputational damage. The original list comprised 65 items and concerned the assess- ment both of concerning communications and concerning approaches. Its length was unwieldy and impractical, and as it was developed over 6 years and used on some 6,000 cases, its was reduced to 38 items through processes similar to those described for the correspondence screen above—co-occurrence and unavailability of data. It was necessary for the items to include both characteristics which were relatively common and those which were rare but of considerable importance when present.This, together with the use of “red-flag” items, meant that the determining of concern level using the tool could not involve a summation of scores, but required a defined pro- cess,withinstructionsastowhich cases should fall into which level. This determined that those using the tool required formal instruction in its use.

Development of the CTAP and QCS for Use in the Private Sector

The catalyst for the development of the pub- lished CTAP from the initial work at FTAC was the recognition of the need for a structured profes- sional judgement tool concerning initial assess- ment of unwanted communications for use in the private sphere, given that no sufficiently specia- lisedtoolwasavailableinthatarea,whilstitshould remain relevant to police settings. It needed to be a comprehensive, fully-fledged and self-contained package, rather than a step in the operating proto- colofapolicingunit.And,whilsttheproblemsthat the CTAP seeks to address are fundamentally the same in the private sphere, there are differences which need to be taken into account. Private firms

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 11

tend to be larger, more complex and more multi- national in nature than their public sector counter- parts: they rely heavily on on-line platforms and electronicformsofcommunication,andhavedone so for longer than organisations in the public sphere, and the management of complaint and grievance tends to be part of core business to a greater extent than in the public sphere. Of note, private organisations tend to be more sensitive to a broader range of risks, in particular the reputa- tional, the financial and the risk of disruption, and they do not have access to the same range of information sources and powers of intervention thatpolicingorganisationsroutinelyemploy.There is also a wider range of organisational roles in private organisations to which the use of a struc- tured threat assessment process is relevant—for instance, security, communications, operations, human resources and legal managers, so care- fully thought-out training packages needed to be incorporated into the development of a relevant system. A clearer recognition and understanding of the needs of organisations in the private sphere was achieved by the authors through its interac- tions with corporate entities and through corporate case-work, insights which were incorporated into the development work on the CTAP. Taking into account the above concerns, the eventual CTAP was produced incorporating 25 core items. The development of the QCS paralleled that of the CTAP. Both cover the same ground, in that the QCS reflects in simpler fashion the content of the CTAP.

Reliability Testing: Method

Study Sample

The study sample concerned 102 consecutive cases referred for risk assessment and/or manage- ment to Theseus, a risk assessment company. This waschosenbecausethecompany was setuponthe FTAC model and adopted FTAC’s data collection and risk assessment methodology. It was also the setting in which the final form of the CTAP was developed. Fifty of the cases came from the cor- porateworld, 38concerned the talent sector and14 were private clients. Each of the 102 cases was examined and scored independently on both the CTAP and QCS by two separate raters, neither of whom was a trained psychologist or psychiatrist, and neither of whom was an author of the CTAP.

Both worked at Theseus and had been trained in the use of the CTAP. This was not an exercise that involved vignettes or that was necessarily limited to single pieces of correspondence: rather, it con- cerned “real world” cases, in which the informa- tion available to the raters was that presented at referral. The amount of material varied between cases from very little to considerable, generally according to how long the behaviour of concern had been occurring. Some cases included summa- riesfromthereferrers,whilstotherswerelimitedto the correspondence itself. The number of items of correspondence per relevant case ranged from one to several hundred. Of the 102 cases rated, 97 (95.1%) involved correspondence, of which 41 also involved some form of intrusive behaviour, ranging from turning up at the person’s place of work to attempts at approaching the addressee. Five cases involved some form of approach, with- out correspondence. Whilst the CTAP is intended for written communications, these were included inthesample,usingrecordsofverbaloutput,asthe sample was intended to represent the “real world” of threat assessment work.

Inter-Rater Reliability Procedure

Both the CTAP and the QCS were rated for each case, and the results (together with a range of other data items) were entered into a database using SPSS, version 26 (IBM Corp, 2019). Inter- rater reliability was then calculated for each item in the CTAP and the QCS. Particular note was taken of the level of agreement on “red flag” items, as the presence of one of these determines an overall “high concern” finding, independent of any other scoring. Inter-rater reliability was then examined for the outcome measures of the QCS and of the CTAP. For the QCS, the out- come decision was whether the case needed referral to someone else, this being determined by the scoring of at least one QCS item as positive. For the CTAP, inter-rater agreement on concern level judgement in each case was compared. This was seen as a particularly important metric, both because it represented the main outcome measure of the CTAP rating, and because it incor- porated a further element of judgement by each rater, the translation of the findings on the individ- ualCTAPitemsintoanoverall concernjudgement in the manner set out in the CTAP manual. The total scores for the QCS and the CTAP

were then calculated in each case for each rater,

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

12 JAMES ET AL.

and the inter-rater reliability of the total scores was calculated. It should be noted that the total- ling of scores is not part of the procedure for using either theQCSortheCTAP. TheQCSis designed to give a “positive” result, if only one item is judged present. And the CTAP contains a mixture of rare and relatively common risk factors, which cannot simply be summated, as well as incorpo- rating “red flag” items, the presence of one alone being sufficient for the case to be judged as being of high concern. Nevertheless, it was decided to include this measure as a further form of check on inter-rater consistency.

Test–Retest

Twenty cases were then randomly selected from the first part of the database and rated again on the CTAP by one of the original raters at a minimum of 2 years after the original rating, without reference to the original rating results. The two ratings were then compared to give mea- sures of test–retest reliability for the individual CTAP items and for the concern judgement. The period of 2 years was selected as long enough for the rater no longer to retain memory of the original ratings.

Concern Levels

Lastly, the concern level from the initial CTAP rating for each case was compared with the con- cernlevelatthepointTheseusclosedthecasefrom its books or last received feedback on progress. Thereasoningforthiswasasfollows.TheCTAPis not a predictive instrument, given that it does not aim to reach a judgement on future risk, but rather to aid those engaged in the initial assessment of a case to decide upon the need for, and priority to be accorded to, any initial intervention. As such, the CTAP is not open to any simple exercise in assessing predictive validity. However, it is possible to use a change in concern levels to gain some indication as to its performance. The CTAP is balanced conservatively to accord pri- ority to a case presenting with certain defined features until further information can be gath- ered which will either confirm the need for further intervention, or enable the concern level to be reduced. In other words, it is designed to avoid false negatives. Were it to be failing to do this, then it would be expected that there would be a proportion of cases in which the concern

level would rise subsequent to the initial assess- ment. Consequently, the absence of any increase in concern level between initial and final contact would constitute evidence that the CTAP was not underestimating initial concern. Alterna- tively, if the CTAP were consistently overesti- mating the priority that should be accorded to cases, then it would be expected that there would be a high proportion of cases initially rated as moderate or high concern which were subse- quently reduced to low. Finally, if the initial level of priority suggested by the initial CTAP application were appropriate, then a proportion of cases would have been reduced in concern level through Theseus intervention, given that part of its purpose is to aid in the drawing up and execution of management plans to reduce or contain threat. It would not be anticipated that all cases would be reduced to a low level of concern, in part because risk containment does not necessarily equate to risk reduction, and in part because those receiving suggested manage- ment plans may choose not to put them into effect.

Statistical Analysis

To ascertain a coefficient of agreement between the two raters on categorical variables, specifically the ratings for each item on CTAP and QCS, Cohen’s kappa (κ) was used (Cohen, 1960), kappa being a means of ascertaining the propor- tion of cases in which there was agreement which takes account of the level of agreement that would be expected based simply on chance (Agresti, 2013). Kappa values greater that zero represent better-than-chance agreement between the two raters, to a maximum value of +1, which indicates perfect agreement. Different classifi- cations have been suggested for the strength of agreement based on the value of Cohen’s κ. Here, the guidelines used were taken from Altman (1999), originally adapted from Landis and Koch (1977). A value of less than 0.20 is classi- fied as poor agreement; 0.21–0.40 fair; 0.41– 0.60 moderate; 0.61–0.80 good; and 0.81–1.00 very good. Confidence intervals were calculated as ±(1.96 × SE). To compare total scores for the CTAP between

raters and total scores for the QCS between raters, the intraclass correlation coefficient (ICC) was used, ICC being a measure which reflects both degree of correlation and agreement between

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 13

measurements. Selecting from the ten forms of model set out by McGraw and Wong (1996), a two-way random model was used, absolute agreement type, single rater, this being selected to allow generalisation from the current raters to a larger set of possible raters. The ICC values were interpreted according to the thresholds set out by Shrout (1998): virtually none (0.00–0.10), slight (0.11–0.40),fair(0.41–0.60),moderate(0.61–0.80), and substantial (0.80–1.0). To examine the correlation between total

CTAP scores and total QCS scores for each rater, and to compare judgements as to concern level, Kendall’s tau-b (τb) was used (Agresti, 2010; Kendall, 1938). Test–retest reliability was exam- ined using Cohen’s κ.

Results

Inter-Rater Reliability for CTAP Items

The percentage agreement and kappa values, with 95% confidence intervals, are given for the CTAP in Table 2, together with the proportion of positive responses by each rater for each item. Some items were infrequently scored as present, with there being no cases in this sample which were positive for Item 24. Of the 25 items, kappa was not calculated for three items for the following reasons: Item 20, sample too small to calculate; Item 24, no data; Item 25, score for Rater 1 is a constant. For 16 of the 25 items, the kappa value fell withinthe band 0.81–1.00. Items1, 6, 9, 14, 15 fell within the band 0.61–0.80. One item, Item 21, producedtheresultκ=0.499(0.301–0.697)which is in the “moderate” agreement band. Percentage agreement fell below 95% for only five items (1, 6, 9, 11, and 21).

“Red Flag” CTAP Items

Of the five items, the percentage agreement rangedbetween97%and100%.Thespecifickappa values for each item with 95% confidence limits were as follows: end-of-tether’ language 0.830 [0.642–1.000]; suicidal ideation 1.000; homi- cidal ideation 0.913 [0.756–1.000]; and threat to personal integrity 0.936 [0.811–1.000]. One red flag item (delusions of jealousy) was present in too few cases (two) to allow the calculation of a kappa value.

Inter-Rater Reliability for QCS Items

The percentage agreement and kappa values for the 18 QCS items, with 95% confidence intervals, are given in Table 3, together with the proportion of cases in which each rater found each item positive/present. For Items 1 and 15, kappa was notcalculated,asatleastoneraterproportionwasa constant. For the remaining 16 items, kappa fell into the agreement band 0.81–1.00. All cases scored at least one positive item for each rater. Percentage agreement for the eighteen items lay between 90% and 100%

Total CTAP and QCS Scores

The intraclass correlation coefficient (ICC) for total CTAP score between raters was 0.962, 95% CI [0.944–0.974]. For total QCS score, the ICC was 0.954, 95% CI [0.932–0.969].

Level of Concern

There was a strong positive correlation between the judgements as to level of concern τb = 0.899, p < .001. There was absolute agree- ment in 92% of cases.

Test–Retest Reliability

Therewas100%agreementbetweentheoriginal testandthere-teston19ofthe 25items.Percentage agreement and Kappa values (all p < .005), with 95% confidence intervals, were as follows for the remaining items: Item 1; 80%, 0.718 [0.432– 1.000]: Item 3; 95%, 0.898 [0.704–1.000]: Item 5; 95%, 0.900 [0.710–1.000]: Item 11; 95%, 0.886 [0.670–1.000]: Item 19; 90%, 0.750 [0.427– 1.000]: Item 21; 95%, 0.623 [0.166–1.000].

Change in Concern Levels

At initial assessment, 14 cases (13.7%) were assessed as being of low concern, 57 cases (55.9%) as of moderate concern, and 31 cases (30.4%) as of high concern. Theseus intervention lasted less than 2 months in 86 cases (84.3%), between 2 months and 1 year in 12 cases (11.7%), including one case which was still open, and between 1 and 3 years in 4 cases (3.9%). At the end of Theseus’ involvement, all the low concern cases remained as low concern. Of the 57 cases of moderate concern, 22 (39%) remained

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

14 JAMES ET AL.

of moderate concern and 35 (61%) had been reduced to low concern. None had increased to highconcern.Ofthe31casesofhighconcern,one case (3%) remained of high concern, this being the case that was still open: of the remainder, 23 (74%) had been reduced to moderate concern and 7 (23%) to low concern.

Discussion

The CTAP is unusual in being designed spe- cifically for the purpose of allocating priority in the initial assessment of concerning communica- tions and for the making of quick decisions in response to limited information in a dynamic,

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

Table 2 Inter-Rater Reliability of CTAP Items

Item (brief characterisation) Percentage agreement

Proportion of yes responses

(Rater 1, Rater 2) (%)

Kappa [95% CI]

in all cases, p < .005

1. Threats 88.24 48, 51 0.806 [0.708–0.904]

2. Declaration of intent 97.06 42, 45 0.940 [0.873–1.000]

3. Evidence of displacement 96.08 43, 45 0.920 [0.844–0.996]

4. Extremes of anger 97.06 17, 17 0.929 [0.833–1.000]

5. Escalation in anger 97.06 36, 39 0.937 [0.866-1.000]

6. Quest for justice 91.00 17, 18 0.744 [0.587–0.901]

7. Demands to change behaviour 95.00 36, 37 0.892 [0.800–0.984]

8. Demand for money/apology 97.03 26, 23 0.919 [0.829–1.000]

9. Prolific correspondence 91.18 27, 26 0.771 [0.628–0.914]

10. Awareness of personal details 96.04 36, 32 0.911 [0.827–0.995]

11. History of intrusive behaviours 92.08 28, 34 0.818 [0.698-0.938]

12. End-of-tether 97.06 6, 8 0.830 [0.642–1.000]

13. Suicidal ideation 100 8, 8 1.000 14. Sexually aggressive 95.01 6, 7 0.666

[0.411–0.921] 15. Interest in attackers 99.00 2, 3 0.795

[0.403–1.000] 16. Reference to weapons 97.06 10, 9 0.942

[0.828–1.000] 17. History of violence 98.97 3, 4 0.905

[0.721–1.000] 18. Homicidal ideation 99.02 4, 4 0.913

[0.756–1.000] 19. Delusions of love 96.08 19, 18 0.900

[0.804–0.996] 20. Delusions of jealousy 98.04 1, 1 Sample too small to calculate 21. Belief in shared destiny 85.30 8, 7 0.499

[0.301–0.697] 22. Imposters or possessed 100 3, 3 1.000 23. Threat to personal integrity 99.02 8, 9 0.936

[0.811–1.000] 24. Belief in own divinity 100 0, 0 No data 25. Gut reaction 98.04 100, 99 Not calculable: score for Rater 1 a constant

Note. CTAP = Communications Threat Assessment Protocol.

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 15

real-time setting. It constitutes a promising instru- ment designed to have considerable utility in a range of settings, its strengths being that: it is a general tool looking at risk as a whole, rather than simply violence; it is designed to have wide appli- cability; it incorporates standardised approaches both for recipients of communications (or their correspondence staff),and forthe securityor threat assessment personnel who are required to conduct an initial evaluation of correspondence referred to them; it has been evolved over a number of years through practical application, which renders it straightforward and user-friendly. The foundations of the CTAP were laid through

dailyuseofearlieriterationsinapolice/NHSthreat assessment setting over an 8-year period on real

cases as they unfolded, the cases being many thousand in number. It incorporates knowledge accumulated by leading threat assessment agen- ciesontwocontinents,aswellasleadingexpertsin the fields of threat, harassment, stalking and vio- lence and mental illness. The items of the CTAP are firmly anchored in research evidence, as illus- trated in Table 1. It was developed alongside two other instruments, the SRP and the SASH, with which it shares some content and structural fea- tures, and both of which have been shown to have useful reliability and predictive validity. Its even- tual development into the comprehensive, self- containedCTAPpackagefollowedtherecognition of the need for a structured professional judgement tool for use in the private sphere, and involved a

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

Table 3 Inter-Rater Reliability for QCS Items

Item Percentage agreement

Proportion of yes responses (Rater 1, Rater 2) (%)

Kappa [95% CI]

in all cases, p < .005

1. 100 100, 100 Not computed as case proportions are constants.

2. 99% 53, 54 0.980 [0.941–1.000]

3. 98% 25, 25 0.947 [0.875–1.000]

4. 96% 45, 45 0.920 [0.845–0.996]

5. 99% 12, 13 0.954 [0.866–1.000]

6. 98% 34, 32 0.955 [0.894–1.000]

7. 95% 28, 31 0.883 [0.783–0.983]

8. 90% 26, 26 0.742 [0.591–0.893]

9. 99% 11, 10 0.947 [0.843–1.000]

10. 100% 7, 7 1.000 11. 98% 6, 6 0.823

[0.584–1.000] 12. 98% 13, 11 0.906

[0.777–1.000] 13. 96% 33, 33 0.912

[0.828–0.996] 14. 97% 8, 10 0.878

[0.712–1.000] 15. 99% 0, 1 Not computed: figure for Rater 1 is a constant. 16. 94% 25, 27 0.845

[0.725–0.965] 17. 94% 21, 24 0.857

[0.736–0.976] 18. 96% 30, 34 0.911

[0.823–0.997]

Note. QCS = Quick Correspondence Screen.

16 JAMES ET AL.

further step—the incorporation of the perspective and requirements of non-public organisations. Additional experience of its use has been accu- mulated over 7 years in the assessment of cases involving prominent individuals, celebrities and corporate entities. Overall, its provenance and research roots go some way to establishing its content validity. This is the first study that has examined the

reliability of the CTAP. The results for inter-rater reliability are encouraging; the CTAP demon- strated very good agreement for most of its items. One CTAP item (belief in shared destiny) pro- duced a disappointing kappa value of 0.499, 95% CI [0.301–0.697], which is in the moderately good agreement band, with confidence intervals ranging from fair to good. This item requires the rater to make a judgement as to the degree of conviction that is present in the mind of the indi- vidual under consideration. It may be that this is a more difficult judgement to make than with factual items: or it may be that the wording of the CTAP or the training module offer insufficient guidance. It is of note that the test–retest reliability was also poorest on this item. It is particularly important that there should be

good inter-rater reliability on the “red flag” items, given their particular role in determining that a case should be judged of high concern. Agree- ment was in the very good range for the four items for which it was possible to calculate results. However, for one red flag item (delusions of jealousy), there were insufficient cases in the sample of 102 to allow analysis. This is under- standable in terms of the way the CTAP is con- structedtoincludebothrelativelycommonfeatures and uncommon characteristics which, when pres- ent, are of such importance that their incorporation into a threat assessment instrument is warranted. A larger sample and/or repetition of this exercise with a sample of different origin would be necessary to test reliability more robustly for this item. The test–retest reliability for the CTAP was

examined on a smaller sample of twenty cases with at least 2 years between evaluations. Agree- ment fell in the “very good” range for 21 of the 25 items and in the “good” range for the remain- der. This is promising, though repeating the exercise on a larger sample would be desirable. But of particular note in this study was the high level of agreement between raters as to level of concern. This is a separate part of the CTAP process, for which instructions are given in the

manual.Evidently, it is not only of importance that there is inter-rater agreement on the individual CTAP items: there needs also to be consistency between raters in the manner that these are con- verted into a judgement as to level of concern, this after all being the final “product” of the exercise. The final part of this study concerned an indi-

cator as to whether the level of concern allocated is likely tohave been appropriate. The purpose of the CTAP is to triage cases at initial assessment into three concern levels as an indication of the priority that they should be accorded. As explained in detail above, the pattern of change in priority level between initial assessment and case closure can offer information as to whether the CTAP is not pronetogivingfalsenegatives,ora preponderance of false positives. The results indicate that neither is likely to have been the case. There is little in the literature with which to compare these particular measures (James, Kerrigan, et al., 2010), but this study suggests that they may be of use. Given that the CTAP’s purpose is not a predictive one, ex- ercises in predictive validity present problems in design. A companion paper to this one examines the ability of CTAP items to distinguish between those concerning correspondents who go on to approach and those that do not, and examines the relationship between CTAP items and persistence.

Limitations

This is the first such study of the CTAP and confirmation of its results by other groups would be desirable in order to offer a firmer foundation foritsconclusions andtoensure thatitsresultscan be generalised. The study involved two raters, and its repetition with a greater number would be instructive. As regards the QCS, the results here should be treated with caution, in that a corre- spondence screen intended for use by correspon- dence handlers was completed by individuals directly involved in threat assessment. This might have resulted in an artificially good performance of the QCS in terms of inter-rater reliability. A limitation of the CTAP itself is that it em-

ploys the use of a manual and requires standar- dised training. However, it is not unique in this respect, thetraininglastshalfaday,andtheCTAP itself can be completed quickly by those familiar with its use. A limitation of the present study is that there were insufficient cases concerning two of the CTAP items to produce kappa values. This is the consequence of using real world cases,

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 17

rather than constructed vignettes. However, the use of real cases in a threat assessment setting is also one of the strengths of the study, given that initial case material does not always contain clear data and often involves a degree of ambiguity which can make cases more difficult to rate.

Directions for Future Research

As stated above, this study has been the first of a two-part project, the second part of which concerns the predictive validity of the CTAP. The first need is for the repetition of these studies, using similar methodology and training, to see if the results are replicated. This should ideally be undertaken on case series from different origins, bothinterms ofsector(e.g.,policing) andinterms of country, population configuration, and lan- guage. It would be preferable if these were inde- pendent of the authors of the CTAP, given that there is some evidence of a researcher allegiance effect in studies conducted by the authors of risk assessment instruments (Singh et al., 2013). As fortheQCS,thereisevidentlyscopeforrepetition of inter-rater reliability exercises using lightly trained correspondence personnel (the skillset for which it is intended), rather than threat asses- sors,asthis wouldgive amore accurate indication of real-world conditions. Finally, it is likely that there are other approaches to the standardised assessment of concerning correspondence which have yet to be published. It would be a useful exercise if the performance of these could be measured against the CTAP, given the current absence of published instruments with which the CTAP could be compared.

Conclusion

The CTAP has been widely used in different settings over the last 7 years, and development versions for a longer period still. The CTAP appears to be a useful tool in the identification and initial prioritisation of concerning commu- nications, both in the public and private spheres. However, this is the first published study relating its content to the background literature on risk factors and evaluating the tool’s reliability. Its results are encouraging. It suggests that CTAP has a firm, up-to-date research base and that both the CTAP and the QCS can be applied reliably. The CTAP appears to be a useful addition to the

stock of instruments which encourage systematic and defensible threat assessment practice.

References

Agresti, A. (2010). Analysis of ordinal categorical data (2nd ed.). Wiley. https://doi.org/10.1002/ 9780470594001

Agresti, A. (2013). Categorical data analysis (3rd ed.). Wiley.

Akhtar, S., & Morrison, C. M. (2019). The prevalence and impact of online trolling of UK Members of Parliament. Computers in Human Behavior, 99, 322–327. https://doi.org/10.1016/j.chb.2019.05.015

Altman, D. G. (1999). Practical statistics for medical research. Chapman & Hall/CRC Press.

Barry-Walsh, J., James, D. V., & Mullen, P. E. (2020). Fixated Threat Assessment Centers: Preventing harm and facilitating care in public figure threat cases and those thought to be at risk of lone-actor grievance- fueled violence. CNS Spectrums, 25, 630–637. https://doi.org/10.1017/S1092852920000152

Bixler, B. S., Dunn, J., & Grundland, T. (2021). Operations of the Los Angeles Police Threat Man- agement Unit and Crisis Support Response Section. In J. R. Meloy & J. Hoffmann (Eds.), International handbook of threat assessment (2nd ed., pp. 545– 570). Oxford University Press.

Bjelland, H. F., & Bjørgo, T. (2014). Trusler og trus- selhendelser mot politikere: En spørreundersøkelse blant norske stortingsrepresentanter og regjerings- medlemmer [Threats and threat events against politi- cians: A survey of Norwegian members of parliament and government officials]. Politihøgskolen.

Bonta, J., Law, M., & Hanson, K. (1998). The prediction of criminal and violent recidivism among mentally disordered offenders: A meta-analysis. Psychological Bulletin, 123(2), 123–142. https://doi.org/10.1037/ 0033-2909.123.2.123

Brottsförebyggande rådet (Crime Prevention Council). (2012). Politikernas Trygghets undersökning 2012: Förtroendevaldas utsatthet och oro för hot, våld och trakasserier Rapport 2012:14 [Politicians: The secu- rity survey 2012: Elected officials vulnerability and concern for the threats, violence and harassment. Report 2012:14]. Fritzes.

Calhoun, F. S. (1998). Hunters and Howlers: Threats and violence against federal judicial officials in the United States, 1789–1993. Department of Justice, U.S. Marshals Service.

Calhoun, F. S., & Weston, S. W. (2003). Contemporary threat management. Specialized Training Services.

Calhoun, F. S., & Weston, S. W. (2009). Threat assessment and management strategies. CRC Press.

Carbone, J. T., Holzer, K. J., Vaughn, M. G., & DeLisi, M. (2020). Homicidal ideation and forensic psychopathology: Evidence from the 2016 Nation- wide Emergency Department Sample (NEDS).

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

18 JAMES ET AL.

Journal of Forensic Sciences, 65(1), 154–159. https://doi.org/10.1111/1556-4029.14156

Cawood, J. S., & Corcaran, M. H. (2003). Violence assessment and Intervention: The practitioner’s handbook. CRC Press. https://doi.org/10.1201/ 9780203500057

Cawood, J. S., Scalora, M. J., & Viñas-Racionero, R. (2020). The Cawood assessment grid for organisa- tional (workplace) violence: Initial testing fir inter- rater reliability and predictive validity. Journal of Threat Assessment and Management, 7, 200–213. https://doi.org/10.1037/tam0000153

Churcher, F. P., & Nesca, M. (2013). Risk factors for violence in stalking perpetration: A meta-analysis. FWU Journal of Social Sciences, 7, 100–112.

Clemmow, C., Gill, P., Corner, E., Farnham, F., Taylor, R., Wilson, S., Taylor, A., & James, D. (2021) A data-driven classification of outcome be- haviours in those who cause concern to British public figures. Psychology, Public Policy, and Law, 27(4), 509–521. https://doi.org/10.1037/la w0000269

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46. https://doi.org/10.1177/ 001316446002000104

Coid, J. W., Ullrich, S., Kallis, C., Keers, R., Barker, D., Cowden, F., & Stamps, R. (2013). The relation- ship between delusions and violence: Findings from the East London first episode psychosis study. JAMA Psychiatry, 70(5), 465–471. https://doi.org/ 10.1001/jamapsychiatry.2013.12

Cook, D. A., & Beckman, T. J. (2006). Current concepts in validity and reliability for psychometric instruments: Theory and application. The American Journal of Medicine, 119(2), 166.e7–166.e16. https:// doi.org/10.1016/j.amjmed.2005.10.036

de Becker, G. (1996). The gift of fear. Random House. De Groot, I. N. J., Drost, L. F., Boutellier, J. C. J., & Voerman, B. E. (2010). Bedreigers van politici: Risicos en interventiemogelijkheden [People who threaten politicians: Risks and intervention possi- bilities]. Verwey-Jonker Instituut.

DeLisi, M., Tahja, K., Drury, A. J., Caropreso, D., Elbert, M. J., & Heinrichs, T. (2017). The criminol- ogy of homicidal ideation: Associations with criminal careers and psychopathology among federal correc- tional clients. American Journal of Criminal Justice, 42, 554–573. https://doi.org/10.1007/s12103-016- 9371-5

Dietz, P., & Martell, D. A. (2010). Commentary: Approaching and stalking public figures—A pre- requisite to attack. The Journal of the American Academy of Psychiatry and the Law, 38(3), 341– 348. http://www.jaapl.org/content/38/3/341

Dietz, P., Matthews, D., Van Duyne, C., Martell, D., Parry, C., Stewart, T., van Duyne, C., & Crowder, J.

(1991) Threatening and otherwise inappropriate let- ters to Hollywood celebrities. Journal of Forensic Sciences, 36, 185–209. https://doi.org/10.1520/ JFS13019J. ISSN 0022-1198.

Dietz, P. E., & Martell, D. A. (1989). Mentally disor- dered offenders in pursuit of celebrities and poli- ticians. National Institute of Justice.

Dietz, P. E., Matthews, D. B., Martell, D. A., Stewart, T. M., Hrouda, D. R., & Warren, J. (1991). Threat- ening and otherwise inappropriate letters to members of the United States Congress. Journal of Forensic Sciences, 36(5), 1445–1468. https://doi.org/10.1520/ JFS13165J

Douglas, K. S., & Otto, R. K. (Eds.). (2020). Handbook of violence risk assessment (2nd ed.). Routledge. https://doi.org/10.4324/9781315518374

Dunn, J. (2014). The Los Angeles Police Department Threat Management Unit. In J. R. Meloy & J. Hoffmann (Eds.), International handbook of threat assessment (pp. 285–298). Oxford University Press.

Every-Palmer, S., Barry-Walsh, J., & Pathé, M. (2015). Harassment, stalking, threats and attacks targeting New Zealand politicians: A mental health issue. The Australian and New Zealand Journal of Psychiatry, 49, 634–641. https://doi.org/10.1177/ 0004867415583700

Farnham, F. R., & Busch, K. G. (2016). International perspectives on the protection of public officials. Behavioral Sciences & the Law, 34, 597–601. https://doi.org/10.1002/bsl.2266

Fein, R. A., & Vossekuil, B. (1998). Preventing attacks on public officials and public figures: A secret service perspective. In J. R. Meloy (Ed.), The psy- chology of stalking (pp. 175–191). Academic Press.

Fein, R. A., & Vossekuil, B. (1999). Assassination in the United States: An operational study of recent assassins, attackers, and near-lethal approachers. Journal of Forensic Sciences, 44(2), 321–333. https://doi.org/10.1520/JFS14457J

Fein, R. A., & Vossekuil, B. (2000). Protective intel- ligence and threat assessment investigations: A guide for state and local law enforcement officials. U.S. Dept. of Justice, Office of Justice Programs, National Institute of Justice.

Fixated Research Group. (2006). Report of the fixated research group. Home Office.

Gill, P., Corner, E., Farnham, F., Wilson, S., March- ment, Z., Taylor, A., Taylor, R., & James, D. (2021). Predictors of varying levels of risks posed by fixated individuals to British public figures. Journal of Forensic Sciences, 66(4), 1364–1376. https://doi.org/10.1111/1556-4029.14708

Hamers, H., Borgesius, E., & Verschure, S. (2010). Mensen met een psychische stoornis die publieke personen bedreigen: Een aanpak gericht op zorg [People with mental disorders who threaten public

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 19

figures: an approach focused on facilitating care]. Zorg Consult.

Hart, S., & Logan, C. (2011). Formulation of violence risk using evidence-based assessments: the structured professional judgement approach. In P. Sturmey & M. McMurran (Eds.), Forensic case formulation (1st ed., pp. 83–107). Wiley. https://doi.org/10.1002/ 9781119977018.ch4

Hehemann, K., van Nobelen, D., Brandt, C., & McEwan, T. E. (2017). The reliability and predictive valid- ity of the Screening Assessment for Stalking and Harassment (SASH). Journal of Threat Assessment and Management, 4(3), 164–177. https://doi.org/10 .1037/tam0000085

Hodgins, S., Hiscoke, U. L., & Freese, R. (2003). The antecedents of aggressive behavior among men with schizophrenia: A prospective investigation of pa- tients in community treatment. Behavioral Sciences and The Law, 21(4), 523–546. https://doi.org/10 .1002/bsl.540

Hoffman, J., & Sheridan, L. (2008). Stalking, threat- ening and attacking corporate figures. In J. R. Meloy, L. Sheridan, & J. Hoffman (Eds.), Stalking, threatening and attacking public figures: A psycho- logical and behavioral analysis (pp. 123–142). Oxford University Press. https://doi.org/10.1093/ med:psych/9780195326383.003.0006

IBM Corp. (2019). SPSS statistics, version 26.0. James, D. V. (2010). Protecting the prominent? A research journey with Paul Mullen. Criminal Behaviour and Mental Health, 20(3), 242–250. https://doi.org/10.1002/cbm.769

James, D. V. (2015). Another helping of “intellectual hash”: Commentary on “Assassins of Rulers” (Mac- Donald, 1912). Journal of Threat Assessment and Management, 1, 228–240. https://doi.org/10.1037/ tam0000027

James, D. V., & Farnham, F. R. (2016). Outcome and efficacy of interventions by a public figure threat assessment and management unit: A mirrored study of concerning behaviors and police contacts before and after intervention. Behavioral Sciences & the Law, 34, 660–680. https://doi.org/10.1002/bsl.2255

James, D. V., Farnham, F. R., Sukhwal, S., Jones, K., Carlisle, J., & Henley, S. (2016). Aggressive/intrusive behaviours, harassment and stalking of members of the United Kingdom parliament: A prevalence study and cross-national comparison. Journal of Forensic Psychiatry & Psychology, 27, 177–197. https://doi.org/10.1080/14789949.2015.1124908

James, D. V., Farnham, F. R., & Wilson, S. P. (2014). The Fixated Threat Assessment Centre: Implement- ing a joint policing and psychiatric approach to risk assessment and management in public figure threat cases. In J. R. Meloy & J. Hoffmann (Eds.), Interna- tional handbook of threat assessment (pp. 299–320). Oxford University Press.

James, D. V., Kerrigan, T., Forfar, R., Farnham, F., & Preston, L. (2010). The Fixated Threat Assessment Centre: Preventing harm and facilitating care. Journal of Forensic Psychiatry & Psychology, 21, 521–536. https://doi.org/10.1080/14789941003596981

James,D. V.,MacKenzie, R. D.,& Farnham,F.R. (2014). Communications Threat Assessment Protocol-25: Guidelines for the initial assessment of problem- atic communications. Theseus.

James, D. V., McEwan, T. E., MacKenzie, R. D., Meloy, J. R., Mullen, P. E., Pathé, M. T., Farnham, F. R., Preston, L. F., & Darnley, B. (2010). Persistence in stalking: A comparison of associa- tions in general forensic and public figure samples. Journal of Forensic Psychiatry & Psychology, 21, 283–305. https://doi.org/10.1080/1478994090 3388994

James, D. V., Meloy, J. R., Mullen, P. E., Pathé, M. T., Farnham, F. R., Preston, L. F., & Darnley, B. J. (2010). Abnormal attentions toward the British Royal family: Factors associated with approach and escalation. The Journal of the American Acad- emy of Psychiatry and the Law, 38, 329–340. http:// www.fixatedthreat.com/perch/resources/factors- associated-with-approach-and-escalation.pdf

James, D. V., Mullen, P. E., Meloy, J. R., Pathé, M. T., Farnham, F. R., Preston, L., & Darnley, B. (2007). The role of mental disorder in attacks on European politicians 1990–2004. Acta Psychiatrica Scandi- navica, 116, 334–344. https://doi.org/10.1111/j .1600-0447.2007.01077.x

James, D. V., Mullen, P. E., Meloy, J. R., Pathé, M. T., Preston, L., Darnley, B., Farnham, F. R., & Scalora, M. J. (2010). Stalkers and harassers of British royalty: An exploration of proxy behaviours for violence. Behavioral Sciences & the Law, 29(1), 64–80. https://doi.org/10.1002/bsl.922

James, D. V., Mullen, P. E., Pathé, M. T., Meloy, J. R., Farnham, F. R., Preston, L., & Darnley, B. (2008). Attacks on the British Royal family: The role of psychotic illness. The Journal of the American Academy of Psychiatry and the Law, 36, 59–67. https://fixatedthreat.com/perch/resources/attacks-on- the-british-royal-family.pdf

James, D. V., Mullen, P. E., Pathé, M. T., Meloy, J. R., Preston, L. F., Darnley, B., & Farnham, F. R. (2009). Stalkers and harassers of royalty: The role of mental illness and motivation. Psychological Medicine, 39(9), 1479–1490. https://doi.org/10 .1017/S0033291709005443

James, D. V., & Sheridan, L. P. (2020). What works in risk assessment in stalking cases? In J. S. Wormith, L. A. Craig, & T. E. Hogue (Eds.), The Wiley handbook of what works in violence risk management: Theory, research and practice (pp. 527–542). Wiley. https://doi.org/10.1002/97811193 15933.ch27

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

20 JAMES ET AL.

James, D. V., Sukhwal, S., Farnham, F. R., Evans, J., Barrie, C., Taylor, A., & Wilson, S. P. (2016). Harassment and stalking of members of the United Kingdom Parliament: Associations and consequences. Journal of Forensic Psychiatry & Psychology, 27, 309–330. https://doi.org/10.1080/14789949.2015 .1124909

Kendall, M. (1938). A new measure of rank correlation. Biometrika, 30(1–2), 81–89. https://doi.org/10.2307/ 2332226

Kirkbride, J. B., Errazuriz, A., Croudace, T. J., Morgan, C., Jackson, D., Boydell, J., Murray, R. M., & Jones, P. B. (2012). Incidence of schizophrenia and other psychoses in England, 1950–2009: A sys- tematic review and meta-analyses [Bottom of Form]. PLOS ONE, 7(3), Article e31660. https://doi.org/10 .1371/journal.pone.0031660

Kropp, P. R., Hart, S. D., & Lyon, D. R. (2008). Guidelines for Stalking Assessment and Management (SAM): User manual. ProActive ReSolutions Inc.

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174. https://doi.org/10.2307/2529310

Laschi, M., & Lombroso, C. (1886). Le délit politique [Political crime]. In Actes du Premier Congrès International d’Anthropologie Criminelle, Rome, 1885 (pp. 379–389). Bocca Frères.

Link, B. G., Stueve, A., & Phelan, J. (1998). Psychotic symptoms and violent behaviors: Probing the com- ponents of “threat/control-override” symptoms. Social Psychiatry and Psychiatric Epidemiology, 33(Suppl. 1), S55–S60. https://doi.org/10.1007/ s001270050210

MacKenzie, R. D., McEwan, T. E., Pathé, M. T., James, D. V., Ogloff, J. R. P., & Mullen, P. E. (2009). The stalking risk profile: Guidelines for the assessment and management of stalkers. Stalk Inc.

Marquez, A., & Scalora, M. J. (2011). Problematic approach of legislators: Differentiating stalking from isolated incidents. Criminal Justice and Behavior, 38(11), 1115–1126. https://doi.org/10.1177/00938548 11420847

McElhaney, M. (2004) Aggression in the workplace: Preventing and managing high-risk behaviour. Author House.

McEwan, T. E., MacKenzie, R. D., Mullen, P. E., & James, D. V. (2012). Approach and escalation in stalking. Journal of Forensic Psychiatry & Psychology, 23(3), 392–409. https://doi.org/10.1080/14789949 .2012.679008

McEwan, T. E., Mullen, P. E., & MacKenzie, R. (2009). A study of the predictors of persistence in stalking situations. Law and Human Behavior, 33(2), 149–158. https://doi.org/10.1007/s10979-008- 9141-0

McEwan, T. E., Mullen, P. E., & MacKenzie, R. (2010). Suicide amongst stalkers. Journal of

Forensic Psychiatry & Psychology, 2(4), 514–520. https://doi.org/10.1080/14789940903564370

McEwan, T. E., Shea, D. E., Daffern, M., MacKenzie, R. D., Ogloff, J. R. P., & Mullen, P. E. (2018). The reliability and predictive validity of the stalking risk profile. Assessment, 25(2), 259–276. https://doi.org/ 10.1177/1073191116653470

McEwan, T. E., Strand, S. M., MacKenzie, R. D., & James, D. V. (2017). Screening Assessment for Stalking and Harassment (SASH): Guidelines for application and interpretation. Stalk Inc.

McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coeffi- cients. Psychological Methods, 1, 30–46. https:// doi.org/10.1037/1082-989X.1.1.30

Meloy, J. R. (2014). Approaching and attacking public figures: A contemporary analysis of communica- tions and behavior. Journal of Threat Assessment and Management, 1(4), 243–261. https://doi.org/10 .1037/tam0000024

Meloy, J. R. (2016) TRAP-18 (Terrorist Radicali- zation Assessment Protocol-18). User manual (version 1.0). Global Institute of Forensic Research, Inc.

Meloy, J. R. (2018). The operational development and empirical testing of the Terrorist Radicalization Assessment Protocol (TRAP-18). Journal of Per- sonality Assessment, 100(5), 483–492. https:// doi.org/10.1080/00223891.2018.1481077

Meloy, J. R., Amman, M., & Hoffman, J. (2021). Public figure stalking and attacks. In J. R. Meloy & J. Hoffmann (Eds.), International handbook of threat assessment (2nd ed., pp. 166–188). Oxford University Press. https://doi.org/10.1093/med- psych/9780190940164.003.0009

Meloy, J. R., & Gill, P. (2016). The lone-actor terrorist and the TRAP-18. Journal of Threat Assessment and Management, 3(1), 37–52. https://doi.org/10 .1037/tam0000061

Meloy, J. R., & Hoffmann, J. (Eds.). (2014). Inter- national handbook of threat assessment. Oxford University Press.

Meloy, J. R., & Hoffmann, J. (Eds.). (2021). Inter- national handbook of threat assessment (2nd ed.). Oxford University Press. https://doi.org/10.1093/ med-psych/9780190940164.001.0001

Meloy, J. R., Hoffmann, J., Deisinger, E. R. D., & Hart, S. (2021). Threat assessment and threat man- agement. In J. R. Meloy & J. Hoffmann (Eds.), International handbook of threat assessment (2nd ed., pp. 3–21). Oxford University Press. https://doi.org/10 .1093/med-psych/9780190940164.003.0001

Meloy, J. R., Hoffmann, J., Guldimann, A., & James, D. (2012). The role of warning behaviors in threat assessment: An exploration and suggested typology. Behavioral Sciences & the Law, 30, 256–279. https:// doi.org/10.1002/bsl.999

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 21

Meloy, J. R., James, D. V., Farnham, F. R., Mullen, P. E., Pathé, M., Darnley, B., & Preston, L. (2004). A research review of public figure threats, approaches, attacks, and assassinations in the United States. Journal of Forensic Sciences, 49(5), 1086–1093. https://doi.org/10.1520/JFS2004102

Meloy, J. R., James, D. V., Mullen, P. E., Pathé, M. T., Farnham, F. R., Preston, L. F., & Darnley, B. J. (2011). Factors associated with escalation and prob- lematic approaches toward public figures. Journal of Forensic Sciences, 56(Suppl. 1), S128–S135. https://doi.org/10.1111/j.1556-4029.2010.01574.x

Meloy, J. R., Mohandie, K., & Green, M. (2008). A forensic investigation of those who stalk celebrities. In J. R. Meloy, L. Sheridan, & J. Hoffman (Eds.), Stalking, threatening and attacking public figures (pp. 37–54). Oxford University Press. https://doi.org/ 10.1093/med:psych/9780195326383.003.0002

Meloy, J. R., & O’Toole, M. E. (2011). The concept of leakage in threat assessment. Behavioral Sciences & the Law, 29(4), 513–527. https://doi.org/10.1002/ bsl.986

Meloy, J. R., Sheridan, L., & Hoffman, J. (Eds.). (2008a). Stalking, threatening and attacking public figures: A psychological and behavioral analysis. Oxford University Press. https://doi.org/10.1093/ med:psych/9780195326383.001.0001

Meloy, J. R., Sheridan, L., & Hoffman, J. (2008b). Public figure stalking, threats and attacks: the state of the science. In J. R. Meloy, L. Sheridan, & J. Hoffman (Eds.), Stalking, threatening and attacking public figures (pp. 3–34). Oxford University Press. https://doi.org/10.1093/med:psych/9780195326383 .003.0001

Meloy, J. R., White, S. G., & Hart, S. (2013). Workplace assessment of targeted violence risk: The development and reliability of the WAVR-21. Journal of Forensic Sciences, 58(5), 1353–1358. https://doi.org/10.1111/1556-4029.12196

Mohandie, K., Meloy, J. R., McGowan, M. G., & Williams, J. (2006). The RECON typology of stalking: Reliability and validity based upon a large sample of North American stalkers. Journal of Forensic Sciences, 51, 147–155. https://doi.org/10.1111/j.1556- 4029.2005.00030.x

Monahan, J., Steadman, H., Silver, E., Appelbaum, P., Robbins, P., Mulvey, E., Roth, L., Grisso, T., & Banks, S. (2001). Rethinking risk assessment: The MacArthur study of mental disorder and violence. Oxford University Press.

Monahan, J., Steadman, H. J., Appelbaum, P. S., Grisso, T., Mulvey, E. P., Roth, L. H., Robbins, P. C., Banks, S., & Silver, E. (2006). The classifica- tion of violence risk. Behavioral Sciences & the Law, 24(6), 721–730. https://doi.org/10.1002/bsl.725

Mullen, P., James, D., Meloy, J. R., Pathé, M. T., Farnham, F. R., Preston, L., Darnley, B., & Berman,

J. (2009). The fixated and the pursuit of public figures. Journal of Forensic Psychiatry & Psychology, 20, 33–47. https://doi.org/10.1080/14789940802197074

Mullen, P. E., James, D. V., Meloy, J. R., Pathé, M., Farnham, F., Preston, L., & Darnley, B. (2008). The role of psychotic illnesses in attacks on public figures. In J. R. Meloy, L. Sheridan, & J. Hoffman (Eds.), Stalking, threatening and attacking public figures: A psychological and behavioral analysis (pp. 55–82). Oxford University Press. https://doi.org/ 10.1093/med:psych/9780195326383.003.0003

Mullen, P. E., & Lester, G. (2006). Vexatious litigants and unusually persistent complainants and peti- tioners: From querulous paranoia to querulous behaviour. Behavioral Sciences & the Law, 24(3), 333–349. https://doi.org/10.1002/bsl.671

Mullen, P. E., & Maack, L. H. (1985). Jealousy, pathological jealousy and aggression. In D. P. Farrington & J. Gunn (Eds.), Aggression and dan- gerousness (pp. 103–126). Wiley.

Muller, E. R., Bron, R. P., de Hoog, D., & van der Varst, L. P. (2010). De aanpak van verwarde bedreigers in Nederland [The approach to threats by people with mental health disorders in The Netherlands]. COT Instituut voor Veiligheids-en Crisismanagement [COT Institute for Safety and Crisis Management].

Narud, K., & Dahl, A. V. (2015). Stalking experiences reported by Norwegian members of parliament compared to a general population sample. Journal of Forensic Psychiatry & Psychology, 26(1), 116– 131. https://doi.org/10.1080/14789949.2014.981564

Nelen, H., Leeuw, B., Bakker, F., & Herrenberg, T. (2012). Pilot Dreigingsmanagement: Een ex ante evaluatie [Pilot threat management: an ex ante evaluation]. WODC, Ministerie van Veiligheid en Justitie [Research and Documentation Centre, Ministry of Justice].

Nelen, H., Mol, M., Plaisier, J., & Peters, M. (2013). Pilot dreigingsmanagement. De implementatie en wijze van uitvoering onder de loep [Threat man- agement pilot. Implementation and performance under the microscope]. WODC, Ministerie van Veiligheid en Justitie [Research and Documentation Centre, Ministry of Justice].

Nijdam, M. J, Olff, M., de Vries, M., Martens, W. J., & Gersons, B. P. R. (2008). Psychosociale effecten van dreigingen beveiliging. [Psycho-social effects of threats and security]. NCTb [National Coordinator for Counter-Terrorism].

Novaco, R. W. (2011). Anger dysregulation: Driver of violent offending. Journal of Forensic Psychiatry & Psychology, 22(5), 650–668. https://doi.org/10.1080/ 14789949.2011.617536

Novaco, R. W. (2013). Reducing anger-related offending: What works. In L. A. Craig, L. Dixon, & T. A. Gannon (Eds.), What works in offender rehabilitation: An

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

22 JAMES ET AL.

evidence-based approach to assessment and treatment (pp. 211–236). Wiley Blackwell. https://doi.org/10 .1002/9781118320655.ch12

Pathé, M. (2017). Stalking public figures: The fixated loner. In W. S. G. Petherick (Ed.), The psychology of criminal and antisocial behavior (pp. 295–320). Academic Press. https://doi.org/10.1016/B978-0- 12-809287-3.00009-2

Pathé, M., Haworth, D., & Lowry, T. (2016). Managing the risk posed by fixated persons at major events: A joint police-mental health intelligence approach. Journal of Policing, Intelligence and Counter Terror- ism, 11(1), 63–72. https://doi.org/10.1080/18335330 .2016.1161226

Pathé, M. T., Haworth, D. J., Goodwin, T., Holman, A. G., Amos, S. J., & Day, L. (2017). Establishing a joint agency response to the threat of lone-actor grievance-fuelled violence. Journal of Forensic Psychiatry & Psychology, 29(1), 37–52. https:// doi.org/10.1080/14789949.2017.1335762

Pathé, M. T., Lowry, T. J., Haworth, D. J., Webster, D., Winterbourne, P., & Briggs, C. J. (2015). Assessing and managing the threat posed by fixated persons in Australia. Journal of Forensic Psychiatry & Psychology, 26(4), 425–438. https://doi.org/10.1080/ 14789949.2015.1037332

Pathé, M. T., Lowry, T. J., Haworth, D. J., Winter- bourne, P., & Day, L. (2016). Public figure fixation: Cautionary findings for mental health practitioners. Behavioral Sciences & the Law, 34(5), 681–692. https://doi.org/10.1002/bsl.2252

Régis, E. (1890). Les régicides dans l’histoire et dans le présent [‘Regicides’ in history and the present day]. Maloine.

Riddle, F. J., Ferriman, M., Farmer, D. G., & Baillie, K. M. (2019). Towards a national strategy for managing fixated persons in Australia. The Journal of Psychiatry & Law, 26(3), 457–467. https:// doi.org/10.1080/13218719.2018.1506722

Scalora, M. J., Baumgartner, J. V., Hatch-Maillette, M. A., Covell, C. N., Palarea, R. E., Krebs, J. A., Washington, D. O., Zimmerman, W., & Callaway, D. (2002). Risk factors for approach behavior toward the U.S. congress. Journal of Threat Assessment, 2, 35–55. https://doi.org/10.1300/J177v02n02_03

Scalora, M. J., Baumgartner, J. V., & Plank, G. L. (2003). The relationship of mental illness to targeted contact behavior toward state government agencies and officials. Behavioral Sciences & the Law, 21, 239–249. https://doi.org/10.1002/bsl.525

Scalora, M. J., Baumgartner, J. V., Zimmerman, W., Callaway, D., Hatch Maillette, M. A., Covell, C. N., Palarea, R. E., Krebs, J. A., & Washington, D. O. (2002). An epidemiological assessment of prob- lematic contacts to members of congress. Journal of Forensic Sciences, 47, 1360–1364. https://doi.org/ 10.1520/JFS15575J

Scalora, M. J., Zimmerman, W. J., & Wells, D. G. (2008). Use of threat assessment for the protection of the United States Congress. In J. R. Meloy, L. Sheridan, & J. Hoffman (Eds.), Stalking, threat- ening and attacking public figures: A psychological and behavioral analysis (pp. 425–434). Oxford University Press. https://doi.org/10.1093/med: psych/9780195326383.003.0019

Schlesinger, L. B. (2006). Celebrity stalking, homi- cide, and suicide: A psychological autopsy. Inter- national Journal of Offender Therapy and Comparative Criminology, 50(1), 39–46. https:// doi.org/10.1177/0306624X05276461

Schoeneman, K. A., Scalora, M. J., Darrow, C. D., McLawsen, J. E., Chang, G. H., & Zimmerman, W. J. (2011). Written content indicators of prob- lematic approach behavior toward political officials. Behavioral Sciences & the Law, 29, 284–301. https://doi.org/10.1002/bsl.977

Schoeneman-Morris, K. A., Scalora, M. J., Chang, G. H., Zimmerman, W. J., & Garner, Y. (2007). A comparison of email versus letter threat contacts toward members of the United States Congress. Journal of Forensic Sciences, 52, 1142–1147. https://doi.org/10.1111/j.1556-4029.2007.00538.x

Shaffer, C. S., Fuller, E. K., & Guy, L. S. (2020). Brief and emerging violence risk assessment measures. In K. S. Douglas & R. K. Otto (Eds.), Handbook of violence risk assessment (2nd ed., pp. 52–69). Routledge. https://doi.org/10.4324/ 9781315518374-4

Shrout, P. E. (1998). Measurement reliability and agreement in psychiatry. Statistical Methods in Medical Research, 7, 301–317. https://doi.org/10 .1177/096228029800700306

Silva, A. J., Ferrari, M. M., Leong, G. B., & Penny, G. (1998). The dangerousness of persons with delusional jealousy. The Journal of the American Academy of Psychiatry and the Law, 26(4), 607–623.

Silva, J. A., Harry, B. E., Leong, G. B., & Weinstock, R. (1996). Dangerous delusional misidentification and homicide. Journal of Forensic Sciences, 41(4), 641–644.

Singh, J. P., Desmarais, S. L., Hurducas, C., Arbach- Lucioni, K., Condemarin, C., Dean, K., Doyle, M., Folino, J. O., Godoy-Cervera, V., Grann, M., Ho, R. M. Y., Large, M. M., Nielsen, L. H., Pham, T. H., Rebocho, M. F., Reeves, K. A., Rettenberger, M., de Ruiter, C., Seewald, K., & Otto, R. K. (2014). International perspectives on the practical application of violence risk assessment: A global survey of 44 countries. International Journal of Forensic Mental Health, 13, 193–206. https://doi.org/10.1080/ 14999013.2014.922141

Singh, J. P., Grann, M., & Fazel, S. (2013). Authorship bias in violence risk assessment? A systematic review

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

CTAP: DEVELOPMENT AND INTER-RATER RELIABILITY 23

and meta-analysis. PLOS ONE, 8, Article e72484. https://doi.org/10.1371/journal.pone.0072484

Sizoo, B. B., & van Nobelen, D. (2021). Effectiveness of interventions by the Threat Management Team of the Netherlands Police Agency: A retrospective study. Journal of Threat Assessment and Management, 8(1–2), 36–47. https://doi.org/10.1037/tam0000159

Skeem, J., & Monahan, J. (2011). Current directions in violence risk assessment. Current Directions in Psychological Science, 20(1), 38–42. https://doi.org/ 10.1177/0963721410397271

Slovic, P., Finucane, M. L., Peters, E., & MacGregor, D. G. (2004). Risk as analysis and risk as feelings: Some thoughts about affect, reason, risk, and ratio- nality. Risk Analysis: An Official Publication of the Society for Risk Analysis, 24(2), 311–322. https:// doi.org/10.1111/j.0272-4332.2004.00433.x

Swanson, J. W., Swartz, M. S., Van Dorn, R. A., Elbogen, E. B., Wagner, H. R., Rosenheck, R. A., Stroup, T. S., McEvoy, J. P., & Lieberman, J. A. (2006). A national studyof violent behavior inpersons withschizophrenia. Archives of General Psychiatry, 63(5), 490–499. https://doi.org/10.1001/archpsyc.63.5.490

Wallin, L., & Wallin, S. (2014). Politikernas trygghet- sundersökning 2013: Utsatthet och oro för trakas- serier, hot och våld [Politicians security survey 2013: Vulnerability and concern—Harassment, threats and violence]. National Council for Crime Prevention.

White, S. G., & Meloy, J. R. (2016). The WAVR-21: A structured professional guide for the workplace assessment of violence risk (3rd ed.). Specialised Training Services.

Wilson, S., Dempsey, C., Farnham, F., Manze, T., & Taylor, T. (2018). Stalking risks to celebrities and public figures. British Journal of Psychiatric Advances, 24, 152–160. https://doi.org/10.1192/bja.2017.22

Wilson, S., Farnham, F., Taylor, A., & Taylor, R. (2019). Reflections on working in public-figure threat management. Medicine, Science and the Law, 59(4), 275–281. https://doi.org/10.1177/ 0025802419861168

Wilson, S. P., Pathé, M. T., Farnham, F. R., & James, D. V. (2021). The fixated threat assessment centers. In J. R. Meloy & J. Hoffmann (Eds.), International handbook of threat assessment (2nd ed., pp. 475–491). Oxford University Press.

Wormith, J. S., Craig, L. A., & Hogue, T. E. (Eds.). (2020). The Wiley handbook of what works in violence risk management: Theory, research and practice. Wiley. https://doi.org/10.1002/ 9781119315933

Received June 22, 2021 Revision received November 30, 2021

Accepted December 12, 2021 ▪

T h is d oc u m en t is co p y ri g h te d b y th e A m er ic an

P sy ch o lo g ic al

A ss o ci at io n o r o n e o f it s al li ed

p u b li sh er s.

T h is ar ti cl e is in te nd ed

so le ly

fo r th e p er so na l u se

o f th e in d iv id u al

u se r an d is n o t to

b e d is se m in at ed

b ro ad ly .

24 JAMES ET AL.