Competencies

Critical Review of Dual Diagnosis Training for Mental Health Professionals

Pernille Pinderup1 & Birgitte Thylstrup2 & Morten Hesse2

Published online: 19 May 2016 # Springer Science+Business Media New York 2016

Abstract To review evidence on the effects of training programs in dual diagnosis treatment for mental health professionals. Three databases were searched. Included studies were evalu- ated by an adapted version of Kirkpatrick’s Training Evaluation Model, which evaluates participant perception of training, the effect on professional competencies, transfer of training, and the effect on the patients. Overall findings from the eleven included studies suggested that participants valued the training, increased some professional competencies, and that some transfer of training occurred. The effect at the patient level showed mixed results. Training mental health professionals in dual diagnosis treatment may have a positive effect on profes- sional competencies and clinical practice. Any conclusion regarding the overall training effect is premature due to limitations in study designs. Future studies on the effects of dual diagnosis training programs for mental health professionals should involve control groups, validated measures, follow-ups, and patient outcomes.

Keywords Dual diagnosis . Training . Comorbidity. Mental illness . Substance use disorder

The term dual diagnosis (DD) describes the coexistence of one or more mental illnesses (MI) and substance use disorders (SUD) (Todd et al. 2004). DD is often associated with early onset, beginning in youth, and a chronic course (Di Lorenzo et al. 2014), and is associated with higher rates of relapse, poorer compliance to treatment, and more psychiatric symptoms, compared to MI (Archie and Gyomorey 2009; Zammit et al. 2008). Furthermore, DD is associated with higher risk of re-hospitalizations (Archie and Gyomorey 2009; Haywood et al. 1995; Schmidt et al. 2011), increased suicide risk (Soyka et al. 2001), violence/delinquency

Int J Ment Health Addiction (2016) 14:856–872 DOI 10.1007/s11469-016-9665-3

* Pernille Pinderup [email protected]

1 Competence Centre for Dual Diagnosis, Mental Health Centre Sct. Hans, Mental Health Services – Capital Region of Denmark, Boserupvej 2, 4000 Roskilde, Denmark

2 Centre for Alcohol and Drug Research, Aarhus University, Artillerivej 90, 2nd Floor, 2300 Copenhagen, Denmark

(Soyka 2000), incarceration (McNiel et al. 2005), unemployment (Laudet et al. 2002), homelessness (Olfson et al. 1999), and greater risk for infections such as HIV and hepatitis (Rosenberg et al. 2001).

DD occurs at rates that exceed chance by far. For instance, the Epidemiological Catchment Area (ECA) study found that the rate of lifetime SUD in general population was 17% compared to 48% for patients with schizophrenia and 56% for patients with bipolar disorder (Regier et al. 1990). Also, the National Comorbidity Survey (NCS) from 1996 found that 41.0–65.5% of individuals with a lifetime SUD suffers from at least one MI, and that 50.9% of individuals with an MI have at least one SUD (Kessler et al. 1996). In Denmark, a recent study found that the prevalence of any lifetime SUD among patients with MI to be 37% for schizophrenia, 35% for schizotypal disorder, 28% for other psychoses, 32% for bipolar disorder, 25% for depression, 25% for anxiety, 11% for OCD, 17% for PTSD, and 46% personality disorders (Toftdahl et al. 2015). Failure to identify and treat cases of DD has severe consequences for both the patient and society. Despite the evidence for using an integrated treatment approach, where both the MI and SUD are treated as primary disorders (Drake et al. 2004; Mangrum et al. 2006; Mueser et al. 2003), many patients often receive treatment for one disorder only (Drake and Mueser 2000). One of the reasons is that SUD can be difficult to differentiate from MI symptoms because of the acute or chronic effects (Hansen et al. 2000). Another reason is that mental health professionals often lack appropriate clinical competencies to detect and treat DD (Barry et al. 2002; Cleary et al. 2009; Griffin et al. 2008; Morojele et al. 2012), which has also been associated with negative attitudes towards the patient group (Adams 2008; Richmond and Foster 2003).

One way to overcome these difficulties is to offer mental health professionals training in DD and DD treatment. In this context, training can be understood as a planned and systematic effort to modify or develop knowledge, skills, and attitudes through learning experience, in order to achieve effective performance (Buckley and Caple 2007). Renner (2004) suggests that training in DD should focus on enhancing professionals’ knowledge of MI and SUD, improve the clinical skills, and reduce counterproductive attitudes, and studies have highlighted the necessity of offering training in DD treatment to mental health professionals (Grella 2003; O’Gara et al. 2005; Ralley et al. 2009). Furthermore, mental health professionals also frequently request DD training (Happell et al. 2002; Howard and Holmshaw 2010; Ryrie and McGowan 1998; Schulte et al. 2010; Siegfried et al. 1999) that involves identifi- cation of substance withdrawal, management of alcohol and substance detoxification (Happell et al. 2002), interactions between alcohol, drugs, and prescribed medications (Ryrie and McGowan 1998), and therapeutic techniques (Howard and Holmshaw 2010). Accordingly, the British National Institute for Health and Care Excellence recommends that mental health professionals should receive continuous training in DD and DD treatment in the clinical guidelines on psychosis and co-existing substance use (NICE 2011).

Despite these recommendations, a critical review of the effect of DD training programs has, to our knowledge, not yet been conducted (Schulte et al. 2010; Siegfried et al. 1999). Such critical review is important for several reasons. Firstly, training programs for mental health professionals on SUD and co-morbidity issues have been criticized for being patchy and inadequate in a number of countries (Munro et al. 2007), and there is therefore a need to develop DD training programs that are more suitable than the current training programs. The present review will provide an overview of the research on DD training programs, which could help improving current and future DD training programs. Secondly, there is a need to explore

Int J Ment Health Addiction (2016) 14:856–872 857

which effects DD training programs have on mental health professionals’ competencies, their clinical practice, and their patients. If time and resources invested in training mental health professionals do not have an effect on these outcomes, it might be more useful to improve DD treatment in other ways.

The primary aim of this paper is to conduct a critical review of the literature on DD training programs for mental health professionals. A secondary aim is to examine whether the training programs result in professional competencies that could be put into practice and improve patient outcomes, such as diminishing psychiatric symptoms and substance use.

Conceptual framework for the analysis

One of the most widely used models to evaluate training programs is Kirkpatrick’s Training Evaluation Model (O’Neill et al. 2004). The model specifies four levels of evaluation criteria that measure separate but related impacts of training (Kirkpatrick 1998), and it has been adapted slightly for this review (see Fig. 1). The first level, reaction, describes the participants’ perception of the DD training program. This level is fundamental, since participants might not be motivated to learn if they do not react favorably to the content and delivery of the training program (O’Neill et al. 2004). The second level, learning, refers to the participants’ improve- ments in professional knowledge, skills, and abilities. We have added attitudes to this level because attitudes toward patients with DD play a central role for quality of treatment. The third level, behavior, refers to transfer of training in the form of changes in professional work practices and behaviors. The fourth level, results, refers to organizational outcomes, which in this context is operationalized as improvements in patients’ psychiatric symptoms and sub- stance use. It is recommended that an evaluation strategy should start at the first level and move up through the levels in sequence (Kirkpatrick 1998; O’Neill et al. 2004).

4.

Evaluation of results (changes in patient outcomes)

3.

Evaluation of behavior (changes to work practices and

behaviors)

2.

Evaluation of learning (improvements in knowledge, skills and

attitudes)

1.

Evaluation of reaction

(perception of the training)

Fig. 1 The adapted version of Kirkpatrick’s training evaluation model (Kirkpatrick 1998)

858 Int J Ment Health Addiction (2016) 14:856–872

Methods

The review was conducted in accordance with relevant items from the PRISMA (The Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines, which consists of a 27-item checklist and a four-phase flow diagram (Moher et al. 2009).

Literature search

The electronic databases Web of Science, PsycINFO, and SCOPUS were searched using a pre- defined search string (see Appendix A for the full search string). Preliminary searches were conducted in all three databases prior to defining the search string. The search string included terms related to 1) SUD, 2) MI, 3) training outcomes, 4) training programs, and 5) DD, in order to ensure that records contained at least one search term from each category. The search terms had to occur in the title, abstract, keywords, or identifiers. In order to increase the number of relevant records, a proximity search was added to the string, such that records containing ‘professionals’ training’ was obtained while records containing ‘physical training’ were eliminated. A range of possible training outcomes (attitude, perception, knowledge, stigma, satisfaction etc.) was included in the string to ensure identifying relevant records.

The search was restricted to English language studies, published between January 1990 and September 15, 2014. The year 1990 was chosen because the DD concept was established in the 1980s (Drake et al. 1996). Additionally, searching in subject-related journals, bibliogra- phies, and citation records, as well as searching Google Scholar, were used to identify relevant studies.

Selection of studies

The review only included studies that examined training programs on DD for mental health professionals. Since different target groups require different training programs that might not be comparable, we focused exclusively on studies of training programs for professionals working in mental health settings, and not training programs for professionals working in substance use settings, students, patients with DD, or caregivers. Finally, this review was restricted to training programs targeting adult patients, as the assessment and treatment of children and adolescents might require other competencies compared to those needed for adults.

Studies that fulfilled the following criteria were included in the review: 1) Involving a training intervention on treatment of both MI and SUD in adult patients; 2) involving professionals from mental health settings; 3) examining training effects such as changes in professional competencies of mental health professionals.

Results

The original search retrieved 988 database and 14 non-database records (see Fig. 2). After duplicates were removed, 767 records remained for initial screening. Of these, 731 records were excluded because their titles or abstracts indicated that the studies did not meet the inclusion criteria. The remaining records (N = 36) were examined in full-text to assess whether the inclusion criteria were met, and whether they addressed a minimum of one of the research

Int J Ment Health Addiction (2016) 14:856–872 859

questions. Studies were included only when there was agreement between all three authors. In total, 16 records originating from 11 studies were finally included in the review (see Table 1). The excluded full-text articles (N = 20) and the reasons for their exclusion are listed in Appendix B.

Data from the included papers were extracted and analyzed within the four levels of the adapted version of Kirkpatrick’s Training Evaluation Model.

Characteristics of the included studies

The included studies differ in terms of geographical origin, study design, and training interventions (see Table 1). The studies were conducted in Australia, England, Ireland, Scotland, and in the US, and half of them used a repeated measure design. The remaining studies were randomized controlled trials (RCT), longitudinal studies, mixed methodology studies, or evaluation studies. Three studies used control groups. The training interventions ranged from 3 hours of training to 12 training days distributed over 10 months. In all studies, professionals conducted the training, but one study also involved lectures by patients.

Records identified through database searching (n = 988)

(PsycInfo = 221, SCOPUS = 473, Web of Science = 294)

Additional records identified through other sources (n = 14)

Records after duplicates removed (n = 767)

Records screened (n = 767)

Records excluded (n = 731)

Full-text articles assessed for eligibility (n = 36)

Full-text articles excluded for not meeting the inclusion

criteria (n = 20)

Studies included in the review (n = 11)

Articles meeting the inclusion criteria

(n = 16),

these originated from 11 studies

Fig. 2 Four-phase flow diagram

860 Int J Ment Health Addiction (2016) 14:856–872

T ab

le 1

S tu dy

ch ar ac te ri st ic s

In cl ud ed

st ud ie s

O ri gi n

P ar ti ci pa nt s

S tu dy

de si g n

C on tr o l

g ro u p

M et h od s

L en g th

o f tr ai ni n g p ro g ra m

C ra ig

et al . (2 0 0 8 ); H u g h es

et al .

(2 0 0 8 a) ; Jo hn so n et al . (2 0 0 7 )

E n g la n d

P ro fe ss io n al s (N

= 7 9 )

P at ie n ts (N

= 2 3 2)

R C T

x D D A * , K A D D * , A A P P Q * * P at ie nt s: A U D IT ,

M A P, B P R S , D A L I

F iv e tr ai n in g d ay s, 1 8 h o f su pe rv is io n (n o

tr ai n in g /s u p er v is io n to

co n tr o l g ro u p )

C lu tt er b u ck

et al . (2 0 0 9 ); C o pe ll o

et al . (2 0 12 ); G ra ha m

(2 0 04 );

G ra ha m

et al . (2 0 06 )

E n g la n d

P ro fe ss io n al s (N

= 6 9 )

P at ie n ts (N

= 5 8 )

L o n g it u d in al

x C o n fi d en ce

an d sk il ls q u es ti o n n ai re * , ra ti n g

sc al es

on C -B IT

sk il ls *,

in te rv ie w s P at ie nt s:

B P R S , S A T S , qu es ti o n n ai re

o n su b st an ce

re la te d be li ef s* *,

d at a o n su b st an ce s

S ix

h al f- da ys

tr ai ni ng

+ su pe rv is io n (t hr ee

te am

s tr ai ne d im

m ed ia te ly , tw o te am

s tr ai n ed

18 m o n th s la te r)

C oo p er

et al . (2 0 0 6 )

E n g la n d

P ro fe ss io n al s (N

= 1 5 1 )

R ep ea te d m ea su re

R at in g sc al e o n p er ce p ti o n o f tr ai ni n g * ,s ki ll s an d

co nf id en ce

sc al e*

P ar ti ci pa nt s re ce iv ed

ei th er

fi ve

or 12

tr ai ni ng

da ys

H es lo p et al . (2 0 1 3 )

A u st ra li a

P ro fe ss io n al s (N

= 18 )

R ep ea te d m ea su re

K A C S Q ** , D A A K Q ** , m ed ic al re co rd s

T hr ee

ho ur s of

tr ai ni ng , on e ho ur

m on th ly

ed u ca ti o n u p d at es

H u g h es

et al . (2 00 8 b )

E n g la n d

P ro fe ss io n al s (N

n o t

re po rt ed )

R ep ea te d m ea su re

Q ue st io nn ai re

on co m pe te nc ie s*

F iv e tr ai ni ng

d ay s

M un ro

et al . (2 0 0 7 )

S co tl an d

P ro fe ss io n al s (N

= 4 9 )

R C T

x K n ow

le d g e te st * , C M P P Q , in te rv ie w s

F ou r tr ai ni ng

da ys

(n o tr ai ni ng

to co nt ro lg ro up )

N aj av it s an d K an u k o ll u (2 00 5 )

U S

P ro fe ss io n al s (N

= 30 )

R ep ea te d m ea su re

C S T *

T hr ee

tr ai ni ng

da ys ,m

on th ly on -s ite

co ns ul ta tio ns , w ee kl y te le ph on e co nf er en ce s

R an i an d B y rn e (2 0 1 2)

Ir el an d

P ro fe ss io n al s (N

= 2 0 )

M ix ed

m et h o d s

Q u es ti o nn ai re

o n p er ce p ti o n of

tr ai n in g an d

co m p et en ci es *,

d ai ly

ev al u at io n s, fo cu s

g ro u p in te rv ie w s

F iv e tr ai n in g d ay s

S ax to n et al . (2 0 11 )

E n g la n d

P ro fe ss io n al s (N

= 1 5 )

R ep ea te d m ea su re

Q u es ti o nn ai re s o n k n o w le d g e, at ti tu d es

an d

tr ai ni n g qu al it y *

F iv e tr ai n in g d ay s

S ci ac ca

an d T h o m p so n (1 9 9 6)

U S

P ro fe ss io n al s (N

= 1 5 )

E v al u a- ti o n st u d y

M u lt ip le -c h oi ce

ex am

*, q u es ti o n n ai re

on k n o w le d g e*

1 2 d ay s tr ai n in g d ay s, in d iv id ua l su pe rv is io n

T o b in

an d B o u lt o n (2 0 09 )

E n g la n d

P ro fe ss io n al s (N

= 1 2 2 )

E v al u a- ti o n st u d y

Q u es ti o nn ai re

o n p er ce p ti o n of

tr ai n in g *

O n e tr ai n in g d ay

O n ly

m et ho d s re la te d to

th e o ut co m es

re p or te d in

th is re vi ew

ar e sh ow

n

* Q ue st io nn ai re

de ve lo pe d fo r th e st ud y ; **

R ef in ed /a da pt ed

ve rs io n o f th e or ig in al qu es ti o nn ai re

A A P P Q A lc o ho l an d A lc oh o l P ro bl em

s P er ce p ti on

Q ue st io nn ai re , A U D IT

A lc oh ol U se

D is or de rs Id en ti fi ca ti on

T es t, B P R S B ri ef ps yc hi at ri c R at in g S ca le , C -B IT

co gn it iv e- b eh av io ra l

in te gr at ed

tr ea tm

en t, C M P P Q

C o- m o rb id it y p ro bl em

s pe rc ep ti on s qu es ti on na ir e, C ST

T he

C li ni ci an

S el f- T es t, D A A K Q

T he

D ru g an d A lc oh ol

K no w le d ge

Q u iz , D A L I D ar th m ou th

A ss es sm

en t o f L if es ty le In st ru m en t, D D A D ua l D ia gn os is A tt it ud es , K A C SQ

K no w le dg e an d C o nf id en ce

S el f- R at ed

Q u es ti on na ir e, K A D D

K no w le d ge

ab o ut

D ua l D ia g no si s, M A P

M au ds le y A dd ic ti on

P ro fi le , SA

T S S ub st an ce

A bu se

T re at m en t S ca le

Int J Ment Health Addiction (2016) 14:856–872 861

Kirkpatrick’s level 1: perception of DD training programs

Seven of the included studies examined participants’ reactions to a DD training program (Cooper et al. 2006; Hughes et al. 2008a, b; Munro et al. 2007; Rani and Byrne 2012; Saxton et al. 2011; Tobin and Boulton 2009). All reported that a substantial proportion of the participants valued the training (see Table 2). Three of the studies examined participant evaluation more closely (Hughes et al. 2008b; Rani and Byrne 2012; Tobin and Boulton 2009). Rani and Byrne (2012) found that the majority of the participants preferred group work, demonstration of skills, discussions, lectures, and the involvement of patients, compared to vignettes, role plays, video recording, and Powerpoint presentations. Hughes et al. (2008b) found that the training participation from work colleagues from drug and alcohol services enhanced the learning experience for many of the mental health professionals, who also valued the work materials provided at the training. Tobin and Boulton (2009) found that the participants considered several areas relevant to their work, and would implement it in clinical practice, including motivational interviewing and using a stage approach to change (Drake and

Table 2 Results

Study Level 1: perception of the training

Level 2: effect on skills, knowledge and attitudes

Level 3: transfer of training

Level 4: effect on patient outcomes

Craig et al. (2008); Hughes et al. (2008a); Johnson et al. (2007)

Satisfaction with the training

- Increased knowledge - No sign. Effect on at- titudes

- Reduced psychiatric symptoms - No sign. Effect on SU

Clutterbuck et al. (2009); Copello et al. (2012); Graham (2004); Graham et al. (2006)

Satisfaction with the training

- Increased confidence and skills, up to 10 years later

Transfer of training occurred

- No sign. Effect on psychiatric symptoms - No sign. Effect on SU

Cooper et al. (2006) Satisfaction with the training

- Increased skills

Heslop et al. (2013) Satisfaction with the training

- Increased knowledge Transfer of training occurred

Hughes et al. (2008b) Satisfaction with the training

- Increased confidence and skills

Munro et al. (2007) Satisfaction with the training

- Increased knowledge - More positive attitudes

Najavits and Kanukollu (2005)

Satisfaction with the training

- Increased knowledge

Rani and Byrne (2012) Satisfaction with the training

- Increased skills and confidence - Increased knowledge

Transfer of training occurred

Saxton et al. (2011) Satisfaction with the training

- Increased knowledge - No sign. Effect on attitudes

Sciacca and Thompson (1996)

Satisfaction with the training

- Increased knowledge Transfer of training occurred

Tobin and Boulton (2009) Satisfaction with the training

862 Int J Ment Health Addiction (2016) 14:856–872

Mueser 2000). All three studies found that the participants recommended longer training programs, in order to acquire the needed competencies. In the study by Tobin and Boulton (2009), the training program lasted 1 day, and in the study by Rani and Byrne (2012), training was delivered 1 day a week over a five-week period. In the study by Hughes et al. (2008b) the training took place once a month for 5 months, a time length that increased the risk of forgetting what had taken place in previous sessions.

Kirkpatrick’s level 2: improvement in skills, knowledge, and attitudes

Three studies examined changes in professional skills by using questionnaires that measured self-perceived changes (Cooper et al. 2006; Graham et al. 2006; Hughes et al. 2008b). All of these studies found that participants in general perceived having increased their skills after the training program. The study by Graham et al. (2006) involved a control group that did not receive training initially, and changes in skills were only seen in the intervention group. The fact that the control group received training at a later stage and then increased their skills following this training, suggests that the training and no other factors was responsible for the change in skills. A few years after the training, a subgroup of the participants was interviewed, and a number of them stated that they felt more confident and skilled compared to 5 years earlier (Clutterbuck et al. 2009), and a follow-up study 10 years later also showed improve- ments in confidence and skills (Copello et al. 2012).

Seven studies (Heslop et al. 2013; Hughes et al. 2008a; Munro et al. 2007; Najavits and Kanukollu 2005; Rani and Byrne 2012; Saxton et al. 2011; Sciacca and Thompson 1996) reported outcomes concerning whether participants increased their knowledge of DD and DD treatment. Two of the included studies used self-rated questionnaires (Rani and Byrne 2012; Saxton et al. 2011), and found that, on average, participants experienced an increase in knowledge following training. Two of the other studies (Hughes et al. 2008a; Najavits and Kanukollu 2005) used a multiple-choice questionnaire to assess their participants’ knowledge of DD treatment pre- and post-training. Najavits and Kanukollu (2005) found a high level of correct responses at baseline and a small increase in knowledge from pre-training to post- training. Since no other measures were included, it is not possible to establish whether the small difference in pre- and post-scores was due to a ceiling effect in the form of an initially high level of knowledge, or whether the questions were too easy. The study by Hughes et al. (2008a) involved a control group that also completed the multiple-choice questionnaire. Improvements in knowledge were only seen in the intervention group, suggesting that training was responsible for the increased knowledge.

Two studies (Heslop et al. 2013; Sciacca and Thompson 1996) included a combination of both self-rated questionnaires and objective knowledge tests, and found that the participants on average increased their knowledge, as measured by both, after the training. The study by Munro et al. (2007) included both a knowledge test with true/false questions and qualitative interviews, together with a control group. The intervention group responded more correctly both post- training and at the six-month follow-up, and the qualitative interviews conducted at a later stage supported the link between training and improvement in knowledge (Watson and Munro 2003).

Three of the studies explored both changes in knowledge and changes in attitudes (Hughes et al. 2008a; Munro et al. 2007; Saxton et al. 2011). The only study that found an effect on attitudes was that of Munro et al. (2007). They found that attitudes in the intervention and control groups were rather negative before training, whereas the intervention group reported significantly more positive attitudes following the training and at the six-month follow-up.

Int J Ment Health Addiction (2016) 14:856–872 863

Kirkpatrick’s level 3: transfer of training

Whereas Level 2 evaluates the short-term effect of a training program in terms of acquired competencies, information on the longer-term outcomes of the training after participants have returned to their workplace is evaluated at Level 3 (O’Neill et al. 2004).

Four of the studies investigated whether the acquired competencies from training were transferred into clinical practice (Graham et al. 2006; Heslop et al. 2013; Rani and Byrne 2012; Sciacca and Thompson 1996). All four studies found that the participants had changed some of their work practices after the training. Rani and Byrne (2012) used focus group interviews 8 weeks after training, and found that some of the participants reported a change in their work practices regarding providing psychoeducation to their patients, and that participants who had not changed their practices explained this by lack of time, current work load, or poor patient attendance.

In the three other studies, the training was part of the implementation of a new treatment method (Graham et al. 2006; Heslop et al. 2013; Sciacca and Thompson 1996). In the study by Heslop et al. (2013), a screening instrument and brief interventions were to be implemented, and the authors reviewed medical records before and after participants completed the training program. Significant improvements were found in the number of drug and alcohol assessments at patient admission, and in the inclusion of drug and alcohol issues in the patients’ manage- ment plan following training, suggesting that some professional work practices had changed as a result of the training program and the implementation process. In the study by Sciacca and Thompson (1996), a new treatment model was to be implemented. Following the training, all the participants led at least one DD treatment group for the first time, suggesting a positive effect of the training and the implementation of a new treatment model. In the study by Graham et al. (2006), changes in teams’ practice were observed following their participation in a training program and the implementation of integrated DD treatment. Post-training involved more attempts to apply the intervention appropriately, improved incorporation of information on substance use into clinical medical case notes, more psychoeducation provided to the patients, and improved therapeutic practices.

Kirkpatrick’s level 4: effects of training on patient outcomes

Only two studies investigated whether training professionals in DD and DD treatment had an effect on patients’ psychiatric symptoms and substance use. In the first study, patients to mental health professionals in an intervention group who received training and supervision were compared to a control group (Craig et al. 2008; Johnson et al. 2007). Patients in the intervention group had significantly lower symptom levels at follow-up compared to patients in the control group, but there were no significant reductions in substance use in either group (Craig et al. 2008). In the second study, results indicated that the training did not affect the patients’ psychiatric symptoms (Graham et al. 2006). There was a reduction in outcomes related to substance use at follow-up, but this was found in both the intervention and control group.

Discussion

Research on the effect of training mental health professionals in DD treatment is an important field that calls for more attention and development, since training mental health professionals often is expensive and takes time from the clinic and the patients.

864 Int J Ment Health Addiction (2016) 14:856–872

Using Kirkpatrick’s Training Evaluation Model, we were able to identify both key findings and areas that require substantial further research. Seven studies reported that DD training programs were positively evaluated by the participants (Cooper et al. 2006; Hughes et al. 2008a, b; Munro et al. 2007; Rani and Byrne 2012; Saxton et al. 2011; Tobin and Boulton 2009), and three of these studies suggested that training programs should involve a wider range of teaching methods, last more than one training day, involve patients as lectures, and that the training should not be spread over a lengthy period of time (Hughes et al. 2008b; Rani and Byrne 2012; Tobin and Boulton 2009). However, the absence of standard measures regarding participants’ reaction to the DD training limits the interpretation of these findings, and more research is needed to determine how to design a suitable training program.

Three studies found that training programs enhance the professional skills of mental health professionals (Cooper et al. 2006; Graham et al. 2006; Hughes et al. 2008b), and seven studies found that training enhanced their professional knowledge (Heslop et al. 2013; Hughes et al. 2008a; Munro et al. 2007; Najavits and Kanukollu 2005; Rani and Byrne 2012; Saxton et al. 2011; Sciacca and Thompson 1996). However, the three studies on changes in attitudes towards patients with DD found mixed results (Hughes et al. 2008a; Munro et al. 2007; Saxton et al. 2011). The effect of training on these three outcomes can be evaluated by two types of measurement tools (O’Neill et al. 2004). One approach involves direct measures, e.g. tests, observations, program-specific questionnaires, role play, job and task simulation, and log books, while the other approach involves less direct measures, e.g. self-report questionnaires. The studies that measured changes in skills or attitudes used less direct measures, however, and the relations between experienced gains and clinical gains are yet to be established, since experienced gains do not necessarily mean that the participants improved their clinical skills or adopted more positive attitudes. Two of the studies that investigated changes in knowledge also used indirect measures (Rani and Byrne 2012; Saxton et al. 2011), and could only show that participants felt more knowledgeable following training, and not whether they actually gained more knowledge. An actual gain in knowledge could have been measured by using a knowledge test or questionnaire (O’Neill et al. 2004), which was done in two of the included studies (Hughes et al. 2008a; Najavits and Kanukollu 2005). However, such studies may be limited due to the use of tests or questionnaires that are too simple, or due to re-test effects. This limitation can be overcome by combining direct measures with less direct measures, which three studies did (Heslop et al. 2013; Munro et al. 2007; Sciacca and Thompson 1996). These studies showed both that the participants perceived a gain in knowledge and that they actually gained more knowledge.

Only four studies examined transfer of training, and all found that professional competen- cies acquired from training were transferred into clinical practice (Graham et al. 2006; Heslop et al. 2013; Rani and Byrne 2012; Sciacca and Thompson 1996). However, the training was part of a broader implementation process in three of the studies, and it is therefore unclear whether the effect was caused by the training program, the implementation process, or the combination of the two. Moreover, in the same three studies, the researchers or the consultants visited the workplace regularly to measure adherence to the new treatment method, or to support the implementation process, and it is likely that these regular meetings supported the transfer of training.

Transfer of training can be measured both by direct measures, e.g. observations, log books, and diaries, and by less direct measures, e.g. self-reports on behavior change (questionnaires, interviews, diaries, and focus groups), and reports of behavior as observed by peers and/or supervisors (O’Neill et al. 2004). Of the four studies measuring transfer of training, Rani and

Int J Ment Health Addiction (2016) 14:856–872 865

Byrne (2012) relied solely on results from focus group interviews, which is a less direct measure that only shows that participants believed that they had changed their practices following training, but not whether changes in clinical practices had actually occurred. The remaining three studies that assessed the transfer of training used direct measures. Heslop et al. (2013) reviewed medical records before and after the training and found that the assessment of drug and alcohol issues improved, and Sciacca and Thompson (1996) observed that the professionals led more DD treatment groups after the training. However, both studies are limited by the use of one single measure. The third study by Graham et al. (2006) involved a range of measures, including observations, observers’ ratings, and interviews which strength- ened the results of their study. Still, none of the studies used validated tests or questionnaires to evaluate the first three levels, whereas Level 4, the effect on patient outcomes, was primarily measured by validated instruments.

Only two studies explored the effect on patient outcomes. The study by Craig et al. (2008) suggested that training was associated with reduction in psychiatric symptoms, while the study by Graham et al. (2006) found no effect on psychiatric symptoms. None of these studies found an effect on substance use. Since only two studies included patients’ outcomes, it is premature to draw any conclusions regarding the effect at the patient level.

Long-term effect of training

Participation in training programs costs both money and time, and it is therefore important to consider the long-term gains of the training. If competencies acquired during training are forgotten few months later, it might be more useful to qualify mental health professionals in other ways. Only two of the identified studies (Graham et al. 2006; Munro et al. 2007) included a follow-up, and one of them showed that gains in skills remained 10 years after the training (Copello et al. 2012; Graham et al. 2006).

Control groups

Intuitively, it is hard to imagine that training mental health professionals in DD treatment would have no effect on their knowledge, skills, or attitudes. A critical question is therefore what a specific training program requires, in order to increase these competencies. None of the included studies compared different training programs, and the lack of control groups was a general problem in most of the studies. Without control groups, it cannot be ruled out that other factors could account for any observed effect, for instance that participants might simply have become more interested in the DD field as a result of their participation in a study. Greater interest in DD could in turn have led the participants to seek more information, resulting in increased skills, knowledge, and attitudes. Another factor that could account for the results is the so-called Hawthorne Effect (Mayo 1933) whereby receiving attention from researchers during study participation may cause participants to feel more competent at the end of the study, regardless of whether they had been through training or not.

Conceptualization

The review found considerable challenges with the conceptualization of the outcome skills. The three studies that explored changes in skills used a questionnaire which measured changes in perceived skills and confidence as a single construct. It is therefore not possible to

866 Int J Ment Health Addiction (2016) 14:856–872

distinguish between improvements in skills and confidence: participants might feel more confident after training without necessarily being more skilled. This argument is supported by the fact that five of the included studies (Heslop et al. 2013; Hughes et al. 2008a; Munro et al. 2007; Rani and Byrne 2012; Saxton et al. 2011) found that the participants’ professional confidence increased after the training program, suggest- ing that specialized training enhances professional confidence. However, again the effect on skills is still unknown.

Future research

There are several areas that require further research, and researchers and practitioners need to work together to develop reliable and valid measures of the key constructs that DD training aims to address. Future research on training effects should combine direct and indirect measures, and focus on changes in professional skills, in knowledge, and in attitudes. Moreover, future research should include control groups, or at least some type of relevant comparison group. Randomization of individual practitioners may often not be feasible, but cluster-randomized studies could be a pragmatic solution in many mental health settings. Ideally, such studies should involve training with various contents and intensities, in order to improve knowledge of what are the important elements in training. Since only one study explored the long-term effects of training and only two studies examined patient outcomes, more research is needed on the long-term effects of DD training programs and the effect on patient outcomes.

Limitations

The literature search was only performed by the first author. However, the search was performed twice to minimize errors, and at both stages, the included studies were agreed upon by all three authors. Another limitation is a possible publication bias, namely that studies with negative findings may not have been reported in the literature.

Conclusion

Mental health professionals have a positive perception of DD training programs, they consider that they gain knowledge from them, and some transfer of training to clinical practice occurs. Whether the professionals actually acquire more skills, change their attitudes, or whether their patients benefit from the training is still unknown. Because of the methodological limitations in the included studies, it would be premature to draw any firm conclusions regarding the effect of DD training programs. Future studies should include patient outcomes, control groups, follow-ups, and validated multiple measures.

Compliance with ethical standards

Funding No grants or financial support were received for this review.

Informed consent No animal or human studies were carried out by the authors for this article.

Disclosures Pernille Pinderup, Birgitte Thylstrup, and Morten Hesse declare that they have no conflict of interest.

Int J Ment Health Addiction (2016) 14:856–872 867

Appendix A: Search string

Search string

Balcohol dependenc*^ OR Bdrug dependenc*^ OR Bdrug use disorder*^ OR Balcohol use disorder*^ OR substance* OR Bsubstance use disorder*^ OR Bdrug abus*^ OR Balcohol abus*^ OR addiction*

AND psychiatr* OR mental OR psychos* OR psychot*

AND attitude* OR perception* OR knowledge OR stigma OR satisfaction OR skill* OR view* OR confidence OR treatment outcome OR efficacy OR competence* OR abilit*

AND (train* OR program* OR educat* OR workshop OR intervent* OR quiz) NEAR/4 (doctor* OR staff OR nurse* OR professional* OR manager* OR psychiatrist* OR physician* OR psychologist* OR clinician* OR therapist* OR worker* OR personnel OR practitioner* OR co-existing OR coexisting OR cooccurring OR co-occurring OR co-morbidity OR comorbidity OR concurrent OR Bdual diagnosis^ OR Bdual disorder^)

AND co-existing OR coexisting OR cooccurring OR co-occurring OR co-morbidity OR comorbidity OR Bdual diagnosis^ OR Bdual disorder^ OR concurrent

Appendix B: Excluded studies

Excluded studies Reason for exclusion

Eden, T., & Hughes, L. (2009). Facilitating the dialogue between service users and participants in a training situation. Advances in Dual Diagnosis, 2(3), 5–7.

Harwood, H. J., Kowalski, J., & Ameen, A. (2004). The need for substance abuse training among mental health professionals. Administration and Policy in Mental Health and Mental Health Services Research, 32(2), 189–205.

Manley, D. S. (2008). Acceptability and applicability of Cue Exposure Therapy as a relapse prevention intervention for individuals who have substance misuse and mental health problems. Mental Health and Substance Use, 1(2), 172–184.

Maxwell, S. (2001). Care of people with dual disabilities in the mental health system: Education vs. attitude rehabilitation. Psychiatric Rehabilitation Skills, 5(1), 197–215.

Moore, J. (2013). Dual diagnosis: training needs and attitudes of nursing staff: Jayne Moore explores nursing staff’s training needs and their attitudes towards patients who misuse substances in a large forensic mental health service. Mental Health Practice, 16(6), 27–31.

Nehlin, C., Fredriksson, A., Gronbladh, L., & Jansson, L. (2012). Three hours of training improve psychiatric staff’s self-perceived knowledge and attitudes toward problem-drinking patients. Drug and Alcohol Review, 31(4), 544–549.

Rassool, G. H. (2006). Professional Education in Co-occurring Disorders: Some Considerations towards Practice Development: Journal of Addictions Nursing, 17(3), 187–191.

Schoener, E. P., Madeja, C. L., Henderson, M. J., Ondersma, S. J., & Janisse, J. J. (2006). Effects of motivational interviewing training on mental health therapist behavior. Drug and Alcohol Dependence, 82(3), 269–275.

Wrong intervention

Cameron, J., Lee, N. K., & Harney, A. (2010). Changes in attitude to, and confidence in, working with comorbidity after training in screening and brief intervention. Mental Health and Substance Use: Dual Diagnosis, 3(2), 124–130.

Caravella, K., Tod, L., & Brown, A.-M. (2012). Awareness into action: How communication skills training enhances traditional substance abuse treatment programs. Journal of Global Drug Policy and Practice, 6(1).

Crowe, T. P., Kelly, P., Pepper, J., McLennan, R., Deane, F. P., & Buckingham, M. (2013). Service Based Internship Training to Prepare Workers to Support the Recovery of People with Co-Occurring Substance Abuse and Mental Health Disorders. International Journal of Mental Health and Addiction, 11(2), 269–280.

Wrong study population

868 Int J Ment Health Addiction (2016) 14:856–872

Excluded studies Reason for exclusion

Hunter, S. B., Watkins, K. E., Wenzel, S., Gilmore, J., Sheehe, J., & Griffin, B. (2005). Training substance abuse treatment staff to care for co-occurring disor- ders. Journal of Substance Abuse Treatment, 28(3), 239–245.

Lee, N., Jenner, L., Baker, A., Ritter, A., Hides, L., Norman, J., … Cameron, J. (2010). Screening and intervention for mental health problems in alcohol and other drug settings: Can training change practitioner behaviour? Drugs: Education, Prevention, and Policy, 18(2), 157–160.

Roussy, V., Thomacos, N., Rudd, A., & Crockett, B. (2013). Enhancing health-care workers’ understanding and thinking about people living with co-occurring mental health and substance use issues through consumer-led training. Health Expectations, 18(5), 1567–81.

Wenzel, S. L., Ebener, P., Hunter, S. B., Watkins, K. E., & Gilmore, J. M. (2005). Research-practice partners assess their first joint project. Science & Practice Perspectives, 3(1), 38–45.

Covell, N. H., Margolies, P. J., Smith, M. F., Merrens, M. R., & Essock, S. M. (2011). Distance Training and Implementation Supports to Scale Up Integrated Treatment for People With Co-occurring Mental Health and Substance Use Disorders. Journal of Dual Diagnosis, 7(3), 162–172.

Davis, K., O’Neill, S., Devitt, T., Baerentzen, B., Little, N., & Wilkniss, S. (2012). Consulting in action: A case study of six community support teams sustaining integrated dual disorder treatment. American Journal of Psychiatric Rehabilitation, 15(4), 313–333.

Lewis, T. (2008). Dual diagnosis education by distance learning. Advances in Dual Diagnosis, 1(2), 13–18.

McKee, S. A., Harris, G. T., & Cormier, C. A. (2013). Implementing Residential Integrated Treatment for Co-occurring Disorders. Journal of Dual Diagnosis, 9(3), 249–259. http://doi.org/10.1080/15504263.2013.807073

Renner Jr., J. A., Quinones, J., & Wilson, A. (2005). Training psychiatrists to diagnose and treat substance abuse disorders. Current Psychiatry Reports, 7(5), 352–359.

Wrong outcomes

References

Adams, M. W. (2008). Comorbidity of mental health and substance misuse problems: a review of workers’ reported attitudes and perceptions. Journal of Psychiatric and Mental Health Nursing, 15(2), 101–108.

Archie, S., & Gyomorey, K. (2009). First episode psychosis, substance abuse and prognosis: a systematic review. Current Psychiatry Reviews, 5(3), 153–163.

Barry, K. R., Tudway, J. A., & Blissett, J. (2002). Staff drug knowledge and attitudes towards drug use among the mentally ill within a medium secure psychiatric hospital. Journal of Substance Use, 7(1), 50–56.

Buckley, R., & Caple, J. (2007). The theory & practice of training. Kogan Page Publishers. Cleary, M., Hunt, G. E., Maheson, S., & Walter, G. (2009). Views of Australian mental health stakeholders on

clients’ problematic drug and alcohol use. Drug and Alcohol Review, 28(2), 122–128. doi:10.1111/j.1465- 3362.2008.00041.x.

Clutterbuck, R., Tobin, D., Orford, J., Copello, A., Preece, M., Birchwood, M., et al. (2009). Exploring the attitudes of staff working within mental health settings toward clients who use cannabis. Drugs: Education, Prevention, and Policy, 16(4), 311–327. doi:10.1080/09687630801945861.

Cooper, P., Doolin, N., Hemming, I., & Rowlands, E. (2006). Participants’ evaluation of a training programme in dual diagnosis. Nursing Standard, 20(25), 48–56. doi:10.7748/ns2006.03.20.25.48.c4083.

Copello, A., Walsh, K., Graham, H., Tobin, D., Fellows, S., Griffith, E., et al. (2012). The impact of training within a comprehensive dual diagnosis strategy: the combined psychosis and substance use (COMPASS) experience. Mental Health and Substance Use, 5(3), 206–216. doi:10.1080/17523281.2012.660191.

Craig, T. K. J., Johnson, S., McCrone, P., Afuwape, S., Hughes, E., Gournay, K., et al. (2008). Integrated care for co-occurring disorders: psychiatric symptoms, social functioning, and service costs at 18 months. Psychiatric Services, 59(3), 276–282.

Int J Ment Health Addiction (2016) 14:856–872 869

Di Lorenzo, R., Galliani, A., Ferri, P., Landi, G., & Guicciardi, A. (2014). A retrospective analysis focusing on a group of patients with dual diagnosis treated by both mental health and substance use services. Neuropsychiatric Disease and Treatment, 1479–1488. doi:10.2147/NDT.S65896

Drake, R. E., & Mueser, K. T. (2000). Psychosocial approaches to dual diagnosis. Schizophrenia Bulletin, 26(1), 105–118. Drake, R. E., Mueser, K. T., Clark, R. E., & Wallach, M. A. (1996). The course, treatment, and outcome of substance

disorder in persons with severe mental illness. The American Journal of Orthopsychiatry, 66(1), 42–51. Drake, R. E., Mueser, K. T., Brunette, M. F., & McHugo, G. J. (2004). A review of treatments for people with

severe mental illnesses and co-occurring substance use disorders. Psychiatric Rehabilitation Journal, 27(4), 360–374.

Graham, H. L. (2004). Implementing integrated treatment for co-existing substance use and severe mental health problems in assertive outreach teams: training issues. Drug and Alcohol Review, 23(4), 463–470. 10.1080/ 09595230412331324581.

Graham, H. L., Copello, A., Birchwood, M., Orford, J., McGovern, D., Mueser, K. T., et al. (2006). A preliminary evaluation of integrated treatment for co-existing substance use and severe mental health problems: impact on teams and service users. Journal of Mental Health, 15(5), 577–591. doi:10.1080/09638230600902633.

Grella, C. E. (2003). Contrasting the views of substance misuse and mental health treatment providers on treating the dually diagnosed. Substance Use & Misuse, 38(10), 1433–1446. doi:10.1081/JA-120023393.

Griffin, S., Campbell, A., & McCaldin, H. (2008). A Bdual diagnosis^ community psychiatric nurse service in Lanarkshire: service innovation. Psychiatric Bulletin, 32(4), 139–142. doi:10.1192/pb.bp.107.015263.

Hansen, S. S., Munk-Jorgensen, P., Guldbaek, B., Solgard, T., Lauszus, K. S., Albrechtsen, N., et al. (2000). Psychoactive substance use diagnoses among psychiatric in-patients. Acta Psychiatrica Scandinavica, 102(6), 432–438. doi:10.1034/j.1600-0447.2000.102006432.x.

Happell, B., Carta, B., & Pinikahana, J. (2002). Nurses’ knowledge, attitudes and beliefs regarding substance use: a questionnaire survey. Nursing & Health Sciences, 4(4), 193–200.

Haywood, T. W., Kravitz, H. M., Grossman, L. S., Cavanaugh, J. L., et al. (1995). Predicting the Brevolving door^ phenomenon among patients with schizophrenic, schizoaffective, and affective disorders. The American Journal of Psychiatry, 152(6), 856–861.

Heslop, K., Ross, C., Osmond, B., & Wynaden, D. (2013). The alcohol smoking and substance involvement screening test (ASSIST) in an acute mental health setting. International Journal of Mental Health and Addiction, 11(5), 583–600. doi:10.1007/s11469-013-9428-3.

Howard, V., & Holmshaw, J. (2010). Inpatient staff perceptions in providing care to individuals with co-occurring mental health problems and illicit substance use: staff perceptions towards patients with co-occurring ill- mental health and illicit substance use. Journal of Psychiatric and Mental Health Nursing, 17(10), 862–872. doi:10.1111/j.1365-2850.2010.01620.x.

Hughes, E., Wanigaratne, S., Gournay, K., Johnson, S., Thornicroft, G., Finch, E., et al. (2008a). Training in dual diagnosis interventions (the COMO study): randomised controlled trial. BMC Psychiatry, 8(1), 12. doi: 10.1186/1471-244X-8-12.

Hughes, L., Tobin, D., McGlynn, P., & Heffernan, K. (2008b). A pilot of team-based dual diagnosis training for assertive outreach teams. Advances in Dual Diagnosis, 1(2), 6–12. doi:10.1108/17570972200800011.

Johnson, S., Thornicroft, G., Afuwape, S., Leese, M., White, I. R., Hughes, E., et al. (2007). Effects of training community staff in interventions for substance misuse in dual diagnosis patients with psychosis (COMO study) cluster randomised trial. The British Journal of Psychiatry, 191(5), 451–452. doi:10.1192/bjp.bp.106.032367.

Kessler, R. C., Nelson, C. B., McGonagle, K. A., Edlund, M. J., Frank, R. G., & Leaf, P. J. (1996). The epidemiology of co-occurring addictive and mental disorders: implications for prevention and service utilization. American Journal of Orthopsychiatry, 66(1), 17–31. doi:10.1037/h0080151.

Kirkpatrick, D. L. (1998). Evaluating training programs : the four levels (2nd ed.). London: Berrett-Koehler Publishers.

Laudet, A., Magura, S., Vogel, H., & Knight, E. (2002). Interest in and obstacles to pursuing work among unemployed dually diagnosed individuals. Substance Use & Misuse, 37(2), 145–170.

Mangrum, L. F., Spence, R. T., & Lopez, M. (2006). Integrated versus parallel treatment of co-occurring psychiatric and substance use disorders. Journal of Substance Abuse Treatment, 30(1), 79–84. doi: 10.1016/j.jsat.2005.10.004.

Mayo, E. (1933). The human problems of an industrial civilization. New York: Macmillan. McNiel, D. E., Binder, R. L., & Robinson, J. C. (2005). Incarceration associated with homelessness, mental

disorder, and co-occurring substance abuse. Psychiatric Services, 56(7), 840–846. Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group D2009]. Preferred

reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Medicine, 6D7], e1000097.http://doi.org/10.1371/journal.pmed.1000097

870 Int J Ment Health Addiction (2016) 14:856–872

Morojele, N. K., Saban, A., & Seedat, S. (2012). Clinical presentations and diagnostic issues in dual diagnosis disorders. Current Opinion in Psychiatry, 25(3), 181–186. doi:10.1097/YCO.0b013e328351a429.

Mueser, K. T., Noordsy, D. L., Drake, R. E., & Fox, L. (2003). Integrated Treatment for Dual Disorders: A Guide to Effective Practice. New York: Guilford Press.

Munro, A., Watson, H. E., & McFadyen, A. (2007). Assessing the impact of training on mental health nurses’ therapeutic attitudes and knowledge about co-morbidity: a randomised controlled trial. International Journal of Nursing Studies, 44(8), 1430–1438. doi:10.1016/j.ijnurstu.2006.07.024.

Najavits, L. M., & Kanukollu, S. (2005). It can be learned, but can it be taught?: results from a state-wide training initiative on PTSD and substance abuse. Journal of Dual Diagnosis, 1(4), 41–51. doi:10.1300/J374v01n04_05.

NICE (2011). Psychosis with coexisting substance misuse: assessment and management in adults and young people. In Clinical guideline 120. London: NICE.

O’Gara, C., Keaney, F., Best, D., Harris, J., Boys, A., Leonard, F., et al. (2005). Substance misuse training among psychiatric doctors, psychiatric nurses, medical students and nursing students in a South London psychiatric teaching hospital. Drugs: Education, Prevention, and Policy, 12(4), 327–336. doi:10.1080/09687630500083691.

O’Neill, M., Addy, D., & Roche, A. M. (2004). Guidelines for evaluating alcohol and other drug education and training programs. Canberra: Australian Government of Department of Health and Ageing.

Olfson, M., Mechanic, D., Hansell, S., Boyer, C. A., & Walkup, J. (1999). Prediction of homelessness within three months of discharge among inpatients with schizophrenia. Psychiatric Services, 50(5), 667–673.

Ralley, C., Allott, R., Hare, D. J., & Wittkowski, A. D2009]. The use of the repertory grid technique to examine staff beliefs about clients with dual diagnosis. Clinical Psychology & Psychotherapy, 16D2], 148–158 http://doi.org/10.1002/cpp.606.

Rani, S., & Byrne, H. (2012). A multi-method evaluation of a training course on dual diagnosis. Journal of Psychiatric and Mental Health Nursing, 19(6), 509–520. doi:10.1111/j.1365-2850.2011.01808.x.

Regier, D. A., Farmer, M. E., Rae, D. S., et al. (1990). Comorbidity of mental disorders with alcohol and other drug abuse: results from the epidemiologic catchment area (ECA) study. JAMA, 264(19), 2511–2518. doi: 10.1001/jama.1990.03450190043026.

Renner, J. A. (2004). How to train residents to identify and treat dual diagnosis patients. Biological Psychiatry, 56(10), 810–816. doi:10.1016/j.biopsych.2004.04.003.

Richmond, I., & Foster, J. (2003). Negative attitudes towards people with co-morbid mental health and substance misuse problems: an investigation of mental health professionals. Journal of Mental Health, 12(4), 393–403. doi:10.1080/0963823031000153439.

Rosenberg, S. D., Goodman, L. A., Osher, F. C., Swartz, M. S., Essock, S. M., Butterfield, M. I., et al. (2001). Prevalence of HIV, hepatitis B, and hepatitis C in people with severe mental illness. American Journal of Public Health, 91(1), 31–37.

Ryrie, I., & McGowan, J. (1998). Staff perceptions of substance use among acute psychiatry inpatients. Journal of Psychiatric and Mental Health Nursing, 5(2), 137–142.

Saxton, L., Lancashire, S., & Kipping, C. (2011). Meeting the training needs of staff working with older people with dual diagnosis. Advances in Dual Diagnosis, 4(1), 36–46. doi:10.1108/17570971111155603.

Schmidt, L. M., Hesse, M., & Lykke, J. (2011). The impact of substance use disorders on the course of schizophrenia–a 15-year follow-up study: dual diagnosis over 15 years. Schizophrenia Research, 130(1– 3), 228–233. doi:10.1016/j.schres.2011.04.011.

Schulte, S. J., Meier, P. S., Stirling, J., & Berry, M. (2010). Dual diagnosis competency among addiction treatment staff: training levels, training needs and the link to retention. European Addiction Research, 16(2), 78–84. doi:10.1159/000277657.

Sciacca, K., & Thompson, C. M. (1996). Program development and integrated treatment across systems for dual diagnosis: mental illness, drug addiction, and alcoholism (MIDAA). The Journal of Mental Health Administration, 23(3), 288–297. doi:10.1007/BF02522303.

Siegfried, N., Ferguson, J., Cleary, M., Walter, G., & Rey, J. M. (1999). Experience, knowledge and attitudes of mental health staff regarding patients’ problematic drug and alcohol use. Australian and New Zealand Journal of Psychiatry, 33(2), 267–273. doi:10.1046/j.1440-1614.1999.00547.x.

Soyka, M. (2000). Substance misuse, psychiatric disorder and violent and disturbed behaviour. The British Journal of Psychiatry, 176(4), 345–350.

Soyka, M., Albus, M., Immler, B., Kathmann, N., & Hippius, H. (2001). Psychopathology in dual diagnosis and non-addicted schizophrenics–are there differences? European Archives of Psychiatry and Clinical Neuroscience, 251(5), 232–238.

Tobin, D., & Boulton, D. (2009). Developing a training programme to support delivery of integrated dual diagnosis treatment in acute inpatient settings. Advances in Dual Diagnosis, 2(3), 14–19. doi: 10.1108/17570972200900020.

Int J Ment Health Addiction (2016) 14:856–872 871

Todd, J., Green, G., Harrison, M., Ikuesan, B. A., Self, C., Baldacchino, A., et al. (2004). Defining dual diagnosis of mental illness and substance misuse: some methodological issues. Journal of Psychiatric and Mental Health Nursing, 11(1), 48–54.

Toftdahl, N. G., Nordentoft, M., & Hjorthøj, C. (2015). Prevalence of substance use disorders in psychiatric patients: a nationwide Danish population-based study. Social Psychiatry and Psychiatric Epidemiology, 1– 12. doi:10.1007/s00127-015-1104-4

Watson, H. E., & Munro, A. (2003). A study of staff training to promote effective care for clients with Bdual diagnosis.^. Glasgow: Caledonian Nursing & Midwifery Research Centre, Glasgow Caledonian University.

Zammit, S., Moore, T. H. M., Lingford-Hughes, A., Barnes, T. R. E., Jones, P. B., Burke, M., et al. (2008). Effects of cannabis use on outcomes of psychotic disorders: systematic review. The British Journal of Psychiatry, 193(5), 357–363. doi:10.1192/bjp.bp.107.046375.

872 Int J Ment Health Addiction (2016) 14:856–872

International Journal of Mental Health & Addiction is a copyright of Springer, 2016. All Rights Reserved.