Social Anxiety

lable at ScienceDirect

Behaviour Research and Therapy 47 (2009) 902–909

Contents lists avai

Behaviour Research and Therapy

j o u r n a l h o m e p a g e : w w w . e l s e v i e r . c o m / l o c a t e / b r a t

Mind the gap: Improving the dissemination of CBT

R. Shafran a, *, D.M. Clark b, C.G. Fairburn c, A. Arntz d, D.H. Barlow e, A. Ehlers b, M. Freeston f, P.A. Garety b, S.D. Hollon g, L.G. Ost h, P.M. Salkovskis b, J.M.G. Williams c, G.T. Wilson i

a School of Psychology and Clinical Language Science, University of Reading, Berkshire RG6 6AL, UK b Institute of Psychiatry, University of London, SE5 8AF, UK c Psychiatry Department, Warneford Hospital, University of Oxford, OX3 7JX, UK d Department of Medical, Clinical, and Experimental Psychology, Maastricht University, PO Box 616, 6200 MD Maastricht, the Netherlands e CAS Psychology, Boston University, One Silber Way, Boston, MA 02215, USA f Ridley Building School of Neurology, Neurobiology and Psychiatry Faculty of Medical Sciences, University of Newcastle, Newcastle upon Tyne Tyne and Wear NE1 7RU, UK g Vanderbilt University, 2201 West End Avenue, Nashville, TN 37240, USA h Stockholm University, SE-106 91 Stockholm, Sweden i Rutgers University, Busch Campus, 604 Bartholomew Rd Piscataway, NJ 08854-8002, USA

Keywords: Psychological treatments Dissemination Evidence-based Manuals Guidelines Training

* Corresponding author. Tel.: þ44 118 378 6668; fa E-mail address: [email protected] (R. Shafra

0005-7967/$ – see front matter � 2009 Elsevier Ltd. doi:10.1016/j.brat.2009.07.003

a b s t r a c t

Empirically supported psychological treatments have been developed for a range of psychiatric disorders but there is evidence that patients are not receiving them in routine clinical care. Furthermore, even when patients do receive these treatments there is evidence that they are often not well delivered. The aim of this paper is to identify the barriers to the dissemination of evidence-based psychological treatments and then propose ways of overcoming them, hence potentially bridging the gap between research findings and clinical practice.

� 2009 Elsevier Ltd. All rights reserved.

The problem

Over the last 30 years considerable progress has been made in developing effective psychological treatments for a wide range of psychiatric disorders. Among these evidence-based treatments forms of cognitive behaviour therapy (CBT) have been consistently shown to be effective across a wide range of disorders. Data from multiple randomized controlled trials (RCTs) indicate that disorder-specific forms of CBT are potent sole interventions in anxiety disorders, eating disorders and unipolar depression. There is also evidence from RCTs and from meta-analyses that cognitive-behavioural therapies are effective at reducing psychotic symptoms and distress, with small –medium effect sizes (e.g., Wykes, Steel, Everitt, & Tarrier, 2008). Promising data suggest that CBT is effective for somatoform disorders (Kroenke, 2007) and schema-focused therapy and dialectical behav- iour therapy are both effective for borderline personality disorder (Farrell, Shaw, & Webber, 2009; Giesen-Bloo et al., 2006; Linehan et al., 2006). CBT may enhance the effects of medication in bipolar disorder (Miklowitz, 2006).

Professional and governmental organisations now recognise the value of CBT and it is strongly advocated for the treatment of

x: þ44 118 378 6715. n).

All rights reserved.

anxiety disorders, unipolar depression and eating disorders in National Guidelines such as those in the UK (e.g., NICE, 2004), USA (e.g., American Psychiatric Association, 2009) and Australia (Ellis, Hickie, & Smith, 2003) However, there is evidence that such empirically supported treatments are rarely available and, even when they are, they are often delivered suboptimally.

Evidence that patients are not receiving CBT

Data from the UK and USA indicate that few patients with a detected psychiatric disorder receive CBT. For generalized anxiety disorder, panic disorder and social phobia, the most common psychosocial treatment in 1996 was psychodynamic (Goisman, Warshaw, & Keller, 1999). Overall in the NCS-R dataset of over 9000 people in the USA, complementary and alternative medicine treat- ments accounted for 31.3% of all mental health visits (Wang et al., 2005). The most common treatment offered to patients with PTSD in the UK is supportive counselling (Ehlers, Gene-Cos, & Perrin, 2009), although UK treatment guidelines recommend trauma-focused psychological treatments as the treatment of choice (NICE, 2005). For the treatment of eating disorders 35–39% of therapists report CBT as their primary approach (Mussell et al., 2000; von Ranson & Robinson, 2006), only 6.9% of patients with bulimia nervosa receive CBT (Crow, Mussell, Peterson, Knopke, & Mitchell, 1999) and fewer

R. Shafran et al. / Behaviour Research and Therapy 47 (2009) 902–909 903

than 4% of general practitioners use national guidelines to inform the management of their patients (Currin et al., 2007) although implementation rates have been reported as higher in academic medical centres (McAlpine, Schroder, Pankratz, & Maurer, 2004). For schizophrenia, fewer than 50% of individuals in the UK with psychosis are offered at least one session of CBT, despite the clinical guidelines recommending that it should be offered to all who have psychotic symptoms (Berry & Haddock, 2008; NICE, 2003, 2009; Healthcare Commission, 2007). It is little surprise that evidence- based psychological therapies are so unavailable given the lack of training provided in them across professional disciplines (Weiss- man et al., 2006). Based on the unmet need for evidence-based psychological treatment, increasing their availability has been identified as a priority in NIMH’s strategic plans (Insel, 2009).

Evidence that CBT is being delivered suboptimally

When CBT is being delivered, it appears that it is often being delivered suboptimally. In an analysis of the NCS-R dataset for mood disorders only (Kessler, Merikangas, & Wang, 2007), no more than 20.9% of all people with 12-month Major Depressive Disorder were considered to have received adequate pharmaco- logical/psychological treatment. Minimally adequate psycholog- ical treatment was defined based on available evidence-based guidelines and at least 8 sessions were required based on the fact that clinical trials demonstrating effectiveness have generally included 8 psychotherapy visits or more. In a small study of OCD, 60% of the sample who reported having CBT did not meet defined minimal criteria for adequacy (Stobie, Taylor, Quigley, Ewing, & Salkovskis, 2007).

Why is CBT so unavailable and so poorly delivered?

The dissemination of the treatments is hampered by a combi- nation of factors. The funding issue is beyond the remit of this paper but it is important to note there is evidence that CBT is a cost- effective treatment (van Asselt et al., 2008; Layard, Clark, Knapp, & Mayraz, 2007; Myhr & Payne, 2006). There has been relatively little written in the cognitive-behavioural literature about barriers to dissemination with some notable exceptions (Addis, 2002; Barlow, 2004; Barlow, Levitt, & Bufka, 1999; Carroll & Rounsaville, 2007; Chorpita & Nakamura, 2004; Stirman, Crits-Christoph, & DeRubeis, 2004; Taylor & Chang, 2008; Weingardt, 2004). The obstacles to dissemination can be divided into two groups. The first concerns commonly held beliefs among clinicians such as (1) the limited relevance of research trials to clinical practice, (2) the therapist is more important in determining outcome than the specific protocol and (3) beliefs about the protocol such as the unhelpfulness of diagnoses, the desirability of mixing and matching preferred parts of different interventions and beliefs that they are implementing protocol when this may not be the case. The second group of obstacles concerns gaps in our knowledge about treatments, their delivery and training modes; filling these gaps and providing more readily available and improved training opportunities for practi- tioners would facilitate the adaptation of research findings to routine clinical practice.

Commonly held beliefs

Research trials have limited applicability to clinical practice

The belief that research trials have limited relevance to clinical practice is based on beliefs about the nature of the patients treated in trials, the generalizability of research findings to routine settings

and the discrepancy between resources in research trials and clinical settings. Each of these views is discussed below.

Beliefs about patients treated in trials. Many clinicians are concerned that treatments are not as effective in regular healthcare as in trials conducted in academic settings because the patients in research trials are perceived to be less severe and less comorbid owing to extensive exclusion criteria in trials. Yet there is increasing evidence that the findings from research studies can and do generalize to routine clinical situations (e.g., Weisz, Weersing, & Henggeler, 2005; Wilson, 2007). The gap between clinical practice and research trials may not be as wide as many perceive and may not necessarily be in the direction of lower severity in trial patients. Patients in routine clinical settings often fail to meet the minimum severity or duration criteria needed for research trials (Stirman, DeRubeis, Crits-Chris- toph, & Rothman, 2005). Many of the more recent trials do allow most comorbidity and only use exclusions that resemble those clinical services would use to determine suitability for their partic- ular psychological treatment. For example, in the trial by DeRubeis et al. (2005), the only excluded disorders were psychosis and substance abuse severe enough to require immediate detoxification. About two-thirds of that sample met criteria for at least one addi- tional DSM disorder, half met criteria for one or more Axis II disorder, and about a third met criteria for substance use disorders. It is also important to clarify that the presence of comorbidity does not necessarily predict poor outcome in the index disorder (e.g., Brown, Antony, & Barlow, 1995). For example, a series of studies have demonstrated that comorbid personality disorders have no negative influence of effects of CBT for various anxiety disorders (Dreessen & Arntz, 1998; Dreessen, Arntz, Luttels, & Sallaerts, 1994; Dreessen, Hoekstra, & Arntz, 1997; Weertman, Arntz, Schouten, & Dreessen, 2005) or eating disorders (Grilo et al., 2007; Rowe et al., 2008). In PTSD, if one allows therapists some flexibility over the number of sessions and allows them to use the relevant CBT interventions for comorbid problems if necessary, then the presence of comorbid disorders does not predict poorer outcome (Duffy, Gillespie, & Clark, 2007; Gillespie, Duffy, Hackmann, & Clark, 2002). Furthermore, if clinical judgement indicates that comorbid depression is interfering with patients’ engagement with an anxiety disorder treatment, it may be helpful to use a protocol such as Beckian Behavioural Acti- vation to help lift mood (Duffy et al., 2007). Unified or trans- diagnostic protocols (see McHugh, Murray, & Barlow, 2009) for multiple diagnoses circumvent many of the issues surrounding comorbidity and so also have the potential to increase the relevance of the research to everyday clinical practice. Despite research trials addressing comorbidity in these ways, treatment manuals used in RCTs do not typically provide guidance for dealing with issues of comorbidity which does little to dispel beliefs that trials don’t deal with comorbidity and it also limits their usefulness for clinicians for whom the majority of their patients are likely to have multiple disorders (Kessler, Chiu, Demler, & Walters, 2005). A notable exception is the manual for the enhanced transdiagnostic treatment for eating disorders (CBT-E) which addresses all forms of commonly encountered comorbidity (Fairburn, 2008). Further development of improved, user-friendly, flexible treatment manuals that address these understandable concerns of clinical practitioners are needed.

Data on the generalizabilty of RCT results to routine settings. The question of how well the results of RCTs replicate with populations that have fewer patient exclusion criteria is an empirical one and there is a need for studies that examine the question of whether or not CBT is less effective in unselected populations than in most RCTs. In the recent transdiagnostic RCT of CBT-E (Fairburn et al., 2009), which had very few exclusion criteria, the results were as good, if not better, than in earlier studies with more restrictive entry

R. Shafran et al. / Behaviour Research and Therapy 47 (2009) 902–909904

criteria (Crow & Peterson, 2009). Large sample sizes will be needed to isolate the effects of specific exclusion criteria. However, it is noteworthy that the few studies that have so far examined gener- alizability but those that that have stayed faithful to the treatment protocol have obtained promising results. For example, 87% of 110 patients in a community mental health centre with a primary diagnosis of panic disorder with or without agoraphobia were panic-free at the end of a 15-session CBT protocol (Wade, Treat, & Stuart 1998). Similarly a study of 416 unselected patients with a primary diagnosis of panic disorder with agoraphobia (treated by 52 therapists in 3 outpatient clinics) found significant improvement in symptoms and no difference between their effect size and the average reported by meta-analytic studies of controlled efficacy research (Hahlweg, Fiegenbaum, Frank, Schroeder, &von Witzleben, 2001). A study by Addis et al. conducted on the ‘front-line’ of mental health service provision found good long term results for those that completed treatment (Addis et al., 2006) for panic disorder, consistent with the data demonstrating that such interventions are feasible to implement in a wide range of settings including primary care (Roy-Byrne et al., 2005). CBT for social phobia can also be applied effectively in the field (Lincoln et al., 2003) as can CBT for psychosis (Morrison et al., 2004). Comparable outcomes to RCTs have also been found in private practice (Persons, Roberts, Zalecki, & Brechwald, 2006). In eating disorders, patients in a trial with almost no exclusion criteria have a comparable outcome to earlier trials with stricter criteria (Fairburn et al., 2009; see Wilson, 2009) For the Omagh bomb post-traumatic stress disorder treatment series (Gil- lespie et al., 2002) there were very few exclusion criteria but the uncontrolled effect size (pre to post) for cognitive therapy was 2.47. This is very similar to uncontrolled effect sizes for cognitive therapy in more tightly defined populations seen in trials from research clinics which were 2.46 (Ehlers et al., 2003) in a trial of recent onset PTSD from road traffic accidents, and 2.69 in a trial of chronic PTSD from a variety of traumas (Ehlers, Clark, Hackmann, McManus, & Fennell, 2005). Similarly good treatment effects have been obtained in an outreach programme that identified and treated survivors of the London bombings with PTSD (Brewin et al., 2008). Exposure- based treatment for PTSD also generalizes well and rape crisis counsellors attained comparable results to those obtained by cognitive behaviour therapy experts after brief training plus regular supervision. The addition of cognitive restructuring did not enhance treatment outcome and hence was not found to be a necessary component of the treatment for dissemination (Foa et al., 2005). Based on these studies, it can be concluded that in several anxiety disorders and in eating disorders, and with some evidence for schizophrenia, appropriately trained therapists have obtained excellent outcomes in largely unrestricted patient populations if treatment integrity is maintained. Overall, there appears to be a methodological challenge in this area of dissemination research and the popular distinction between efficacy and effectiveness trials has not been helpful. Most effectiveness trials differ from efficacy trials in both the quality of treatment delivery and the patients who receive the treatments. The field needs studies that vary only one of these factors at a time to fully address the legitimate concerns of front-line clinicians.

Benefits of additional resources. Another reason that some clinicians believe that research trials are not applicable to clinical practice is that research trials are better resourced, therapists have a lower clinical load and much higher levels of expert continuing supervision. These concerns are likely to be valid. Furthermore, the additional resources facilitate a legitimate systematic (often session-by-session) measurement of clinical outcomes which is still relatively rare in routine mental healthcare settings. If it is not usually known whether patients in routine clinical settings are getting better or not, or that

outcomes are below those that should be expected from the litera- ture, then therapists may be unaware that there is a problem to correct. Healthcare advances on the routine monitoring of outcome rates should improve this situation. Even if standardised measures are used, high missing data rates are common (e.g., Stiles, Barkham, Twigg, Mellor-Clark, & Cooper, 2006) as data collection usually relies on patients providing data at the end of therapy, and a significant subgroup of patients do not attend the last scheduled session. Missing data are a problem in audits of outcome: As the people who are missing tend to have poorer outcome (Clark, 2008), missing data lead to the erroneous impression that patients improve more than is the case on average. Lack of data also precludes benchmarking outcome against those obtained in RCTs. A solution to this problem is the collection of simple outcome measures at each session as this ensures that data are available for the last point of contact. Dissem- ination work following the Omagh and London bombings however, showed that over 90% complete outcome data can be obtained in routine practice if clinicians feel that the measures are helpful for planning therapy and are given every session (Brewin et al., 2008; Gillespie et al., 2002). The measures must be sensitive to change and have norms for non-clinical samples so it is possible to establish recovery rates. This system might usefully be extended to other settings and assessments can be made via the computer or paper (see also Aarons, Sommerfeld, Hecht, Silovsky, & Chaffin, 2009).

Non-specific therapist effects

It has been argued that the scientific data do not demonstrate the superiority of specific interventions and that therapist effects may be more important than the therapy; consequently, it is argued there is no need to implement the specific protocols associated with RCTs (see House & Loewenthal, 2008; Parker & Fletcher, 2007; Wampold, 2001). However, the evidence points to the role of therapist training in obtaining good treatment outcomes rather than the therapist’s personal attributes and style. Studies show that as therapists get better at delivering the treatment, outcomes improve (e.g., DeRubeis et al., 2005). Since the therapists’ personal attributes and style have not changed this implies that the active ingredients for change are within the protocol as opposed to being attributable to non-specific effects of the therapist. In a recent study conducted jointly at the University of Pennsylvania, the original home of cognitive therapy (CT) and Vanderbilt University (DeRubeis et al., 2005), the outcome of patients of the less expe- rienced therapists was worse than that of the more experienced therapists. Increasing the therapists’ ability to deliver the treatment protocol was followed by an increment in rates of response. Within eating disorders, there is a progressive decrease in drop-out rate with the same therapist after increased training (Fairburn, 2009, personal communication). In panic disorder success rates improve from 10% to 60% after training (Grey, Salkovskis, Quigley, Clark, & Ehlers, 2008). Other research has shown a rather large impact of therapist experience on the outcome of cognitive therapy for personality disorders (Cohen’s d ¼ .73; Weertman & Arntz, 2007).

It can therefore be concluded that therapist effects are not immutable characteristics of the individual but as competence improves, outcome improves.

Further evidence against the argument that non-specific factors are more important than specific interventions comes from studies investigation the quality of the therapeutic alliance during the course of treatment. In studies of depression, therapeutic alliance does not drive subsequent reductions in depression but the alliance changes due to earlier symptom improvement (DeRubeis & Feeley, 1990; Feeley, DeRubeis, & Gelfand, 1999). Again this suggests that change in cognitive therapy is more a consequence of adherence to modality-specific therapist strategies than the quality of the

R. Shafran et al. / Behaviour Research and Therapy 47 (2009) 902–909 905

therapeutic relationship, which itself appears to be more a con- sequence of earlier symptom change.

Finally, non-specific factors might even be less non-specific than some theorists think. For instance, in a RCT on the treatment of BPD, therapeutic alliance appeared to be significantly higher in the CBT condition than in the psychodynamic condition (Spinhoven, Giesen-Bloo, Dyck van, Kooiman, & Arntz, 2007).

Overall, the available data suggest that effect size of therapist effects in CBT trials with trained therapists is small. The data from panic disorder suggest that 10% of variance in outcome is due to therapist effects for both CBT and medication and 40% is due to technique (Huppert et al., 2001; using the methods that Crits- Christoph used in his 1990 meta-analysis of therapist effects to calculate variance accounted for by therapist and then treatment vs placebo response to estimate treatment effects; Crits-Christoph, Beebe, & Connolly, 1990) In a review of over 6000 patients in clinical practice, 5% of variance in outcome is due to the therapist (Wampold & Brown, 2005). No therapist effects have been found in large RCTs of bulimia nervosa (e.g., Loeb et al., 2005; Wilson et al., 1999). However, there is a gap in the literature in that many CBT trials fail to analyse therapist effects and we recommend that they should do so in the future. There is a need to identify and reliably assess the skill level that is required for a therapist to obtain good outcomes and this may vary across disorders or treatment protocols.

Beliefs about the protocol

Clinicians often report using CBT and clients report receiving it, yet the content of the sessions often does not resemble the evidence-based protocol (Carroll & Rounsaville, 2007; Stobie et al., 2007) but has been significantly adapted. There are at least three reasons for this problem: First, many clinicians are reluctant to use diagnostic labels and/or are not trained to diagnose clients in the same formal, standardised way as in research trials. For example, many patients with PTSD receive CBT treatments for panic attacks or for depression in clinical practice without addressing the trauma. Second, many clinicians believe that there is no disadvantage to selecting only the parts of the treatment protocols that they like and mixing them with other interventions on the basis of their personal preference. This is reflected in almost all of 2281 prac- tising psychologists in the USA stating that they use a variety of approaches (Taylor & Chang, 2008) and is consistent with the earlier finding that only a minority of clinicians use CBT as their primary approach for eating disorders (von Ranson & Robinson, 2006). Clinicians often have concerns about the safety of exposure therapy that keep them from using that aspect of the protocol despite data to the contrary (Becker, Zayfert, & Anderson, 2004). Understandably these clinicians often argue that their personal experience with their particular client groups in their own work setting is superior to evidence collected on a specific disorder in a University setting in a well funded trial. Such beliefs are likely to contribute to ‘therapist drift’ from protocols which may negatively impact therapy (Waller, 2009) and make it difficult to maintain treatment integrity (Perepletchikova, Hilt, Chereji, & Kazdin, 2009). However, elsewhere in this special issue it is argued that it is necessary to allow for the local modification of protocols to facili- tate dissemination whilst retaining treatment efficacy (Ruzek & Rosen, 2009). Such local modification would allow clinicians’ to feel free to make professional decisions and increase their will- ingness to change their habits. Adhering to the protocol, when accompanied by appropriate supervision to allow the necessary adaptation at a local level appears to reduce staff turnover (Aarons et al., 2009). Balancing the different demands presents a significant challenge, Researchers need to provide practitioners with guide- lines that would facilitate local adaptations without losing efficacy.

Third, there is no requirement to state whether the protocol being imparted has been evaluated and is recommended in practice guidelines; clinicians attending workshops on ‘CBT’ will believe they are knowledgeable about the empirically supported protocols when they are not. There is no regulation of advertisements for training, and a content analysis of 261 unique advertisements for psychotherapy workshops appearing in two bimonthly clinical magazines in the USA found that only 8% (19/261) of the adver- tisements noted whether the treatment presented in the workshop was evidence-based (Cook, Weingardt, Jaszka, & Wiesner, 2008).

Gaps in knowledge about CBT

Gaps in our current knowledge about training, measuring competence, the mechanism of action of CBT and the minimum dose required for treatment limit the adoption of the protocols to clinical settings.

Lack of knowledge about how to effectively convey CBT skills

It has been argued above that patient outcome improves with improved clinician training and increasing competence. Most RCTs that have shown large effect sizes of treatment have used inten- sively trained therapists who are also supervised throughout the trial. Failing to do this well can have serious implications for the study (Jacobson & Hollon, 1996). Subsequent dissemination of treatment procedures is often by short workshops from the treat- ment developers or as part of general CBT courses. There has been a large expansion of professional society linked workshops and courses but very little research on best ways of how to teach therapeutic skills, and what sorts of improvement in skills can be expected after short or longer training workshops, or with and without ongoing supervision following the workshops obtained. There is therefore an urgent need for research on efficient ways of disseminating treatment procedures. The empirical question of how best to train therapists has rarely been addressed. The little research to date indicates that supervision is necessary to improve competency in CBT for substance abuse (Sholomskas et al., 2005; see Walters, Matson, Baer, & Ziedonis, 2005 for a review) and palliative care (Mannix et al., 2006) and to aid clinicians in learning motivational interviewing (Carroll & Rounsaville, 2007; Miller, Yahne, Moyers, Martinez, & Pirritano, 2004). There remains an urgent need for research on efficient ways of disseminating treat- ment (see for example the study by Dimeff et al., 2009). At a minimum, dissemination would be aided by having treatment manuals from the RCTs made easily available and at a reasonable cost.

It is likely that different amounts of training will be needed for different interventions. Research is required both on the mode of training (i.e., manuals, expert workshop, longer term courses with supervision, web-based programmes) and the level of training. It is likely that modest training is sufficient for some interventions (e.g., training graduate mental health workers to deliver behav- ioural activation) but further training is required for more complex interventions such as trauma-focused CBT, schema-focused therapy or CBT for psychosis. Rollinson et al. (2007) using an adherence scale especially developed for CBT for psychosis (Rollinson et al., 2008), found that clinicians trained in routine service settings did not demonstrate the full repertoire of skills recommended by the treatment manual, in contrast to those who had been specifically trained to offer therapy in a research trial. It is noteworthy that in studies reported in this special issue, twenty hours of training was insufficient to raise clinicians’ dialectical behaviour therapy skills to anything above ‘minimal-moderate’ (Dimeff et al., 2009) but fifty hours led to clinicians’ being competent to deliver schema-focused

R. Shafran et al. / Behaviour Research and Therapy 47 (2009) 902–909906

therapy (Nadort et al., 2009). Such research will need to measure patient outcomes (almost never done in existing training pro- grammes) and should explore the use of novel methods of training.

A different approach was taken in this Giesen-Bloo et al. (2006) trial. Therapists were regular mental healthcare therapists, patients were recruited from their clinics and treated in that same context, and the number of therapists was large (more than 20 in each condition). Many of the therapists were also involved in regular training and supervision of trainees. This meant that this trial a study mixed features of efficacy and effectiveness research. Though effects were smaller than in a more controlled environ- ment, one of the advantages of such an approach is that a whole new generation of trainers and supervisors is created that can disseminate the new treatment very quickly. Blended learning (systemically integrating multiple methods of information provi- sion), and the formation of community-partnership research are all innovative suggestions for facilitating training and dissemination (Andersson, 2009; Becker, Stice, Shaw, & Woda, 2009; Cucciare, Weingardt, & Villafranca, 2008).

Difficulties in measuring quality of therapy

Measures of the extent to which therapists adhere to a therapy and how well they implement the therapy need further develop- ment. The potential value of such measures was nicely illustrated by Arntz’s recent trial of CBT vs psychodynamic treatment for borderline personality disorder in which better adherence predicted better outcome with CBT and poorer outcome with psychodynamic treatment (Giesen-Bloo et al., 2006). Measuring quality of CBT is not easy. Treatment quality can be assessed using either measures of adherence or competence. Adherence refers to the extent to which the therapy is implemented as intended (do the therapists do what they are supposed to do), whereas competence refers to the quality with which that therapy is implemented (do the therapists do that therapy well). Adherence typically can be rated by non-experts trained to recognise the various components of the treatment, whereas competence typically requires ratings by clinicians expert in the particular modality. The Collaborative Study Psychotherapy Rating Scale (CSPRS), which consists of items assessing cognitive therapy, interpersonal psychotherapy, and the clinical management component of pharmacotherapy, among others, is a widely used measure of adherence to different treat- ments of depression (Hollon, Evans, Elkin, & Lowery, 1984). In a pair of studies, DeRubeis et al. have shown that adherence to modality- specific cognitive and behaviour strategies early in cognitive therapy leads to subsequent reductions in depression (DeRubeis & Feeley, 1990; Feeley et al., 1999).

Lack of knowledge about the mechanism of action of CBT

To date CBT research has mainly focused on developing and evaluating treatment programmes. Although the treatment devel- opers assume that their treatments work because they target key factors in the etiology or maintenance of the treated disorder, relatively little is known about whether these factors are actually the mechanisms of action. This is unfortunate as knowledge about mechanisms is likely to help researchers develop more efficient versions of the treatments. Definitive analyses of mechanisms require researchers to frequently measure hypothesized mediators and symptoms during the course therapy. In this way, it will be possible to determine whether change in a potential mediator precedes symptom change (Murphy, Cooper, Hollon, & Fairburn, 2009). This can then help in is the next stage in refining a treatment and it would aid dissemination by allowing treatments to be more efficient and cost-effective.

Lack of knowledge about the minimum dose that patients require

Researchers have started to develop and evaluate low therapist input versions of empirically validated CBT programmes, with some promising results. Examples are self-study assisted CBT, computer- ized CBT and Internet based CBT. Further research in this area is needed using rigorous designs. Although low-intensity treatment appear to be an obvious solution to the problem of dissemination, they may not be as helpful as one might think. What is the evidence for low-intensity treatment working? There is promising evidence that self-study assisted CBT (with a fully qualified therapist) and fewer individual sessions supplemented with self-study/compu- terised CBT can do as well as full CBT (see Andrews & Titov, 2009; Mitchell et al., 2008). However, guided self-help in general without seeing a skilled therapist in PTSD appears to have a poor outcome (Ehlers et al., 2003). A companion article in this issue presents data speaking to the efficacy of computerized CBT and Internet CBT that suggests that these approaches might have a role to play in treat- ment provision, particularly for mild to moderate cases (Andrews & Titov, 2009). However, the overall recovery rates seem to be less than with leading therapist-delivered CBT programmes. de Graaf et al. (2009) found Internet delivered computerized CBT for depression without therapist support to be disappointing in its effects and no better than usual GP care. It is necessary not just to report improvement but also for a significant proportion of patients to achieve outcomes as good as outcomes for full treatment (i.e., recovery). Some studies have reported low take-up or high drop-out rates suggesting the CCBT is only appropriate for a subset of patients (e.g., de Graaf et al., 2009). Although group and individual CBT are sometimes similarly effective (e.g., for obsessive compulsive disorder; Anderson & Rees, 2007), for depression, social phobia and more complex cognitive interventions the individual format appears to be more effective (e.g., Mörtberg et al., 2007; Stangier, Hei- denreich, Peitz, Lauterbach, & Clark, 2003). There does not appear to be any difference in outcome according to the format in which supplementary material is delivered (Hirai & Clum, 2006).

Given that there are low therapist input treatments that work for some people, it remains to be established how they should best be utilised within a healthcare system. Important questions remain such as when patients should be ‘stepped up’ to more intense treatments. Does going through a low-intensity intervention impact negatively on subsequent ones? The answer, for eating disorders, appears to be ‘no’ (Mitchell et al., 2008) although there are lower take-up rates of therapies offered for second-stage interventions after first-stage therapies have not been effective. The cost implications of such a ‘stepped care’ approach have also yet to be established because often those in low-intensity conditions utilise other external sources (Treasure et al., 1996). The ‘stepped care’ approach used within healthcare systems is one born of ignorance because it is not yet possible to match a person to treatment. Finding methods to establish which patients would benefit from lower intensity interventions and which require more face-to-face contact with highly skilled therapists is an important area for future research.

Key recommendations

A number of key recommendations are made to facilitate the utilisation of empirically supported CBT protocols in routine practice.

� Treatment developers should state how the existing trials address comorbidity and produce treatment guidelines and manuals; such manuals should be easily accessible and avail- able at a reasonable cost.

R. Shafran et al. / Behaviour Research and Therapy 47 (2009) 902–909 907

� Clinicians should have easy access to training in diagnostic assessments and routine outcome measures. They should be encouraged to use outcome measures at regular intervals during treatment to monitor progress. � Effectiveness studies should provide adequate training and

supervision for therapists when studying how well treatments work in routine clinical populations. � CBT trials and effectiveness studies should be analysed for

therapist effects and should establish the effects of levels of training on clinician competence and patient outcomes. � The skill level that is required for a therapist to obtain good

outcomes should be identified; this requires reliable assess- ment measures of competence. � There is a need for more research on efficient ways of

disseminating treatment procedures. � The mechanisms of action of efficacious treatments should be

studied. � Methods to establish which patients would benefit from lower

intensity interventions and which require more face-to-face contact are required.

The above recommendations are feasible to implement and not all require additional funds. Implementation of the recommenda- tions will help to ensure that people in need are receiving the empirically supported forms of CBT that stand the best chance of helping them overcome their mental health problems.

Acknowledgements

The stimulus for this paper was a meeting on the Dissemination of Psychological Treatments held at the Wellcome Trust, London (December 2006) convened by David M. Clark and Christopher G. Fairburn. We are grateful to the Wellcome Trust for funding this meeting and to Carolyn Black Becker for helpful comments on an earlier version of this manuscript.

References

Aarons, G. A., Sommerfeld, D. H., Hecht, D. B., Silovsky, J. F., & Chaffin, M. J. (2009). The impact of evidence-based practice implementation and fidelity monitoring on staff turnover: evidence for a protective effective. Journal of Consulting and Clinical Psychology, 77, 270–280.

Addis, M. E. (2002). Methods for disseminating research products and increasing evidence-based practice: promises, obstacles and future directions. Clinical Psychology: Science and Practice, 9, 367–378.

Addis, M. E., Hatgis, C., Cardelmil, E., Jacob, K., Krasnow, A. D., & Mansfield, A. (2006). Effectiveness of cognitive-behavioral treatment for panic disorder versus treatment as usual in a managed care setting: 2-year follow-up. Journal of Consulting and Clinical Psychology, 74, 377–385.

American Psychiatric Association. (2009). Treatment of patients with panic disorder (2nd ed.).. APA Practice Guidelines.

Andersson, G. (2009). Using the Internet to provide cognitive behaviour therapy. Behaviour Research and Therapy, 47, 175–180.

Anderson, R. A., & Rees, C. S. (2007). Group versus individual cognitive-behavioural treatment for obsessive-compulsive disorder: A controlled trial. Behaviour Research and Therapy, 45, 123–137.

Andrews, G. A., & Titov, N. (2009). Hit and miss: innovation and the dissemination of evidence based psychological treatments. Behaviour Research and Therapy, 47(11), 974–979.

van Asselt, A. D. I., Dirksen, C. D., Arntz, A., Giesen-Bloo, J. H., van Dyck, R., Spinhoven, Ph., et al. (2008). British Journal of Psychiatry, 192, 450–457.

Barlow, D. H. (2004). Psychological treatments. The American Psychologist, 59, 869–878.

Barlow, D. H., Levitt, J. T., & Bufka, L. F. (1999). The dissemination of empirically supported treatments: a view to the future. Behaviour Research and Therapy, 37(Suppl. 1), S147–S162.

Becker, C. B., Stice, E., Shaw, H., & Woda, S. (2009). Use of empirically supported interventions for psychopathology: can the participatory approach move us beyond the research-to-practice gap? Behaviour Research and Therapy, 47, 265–274.

Becker, C. B., Zayfert, C., & Anderson, E. (2004). A survey of psychologists’ attitudes towards and utilization of exposure therapy for PTSD. Behaviour Research and Therapy, 42, 277–292.

Berry, K., & Haddock, G. (2008). The implementation of the NICE guidelines for schizophrenia: barriers to the implementation of psychological interventions and recommendations for the future. Psychology and Psychotherapy: Theory Research and Practice, 81, 419–436.

Brewin, C. R., Scragg, P., Robertson, M., Thompson, M., d’Ardenne, P., & Ehlers, A. (2008). Promoting mental health following the London bombings: a screen and treat approach. Journal of Traumatic Stress, 21, 3–8.

Brown, T. A., Antony, M. M., & Barlow, D. H. (1995). Diagnostic comorbidity in panic disorder: effect on treatment outcome and course of comorbid diagnoses following treatment. Journal Consulting and Clinical Psychology, 63, 408–418.

Carroll, K. M., & Rounsaville, B. J. (2007). A vision of the next generation of behavioral therapies research in the addictions. Addiction, 102, 850–862.

Chorpita, B. J., & Nakamura, B. J. (2004). Four considerations for dissemination of intervention innovations. Clinical Psychology: Science and Practice, 11, 364–367.

Clark, D. M. Improving access to psychological therapies in England: a government initiative. Presentation, Symposium 57, 42nd Annual Convention, Association of Behavioral and Cognitive Therapy, Orlando, 13–16 November, 2008.

Cook, J. M., Wiengardt, K. R., Jaszka, J., & Weisner, M. (2008). A content analysis of advertisements for psychotherapy workshops: implications for disseminating empirically supported treatments. Journal of Clinical Psychology, 64, 296–307.

Crits-Christoph, P., Beebe, K. L., & Connolly, M. B. (1990). Therapist effects in the treatment of drug dependence: implications for conducting comparative treatment studies. NIDA Research Monograph, 104, 39–49.

Crow, S., Mussell, M. P., Peterson, C., Knopke, A., & Mitchell, J. (1999). Prior treat- ment received by patients with bulimia nervosa. International Journal of Eating Disorders, 25, 39–44.

Crow, S., & Peterson, C. B. (2009). Refining treatments for eating disorders. American Journal of Psychiatry, 166, 266–267.

Cucciare, M. A., Weingardt, K. R., & Villafranca, S. (2008). Using blended learning to implement evidence-based psychotherapies. Clinical Psychology: Science and Practice, 15, 299–307.

Currin, L., Waller, G., Treasure, J., Nodder, J., Stone, C., Yeomans, M., et al. (2007). The use of guidelines for dissemination of ‘best practice’ in primary care of patients with eating disorders. International Journal of Eating Disorders, 40, 476–479.

DeRubeis, R. J., & Feeley, M. (1990). Determinants of change in cognitive therapy for depression. Cognitive Therapy & Research, 14, 469–482.

DeRubeis, R. J., Hollon, S. D., Amsterdam, J. D., Shelton, R. C., Young, P. R., Salomon, R. M., et al. (2005). Cognitive therapy vs. medications in the treatment of moderate to severe depression. Archives of General Psychiatry, 62, 409–416.

Dimeff, L. A., Koerner, K., Woodcock, E. A., Beadnell, B., Brown, M. Z., Skutch, J. M., et al. (2009). Which training method works best? A randomized controlled trial comparing three methods of training clinicians in dialectical behavior therapy skills. Behaviour Research and Therapy, 47(11), 921–930.

Dreessen, L., & Arntz, A. (1998). The impact of personality disorders on treatment outcome of anxiety disorders: best-evidence synthesis. Behaviour Research and Therapy, 36, 483–504.

Dreessen, L., Arntz, A., Luttels, C., & Sallaerts, S. (1994). Personality disorders do not influence the results of cognitive behavior therapies for anxiety disorders. Comprehensive Psychiatry, 35, 265–274.

Dreessen, L., Hoekstra, R., & Arntz, A. (1997). Personality disorders do not influence the results of cognitive and behavior therapy for obsessive compulsive disorder. Journal of Anxiety Disorders, 11, 503–521.

Duffy, M., Gillespie, K., & Clark, D. M. (2007). Post-traumatic stress disorder in the context of terrorism and other civil conflict in Northern Ireland: randomised controlled trial. British Medical Journal, 334(7604), 1147.

Ehlers, A., Clark, D. M., Hackmann, A., McManus, F., & Fennell, M. (2005). Cognitive therapy for post-traumatic stress disorder: development and evaluation. Behaviour Research and Therapy, 43, 413–431.

Ehlers, A., Clark, D. M., Hackmann, A., McManus, F., Fennell, M., Herbert, C., et al. (2003). A randomized controlled trial of cognitive therapy, a self-help booklet, and repeated assessments as early interventions for posttraumatic stress disorder. Archives of General Psychiatry, 60, 1024–1032.

Ehlers, A., Gene-Cos, N., & Perrin, S. (2009). Low recognition of posttraumatic stress disorder in primary care. London Journal of Primary Care, 1, 36–42.

Ellis, P. M., Hickie, I., & Smith. (2003). Summary of guideline for the treatment of depression. Australasian Psychiatry, 11(1), 34–38.

Fairburn, C. G. (2008). Cognitive behavior therapy and eating disorders. New York: Guilford Press.

Fairburn, C. G., Cooper, Z., Doll, H. A., O’Connor, M. E., Bohn, K., Hawker, D. M., et al. (2009). Transdiagnostic cognitive-behavioral therapy for patients with eating disorders: a two-site trial with 60-week follow-up. American Journal of Psychiatry, 166, 311–319.

Farrell, J. M., Shaw, I., & Webber, M. A. (2009). A schema-focused approach to group psychotherapy for outpatients with borderline personality disorder: a randomized controlled trial. Journal of Behavior Therapy and Experimental Psychiatry, 40, 317–328.

Feeley, M., DeRubeis, R. J., & Gelfand, L. A. (1999). The temporal relation of adher- ence and alliance to symptom change in cognitive therapy for depression. Journal Consulting and Clinical Psychology, 67, 578–582.

Foa, E. B., Hembree, E. A., Cahill, S. P., Rauch, S. A., Riggs, D. S., Feeny, N. C., et al. (2005). Randomized trial of prolonged exposure for PTSD with and without cognitive restructuring: outcome at academic and community clinics. Journal of Consulting and Clinical Psychology, 73, 955–964.

Giesen-Bloo, J., van Dyck, R., Spinhoven, P., van Tilburg, W., Dirksen, C., van Asselt, T., et al. (2006). Outpatient psychotherapy for borderline personality disorder:

R. Shafran et al. / Behaviour Research and Therapy 47 (2009) 902–909908

randomized trial of schema-focused therapy vs. transference-focused psycho- therapy. Archives of General Psychiatry, 63, 649–658.

Gillespie, K., Duffy, M., Hackmann, A., & Clark, D. M. (2002). Community based cognitive therapy in the treatment of posttraumatic stress disorder following the Omagh bomb. Behaviour Research and Therapy, 40, 345–357.

Goisman, R. M., Warshaw, M. G., & Keller, M. B. (1999). Psychosocial treatment prescriptions for generalized anxiety disorder, panic disorder and social phobia, 1991–1996. American Journal of Psychiatry, 156, 1819–1821.

de Graff, L. E., Gerhards, S. A. H., Arntz, A., Riper, H., Metsemakers, J. F. M., Evers, S. M. A. A., et al. (2009). Clinical effectiveness of online computerized cognitive behavioural therapy without support for depression in primary care: a randomized trial. British Journal of Psychiatry, 195, 73–80.

Grey, N., Salkovskis, P., Quigley, A., Clark, D. M., & Ehlers, A. (2008). Dissemination of cognitive therapy for panic disorder in primary care. Behavioural and Cognitive Psychotherapy, 36, 509–520.

Grilo, C. M., Pagano, M. E., Skodol, A. E., Sanislow, C. A., McGlashan, T. H., Gunderson, J. G., et al. (2007). Natural course of bulimia nervosa and eating disorder not otherwise specified: 5-year prospective study of remissions, relapses, and the effects of personality disorder psychopathology. Journal of Clinical Psychiatry, 68, 738–746.

Hahlweg, K., Fiegenbaum, W., Frank, M., Schroeder, B., & von Witzleben, I. (2001). Short- and long-term effectiveness of an empirically supported treatment for agoraphobia. Journal of Consulting and Clinical Psychology, 69, 375–382.

Hirai, M., & Clum, G. A. (2006). A meta-analytic study of self-help interventions for anxiety problems. Behavior Therapy, 37, 99–111.

Hollon, S. D., Evans, M. D., Elkin, I., Lowery, A. System for rating therapies for depression. Paper presented at the 92nd Annual Convention of the American Psychological Association, Toronto, Ontario, Canada, August 1984.

House, R., & Loewenthal, D. (2008). Against and for CBT: Towards a constructive dialogue? Hertfordshire, UK: PCCS Books.

Huppert, J. D., Bufka, L. F., Barlow, D. H., Gorman, J. M., Shear, M. K., & Woods, S. W. (2001). Therapists, therapist variables and cognitive behavioral therapy outcome in a multicenter trial for panic disorder. Journal of Consulting and Clinical Psychology, 69, 747–755.

Insel, T. R. (2009). Translating scientific opportunity into public health impact: a strategic plan for research on mental illness. Archives of General Psychiatry, 66, 128–133.

Jacobson, N. S., & Hollon, S. D. (1996). Prospects for future comparisons between drugs and psychotherapy: lessons from the CBT-versus-pharmacotherapy exchange. Journal of Consulting and Clinical Psychology, 64, 104–108.

Kessler, R. C., Chiu, W. T., Demler, O., & Walters, E. E. (2005). Prevalence, severity and comorbidity of 12 month DSM-IV disorders in the national comorbidity survey replication. Archives of General Psychiatry, 62, 617–627.

Kessler, R. C., Merikangas, K. R., & Wang, P. S. (2007). Prevalence, comorbidity and service utilization of mood disorders in the United States at the beginning of the twenty-first century. Annual Review of Clinical Psychology, 3, 137–158.

Kroenke, K. (2007). Efficacy of treatment for somatoform disorders: a review of randomized controlled trials. Psychosomatic Medicine, 69, 881–888.

Layard, R., Clark, D. M., Knapp, M., & Mayraz, G. (October 2007). Cost–benefit analysis of psychological therapy. Centre for Economic Performance Discussion Paper No. 829Web site. The Centre for Economic Performance Publications Unit. http:// cep.lse.ac.uk.

Lincoln, T. M., Rief, W., Hahlweg, K., Frank, M., von Witzleben, I., Schroeder, B., et al. (2003). Effectiveness of an empirically supported treatment for social phobia in the field. Behaviour Research and Therapy, 41, 1251–1269.

Linehan, M. M., Comtois, K. A., Murray, A. M., Brown, M. Z., Gallop, R. J., Heard, H. L., et al. (2006). Two-year randomized controlled trial and follow-up of dialectical behavior therapy vs. therapy by experts for suicidal behaviors and borderline personality disorder. Archives of General Psychiatry, 63, 757–766.

Loeb, K. L., Wilson, G. T., Labouvie, E., Pratt, E. M., Hayaki, J., Walsh, B. T., et al. (2005). Therapeutic alliance and treatment adherence in two interventions for bulimia nervosa: a study of process and outcome. Journal of Consulting and Clinical Psychology, 73, 1097–1107.

McAlpine, D. E., Schroder, K., Pankratz, S., & Maurer, M. (2004). Survey of regional health care providers on selection of treatment for bulimia nervosa. Interna- tional Journal of Eating Disorders, 35, 27–32.

McHugh, R. K., Murray, H. W., & Barlow, D. H. (2009). Balancing fidelity and adaptation in the dissemination of empirically-supported treatments: the promise of transdiagnostic interventions. Behaviour Research and Therapy, 47(11), 946–953.

Mannix, K. A., Blackburn, I. M., Garland, A., Gracie, J., Moorey, S., Reid, B., et al. (2006). Effectiveness of brief training in cognitive behaviour therapy techniques for palliative care practitioners. Palliative Medicine, 20, 579–584.

Miklowitz, D. J. (2006). A review of evidence-based psychosocial interventions for bipolar disorder. Journal of Clinical Psychiatry, 67(Suppl. 11), 28–33.

Miller, W. R., Yahne, C. E., Moyers, T. B., Martinez, J., & Pirritano, M. (2004). A randomized trial of methods to help clinicians learn motivational interviewing. Journal of Consulting and Clinical Psychology, 72, 1050–1062.

Mitchell, J. E., Crosby, R. D., Wonderlich, S. A., Crow, S., Lancaster, K., Simonich, H., et al. (2008). A randomized trial comparing the efficacy of cognitive-behavioral therapy for bulimia nervosa delivered via telemedicine versus face-to-face. Behaviour Research and Therapy, 46, 581–592.

Morrison, A. P., Renton, J. C., Williams, S., Dunn, H., Knight, A., Kreutz, M., et al. (2004). Delivering cognitive therapy to people with psychosis in a community

mental health setting: an effectiveness study. Acta Psychiatrica Scandinavia, 110, 36–44.

Mörtberg, E., Clark, D. M., Sundin, O., & Aberg Wistedt, A. (2007). Intensive group cognitive treatment and individual cognitive therapy vs. treatment as usual in social phobia: a randomized controlled trial. Acta Psychiatrica Scandinavia, 115, 142–154.

Murphy, R., Cooper, Z., Hollon, S. D., & Fairburn, C. G. (2009). How do psychological treatment work? Investigating mediators of change. Behaviour Research and Therapy, 47, 1–5.

Mussell, M. P., Crosby, R. D., Crow, S., Knope, A. J., Peterson, C. B., Wonderlich, S. A., et al. (2000). Utilization of empirically supported psychotherapy treatments for individuals with eating disorders: a survey of psychologists. International Journal of Eating Disorders, 27, 230–237.

Myhr, G., & Payne, K. (2006). Cost-effectiveness of cognitive behavioural therapy for mental disorders: implications for public health care funding policy in Canada. Canadian Journal of Psychiatry, 51, 662–670.

Nadort, M., van Dyck, R., Smit, J., Giesen-Bloo, J., Eikelenboom, M., Wensing, M., et al. (2009). Three preparatory studies for promoting implementation of outpatient schema therapy for borderline personality disorder in General Mental Health Care. Behaviour Research and Therapy, 47(11), 938–945.

NICE. (2003). Schizophrenia: Core interventions in the treatment and management of schizophrenia in primary and secondary care. [Full guideline]. London: Gaskell and The British Psychological Society.

NICE. (2004). London. NICE Clinical Guideline No. 9. Eating disorders – Core inter- ventions in the treatment and management of anorexia nervosa, bulimia nervosa and related eating disorders. National Institute for Clinical Excellence/Gaskell and The British Psychological Society.

NICE. (2005). Clinical guideline 26: Posttraumatic stress disorder: The management of PTSD in adults and children in primary and secondary care. London: Gaskell and The British Psychological Society. http://guidance.nice.org/CG26.

NICE. (2009). Schizophrenia: core interventions in the treatment and management of schizophrenia in primary and secondary care. NICE Clinical guideline 82. London: National Institute for Health and Clinical Excellence.

Parker, G., & Fletcher, K. (2007). Treating depression with the evidence-based psychotherapies: a critique of the evidence. Acta Psychiatrica Scandinavia, 115, 352–359.

Perepletchikova, F., Hilt, L. M., Chereji, E., & Kazdin, A. E. (2009). Barriers to implementing treatment integrity procedures: survey of treatment outcome researchers. Journal of Consulting and Clinical Psychology, 77, 212–218.

Persons, J. B., Roberts, N. A., Zalecki, C. A., & Brechwald, W. A. (2006). Naturalistic outcome of case formulation-driven cognitive-behavior therapy for anxious depressed outpatients. Behavior Research and Therapy, 44, 1041–1051.

von Ranson, K. M., & Robinson, K. E. (2006). Who is providing what type of psychotherapy to eating disorder clients? A survey. International Journal of Eating Disorders, 39, 27–34.

Rollinson, R., Haig, C., Warner, R., Garety, P., Kuipers, E., Freeman, D., et al. (2007). The application of cognitive behavioural therapy for psychosis in clinical and research settings. Psychiatry Services, 58, 1297–1302.

Rollinson, R., Smith, B., Steel, S., Jolley, S., Onwumere, J., Freeman, D., et al. (2008). Measuring adherence and competence in CBT for psychosis: a psychometric analysis of an adherence scale. Behavioural and Cognitive Psychotherapy, 36, 163–178.

Rowe, S. L., Jordan, J., McIntosh, V., Carter, F. A., Bulik, C. M., & Joyce, P. R. (2008). Impact of borderline personality disorder on bulimia nervosa. Australian and New Zealand Journal of Psychiatry, 42, 1021–1029.

Roy-Byrne, P. P., Craske, M. G., Stein, M. B., Sullivan, G., Bystritsky, A., Katon, W., et al. (2005). A randomized effectiveness trial of cognitive-behavioral therapy and medication for primary care panic disorder. Archives of General Psychiatry, 62, 290–298.

Ruzek, J. I., & Rosen, R. C. (2009). Disseminating evidence-based treatments for PTSD in organizational settings: a high priority focus area. Behaviour Research and Therapy, 47(11), 980–989.

Sholomskas, D. E., Syracuse-Siewert, G., Rounsaville, B. J., Ball, S. A., Nuro, K. F., & Carroll, K. M. (2005). We don’t train in vain: a dissemination trial of three strategies of training clinicians in cognitive-behavioral therapy. Journal of Consulting and Clinical Psychology, 73, 106–115.

Spinhoven, Ph, Giesen-Bloo, J., Dyck van, R., Kooiman, K., & Arntz, A. (2007). The therapeutic allliance in schema-focused therapy and transference-focused psychotherapy for borderline personality disorder. Journal of Consulting and Clinical Psychology, 75, 104–115.

Stangier, U., Heidenreich, T., Peitz, M., Lauterbach, W., & Clark, D. M. (2003). Cognitive therapy for social phobia: individual versus group treatment. Behaviour Research and Therapy, 41, 991–1007.

Stiles, W. B., Barkham, M., Twigg, E., Mellor-Clark, J., & Cooper, M. (2006). Effectiveness of cognitive-behavioural, person-centred and psychodynamic therapies as practiced in UK National Health Service settings. Psychological Medicine, 36, 555–566.

Stirman, S. W., Crits-Christoph, P., & DeRubeis, R. J. (2004). Achieving successful dissemination of empirically supported psychotherapies: a synthesis of dissemination theory. Clinical Psychology: Science and Practice, 11, 343–359.

Stirman, S. W., DeRubeis, R. J., Crits-Christoph, P., & Rothman, A. (2005). Can the randomized controlled trial literature generalize to nonrandomized patients? Journal of Consulting and Clinical Psychology, 73, 127–135.

Stobie, B., Taylor, T., Quigley, A., Ewing, S., & Salkovskis, P. M. (2007). ‘Contents my vary’: a pilot study of treatment histories of OCD patients. Behavioural and Cognitive Psychotherapy, 35, 273–282.

R. Shafran et al. / Behaviour Research and Therapy 47 (2009) 902–909 909

Taylor, C. B., & Chang, V. Y. (2008). Issues in the dissemination of cognitive-behavior therapy. Nordic Journal of Psychiatry, 62(Suppl. 47), 37–44.

The Healthcare Commission and Commission for Social Care Inspection. (July 2007). Improvement review of adult community mental health services 2005/2006. www.healthcarecommission.org.uk.

Treasure, J., Schmidt, U., Troop, N., Tiller, J., Todd, G., & Turnbull, S. (1996). Sequential treatment for bulimia nervosa incorporating a self care manual. British Journal of Psychiatry, 168, 94–98.

Wade, W. A., Treat, T. A., & Stuart, G. L. (1998). Transporting an empirically sup- ported treatment for panic disorder to a service clinic setting: a benchmarking strategy. Journal of Consulting and Clinical Psychology, 66, 231–239.

Waller, G. (2009). Evidence-based treatment and therapist drift. Behaviour Research and Therapy, 47, 119–127.

Walters, S. T., Matson, S. A., Baer, J. S., & Ziedonis, D. M. (2005). Effectiveness of workshop training for psychosocial addiction treatments: a systematic review. Journal of Substance Abuse Treatment, 29, 283–293.

Wampold, B. E. (2001). The great psychotherapy debate: Models, methods, and find- ings. Mahwah, NJ: Erlbaum.

Wampold, B. E., & Brown, G. S. (2005). Estimating variability in outcomes attrib- utable to therapists: a naturalistic study of outcomes in managed care. Journal of Consulting and Clinical Psychology, 73, 914–923.

Wang, P. S., Lane, M., Olfson, M., Pincus, H. A., Wells, K. B., & Kessler, R. C. (2005). Twelve month use of mental health services in the USA. Results from the National Comorbidity Survey Replication. Archives of General Psychiatry, 62, 629–640.

Weertman, A., & Arntz, A. (2007). Effectiveness of treatment of childhood memories in cognitive therapy for personality disorders: a controlled study contrasting

methods focusing on the present and methods focusing on childhood memo- ries. Behaviour Research and Therapy, 45, 2133–2143.

Weertman, A., Arntz, A., Schouten, E., & Dreessen, L. (2005). Influences of beliefs and personality disorders on treatment outcome in anxiety patients. Journal of Consulting and Clinical Psychology, 73, 936–944.

Weingardt, K. R. (2004). The role of instructional design and technology in the dissemination of empirically supported, manual-based treatments. Clinical Psychology: Science and Practice, 11, 313–331.

Weissman, M. M., Verdeli, H., Gameroff, M. J., Beldsoe, S. E., Betts, K., Mufson, L., et al. (2006). National survey of psychotherapy training in psychiatry, psychology and social work. Archives of General Psychiatry, 63, 925–934.

Weisz, J. R., Weersing, V. R., & Henggleler, S. W. (2005). Jousting with straw men: comment on Westen, Novotny and Thompson-Brenner (2004). Psychological Bulletin, 131, 418–426, discussion 427–433.

Wilson, G. T. (2007). Manual-based treatment: evolution and evaluation. In T. A. Treat, R. R. Bootzin, & T. B. Baker (Eds.), Psychological clinical science: Papers in honor of Richard M. McFall. Mahwah, NJ: Lawrence Erlbaum.

Wilson, G. T. (2009). What BN research is needed. In C. M. Grilo, J. E. Mitchell (Eds.), The treatment of eating disorders: A clinical handbook. The Guilford Press.

Wilson, G. T., Loeb, K. L., Walsh, B. T., Labouvie, E., Petkova, E., Liu, X., et al. (1999). Psychological versus pharmacological treatments of bulimia nervosa: predictors and processes of change. Journal of Consulting and Clinical Psychology, 67, 451–459.

Wykes, T., Steel, C., Everitt, B., & Tarrier, N. (2008). Cognitive behaviour therapy for schizophrenia: effect sizes, clinical models, and methodological rigor. Schizo- phrenia Bulletin, 34, 523–537.