Article Review

Clinical Psychology Review 42 (2015) 62–71

Contents lists available at ScienceDirect

Clinical Psychology Review

The effects of cognitive behavior therapy for adult depression on dysfunctional thinking: A meta-analysis

Ioana A. Cristea a,b,⁎, Marcus J.H. Huibers c,d, Daniel David a, Steven D. Hollon e, Gerhard Andersson f,g, Pim Cuijpers c,d

a Department of Clinical Psychology and Psychotherapy, Babeş-Bolyai University, Cluj-Napoca, Romania b Clinical Psychology Branch, Department of Surgical, Medical, Molecular and Critical Pathology, University of Pisa, Pisa, Italy c Department of Clinical Psychology, VU University, Amsterdam, The Netherlands d EMGO Institute for Health and Care Research, The Netherlands e Department of Psychology, Vanderbilt University, Nashville, USA f Department of Behavioural Sciences and Learning, Linköping University, Sweden g Department of Clinical Neuroscience, Psychiatry Section, Karolinska Institutet, Stockholm, Sweden

H I G H L I G H T S

• We examined the effects of CBT for depression on dysfunctional thoughts. • CBT for depression had a robust medium effect on dysfunctional thoughts. • Effects on dysfunctional thoughts were strongly associated with those on depression. • The only difference between CBT and other psychotherapies was found for the DAS. • There were no differences between CBT and pharmacotherapy on dysfunctional thoughts.

⁎ Corresponding author at: Department of Clinical Psy Branch, Department of Surgical, Medical, Molecular and C

E-mail address: [email protected] (I.A. Cristea).

http://dx.doi.org/10.1016/j.cpr.2015.08.003 0272-7358/© 2015 Elsevier Ltd. All rights reserved.

a b s t r a c t

a r t i c l e i n f o

Article history:

Received 27 January 2015 Received in revised form 24 June 2015 Accepted 12 August 2015 Available online 14 August 2015

Keywords: Cognitive behavior therapy Dysfunctional thoughts: depression Meta-analysis

Background: It is not clear whether cognitive behavior therapy (CBT) works through changing dysfunctional thinking. Although several primary studies have examined the effects of CBT on dysfunctional thinking, no meta-analysis has yet been conducted. Method: We searched for randomized trials comparing CBT for adult depression with control groups or with other therapies and reporting outcomes on dysfunctional thinking. We calculated effect sizes for CBT versus con- trol groups, and separately for CBT versus other psychotherapies and respectively, pharmacotherapy. Results: 26 studies totalizing 2002 patients met inclusion criteria. The quality of the studies was less than optimal. We found a moderate effect of CBT compared to control groups on dysfunctional thinking at post-test (g = 0.50; 95% CI: 0.38–0.62), with no differences between the measures used. This result was maintained at follow-up

(g = 0.46; 95% CI: 0.15–0.78). There was a strong association between the effects on dysfunctional thinking and those on depression. We found no significant differences between CBT and other psychotherapies (g = 0.17; p = 0.31), except when restrict in outcomes to the Dysfunctional Attitudes Scale (g = 0.29). There also was no difference between CBT and pharmacotherapy (g = 0.04), though this result was based on only 4 studies. Discussion: While CBT had a robust and stable effect on dysfunctional thoughts, this was not significantly different from what other psychotherapies or pharmacotherapy achieved. This result can be interpreted as confirming the primacy of cognitive change in symptom change, irrespective of how it is attained, as well as supporting the idea that dysfunctional thoughts are simply another symptom that changes subsequent to treatment.

© 2015 Elsevier Ltd. All rights reserved.

chology and Psychotherapy, Babeş-Bolyai University, Republicii Street 37, 400015 Cluj-Napoca, Romania, Clinical Psychology ritical Pathology, University of Pisa, Via Roma 67, 56126 Pisa, Italy.

63I.A. Cristea et al. / Clinical Psychology Review 42 (2015) 62–71

Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 2. Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

2.1. Identification and selection of studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 2.2. Quality assessment and data extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 2.3. Meta-analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

3. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 3.1. Selection of studies and characteristics of included studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 3.2. Quality assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 3.3. The effects of CBT versus control groups on dysfunctional thinking at post-test and follow-up . . . . . . . . . . . . . . . . . . . . . . . 65 3.4. Association between effects on depression and effects on dysfunctional thinking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 3.5. The effects of CBT versus other treatments on dysfunctional thinking at post-test and follow-up . . . . . . . . . . . . . . . . . . . . . . 67

4. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Role of funding sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 Conflict of interest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

1. Introduction

It is well-established that cognitive behavior therapy (CBT) is effec- tive in the treatment of adult depression (Cuijpers, Berking, Andersson, Quigley, Kleiboer, & Dobson, 2013a), and that it is by far the most researched type of psychotherapy for depression. More than 100 ran- domized trials have shown that CBT is more effective than no treatment (Cuijpers et al., 2013a; Gloaguen, Cottraux, Cucherat, & Blackburn, 1998), that short-term effects are comparable to those of other effective psychotherapies (Barth et al., 2013; Braun, Gregor, & Tran, 2013; Cuijpers, van Straten, Andersson, & van Oppen, 2008a) and pharmaco- therapy (Cuijpers, Sijbrandij, Koole, Andersson, Beekman, & Reynolds, 2013c), that CBT is also efficient in different formats such as guided self-help (Barth et al., 2013; Cuijpers, Donker, van Straten, Li, & Andersson, 2010) and that the effects of acute phase CBT without con- tinuation are comparable to those of continuation pharmacotherapy at the longer-term (Cuijpers, Hollon, van Straten, Bockting, Berking, & Andersson, 2013b).

It is less clear, however, how CBT achieves therapeutic effects. Cogni- tive therapists focus on the impact a patient's present dysfunctional thoughts have on current behavior and future functioning. CBT is centered around cognitive restructuring, consisting of evaluating, em- phatically challenging, and modifying a patient's dysfunctional beliefs. Change of maladaptive beliefs and their consequent conversion into adaptive ones is assumed to be the main vehicle by which CBT produces symptom change (Beck & Dozois, 2011), although behavioral interven- tions are considered to be essential as well. In fact, Beck & Dozois (2011) claimed that research findings “support the primacy of cognition in therapeutic change and are consistent with the idea that there are myriad ways in which to modify cognition” (p.405). Nonetheless, this argument has also been cogently contested. At least two reviews (Kazdin, 2007; Longmore & Worrell, 2007) contended there was not sufficient evidence to sustain the idea that CBT causes symptom im- provements by means of changing cognitions. Indeed, in no uncertain terms, Kazdin (2007) stated that “perhaps we can state more confident- ly now than before that whatever may be the basis of changes with CT, it does not seem to be the cognitions as originally proposed” (p.8).

Moreover, although there are indications that change in dysfunc- tional thoughts is associated with the outcomes of CBT, this evidence is also inconclusive with respect to causality. Some studies have found that change in dysfunctional thinking precedes change in depression during therapy (DeRubeis, Evans, Hollon, Garvey, Grove, & Tuason, 1990; Furlong & Oei, 2002), but some systematic reviews have questioned this (Oei & Free, 1995; Whisman, 1993). It is not clear, there- fore, whether change in dysfunctional thinking is the mechanism through which CBT works to reduce depression, or if an improvement

of depression results in less dysfunctional thinking. In other words, is change in dysfunctional thinking the cause or the consequence of clinical improvement during CBT?

As hard as proving any of these contrary positions might be, the debate on the status of cognitive change in CBT and in psychotherapy in general is further complicated by the difficulty to establish what kind of evidence is actually needed to offer support for the cognitive model of therapeutic change. Older approaches (Whisman, 1993) contended that in order to infer that change in cognitions is indeed the mechanism supporting symptom change, two fundamental tenets have to be met. First, change in dysfunctional thoughts must covary with symptom reduction. Secondly, he argued that this change in thoughts must be specific to CBT, but that is not necessarily true because other therapies could in principle also work through changing thoughts. However, newer conceptualizations (Lorenzo-Luaces, German, & DeRubeis, 2015) of this problem have reasoned that there are in fact four questions that have to be tackled in evaluating the state of the evidence for the cognitive model regarding its validity and specificity: 1. Differential efficacy of procedures on symptoms (i.e., are CBT procedures more efficient than other treatment procedures in reducing depressive symptoms?); 2. Differential efficacy of procedures on cogni- tions (i.e., do CBT procedures lead to more cognitive change than proce- dures in other therapies?); 3. Effects of cognitive change on symptom change (i.e., do changes in cognitions lead to changes in symptoms of depression, irrespective of the procedures that engendered them?); 4. Cognitive specificity (i.e., do changes in cognitions that are the result of CBT restructuring procedures lead to larger changes in symptoms than changes in cognition resulting from non-CBT procedures?).

Studies in which patients are randomized to CBT or to another treat- ment might in theory answer all of these questions if conducted proper- ly, as suggested by Kazdin (2007) (e.g., establishing a timeline, multiple measurements of the outcome and mediator, accounting for confound- ing variables and reverse causality). However, individual studies often do not have sufficient power to reliably examine these differential ef- fects, and consequently paint an inconsistent picture, with some studies finding differences between CBT and other therapies on some measures of thoughts (but not on others), and others not finding differences. Meta-analyses, statistically combining more individual studies, can on the other hand help with the problem of statistical power. While traditional meta-analyses, aggregating averaged data from individual studies, cannot offer any solutions to questions 3 and 4, they can, nonetheless, provide clearer and more reliable answers to the first two questions.

Regarding the first question, three recent meta-analyses found no differences between CBT and other therapies on symptoms of depres- sion (Barth et al., 2013; Braun et al., 2013; Cuijpers et al., 2013a). The

64 I.A. Cristea et al. / Clinical Psychology Review 42 (2015) 62–71

second question has not been approached in a meta-analysis so far. Most studies have shown that CBT for depression does have an effect on dysfunctional thinking, but until now no meta-analysis has been done to examine magnitude of these effects and furthermore whether they are unique or specific to CBT. The last point is essential, given the fundamental opposition between researchers claiming that cognitive change, regardless of how it is achieved, is fundamental to symptom change (Beck & Dozois, 2011; David & Montgomery, 2011; Lorenzo-Luaces et al., 2015) and those arguing that cognitive change is non-essential for symptom change and most likely another conse- quence accompanying successful treatment, due to the action of other causal factors (Garratt, Ingram, Rand, & Sawalani, 2007; Kazdin, 2007; Longmore & Worrell, 2007; Wampold, 2001). It is not possible to exam- ine causality directly using the available evidence primarily because the designs of most individual studies were such that that the outcomes and the presumed mechanisms of change could not truly and reliably be parsed (e.g., measured at the same time point, confounding variables were not accounted for). Cognitive specificity, on the other hand, can be examined using the available evidence, but a distinction should be made between specificity and causality. It was argued that a mechanism like cognitive change can be causal without being specific and specific without being causal (Hollon, DeRubeis, & Evans, 1987). In this vein, finding little evidence for cognitive specificity does not rule out a causal role for cognitive change (although it provides no support for it either). Nevertheless, it does suggest that the original model that cognitive therapy worked via producing cognitive change and that it was distinct from other treatments in that regard may be overly simplistic and not very accurate. It also raises important questions about the added benefits of cognitive procedures, typical of CBT protocols, in changing thoughts.

It has been suggested that a good empirical way to test this “cognitive specificity” conjecture is to examine cognitive change comparatively, contrasting CBT against other therapies, or respectively, pharmacothera- py (Garratt et al., 2007). Particularly the contrast between CBT and phar- macotherapy is relevant, as it was argued that even if dysfunctional thoughts may also change as a consequence of pharmacotherapy, claims that these changes would carry a causal significant have yet to be sub- stantiated (Garratt et al., 2007). While it is theoretically possible that medication carries its effects on depressive symptoms via dysfunctional thoughts and recent reformulations of the general theory of CBT could accommodate this possibility (Beck & Haigh, 2014), these claims have yet to be corroborated empirically.

We decided therefore to conduct a meta-analysis of studies examining the effects of CBT for adult depression on dysfunctional thinking, in which we included studies comparing CBT with control groups, as well studies in which CBT was compared with other treatments (other psychother- apies or pharmacotherapy).

2. Methods

2.1. Identification and selection of studies

We used and updated a database of randomized trials on the psycho- logical treatment of depression that was described in detail elsewhere (Cuijpers, van Straten, Warmerdam, & Andersson, 2008b) and that was used in a series of earlier published meta-analyses (www. evidencebasedpsychotherapies.org). This database has been continu- ously updated through comprehensive literature searches (from 1966 to January 2014). In these searches, we examined 14,902 abstracts from Pubmed, PsycInfo, Embase and the Cochrane Register of Trials. These abstracts were identified by combining terms indicative of psy- chological treatment and depression (both MeSH terms and text words). For this database, we also checked the primary studies from earlier meta-analyses of psychological treatment for depression to en- sure that no published studies were missed. From the 14,902 abstracts, we retrieved 1613 full-text papers for possible inclusion in the database.

We included (a) randomized controlled trials in which (b) CBT (c) was compared to a control condition or another treatment (d) in adults with depression (established through a diagnostic interview or through a cut-off on a self-report scale), and (e) in which the effects on dysfunctional thinking were measured. We included randomized trials in which CBT was compared with a control group, with another psychological treatment, and with pharmacotherapy. Studies in which the combination of CBT and pharmacotherapy was compared with CBT alone or with pharmacotherapy alone were not included. We also excluded studies in younger adults, adolescents or children (≤18 years). Comorbid general medical or psychiatric disorders were not used as an exclusion criterion.

CBT was defined as a therapy in which the therapist focuses on the impact that a patient's present dysfunctional thoughts has on current behavior and functioning (Cuijpers et al., 2013a; Jacobson et al., 1996). CBT helps clients to evaluate, challenge, and modify their dysfunctional beliefs (cognitive restructuring), in part to promote behavioral change and to improve their functioning. Therapists use a psychoeducational approach, teaching patients new ways to cope with stressful situations; however, CBT therapists emphasize homework assignments and outside-of-session activities, through the method of collaborative empiricism, in order to directly experience the value of the proposed changes within therapy sessions (Cuijpers et al., 2013a). CBT can be delivered in more formats, including individual, group, or guided self- help, all of which were included.

All measures broadly aimed at examining dysfunctional thinking were allowed, including the Automatic Thoughts Questionnaire/ATQ (Hollon & Kendall, 1980) and the Dysfunctional Attitudes Scale/DAS (Weissman & Beck, 1978), but also less widely used instruments, such as Crandell Cognitions Inventory (Crandell & Chambless, 1986) and the Irrational Beliefs Survey (Watson, Vassar, Plemel, Herder, Manifold, & Anderson, 1990). We included this wide range of instruments to examine possible differences between the most used instruments (ATQ and DAS) and to compare these with the other instru- ments used. We analyzed both short term (post-test) and longer-term effects (follow-up), even if only a limited number of studies reported this last type of information.

2.2. Quality assessment and data extraction

We assessed the validity of included studies using four criteria of the ‘Risk of bias’ assessment tool, developed by the Cochrane Collaboration (Higgins et al., 2011). This tool assesses possible sources of bias in randomized trials, including the adequate generation of allocation sequence; the concealment of allocation to conditions; the prevention of knowledge of the allocated intervention (masking of outcome asses- sors); and dealing with incomplete outcome data (this was assessed as positive when intention-to-treat analyses were conducted, meaning that all randomized patients were included in the analyses).

We also coded additional aspects of the included studies, including characteristics of the participants, the interventions and the study. Two independent researchers completed the quality assessment and data extraction.

2.3. Meta-analyses

For each comparison between a CBT condition and a control or comparison group, the effect size indicating the difference between the two groups at post-test was calculated (Hedges' g). Because several studies had relatively small sample sizes, we corrected the effect size for small sample bias (Hedges & Olkin, 1985).

In the calculations of effect sizes, we used only those instruments that explicitly measured dysfunctional thinking. If more than one measure was used, the mean of the effect sizes was calculated, so that each comparison yielded only one effect size (using the methods de- scribed in Borenstein, Hedges, Higgins, & Rothstein, 2009). We also

65I.A. Cristea et al. / Clinical Psychology Review 42 (2015) 62–71

calculated effect sizes for the outcomes on depression, using all instru- ments measuring depressive symptoms and pooling effect sizes within the study, before pooling effect sizes across studies, when more than one depression instrument was used. If only dichotomous outcomes for depression were reported without means and standard deviations, we used the procedures described by Borenstein et al. (2009) to calcu- late the standardized mean difference.

To calculate pooled mean effect sizes, we used the computer program Comprehensive Meta-Analysis (version 2.2.021). Because we expected considerable heterogeneity among studies, we used a random effects pooling model in all analyses. As a test of homogeneity of effect sizes, we calculated the I2-statistic as an indicator of heteroge- neity in percentages. A value of 0% indicates no observed heterogeneity, and larger values indicate increasing heterogeneity, with 25% as low, 50% as moderate, and 75% as high heterogeneity. We calculated 95% confidence intervals around I2 (Ioannidis, Patsopoulos, & Evangelou, 2007), using the non-central chi-squared-based approach within the heterogi module for Stata (Orsini, Bottai, Higgins, & Buchan, 2006).

Subgroup analyses were conducted according to the mixed effects model (Borenstein et al., 2009), in which studies within subgroups are pooled with the random effects model, while tests for significant differ- ences between subgroups are conducted with the fixed effects model. For continuous variables, we used meta-regression analyses to test whether there was a significant relationship between the continuous var- iable and effect size, as indicated by a Z-value and an associated p-value.

Publication bias was tested by inspecting the funnel plot on primary outcome measures and by Duval and Tweedie's trim and fill procedure (Duval & Tweedie, 2000), which yields an estimate of the effect size after the publication bias has been taken into account. We also conducted Egger's test for the asymmetry of the funnel plot.

3. Results

3.1. Selection of studies and characteristics of included studies

Fig. 1 presents a flowchart describing the inclusion process. Of the 1613 retrieved full-text papers, 1587 were excluded (Fig. 1), while 26 studies met inclusion criteria. For each study in which CBT was examined, we not only checked the papers in which the main outcomes were reported, but also all papers with secondary analyses that were identified in the searches.

Fig. 1. Flowchart for the inclusion of studies.

In the included studies, 2002 patients participated (907 in CBT, 611 in control conditions, 374 in other psychotherapy conditions, and 110 in the pharmacotherapy conditions). Selected characteristics of the 26 studies are presented in Table 1.

In 18 of the 26 studies patients were recruited from the community, while 6 studies recruited patients from clinical samples (one study re- cruited patients through a prenatal clinic, and another did not report it). Seventeen studies were aimed at adults in general, nine at specific populations, such as older adults, university students, and patients with general medical disorders. In 14 studies a diagnostic interview was used to establish the presence of a depressive disorder, while the remaining 12 studies used a cut-off on a self-report scale to establish the presence of depression. In the 23 studies, a total of 43 psychotherapy conditions were examined, of which 30 were CBT, four non-directive counseling, three behavioral activation therapy, and seven other types of therapy (e.g., interpersonal therapy, problem solving therapy, psy- chodynamic therapy). In 19 of the 43 therapies an individual treatment format was used, 14 used a group format and 10 a guided self-help one. The number of therapy sessions ranged from 2 to 18 with most having 8 to 12 sessions (24 out of 43). A total of 16 of the 26 studies were con- ducted in the US, 3 in Europe, and 7 in other countries. Of the 18 control conditions, 12 were a waiting list, 5 were care-as-usual, and one was pill placebo. Nine studies reported group data at follow-up for both the CBT and a control or another psychotherapy group. Follow-up duration ranged from 1 month to 6 months.

3.2. Quality assessment

The quality of the included studies was not optimal (Table 1). Only five of the included studies reported an adequate sequence generation. Six studies reported allocation to conditions by an independent (third) party. A total of 17 studies reported blinding of outcome assessors or used only self-report outcome measures, and in 13 studies intention- to-treat analyses were conducted. Five studies met all four of the quality criteria, 6 met 2 or 3 criteria; and the remaining 15 studies had a lower quality (0 or 1 of the four criteria).

3.3. The effects of CBT versus control groups on dysfunctional thinking at post-test and follow-up

The overall effect of CBT on dysfunctional thinking compared with the control conditions at post-test was g = 0.51 (95% CI: 0.39–0.62), with low heterogeneity (I2 = 6; 95% CI: 0–45). The results of these anal- yses are presented in Table 2, and the forest plot is given in Fig. 2. When we limited the effects to the ATQ, the effects were comparable (g = 0.56; I2 = 25). The effects according to the DAS were somewhat smaller (g = 0.44) and heterogeneity was moderate (I2 = 50). The other measures of dysfunctional thinking resulted in a somewhat larger effect size (g = 0.77; I2 = 28).

In these analyses we included three studies in which more than one form of CBT was compared with the same control group. This means that multiple comparisons from these studies were included in the same analysis, and these were not independent from each other, which may have resulted in an artificial reduction of heterogeneity and may have affected the pooled effect size. In sensitivity analyses, we examined the possible effects of this by conducting an analysis in which we included only one effect size per study. First, we included only the comparison with the largest effect size from these studies and then we conducted another analysis in which we included only the smallest effect size. As it can be seen from Table 2, the resulting ef- fect sizes as well as the levels of heterogeneity were comparable with the overall analyses.

We found some indication for a small effect of publication bias. After adjustment for publication bias according to Duval and Tweedie's trim and fill procedure, the overall effect size was reduced from 0.51 to

Table 1 Selected characteristics of randomized trials examining the effects of cognitive behavior therapy on dysfunctional thinking.

Recra Definition of depressionb

Target group Conditionsc N Formatd Nse Qual Measure of dysfunct thoughtse

C

Allart-van Dam, 2003 Comm BDI ≥ 10; no MDD Adults 1. CWD 2. CAU

61 41

Grp 12 − − + +

ATQ NL

Bowman, 1995 Comm HRSD ≥ 10 Adults 1. CBT 2. PST 3. WL

10 10 10

Gsh Gsh

4 4

− − − −

ATQ US

Bright et al., 1999 Comm Mood disorder (DSM-III-R/SCID)

Adults 1. CBT prof. 2. CBT paraprof. 3. SUP prof. 4. SUP paraprof.

18 13 22 14

Grp Grp Grp Grp

10 10 10 10

− − + −

ATQ US

Cho, 2008 Through prenatal clinic

BDI ≥ 16 + Mood disorder (SCID)

Pregnant women 1. CBT 2. CAU

12 10

Ind 9 − − + −

ATQ S-Ko

Cramer, 2011 Comm Depression PHQ-9 Women aged 30 to 55 years

1. CBT 2. CAU

48 19

Grp 12 + + + +

ATQ — short UK

Dekker et al., 2012 Through clinics BDI-II (10-28) Hospitalized patients with heart failure

1. CBT 2. CAU

20 21

Ind 2 + + + +

CCI US

Dobkin, 2011 Comm Mood disorder (SCID) Patients with Parkinson's Disease

1. CBT 2. WL

41 39

Ind 10 + + + +

IQ US

Elkin, 1989 (Imber et al., 1990)

Clin MDD (RDC; SADS) Adults 1. CBT 2. IPT 3. Placebo

59 61 62

Ind Ind

16 16

+ + + +

DAS US

Hamamci, 2006 Comm BDI ≥ 19 University students 1. CBT 2. CAU

10 11

Grp 11 − − + −

ATQ; DAS Turk

Hogg, 1988 Clin BDI ≥ 14 University students 1. CBT 2. Interp group 3. WL

13 14 10

Grp Grp

8 8

− − + −

ATQ US

Jamison, 1995 Comm MDD (DSM-III-R) + HRSD ≥ 10 + BDI ≥ 10

Adults 1. CBT 2. WL

33 39

Gsh 4 − − − −

ATQ; DAS US

McKnight, 1992 Comm MDD Adults 1. CBT 2. PHA

22 21

Ind 8 − − − −

PBI US

McNamara & Horan, 1986

Clin BDI ≥ 18 + HRSD ≥ 20 Adults 1. CBT 2. BA 3. SUP

10 10 9

Ind Ind Ind

9 9 9

− − + −

ATQ US

Quilty, 2008 NR MDD (DSM-IV/SCID) Adults 1. CBT 2. IPT 3. PHA

45 46 41

Ind Ind

18 18

− − − −

DAS CAN

Scogin, 1987 Comm HRSD ≥ 10 Elderly 1. CBT 2. WL

9 8

Gsh 4 − − − −

CEQ US

Scogin, 1989 Comm HRSD ≥ 10 Elderly 1. CBT 2. CWD 3. WL

19 21 21

Gsh Gsh

4 4

− − − −

ATQ; DAS US

Selmi, 1991 Comm DD (RDC/SADS) + BDI ≥ 16

Adults 1. cCBT 2. CBT 3. WL

12 12 12

Gsh Ind

6 6

− − + +

ATQ US

Simons, 1984 Clin MDD Adults 1. CBT 2. PHA

14 14

Grp 15 − − − −

ATQ; DAS US

Thompson et al., 1987 Comm MDD (RDC/SADS) Elderly 1. BA 2. CBT 3. PDT

21 17 20

Ind Ind Ind

18 18 18

− − − −

ATQ US

Warmerdam et al., 2008

Comm CES-D ≥ 16 Adults 1. iCBT 2. iPST 3. WL

88 88 87

Gsh Gsh

8 5

+ + + +

DAS NL

Watson et al., 2003 Comm MDD (DSM-IV/SCID) Adults 1. CBT 2. SUP

45 40

Ind Ind

16 16

− − + +

DAS CAN

Wilson et al., 1983 Comm BDI ≥ 17 Adults 1. BA 2. CBT 3. WL

8 8 8

Ind Ind

8 8

− − − −

IBT; NCS (3 subscales); PCS (3 subscales)

AU

Wong, 2008 A Comm MDD (DSM-IV) Adults 1. CBT 2. WL

48 40

Grp 10 − − + +

DAS HK

Wong, 2008 B Comm MDD (DSM-IV) Adults 1. CBT 2. WL

163 159

Grp 10 − + + +

DAS HK

Wright et al., 2005 Clin MDD (DSM-IV/SCID) Adults 1. cCBT 2. CBT 3. WL

13 13 14

Gsh Ind

9 9

− − + +

ATQ; DAS US

Zettle & Rains, 1989 Comm BDI ≥ 20 + MMPI-D T N 70 + HRSD ≥ 14

Women 1. CBT 2. Distancing

10 11

Grp Grp

12 12

− − + −

ATQ; DAS US

a Recr, recruitment; Comm, community recruitment; Clin, recruitment from clinical populations; NR, not reported. b MDD, Major Depressive Disorder; BDI, Beck Depression Inventory; HRSD, Hamilton Rating Scale for Depression; SCID, Structured Clinical Interview for DSM III/IV; PHQ-9, Patient Health

Questionnaire-9; RDC, Research Diagnostic Criteria; SADS, Schedule for Affective Disorders and Schizophrenia; DD, Depressive Disorder; CES—D, Center for Epidemiological Studies—Depression scale; MMPI—D, Minnesota Multiphasic Personality Inventory—Depression scale.

c CWD, Coping with Depression; CAU, care-as-usual; CBT: cognitive behavior therapy; PST, problem-solving therapy; WL, waiting list; prof., professionals; paraprof., paraprofessionals; SUP, non-directive supportive therapy; IPT, interpersonal therapy; Interp group, Interpersonal group process therapy; PHA, pharmacotherapy; BA, Behavioral Activation; cCBT, comput- erized cognitive behavior therapy; PDT, psychodynamic therapy; iCBT, internet cognitive behavior therapy; iPST, internet problem-solving therapy.

d Grp, group; Ind, individual; Gsh, guided self-help. e Dysfunc, dysfunctional; ATQ, Automatic Thoughts Questionnaire; CCI, Crandell Cognitions Inventory; IQ, Inference Questionnaire; DAS, Dysfunctional Attitudes Scale; IBS: Irrational

Beliefs Survey; PBI, Personal Beliefs Inventory; CEQ, Cognitive Error Questionnaire; IBT, Irrational Beliefs Test; PCS, Positive Cognition Schedule; NCS, Negative Cognition Schedule.

66 I.A. Cristea et al. / Clinical Psychology Review 42 (2015) 62–71

Table 2 The effects of cognitive behavior therapies for depression on dysfunctional thinking com- pared to control groups, post-test and follow-up: Hedges' g.

N g 95% CI I2 95% CI pa

Post-test

All studies 21 0.51 0.39–0.62 6 0–45 One effect size per study (highest) 18 0.53 0.41–0.65 0 0–44 One effect size per study (lowest) 18 0.51 0.39–0.63 5 0–46

Specific measures ATQ only 13 0.56 0.35–0.77 25 0–61 0.23 DAS only 10 0.44 0.21–0.66 50 0–74 Other measures 10 0.77 0.45–1.08 28 0–65

Subgroup analyses Recruitment Community 15 0.53 0.37–0.70 29 0–61 0.71

Other 6 0.48 0.14–0.71 0 0–61 Format Individual 7 0.49 0.26–0.73 0 0–58 0.98

Group 6 0.52 0.31–0.74 26 0–70 Guided self-help 8 0.53 0.24–0.73 38 0–71

Target group Adults in general 12 0.55 0.42–0.68 0 0–50 0.31 Specific population 9 0.40 0.15–0.66 27 0–66

Control group Waiting list 15 0.53 0.38–0.68 15 0–54 0.88 Care-as-usual 5 0.50 0.22–0.78 11 0–68 Placebo 1 0.41 –0.06–0.87 b

Definition depression

Diagnosis 10 0.55 0.40–0.69 0 0–53 0.43 Cut-off self-report 11 0.44 0.24–0.65 19 0–60

Quality 3–4 criteria 5 0.42 0.22–0.63 0 0–64 0.30 0–2 criteria 16 0.57 0.40–0.74 23 0–58

Follow-up

All studies 9 0.46 0.15–0.78 48 0–74 One effect size per study (highest) 8 0.36 0.08–0.64 34 0–70 One effect size per study (lowest) 8 0.34 0.08–0.60 25 0–67

Specific measures ATQ only 6 0.59 0.13–1.05 59 0–81 0.34 DAS only 2 0.13 −0.29–0.55 0 c

Other measures 3 0.31 −0.06–0.69 16 0–77

a The p-value in this column indicates whether the effect sizes in subgroup differ signifi- cantly from each other.

b Level of heterogeneity cannot be calculated when the subgroup contains only one study. c 95% confidence interval of I2 cannot be calculated with two studies.

67I.A. Cristea et al. / Clinical Psychology Review 42 (2015) 62–71

0.47 (95% CI: 0.36–0.58; number of imputed studies: 3). Egger's test was not significant, however (p = 0.13).

The effects of CBT on dysfunctional thinking were maintained at follow-up (Table 2). Nine comparisons between CBT and a control group led to a significant g = 0.46 (95% CI: 0.15–0.78), with moderate heterogeneity (I2 = 58; 95% CI: 0–74). This result was robust when considering only one ES per study or when restricting analyses to the

Fig. 2. Forrest plot of effect sizes indicating the difference between studies examining CBT and control groups on dysfunctional thoughts.

ATQ only (there were too few studies for the DAS or for other measures of cognition).

In order to examine possible moderators of outcome and potential sources of heterogeneity, we conducted a series of subgroup analyses. We found no indication that the effect sizes significantly among sub- groups of studies, including recruitment (community versus other re- cruitment strategies), treatment format (individual, group, guided self-help), target group (adults versus more specific target group), type of control group (waiting list, care-as-usual, placebo), definition of depression (according to a diagnostic interview versus self-report), and quality (3–4 criteria versus 0–2 criteria).

3.4. Association between effects on depression and effects on dysfunctional thinking

We examined whether the effects of CBT on dysfunctional thinking were associated with the effects on depression. We conducted a metaregression analysis with the effects on depression as dependent variable and the effects on dysfunctional thinking as predictor. The results of these analyses are presented in Fig. 3. It was found that there was indeed a significant association between the effects on dysfunctional thinking and those on depression (slope: 0.77; 95% CI: 0.33–1.21; p b 0.001).

3.5. The effects of CBT versus other treatments on dysfunctional thinking at post-test and follow-up

We could compare the effects of CBT on dysfunctional thinking with those of other psychotherapies in 14 comparisons. The differential effect size was g = 0.17 (95% CI: −0.05–0.39), which was not significantly dif- ferent from zero (p = 0.14). Heterogeneity was moderate (I2 = 47; 95% CI: 0–70). The results of these analyses are presented in Table 3. The ef- fect according the DAS (g = 0.29) was larger than the effect according to the ATQ (g = 0.04), and it was significantly larger than zero (p b 0.05), although it was based on only five studies.

When the analyses were limited to only one effect size per study, we did not find that this resulted in major changes in effect size or level of heterogeneity (Table 3). We found no indication of significant publica- tion bias. In Duval and Tweedie's trim and fill procedure the adjusted ef- fect size was identical to the unadjusted effect size with no studies missing, and Egger's test was not significant either (p N 0.1).

Follow-up data were available for 5 comparisons (Table 3) and pointed to a significant effect of CBT over other psychotherapies, g = 0.43 (95% CI: 0.05–0.81), with low heterogeneity (I2 = 0; 95% CI: 0–64). However, this result was not confirmed in any of the subse- quent sensitivity analyses, neither when considering one ES per study, nor when restricting analyses to the ATQ, which was the measure reported by all studies.

The subgroup analyses, aimed at examining moderators of outcome and sources of heterogeneity, did not result in any significant difference between subgroups.

Four studies compared the effects of CBT with those of pharmaco- therapy on dysfunctional thinking. The differential effect size indicating the difference between the two treatments at post-test was g = 0.04 (95% CI: −0.22–0.30) with no heterogeneity (I2 = 0; 95% CI: 0–68). Be- cause the number of studies was so small we did no additional analyses. There were no follow-up data for the comparison between CBT and pharmacotherapy.

4. Discussion

While the efficiency of CBT on adult depression has been established, its putative mechanisms of action are less clear. Cognitive restructuring, the transformation of dysfunctional cognitions into more adaptive ones, is believed to be at the core of CBT (Clark & Beck, 2010). In a review striving to untangle the “complicated” (in the authors' own words)

Fig. 3. Effect on depression as predictor of the effect of CBT on dysfunctional thinking: Metaregression analysis.

68 I.A. Cristea et al. / Clinical Psychology Review 42 (2015) 62–71

relationships between cognitive change procedures, cognitive change and symptom change, Lorenzo-Luaces et al. (2015) synthesized the the- ory behind CBT as “engaging in procedures aimed at altering negatively biased beliefs and thinking styles leads to cognitive change, which is the mechanism by which depressive symptoms are reduced.”

Table 3 The effects of cognitive behavior therapies for depression on dysfunctional thinking com- pared to other psychotherapies and pharmacotherapy, at post-test and follow-up: Hedges' g.

N g 95% CI I2 95% CI

pa

Post-test

CBT versus other psychotherapies All studies 14 0.17 −0.05–0.39 47 0–70 One effect size per study (highest) 11 0.19 −0.08–0.47 57 0–76 One effect size per study (lowest) 11 0.20 0.03–0.36 0 0–51 One outlier excluded (McNamara & Horan, 1986)

13 0.13 −0.04–0.30 11 0–54

Specific measuresb

ATQ only 9 0.04 −0.32–0.41 57 0–78 DAS only 5 0.29 0.09–0.49 0 0–64

Subgroup analyses Recruitmentd Community 9 0.01 −0.24–0.25 0 0–54 0.10

Clinical 4 0.41 0.00–0.81 74 0–89 Comparison therapy

Supportive 4 0.02 −0.38–0.41 33 0–77 0.40 Behavioral activation

3 0.50 −0.07–1.07 82 0–92

Other 7 0.16 −0.15–0.48 12 0–63 Format Individual 8 0.35 −0.09–0.62 51 0–76 0.08

Group 4 −0.19 −0.58–0.21 0 0–68 Guided self-help 2 0.08 −0.42–0.58 0 c

Target group Adults in general 10 0.26 0.02–0.51 57 0–77 0.13 Specific population 4 −0.12 −0.54–0.30 0 0–68

Definition depression

Diagnosis 7 0.12 −0.18–0.42 47 0–76 0.35 Cut-off self-report 7 0.25 −0.12–0.62 54 0–79

Quality 3–4 criteria 2 0.20 −0.28–0.68 0 c 0.89 0–2 criteria 12 0.16 −0.10–0.43 54 0–74

CBT versus pharmacotherapy All studies 4 0.04 −0.22–0.30 0 0–68

Follow-up

CBT versus other psychotherapies All studies 5 0.43 0.05–0.81 0 0–64 One effect size per study (highest) 4 0.37 −0.05–0.79 0 0–68 One effect size per study (lowest) 4 0.34 −0.08–0.76 0 0–68

Specific measurese

ATQ only 5 0.34 −0.04–0.72 0 0–64

a The p-value in this column indicates whether the effect sizes in subgroup differ signifi- cantly from each other.

b Only one study used another measure than the ATQ or DAS; therefore we only report the outcomes for ATQ and DAS here.

c The level of heterogeneity cannot be calculated when the subgroup contains only one study.

d The study that did not report recruitment method was excluded from these analyses. e Only one study used the DAS along with the ATQ; therefore we only report the outcomes

for ATQ here.

The crux of this argument is whether cognitive change, be it cognitive processes or content, is the essential, sine qua non mechanism for symp- tom change, irrespective of how it was produced. However, it is also possible that certain therapeutic contexts, like those focusing explicitly on changing dysfunctional thoughts, are more likely to engender cogni- tive change (Lorenzo-Luaces et al., 2015). In this case, we might expect to see differences between CBT and other therapies on these outcomes. At the other end however, a case can be made that, along with change in symptoms, change in cognitions is simply another by-product of other causal factors, such as nonspecific processes, present in all forms of psychotherapy (e.g., the therapeutic relationship). Unfortunately, the circularity of this argument makes it hard to test empirically unless stud- ies examine the pattern of covariation over time in both dysfunctional thoughts and depressive symptoms (see for example Kazdin (2007) for methodological suggestions for implementing this).

Lorenzo-Luaces et al. (2015) also laid out four questions that need to be addressed empirically in order to evaluate causality and speci- ficity regarding the cognitive model of therapeutic change. We fo- cused on one of those questions — the purported differential efficacy of CBT procedures on cognitions, which has not been ad- dressed before in a meta-analysis. More precisely, we wanted to evaluate the magnitude of the effects of CBT on dysfunctional think- ing, whether they co-vary with symptom change and, essentially, whether they are unique to CBT.

The results of our meta-analysis showed that there was indeed a sig- nificant, medium effect of CBT on dysfunctional thoughts, confirming the conclusions of most individual studies. This result remained similar at follow-up. Interestingly, the effect remained very similar when we looked at different types of dysfunctional beliefs outcomes, by reporting results only on the ATQ, the DAS or on other types of measures. Adjust- ment for publication bias also modified the overall effect by very little and heterogeneity was small, both when dysfunctional beliefs were considered globally, and when they were reported on separate instruments.

More importantly for the assumption of the covariance of change in dysfunctional thoughts with change in symptoms, our meta-regression analysis showed a significant linear association between the effects on dysfunctional thinking and those on depression. While this seems to in- dicate that change in dysfunctional thoughts is indeed associated with change in depressive outcomes, there are some caveats that have to be considered. For one, we need to consider the fact that there is a ro- bust correlation between dysfunctional beliefs and depressed mood (Lau, Segal, & Williams, 2004; Miranda, Persons, & Byers, 1990), which means that it is possible that the effects seen on dysfunctional thoughts might be simply due to the covariance with depressive symptoms. As importantly, we have no way of assessing temporal precedence, as dys- functional thoughts and depressive symptoms were assessed at the same temporal moment at post-test. Temporal precedence is essential in order to be able to state that cognitive change is causally involved in symptom change (Kazdin, 2007). It is also possible that another, third variable, such as taking part in treatment, drove changes in both cognitions and depressive symptoms, with no causal link existing between the two.

However, perhaps even more relevantly, our results do not gen- erally support the conjecture of cognitive specificity (which as Hollon et al. (1987) have noted is not to be confused with the as- sumption of causality). Differences between CBT and other psycho- therapies in their impact on dysfunctional thoughts were not significant, except when effects were limited to the DAS (but only 5 studies contributed to this effect). Perhaps even more relevantly, there were also no differences between CBT and pharmacotherapy, though again this was based on only 4 comparisons. We found a sig- nificant difference between CBT and other therapies at follow-up, but this result was based on a small number of comparisons and not retained in any of the subsequent sensitivity analyses. Hence, it is very unstable and most likely a chance finding.

69I.A. Cristea et al. / Clinical Psychology Review 42 (2015) 62–71

The lack of differences between CBT and other psychotherapies is an argument that can be credibly employed by both sides of the debate. On one hand, we could interpret it as showing that not only CBT, but other therapies too, work by changing cognitions, even if they do not target them directly. Consequently, given that we also know from previous meta-analyses that all these psychotherapies have similar effects on symptoms, we could plausibly argue that change in cognitions is most likely essential for symptom change, regardless of how it is achieved. In this vein, another corollary of our findings would be that CBT strate- gies of directly targeting and attempting to change dysfunctional thoughts appear in no way more efficient than the strategies used in other psychotherapies, which do not focus on cognitive restructuring and emphasize other processes (or at least do not explicitly work with dysfunctional beliefs). It is nevertheless possible that other psychother- apies also work with dysfunctional cognitions and may even employ cognitive techniques, even if they don't explicitly consider cognitive restructuring as a key component of their therapeutic protocols. In other words, cognitive change may lead to symptom change, but CBT protocols and strategies are not uniquely or even better equipped to en- gender it. On the other hand, the same findings could be interpreted by reasoning that another plausible explanation is that change in dysfunc- tional beliefs is a consequence of improvement in symptoms, regardless of the treatment. Thus, dysfunctional thinking would represent just an- other depressive symptom, maybe more resistant to change than others, but a symptom nevertheless. Following the same reasoning, it is also possible, as delineated previously, that change in depressive symptoms, as well as change in dysfunctional cognitions, are both driven by a third, unknown factor, without them being causally related.

An exception to the lack of differences between CBT and other psy- chotherapies appeared when effects were measured on the DAS. While we could infer this to mean that CBT is more efficient on the par- ticular types of cognitions measured by the DAS, it should be noted that not only was this contrast based on a small number of comparisons, but that the DAS is also a problematic measure. A number of recent studies (De Graaf, Roelofs, & Huibers, 2009; Moore, Fresco, Segal, & Brown, 2014) have pointed out to problems regarding the validity and psycho- metric properties of the 40 item DAS-A, which was the version used in most studies. Also, the DAS (Weissman & Beck, 1978) was explicitly developed in the context of Beck's theory of depression, measuring maladaptive schemas central to this theory and explicitly targeted by cognitive restructuring in CBT protocols. In fact, as other authors have remarked, a patient who has gone through a CBT course has most likely picked up what types of beliefs his or her therapist views as adaptive (Adler, Strunk, & Fazio, 2015). So it is very well possible that when faced with a self-report measure of these beliefs, demand characteristics may come into play making the patient give “desirable” answers, con- sistent to what was taught in therapy. In this sense, a scale like the DAS might be too contaminated conceptually by the presumed mecha- nisms of cognitive therapy and might function much more as a “test of knowledge” than a true measure of a psychopathological process. On the other hand, it is not surprising that we did not find any differences on the ATQ, a measure that is more state dependent and strongly correlated with depressive symptoms (Clark, 1988).

Interestingly, we found no differences between CBT and pharmaco- therapy, though this result was based on a small number of comparisons and should be interpreted very cautiously. Garratt et al. (2007) argued that the contrast against pharmacotherapy is particularly relevant because claims had yet to be made about pharmacotherapy also acting through the mediation of changing dysfunctional thoughts. Nonethe- less, recently some of these claims have begun to be made (Lorenzo- Luaces et al., 2015). One of the main arguments comes from interpreting behavioral and neuroimaging data showing that in both healthy and de- pressed individuals, antidepressant medication can have effects on emotional processing even before it has effects on mood (Harmer, Goodwin, & Cowen, 2009). However, we believe we have to be very careful before drawing any analogies between the dysfunctional

thoughts that are the focus of CBT protocols, are measured by self- report measures and were analyzed in our meta-analysis, and what is described as emotional processing in behavioral or neuroimaging studies. These latter studies use experimental tasks, often implicit (e.g., implicit face recognition or processing) and the constructs they measure could be at most assimilated with the concept of negative bias. Even in the cognitive theory of depression postulating the mecha- nisms of action of CBT (Beck & Haigh, 2014), biases are separate and in no way equivalent to dysfunctional thoughts or schemas. As such, data on the effects of antidepressant medication on emotional processing, albeit interesting, do not allow us to infer that this medication would also lead to change in dysfunctional thoughts of the type assessed here, before change in other depressive symptoms. It is of course theo- retically possible that CBT would carry its effect by modifying thoughts and recent formulations of the generic theory of CBT insisting on con- cepts like “mode” or “schema” could accommodate this possibility. But until evidence comes along showing how and why pharmacotherapy for depression would also work by modifying dysfunctional thoughts, our own results seem to add to the idea that change in cognitions might be just another consequence of treatment of depression and not a causal element in symptom change.

However, it is also worth noting that some studies have reported an enduring effect of CBT in preventing relapse that was not found for medication (Hollon et al., 2005) and there is some indication this effect might be mediated by changes in CBT specific skills and independent in- session use of CBT principles (Strunk, Brotman, & DeRubeis, 2010). Our results are instead based on data on the content and frequency of dysfunctional thoughts, which is what is conveyed by the most used instruments of evaluating cognitions targeted by CBT and, as a conse- quence, what most studies assessed. It is therefore possible that CBT procedures have a unique effect on the acquisition and implementation of CBT skills instead of on the content and frequency of dysfunctional cognitions.

There are a number of limitations of this meta-analysis. First of all, the number of studies was relatively small, making several comparisons underpowered. Still, it must be said that many, if not all, existent theo- ries about the function of cognitive change in CBT are based on narrative reviews interpreting the results of individual studies and in most of these cases there are studies pointing in both directions. Therefore, a meta-analysis, while not exempt from limitations of statistical power, is in any case more systematic and less subjective. Secondly, the quality of the included studies was subpar, with only 5 out of 26 having a low risk of bias and with more than a half of the included studies having a high risk of bias. As it has been previously shown, the inclusion of lower quality studies might artificially inflate the estimation of the efficiency of CBT for depression (Cuijpers et al., 2013a). We did not find differences between lower and higher quality studies in our sub- group analysis, but it must also be noted that there were only 5 studies with a higher quality, which rendered statistical comparisons difficult. Thirdly, it is possible we might have missed some of the studies eligible for inclusion. Authors sometimes include results on cognitive variables in a separate paper from results on the main outcomes and while we did search for additional papers related to the main report, it is possible we might have missed some.

Finally, some problems were inherent in the literature we reviewed. Our results are based on self-report measures focused on the content and frequency of dysfunctional thoughts, but it is possible that the spe- cific effects of CBT are more visible on other types of processes such as cognitive therapy skills, as some researchers have suggested (Adler et al., 2015). Interestingly, there is also some evidence of the mediating role of these skills in relapse prevention following CBT, so this might be a useful venue to explore (Strunk et al., 2010). Another concern has to do with the impossibility of verifying treatment integrity, meaning that it is possible that cognitive procedures were not correctly imple- mented in the CBT arms, across studies, or conversely, that spill-over ef- fects, in which cognitive change procedures were also implemented in

70 I.A. Cristea et al. / Clinical Psychology Review 42 (2015) 62–71

the other psychotherapies, may have been present. Therapist adherence to the therapy protocol might have played a role in distinguishing the effects of CBT from those of other therapies, but only a couple of com- parative trials quantified and reported adherence data using previously validated scales. However, a recent meta-analysis (Webb, Derubeis, & Barber, 2010) found a non-significant, almost zero association, between therapist adherence and symptom outcomes (r = 0.02), which makes it unlikely that therapist adherence would have significantly moderated these results. Follow-up data might have helped expose more nuanced differences between CBT and other psychotherapies, or respectively pharmacotherapy. However, given that only a small number of studies provided follow-up data on dysfunctional thinking, we were unable to reliably address this question. We also note that time to follow-up was variable across studies, a factor increasing heterogeneity and raising doubts about combining these studies.

The remaining quandary is how future studies could be constructed so as to permit a better assessment of therapeutic mechanisms and to offer a more valid answer to the issue of whether cognitive change is a cause or a consequence of symptom change. Kazdin (2007) and Lorenzo-Luaces et al. (2015) both laid out a set of valuable suggestions, such as frequent assessments of both mediators and symptoms, particu- larly early in therapy, so as to ensure that mediators are tested in reference to their capacity to predict change that is subsequent to their assessment. Other relevant suggestions included looking at long term outcomes of patients treated with different treatment modalities, being mindful of the composition of the study population (e.g., spontaneous re- mitters or extreme non-responders), moving away from measures that conflate cognitive change with symptom change, as do most self-report measures of dysfunctional thinking used in trials, and considering patient level variability (i.e., for some patients explicit techniques of cognitive change might be essential, while irrelevant for others).

In conclusion, our meta-analysis showed that while CBT has a robust and stable effect on dysfunctional thoughts, this is not significantly dif- ferent from what other psychotherapies or even pharmacotherapy achieve. We did find differences between CBT and other psychother- apies when restricting effects to the DAS, but since this is a measure deeply embedded in the theory of CBT it is impossible to discern wheth- er patients are reporting true therapeutic changes or whether they are just giving responses they know to be right after having been in therapy. Nonetheless, while it may be hard to escape the circularity of the “what came first” debate on whether cognitive change is always the causal process of symptom change, even if not targeted directly in the course of therapy, or whether it is simply another symptom of depression, we are at least safe in concluding that change in dysfunctional beliefs is not a process unique or specific to CBT.

Role of funding sources

There was no funding for this study. The authors benefited from no financial support for conducting this research.

Contributors

PC had the original idea for this paper. PC and IC did the searches, the data extraction and analyses, and wrote the first draft of the paper. All authors (IC, MH, DD, SH, GA, and PC) read all versions of the text of the paper critically and contributed significantly to the content. All the authors have reviewed the present version of the manuscript and approved it for submission.

Conflict of interest

None of the authors disclosed any conflict of interest.

References

Adler, A.D., Strunk, D.R., & Fazio, R.H. (2015). What changes in cognitive therapy for depression? An examination of cognitive therapy skills and maladaptive beliefs. Behavior Therapy, 46(1), 96–109. http://dx.doi.org/10.1016/j.beth.2014.09.001.

Allart-van Dam, E., Hosman, C. M. H., Hoogduin, C. A. L., & Schaap, C. P. D. R. (2003). The coping with depression course: Short-term outcomes and mediating effects of a ran- domized controlled trial in the treatment of subclinical depression. Behavior Therapy, 34(3), 381–396. http://dx.doi.org/10.1016/S0005-7894(03)80007-2.

Barth, J., Munder, T., Gerger, H., Nüesch, E., Trelle, S., Znoj, H., et al. (2013). Comparative efficacy of seven psychotherapeutic interventions for patients with depression: A network meta-analysis. PLoS Medicine, 10(5), e1001454. http://dx.doi.org/10.1371/ journal.pmed.1001454.

Beck, A.T., & Dozois, D.J.A. (2011). Cognitive therapy: Current status and future directions. Annual Review of Medicine, 62, 397–409. http://dx.doi.org/10.1146/annurev-med- 052209-100032.

Beck, A.T., & Haigh, E.A.P. (2014). Advances in cognitive theory and therapy: The generic cognitive model. Annual Review of Clinical Psychology, 10, 1–24. http://dx.doi.org/10. 1146/annurev-clinpsy-032813-153734.

Borenstein, M., Hedges, L.V., Higgins, J.P.T., & Rothstein, H.R. (2009). Introduction to meta- analysis. Chichester, UK: Wiley.

Bowman, D., Scogin, F., & Lyrene, B. (1995). The Efficacy of Self-Examination Therapy and Cognitive Bibliotherapy in the Treatment of Mild to Moderate Depression. Psychotherapy Research, 5(2), 131–140. http://dx.doi.org/10.1080/10503309512331331256.

Bright, J. I., Baker, K. D., & Neimeyer, R. A. (1999). Professional and paraprofessional group treatments for depression: a comparison of cognitive-behavioral and mutual support interventions. Journal of Consulting and Clinical Psychology, 67(4), 491–501.

Braun, S.R., Gregor, B., & Tran, U.S. (2013). Comparing bona fide psychotherapies of depression in adults with two meta-analytical approaches. PloS One, 8(6), e68135. http://dx.doi.org/10.1371/journal.pone.0068135.

Cho, H. J., Kwon, J. H., & Lee, J. J. (2008). Antenatal Cognitive-behavioral Therapy for Pre- vention of Postpartum Depression: A Pilot Study. Yonsei Medical Journal, 49(4), 553–562. http://dx.doi.org/10.3349/ymj.2008.49.4.553.

Clark, D.A. (1988). The validity of measures of cognition: A review of the literature. Cognitive Therapy and Research, 12(1), 1–20. http://dx.doi.org/10.1007/BF01172777.

Clark, D.A., & Beck, A.T. (2010). Cognitive theory and therapy of anxiety and depression: Convergence with neurobiological findings. Trends in Cognitive Sciences, 14(9), 418–424. http://dx.doi.org/10.1016/j.tics.2010.06.007.

Cramer, H., Salisbury, C., Conrad, J., Eldred, J., & Araya, R. (2011). Group cognitive behav- ioural therapy for women with depression: pilot and feasibility study for a randomised controlled trial using mixed methods. BMC Psychiatry, 11, 82. http://dx. doi.org/10.1186/1471-244X-11-82.

Crandell, C.J., & Chambless, D.L. (1986). The validation of an inventory for measuring depressive thoughts: The Crandell cognitions inventory. Behaviour Research and Therapy, 24(4), 403–411. http://dx.doi.org/10.1016/0005-7967(86)90005-7.

Cuijpers, P., Berking, M., Andersson, G., Quigley, L., Kleiboer, A., & Dobson, K.S. (2013a). A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments. Rev. Can. Psychol., 58(7), 376–385.

Cuijpers, P., Donker, T., van Straten, A., Li, J., & Andersson, G. (2010). Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A sys- tematic review and meta-analysis of comparative outcome studies. Psychological Medicine, 40(12), 1943–1957. http://dx.doi.org/10.1017/S0033291710000772.

Cuijpers, P., Hollon, S.D., van Straten, A., Bockting, C., Berking, M., & Andersson, G. (2013b). Does cognitive behaviour therapy have an enduring effect that is superior to keeping patients on continuation pharmacotherapy? A meta-analysis. BMJ Open, 3(4)http:// dx.doi.org/10.1136/bmjopen-2012-002542.

Cuijpers, P., Sijbrandij, M., Koole, S.L., Andersson, G., Beekman, A.T., & Reynolds, C.F. (2013c). The efficacy of psychotherapy and pharmacotherapy in treating depressive and anxiety disorders: A meta-analysis of direct comparisons. World Psychiatry: Official Journal of the World Psychiatric Association (WPA), 12(2), 137–148. http://dx. doi.org/10.1002/wps.20038.

Cuijpers, P., van Straten, A., Andersson, G., & van Oppen, P. (2008a). Psychotherapy for depression in adults: A meta-analysis of comparative outcome studies. Journal of Consulting and Clinical Psychology, 76(6), 909–922. http://dx.doi.org/10.1037/a0013075.

Cuijpers, P., van Straten, A., Warmerdam, L., & Andersson, G. (2008b). Psychological treatment of depression: A meta-analytic database of randomized studies. BMC Psychiatry, 8(36)http://dx.doi.org/10.1186/1471-244X-8-36.

David, D., & Montgomery, G.H. (2011). The scientific status of psychotherapies: A new evaluative framework for evidence-based psychosocial interventions. Clinical Psychology: Science and Practice, 18(2), 89–99. http://dx.doi.org/10.1111/j.1468- 2850.2011.01239.x.

De Graaf, L.E., Roelofs, J., & Huibers, M.J.H. (2009). Measuring dysfunctional attitudes in the general population: The Dysfunctional Attitude Scale (form A) revised. Cognitive Therapy and Research, 33(4), 345–355. http://dx.doi.org/10.1007/s10608- 009-9229-y.

Dekker, R. L., Moser, D. K., Peden, A. R., & Lennie, T. A. (2012). Cognitive therapy improves three-month outcomes in hospitalized patients with heart failure. Journal of Cardiac Failure, 18(1), 10–20. http://dx.doi.org/10.1016/j.cardfail.2011.09.008.

DeRubeis, R.J., Evans, M.D., Hollon, S.D., Garvey, M.J., Grove, W.M., & Tuason, V.B. (1990). How does cognitive therapy work? Cognitive change and symptom change in cogni- tive therapy and pharmacotherapy for depression. Journal of Consulting and Clinical Psychology, 58(6), 862–869.

Dobkin, R. D., Menza, M., Allen, L. A., Gara, M. A., Mark, M. H., Tiu, J., ... Friedman, J. (2011). Cognitive-behavioral therapy for depression in Parkinson’s disease: a randomized, controlled trial. The American Journal of Psychiatry, 168((10), 1066–1074. http://dx. doi.org/10.1176/appi.ajp.2011.10111669.

71I.A. Cristea et al. / Clinical Psychology Review 42 (2015) 62–71

Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463.

Elkin, I., Shea, M. T., Watkins, J. T., Imber, S. D., Sotsky, S. M., Collins, J. F., ... Docherty, J. P. (1989). National Institute of Mental Health Treatment of Depression Collaborative Research Program. General effectiveness of treatments. Archives of General Psychiatry, 46(11), 971–982 (discussion 983).

Furlong, M., & Oei, T.P.S. (2002). Changes to automatic thoughts and dysfunctional atti- tudes in group CBT for depression. Behavioural and Cognitive Psychotherapy, 30(03), 351–360. http://dx.doi.org/10.1017/S1352465802003107.

Garratt, G., Ingram, R.E., Rand, K.L., & Sawalani, G. (2007). Cognitive processes in cognitive therapy: Evaluation of the mechanisms of change in the treatment of depression. Clinical Psychology: Science and Practice, 14(3), 224–239. http://dx.doi.org/10.1111/j. 1468-2850.2007.00081.x.

Gloaguen, V., Cottraux, J., Cucherat, M., & Blackburn, I.M. (1998). A meta-analysis of the effects of cognitive therapy in depressed patients. Journal of Affective Disorders, 49(1), 59–72.

Hamamci, Z. (2006). Integrating psychodrama and cognitive behavioral therapy to treat moderate depression. The Arts in Psychotherapy, 33(3), 199–207. http://dx.doi.org/ 10.1016/j.aip.2006.02.001.

Harmer, C.J., Goodwin, G.M., & Cowen, P.J. (2009). Why do antidepressants take so long to work? A cognitive neuropsychological model of antidepressant drug action. The British Journal of Psychiatry: the Journal of Mental Science, 195(2), 102–108. http:// dx.doi.org/10.1192/bjp.bp.108.051193.

Hedges, L.V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.

Higgins, J.P.T., Altman, D.G., Gotzsche, P.C., Juni, P., Moher, D., Oxman, A.D., et al. (2011). The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ, 343(oct18 2)http://dx.doi.org/10.1136/bmj.d5928 d5928–d5928.

Hogg, J. A., & Deffenbacher, J. L. (1988). A comparison of cognitive and interpersonal-pro- cess group therapies in the treatment of depression among college students. Journal of Counseling Psychology, 35(3), 304–310. http://dx.doi.org/10.1037/0022-0167.35.3. 304.

Hollon, S.D., DeRubeis, R.J., & Evans, M.D. (1987). Causal mediation of change in treatment for depression: Discriminating between nonspecificity and noncausality. Psychological Bulletin, 102(1), 139–149.

Hollon, S.D., DeRubeis, R.J., Shelton, R.C., Amsterdam, J.D., Salomon, R.M., O'Reardon, J.P., et al. (2005). Prevention of relapse following cognitive therapy vs medications in moderate to severe depression. Archives of General Psychiatry, 62(4), 417–422. http://dx.doi.org/10.1001/archpsyc.62.4.417.

Hollon, S.D., & Kendall, P.C. (1980). Cognitive self-statements in depression: Development of an automatic thoughts questionnaire. Cognitive Therapy and Research, 4(4), 383–395. http://dx.doi.org/10.1007/BF01178214.

Imber, S. D., Pilkonis, P. A., Sotsky, S. M., Elkin, I., Watkins, J. T., Collins, J. F., ... Glass, D. R. (1990). Mode-specific effects among three treatments for depression. Journal of Consulting and Clinical Psychology, 58(3), 352–359.

Ioannidis, J.P.A., Patsopoulos, N.A., & Evangelou, E. (2007). Uncertainty in heterogeneity estimates in meta-analyses. BMJ (Clinical Research Ed.), 335(7626), 914–916. http:// dx.doi.org/10.1136/bmj.39343.408449.80.

Jacobson, N.S., Dobson, K.S., Truax, P.A., Addis, M.E., Koerner, K., Gollan, J.K., et al. (1996). A component analysis of cognitive-behavioral treatment for depression. Journal of Consulting and Clinical Psychology, 64(2), 295–304.

Jamison, C., & Scogin, F. (1995). The outcome of cognitive bibliotherapy with depressed adults. Journal of Consulting and Clinical Psychology, 63(4), 644–650.

Kazdin, A.E. (2007). Mediators and mechanisms of change in psychotherapy research. Annual Review of Clinical Psychology, 3, 1–27. http://dx.doi.org/10.1146/annurev. clinpsy.3.022806.091432.

Lau, M.A., Segal, Z.V., & Williams, J.M.G. (2004). Teasdale's differential activation hypoth- esis: Implications for mechanisms of depressive relapse and suicidal behaviour. Behaviour Research and Therapy, 42(9), 1001–1017. http://dx.doi.org/10.1016/j.brat. 2004.03.003.

Longmore, R.J., & Worrell, M. (2007). Do we need to challenge thoughts in cognitive be- havior therapy? Clinical Psychology Review, 27(2), 173–187. http://doi.org/10.1016/j. cpr.2006.08.001

Lorenzo-Luaces, L., German, R.E., & DeRubeis, R.J. (2015). It's complicated: The relation be- tween cognitive change procedures, cognitive change, and symptom change in cog- nitive therapy for depression. Clinical Psychology Review, 41, 3–15.

McKnight, D. L., Nelson-Gray, R. O., & Barnhill, J. (1992). Dexamethasone suppression test and response to cognitive therapy and antidepressant medication. Behavior Therapy, 23(1), 99–111. http://dx.doi.org/10.1016/S0005-7894(05)80311-9.

McNamara, K., & Horan, J. J. (1986). Experimental construct validity in the evaluation of cognitive and behavioral treatments for depression. Journal of Counseling Psychology, 33(1), 23–30. http://dx.doi.org/10.1037/0022-0167.33.1.23.

Miranda, J., Persons, J.B., & Byers, C.N. (1990). Endorsement of dysfunctional beliefs depends on current mood state. Journal of Abnormal Psychology, 99(3), 237–241.

Moore, M.T., Fresco, D.M., Segal, Z.V., & Brown, T.A. (2014). An exploratory analysis of the factor structure of the Dysfunctional Attitude Scale—Form A (DAS). Assessment, 21(5), 570–579. http://dx.doi.org/10.1177/1073191114524272.

Oei, T.P.S., & Free, M.L. (1995). Do cognitive behaviour therapies validate cognitive models of mood disorders? A review of the empirical evidence. International Journal of Psychology, 30(2), 145–180. http://dx.doi.org/10.1080/00207599508246564.

Orsini, N., Bottai, M., Higgins, J., & Buchan, I. (2006). HETEROGI: Stata module to quantify heterogeneity in a meta-analysis. Stata (Retrieved from http://econpapers.repec.org/ software/bocbocode/s449201.htm).

Quilty, L. C., McBride, C., & Bagby, R. M. (2008). Evidence for the cognitive mediational model of cognitive behavioural therapy for depression. Psychological Medicine, 38(11), 1531–1541. http://dx.doi.org/10.1017/S0033291708003772.

Scogin, F., Hamblin, D., & Beutler, L. (1987). Bibliotherapy for depressed older adults: a self-help alternative. The Gerontologist, 27(3), 383–387.

Scogin, F., Jamison, C., & Gochneaur, K. (1989). Comparative efficacy of cognitive and be- havioral bibliotherapy for mildly and moderately depressed older adults. Journal of Consulting and Clinical Psychology, 57(3), 403–407.

Selmi, P. M., Klein, M. H., Greist, J. H., Sorrell, S. P., & Erdman, H. P. (1991). Computer-ad- ministered therapy for depression. M.D. Computing : Computers in Medical Practice, 8(2), 98–102.

Simons, A. D., Garfield, S. L., & Murphy, G. E. (1984). The process of change in cognitive therapy and pharmacotherapy for depression. Changes in mood and cognition. Archives of General Psychiatry, 41(1), 45–51.

Strunk, D.R., Brotman, M.A., & DeRubeis, R.J. (2010). The process of change in cognitive therapy for depression: Predictors of early inter-session symptom gains. Behaviour Research and Therapy, 48(7), 599–606. http://dx.doi.org/10.1016/j.brat.2010.03.011.

Thompson, L. W., Gallagher, D., & Breckenridge, J. S. (1987). Comparative effectiveness of psychotherapies for depressed elders. Journal of Consulting and Clinical Psychology, 55(3), 385–390.

Warmerdam, L., van Straten, A., Twisk, J., Riper, H., & Cuijpers, P. (2008). Internet-based treatment for adults with depressive symptoms: randomized controlled trial. Journal of Medical Internet Research, 10(4), e44. http://dx.doi.org/10.2196/jmir.1094.

Wampold, B.E. (2001). The great psychotherapy debate: Models, methods, and findings. Mahwah, NJ: Lawrence Erlbaum Associates.

Watson, C.G., Vassar, P., Plemel, D., Herder, J., Manifold, V., & Anderson, D. (1990). A factor analysis of Ellis' irrational beliefs. Journal of Clinical Psychology, 46(4), 412–415.

Watson, J. C., Gordon, L. B., Stermac, L., Kalogerakos, F., & Steckley, P. (2003). Comparing the effectiveness of process-experiential with cognitive-behavioral psychotherapy in the treatment of depression. Journal of Consulting and Clinical Psychology, 71(4), 773–781.

Webb, C.A., Derubeis, R.J., & Barber, J.P. (2010). Therapist adherence/competence and treatment outcome: A meta-analytic review. Journal of Consulting and Clinical Psychology, 78(2), 200–211. http://dx.doi.org/10.1037/a0018912.

Weissman, A.N., & Beck, A.T. (1978). Development and validation of the Dysfunctional Attitudes Scale. Presented at the American Educational Research Association. Toronto: Ontario, Canada.

Wilson, P. H., Goldin, J. C., & Charbonneau-Powis, M. (1983). Comparative efficacy of be- havioral and cognitive treatments of depression. Cognitive Therapy and Research, 7(2), 111–124. http://dx.doi.org/10.1007/BF01190064.

Whisman, M.A. (1993). Mediators and moderators of change in cognitive therapy of depression. Psychological Bulletin, 114(2), 248–265.

Wong, D. F. K. (2008). Cognitive and Health-Related Outcomes of Group Cognitive Behav- ioural Treatment for People With Depressive Symptoms in Hong Kong: Randomized Wait-List Control Study. Australian and New Zealand Journal of Psychiatry, 42(8), 702–711. http://dx.doi.org/10.1080/00048670802203418.

Wright, J. H., Wright, A. S., Albano, A. M., Basco, M. R., Goldsmith, L. J., Raffield, T., & Otto, M. W. (2005). Computer-assisted cognitive therapy for depression: maintaining effi- cacy while reducing therapist time. The American Journal of Psychiatry, 162(6), 1158–1164. http://dx.doi.org/10.1176/appi.ajp.162.6.1158.

Zettle, R. D., & Rains, J. C. (1989). Group cognitive and contextual therapies in treatment of depression. Journal of Clinical Psychology, 45(3), 436–445.