due is 24 hours

1 3

Eur Child Adolesc Psychiatry (2017) 26:47–55 DOI 10.1007/s00787-016-0863-0

ORIGINAL CONTRIBUTION

Defining cognitive‑behavior therapy response and remission in pediatric OCD: a signal detection analysis of the Children’s Yale‑Brown Obsessive Compulsive Scale

Gudmundur Skarphedinsson1 · Alessandro S. De Nadai2 · Eric A. Storch2,3,4,5 · Adam B. Lewin2 · Tord Ivarsson1

Received: 12 June 2015 / Accepted: 4 May 2016 / Published online: 21 May 2016 © Springer-Verlag Berlin Heidelberg 2016

Keywords Obsessive–compulsive disorder · Cognitive- behavioral treatment · Children’s Yale-Brown Obsessive– Compulsive Scale · Treatment

Introduction

Obsessive–compulsive disorder (OCD) is a chronic and disabling disorder [1–3] with a population prevalence rate of 1–2 % [2, 4–6]. Studies have shown the efficacy of cognitive-behavior therapy (CBT) and serotonin reuptake inhibitors [7, 8]. Expert guidelines recommend CBT as the first-line treatment for mild and moderate OCD, and com- bined medication and CBT for moderate to severe OCD [9] and unsuccessful CBT [9, 10].

The Children’s Yale-Brown Obsessive–Compulsive Scale (CY-BOCS) [11] is the gold standard; dimensional measure of symptom severity in pediatric OCD is the most commonly used treatment outcome measure [12]. It is a semi-structured interview of OCD symptomatology with acceptable psychometric properties [11, 13, 14]. Other measures frequently used are the Clinical Global Impres- sions Scales of severity and improvement (CGI-S/CGI-I) [15, 16] measuring the overall illness severity and treat- ment improvement, respectively.

Despite the existence of these instruments, few defini- tions for characterizing treatment response (no longer fully symptomatic, but may continue to evidence some minimal symptoms, e.g., [17]) or remission (no longer meets syn- dromal criteria [18]) are consistently used or have empiri- cal support. Consequently, it has been difficult to compare outcomes across studies and apply clinical results to clini- cal practice [14]. For instance, the operational definition of treatment response has ranged from 20 % [19] to 50 % [20, 21] symptom reduction on the CY-BOCS (See [17] for a

Abstract The objective of the study was to examine the optimal Children’s Yale-Brown Obsessive–Compulsive Scale (CY-BOCS) percent reduction and raw cutoffs for predicting cognitive-behavioral treatment (CBT) response among children and adolescents with obsessive–compul- sive disorder (OCD). The sample consisted of children and adolescents with OCD (N = 241) participating in the first step of the Nordic long-term OCD treatment study and receiving 14 weekly sessions of CBT in the form of expo- sure and response prevention. Evaluations were conducted pre- and post-treatment, included the CY-BOCS, Clinical Global Impressions—severity/improvement. The results showed that the most efficient CY-BOCS cutoffs were 35 % reduction for treatment response, 55 % reduction for remission, and a post-treatment CY-BOCS raw total score of 11 for treatment remission. Overall, our results diverge from previous research on pediatric OCD with more con- servative cutoffs (higher cutoff reduction for response and remission, and lower raw score for remission). Further research on optimal cutoffs is needed.

* Gudmundur Skarphedinsson [email protected]

1 Center for Child and Adolescent Mental Health, Eastern and Southern Norway, Nydalen, Postbox 4623, 0405 Oslo, Norway

2 Department of Pediatrics, Psychology and Psychiatry, University of South Florida, St. Petersburg, FL, USA

3 Department of Health Policy and Management, University of South Florida, Tampa, FL, USA

4 Rogers Behavioral Health–Tampa Bay, Tampa, FL, USA 5 All Children’s Hospital, Johns Hopkins Medicine, St.

Petersburg, FL, USA

48 Eur Child Adolesc Psychiatry (2017) 26:47–55

1 3

thorough overview). Similarly, the operational definition of treatment remission has varied from a raw score of 7 [22], 8 [23], 10 [24–26], and 12 [27] on the post-treatment CY- BOCS total score.

Previous studies in OCD and anxiety [17, 28–30] sug- gest using the rating for improvement (CGI-I) and severity (CGI-S) conducted by the clinician as a benchmark. Typi- cally signal detection analysis (receiver operational char- acteristics) [31] has been used to compare judgments of response and remission with symptom reduction on symp- tom measures (e.g., the CY-BOCS). In particular, signal detection analysis focuses on classifying the rate of true and false diagnostic judgments at various thresholds (cut- offs) to identify the sensitivity and specificity of a measure at various discrimination thresholds. Storch et al. [29] used this method by comparing the CGI-I/S with the CY-BOCS total score in 109 children and adolescents that were treated with CBT for 14 weeks. The most efficient CY-BOCS cut- off score for treatment response was a 25 % reduction or higher with positive and negative predictive values of .96 and .86, respectively. This means that 96 % of all par- ticipants that experienced 25 % reduction or higher truly responded according to the CGI-I, and 86 % of the par- ticipants with lower than 25 % reduction did not respond according to the CGI-I.

Likewise, 45 % reduction or higher was the most effec- tive cutoff for remission with positive and negative pre- dictive values of .96 and .72, respectively. That means that 96 % of all participants with 45 % reduction or more were true remitters according to the CGI-S, and 72 % of all participants with less than 45 % reduction did not remit according to the CGI-S. A CY-BOCS total raw score of 14 or less was the most effective cutoff for remission based on the CGI-I with positive and negative predictive value of .96 and .79, respectively. This means that 96 % of all participants with a total raw score of 14 or less were true remitters according to the CGI-S, and 79 % of all partici- pants with a total raw score of 15 or higher did not remit according to the CIG-S. Among adults, 30 or 45 % reduc- tion on the Yale-Brown Obsessive–Compulsive Scale was the most efficient cutoff for response using the CGI-I, and 40–55 % reduction was optimal cutoff for remission using the CGI-S [17, 30]. Twelve was the most optimal raw cut- off score for remission [30]. The aim of this study is to replicate Storch et al. [29] using a large and cross-cultural sample of pediatric OCD patients treated with CBT in community child and adolescent clinics in Denmark, Swe- den, and Norway within the Nordic long-term OCD treat- ment study (NordLOTS) [32]. NordLOTS offers a large and unique sample (N = 269) consisting of well-charac- terized youth who were medication-free (SSRI) prior to treatment initiation. More specifically, this investigation

examines whether the proposed cutoffs identified by Storch et al. are consistent with this cross-cultural sample from the NordLOTS study.

Methods

Participants

Two hundred and sixty-nine participants (138 females) with a primary OCD diagnosis were included in the Nor- dLOTS Step 1. All participants were free from medication specifically intended for OCD such as serotonin reuptake inhibitors or antipsychotics. The design and methods of the NordLOTS trial have been described elsewhere [26, 32, 33]. Briefly, NordLOTS included three main steps. For Step 1, all participants received 14 weekly sessions of family-based CBT in the form of exposure and response prevention.

The mean baseline CY-BOCS total score was 24.6 (SD = 5.1), subscale scores for obsession 12.3 (SD = 2.8) and compulsion 12.3 (SD = 2.7), which is comparable to the average range for treatment-seeking youth in OCD specialty centers [14]. Child age ranged from 7 to 17 years (M = 12.8, SD = 2.7). The sample was primarily of Scan- dinavian ethnicity as 97 % (n = 261) had one or both par- ents of Scandinavian origin. Almost half (40.5 %, n = 109) of the patients had one or more comorbid psychiatric disor- der, with common comorbid conditions including anxiety disorders (19.3 %, n = 52), tic disorders (18.6 %, n = 49), and attention-deficit hyperactivity disorder (8.9 %, n = 24), depressive disorders (3.7 %, n = 10), and oppositional defi- ant disorder/conduct disorder (3.7 %, n = 10).

Patients were included in the study if they fulfilled the following criteria: (1) primary diagnosis of OCD in accord- ance with the criteria in the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revi- sion (DSM-IV) [34], (2) CY-BOCS entry score ≥16, (3) 7–17 years of age, and (4) patients with attention-deficit hyperactivity disorder (ADHD) were eligible, after having been stabilized on medication for at least 3 months prior to entry. Exclusion criteria included: (1) the presence of other psychiatric disorders having a higher treatment prior- ity (i.e., psychosis and severe depression), (2) any specific developmental disorder (i.e., autism spectrum disorders). However, a diagnosis of PDD NOS was allowed as long as OCD was judged to be the primary disorder based on the respective clinical global impression-severity (CGI) scores, (3) a previous failed trial of exposure-based CBT for OCD within 6 months of inclusion, (4) medication treatment with an SRI less than 6 months of inclusion, and (5) inadequate language proficiency by the patient or the parent. See Torp

49Eur Child Adolesc Psychiatry (2017) 26:47–55

1 3

and colleagues [26] for a detailed description of the Nord- LOTS sample.

Procedure

The trial was approved by the Norwegian, Swedish and Danish Committees for Medical and Health Research Eth- ics and the Medical Products Agencies. The project was registered in Current Controlled Trials (www.controlled- trials.com ISRCTN66385119). Informed consent was provided by parent(s) or guardian(s), and informed assent obtained from children 11 years of age or older. A diagno- sis of OCD was made prior to the initial assessment (see below). Assessment points were at baseline, weeks 7 and 14 (post-treatment), assessed by independent evaluators (IEs) using the CY-BOCS.

After assessment at baseline, participants received 14 weekly sessions of exposure-based CBT regime consisting of 75 min. Parents were expected to accompany their chil- dren to all sessions. The children were seen together with their parents in 6 of the 14 sessions (sessions 1–3, 5, 11, and 14). In the remaining sessions, the child was treated individually for 45 min, and then, the parents were seen with or without the child for an additional 30 min. This extra time was added specifically to address issues regard- ing the parents’ involvement in therapy and their attitude and feelings about their child’s OCD symptoms. Treatment is detailed in Torp et al. [26, 35].

Measures

The Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime Version (K-SADS-PL) [36] is a semi-structured diagnostic inter- view that assesses a variety of child psychopathology. The K-SADS-PL demonstrates favorable psychometric prop- erties, with an excellent inter-rater reliability of 98 % and a 1–5-week test–retest kappa of .80 for anxiety disorder diagnosis [36]. Convergent and divergent validity [37] and the inter-rater reliability [38] of the K-SADS-PL have been documented in a Nordic sample of adolescents. Also, the K-SADS-PL has been used in previous OCD treatment tri- als [39, 40]. Symptoms can be classified as “not present”, “possible”, “in remission” or “certain”. In this study, OCD diagnoses and comorbidity were based on symptoms clas- sified as “certain” only. The K-SADS-PL was used for diagnostic assessment at baseline of Step 1. All interviews were conducted by experienced clinicians, trained by the NordLOTS research group.

The CY-BOCS evaluates the severity of obsessions and compulsions, using ten items across five dimensions (time occupied by symptoms, interference, distress, resistance, and degree of control over symptoms). The total severity

score can range from 0 to 40. CY-BOCS total scores in the range of 14–24 are considered moderate, 25–30 moderate- severe, and over 30 severe (based on a normative study of 815 treatment-seeking youth [14]). The CY-BOCS shows reasonable reliability and validity [41–43]. In particular, high internal consistency for the total score .87 [41] and good to excellent inter-rater agreement [intra-class cor- relation coefficient (ICC)] have been reported [41], .84, . 91, and .68 for total score, obsessions, and compulsions, respectively). In the NordLOTS sample, the inter-rater agreement (ICC) was .92, .94, and .87 for total score, obsessions, and compulsions, respectively [26].

The clinical global impression scales [44] are two single-item clinician ratings that measure severity and improvement. The CGI-S scale was used by the clinician to rate the global severity of symptom severity. Ratings range from 0 (no illness) to 6 (extremely severe). It correlates strongly with the CY-BOCS total score in pediatric OCD patients, and is widely used and has been shown to be treat- ment-sensitive [15]. Consistent with Storch et al. [29] we used a score of 0 or 1 (no illness or mild illness) for remis- sion. The CGI-I scale was used to assess overall clinical improvement based on symptoms observed and impairment reported using a seven-point scale ranging from 0 (very much worse) to 6 (very much improved). Consistent with Storch et al. [29] ratings of 5 (much improved) or 6 (very much improved) designated treatment response.

Statistical analysis

To evaluate the properties of test cutoffs, we used receiver operating characteristic (ROC) analyses that are origi- nally based on signal detection theory [31]. We evaluated a number of ROC parameters, including sensitivity, speci- ficity, positive predictive value, negative predictive value, and efficiency. Sensitivity is defined as the proportion of true positives captured by the test cutoff, and specificity is defined as the proportion of true negatives that fall below the test cutoff. The positive predictive value is the probabil- ity that all patients who meet the gold standard also exceed the test cutoff, and negative predictive value is the probabil- ity that all patients who do not meet the gold standard do not exceed the cutoff. Efficiency (also called accuracy) is the probability that the cutoff diagnosis and the gold stand- ard agree.

Consistent with analyses in previous studies [17, 29] we broke up the CY-BOCS percentage reduction into 5 % interval cutoffs to examine the different discrimination accuracy compared to remission (CGI-S) and response (CGI-I). We also examined the properties of different CY- BOCS raw score cutoffs in predicting symptom remission (CGI-S). However, raw scores were not used to predict response.

50 Eur Child Adolesc Psychiatry (2017) 26:47–55

1 3

We also used quality receiver operation characteristic methods (QROC) to correct for errors in the CGIs [45, 46]. QROC reports specific forms of weighted κ statistics to meas- ure quality of specificity κ(0.0), quality of efficiency κ(0.5), and, quality of sensitivity κ(1.0). For each measure, a value of 0 indicates that one cannot discriminate the cutoff from chance, while a value of 1.00 indicates that one can perfectly discriminate between patients that responded/remitted. As previous studies [29], the current study focused on maximiz- ing efficiency κ(0.5) by selecting the most efficient cutoffs and minimizing false-positive and false-negative equally. We conducted the analyses using the ROC4 program (available at http://www.stanford.edu/~yesavage/ROC.html).

Results

Missing data

Data with both participants’ baseline and week-14 meas- ures were included. No data were missing for the CGI and CY-BOCS at baseline. At week 14, 29 (10.8 %) of the CY- BOCS were missing and 30 (11.1 %) of the CGI ratings made by therapists. We compared whether completers and non-completers differed on all predictors (used in a pre- vious study [35]) at baseline and week 7. No significant group differences were detected at the p < .05 level.

Sample characteristics

The mean CY-BOCS at baseline was 24.60 (SD = 6.56), and at week 14, 11.40 (6.70). As reported in Torp et al.

[26] the pre-post difference was statistically significant (F[1, 479] = 130.434, p < .001). The mean reduction from baseline to week 14 was 52.9 % (SD = 30.9 %). The mean baseline CGIs was 3.43 (SD = 0.84), and the mean week- 14 CGIS was 1.78 (SD = 1.19). The difference between baseline and week-14 was significant using a paired sam- ples t test (t(215) = 17.86, p < .001). The mean reduction was 54.5 % (SD = 39.6 %). The mean CGI-I at week 14 was 4.82 (SD = 1.00). At week 14, 48.3 % (n = 130) met criteria for remission using the CGIS. Similarly, 61.7 % (n = 166) were responders using the CGI-I.

Predicting treatment response with CY‑BOCS percent reduction

Table 1 shows series of CY-BOCS percentage symptom reduction cutoffs used to predict treatment response using the dichotomized CGI-I, along with ROC statistics for each cutoff. We conducted ROC analyses for each 5 % symptom reduction on the CY-BOCS. We found maximum efficiency (.61) at a cutoff of 35 % with positive predictive value of .88 and negative predictive value of .74, indicating a false- positive rate of 12 % and a false-negative rate of 26 %.

Predicting remission with CY‑BOCS percent reduction

Table 2 shows a series of CY-BOCS percent reduction cut- offs and ROC analyses. The maximum quality of efficiency (.74) was found at 55 % symptom reduction on the CY- BOCS, with positive and negative predictive values at .83 and .92, respectively, corresponding to false-positive rate of 17 % and false-negative rate of 8 %.

Table 1 Prediction of treatment response at varying CY-BOCS percentage reduction cutoffs

* Optimal cut-off value based on the Kappa (κ(0.5))

Value Sensitivity Specificity Efficiency κ(0) κ(0.5) κ(1) Positive predictive value

Negative predictive value

≥5 1.00 0.24 0.77 0.18 0.31 1.00 0.75 1.00 ≥10 0.99 0.27 0.77 0.20 0.33 0.93 0.75 0.95 ≥15 0.99 0.34 0.79 0.25 0.40 0.89 0.77 0.93 ≥20 0.96 0.38 0.78 0.28 0.40 0.75 0.78 0.82 ≥25 0.95 0.45 0.79 0.33 0.45 0.69 0.79 0.79 ≥30 0.93 0.58 0.82 0.46 0.55 0.69 0.83 0.78 ≥35* 0.89 0.72 0.83 0.60 0.61 0.62 0.88 0.74 ≥40 0.85 0.76 0.82 0.63 0.59 0.55 0.89 0.69 ≥45 0.78 0.82 0.80 0.71 0.60 0.46 0.91 0.63 ≥50 0.73 0.84 0.76 0.71 0.50 0.39 0.91 0.58 ≥55 0.68 0.88 0.74 0.76 0.47 0.34 0.93 0.55 ≥60 0.60 0.89 0.69 0.76 0.41 0.28 0.93 0.50 ≥60 0.49 0.89 0.62 0.71 0.30 0.19 0.91 0.44 ≥70 0.42 0.91 0.57 0.71 0.25 0.15 0.91 0.41

51Eur Child Adolesc Psychiatry (2017) 26:47–55

1 3

Comparison of treatment response and remission

Figure 1 depicts the κ(0.5) over the series of percent reduction cutoffs on the CY-BOCS on predicting treatment response and remission. Response shows that the maximal κ(0.5) is at 35 % reduction on the CY-BOCS, while the maximal κ(0.5) for remission is at 55 % reduction.

Predicting remission with CY‑BOCS total scores

Table 3 shows a series of CY-BOCS post-treatment raw cutoff scores with ROC analyses that we used to predict the dichotomized CGI-S symptom reduction (mild or no ill- ness). Maximal quality of efficiency (.69) was CY-BOCS

raw cutoff score of ≤11 with positive predictive value of .88 (corresponding to 12 % false-positive) and negative predic- tive value .81 (corresponding to 19 % false-negative).

Discussion

It is important to standardize treatment outcome criteria so as to facilitate comparisons across trials and establishment of treatment guidelines. Currently, the inconsistent definition of terms such as response or remission is the rule [17, 47, 48]. For instance, the range of cutoffs using the CY-BOCS per- cent reduction has varied from 20 to 50 % in different trials. This variation substantially hinders the ability to determine

Table 2 Prediction of treatment remission (per CGI-S at varying CY-BOCS reduction cutoffs

* Optimal cut-off value based on the Kappa (κ(0.5))

Value Sensitivity Specificity Efficiency κ(0) κ(0.5) κ(1) Positive predictive value

Negative predictive value

≥5 1.00 0.14 0.53 0.07 0.13 1.00 0.50 1.00 ≥10 1.00 0.16 0.55 0.08 0.15 1.00 0.50 1.00 ≥15 1.00 0.21 0.57 0.11 0.19 1.00 0.52 1.00 ≥20 1.00 0.26 0.60 0.14 0.25 1.00 0.53 1.00 ≥25 1.00 0.32 0.63 0.18 0.30 1.00 0.56 1.00 ≥30 0.99 0.42 0.68 0.24 0.39 0.96 0.59 0.98 ≥35 0.99 0.55 0.75 0.35 0.52 0.97 0.65 0.99 ≥40 0.98 0.61 0.78 0.41 0.57 0.95 0.68 0.98 ≥45 0.95 0.70 0.81 0.50 0.63 0.87 0.73 0.94 ≥50 0.94 0.77 0.85 0.58 0.69 0.86 0.77 0.93 ≥55* 0.91 0.84 0.87 0.68 0.74 0.82 0.83 0.92 ≥60 0.82 0.86 0.84 0.69 0.68 0.67 0.83 0.85 ≥60 0.69 0.89 0.80 0.71 0.59 0.51 0.84 0.77 ≥70 0.64 0.95 0.80 0.83 0.60 0.47 0.91 0.75

Fig. 1 Quality index of effi- ciency [κ(0.5)] for the predic- tive values of the CY-BOCS percent cutoffs corresponding to response and remission using CGI-I and CGI-S

52 Eur Child Adolesc Psychiatry (2017) 26:47–55

1 3

the most favorable target for OCD symptom reduction across trials. When a criterion is too conservative, fewer true posi- tives are identified (decreased sensitivity). Likewise, when a criterion is less conservative, one captures a higher propor- tion of true positives (sensitivity is increased) but with less specificity. The aim of the current study was to replicate previous research [29] in a Scandinavian-majority sample by examining series of CY-BOCS percent reduction that cor- respond with treatment response and remission, and series of raw CY-BOCS scores that correspond to remission.

We found that 35 % reduction on the CY-BOCS had optimal correspondence (i.e., sensitivity and specific- ity were equally optimal at that cutoff) with treatment response. The rate of false-positives (children classi- fied as responders without substantial improvement) was 12 %, and the rate of false-negatives (failure to classify true responders) was 26 %. These results are in line with previous research, both the recent Delphi survey among 468 experts in pediatric and/or adult OCD [49] and stud- ies using the same analyses as us with adult OCD data [30, 50]. However, the results are somewhat different than those previously found in pediatric OCD where Storch and colleagues [29] found optimal cutoff at 25 % symp- tom reduction. Also, their false-positive (4 %) and false- negative (14 %) rates were lower, indicating better classi- fication. One explanation for the differences in the percent reduction threshold is that a higher symptom reduction may have corresponded to greater improvement in this Scandinavian sample compared to the sample in Storch

and colleagues. The consequence of lowering the thresh- old corresponding to Storch and colleagues (25 % reduc- tion) is that one increases children classified as responders without substantial improvement from 12 to 21 %. How- ever, the false-negative rate decreases from 26 to 21 %. Consequently, this produces a lower quality of efficiency.

With regards to remission, a 55 % symptom reduction was identified as optimal with 17 % false-positive rate and 8 % false-negative rate. This is a similar cutoff as found by Storch and colleagues [29] who showed that 50 % symp- tom reduction had optimal correspondence with remission with false-positive rate of 4 % and false-negative rate of 28 %. Based on our data, 50 % symptom reduction cut- off means that we increase the false-positive rate to 23 % (increase in children classified as responders without sub- stantial improvement), while decreasing the false-negative rate to 8 % (reducing the failure to classify true respond- ers). We found that the raw cutoff score of 11 was optimal correspondence with remission with false-positive rate of 12 % and false-negative rate of 19 %. This cutoff is simi- lar to many treatment studies that have consensually used a post-treatment raw score of 10 or lower on the CY-BOCS as an indication of remission [24–26] and to the recent results of the Delphi survey showing that a post-treatment raw score of 12 or lower indicated remission [49]. How- ever, Storch and colleagues found a higher post-treatment raw cutoff score (CY-BOCS ≤ 14) than in the current study. Their cutoff score had a maximum efficiency with 4 % false-positives and 21 % false-negatives. Based on our

Table 3 Prediction of treatment remission (per CGI-S at varying CY-BOCS raw cutoff scores according to therapists’ and independent evaluator (IE), respectively

* Optimal cut-off value based on the Kappa (κ(0.5))

Value Sensitivity Specificity Efficiency κ(0) κ(0.5) κ(1) Positive predictive value

Negative predictive value

≤5 0.99 0.45 0.74 0.30 0.45 0.93 0.68 0.96 ≤6 0.97 0.51 0.76 0.35 0.50 0.88 0.70 0.93 ≤7 0.96 0.56 0.78 0.39 0.53 0.86 0.72 0.92 ≤8 0.95 0.60 0.79 0.43 0.57 0.85 0.74 0.92 ≤9 0.94 0.72 0.84 0.56 0.67 0.83 0.80 0.91 ≤10 0.87 0.80 0.84 0.64 0.67 0.70 0.84 0.84 ≤11* 0.82 0.87 0.85 0.75 0.69 0.64 0.88 0.81 ≤12 0.79 0.90 0.84 0.79 0.69 0.60 0.90 0.79 ≤13 0.70 0.96 0.82 0.89 0.64 0.50 0.95 0.73 ≤14 0.65 0.99 0.80 0.97 0.62 0.45 0.99 0.70 ≤15 0.59 1.00 0.78 1.00 0.56 0.39 1.00 0.67 ≤16 0.51 1.00 0.73 1.00 0.49 0.32 1.00 0.63 ≤17 0.47 1.00 0.71 1.00 0.45 0.29 1.00 0.61 ≤18 0.42 1.00 0.69 1.00 0.40 0.25 1.00 0.59 ≤19 0.37 1.00 0.66 1.00 0.35 0.21 1.00 0.57 ≤20 0.33 1.00 0.64 1.00 0.31 0.19 1.00 0.56

53Eur Child Adolesc Psychiatry (2017) 26:47–55

1 3

data, the consequences of increasing the raw cutoff score of 14 would result in decreasing the quality of efficiency slightly (.62), the false-positive rate would decrease (1 %) (less children classified as responders without substantial improvement). However, this also results in a high increase of false-negatives of 30 % (increasing the failure to classify true responders).

Taken together, we found similar percentage symptom reduction cutoff for remission as Storch and colleagues [29]. However, the symptom reduction cutoff for response and the raw cutoff score for remission were more conserva- tive than those found by Storch and colleagues. However, the quality of the efficiency was generally higher in Storch and colleagues (.77, .72, and .77 for CY-BOCS percent reduction vs. response, remission, and CY-BOCS raw score vs. remission respectively). In our study, the quality of effi- ciency was .61, .74, and .69 for CY-BOCS percent reduc- tion vs. response, remission, and CY-BOCS raw score vs. remission, respectively.

Not surprisingly, remission had a higher threshold than response, consistent with the concepts of response and remission [18]. As response is characterized as an improve- ment in symptoms following treatment (e.g., 5–50 % [51–53]), some responders, although they benefited from a treatment, may still have an active disorder that needs fur- ther improvement. However, remission is a more stringent criterion that requires that patients improve to the degree of virtual absence of syndromal criteria [18, 47, 54].

Strengths and limitations

Within the strengths of this study (e.g., well-character- ized sample, methodological rigor, multi-site nature), all patients were free from OCD medications, and all under- went the same standardized 14 weekly sessions of CBT. Also, treatment was conducted in regular community child and adolescent psychiatry clinics, and few exclusionary cri- teria were applied, making the results applicable to clinic- referred youth.

However, the study has several limitations. For instance, our kappa values for the correspondence between percent reduction and response, and CY-BOCS raw score and remission are lower than those reported in Storch and col- leagues, possibly indicating more variation in scoring the CGIs and the CY-BOCS. Whether this indicates real dif- ferences which may be attributed to factors, such as cross- cultural differences or measurement errors of the translated CY-BOCS merits evaluation in subsequent studies. Also, the CGI ratings were scored by raters doing the CY-BOCS and may have been influenced by the CY-BOCS. We also note as a limitation that the inter-rater agreement of the K-SADS has not been evaluated. However, the K-SADS

diagnoses were obtained by combining all available data to generate diagnosis [55]. Conducting an invariance testing or confirmatory factor analysis (CFA) by combining both datasets is warranted but beyond the scope of this paper. A separate paper will report the results of the CFA for Scandi- navian and US versions of the CY-BOCS [56].

Implications

Our findings have implications for research and practice. By establishing percent reduction and raw total score on the CY-BOCS that corresponds to response and remission clinicians and researchers alike can more easily assess the treatment progress of individual child against the standard outcomes obtained in clinical trials. Established cutoffs can assist in decision-making, for instance, by informing whether to switch or augment treatment in the absence of response, continue treatment in the presence of response without remission, or provide booster sessions rather than regular treatment in the presence of remission. For example, if a patient does not obtain 35 % symptom reduction after 14 weeks of CBT, the clinician might con- sider changing the treatment. However, if the patient has obtained at least 35 % symptom reduction but still has a high CY-BOCS total score (above 11), he should receive further CBT sessions until he reaches the threshold of remission.

Acknowledgments The authors would like to thank the patients and their parents that participated in the Nordic long-term OCD treatment study (NordLOTS) and the NordLOTS researcher group. Funding was applied to each national site as well as some central funding. We thank the following for their contributions: Trygfonden, The Danish Council for Strategic Research, Pulje til styrkelse af psykiatrisk for- skning i Region Midtjylland, The Center for Child and Adolescent Mental Health, Eastern and Southern Norway (RBUP), Stiftelsen Clas Groschinskys Minnesfond, Norwegian Research Council, Norwe- gian ExtraFoundation, and (De Nadai) National Institutes of Health under Ruth L. Kirschstein National Research Service Award number F31MH094095 from the National Institute of Mental Health.

Compliance with ethical standards

Conflicts of interest Tord Ivarsson is involved in Speakers Bureau for Shire, Sweden. Dr. Storch has received grant funding from NIH, the Centers for Disease Control and Prevention, the Agency for Healthcare Research and Quality, the International OCD Foundation, and Ortho- McNeil Scientific Affairs Pharmaceuticals. He has received textbook honorarium from Springer, the American Psychological Association, Wiley Publishers, and Lawrence Erlbaum Associates. He is an educa- tional consultant for Rogers Memorial Hospital. He serves as a con- sultant for Prophase, Inc. and CroNos, Inc., and serves on the Speak- er’s Bureau and Scientific Advisory Board of the International OCD Foundation. He has received research support from the AllChildren’s Hospital Guild Endowed Chair.Dr. Adam Lewin Research support from the International OCD Foundation and All Children’s Hospi- tal. Honorarium from Oxford Press, Springer, and Children’s Tumor Foundation. Travel Support from the Tourette Syndrome Association,

54 Eur Child Adolesc Psychiatry (2017) 26:47–55

1 3

American Psychological Association, Society for Clinical Child and Adolescent Psychology. Scientific Advisory Board for International OCD Foundation.

References

1. Rasmussen SA, Eisen JL (1992) The epidemiology and clinical features of obsessive compulsive disorder. Psychiatr Clin North Am 15:743–758

2. Ruscio AM, Stein DJ, Chiu WT, Kessler RC (2010) The epi- demiology of obsessive–compulsive disorder in the national comorbidity survey replication. Mol Psychiatry 15:53–63

3. Stewart SE, Geller DA, Jenike M, Pauls D, Shaw D, Mullin B, Faraone SV (2004) Long-term outcome of pediatric obsessive– compulsive disorder: a meta-analysis and qualitative review of the literature. Acta Psychiat Scand 110:4–13

4. Flament MF, Whitaker A, Rapoport JL, Davies M, Berg CZ, Kalikow K, Sceery W, Shaffer D (1988) Obsessive compulsive disorder in adolescence: an epidemiological study. J Am Acad Child Adolesc Psychiatry 27:764–771

5. Valleni-Basile LA, Garrison CZ, Jackson KL, Waller JL, McK- eown RE, Addy CL, Cuffe SP (1994) Frequency of obses- sive–compulsive disorder in a community sample of young adolescents. J Am Acad Child Adolesc Psychiatry 33:782–791 (Erratum appears in J Am Acad Child Adolesc Psychiatry. 1995 Feb;34(2):128–129; PMID: 7896644)

6. Rapoport JL, Inoff-Germain G, Weissman MM, Greenwald S, Nar- row WE, Jensen PS, Lahey BB, Canino G (2000) Childhood obses- sive–compulsive disorder in the NIMH MECA study: parent versus child identification of cases. Methods for the epidemiology of child and adolescent mental disorders. J Anxiety Disord 14:535–548

7. Skarphedinsson G, Hanssen-Bauer K, Kornor H, Heiervang ER, Landro NI, Axelsdottir B, Biedilae S, Ivarsson T (2015) Stand- ard individual cognitive behaviour therapy for paediatric obses- sive–compulsive disorder: a systematic review of effect estimates across comparisons. Nord J Psychiatry 69:81–92

8. Ivarsson T, Skarphedinsson G, Kornor H, Axelsdottir B, Biedi- lae S, Heyman I, Asbahr F, Thomsen PH, Fineberg N, March J (2015) The place of and evidence for serotonin reuptake inhibi- tors (SRIs) for obsessive compulsive disorder (OCD) in children and adolescents: views based on a systematic review and meta- analysis. Psychiatry Res 227:93–103

9. Geller D, March J (2012) Practice parameter for the assessment and treatment of children and adolescents with obsessive–com- pulsive disorder. J Am Acad Child Adolesc Psychiatry 51:98–113

10. NICE (2005) Obsessive compulsive disorder (OCD) and body dysmorphic disorder (BDD). In: Excellence NIfHaC (ed) National Institute for Health and Clinical Excellence, London

11. Scahill L, Riddle MA, McSwiggin-Hardin M, Ort SI, King RA, Goodman WK, Cicchetti D, Leckman JF (1997) Children’s Yale- Brown obsessive compulsive scale: reliability and validity. J Am Acad Child Adolesc Psychiatry 36:844–852

12. Lewin A, Piacentini J (2010) Evidence-based assessment of child obsessive compulsive disorder: recommendations for clini- cal practice and treatment research. Child Youth Care Forum 39:73–89

13. Storch EA, Murphy TK, Bagner DM, Johns NB, Baumeister AL, Goodman WK, Geffken GR (2006) Reliability and validity of the child behavior checklist obsessive–compulsive scale. J Anxi- ety Disord 20:473–485

14. Lewin AB, Piacentini J, De Nadai AS, Jones AM, Peris TS, Gef- fken GR, Geller DA, Nadeau JM, Murphy TK, Storch EA (2014) Defining clinical severity in pediatric obsessive–compulsive dis- order. Psychol Assess 26:679–684

15. Busner J, Targum SD (2007) The clinical global impressions scale: applying a research tool in clinical practice. Psychiatry 4:28–37

16. Garvey MA, Perlmutter SJ, Allen AJ, Hamburger S, Lougee L, Leonard HL, Witowski ME, Dubbert B, Swedo SE (1999) A pilot study of penicillin prophylaxis for neuropsychiatric exac- erbations triggered by streptococcal infections. Biol Psychiatry 45:1564–1571

17. Tolin DF, Abramowitz JS, Diefenbach GJ (2005) Defining response in clinical trials for obsessive–compulsive disorder: a signal detection analysis of the Yale-Brown obsessive compul- sive scale. J Clin Psychiatry 66:1549–1557

18. Frank E, Prien RF, Jarrett RB, Keller MB, Kupfer DJ, Lavori PW, Rush AJ, Weissman MM (1991) Conceptualization and rationale for consensus definitions of terms in major depressive disorder. Remission, recovery, relapse, and recurrence. Arch Gen Psychiatry 48:851–855

19. Pigott TA, Pato MT, Bernstein SE, Grover GN, Hill JL, Tolliver TJ, Murphy DL (1990) Controlled comparisons of clomipramine and fluoxetine in the treatment of obsessive–compulsive dis- order. Behavioral and biological results. Arch Gen Psychiatry 47:926–932

20. Cottraux J, Note I, Yao SN, Lafont S, Note B, Mollard E, Bou- vard M, Sauteraud A, Bourgeois M, Dartigues JF (2001) A ran- domized controlled trial of cognitive therapy versus intensive behavior therapy in obsessive compulsive disorder. Psychother Psychosom 70:288–297

21. Simpson HB, Gorfinkle KS, Liebowitz MR (1999) Cognitive- behavioral therapy as an adjunct to serotonin reuptake inhibitors in obsessive–compulsive disorder: an open trial. J Clin Psychia- try 60:584–590

22. Sookman D, Steketee G (2007) Directions in specialized cognitive behavior therapy for resistant obsessive–compulsive disorder: the- ory and practice of two approaches. Cognit Behav Pract 14:1–17

23. Wagner KD, Cook EH, Chung H, Messig M (2003) Remission status after long-term sertraline treatment of pediatric obsessive– compulsive disorder. J Child Adoles Psychopharmacol 13(Suppl 1):S53–S60

24. Piacentini J, Bergman RL, Chang S, Langley A, Peris T, Wood JJ, McCracken J (2011) Controlled comparison of family cogni- tive behavioral therapy and psychoeducation/relaxation training for child obsessive–compulsive disorder. J Am Acad Child Ado- lesc Psychiatry 50:1149–1161

25. POTS Study Team (2004) Cognitive-behavior therapy, sertraline, and their combination for children and adolescents with obses- sive–compulsive disorder: the Pediatric OCD Treatment Study (POTS) randomized controlled trial. JAMA 292:1969–1976

26. Torp NC, Dahl K, Skarphedinsson G, Thomsen P, Valderhaug R, Weidle B, Melin K, Hybel K, Nissen JB, Lenhard F, Wentzel- Larsen T, Franklin M, Ivarsson T (2015) Effectiveness of cogni- tive behavior treatment for pediatric obsessive–compulsive disor- der: acute outcomes from the Nordic long-term OCD treatment study (NordLOTS). Behav Res Ther 64:15–23

27. Turner CM, Mataix-Cols D, Lovell K, Krebs G, Lang K, Byford S, Heyman I (2014) Telephone cognitive-behavioral therapy for adolescents with obsessive–compulsive disorder: a randomized controlled non-inferiority trial. J Am Acad Child Adolesc Psy- chiatry 53:1298–1307.e1292

28. Caporino NE, Brodman DM, Kendall PC, Albano AM, Sherrill J, Piacentini J, Sakolsky D, Birmaher B, Compton SN, Ginsburg G, Rynn M, McCracken J, Gosch E, Keeton C, March J, Walkup JT (2013) Defining treatment response and remission in child anxi- ety: signal detection analysis using the pediatric anxiety rating scale. J Am Acad Child Adolesc Psychiatry 52:57–67

29. Storch EA, Lewin AB, De Nadai AS, Murphy TK (2010) Defin- ing treatment response and remission in obsessive–compulsive

55Eur Child Adolesc Psychiatry (2017) 26:47–55

1 3

disorder: a signal detection analysis of the children’s Yale-Brown obsessive compulsive scale. J Am Acad Child Adolesc Psychia- try 49:708–717

30. Lewin AB, De Nadai AS, Park J, Goodman WK, Murphy TK, Storch EA (2011) Refining clinical judgment of treatment outcome in obsessive–compulsive disorder. Psychiatry Res 185:394–401

31. Swets JA, Pickett RM (1982) Evaluation of Diagnostic Systems: Methods from signal detection theory. Academic Press, New York

32. Thomsen PH, Torp NC, Dahl K, Christensen K, Englyst I, Melin KH, Nissen JB, Hybel KA, Valderhaug R, Weidle B, Skarphed- insson G, von Bahr PL, Ivarsson T (2013) The Nordic long-term OCD treatment study (NordLOTS): rationale, design, and meth- ods. Child Adolesc Psychiatry Mental Health 7:41

33. Skarphedinsson G, Weidle B, Thomsen PH, Dahl K, Torp NC, Nissen JB, Melin KH, Hybel K, Valderhaug R, Wentzel-Larsen T, Compton SN, Ivarsson T (2015) Continued cognitive-behavior therapy versus sertraline for children and adolescents with obses- sive–compulsive disorder that were non-responders to cognitive- behavior therapy: a randomized controlled trial. Eur Child Ado- lesc Psychiatry 24:591–602

34. American Psychiatric Association (2000) Diagnostic and Statis- tical Manual of Mental Disorders DSM-IV-TR Fourth Edition (Text Revision). American Psychiatric Association, Washington, DC

35. Torp NC, Dahl K, Skarphedinsson G, Thomsen PH, Valderhaug R, Weidle B, Hybel K, Compton S, Ivarsson T (2015) Predictors associated with improved cognitive-behavioral therapy outcome in pediatric obsessive–compulsive disorder. J Am Acad Child Adolesc Psychiatry 54:200–207

36. Kaufman J, Birmaher B, Brent D, Rao U et al (1997) Schedule for affective disorders and schizophrenia for school-age children- present and lifetime version (K-SADS-PL): initial reliability and validity data. J Am Acad Child Adolesc Psychiatry 36:980–988

37. Lauth B, Arnkelsson GB, Magnusson P, Skarphedinsson G, Fer- rari P, Petursson H (2010) Validity of K-SADS-PL (schedule for affective disorders and schizophrenia for school-age children- present and lifetime version) depression diagnoses in an adoles- cent clinical population. Nord J Psychiatry 64:409–420

38. Lauth B, Magnusson P, Ferrari P, Petursson H (2008) An Ice- landic version of the kiddie-SADS-PL: translation, cross-cul- tural adaptation and inter-rater reliability. Nord J Psychiatry 62:379–385

39. Valderhaug R, Larsson B, Gotestam KG, Piacentini J (2007) An open clinical trial of cognitive-behaviour therapy in children and adolescents with obsessive–compulsive disorder administered in regular outpatient clinics. Behav Res Ther 45:577–589

40. Freeman JB, Garcia AM, Coyne L, Ale C, Przeworski A, Himle M, Compton S, Leonard HL (2008) Early childhood OCD: pre- liminary findings from a family-based cognitive-behavioral approach. J Am Acad Child Adolesc Psychiatry 47:593–602

41. Scahill L, Riddle MA, McSwiggin-Hardin M, Ort SI (1997) Children’s Yale-Brown obsessive compulsive scale: reliability and validity. J Am Acad Child Adolesc Psychiatry 36:844–852

42. Storch EA, Murphy TK, Geffken GR, Soto O, Sajid M, Allen P, Roberti JW, Killiany EM, Goodman WK (2004) Psychometric evaluation of the children’s Yale-Brown obsessive–compulsive scale. Psychiatry Res 129:91–98

43. Gallant J, Storch EA, Merlo LJ, Ricketts ED, Geffken GR, Goodman WK, Murphy TK (2008) Convergent and discrimi- nant validity of the children’s Yale-Brown obsessive compulsive scale-symptom checklist. J Anxiety Disord 22:1369–1376

44. Guy W (1976) ECDEU assessment manual for psychopharma- cology. Government Printing Office, Washington

45. Kraemer H, Periyakoil VS, Noda A (2002) Kappa coefficients in medical research. Stat Med 21:2109–2129

46. Kraemer H (1992) Evaluating medical tests: objective and quan- titative guidelines. Sage Publications, Thousand Oaks

47. Pallanti S, Hollander E, Bienstock C, Koran L, Leckman J, Marazziti D, Pato M, Stein D, Zohar J, International Treatment Refractory OCDC (2002) Treatment non-response in OCD: methodological issues and operational definitions. Int J Neu- ropsychopharmacol 5:181–191

48. Leucht S, Davis JM, Engel RR, Kane JM, Wagenpfeil S (2007) Defining ‘response’ in antipsychotic drug trials: recommenda- tions for the use of scale-derived cutoffs. Neuropsychopharma- cology 32:1903–1910

49. Mataix-Cols D, de la Cruz LF, Nordsletten AE, Lenhard F, Iso- mura K, Simpson HB (2016) Towards an international expert consensus for defining treatment response, remission, recovery and relapse in obsessive–compulsive disorder. World Psychiatry 15:80–81

50. Farris SG, McLean CP, Van Meter PE, Simpson HB, Foa EB (2013) Treatment response, symptom remission, and wellness in obsessive–compulsive disorder. J Clin Psychiatry 74:685–690

51. Frank E, Kupfer DJ, Wagner EF, McEachran AB, Cornes C (1991) Efficacy of interpersonal psychotherapy as a maintenance treatment of recurrent depression: contributing factors. Arch Gen Psychiatry 48:1053–1059 [Erratum appears in Arch Gen Psy‑ chiatry 1992 May; 49(5):401]

52. Kennard BD, Emslie GJ, Mayes TL, Nakonezny PA, Jones JM, Foxwell AA, King J (2014) Sequential treatment with fluoxetine and relapse–prevention CBT to improve outcomes in pediatric depression. Am J Psychiatry 171:1083–1090

53. Franklin ME, Sapyta J, Freeman JB, Khanna M, Compton S, Almirall D, Moore P, Choate-Summers M, Garcia A, Edson AL, Foa EB, March JS (2011) Cognitive behavior therapy augmen- tation of pharmacotherapy in pediatric obsessive–compulsive disorder: the pediatric OCD treatment study II (POTS II) rand- omized controlled trial. J Am Med Assoc. 306:1224–1232

54. Eisen JL, Goodman WK, Keller MB, Warshaw MG, DeMarco LM, Luce DD, Rasmussen SA (1999) Patterns of remission and relapse in obsessive–compulsive disorder: a 2-year prospective study. J Clin Psychiatry 60:346–351 (quiz 352)

55. Leckman JF, Sholomskas D, Thompson D, Belanger A, Weiss- man MM (1982) Best estimate of lifetime psychiatric diagnosis: a methodological study. Arch Gen Psychiatry 39:879–883

56. Højgaard DRMA, Mortensen E, Ivarsson T, Valderhaug R, Hybel K, Skarphedinsson G, Dahl K, Weidle B, Torp N, Grados M (2015) Child and adolescent OCD symptom patterns: a factor analytic study. In: 16th International ESCAP congress, Madrid, Spain, pp S187–S187

European Child & Adolescent Psychiatry is a copyright of Springer, 2017. All Rights Reserved.