Social Anxiety
Reeb79
Arandomizedcontrolledtrialexaminingtheefficacyofaninternetbasedcognitivebehavioraltherapyprogranforadolescentswithanxietydisorders.pdf
RESEARCH ARTICLE
A randomized controlled trial examining the
efficacy of an internet-based cognitive
behavioral therapy program for adolescents
with anxiety disorders
Silke StjerneklarID 1*, Esben Hougaard1, Lauren F. McLellan2, Mikael Thastum1
1 Department of Psychology and Behavioral Sciences, School of Business and Social Sciences, Aarhus
University, Aarhus, Denmark, 2 Centre for Emotional Health, Department of Psychology, Macquarie
University, New South Wales, North Ryde, Australia
Abstract
Background
Anxiety disorders are highly prevalent in adolescence, but access to health care services is
limited and only few receive professional help. Internet-based cognitive behavioral therapy
(ICBT) has been proposed to increase accessibility and reduce costs of treatment.
Objective
The study evaluated the efficacy of a Danish version of the guided ICBT program ChilledOut
Online, developed at the Centre for Emotional Health, Macquarie University, Australia.
Method
At the Centre for Psychological Treatment of Children and Adolescents, Aarhus University,
Denmark, a randomized controlled trial was conducted with 70 adolescents (13–17 years)
with anxiety disorders according to DSM-IV. Participants were randomly assigned to a 14-
weeks therapist-guided ICBT or to a waitlist condition. Outcomes were evaluated post-treat-
ment and at 3- and 12-month follow-up.
Results
At post-treatment, the ICBT group significantly outperformed the waitlist condition with mod-
erate to large between-group effect sizes on diagnostic severity and anxiety symptoms
rated by clinicians, and by adolescents and their parents. Forty percent of adolescents in
ICBT were free of their primary diagnosis compared to 16% in the waitlist condition. Treat-
ment gains were maintained at 3- and 12-month follow-up.
Conclusion
Results of the study provide support for the efficacy of guided ICBT for adolescents with
anxiety disorders.
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 1 / 26
a1111111111
a1111111111
a1111111111
a1111111111
a1111111111
OPEN ACCESS
Citation: Stjerneklar S, Hougaard E, McLellan LF,
Thastum M (2019) A randomized controlled trial
examining the efficacy of an internet-based
cognitive behavioral therapy program for
adolescents with anxiety disorders. PLoS ONE 14
(9): e0222485. https://doi.org/10.1371/journal.
pone.0222485
Editor: Andrea Martinuzzi, IRCCS E. Medea, ITALY
Received: August 24, 2018
Accepted: August 30, 2019
Published: September 18, 2019
Copyright: © 2019 Stjerneklar et al. This is an open access article distributed under the terms of the
Creative Commons Attribution License, which
permits unrestricted use, distribution, and
reproduction in any medium, provided the original
author and source are credited.
Data Availability Statement: The data that form
the basis of the manuscript submitted to you
comprises information of both child and caregiver,
and all participants were guaranteed full anonymity.
As such and in line with Danish Data Protection
Agency regulations, the data used in this study are
not publicly available because they contain
sensitive and potentially identifiable information.
Unidentifiable data excerpts can be made available
upon request to the corresponding author
([email protected]). Further data access
Trial registration
ClinicalTrials.gov: NCT02535403.
Introduction
Anxiety disorders are common in adolescence with 5–12% of youths from western countries
suffering from the disorder [1–3]. When left untreated, anxiety disorders in adolescence tend
to have a chronic developmental course, often predicting anxiety and depression into adult-
hood [4–6]. Cognitive behavioral therapy (CBT) has proven effective in treating adolescent
anxiety disorders with studies showing large treatment effects in individual and group treat-
ment formats [7]. However, access to health care services is limited and it is estimated that
only around 25% of clinically anxious youth receive professional help [8–10]. Several reasons
have been suggested for this, including insufficient knowledge about and limited accessibility
of available mental health services, lack of trained therapists, and high therapy costs [11, 12].
Adolescents may be especially reluctant to seek help for emotional problems, even if treatment
facilities are available [13–15]. Among the most frequently mentioned help-seeking barriers
for adolescents are perceived social stigma [12, 16], concerns about privacy and confidentiality
[12], preference for self-reliance [17, 18], and worries concerning treatment costs, transporta-
tion or waiting times [19]. Clearly, there is a need for alternative treatment delivery methods
for anxiety disordered youths that strive to overcome such barriers.
Internet-based CBT (ICBT) has been proposed as a means to increase access to, and reduce
costs of, psychological interventions. ICBT typically provides therapeutic content comparable
to that of regular face-to-face CBT and is typically presented in modules on a weekly basis [20,
21]. ICBT interventions can be either self-directed or therapist-guided. Guidance may refer to
any sort of support from a coach or a therapist, for example automated reminders, asynchro-
nous email correspondence, brief scheduled phone calls or real-time chat [22]. Generally,
research suggests that regular contact with a therapist or a coach, providing some sort of sup-
port to the user, substantially increases program usage, minimizes drop-out and improves out-
come [23–25].
ICBT for adults has been widely researched with substantial evidence of effectiveness for a
range of anxiety disorders [22, 26–30]. ICBT may be particularly appropriate for adolescents
as it has the potential to overcome many of the barriers to seeking help for this age group, par-
ticularly less social stigma, greater anonymity, a higher degree of self-determination, reduced
expenses, and eliminated travel time [27, 31–34]. Moreover, as ‘digital natives’ adolescents are
highly skilled in the use of computer technologies, engaging in daily activities on the internet
and commonly seeking health information online [35–38].
An emerging body of research shows promising results for the ICBT treatment of anxiety
disorders in children and adolescents. Four recent meta-analyses of children, adolescents and
young adults (age range 5–25) with anxiety and/or depressive symptoms [39–42] indicate that
ICBT can be an effective intervention. In a systematic meta-review focusing more broadly on
digital health interventions (including wearable technologies, smartphone apps, computer-
assisted and internet-based therapies among others) for children, adolescents and young
adults, through meta-analyses comparing the experimental condition to no treatment condi-
tions (e.g. waitlist, placebo) Hollis et al. [43] demonstrated moderate-to-large effect sizes (ESs)
on anxiety symptoms (Hedges’ g = 0.53–1.41). None of the above mentioned meta-analyses examined the effect of ICBT for adolescents specifically, but those that investigated age as a
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 2 / 26
requests may be sent to The Technology Transfer
Office at Aarhus University ([email protected]).
Funding: The study was funded in part by
Trygfonden (grant id: 100886) and by Edith and
Godtfred Kirk Christiansens Fund (grant id: 21-
5675). The funders had no role in the study design,
data collection and analysis, decision to publish, or
preparation of the manuscript.
Competing interests: SS and LM took part in the
development of the ChilledOut Online program;
however, they do not receive any financial benefits
from its use. This does not alter our adherence to
PLOS ONE policies on sharing data and materials.
The remaining authors declare no conflicting
interests.
moderator of outcome [39–41] found better results for older youths compared to younger,
although age ranges varied across studies.
When looking exclusively at ICBT with anxious adolescents, studies are sparse [44–48]. In
an RCT including 19 speech-anxious high school students (aged 15–21), Tillfors et al. [44]
showed that nine weeks of guided ICBT adapted from a self-help manual [49] reduced social
anxiety at post-treatment with a large ES (Cohen’s d = 1.28) compared to a waitlist control (WL) group. Spence et al. [46] conducted an RCT with 115 clinically anxious adolescents
(aged 12–18) comparing the 12-week generic ICBT program BRAVE for teenagers-ONLINE
with clinic-based face-to-face CBT. At post-treatment both groups showed significant reduc-
tions in anxiety without significant differences between them. Thirty four percent were free of
their primary anxiety diagnosis in the ICBT condition (intent to treat) compared with 30% in
the face-to-face condition. Also, both the online group and the face-to-face group showed
comparable reductions in self-rated anxiety symptoms on The Spence Children’s Anxiety
Scale [SCAS; [50]] Child version post-treatment [46]. Lenhard et al. [48] conducted an RCT
with 67 adolescents (aged 12–17) diagnosed with obsessive-compulsive disorder (OCD).
Results showed that the 12-week clinician and parent supported ICBT-intervention was supe-
rior to WL on the clinician-administered interview Children’s Yale-Brown Obsessive-Com-
pulsive Scale [51] with an ES of d = 0.69 (p < 0.001). The present study investigates a Danish version of the ChilledOut Online program [52]
developed at the Centre for Emotional Health, Macquarie University, Australia. The program
builds on the structure and content of the manualized group-CBT program Chilled [53] for adolescents with anxiety disorders. The program has not yet been evaluated, although a prior
CD-ROM version, the Cool Teens program [54, 55], has been demonstrated efficacious in an RCT with 43 adolescents [56]. An initial feasibility study indicated that the Danish version of
ChilledOut Online could be a feasible psychological intervention for adolescents with anxiety
disorders with promising results [57].
Aim and hypotheses
The present study examined the efficacy of ChilledOut Online using a randomized waitlist-
controlled design with follow-up assessments at 3- and 12-month post-intervention. We
hypothesized that the ICBT intervention would demonstrate superiority to the WL condition
primarily by reducing diagnostic severity and anxiety symptoms, but also by reducing symp-
toms of depression and improving self-efficacy and quality of life. Additionally, we expected
these outcomes to be maintained at follow-ups.
Methods
Participants and recruitment
The study took place at the Centre for Psychological Treatment of Children and Adolescents
(CEBU), a research and teaching facility at the Department of Psychology and Behavioral
Sciences, Aarhus University, Denmark. The study was approved on May 7, 2015 by the local
Ethics Committee of Central Denmark Region (1-10-72-98-15) and by the Danish Data Pro-
tection Agency. Furthermore, the trial was registered within ClinicalTrials.gov, registration
number: NCT02535403.
Inclusion criteria were: (a) 13 to 17 years of age; (b) a primary anxiety disorder diagnosis
according to the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM–IV; [58]); (c) direct access to a home computer with internet; and (d) the ability to read and write
in Danish. Exclusion criteria were: (a) severe comorbid depression (Clinical Severity
Rating > 5 as measured with the Anxiety Disorders Interview Schedule for DSM-IV, Child
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 3 / 26
and Parent Versions (ADIS-IV C/P; [59]); (b) substance abuse; (c) current severe self-harm or
suicidal ideation; (d) pervasive developmental disorder; (e) learning disorder or intellectual
disability; and (f) psychotic symptoms.
Participants were recruited from May 15 to December 18, 2015 and the last follow-up
assessment was obtained by March 1, 2017. Although the recruitment process was initiated
prior to trial registration (August 28, 2015), baseline assessments and enrolment was not com-
menced until registration had been confirmed. Families had referred themselves to the CEBU
secretary in response to postings on the website or recommendations from local community
health services. Interested families were invited to send in a brief description of the adoles-
cent’s major problems. If deemed eligible for the present study, families were contacted for a
semi-structured diagnostic telephone interview and sent a package of online questionnaires
(see full description under Measures). Those who met inclusion criteria were given verbal and
written information about the study procedures. After adolescents and their parents provided
written consent, they were included in the study and randomized to condition. The randomi-
zation sequence was created with an online computer random number generator using per-
muted block design with a fixed block size of 10 at a 1:1 allocation ratio to the ICBT or the WL
condition. The sequence list was kept concealed from researchers and therapists, stored by an
external secretary at the University who administered group assignment to included partici-
pants according to the randomization sequence. Participants in both conditions were encour-
aged not to engage in other forms of treatment nor make changes to their use of psychiatric
medication during the acute treatment and waitlist period.
Procedures
Adolescents randomized to the WL condition were instructed to wait 14 weeks for treatment.
During this period, participants did not have any form of planned contact with the project
team. After 14 weeks, adolescents and parents recompleted the assessment measures, took part
in a second telephone interview, and were offered the ICBT treatment. Adolescents random-
ized to the ICBT condition were informed of their allocation over the phone by their appointed
therapist and provided introductory information about the program. During this 20-minute
introductory phone call, the course of the intervention was summarized, adolescents were
given a brief introduction to the program website and functions, and therapists and adoles-
cents agreed upon the time and dates of their weekly supportive phone calls. The adolescent
then received a letter outlining the treatment start date and program information, i.e. the pro-
gram URL, a personal username, a temporary password, the therapist e-mail address, a photo
of the therapist, and a calendar plan of the future therapist phone calls. All therapist calls were
recorded using Crystal Gears1 Ver. 2.00 RTM for supervision purposes and to ensure integ-
rity of the intervention.
Adolescents’ diagnostic status was assessed at baseline (pre), after the intervention (post),
and at three-month follow-up with the ADIS-IV C/P [59] administered via telephone. The
interviews were conducted by trained graduate psychology students working at the Center,
closely supervised by clinical psychologists, experienced in using the ADIS-IV. Assessors were
blind to group allocation at pre-assessment and of participants’ prior diagnoses at post and fol-
low-up. Assessors were also blind to group allocation at post assessment, although most fami-
lies did reveal their allocation status during the post interview. All interviews were recorded
using Crystal Gears1 Ver. 2.00 RTM. One of two trained assessors re-assessed 14 (20%) of the
audio-recorded baseline interviews to allow for subsequent evaluation of interrater reliability.
Adolescents and both parents separately completed online self-report questionnaires at pre,
post, three- and twelve-month follow-up. Assessment at 12-month follow-up only included
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 4 / 26
SCAS, CALIS and WHO-5 (see reference to the scales below). The questionnaires were trans-
lated into Danish using the forward-backward translation process and administered through
an electronic data collection platform, SurveyXact. Families failing to complete the question-
naires at any assessment point received three weekly e-mail reminders followed by a phone
call after four weeks from their appointed therapist.
Measures
Primary outcome measures.
Clinician-rated diagnostic severity. The ADIS-IV C/P [59] is a semi-structured diagnostic
interview conducted with the adolescent and one parent separately to assess the diagnostic cri-
teria of anxiety disorders in accordance with DSM-IV [58]. In addition to anxiety disorders,
the ADIS-IV permits assessment of other disorders often associated with anxiety, including
dysthymia, depression, ADHD, oppositional defiant disorder, and conduct disorder. The
severity of each diagnosis is evaluated by a clinician on a nine-point Likert scale (0 = not at all
disturbing; 8 = severely disturbing), the Clinical Severity Rating (CSR), using information
from both adolescent and parent. Disorders were considered clinically impairing if clinicians
provided a CSR of 4 or greater. When more than one disorder was present, clinicians identi-
fied the primary diagnosis (defined as the most impairing disorder). The ADIS-IV has shown
good to excellent test-retest reliability for the presence of specific diagnoses and CSR [60].
Concurrent validity of the anxiety disorders section has also been established [61]. Administer-
ing the ADIS-IV over the telephone has also yielded high inter-rater reliability and validity,
comparable to face-to-face administration [62]. The inter-rater reliability (Cohen’s Kappa) for
the primary anxiety diagnoses in this study was excellent, κ = 0.80 (95% CI: 0.556–1.038; p < 0.001). The intraclass correlation coefficient (ICC; two-way random effects model, consis- tency, individual raters), was fair for the CSR of the primary anxiety diagnosis (CSRprim),
ICC = 0.419 (95% CI: -0.121–0.768; p = 0.060) and good for the CSR of all anxiety diagnoses (CSRall), ICC = 0.731 (95% CI: 0.348–0.905; p = 0.001).
Adolescent- and parent-rated anxiety symptoms. The Spence Children’s Anxiety Scale
(SCAS; [50]) measures self-rated anxiety symptoms in six domains: social phobia (SoP), gener-
alized anxiety disorder (GAD), specific phobia (SP; ‘fear of physical injury’), separation anxiety
disorder (SAD), OCD, and panic disorder (PD) with agoraphobia (AP). However, in the pres-
ent study only the total score measure of overall symptom severity was used. The questionnaire
contains 38 items, each rated on a four-point Likert scale from zero to three. Higher scores
indicate higher anxiety symptom levels. The scale is filled out by the adolescent (SCAS-C) and
by the each of the parents (SCAS-P). The SCAS-C includes six additional positive filler items.
The Danish version of the SCAS has demonstrated good to excellent internal consistency for
the total score in a clinical, as well as a community sample, and good test-retest reliability in a
community sample [63]. In the present study, internal consistency (Cronbach’s α) for the total score was good to excellent for adolescent (α = 0.88), mother (α = 0.89), and father (α = 0.90) report versions of the scale.
Secondary measures.
Anxiety life interference. The Child Anxiety Life Inference Scale (CALIS; [64]) measures the
interference of youth anxiety on various areas of life functioning including family, friends,
school, and extracurricular activities. Items are evaluated separately by the adolescent (nine
items) and their parents (nine items) on five-point Likert scales (0 = not at all; 4 = a great
deal). Parents also rate the interference of the adolescents’ anxiety on their own life including
their relationship with spouse, friends, family, career choices and general level of stress (seven
items). For the present study, only the total scores on parent reports, i.e. the sum of the rated
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 5 / 26
interference on child and parent life (16 items), was used. The CALIS has demonstrated satis-
factory internal consistency and moderate retest reliability in its original English language ver-
sion [64]. Internal consistency in the current sample (Cronbach’s α) was acceptable for adolescent reports (α = 0.77), and good for mother and father reports (α = 0.87 and α = 0.89, respectively).
Depressive symptoms. The short version of the Moods and Feelings Questionnaire
(S-MFQ; [65]) measures youth depressive symptoms within the last two weeks as evaluated
separately by adolescents and parents. The questionnaire consists of 13 items rated on a three-
point Likert scale (0 = not true; 2 = true). The English version of the S-MFQ has shown good
internal consistency on both youth self-report and parent-report in [65]. In the present study,
internal consistency (Cronbach’s α) was excellent for adolescent report (α = 0.90), and good for mother and father reports (α = 0.88 and α = 0.87, respectively).
Self-efficacy. The Self-Efficacy Questionnaire for Children (SEQ-C; [66]) is a 24-item mea-
sure of self-efficacy assessing youth perceptions of personal strengths and competencies in
three domains: academic (ability to succeed in school and display appropriate learning behav-
iors), social (ability to get along with and relate to peers), and emotional (ability to regulate
negative emotions). Items are rated on a five-point Likert scale (1 = not at all; 5 = very well). In
its original English version, the SEQ-C has shown high internal reliability [66], good internal
consistency [67], and criterion validity has been established [68]. In the current study, internal
consistency (Cronbach’s α) for the total scale was good, α = 0.89. Mental well-being. The WHO-5 Well-being Index Questionnaire (WHO-5; [69]) is a five-
item self-report measure of respondents subjective mental well-being during the previous two
weeks covering positive mood, vitality and general interest. Each item is rated on a six-point
Likert scale (0 = not present; 5 = constantly present). Raw scores range from zero to 25. When
interpreting changes in WHO-5 scores, the scale is translated into 0–100%, and the percentage
value is used, calculated by multiplying raw scores by four. By convention, a score between 100
and 50 indicate an acceptable life-quality, a score between 49 and 30 indicate a moderately
reduced life-quality, while a score � 30 indicate a pathologically reduced well-being, i.e. mod-
erate depression. The English version of the scale has demonstrated excellent internal consis-
tency and good criterion validity [70, 71]. Internal consistency (Cronbach’s α) in the present sample was good, α = 0.86.
Computer experience. At pre-treatment, adolescents were asked to rate their computer
experience by answering the question: ‘How comfortable do you feel using the computer and
the internet?’ rated on a four-point Likert scale (1 = not comfortable at all; 4 = very
comfortable).
Treatment satisfaction. Satisfaction with the ICBT intervention was measured in adoles-
cents and parents post treatment using a short questionnaire adapted from The Experience of
Service Questionnaire [72, 73]. Separate versions were used for adolescents (seven items) and
parents (nine items), rating affirmative statements on a three-point Likert scale (1 = not true;
2 = partly true; 3 = true), e.g. ‘The treatment helped me/my child’, ‘We feel better in the family
now compared to before the treatment’, and ‘If a friend needed this type of help, I would rec-
ommend him/her to contact the Centre’. The questionnaire ended with an open section invit-
ing respondents to provide any feedback in relation to the intervention.
Program activity and support. Adolescents’ program activity was automatically registered
at the website server as number of logins, number of module visits and number of module
components activated. Therapists kept a record of all contact with adolescents and parents,
noting the duration of each phone call and all e-mail correspondence throughout the interven-
tion. Moreover, adolescents were asked to rate their average time spent per week on program
relevant activities (on- and offline separately) at post treatment, and parents were asked to
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 6 / 26
report their average time spent per week helping their child with program relevant activities
(on- and off-line combined).
The following measures were administered but not reported here: The Strengths and Diffi- culties Questionnaire [74, 75]; The Working Alliance Inventory-Short Form [76].
Treatment. ChilledOut Online is a treatment program based on the Cool Kids and
Chilled anxiety management program developed at Macquarie University, Sydney, Australia
[52]. The program teaches CBT strategies for adolescents through eight online modules of
approximately 30 minutes, with a focus on psychoeducation, cognitive restructuring and
graded exposure. To allow for flexibility and personal learning preferences, adolescents were
able to access all modules at treatment start. To guide progress through the program, adoles-
cents were however encouraged to complete all eight modules (and module content) in the
order they appeared within the 14-week intervention period, after which they would have
access to the web site for another three months. Program content such as goal setting, realistic
thinking, problem solving, and assertiveness, is presented through a combination of multime-
dia formats, i.e. text, audio, illustrations, cartoons, worksheets, and video vignettes. An over-
view of module content is presented in Table 1. The program includes six video cases of
adolescents with different anxiety issues, who reappear throughout the program as they dem-
onstrate various program skills to manage their problems. Furthermore, each module contains
homework practice tasks that adolescents are encouraged to complete. Once a week, partici-
pants are asked to rate nine statements concerning the interference of anxiety in the following
areas of their lives: (1) getting along with parents, (2) getting along with siblings, (3) spending
time with friends outside school, (4) finishing homework assignments, (5) spending time with
Table 1. Module content and homework practice tasks.
Module title Module content Homework practice tasks
1. Understanding
anxiety
• How to use the program
• Psychoeducation about anxiety
• Complete an anxiety self-assessment questionnaire to get overview of
present anxiety issues
2. Setting goals • Learning to set goals
• Rewards
• Measuring feelings on a worry scale
• Identify goals
• Make a list of possible rewards
3. Realistic thinking I • Linking thoughts and feelings
• Negative thinking
• Identifying and challenging unrealistic thoughts (cognitive
restructuring)
• Practice realistic thinking
• Rewards
4. Stepladders I • Identifying and categorizing fears and worries
• Graded exposure using stepladders
• Create the first stepladder
• Plan the execution of the stepladder
• Practice steps until goal is reached
5. Stepladders II • Revising old and creating new stepladders
• Behavioral experiments
• Help solving stepladder barriers and difficulties
• Create more stepladders
• Practice behavioral experiments
6. Realistic thinking II • Simplifying realistic thinking (in my mind)
• Acting as if
• Surfing emotions
• Make list of useful questions and thoughts for realistic thinking
• Continue working on stepladders
7. Other coping skills • Problem solving
• Constructive feedback
• Assertive communication
• Calming activities
• Practice problem solving and assertiveness
8. Staying chilled • Skills overview and maintenance
• Relapse prevention
• Continue to practice skills
Note. Table originally published in Stjerneklar et al. [57]
https://doi.org/10.1371/journal.pone.0222485.t001
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 7 / 26
classmates in recess, (6) play sports, (7) attending leisure activities, (8) completing daily rou-
tines, and (9) general mood. The statements enable participants to view a progress chart of
their weekly total scores and thus track changes during the intervention. For the present study,
the original Australian version of ChilledOut Online [52] was translated and revised according
to Danish language. Revisions primarily involved recording of new video material with Danish
characters; all other translations were as close to the original program as possible.
Adolescents received a weekly phone call from their therapist set to a duration of approxi-
mately 20 minutes, with a focus on assisting learning of program skills and strategies, offering
technical assistance, giving feedback about homework tasks, and encouraging program use
during times of low motivation. The therapists were graduate psychology students in a clinical
training course supervised by psychologists working at CEBU. All therapists received training
in cognitive behavioral therapy and a thorough introduction to the program. Moreover, all
therapists were equipped with a semi-structured “manual” with topics to be dealt with in the
calls. The therapists had access to participants’ module activities and worksheets throughout
the program. At three-month follow-up, adolescents received a booster phone call from their
therapist with the aim of consolidating previously learned skills.
Shortly before treatment start, parents received the ChilledOut Parent Companion describ- ing their role in treatment and providing them with an explanation of the program’s core treat-
ment strategies (psychoeducation, goal setting, cognitive restructuring, and graded exposure).
Furthermore, the handout included advice on how to support their teenager in his/her efforts
to overcome anxiety. Parents are encouraged to lend support and motivational encouragement
to their child during treatment. However, they are also advised to be sensitive to their child’s
need for autonomy. Within the first two weeks of the intervention, parents received a brief
phone call in which the therapist introduced him- or her-self and answered any questions
about the program. During the remaining intervention period, parents were invited to contact
the therapist through e-mail and, if needed, they were offered a short phone call from the ther-
apist. Based on parent feedback from a previous feasibility trial of the ChilledOut Online pro-
gram [57], parents were invited to sign up for a closed online network to facilitate group
exchanges and support.
Statistical analyses
Allowing for approximately 20% dropout (cf. [77]), the study was acceptably powered to detect
a large ES (Cohen’s d = 0.80) with a power of .80 and two-tailed α = .05 [78]. Baseline group differences were explored using independent samples t-tests, χ2-tests or
Fisher’s exact test (two-sided) as appropriate. Pre–post comparisons between the two condi-
tions were analyzed as (1) number of participants free of primary or all anxiety diagnoses at
post-treatment; (2) degree of change on continuous outcome measures from pre to post; and
(3) number of participants with clinically significant change on the SCAS.
Mixed linear models (MLMs) were used to compare the two conditions from pre to post on
continuous outcome variables. MLMs tolerate missing values and thus do not unnecessarily
compromise statistical power. All MLMs were based on the intention-to-treat sample (N = 70)
without any ad hoc imputations of missing values, which is recommended over other proce-
dures of handling missing data in longitudinal clinical trials [79]. All MLMs were estimated
with the full maximum likelihood method. Data were hierarchically arranged in two levels,
with time at level 1 nested within individuals at level 2. For the MLMs of the pre-post compari-
sons, fixed effects were specified for time (pre and post), group (ICBT and WL), and the
time × group interaction. Models included a random intercept. With three dependent vari- ables on SCAS, CALIS and S-MFQ (adolescents, mothers and fathers respectively), for these
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 8 / 26
measures we employed a family-wise Bonferroni-corrected significance level of p = 0.017 (0.05/3) for each set of analyses.
Using Jacobson and Truax’s [80] method, the reliable change index (RCI) and clinical cut-
off score were determined on SCAS. The formula used was: RCI ¼ 1:96 �ð ffiffiffi 2 p �ðSD�
ffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 � rxx
p Þ
2 Þ,
where SD is the standard deviation of the standardized sample and rxx is the reliability of the measure (internal consistency, Cronbach’s α). The RCI was calculated using standard devia- tions and internal consistency ratings from Danish norm data [63]. The clinical cut-off score
(CScut-off) defined as the mid-point between the clinical and non-clinical population, was cal-
culated with the formula: CScut−off = ((Mclin×SDnorm)+(Mnorm×SDclin))�(SDnorm+SDclin). The CScut-off was calculated from Danish community and clinical norms split into gender and age
groups reported in Arendt et al. [63]. As suggested by Jacobson and Truax [80], participants
achieving statistically reliable change according to the RCI were considered improved, whereas
those achieving both statistically reliable and clinical change were considered recovered. Cal-
culations of improvement and recovery were based on participants who completed
questionnaires.
Only participants randomized to the ICBT condition were included in the follow-up analy-
ses. Analyses of treatment outcome maintenance were conducted on SCAS, CALIS and
WHO-5 from pre to 3- and 12-month follow-up and for ADIS CSR from pre to 3-month fol-
low-up using MLMs. For models including more than two measuring points (e.g., analyses of
treatment maintenance from pre to 3- or 12-month follow-up), the slope was specified as ran-
dom if it significantly improved model fit as evaluated by a change in the –2LL fit statistics
[81].
For all analyses ES calculations were based on observed values and expressed as Cohen’s d, with 0.2, 0.5, and 0.8 considered a small, medium, and large ES, respectively [78]. For the
between group analyses, Cohen’s d was derived by the following formula: d = (MdiffICBT�S- DICBTpooled)−(MdiffWL�SDWLpooled), were Mdiff ICBT and Mdiff WL are the means of the pre-post difference scores for the ICBT and WL group respectively, and SDICBT pooled and SDWL pooled
are the pooled standard deviations of the two groups, calculated by the formula: SDpooled ¼ p ððSD2pre þ SD
2
postÞ� 2Þ. Within-group ES’s were calculated by dividing the difference
between the pre- and post-assessment means with the pooled standard deviation of those two
means: d = (Mpre−Mpost)�SDpooled. Study dropouts were defined as participants who had not remained in contact with the
therapist for the entire 14 weeks of intervention. Otherwise participants were considered com-
pleters, regardless of their number of completed modules or amount of completed therapist
calls.
All statistical analyses were carried out using IBM1 SPSS1 statistics, v.24.0 (Armonk, NY:
IBM Corp.).
Results
Study flow and baseline participant characteristics
A total of 120 adolescents were recruited. Eighty-four adolescents presented themselves with
eligible problems and thus received questionnaires and phone interviews (see study flow-
chart, Fig 1). Fourteen adolescents were excluded, 11 of which did not meet the inclusion crite-
ria and three declined to participate despite their initial interest. Seventy adolescents aged 13–
17 (M = 15.0, SD = 1.30) met inclusion criteria and were randomly assigned to ICBT (n = 35,
83% females) or WL (n = 35, 73% females).
Five participants (7%) dropped out prior to post-assessments. Two participants (ICBT)
dropped out due to loss of motivation, one (WL) experienced a worsening in symptoms during
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 9 / 26
Fig 1. Study flow-chart.
https://doi.org/10.1371/journal.pone.0222485.g001
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 10 / 26
the waitlist period (he was offered treatment through the municipality), and two (WL) gave no
explanation. No significant differences were found on primary outcome pre-treatment scores
(CSR and SCAS-C/P), gender, age, number of anxiety diagnoses, age at anxiety onset, previous
therapeutic treatment, primary diagnosis or computer experience between dropouts and com-
pleters. Ninety-six percent (63/65) of the adolescent study completers returned the self-report
questionnaire at post, while 100% (65/65) took part in the diagnostic post-interview. At
3-month follow-up, 82% (27/33) of the adolescent study completers (ICBT condition only)
returned the self-report questionnaire, while 85% (28/33) completed the diagnostic interview.
At 12-month follow-up, 76% (25/33) of the adolescent study completers returned the self-
report questionnaire. Reasons for non-completion of questionnaires and interviews are largely
unknown, as most of these participants could not be reached.
Baseline characteristics are presented in Table 2. Fifty (71%) adolescents lived with both of
their parents. The sample was relatively high in socioeconomic status, with 46% households
having an annual income � 100.000 EUR (the average personal income in Denmark was
41.363 EUR in 2015; [82]). In terms of primary diagnosis, 28 (40%) had SoP, 11 (16%) had
GAD, six (9%) had SP, eight (11%) had OCD, three (4%) had PD, nine (13%) had SAD, two
(3%) had PD with AP, and three (4%) had AP without PD. Fifty-three (76%) met the diagnos-
tic criteria for a comorbid diagnosis, 51 (73%) were diagnosed with a comorbid anxiety disor-
der, and six (9%) with a mood disorder. The mean number of anxiety diagnoses per adolescent
at pre-treatment was 2.11 (SD = 0.93). No significant between-group differences were found
for any of the primary outcome measures at pre-treatment. Likewise, no significant differences
were found for any of the demographic variables except annual household income, t (68) = 3.345, p = 0.001., where families in the WL condition had higher income than families in the ICBT condition.
Regarding computer experience, thirty-three (94%) of the adolescents randomized to the
ICBT condition reported feeling either ‘fairly confident’ or ‘very confident’ using computer
and internet. Two (6%) reported feeling only ‘a little confident’ and none reported ‘not at all
confident’.
Program activity and support
Participants (ICBT group minus drop-outs, n = 33) completed a mean of 5.4 (SD = 2.37, range
0–8) of the eight modules available. Modules were registered as completed if participants had
activated � 80% of core module components (i.e., video instructions, examples and practice
tasks excluding downloads and worksheets). According to website server logs, participants
activated a mean of 74% (SD = 23.17, range 32–100%) of all module components. Total num-
ber of logins to the program web site ranged from seven to 51 (M = 24.4, SD = 10.57). Table 3
presents an overview of participants’ program activity and amount of therapeutic support. Ten
participants (30%) completed all eight modules, three participants (9%) completed seven mod-
ules, five (15%) completed six modules, three (9%) completed five modules, two (6%) com-
pleted four modules, six (18%) completed three modules, three (9%) completed two modules,
and no participants completed only one module. One participant (3%) completed zero mod-
ules according to the definition of � 80% module component activation. As participants were
only advised to complete modules in the numerical order they appeared, but allowed to access
them in the order they preferred, approximately one third (27%) completed modules in a ran-
dom order (e.g., module 1, 6 and 7), while the remaining completed them as intended. Partici-
pants received a mean of 11.0 phone calls from the therapist (SD = 1.93, range 5–14) during
the intervention period, with an average call duration of 19.5 minutes (SD = 8.80, range 2–56).
Parents received a mean of 1.3 therapist phone calls (SD = 0.64, range 0–3) with an average call
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 11 / 26
Table 2. Sample characteristics.
ICBT (n = 35) Waitlist (n = 35) Total (N = 70)
Females (%) 29 (83) 26 (74) 55 (79)
Adolescent age, M (SD) 15.22 (1.32) 14.83 (1.28) 15.03 (1.30)
Living with both parents (%) 23 (66) 27 (77) 50 (71)
Parent age, mothers/fathers, M (SD) 45.86/48.84 (4.47/5.92) 46.31/47.09 (4.50/4.64) 46.08/47.95 (4.46/5.34)
Level of education of mothers/fathers (%)
University / Doctoral degree 7/8 (20/23) 4/9 (11/26) 11/17 (16/24)
Vocational / College 26/23 (74/66) 31/19 (89/54) 57/42 (81/60)
Primary / High school 2/0 (6/0) 0/3 (0/9) 2/3 (3/4)
NA 0/4 (0/11) 0/4 (0/11) 0/8 (0/11)
Annual household income in Euro, M (SD) 79566.99 (33885.81) 108829.05 (39124.18) 94198.02 (39207.54)
>150.000 (%) 0 (0) 6 (17) 6 (9)
100.000–149.999 (%) 12 (34) 14 (40) 26 (37)
50.000–99.999 (%) 15 (43) 14 (40) 29 (41)
<50.000 (%) 8 (23) 1 (3) 9 (13)
Age of anxiety onset a , M (SD) 8.6 (4.16) 9.2 (3.47) 8.9 (3.81)
Primary diagnoses (%)
Social phobia 14 (40) 14 (40) 28 (40)
Generalized anxiety disorder 6 (17) 5 (14) 11 (16)
Specific phobia 4 (11) 2 (6) 6 (9)
Obsessive-compulsive disorder 3 (9) 5 (14) 8 (11)
Separation anxiety disorder 2 (6) 7 (20) 9 (13)
Panic disorder 3 (9) 0 (0) 3 (4)
Panic disorder with AP 1 (3) 1 (3) 2 (3)
AP without panic disorder 2 (6) 1 (3) 3 (4)
Comorbid diagnoses (%)
Anxiety disorders 29 (83) 22 (63) 51 (73)
Mood disorders 3 (9) 3 (9) 6 (9)
Externalizing disorders 0 (0) 0 (0) 0 (0)
No comorbidity 5 (14) 12 (34) 17 (24)
Number of anxiety diagnoses per adolescent, M (SD) 2.29 (0.93) 1.94 (0.91) 2.11 (0.93)
On psychopharmacological medication (%) 5 (14) 2 (6) 7 (10)
Previous therapeutic treatment (%) 21 (60) 22 (63) 43 (61)
Note. AP: Agoraphobia a
Age of onset is based on the mean age from mother and father reports.
https://doi.org/10.1371/journal.pone.0222485.t002
Table 3. Program activity and support for the ICBT condition, completer sample (n = 33); mean (SD) {range}.
#
modules
completed
#
components
activated a
#
log ins
#
calls T/A
Mean duration a
of calls T/
A
#
emails T/A
#
calls T/P
Mean duration b
of calls T/
P
#
emails T/P
5.4 (2.37)
{0–8}
74.4 (23.17)
{32.3–100}
24.4
(10.57)
{7–51}
11.0
(1.93)
{5–14}
19.5 (8.80)
{2–56}
0.2 (0.65)
{0–3}
1.3
(0.64)
{0–3}
24.0 (10.62)
{3–45}
0.6 (1.23)
{0–6}
Note. #: number; T/A: between therapist and adolescent; T/P: between therapist and parent. a
Components activated is reported as percentages per module b
Duration is reported in minutes
https://doi.org/10.1371/journal.pone.0222485.t003
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 12 / 26
duration of 24.0 minutes (SD = 10.62, range 3–45). Twenty-two mothers (63%) and six fathers
(17%) joined the online parent network during the study period. The majority of postings and
responses in the community originated from four mothers, accounting for 76% of the online
activity.
Primary outcomes
At post-treatment, of those with diagnostic post-interview data significantly more adolescents
in the ICBT condition (40%) were free of their primary diagnosis, compared with adolescents
in the WL condition (16%), χ2(1) = 4,89, p = .027 (Table 4 shows frequency of improvement and recovery for the intention-to-treat sample). Based on the odds ratio, the odds of a partici-
pant being free of their primary diagnosis were 3.60 times higher if they received ICBT than if
they were on WL. Also, significantly more adolescents in the ICBT condition (29%) were free
of all anxiety diagnoses at post, compared with adolescents in the WL condition (3%), χ2(1) = 7,89, p = .005. The odds of a participant being free of all anxiety diagnoses were 12.38 times
higher if they received ICBT than if they were on WL.
Table 5 shows mean scores, standard deviations, ES’s and p-values for all outcome scales at pre and post for the ICBT and WL condition and at follow-up for the ICBT condition. The
number of adolescents in the ICBT group free of their primary diagnosis at post-treatment
(intention-to-treat sample) was distributed as follows: SAD = 0% (0/2), SoP = 21% (3/14),
Table 4. Frequency of improvement and recovery for the intention-to-treat sample; n (%).
Post-treatment 3-month f/up 12-month f/up
Free of primary diagnosis
ICBT 14/35 (40%) 12/33 (36%) - -
WL 5/32 (16%)
Free of all anxiety diagnoses
ICBT 10/35 (29%) 10/33 (30%) - -
WL 1/32 (3%)
Improved (SCAS-Cadol)
ICBT 22/32 (69%) 18/27 (67%) 12/25 (48%)
WL 8/31 (26%)
Recovered (SCAS-Cadol)
ICBT 14/32 (44%) 14/27 (52%) 10/25 (40%)
WL 2/31 (6%)
Improved (SCAS-Pmother)
ICBT 24/35 (69%) 23/33 (70%) 20/31 (65%)
WL 7/32 (22%)
Recovered (SCAS-Pmother)
ICBT 9/35 (26%) 13/33 (39%) 14/31 (45%)
WL 2/32 (6%)
Improved (SCAS-Pfather)
ICBT 9/25 (35%) 14/23 (61%) 14/21 (67%)
WL 5/27 (19%)
Recovered (SCAS-Pfather)
ICBT 1/25 (4%) 7/23 (30%) 9/21 (43%)
WL 2/27 (7%)
Note. SCAS-C/P: Spence Children’s Anxiety Scale Child/Parent version. f/up: follow-up.
https://doi.org/10.1371/journal.pone.0222485.t004
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 13 / 26
Table 5. Means, standard deviations and effect sizes of primary and secondary outcomes.
Unadjusted mean (standard deviation) {valid n} Effect size; p-value Pre-treatment Post-
treatment
3-month f/up 12-month f/up Within-group Within-group Between-
group
Within-
group
Within-
group
pre to post pre to f/up ICBT vs. WL post to 3 f/up 3 to 12 f/up
ADIS CSRprim
ICBT 6.69 (0.68)
{35}
3.83 (2.65)
{35}
3.88 (2.56)
{33}
d = 1.48, p < .001
d = 1.50, p < .001
a d = .65, p =
.022
d = -.02, p = .495
WL 6.54 (0.92)
{35}
5.09 (2.29)
{32}
d = .83, p = .001
ADIS CSRall
ICBT 13.17 (5.28)
{35}
6.89 (4.56)
{35}
6.39 (4.64)
{33}
d = 1.27, p < .001
d = 1.36, p < .001
a d = .83, p =
.002
d = .11, p = .482
WL 11.17 (4.42)
{35}
9.28 (4.13)
{32}
d = .44, p = .012
SCAS-Cadol
ICBT 47.29 (15.38)
{35}
31.88 (16.06)
{32}
28.74 (16.08)
{27}
33.36 (12.64)
{25}
d = .98, p < .001
d = .99, p < .001
b d = .68, p <
.001
d = .20, p = .054
d = -.32, p = .056
WL 45.60 (15.79)
{35}
40.19 (19.90)
{31}
d = .30, p = .038
SCAS-Pmother
ICBT 48.23 (17.28)
{35}
29.43 (13.60)
{35}
28.73 (16.57)
{33}
28.58 (18.58)
{31}
d = 1.21, p < .001
d = 1.10, p < .001
b d = 1.12, p <
.001
d = .05, p = .548
d = .01, p = .849
WL 42.71 (14.85)
{35}
41.31 (17.54)
{32}
d = .09, p = .632
SCAS-Pfather
ICBT 41.45 (19.18)
{31}
32.46 (17.29)
{26}
28.33 (15.33)
{24}
23.55 (18.77)
{22}
d = .49, p = .001
d = .94, p < .001
b d = .46, p =
.011
d = .25, p = .007
d = .28, p = .484
WL 37.41 (12.75)
{32}
36.96 (14.35)
{27}
d = .03, p = .579
CALISadol
ICBT 13.54 (6.56)
{35}
10.59 (7.65)
{32}
8.93 (7.63)
{27}
9.72 (8.55) {25} d = .41, p = .007
d = .50, p = .002
b d = .21, p =
.254
d = .22, p = .183
d = -.10, p = .663
WL 13.97 (6.73)
{35}
12.42 (8.65)
{31}
d = .20, p = .180
CALISmother
ICBT 32.14 (11.36)
{35}
22.49 (12.67)
{35}
23.21 (16.33)
{33}
20.61 (14.96)
{31}
d = .80, p < .001
d = .87, p < .001
b d = .93, p <
.001
d = -.05, p = .856
d = .17, p = .450
WL 33.71 (11.91)
{35}
35.34 (13.47)
{32}
d = -.13, p = .513
CALISfather
ICBT 29.42 (14.07)
{31}
25.89 (15.58)
{27}
24.21 (14.53)
{24}
16.59 (13.50)
{22}
d = .24, p = .049
d = .93, p < .001
b d = .20, p =
.227
d = .11, p = .195
d = .54, p = .044
WL 28.91 (9.74)
{32}
28.52 (13.00)
{27}
d = .03, p = .574
WHO-5
ICBT 44.69 (21.44)
{35}
49.50 (21.69)
{32}
55.85 (25.21)
{27}
48.48 (25.98)
{25}
d = .22, p = .143
d = .16, p = .129
b d = -.04, p =
.945
d = .27, p = .100
d = -.29, p = .105
WL 48.91 (19.19)
{35}
54.06 (20.39)
{31}
d = .26, p = .135
S-MFQadolescent
ICBT 10.46 (6.47)
{35}
8.06 (7.77)
{32}
d = .34, p = .014
d = -.11, p = .932
WL 10.94 (7.07)
{35}
7.77 (7.14)
{31}
d = .45, p = .010
S-MFQmother
(Continued)
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 14 / 26
SP = 50% (2/4), PD = 67% (2/3), PD with AP = 0% (0/1), AP without PD = 0% (0/2),
GAD = 83% (5/6), OCD = 67% (2/3). Although numerically diverse, a Fisher’s Exact test with
eight groups (all diagnoses) revealed a non-significant association (p = .062) between type of primary anxiety diagnosis pre-treatment and whether or not the adolescents were free of these
post-treatment (Fisher’s Exact test value = 11.05).
Pre-post MLM analyses revealed significant effects on clinician-rated severity of primary
diagnosis (CSRprim; d = 0.65, p = 0.022) and all diagnoses (CSRall; d = 0.83, p = 0.002) (see Table 4 and Figs 2 and 3), demonstrating significantly larger improvements in diagnostic
severity in the ICBT group compared to WL. For anxiety symptoms, pre-post comparisons on
the SCAS revealed significant time × condition interactions for adolescents (d = 0.68,
Table 5. (Continued )
Unadjusted mean (standard deviation) {valid n} Effect size; p-value Pre-treatment Post-
treatment
3-month f/up 12-month f/up Within-group Within-group Between-
group
Within-
group
Within-
group
pre to post pre to f/up ICBT vs. WL post to 3 f/up 3 to 12 f/up
ICBT 9.91 (5.79)
{35}
5.77 (5.25)
{35}
d = .75, p < .001
d = .60, p = .008
WL 10.43 (6.15)
{35}
9.47 (6.37)
{32}
d = .15, p = .360
S-MFQfather
ICBT 8.26 (5.43)
{31}
7.15 (5.86)
{27}
d = .20, p = .103
d = .07, p = .813
WL 7.88 (4.89)
{32}
6.56 (4.85)
{27}
d = .27, p = .229
SEQ-Ctotal
ICBT 70.71 (13.88)
{35}
75.25 (16.55)
{32}
d = .30, p = .008
d = .12, p = .367
WL 71.23 (15.76)
{35}
74.00 (16.27)
{31}
d = .17, p = .174
SEQ-Cacademic
ICBT 27.17 (7.14)
{35}
27.13 (8.04)
{32}
d = -.01, p = .970
d = -.08, p = .449
WL 26.11 (8.49)
{35}
26.71 (7.84)
{31}
d = .07, p = .270
SEQ-Csocial
ICBT 25.09 (5.46)
{35}
26.06 (6.11)
{32}
d = .17, p = .133
d = .04, p = .668
WL 26.20 (6.79)
{35}
27.06 (6.32)
{31}
d = .13, p = .412
SEQ-Cemotional
ICBT 18.46 (5.92)
{35}
22.06 (5.87)
{32}
d = .61, p < .001
d = .38, p = .026
WL 18.91 (5.26)
{35}
20.23 (6.35)
{31}
d = .23, p = .205
Note. ADIS: Anxiety Disorder Interview Schedule for DSM-IV; CSR: Clinical Severity Rating; SCAS-C/P: Spence Children’s Anxiety Scale Child/Parent version; CALIS: Child Anxiety Life Inference Scale; WHO-5: WHO-5 Well-being Index Questionnaire; S-MFQ: short version of the Moods and Feelings Questionnaire; SEQ-C: The
Self-Efficacy Questionnaire for Children; f/up: follow-up. a
Pre to 3-month follow-up b
Pre to 12-month follow-up. For the SCAS, CALIS and S-MFQ, a family-wise Bonferroni-corrected significance level of p = 0.017 was deployed. Effect size calculations based on observed values and expressed as Cohen’s d. Positive effect sizes indicate improvement.
https://doi.org/10.1371/journal.pone.0222485.t005
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 15 / 26
p < 0.001), mothers (d = 1.12, p < 0.001) and fathers (d = 0.46, p = 0.011), indicating signifi- cantly greater reductions in adolescent anxiety symptoms in the ICBT group compared to WL.
Secondary outcomes
Significant time × condition interactions were found for mother reports on life interference (CALIS; d = 0.93, p < 0.001), mother reports on symptoms of depression (S-MFQ; d = 0.60, p = 0.008) and adolescent report on the emotional subscale of self-efficacy (SEQ-C; d = 0.38, p = 0.026). For the remaining secondary outcome measures (including adolescent and father reported CALIS, adolescent and father reported S-MFQ, WHO-5, SEQ-C total scale, and
SEQ-C academic and social subscales), no significant interaction effects were found (see
Table 4).
Improvement and recovery rates
According to adolescent reported anxiety symptoms on SCAS-C, significantly more participants
in the ICBT condition (69%) than in the WL condition (26%) were classified as improved at post-
treatment (χ2 (1) = 11.62, p = .001). Also, significantly more participants in the ICBT condition (44%) than in the WL condition (6%) were classified as recovered (χ2 (1) = 11.56, p = .001) accord- ing to adolescent SCAS-C report. For the post-treatment mother-reported SCAS-P, significantly
more participants in the ICBT condition (69%) than in the WL condition (22%) were considered
improved (χ2 (1) = 14.66, p < .001). There were, however, no significant differences in number of participants classified as recovered (χ2 (1) = 4.62, p = .032) in the ICBT condition (26%) compared
Fig 2. Mean CSR pre-post for primary diagnosis.
https://doi.org/10.1371/journal.pone.0222485.g002
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 16 / 26
to the WL condition (6%). For father reported SCAS-P, the number of participants improved
post-treatment in the ICBT condition (35%) was not significantly larger than those improved in
the WL condition (19%, χ2 (1) = 2.02, p = .156). Similarly, participants classified as recovered according to father reported SCAS-P at post-treatment did not differ significantly (Fisher’s exact
test: p = 1.00 (two sided)) in the ICBT group (4%) compared to the WL group (7%).
Treatment maintenance
At 3-month follow-up, 36% adolescents were free of their primary anxiety diagnosis and 30%
were free of all anxiety diagnoses. MLM within-group analyses of the CSRprim and CSRall from
pre-treatment to 3-month follow-up showed large and significant improvements (d = 1.50, p < 0.001; and d = 1.36, p < 0.001, respectively), while analyses from post to 3-month follow- up revealed small and non-significant changes (d = -0.02, p = 0.495; and d = 0.11, p = 0.482, respectively).
Within-group analyses of the SCAS reports from pre to 12-month follow-up (see Fig 4 and
Table 4) revealed large and highly significant improvements for all responders (adolescents:
d = .99, p < .001; mothers: d = 1.10, p < .001; and fathers: d = .94, p < .001). Analyses from post to 3-month follow-up revealed significant improvements for the father report on SCAS-P
(d = 0.25, p = 0.007). Otherwise, analyses generally showed non-significant positive changes in the follow-up periods.
At 3-month follow-up, the number of adolescents classified as improved on the SCAS was
67% according to adolescent report, 70% according to mother report and 61% according to
Fig 3. Mean CSR pre-post for sum of all anxiety diagnoses.
https://doi.org/10.1371/journal.pone.0222485.g003
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 17 / 26
father report. The number classified as recovered was 52% according to adolescent report, 39%
according to mother report, and 30% according to father report. At 12 month follow-up, the
number of adolescents categorized as improved on the SCAS was 48% according to adolescent
report, 65% according to mother report, and 67% according to father report. The number clas-
sified as recovered was 40% according to adolescent report, 45% according to mother report,
and 43% according to father report.
For the CALIS reports, progress was also maintained during the follow-up period according
to adolescent, mother and father reports at 3- and 12-month follow-up, with within-group
analyses from pre to 12-month follow-up showing significant improvements of moderate to
large ES’s (adolescents: d = 0.50, p = 0.002; mothers: d = 0.868, p < 0.001; and fathers: d = 0.931, p < 0.001). Analyses of participants’ emotional well-being scores on the WHO-5 indicated no change, as the total treatment effect from baseline to 12-month follow-up was
small and non-significant (d = 0.16, p = 0.129). Similarly, effects for both follow-up periods analyzed separately were small and non-significant (see Table 4).
Treatment satisfaction
Overall, adolescents were satisfied with the intervention, with a mean of 17.3 (SD = 3.64) out
of a maximum score of 21. Seventeen (55%) rated the statement “the treatment helped me” to
be ‘true’, nine (29%) to be ‘partly true’, and five (16%) not to be true. Eighteen adolescents
(58%) would recommend the treatment to a friend, four (13%) would not recommend it, and
the remaining nine (29%) answered ‘partly true’ to the statement. Adolescents’ qualitative
comments on the treatment were generally highly positive. Two (7%) would, however, have
liked more time, and one (3%) expressed the need for face-to-face sessions with the therapist.
Mothers and fathers were also generally satisfied with the intervention with a mean rating of
23.1 (SD = 3.66) and 22.2 (SD = 3.19), respectively, out of max 27.
Participants and their parents were also asked post-treatment whether the treatment had
caused them/their child to feel worse. One adolescent (3%) rated the statement to be true,
while three adolescents (10%) rated it to be ‘partly true’. One mother (3%) rated the statement
to be true, while two mothers (6%) answered ‘partly true’ to the statement. No fathers rated the
statement to be ‘true’, but three (12%) rated it to be ‘partly true’. Regarding those (n = 2) who
rated the above statement to be true, additional qualitative information from questionnaires
Fig 4. Mean SCAS total scores at pre, post, 3- and 12-month follow-up for adolescents, mothers and fathers.
https://doi.org/10.1371/journal.pone.0222485.g004
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 18 / 26
and booster-sessions was collected but did not indicate requirements of further clinical
intervention.
Discussion
The present study evaluated the efficacy of therapist-guided ICBT for adolescents with anxiety
disorders. To the best of our knowledge, the study is the first randomized evaluation of the
ChilledOut Online program, and it is the first systematic evaluation of an ICBT intervention
for this population in Denmark.
Lending support to our main hypothesis, participants receiving ICBT demonstrated signifi-
cant improvements at post-treatment compared to participants in the WL condition across all
raters on diagnostic severity and level of anxiety symptoms (CSR and SCAS). The between-
group ES’s found for CSRprim and CSRall in the present study (d = 0.65 and d = 0.83, respec- tively) are in the lower end of those found in other similar WL controlled studies; for instance
by Lenhard et al. [48]: d = 0.69; and Vigerland et al. [83]: d = 1.66. In their study of the Cool Teens CD-ROM, Wuthrich et al. [56] showed a between-group ES, d = 1.35. The within-treat- ment pre-post effects on diagnostic severity were, however, quite similar in our study to that of
other studies; for instance a d = 1.48 on CSRprim compared to ds 1.32–1.69 [56, 83, 84]. The lower between-group ES in the present study may be explained by the large improvement in
the WL group, d = 0.83. Seven of the 35 WL participants (20%) reported having had contact with mental health services during the WL period, three had received hypnosis in private prac-
tice, one CBT with the GP, and three had contacted the local psychiatric hospital, possibly
explaining part of the improvements found in the WL group. Large changes in waitlist partici-
pants is however not unprecedented as previous studies of CBT with youth/children has found
similar outcomes (e.g., [85]). Despite these improvements, most WL participants (32 of 35)
received ICBT after the waitlist period.
The study demonstrated significant treatment effects on both the adolescent and parent rat-
ings of anxiety symptoms on SCAS, but with large differences in the size of the effect: SCAS-C
d = 0.68; SCAS-Pmother d = 1.12; and SCAS-Pfather d = 0.46. Moderate between-group treatment effects for child and adolescent self-rated anxiety symptoms on SCAS-C has previously been
shown [84] but from an overall perspective on the literature, these self-ratings tend to fluctuate
from small and/or non-significant [46, 48, 83] to large and highly significant [44]. It may be
that the SCAS-C does not adequately frame the symptoms as experienced by adolescents. It
may also be a testimony of the adolescents’ emotional self-awareness being under development
with accompanying variances in their ability to detect and report changes. The outcome on
mother reported SCAS-P (d = 1.12) is, on the other hand, larger than those reported in previ- ous studies (d = 0.16–0.69; [46, 48, 56, 83, 84, 86]). It is not common for CBT studies to mea- sure both mother- and father-reported anxiety. The present study is, to the best of our
knowledge, the first RCT of ICBT to include separate reports from both parents. However, a
similar difference in parents’ ratings of anxiety symptoms was found in a prior study of face-
to-face CBT with children and adolescents at CEBU [87] with mothers reporting larger
improvements than fathers. More research is needed to further investigate the divergence of
anxiety symptom scores between adolescents, mothers and fathers.
Results of the study partially supported our second hypothesis, that the ICBT treatment
would reveal better outcome in depressive symptoms, self-efficacy and quality of life compared
to the WL condition. Significant improvements were found on the mother ratings of depres-
sive symptoms (S-MFQ), mother ratings of anxiety life interference (CALIS) and self-ratings
of the emotional self-efficacy sub-scale (SEQ-C emotional). Two previous ICBT studies treat-
ing adolescents with anxiety disorders also measured depressive symptoms, with one [48]
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 19 / 26
showing no effects and the other [44] demonstrating large effects (d = 1.39). As was found in this study, it is not unusual to find smaller outcomes (and/or less frequently significant find-
ings) on depression than on anxiety symptoms (e.g., [88]) in studies using CBT to treat anxiety
disorders.
Supporting our third hypothesis, results at 3- and 12-month follow-up assessment demon-
strated a maintenance of treatment effects for all included measures. These results are similar
to those of Wuthrich et al. [56], Spence et al. [84], and Tillfors et al. [44], who showed treat-
ment maintenance at 3-, 6-, and 12-month follow-up, respectively. However, our results differ
from those of Lenhard et al. [48], Vigerland et al. [83], and Spence et al. [46], whom have all
demonstrated further anxiety symptom improvements during the follow-up periods of 3-, 3-
and 12-month, respectively. Future studies including longer follow-up periods might help illu-
minate how improvements achieved through ICBT for children and adolescents evolve over
time.
Limitations
The study has a number of limitations that should be considered when interpreting the results.
Although improvements in the ICBT group suggests that recovery could be attributed to the
specific intervention and not to confounding factors such as spontaneous remission or
repeated assessment, an active control condition would be required to rule out potential effects
of such unspecific factors. The study was not controlled in the follow-up period since partici-
pants in the WL condition received treatment during this period. Rater blindness to treatment
condition was often broken post-treatment and at follow-up. Moreover, the use of self-referral
and the high socioeconomic status of participant families could have implications for gener-
alizability of results, although it is difficult to predict the direction of potential effects. Lastly,
the majority of CSR ratings were based on ADIS interviews conducted by students. Although
thoroughly trained and supervised, they had only modest prior assessment experience com-
pared to trained psychologists used in other studies. Combined with low variability in CSR of
primary diagnosis (all were scored 6–8) and the use of telephone-recorded interviews in the
reliability checks, this might help explain the moderate inter-rater reliability scores found in
the study.
Conclusion
This study demonstrated the Danish version of ChilledOut Online to be efficacious and feasi-
ble in relieving symptoms of anxiety in adolescents. As such, the study supports previous find-
ings of similar guided ICBT interventions and helps build a strong foundation for future
research in and implementation of ICBT in mental health services for adolescents with anxiety
disorders.
Supporting information
S1 File. CONSORT checklist.
(PDF)
S2 File. Trial study protocol (ethics committee).
(PDF)
S3 File. Trial study protocol_Danish version.
(PDF)
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 20 / 26
Acknowledgments
The authors would like to thank our colleagues at Macquarie University for the permission to
use the program. The authors also wish to thank clinic secretary Marianne Bjerregaard Madsen
for her incommensurable diligence and helpfulness, as well as all the students who took part in
the study. Lastly, we wish to express our sincere gratitude to associate professor Mia Skytte
O’Toole for her valuable advice and guidance in the area of multilevel modeling.
Author Contributions
Conceptualization: Silke Stjerneklar, Esben Hougaard, Mikael Thastum.
Data curation: Silke Stjerneklar.
Formal analysis: Silke Stjerneklar.
Funding acquisition: Esben Hougaard, Mikael Thastum.
Investigation: Silke Stjerneklar, Mikael Thastum.
Methodology: Silke Stjerneklar, Esben Hougaard, Mikael Thastum.
Project administration: Silke Stjerneklar.
Software: Silke Stjerneklar, Lauren F. McLellan.
Supervision: Silke Stjerneklar, Esben Hougaard, Mikael Thastum.
Validation: Esben Hougaard, Mikael Thastum.
Writing – original draft: Silke Stjerneklar.
Writing – review & editing: Silke Stjerneklar, Esben Hougaard, Lauren F. McLellan, Mikael
Thastum.
References 1. Costello EJ, Egger HL, Copeland W, Erkanli A, Angold A. The developmental epidemiology of anxiety
disorders: Phenomenology, prevalence, and comorbidity. In: Silverman WK FA, editor. Anxiety disor-
ders in children and adolescents. 2. ed. Cambridge: Cambridge University Press; 2011. p. 56–75.
2. Polanczyk GV, Salum GA, Sugaya LS, Caye A, Rohde LA. Annual research review: A meta-analysis of
the worldwide prevalence of mental disorders in children and adolescents. Journal of child psychology
and psychiatry, and allied disciplines. 2015; 56(3):345–65. Epub 2015/02/05. https://doi.org/10.1111/
jcpp.12381 PMID: 25649325.
3. Merikangas KR, He JP, Burstein M, Swanson SA, Avenevoli S, Cui L, et al. Lifetime prevalence of men-
tal disorders in U.S. adolescents: results from the National Comorbidity Survey Replication—Adoles-
cent Supplement (NCS-A). Journal of the American Academy of Child and Adolescent Psychiatry.
2010; 49(10):980–9. Epub 2010/09/22. https://doi.org/10.1016/j.jaac.2010.05.017 PMID: 20855043;
PubMed Central PMCID: PMC2946114.
4. Beesdo-Baum K, Knappe S. Developmental epidemiology of anxiety disorders. Child Adolesc Psychiatr
Clin N Am. 2012; 21(3):457–78. Epub 2012/07/18. https://doi.org/10.1016/j.chc.2012.05.001 PMID:
22800989.
5. Kessler RC, Avenevoli S, Costello EJ, Georgiades K, Green JG, Gruber MJ, et al. Prevalence, persis-
tence, and sociodemographic correlates of DSM-IV disorders in the National Comorbidity Survey Repli-
cation Adolescent Supplement. Archives of General Psychiatry. 2012; 69(4):372–80. https://doi.org/10.
1001/archgenpsychiatry.2011.160 PMID: 22147808; 2012-09587-005.
6. Kessler RC, Petukhova M, Sampson NA, Zaslavsky AM, Wittchen H-U. Twelve-month and lifetime
prevalence and lifetime morbid risk of anxiety and mood disorders in the United States. International
Journal of Methods in Psychiatric Research. 2012; 21(3):169–84. https://doi.org/10.1002/mpr.1359
PMID: 22865617
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 21 / 26
7. James AC, James G, Cowdrey FA, Soler A, Choke A. Cognitive behavioural therapy for anxiety disor-
ders in children and adolescents. The Cochrane database of systematic reviews. 2013;(6):Cd004690.
Epub 2013/06/05. https://doi.org/10.1002/14651858.CD004690.pub3 PMID: 23733328.
8. Merikangas KR, He J-P, Brody D, Fisher PW, Bourdon K, Koretz DS. Prevalence and treatment of men-
tal disorders among US children in the 2001–2004 NHANES. Pediatrics. 2010; 125(1):75–81. https://
doi.org/10.1542/peds.2008-2598 PMID: 20008426; 2010-17117-014.
9. Wang PS, Aguilar-Gaxiola S, Alonso J, Angermeyer MC, Borges G, Bromet EJ, et al. Use of mental
health services for anxiety, mood, and substance disorders in 17 countries in the WHO world mental
health surveys. The Lancet. 2007; 370(9590):841–50. http://dx.doi.org/10.1016/S0140-6736(07)
61414-7. PMID: 621878556; 2007-13901-003.
10. Shafran R, Clark DM, Fairburn CG, Arntz A, Barlow DH, Ehlers A, et al. Mind the gap: Improving the dis-
semination of CBT. Behav Res Ther. 2009; 47(11):902–9. Epub 2009/08/12. https://doi.org/10.1016/j.
brat.2009.07.003 PMID: 19664756.
11. Stallard P, Udwin O, Goddard M, Hibbert S. The Availability of Cognitive Behaviour Therapy Within Spe-
cialist Child and Adolescent Mental Health Services (CAMHS): A National Survey. Behavioural and
Cognitive Psychotherapy. 2007; 35(4):501–5. Epub 2007/05/31. https://doi.org/10.1017/
S1352465807003724
12. Gulliver A, Griffiths KM, Christensen H. Perceived barriers and facilitators to mental health help-seeking
in young people: A systematic review. BMC psychiatry. 2010;10. https://doi.org/10.1186/1471-244X-
10-10 PMID: 860176004; 2011-02872-001.
13. Adolescents Tillett J. and Informed Consent: Ethical and Legal Issues. The Journal of Perinatal & Neo-
natal Nursing. 2005; 19(2):112–21. PMID: 621153809; 2006-03909-002.
14. Zachrisson HD, Rodje K, Mykletun A. Utilization of health services in relation to mental health problems
in adolescents: a population based survey. BMC public health. 2006; 6:34. Epub 2006/02/17. https://
doi.org/10.1186/1471-2458-6-34 PMID: 16480522; PubMed Central PMCID: PMC1386725.
15. Reavley NJ, Cvetkovski S, Jorm AF, Lubman DI. Help-Seeking for Substance Use, Anxiety and Affec-
tive Disorders Among Young People: Results from the 2007 Australian National Survey of Mental
Health and Wellbeing. Australian & New Zealand Journal of Psychiatry. 2010; 44(8):729–35. https://doi.
org/10.3109/00048671003705458 PMID: 20636194
16. Nearchou FA, Bird N, Costello A, Duggan S, Gilroy J, Long R, et al. Personal and perceived public men-
tal-health stigma as predictors of help-seeking intentions in adolescents. J Adolesc. 2018; 66:83–90.
Epub 2018/05/26. https://doi.org/10.1016/j.adolescence.2018.05.003 PMID: 29800758.
17. Rickwood DJ, Deane FP, Wilson CJ. When and how do young people seek professional help for mental
health problems? The Medical journal of Australia. 2007; 187(7 Suppl):S35–9. Epub 2007/11/28. PMID:
17908023.
18. Wilson CJ, Bushnell JA, Caputi P. Early access and help seeking: practice implications and new initia-
tives. Early intervention in psychiatry. 2011; 5 Suppl 1:34–9. Epub 2011/01/19. https://doi.org/10.1111/j.
1751-7893.2010.00238.x PMID: 21208389.
19. Elliott BA, Larson JT. Adolescents in mid-sized and rural communities: Foregone care, perceived barri-
ers, and risk factors. Journal of Adolescent Health. 2004; 35(4):303–9. https://doi.org/10.1016/j.
jadohealth.2003.09.015 PMID: 15450544; 2004-19430-007.
20. Andersson G, Titov N. Advantages and limitations of Internet-based interventions for common mental
disorders. World Psychiatry. 2014; 13(1):4–11. Epub 2014/02/06. https://doi.org/10.1002/wps.20083
PMID: 24497236; PubMed Central PMCID: PMC3918007.
21. Barak A, Klein B, Proudfoot JG. Defining internet-supported therapeutic interventions. Annals of behav-
ioral medicine: a publication of the Society of Behavioral Medicine. 2009; 38(1):4–17. Epub 2009/09/30.
https://doi.org/10.1007/s12160-009-9130-7 PMID: 19787305.
22. Andersson G. Internet-Delivered Psychological Treatments. Annu Rev Clin Psychol. 2016; 12:157–79.
Epub 2015/12/15. https://doi.org/10.1146/annurev-clinpsy-021815-093006 PMID: 26652054.
23. Johansson R, Andersson G. Internet-based psychological treatments for depression. Expert Review of
Neurotherapeutics. 2012; 12(7):861–70. https://doi.org/10.1586/ern.12.63 PMID: 22853793
24. Kleiboer A, Donker T, Seekles W, van Straten A, Riper H, Cuijpers P. A randomized controlled trial on
the role of support in Internet-based problem solving therapy for depression and anxiety. Behav Res
Ther. 2015; 72:63–71. Epub 2015/07/21. https://doi.org/10.1016/j.brat.2015.06.013 PMID: 26188373.
25. Baumeister H, Reichler L, Munzinger M, Lin J. The impact of guidance on Internet-based mental health
interventions—A systematic review. Internet Interventions. 2014; 1(4):205–15. http://dx.doi.org/10.
1016/j.invent.2014.08.003.
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 22 / 26
26. Griffiths KM, Christensen H. Review of randomised controlled trials of internet interventions for mental
disorders and related conditions. Clinical Psychologist. 2006; 10(1):16–29. http://dx.doi.org.ez.
statsbiblioteket.dk:2048/10.1080/13284200500378696.
27. Christensen H, Batterham P, Calear A. Online interventions for anxiety disorders. Current opinion in
psychiatry. 2014; 27(1):7–13. https://doi.org/10.1097/YCO.0000000000000019 PMID: 24257123;
2013-43204-002.
28. Hedman E, Ljotsson B, Lindefors N. Cognitive behavior therapy via the Internet: a systematic review of
applications, clinical efficacy and cost-effectiveness. Expert review of pharmacoeconomics & outcomes
research. 2012; 12(6):745–64. Epub 2012/12/21. https://doi.org/10.1586/erp.12.67 PMID: 23252357.
29. Richards D, Richardson T, Timulak L, McElvaney J. The efficacy of internet-delivered treatment for gen-
eralized anxiety disorder: A systematic review and meta-analysis. Internet Interventions. 2015; 2
(3):272–82. http://dx.doi.org/10.1016/j.invent.2015.07.003.
30. Spek V, Cuijpers P, Nyklı́ček I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: A meta-analysis. Psychological Medicine. 2007; 37(3):319–28.
https://doi.org/10.1017/S0033291706008944 PMID: 17112400; 2007-03729-002.
31. Cunningham MJ, Wuthrich VM, Rapee RM, Lyneham HJ, Schniering CA, Hudson JL. The Cool Teens
CD-ROM for anxiety disorders in adolescents: a pilot case series. Eur Child Adolesc Psychiatry. 2009;
18(2):125–9. Epub 2008/06/20. https://doi.org/10.1007/s00787-008-0703-y PMID: 18563472.
32. Donker T, Blankers M, Hedman E, Ljotsson B, Petrie K, Christensen H. Economic evaluations of Inter-
net interventions for mental health: a systematic review. Psychol Med. 2015:1–20. Epub 2015/08/04.
https://doi.org/10.1017/s0033291715001427 PMID: 26235445.
33. Hedman E, Andersson E, Ljótsson B, Andersson G, Rück C, Lindefors N. Cost-effectiveness of Inter-
net-based cognitive behavior therapy vs. cognitive behavioral group therapy for social anxiety disorder:
Results from a randomized controlled trial. Behaviour Research and Therapy. 2011; 49(11):729–36.
https://doi.org/10.1016/j.brat.2011.07.009 PMID: 21851929
34. Lenhard F, Ssegonja R, Andersson E, Feldman I, Ruck C, Mataix-Cols D, et al. Cost-effectiveness of
therapist-guided internet-delivered cognitive behaviour therapy for paediatric obsessive-compulsive
disorder: results from a randomised controlled trial. BMJ open. 2017; 7(5):e015246. Epub 2017/05/19.
https://doi.org/10.1136/bmjopen-2016-015246 PMID: 28515196.
35. Ho J, Corden ME, Caccamo L, Tomasino KN, Duffecy J, Begale M, et al. Design and evaluation of a
peer network to support adherence to a web-based intervention for adolescents. Internet Interventions.
2016; 6:50–6. https://doi.org/10.1016/j.invent.2016.09.005 PMID: 27722095
36. Stallard P, Velleman S, Richardson T. Computer use and attitudes towards computerised therapy
amongst young people and parents attending child and adolescent mental health services. Child and
Adolescent Mental Health. 2010; 15(2):80–4. http://dx.doi.org/10.1111/j.1475-3588.2009.00540.x.
PMID: 622181267; 2010-06698-003.
37. Subrahmanyam K, Lin G. Adolescents on the net: Internet use and well-being. Adolescence. 2007; 42
(168):659–77. PMID: 18229503; 2007-19818-003.
38. Nicholas J, Oliver K, Lee K, O’Brien M. Help-seeking behaviour and the Internet: An investigation
among Australian adolescents. Australian e-Journal for the Advancement of Mental Health. 2004; 3
(1):16–23. https://doi.org/10.5172/jamh.3.1.16
39. Ebert DD, Zarski AC, Christensen H, Stikkelbroek Y, Cuijpers P, Berking M, et al. Internet and com-
puter-based cognitive behavioral therapy for anxiety and depression in youth: a meta-analysis of ran-
domized controlled outcome trials. PloS one. 2015; 10(3):e0119895. Epub 2015/03/19. https://doi.org/
10.1371/journal.pone.0119895 PMID: 25786025.
40. Pennant ME, Loucas CE, Whittington C, Creswell C, Fonagy P, Fuggle P, et al. Computerised therapies
for anxiety and depression in children and young people: A systematic review and meta-analysis.
Behaviour Research and Therapy. 2015; 67:1–18. https://doi.org/10.1016/j.brat.2015.01.009 PMID:
25727678; 2015-12473-002.
41. Podina IR, Mogoase C, David D, Szentagotai A, Dobrean A. A Meta-Analysis on the Efficacy of Tech-
nology Mediated CBT for Anxious Children and Adolescents. Journal of Rational-Emotive & Cognitive-
Behavior Therapy. 2016; 34(1):31–50. https://doi.org/10.1007/s10942-015-0228-5
42. Ye X, Bapuji SB, Winters SE, Struthers A, Raynard M, Metge C, et al. Effectiveness of internet-based
interventions for children, youth, and young adults with anxiety and/or depression: a systematic review
and meta-analysis. BMC health services research. 2014; 14:313. Epub 2014/07/20. https://doi.org/10.
1186/1472-6963-14-313 PMID: 25037951; PubMed Central PMCID: PMC4110069.
43. Hollis C, Falconer CJ, Martin JL, Whittington C, Stockton S, Glazebrook C, et al. Annual Research
Review: Digital health interventions for children and young people with mental health problems—A sys-
tematic and meta-review. Journal of Child Psychology and Psychiatry. 2017; 58(4):474–503. https://doi.
org/10.1111/jcpp.12663 PMID: 27943285; 2016-60086-001.
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 23 / 26
44. Tillfors M, Andersson G, Ekselius L, Furmark T, Lewenhaupt S, Karlsson A, et al. A randomized trial of
internet-delivered treatment for social anxiety disorder in high school students. Cognitive Behaviour
Therapy. 2011; 40(2):147–57. https://doi.org/10.1080/16506073.2011.555486 PMID: 25155815; 2011-
14818-007.
45. Spence SH, Donovan CL, March S, Gamble A, Anderson R, Prosser S, et al. Online CBT in the treat-
ment of child and adolescent anxiety disorders: Issues in the development of BRAVE-ONLINE and two
case illustrations. Behavioural and Cognitive Psychotherapy. 2008; 36(4):411–30. http://dx.doi.org/10.
1017/S135246580800444X. PMID: 621613258; 2008-15717-004.
46. Spence SH, Donovan CL, March S, Gamble A, Anderson RE, Prosser S, et al. A randomized controlled
trial of online versus clinic-based CBT for adolescent anxiety. Journal of consulting and clinical psychol-
ogy. 2011; 79(5):629–42. https://doi.org/10.1037/a0024512 PMID: 21744945; 2011-14202-001.
47. Lenhard F, Vigerland S, Andersson E, Ruck C, Mataix-Cols D, Thulin U, et al. Internet-delivered cogni-
tive behavior therapy for adolescents with obsessive-compulsive disorder: an open trial. PloS one.
2014; 9(6):e100773. https://doi.org/10.1371/journal.pone.0100773 PMID: 24949622; PubMed Central
PMCID: PMC4065049.
48. Lenhard F, Andersson E, Mataix-Cols D, Ruck C, Vigerland S, Hogstrom J, et al. Therapist-Guided,
Internet-Delivered Cognitive-Behavioral Therapy for Adolescents With Obsessive-Compulsive Disor-
der: A Randomized Controlled Trial. Journal of the American Academy of Child and Adolescent Psychi-
atry. 2017; 56(1):10–9.e2. Epub 2016/12/21. https://doi.org/10.1016/j.jaac.2016.09.515 PMID:
27993223.
49. Tillfors M, Carlbring P, Furmark T, Lewenhaupt S, Spak M, Eriksson A, et al. Treating university stu-
dents with social phobia and public speaking fears: Internet delivered self-help with or without live group
exposure sessions. Depression and Anxiety. 2008; 25(8):708–17. https://doi.org/10.1002/da.20416
PMID: 18729147; 2008-13151-008.
50. Spence SH. A measure of anxiety symptoms among children. Behaviour Research and Therapy. 1998;
36(5):545–66. https://doi.org/10.1016/s0005-7967(98)00034-5 PMID: 9648330; 1998-10708-006.
51. Scahill L, Riddle MA, McSwiggin-Hardin M, Ort SI, King RA, Goodman WK, et al. Children’s Yale-Brown
Obsessive Compulsive Scale: Reliability and validity. Journal of the American Academy of Child & Ado-
lescent Psychiatry. 1997; 36(6):844–52. http://dx.doi.org/10.1097/00004583-199706000-00023. PMID:
619078361; 1997-04647-022.
52. Lyneham HJ, McLellan LF, Cunningham M, Wuthrich V, Schniering C, Hudson JL, et al. ChilledOut
Online Program. Sydney: Centre for Emotional Health, Macquarie University; 2014.
53. Rapee RM, Lyneham HJ, Schniering CA, Wuthrich V, Abbott MA, Hudson JL, et al. The cool kids child
and adolescents anxiety program–therapist manual. Sydney, NSW: Centre for Emotional Health, Mac-
quarie University; 2006.
54. Wuthrich VM, Cunningham MJ, Rapee RM. Cool Teens: A Computerized Intervention for Anxious Ado-
lescents. The Wiley-Blackwell Handbook of The Treatment of Childhood and Adolescent Anxiety: John
Wiley & Sons, Ltd.; 2013. p. 249–74.
55. Cunningham M. Overview of design approaches in the Cool Teens CD and other computer programs
for adolescent anxiety. E-Journal of Applied Psychology. 2008; 4(2):6–11. PMID: 851233202; 2010-
10939-002.
56. Wuthrich VM, Rapee RM, Cunningham MJ, Lyneham HJ, Hudson JL, Schniering CA. A randomized
controlled trial of the Cool Teens CD-ROM computerized program for adolescent anxiety. Journal of the
American Academy of Child & Adolescent Psychiatry. 2012; 51(3):261–70. http://dx.doi.org/10.1016/j.
jaac.2011.12.002. PMID: 964197898; 2012-05353-009.
57. Stjerneklar S, Hougaard E, Nielsen AD, Gaardsvig MM, Thastum M. Internet-based cognitive behav-
ioral therapy for adolescents with anxiety disorders: A feasibility study. Internet Interventions. 2018;
11:30–40. https://doi.org/10.1016/j.invent.2018.01.001 PMID: 30135757
58. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Fourth Edi-
tion. 4th ed. ed. Washington, DC: American Psychiatric Association; 1994.
59. Silverman WK, Albano AM. Anxiety Disorders Interview Schedule for Children for DSM–IV: Child and
Parent Versions. San Antonio: TX: Psychological Corporation; 1996.
60. Silverman WK, Saavedra LM, Pina AA. Test-retest reliability of anxiety symptoms and diagnoses with
anxiety disorders interview schedule for DSM-IV: Child and parent versions. Journal of the American
Academy of Child & Adolescent Psychiatry. 2001; 40(8):937–44. http://dx.doi.org/10.1097/00004583-
200108000-00016. PMID: 619583840; 2001-01750-010.
61. Wood JJ, Piacentini JC, Bergman RL, McCracken J, Barrios V. Concurrent validity of the anxiety disor-
ders section of the Anxiety Disorders Interview Schedule for DSM-IV: child and parent versions. Journal
of clinical child and adolescent psychology: the official journal for the Society of Clinical Child and
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 24 / 26
Adolescent Psychology, American Psychological Association, Division 53. 2002; 31(3):335–42. Epub
2002/08/02. https://doi.org/10.1207/s15374424jccp3103_05 PMID: 12149971.
62. Lyneham HJ, Rapee RM. Agreement Between Telephone and In-Person Delivery of a Structured Inter-
view for Anxiety Disorders in Children. Journal of the American Academy of Child & Adolescent Psychi-
atry. 2005; 44(3):274–82. http://dx.doi.org/10.1097/00004583-200503000-00012. PMID: 620692159;
2005-02313-012.
63. Arendt K, Hougaard E, Thastum M. Psychometric properties of the child and parent versions of Spence
Children’s Anxiety Scale in a Danish community and clinical sample. Journal of Anxiety Disorders.
2014; 28(8):947–56. https://doi.org/10.1016/j.janxdis.2014.09.021 PMID: 25445085
64. Lyneham HJ, Sburlati ES, Abbott MJ, Rapee RM, Hudson JL, Tolin DF, et al. Psychometric properties
of the Child Anxiety Life Interference Scale (CALIS). Journal of Anxiety Disorders. 2013; 27(7):711–9.
https://doi.org/10.1016/j.janxdis.2013.09.008 PMID: 24135256; 2013-41121-012.
65. Angold A, Costello EJ, Messer SC, Pickles A. Development of a short questionnaire for use in epidemio-
logical studies of depression in children and adolescents. International Journal of Methods in Psychiatric
Research. 1995; 5(4):237–49. PMID: 618820158; 1996-02633-002.
66. Muris P. A brief questionnaire for measuring self-efficacy in youths. Journal of Psychopathology and
Behavioral Assessment. 2001; 23(3):145–9. http://dx.doi.org/10.1023/A:1010961119608. PMID:
619816969; 2002-10117-002.
67. Muris P. Relationships between self-efficacy and symptoms of anxiety disorders and depression in a
normal adolescent sample. Personality and Individual Differences. 2002; 32(2):337–48. http://dx.doi.
org/10.1016/S0191-8869(01)00027-7. PMID: 619720149; 2002-00167-010.
68. Suldo SM, Shaffer EJ. Evaluation of the Self-Efficacy Questionnaire for Children in Two Samples of
American Adolescents. Journal of Psychoeducational Assessment. 2007; 25(4):341–55. https://doi.org/
10.1177/0734282907300636
69. Topp CW, Østergaard SD, Søndergaard S, Bech P. The WHO-5 Well-Being Index: A Systematic Review of the Literature. Psychotherapy and Psychosomatics. 2015; 84(3):167–76. https://doi.org/10.
1159/000376585 PMID: 25831962
70. Löwe B, Spitzer RL, Gräfe K, Kroenke K, Quenter A, Zipfel S, et al. Comparative validity of three screen-
ing questionnaires for DSM-IV depressive disorders and physicians’ diagnoses. Journal of affective dis-
orders. 2004; 78(2):131–40. https://doi.org/10.1016/s0165-0327(02)00237-9 PMID: 14706723
71. Brähler E, Mühlan H, Albani C, Schmidt S. Teststatistische Prüfung und Normierung der deutschen Ver-
sionen des EUROHIS-QOL Lebensqualität-Index und des WHO-5 Wohlbefindens-Index. Diagnostica.
2007; 53(2):83–96. https://doi.org/10.1026/0012-1924.53.2.83
72. Brown A, Ford T, Deighton J, Wolpert M. Satisfaction in Child and Adolescent Mental Health Services:
Translating Users’ Feedback into Measurement. Administration and Policy in Mental Health and Mental
Health Services Research. 2014; 41(4):434–46. https://doi.org/10.1007/s10488-012-0433-9 PMID:
22829193
73. Attride-Stirling J. Development of methods to capture users’ views of CAMHS in clinical governance
reviews. Cambridge, UK: Commission for Health Improvement; 2002.
74. Goodman R, Meltzer H, Bailey V. The strengths and difficulties questionnaire: A pilot study on the valid-
ity of the self-report version. European Child & Adolescent Psychiatry. 1998; 7(3):125–30. https://doi.
org/10.1007/s007870050057
75. Goodman R. The Strengths and Difficulties Questionnaire: A Research Note. Journal of Child Psychol-
ogy and Psychiatry. 1997; 38(5):581–6. https://doi.org/10.1111/j.1469-7610.1997.tb01545.x PMID:
9255702
76. Tracey TJ, Kokotovic AM. Factor structure of the Working Alliance Inventory. Psychological Assess-
ment: A Journal of Consulting and Clinical Psychology. 1989; 1(3):207–10. http://dx.doi.org/10.1037/
1040-3590.1.3.207. PMID: 617705879; 1990-03343-001.
77. Melville KM, Casey LM, Kavanagh DJ. Dropout from Internet-based treatment for psychological disor-
ders. The British journal of clinical psychology / the British Psychological Society. 2010; 49(Pt 4):455–
71. Epub 2009/10/06. https://doi.org/10.1348/014466509x472138 PMID: 19799804.
78. Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2 ed. Hillsdale, NJ: Lawrence Erl-
baum Associates; 1988.
79. Chakraborty H, Gu H. A Mixed Model Approach for Intent-to-Treat Analysis in Longitudinal Clinical Tri-
als with Missing Values. RTI Press publication No MR-0009-0903 [Internet]. 2009 31.01.2018. Available
from: https://doi.org/10.3768/rtipress.2009.mr.0009.0903 PMID: 30896910
80. Jacobson NS, Truax P. Clinical significance: A statistical approach to defining meaningful change in
psychotherapy research. Journal of consulting and clinical psychology. 1991; 59(1):12–9. https://doi.
org/10.1037//0022-006x.59.1.12 PMID: 2002127; 1991-16094-001.
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 25 / 26
81. Heck RH, Thomas SL, Tabata LN. Multilevel and Longitudinal Modeling with IBM SPSS (Quantitative
Methodology Series). 2 ed. New York and London: Routledge/Taylor & Francis; 2014.
82. Bisgaard MP, Poulin PD. Statistical Yearbook. 121th ed: Statistics Denmark; 2017 june 2017. 568 p.
83. Vigerland S, Ljótsson B, Thulin U, Öst L-G, Andersson G, Serlachius E. Internet-delivered cognitive
behavioural therapy for children with anxiety disorders: A randomised controlled trial. Behaviour
Research and Therapy. 2016; 76:47–56. https://doi.org/10.1016/j.brat.2015.11.006 PMID: 26649465;
2015-59077-007.
84. Spence SH, Donovan CL, March S, Kenardy JA, Hearn CS. Generic versus disorder specific cognitive
behavior therapy for social anxiety disorder in youth: A randomized controlled trial using internet deliv-
ery. Behav Res Ther. 2017; 90:41–57. Epub 2016/12/19. https://doi.org/10.1016/j.brat.2016.12.003
PMID: 27988427.
85. Thirlwall K, Cooper PJ, Karalus J, Voysey M, Willetts L, Creswell C. Treatment of child anxiety disorders
via guided parent-delivered cognitive-behavioural therapy: randomised controlled trial. The British jour-
nal of psychiatry: the journal of mental science. 2013; 203(6):436–44. Epub 2013/08/24. https://doi.org/
10.1192/bjp.bp.113.126698 PMID: 23969483.
86. March S, Spence SH, Donovan CL. The efficacy of an Internet-based cognitive-behavioral therapy
intervention for child anxiety disorders. Journal of Pediatric Psychology. 2009; 34(5):474–87. https://
doi.org/10.1093/jpepsy/jsn099 PMID: 18794187; 2009-08292-004.
87. Arendt K, Thastum M, Hougaard E. Efficacy of a Danish version of the Cool Kids program: a random-
ized wait-list controlled trial. Acta Psychiatr Scand. 2015. https://doi.org/10.1111/acps.12448 PMID:
26016532.
88. Cuijpers P, Sijbrandij M, Koole S, Huibers M, Berking M, Andersson G. Psychological treatment of gen-
eralized anxiety disorder: a meta-analysis. Clin Psychol Rev. 2014; 34(2):130–40. Epub 2014/02/04.
https://doi.org/10.1016/j.cpr.2014.01.002 PMID: 24487344.
ICBT for adolescents with anxiety disorders
PLOS ONE | https://doi.org/10.1371/journal.pone.0222485 September 18, 2019 26 / 26
Copyright of PLoS ONE is the property of Public Library of Science and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
