two essay 3 pages each.

A Meta-Analysis of the Effects of Internet- and Computer- Based Cognitive-Behavioral Treatments for Anxiety

Mark A. Reger

Madigan Army Medical Center and University of Washington School of Medicine

Gregory A. Gahm

Madigan Army Medical Center

Internet-and computer-based cognitive-behavioral treatments have been introduced as novel approaches to deliver standard, quality treatment that may reduce barriers to care. The purpose of this review is to quantitatively summarize the literature examining the treatment effects of Internet- or computer-based treatment (ICT) on anxiety. Nineteen randomized controlled ICT trials were identified and subjected to fixed and random effects meta-analytic techniques. Weighted mean effect sizes (Cohen’s d) showed that ICT was superior to waitlist and placebo assignment across outcome measures (ds5.49–1.14). The effects of ICT also were equal to therapist-delivered treatment across anxiety disorders. However, conclusions were limited by small sample sizes, the rare use of placebo controls, and other methodological problems. In addition, the number of available studies limited the opportunity to conduct analyses by diagnostic group; there was preliminary support for the use of ICT for panic disorder and phobia. Large, well-designed, placebo-controlled trials are needed to confirm and extend the results of this meta-analysis. & 2008 Wiley Periodicals, Inc. J Clin Psychol 65: 53–75, 2009.

Keywords: Internet; computer; cognitive-behavioral therapy; meta- analysis; anxiety

Dozens of randomized, controlled clinical trials have demonstrated that cognitive- behavioral therapy (CBT) improves symptoms of anxiety (Deacon & Abramowitz, 2004; Scott, 2001). Despite this body of research, multiple factors still limit the utility

Correspondence concerning this article should be addressed to: Gregory Gahm, Ph.D., Director, Telehealth and Technology Directorate, Defense Centers of Excellence for Psychological Health and Traumatic Brain Injury, Bldg. 9040, Fitzsimmons Drive, Attn: OMAMC 9933A, Tacoma, WA 98431- 1100; e-mail: [email protected]

JOURNAL OF CLINICAL PSYCHOLOGY, Vol. 65(1), 53-75 (2009) & 2008 Wiley Periodicals, Inc. Published online in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/jclp.20536

m

m

54 Journal of Clinical Psychology, January 2009

of efficacious psychotherapies. Many individuals avoid or delay seeking treatment because the very nature of their psychological disorder limits travel outside the home (e.g., agoraphobia) or because of geographic or mobility limitations. In addition, clinicians who are trained to provide efficacious treatments may be in short supply (Scott, 2001); this problem is surely exacerbated in rural locations with poor access to treatment resources. Among therapists who practice CBT, training and follow- through vary widely (Sholomskas et al., 2005). When high-quality treatment resources are available, a fear of stigma can influence an individual’s decision to access care (Hoge et al., 2004; Wells, Robins, Bushnell, Jarosz, & Oakley-Browne, 1994). Despite improved outreach and educational efforts, treatment stigma continues to affect young populations (Chandra & Minkovitz, 2006), and the problem may be exacerbated in some ethnic minority groups (e.g., African Americans; Das, Olfson, McCurtis, & Weissman, 2006), and special occupational groups such as military personnel and others (Carter, Buckey, Greenhalgh, Holland, & Hegel, 2005; Hoge et al., 2004). Thus, additional efforts are needed to continue to reduce barriers to treatment.

Internet- and computer-based treatments have been proposed as a delivery approach that may minimize several barriers to care while providing the opportunity to deliver standardized, empirically supported treatment. There is significant interest in a diverse array of Internet and computer-based treatments (ICTs). For example, computer-based approaches can be used to facilitate communication between patients and providers. Some methods seek to simply ‘‘extend’’ the benefits of traditional face-to-face therapy between sessions with the use of e-mail or other Internet-based communications. Alternatively, technological equipment can be used as the primary communication device to support therapy across geographical distances (e.g., video teleconferencing; Modai et al., 2006).

Technology also can be used to automate or manualize routine aspects of treatment. For example, software can be developed to train patients in basic cognitive-behavioral skills such as relaxation training, or in cognitive restructuring (Litz, Engel, Bryant, & Papa, 2007). Standard personal computers (PCs) can be used to train patients on a single CBT skill or to deliver an entire standardized treatment plan. As technological advances continue at an unprecedented rate, the use of highly interactive software that utilizes audio, video, animations, and advanced graphics is possible. Branching logic can be used to create response-dependent experiences that attempt to shape treatment based on the needs of an individual patient (Carter et al., 2005). Specific programs differ in their degree of interactivity, ranging from primarily text-based approaches to multimedia experiences that extend what is possible through traditional self-help manuals.

Specialized software and equipment also have been explored as possible tools to facilitate psychotherapy. Virtual reality immerses a participant into a realistic, computer-generated world. Hardware such as head mounted displays (HMDs), motion tracking, tactile feedback, olfactory stimuli, and naturalistic navigation devices help deliver three-dimensional virtual environments in a believable manner. A growing body of literature is exploring the use of virtual reality to facilitate a variety of therapeutic goals (e.g., exposure; Difede et al., 2007).

Interest also is growing in the use of other technologies for therapeutic goals, including cell phones, handheld video devices, personal digital assistants, bulletin boards, Internet chat rooms, blogs, and so on (e.g., Bang, Timpka, Eriksson, Holm, & Nordin, 2007). Different technologies, by their nature, imply varying levels of therapist involvement. Some ICTs are proposed as pure ‘‘self-help’’ solutions while

Journal of Clinical Psychology DOI: 10.1002/jclp

A Meta-Analysis of Internet-Based CBT 55

others assume regular, face-to-face clinician contact that utilizes a technological tool to accomplish some therapy goals.

While the nature of ICTs differ, in general they may have potential to reduce some barriers to care (Andersson, Bergstrom, Carlbring, & Lindefors, 2005; Emmelkamp, 2005). ICTs may improve access to treatment for individuals who require psychological services, but whose transportation options are limited due to anxiety, other mental health problems, physical disabilities, or other medical complications. ICT also has the potential to improve access to care in rural areas, and may provide the opportunity to reach individuals with mental disorders who would otherwise avoid treatment because of a fear of stigma. Furthermore, early evidence has suggested that computer-based CBT can be cost-effective (McCrone et al., 2004). Although ICT may be contraindicated in certain cases, it may provide a flexible tool to address a variety of treatment goals for a subgroup of patients.

The purpose of this review is to quantitatively summarize the literature that has examined the treatment effects of Internet- or computer-based CBT on anxiety. For the purposes of this article, we focused on Internet- and computer-based approaches that utilized software on a standard PC to automate the delivery of CBT training (In the sections that follow, the term ICT is used to refer to this subgroup of technological approaches.) Several qualitative reviews of this literature have been conducted and have suggested, in general, that ICT may be a viable option for some patients (e.g., Andersson et al., 2005; Emmelkamp, 2005). However, a quantitative analysis provides a clearer understanding of the magnitude of the treatment effects, and allows for a comparison of ICT to treatment as usual, placebo, or waitlist assignment. Efforts also can address the variability noted in the current research literature.

The present review examines the effects of ICT on five types of clinical measures (Depression, Anxiety, General Distress, Dysfunctional Thinking, and Functioning/ Quality of Life) in individuals with anxiety using meta-analytic methodologies.

Method

Selection of Studies

We attempted to include all studies on the effects of Internet- or computer-based CBT for anxiety published through 2007. A preliminary search for articles was conducted using PubMed, PsychInfo, and the Cochrane Central Register of Controlled Trials with keywords identified through a search of the database index terms. For example, the Medical Subject Heading (MeSH) terms used for indexing articles for MEDLINE/ PubMed were searched, and the following search terms were identified: Internet, computers, computer-assisted therapy, behavior therapy, and anxiety disorders (which includes phobic disorders, panic disorder, PTSD, etc.). Terms from the index were selected to be intentionally broad to maximize the chances of identifying all articles. For example, behavior therapy includes multiple subcategories such as cognitive therapy, cognitive behavior therapy, relaxation techniques, and so on. Reference lists of all relevant articles were searched manually to locate any other studies that were not identified by the preliminary search. Corresponding authors of relevant articles were e- mailed to request any additional unpublished manuscripts or data they may possess. One ‘‘in press’’ article was identified.

All studies were included that met the following eligibility criteria: (a) randomized controlled trial (RCT) of individual CBT delivered with Internet or computer-based software; (b) included adult participants with anxiety; (c) administered outcome measures assessing levels of anxiety, depression, general distress, dysfunctional

Journal of Clinical Psychology DOI: 10.1002/jclp

56 Journal of Clinical Psychology, January 2009

thinking, or functioning/quality of life; and (d) reported sufficient information about the study results to allow computation of an effect size (ES) estimate. Studies were excluded when the computer hardware used to conduct the treatment required more than a PC and an Internet connection. Thus, studies utilizing equipment such as a virtual reality head mount display were excluded. Three studies were identified that appeared to meet inclusion criteria, but did not report sufficient statistical information for the computation of an ES. Letters were sent to the corresponding authors of these studies requesting additional information; one response was received, allowing the study to be included (Klein & Richards, 2001). The final results of the literature search produced 19 studies that met the aforementioned criteria. A list of excluded studies is available upon request. Participant characteristics and ICT treatment details for each study are presented in Tables 1 and 2, respectively.

Methodological Quality

The methodological quality of each included primary study was rated utilizing a rating scheme recommended in a recent review of systems available for rating the strength of scientific evidence (West et al., 2002). The methodological ratings of the RCTs were based on four broad categories (study population, interventions, measurement of effect, and data presentation) that were divided into specific weighted criteria based on each criterion’s importance for validity and clinical relevance (van der Heijden, van der Windt, Kleijnen, Koes, & Bouter, 1996). Specific criteria were modified when the characteristics of interest were specific to medical RCTs in the original system. The detailed criteria utilized in this study are available upon request. Briefly, studies were rated based upon (a) patient selection [possible points (pp) 5 4], (b) randomization procedure (pp 5 5), (c) sample size (pp 5 15), (d) comparability of groups (pp 5 6), (e) management of dropouts (pp 5 6), (f) rates of loss-to-follow-up (pp 5 5), (g) description of treatments (pp 5 8), (h) management of co-interventions (pp54), (i) blinding of patients (pp54), (j) relevant outcome measures (pp56), (k) blinded outcome measurement (pp53), (l) duration of follow-up (pp54), and (m) adequate analysis and presentation (pp58). Thus, RCTs were rated on a total of 78 possible points.

Data Coding

The following information was extracted from the published articles for each coded effect: (a) number of participants and their demographics, (b) participants’ diagnosis, (c) nature of ICT, (d) type of control group used [waitlist, placebo, or therapist-delivered treatment as usual (TAU)], (e) outcome measures administered, (f) summary statistics required for computation of ESs, and (g) amount of clinician contact during ICT.

Outcome measures were categorized into the following five domains: Depression, Anxiety, General Distress, Dysfunctional Thinking, and Functioning/Quality of Life. Each measure was coded in a single domain that was judged to best represent the primary area measured. Measures of functional abilities and quality of life were grouped together based on research showing that functional skills are a primary component of quality of life (Quilty, Van Ameringen, Mancini, Oakman, & Farvolden, 2003). Table 3 lists the tests that were included in each outcome domain.

Journal of Clinical Psychology DOI: 10.1002/jclp

A Meta-Analysis of Internet-Based CBT 57

Journal of Clinical Psychology DOI: 10.1002/jclp

Table 1

Participant Characteristics in Primary Studies

Study N Andersson

69% had at least some college –

Recruited by means of newspaper and magazine articles, and an Internet link

Social Phobia

et al., 2006

Carlbring et al., 2005

Recruited from a waitlist of individuals interested in Internet treatment.

Met DSM-IV criteria for a diagnosis of panic disorder as confirmed by the SCID; panic disorder was the primary problem with a duration of at least 1 year. All met DSM-IV criteria for a diagnosis of panic disorder; panic disorder was the primary problem with a duration of at least 1 year; all participants had at least one panic attack or limited symptoms

Carlbring, Westling, Ljungstrand, Ekselius, & Andersson, 2001

–

Recruited with advertising and Web links.

Gilroy, Kirkby, Daniels, Menzies, & Montgomery, 2000 Grime, 2004

45 33.1

–

Recruited via newspaper advertisements and public notices.

attack during the pretreatment baseline. Specific Phobia (Spiders) based on the CIDI; minimum duration of phobia of 1 year.

Heading et al., 2001 Hirai & Clum, 2005

–

Kenardy, Dow, et al., 2003

186 36.8

Recruited by means of referrals from general practitioners and through the media.

Participants met DSM-IV criteria for panic disorder, confirmed by the SCID; current episode durationZ3 months; all considered panic their main problem. Elevated anxiety sensitivity (Anxiety Sensitivity Index 4 24).

Kenardy, 83 McCafferty, & Rosa, 2003 Kline & Richards, 2001 22 Klein, Richards, & 55 Austin, 2006

19.9

All were first-year university psychology students.

M age 64 37.3 49 35.0 41 34.0

M education/ alternative

Recruitment

Diagnosis/symptoms

48 39.0 40 34.9 27 29.4

–

Recruited through a London NHS occupational health department. Recruited through newspaper and notice board advertisements

Missed 10 or more days of work due to stress, anxiety, or depression in the past 6 months and GHQ-12 Z4 Met DSM-IV criteria for Specific Phobia (Spiders) based on CIDI-Auto 2.1.

45% were university students 33% had some postsecondary education All were first year psychology students 11.7

Recruited with advertisements, and from the introductory psychology class student pool

All participants met at least the reexperiencing and avoidance criteria for PTSD (DSM-IV).

40.8 ––

– Recruited by contacts to a study Web site.

Met DSM-IV criteria for a primary diagnosis of panic disorder Primary diagnosis of panic disorder as assessed by the ADIS-IV

58 Journal of Clinical Psychology, January 2009

Journal of Clinical Psychology DOI: 10.1002/jclp

TABLE 1. Continued

M age M education/ Study N alternative

Recruitment Recruited through the Internet or advertisements;

Diagnosis/symptoms

Knaevelsrud & 96 35 Maercker, 2007

44% had university degree –

participants had a primary DSM-IV diagnosis of panic disorder, as assessed by the ADIS-IV. Recruited undergraduate psychology students

Sx of PTSD (70% scored above a cut-point for PTSD on the IES).

Lange, van de Ven, 25 22.0 & Schrieken, 2001

All participants had experienced a traumatic event (including loss of a beloved one) at least 3 months prior to the study and were excluded for ‘‘not (suffering) from (PTSD) or pathological grief.’’ Symptoms of PTSD; 90% scored above IES cut-off for PTSD. All participants had

Lange, van de Ven, 101 39.0 Schrieken, & Emmelkamp, 2003

–

Participants were recruited through the Interapy Web site.

Litz, Engel, Bryant, & 45 39.2 Papa, 2007 Marks, Kenwright, 90 38.0 McDonough, Whittaker,

– 11.0 years

Recruited through advertisements and Web sites

experienced a traumatic event (including events such as sudden loss of a beloved one, divorce, loss of job) at least 3 months prior to the study. PTSD

& Mataix-Cols, 2004 Orbach, Lindsay, 58 23.7 & Grey, 2007 Richards, Klein, 32 36.6 & Austin, 2006 Zetterqvist, 63 39.2 Maanmies, Strom, & Andersson, 2003

3.1 years at University 13.2 years

Participants responded to notices in general practitioner practices or phobia self-help groups, or were referred by health professionals. e-mail invitation circulated to graduate students

All participants met DSM-IV criteria for agoraphobia without panic disorder, panic disorder with agoraphobia, social phobia, and/or simple phobia as measured by semistructured interview. Test anxiety

–

Recruited from visitors to a Web site or through local and national print and electronic media Recruited through newspaper articles or through a Web site for health.

Met DSM-IV criteria for a primary diagnosis of panic disorder as assessed by the AIDI-IV. Stress. Levels of depression and anxiety were ‘‘checked’’ on screening instruments.

Note. N represents the no. of participants from each primary study included in the meta-analysis. Participants met diagnostic criteria for each disorder listed unless otherwise specified. CBT 5 cognitive-behavioral treatment; GHQ 5 General Health Questionnaire; PTSD 5 posttraumatic stress disorder; TAU 5 treatment as usual; CIDI 5 Composite International Diagnostic Interview; ADIS 5 Anxiety Disorders Interview Schedule; IES 5 Impact of Event Scale.

A Meta-Analysis of Internet-Based CBT 59

Journal of Clinical Psychology DOI: 10.1002/jclp

Table 2

Characteristics of Treatment and Control Groups in Primary Studies

Study

Nature of ICT

Control group(s)

Nature of placebo –

Clinician contact with ICT group

Andersson et al., 2006

Internet-delivered self-help program divided into nine modules. Topics included an introduction, a model of social phobia and CBT, modifying cognitive distortions, conducting behavioral experiments, setting goals, exposure and reality testing, shifting focus, listening skills, setting boundaries and assertiveness, perfectionism and self-confidence, and relapse prevention. In each module, participants posted a message in a discussion forum about a specific topic. PC: M of 7.5 sessions.

Waitlist

Individual feedback via e-mail on questions at the end of each module (approximately 3 hr per participant). Participants invited to complete two 3-hr group exposure sessions; 59% completed second session.

Carlbring et al., 2001

Internet-delivered self-help program divided into six modules, including psychoeducation, breathing retraining, modifying interpretations of physical symptoms, interoceptive exposure, in vivo exposure, and relapse prevention and assertiveness training. Each module ended with five to eight questions on which participants received e-mailed feedback. PC: All

Waitlist

–

No personal contact; individual feedback via e- mail on questions at the end of each module; e- mail to inquire about progress; M contact time (via e-mail) 5 90 min

Carlbring et al., 2005

Completers 5 100%. Internet-delivered self-help program divided into 10 modules, including (1–2) psychoeducation and socialization, (3) breathing retraining and hyperventilation test, (4–5) cognitive restructuring, (6–7) interoceptive exposure, (8–9) exposure in vivo, (10) relapse prevention and assertiveness training. Participants also posted at least one message in an online discussion group during each module. Additioanally, each module ended with five to eight questions and an

TAU

–

Individual feedback was provided via e-mail on homework for each module; participants could send unlimited e-mail (M 5 15.4); M therapist time 5 150 min per participant

60 Journal of Clinical Psychology, January 2009

Journal of Clinical Psychology DOI: 10.1002/jclp

TABLE 2. Continued

Study

Nature of ICT

Control group(s)

Nature of placebo

Clinician contact with ICT group

Gilroy et al., 2000

interactive multiple-choice quiz. Feedback on homework was provided via e-mail. PC: M of 7.4 sessions. Computer-aided vicarious exposure in which participants used a computer mouse to direct a screen figure into scenarios with pictures of spiders, or plastic, dead, or live spiders. A game score is calculated, and an anxiety thermometer rises as the figure approaches the spiders. Beating the Blues: Eight sessions of interactive computerized CBT. Cognitive components explore automatic thoughts, thinking errors, and distraction, challenging unhelpful thinking, core beliefs, and attributional style. Behavioral components include activity scheduling, task breakdown, problem solving, sleep management, relaxation training and biofeedback, planning and prioritizing, and graded exposure. PC: M of 6.4 sessions.

Placebo, Progressive muscle relaxation delivered with an TAU audiotape. Authors cited evidence showing that

Five min in the initial session to ensure participant was comfortable using the program

Grime, 2004

TAU –

ICT was administered in a private room in the Occupational Health Department. The author reviewed weekly progress reports to monitor for adverse events. Treatment and control groups were allowed to continue conventional care.

Heading et al., 2001

Computerized symbolic modeling treatment in which participants guide a computer figure into an elevator. An onscreen thermometer displays increasing anxiety/panic as the figure approaches, enters, remains in, or travels in the elevator, but a score on the screen increases when the computer figure is in these situations. PC 5 100%.

Waitlist, – TAU

Therapist worked with the patient for the first 5 min of session.

Hirai & Clum, 2005

Internet-based, cognitive-behavioral self-change program composed of information, breathing retraining, muscle relaxation, imagery-induced relaxation, cognitive restructuring, and written exposure modules.

Waitlist –

Contact was made only to prompt participants to take assessments or mastery tests or to provide information about the timeline for completion of the program. Technical assistance was allowed.

relaxation is ineffective for specific phobia.

A Meta-Analysis of Internet-Based CBT 61

Journal of Clinical Psychology DOI: 10.1002/jclp

Kenardy, Dow et al, 2003

Palmtop computer program included a self- statement module, a breathing control module, and a situational and interoceptive exposure module. PC: M of 5.9 sessions.

Waitlist, TAU

–

The computer-based treatment augmented therapist-delivered CBT. Total therapist contact time was 6 hr.

Kenardy, McCafferty, & Rosa, 2003

Online Anxiety Prevention Program included psychoeducation, relaxation training, interoceptive exposure, cognitive restructuring, and relapse prevention. Each session required participants to practice skills and record progress. PC: accessed software M of 7.76 times with 90.4 min per access.

Waitlist

–

None noted

Klien et al., 2001

Internet-based program focused on the nature, causes, and management of panic. Topics included negative self-statements, errors in thinking, and techniques for overcoming cognitive errors.

Placebo

Self-monitoring

No therapeutic contact. Investigators assisted participants in accessing and navigating the program, and checked to ensure they were accessing it during the treatment phase

Klien et al., 2006

Panic Online (delivered via the Internet) included an introductory module, four learning modules, and a relapse-prevention module. Treatment methods included controlled breathing, cognitive restructuring, and interoceptive and situational exposure. Interapy was an online standardized treatment that consisted of three treatment phases (Self- Confrontation, Cognitive Restructuring, Social Sharing and Farewell Ritual). Each phase began with an information section and then required participants to write essays on given topics.

Placebo, TAU

Information control in which participants were instructed to reread Internet-based informational program about panic. Clinical student called controls weekly to assess panic and encourage self-monitoring. Received no active CBT.

Individualized e-mail support and feedback; actual therapist time: M 5 332.5 min per participant

Knaevelsrud & Maercker, 2007

Waitlist

–

At midpoint and end of each treatment phase, therapists provided Internet-based feedback and instructions tailored to patients’ needs. Provided recognition of patients work, positive feedback, motivation, and encouragement to voice concerns. Assisted patients in focusing on painful memories during self-confrontation phase

Lange et al., 2001

Interapy was an online standardized treatment that consisted of three treatment phases (Actualization/Confrontation, Cognitive Reappraisal, Sharing and Farewell Ritual). Each phase began with an information section and then required participants to write essays on given topics.

Waitlist

–

During the middle of each treatment phase, therapists provided participants with feedback on their essays via the Interapy Web site, and provided instructions on how to proceed.

62 Journal of Clinical Psychology, January 2009

Journal of Clinical Psychology DOI: 10.1002/jclp

TABLE 2. Continued

Study

Nature of ICT

Control group(s)

Nature of placebo –

Clinician contact with ICT group

Lange et al., 2003

Interapy was an online standardized treatment that consisted of three treatment phases (Actualization/Confrontation, Cognitive Reappraisal, Sharing and Farewell Ritual). Each phase began with an information section and then required participants to write essays on given topics. PC: Completers 5 100%. DE-STRESS was an Internet-delivered program composed of the following components: self- monitoring triggers, developing a hierarchy of trauma triggers, stress management, graduated self-guided, in vivo exposure, trauma writing sessions, a review of progress, and relapse prevention.

Waitlist

During the middle of each treatment phase, therapists provided participants with feedback on their essays via the Interapy Web site, and provided instructions on how to proceed.

Litz et al., 2007

Placebo

Self-monitored daily non-trauma-related concerns and hassles and wrote online about these experiences. Psychoeducational materials were available, but no skills training or prescriptions for proactive steps were provided

Study therapist assisted with hierarchy of stressful situations and provided training in stress management and cognitive reframing; called participants to assess readiness to complete trauma narrative

Marks et al., 2004

FearFighter was a computer-based system with Placebo,

Computer-guided self-relaxation techniques with up to 20 min of clinician contact per session focused on relaxation

Protocol included 20 min of coaching, progress discussion, and treatment advice during each of six sessions; actual mean total therapist time per participant was 76 min.

Orbach et al., 2007

Placebo

Internet-based program consisting of four modules, including psychoeducation, relaxation training (based only on music and breathing), a thought diary, and ‘‘brain puzzles.’’

No therapeutic contact. Investigators introduced the Internet-based programs to participants (some by phone and some face-to face).

Richards et al., 2006

Panic Online was an Internet-delivered program with four learning modules and introductory and relapse prevention modules. CBT methods included controlled breathing, progressive

Placebo

Information-Only Controls received no active CBT. They completed assessments and received weekly e-mail to check on panic status and provided minimal support. Controls were

E-mail contact to guide participants through the program. E-mailed weekly feedback based on weekly panic summary information entered in the program. M therapist time 5 342.8 min.

nine steps including (1–3) introduction, rationale, and use of a ‘‘co-therapist;’’ (4) identifying triggers and writing problem statements; (5) developing exposure homework; (6) panic coping skills; (7) practicing coping during exposure; (8) review, feedback, and modifying goals; and (9) troubleshooting. Internet-based treatment program consisting of six modules, including psychoeductaion, relaxation training, rational thinking, study skills training, and two modules on controlling stress. PC: M of 5.2 sessions

TAU

A Meta-Analysis of Internet-Based CBT 63

Journal of Clinical Psychology DOI: 10.1002/jclp

Zetterqvist et al., 2003

Internet-based stress-management program consisting of six treatment modules, including relaxation training, problem solving, time management, and cognitive and behavioral restructuring. An optional information section included sleeping problems, nutrition and eating habits, exercise, work stress, and social life and assertiveness. PC: M of 4.2 sessions

Waitlist –

E-mail feedback on submitted forms and correspondence on individual issues of stress management or technical problems.

muscle relaxation, cognitive restructuring, and interoceptive and situational exposure. A second group used Panic and Stress Online, which was the same except that it included six learning modules on stress.

encouraged to reread information on a panic resource program that contained no active CBT.

Note. PC 5 Patient Compliance

64 Journal of Clinical Psychology, January 2009

Journal of Clinical Psychology DOI: 10.1002/jclp

Table 3

Measures Categorized by Clinical Domain

Beck Depression Inventory Montgomery–Asberg

4 3 2 2 2 1 1 1 1

Body Sensations Questionnaire Beck Anxiety Inventory Fear Questionnaire Impact of Events Scale

4 4 4 4 3 3

Anticipatory Fear Rating Scale 1 Anxiety Hierarchy Questionnaire Brief Symptom Inventory-Anxiety 1 Generalized Anxiety Rating Scale 1 Leibowitz Social Anxiety Scale 1 Main Problem and Goals Rating 1

Depression measures Test k

Anxiety measures Test k

Test k Weekly Panic Attack Frequency 2

Depression Rating Scale Depression, Anxiety, Stress Scales-

Depression Scale Hospital Anxiety and Depression

Body Vigilance Scale Mobility Inventory for

Scales-Depression Subscale Symptom Checklist-90-Depression

Agoraphobia Behavioral Assessment Test Clinician Agoraphobia Rating Clinician Panic Disorder Rating Depression, Anxiety, Stress Scales-

2 2 2

Scales Panic Frequency 1 Panic Frequency and Severity 1

Subscale Brief Symptom Inventory-

Depression Center for Epidemiological Studies

Anxiety Scale Hospital Anxiety and Depression

2

Ratings 1 PTSD Symptom Scale-Interview

Depression Scale Generalized Depression Rating

Scales-Anxiety Panic Disorder Severity Scale Phobic Targets Spider Questionnaire State-Trait Anxiety Inventory Symptom Checklist-90-Anxiety Subscale Subjective Units of Distress 2

Version 1 Social Interaction Anxiety Scale Social Phobia Scale 1 Social Phobia Screening

Scale Profile of Mood States-

2 2 2 2 2 2

Depressiveness

Questionnaire 1 Stressful Responses Questionnaire 1 Test Anxiety Inventory 1

Domain

A Meta-Analysis of Internet-Based CBT 65

Journal of Clinical Psychology DOI: 10.1002/jclp

General Distress

Domain Dysfunctional Thinking Functioning/QOL

Test k Depression, Anxiety, Stress Scales-

Test k Test k

Stress Scale Symptom Checklist-90 Hospital Anxiety and Depression

2 2

Agoraphobic Cognitions Questionnaire

6 Quality of Life Inventory 3 Work and Social Adjustment Scale 3 2 WHO Quality of Life Questionnaire 1

Scale Profile of Mood States Outcome Study Self-Report Form

1 1 1

Anxiety Sensitivity Index Anxiety Sensitivity Profile 2 Active Coping with Trauma Scale-

Questionnaire-Modified 1 Note. k 5 No. of journal articles included in the meta-analysis that used the measure.

General Strategy 1 Attributional Style Questionnaire 1 Catastrophic Cognitions

66 Journal of Clinical Psychology, January 2009

To compare the effects of ICT to the effects of waitlist, placebo or TAU, studies were further divided based on type of control group that was utilized. Ten RCTs included a waitlist control group, seven included a placebo, and seven articles utilized a TAU control group (Five articles compared ICT to both waitlist/placebo and TAU control groups.) Placebos included both activities without active CBT features that the participants thought may be helpful and ‘‘attention placebos.’’ TAU was defined as traditional face-to-face treatment provided by a therapist.

All participants were initially analyzed together (Any Anxiety Disorder). Studies were then subdivided on the basis of specific anxiety disorders when at least two primary studies were found that had examined a specific diagnosis. Coding was possible for studies of posttraumatic stress disorder (PTSD; five primary studies), panic disorder (six primary studies), and phobia (three studies).

Statistical Analysis

For each study, Cohen’s d was calculated as the posttreatment difference between the control and treatment group means divided by the pooled SD. Formulas were adjusted so that a positive d indicated that the ICT group showed less symptomatology than did the control group on any given assessment instrument. Thus, each ES represents the posttreatment difference between the ICT and control groups on a given clinical measure (e.g., Beck Depression Inventory) in SD units.

To adjust ESs for sample size, the weight given to each study ES was inversely proportional to the conditional variance in the study (Hedges & Olkin, 1985). When a study contributed more than one individual ES to an analysis (e.g., administration of multiple measures), a mean was calculated such that each primary study contributed only a single ES, as recommended by Rosenthal (1991). Weighted mean ESs were calculated separately for each type of control-group comparison (waitlist, placebo, or TAU) by each type of clinical measure, as described earlier.

The goal of all fixed effects meta-analyses is to aggregate only the data that are thought to share a common population ES. The test statistic Q was used, as outlined by Hedges and Olkin (1985), to test the hypothesis that the observed variance in study ESs that make up a mean ES is within the range that can be reasonably expected by chance if all studies share a common population ES. Random effects models (Hedges & Olkin, 1985) were used when Q was significant (po.05), indicating heterogeneity of the variance.

The test statistic Q also was used as an analogue to the analysis of variance for ESs, as outlined by Hedges and Olkin (1985). ESs were compared between studies that enrolled participants with a Diagnostic and Statistical Manual of Mental Disorders (DSM) diagnoses and studies that enrolled subclinical, nondiagnosed participants. In addition, we compared ESs for studies with and without face-to-face clinician contact during ICT. The procedures followed here differ from an ANOVA in that an estimate of unsystematic error was incorporated in the weights of the ESs. Therefore, there was no need to use the separate error term required by an F test. Since each comparison included only two mean ESs, no contrasts were needed.

Results

Methodological Quality of the Primary Studies

Of a total possible method score of 78 for each study, only eight studies scored above 40, indicating that methodological problems persisted in many of the studies (data

Journal of Clinical Psychology DOI: 10.1002/jclp

A Meta-Analysis of Internet-Based CBT 67

available upon request). In general, studies proved to be methodologically sound with respect to (A) patient selection, (D) comparability of groups, (G) description of treatments, (J) relevant outcome measures, and (M) adequate analysis and presentation. The most prevalent limitations to the primary studies included (C) small sample sizes, (F) high rates of loss-to-follow-up, (I) unblinded patients, and (K) unblinded assessment of outcome measures.

Tests of Homogeneity of Variance

The within-group homogeneity of the ESs was tested, and the results are presented in Table 4. Thirteen of the 17 tests were not significant; thus, despite differences in study characteristics, most of the planned study groupings were statistically appropriate to combine under the assumptions of a fixed effects analysis. More specifically, when results from all outcome measures were combined, Q statistics were significant for TAU-controlled studies (po.05); heterogeneity was not detected for waitlist or placebo-controlled ICT studies. When ESs were disaggregated by type of outcome measure, Q was significant for Anxiety Measures for both waitlist- and TAU-controlled studies. The Q test for Dysfunctional Thinking measures also was significant for waitlist-controlled studies. Random effects analyses were conducted when Q was significant.

Mean ESs

Mean ESs were calculated for five types of clinical outcome measures (Depression, Anxiety, General Distress, Dysfunctional Thinking, and Functioning/Quality of Life) within each of four diagnostic groups (Any Anxiety Disorder, PTSD, Panic Disorder, and Phobia). In addition, to compare the effects of ICT to the effects of waitlist, placebo and TAU, studies with different control groups were aggregated separately. These data are shown in Table 4, which also includes the variance and confidence intervals (CI) for the mean weighted ESs.

Although classification systems for ESs facilitate communication, they are based on arbitrary distinctions between magnitudes. The importance of a ‘‘small’’ ES depends on the nature of the question. In addition, the number of studies identified in this review is small. However, for the purposes of facilitating discussion, Cohen’s (1988) classification system is used to describe the results. In his system, ESs of .20, .50, and .80 are classified as ‘‘small,’’ ‘‘moderate,’’ or ‘‘large,’’ respectively. These terms are used later to describe the results in the current study.

Following treatment, participants receiving ICT showed fewer symptoms than did the waitlist and placebo controls across all types of clinical measures and when all outcomes were analyzed together (po.05). The ESs were moderate to large in almost all cases. In addition, the benefits of ICT were equivalent or superior to TAU in all analyses. Participants showed fewer symptoms of depression following ICT compared to TAU, but this finding requires replication as it was based on only three RCTs.

ESs within specific diagnostic groups were analyzed separately for available data. Four waitlist-controlled studies examined participants with symptoms of PTSD. A moderate, significant weighted mean ES of .75 (95% CI 5 .49, 1.01) was observed in these studies across outcome measures. A large, mean ES of 1.2 (CI 5 .87, 1.58) was observed across outcomes after synthesizing the results from two waitlist-controlled studies of panic disorder. In addition, analysis of the outcomes in three waitlist-

Journal of Clinical Psychology DOI: 10.1002/jclp

68 Journal of Clinical Psychology, January 2009

Table 4

Weighted Mean Effect Sizes (ESs) by Type of Measure

ICT vs. Waitlist ICT vs. Placebo

ICT vs. TAU

All measures M ES (95% CI) .76 (.60, .92) .86 (.61, 1.11) k 10 7 7

Variance

Q

.0069 15.23

.0164 5.80

.03 ( .35, .41) .0377

Anxiety measures M ES (95% CI) k 10 6 7

Variance

Q

.77 (.56, .98) .0115

.88 (.70, 1.31) .0192

.00 ( .38, .38) .0376

Depression measures M ES (95% CI)

18.08 .89 (.69, 1.08)

6.83

13.46

.57 (.22, .92) .0325 .0323

Dysfunctional Thinking M ES (95% CI) 1.14 (.43, 1.85) .70 (.26, 1.15) .25 ( .02, .53)

k434

.49 (.14, .84) k843

Variance

Q

General Distress M ES (95% CI) k42– Variance .0151 .0708 – Q 2.82 .29 –

Functioning/QOL M ES (95% CI)

.0100 6.02

1.67 0.95 .58 –

.48 (.24, .72)

Variance

Q

Q 4.20 .27 4.99

Note. k 5 no. of studies in the analysis; ICT 5 Internet- or computer-based treatment; TAU 5 treatment as usual. Value differs from 0, po.05. Where Q is significant, results are reported from random effects analyses. Mean ESs are presented for groups with two or more studies; dashes indicate insufficient data available.

controlled studies that examined participants with phobia revealed a moderate, mean ES of .66 (CI 5 .30, 1.02).

Studies that utilized a placebo control were aggregated for panic disorder and phobia. Three studies examined participants with panic disorder and, together, revealed a large, mean ES of .93 (CI 5 .49, 1.38). For phobia, the mean ES for three studies was .81 across outcome measures (CI 5 .44, 1.19).

Among TAU-controlled studies, data were available to analyze three studies of panic disorder. There was no difference between ICT and TAU outcomes in these studies (ES 5 .30, CI 5 .001, .61). There were insufficient data available to examine other results for specific diagnostic groups.

Effects of Participant or Treatment Characteristics

Among the 10 waitlist-controlled ICT studies, four studied participants who met full DSM diagnostic criteria for an anxiety disorder while six studied participants who met partial criteria or were below diagnostic thresholds. There was no significant

.1314 17.42

.0513 .28

.0196 1.25

.57 (.23, .91) k334

.71 (.29, .1.14) Variance .0307 .0473 .0261

13.12

.02 ( .33, .30)

Journal of Clinical Psychology DOI: 10.1002/jclp

A Meta-Analysis of Internet-Based CBT 69

difference between the mean weighted ES for the diagnosed groups (ES5.93; CI 5 .66, 1.20) and subclinical groups across outcome measures (ES 5 .66; CI 5 .45, .86).

Only one placebo- and one TAU-controlled study used subclinical samples, so these analyses were not repeated for these groups.

To examine the impact of clinician contact on treatment outcomes, waitlist- controlled ICT studies were divided into two groups based on whether ICT participants had face-to-face clinician contact (k 5 3) or no clinician contact except via e-mail, in some cases (k 5 7). There was no significant difference in ESs for those with clinician contact (ES 5 .91; CI 5 .61, 1.21) and those without clinician contact (ES 5 .70; CI 5 .50, .89). Similarly, among placebo-controlled studies, there was no difference between ICT studies with (ES5.91; CI5.61, 1.21; k53) and without (ES5.85; CI551, 1.18; k54) clinician contact as a part of treatment. The same nonsignificant pattern emerged when TAU-controlled studies with (ES 5 .04; CI 5 .22, .31; k 5 5) and without (ES 5 .26; CI 5 .17, .68; k 5 2) clinician contact were compared.

Publication Bias

To evaluate the impact of publication bias on the results of the meta-analysis, a funnel plot was constructed to display the ICT ESs against their respective sample sizes. The resulting scatterplot should resemble an inverted funnel since the precision of an ES estimate increases as the sample size increases. Publication bias is suspected when a ‘‘hole’’ in the area representing smaller ESs creates a degree of asymmetry in the funnel plot.

Since both reporting bias and retrieval bias are of concern (Greenhouse & Iyengar, 1994), all individual ESs from all studies were plotted (Fig. 1). No publication bias was detected.

Figure 1. Funnel plot of all ICT effect sizes by the size of the sample. Journal of Clinical Psychology DOI: 10.1002/jclp

70 Journal of Clinical Psychology, January 2009

Discussion

The results of this meta-analysis provide preliminary support for the use of Internet- and computer-based CBT for the treatment of anxiety. The benefits of ICT were superior to waitlist or placebo assignment, although the number of placebo- controlled studies was small (n 5 7), which limits conclusions. The clinical effects of ICT were equal to traditional therapist-delivered treatment in the few studies available for analyses (n57). One of the primary conclusions of this systematic review is that additional high-quality research is needed in this field. Placebo- controlled studies were rare, and many of the studies suffered from small sample sizes and high dropout rates.

In addition, conclusions are limited by the fact that it was not possible to conduct these comparisons by diagnostic group in most cases. Where sufficient data were available, results provided tentative support for ICT. For participants with panic disorder, ICT may be superior to no treatment (i.e., waitlist control) and to placebo assignment; the benefits of ICT also were equivalent to TAU in the few studies available. For participants with a phobia, posttreatment symptoms were lower following ICT compared to both waitlist and placebo assignments. For participants with putative PTSD, there was preliminary support for ICT compared to no treatment. There was one well-designed placebo-controlled study for PTSD that generally supported ICT (Litz et al., 2007), but it was a small proof-of-concept trial. Therefore, there are limited data to support the use of ICTs for PTSD at this time. Sufficient data were not available to compare ICT to controls for other diagnostic groups. Large placebo-controlled trials that examine participants with well-defined mental disorders are needed to confirm the results of this meta-analysis.

Results demonstrating that the effects of ICT were equivalent or superior to TAU were unexpected. Failure to detect superior effects of TAU did not appear to be related to a problem with power, as ICT showed slightly higher ESs in the majority of the nonsignificant analyses. Since six of the seven TAU-controlled studies utilized CBT approaches in the TAU conditions, it is unlikely that the equivalence of ICT is due to poor TAU treatment approaches. However, while empirically supported CBT techniques were employed, most studies did not appear to utilize empirically supported manualized treatments. Thus, the efficacy of specific TAU treatments is unknown in some cases.

It is possible that differences in treatment ‘‘dose’’ (e.g., number of ‘‘sessions’’) impacted comparisons of ICT and TAU. The use of TAU control groups implies that efforts were made in the RCTs to provide equivalent therapist-delivered ‘‘doses’’ of the same treatment; many authors were explicit about such attempts, but at least one trial was not properly controlled. In addition, it may be difficult in practice for a therapist to provide treatment in a format that properly parallels computerized modules. Therefore, ICT’s superior treatment of depression may reflect a dose–response effect. Alternatively, a structured, computer presentation of treatment techniques may be more efficacious than a presentation by a therapist who may use clinical judgment to modify empirically supported procedures. However, it is unclear why ICT’s superior treatment outcomes might apply only to depression; since only three RCTs were included in this analysis, replication of this finding in future RCTs would be helpful.

The results also showed that there was no difference in ESs between participants with anxiety disorders and participants with subclinical symptoms. These preliminary results suggest that ICT may be useful in stepped-care models of

Journal of Clinical Psychology DOI: 10.1002/jclp

A Meta-Analysis of Internet-Based CBT 71

treatment. Stepped-care models are based on the assumption that the same treatment approaches are not indicated for all clients (Bower & Gilbody, 2005). Some clients may require only psychoeducation while others might require careful, face-to-face risk management by a therapist. Multiple options for incorporating ICT into stepped-care treatment approaches exist, including use of ICT for pure ‘‘self-help’’ delivery (i.e., no therapist contact), minimal therapist contact, or treatment as usual with ICT used as an adjunct to care.

In addition, there was no difference in treatment outcome between participants who received face-to-face clinician contact compared to those with no clinician contact. Although minimal therapist contact may not significantly improve treatment outcome, there may be other reasons for maintaining contact with patients during ICT. For example, clinician contact may improve treatment compliance, and certainly patients at risk for self-harm must be followed by a clinician, regardless of the use of ICT. It also is possible that treatment outcome may differ by ‘‘dose’’ of clinician contact. It was not possible to reliably code amount of clinician contact as a continuous variable in the meta-analysis due to the variability in study reporting formats. Future studies should further evaluate the impact of clinician contact on treatment outcome.

Although ICT may be efficacious, this meta-analysis does not address the ethical, legal, and philosophical questions related to the use of ICT. ICT does not yet have general acceptance in many countries. Questions are still being raised regarding privacy, confidentiality, risk management issues, liability, and patient contra- indications. These areas of concern are contrasted with the potential benefits that this methodology offers. Although a thorough review of these issues is beyond the scope of this article (for reviews, see Fenichel et al., 2002; Hsiung, 2001), recent concerns have been raised that specific subgroups of the population may experience elevated fears about treatment stigma (Hoge et al., 2004). Military personnel, police officers, firefighters, pilots, and others may fear real and perceived consequences to pursuing psychological services (Carter et al., 2005; Hoge et al., 2004). Significant numbers of individuals may prefer ICT to walking into a counseling center, sitting in a waiting room, and talking with a therapist. The assumption that a traditional therapist is generally preferred over ICT may be inaccurate for significant subgroups of the population, but this possibility requires additional study.

ICT also may be useful as a research tool to explore the essential components of CBT. ICT allows researchers to add, modify, or delete specific treatment interventions while truly leaving all other treatment components unchanged for a comparison group. For example, Schneider, Mataix-Cols, Marks, and Bachofen (2005) recently examined the differences between Internet-guided treatment of panic and phobia with or without exposure instructions. Thus, ICT may provide a tool to help refine CBT theory and practice.

Additional research is needed on a variety of topics related to ICT. Many of the studies reviewed here used small sample sizes, and there were insufficient data to compare the effects of ICT to placebo or TAU in some analyses. In addition, participants in the primary studies were generally well educated, and all could presumably use a computer. It is unknown how well these results generalize to the general population. Further research is needed on the relationship between the effects of ICT and a client’s age, education, computer familiarity, and socioeconomic status to inform potential ICT treatment procedures. Demographic variables also may be directly or indirectly associated with treatment compliance through a variety

Journal of Clinical Psychology DOI: 10.1002/jclp

72 Journal of Clinical Psychology, January 2009

of mechanisms such as cultural preferences, language barriers, literacy, or access to technology. Population-based studies addressing access to, and use of, various forms of technology will be helpful in guiding the development of tools that are likely to be utilized by target populations.

There are several limitations of the current study. As noted earlier, the conclusions from this study are limited by the methodological problems observed in the primary studies. Large, well-designed, randomized, placebo-controlled trials of homogenous clinical samples are needed. In addition, every meta-analysis struggles with the conflict between the goal of data synthesis versus the problem of between-study variability. The current meta-analysis suffers from variability in participant characteristics, treatment approaches, clinical scales, and other methodological differences between studies. Grouping tests by clinical domain reduces only some of the variability because many measures tap multiple domains, and each measure may assess different aspects of a given domain. Development of a standardized battery of measures that can be used across Internet- and computer-based treatment studies may be helpful.

Despite the apparent differences between studies, however, there was statistical support for the decision to combine these studies into the various groups in the majority of the analyses. Tests of the homogeneity of the variance suggested that the mean ESs do estimate a single population ES in most cases. Where this was not true, a random effects approach was used, as is typical for cases where heterogeneity is detected (Hedges & Olkin, 1985).

The importance of developing cost-effective approaches to delivering empirically supported treatments for anxiety that reduce barriers to care is likely to grow as awareness of mental health issues continues to improve. The results of this study provide preliminary evidence that support additional research on the benefits of Internet- and computer-based CBT. As society becomes increasingly comfortable with, and reliant upon, the use of computers and the Internet for more of their routine business and healthcare, opportunities to apply ICT approaches are likely to continue to grow.

Note: References marked with an asterisk indicate studies included in the meta- analysis.

References:

Reger, M. A., & Gahm, G. A. (2009). A meta-analysis of the effects

           of Internet and computer-based cognitive-behaviour treatments

           for anxiety. Journal of Clinical Psychology, 65(1), 5375. 

           doi:10.1002/jclp.20536