This article was downloaded by: [University of Vermont] On: 09 January 2014, At: 08:40 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Journal of Clinical Child & Adolescent Psychology Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/hcap20

Technology-Enhanced Program for Child Disruptive Behavior Disorders: Development and Pilot Randomized Control Trial Deborah J. Jones a , Rex Forehand b , Jessica Cuellar a , Justin Parent b , Amanda Honeycutt c , Olga Khavjou c , Michelle Gonzalez a , Margaret Anton a & Greg A. Newey d a Department of Psychology , University of North Carolina at Chapel Hill b Department of Psychology , University of Vermont c RTI International d Research Technology Solutions Published online: 07 Aug 2013.

To cite this article: Deborah J. Jones , Rex Forehand , Jessica Cuellar , Justin Parent , Amanda Honeycutt , Olga Khavjou , Michelle Gonzalez , Margaret Anton & Greg A. Newey (2014) Technology-Enhanced Program for Child Disruptive Behavior Disorders: Development and Pilot Randomized Control Trial, Journal of Clinical Child & Adolescent Psychology, 43:1, 88-101, DOI: 10.1080/15374416.2013.822308

To link to this article: http://dx.doi.org/10.1080/15374416.2013.822308

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/loi/hcap20

http://www.tandfonline.com/action/showCitFormats?doi=10.1080/15374416.2013.822308

http://dx.doi.org/10.1080/15374416.2013.822308

http://www.tandfonline.com/page/terms-and-conditions

Technology-Enhanced Program for Child Disruptive Behavior Disorders: Development and Pilot Randomized Control Trial

Deborah J. Jones

Department of Psychology, University of North Carolina at Chapel Hill

Rex Forehand

Department of Psychology, University of Vermont

Jessica Cuellar

Department of Psychology, University of North Carolina at Chapel Hill

Justin Parent

Department of Psychology, University of Vermont

Amanda Honeycutt and Olga Khavjou

RTI International

Michelle Gonzalez and Margaret Anton

Department of Psychology, University of North Carolina at Chapel Hill

Greg A. Newey

Research Technology Solutions

Early onset disruptive behavior disorders are overrepresented in low-income families; yet these families are less likely to engage in behavioral parent training (BPT) than other groups. This project aimed to develop and pilot test a technology-enhanced version of one evidence-based BPT program, Helping the Noncompliant Child (HNC). The aim was to increase engagement of low-income families and, in turn, child behavior outcomes, with potential cost-savings associated with greater treatment efficiency. Low-income families of 3- to 8-year-old children with clinically significant disruptive behaviors were randomized to and completed standard HNC (n ¼ 8) or Technology- Enhanced HNC (TE-HNC; n ¼ 7). On average, caregivers were 37 years old; 87% were female, and 80% worked at least part-time. More than half (53%) of the youth were boys; the average age of the sample was 5.67 years. All families received the standard HNC program; however, TE-HNC also included the following smartphone enhancements: (a) skills video series, (b) brief daily surveys, (c) text message reminders, (d) video

Funding for this project was provided by NIMH 1R34MH082956 (ClinicalTrials.gov Identifier: NCT01367847). We acknowledge the following

individuals who were instrumental in our pilot and feasibility study: Joel Sherrill, Program Chief, Child and Adolescent Psychosocial Intervention

Program, for his guidance on this project; Our clinician advisory panel, which included April Harris-Britt, Ph.D., AHB Psychological Associates;

Kathryn J. Smith, MSW, Center for Child and Family Health; Jennifer Youngstrom, Ph.D., UNC Child and Family Clinic; Sarah Stearns,

Ph.D., Dartmouth-Hitchcock Psychiatry Associates; and Timothy Verduin, Ph.D., NYU Child Study Center, for their review and input on all

aspects of the TE-HNC program; finally, we thank the therapists and families who participated in this project for their time and contributions. Correspondence should be addressed to Deborah J. Jones, Department of Psychology, University of North Carolina at Chapel Hill, Chapel Hill,

NC 27599. E-mail: [email protected]

Journal of Clinical Child & Adolescent Psychology, 43(1), 88–101, 2014

ISSN: 1537-4416 print=1537-4424 online

DOI: 10.1080/15374416.2013.822308

D ow

nl oa

de d

by [

U ni

ve rs

it y

of V

er m

on t]

a t

08 :4

0 09

J an

ua ry

2 01

recording home practice, and (e) midweek video calls. TE-HNC yielded larger effect sizes than HNC for all engagement outcomes. Both groups yielded clinically significant improvements in disruptive behavior; however, findings suggest that the greater program engagement associated with TE-HNC boosted child treatment outcome. Further evidence for the boost afforded by the technology is revealed in family responses to postassessment interviews. Finally, cost analysis suggests that TE-HNC families also required fewer sessions than HNC families to complete the program, an efficiency that did not compromise family satisfaction. TE-HNC shows promise as an innovative approach to engaging low-income families in BPT with potential cost-savings and, therefore, merits further investigation on a larger scale.

Disruptive behaviors (e.g., noncompliance, defiance, aggression) are among the most common reasons children are referred to mental health care, with the 12-month prevalence of disruptive behavior disorders (DBDs) worldwide second only to anxiety disorders in childhood and adolescence (see Merikangas, Nakamura, & Kessler, 2009, for a review). Although several models have evolved to explain the etiology of the early onset of DBDs in particular, at its core each model highlights the central role of early parenting and parent–child inter- actions (e.g., bridging model, Shaw & Bell, 1993; cascade model, Dodge et al., 2009; early onset type, Moffitt et al., 2008). Both theoretical and empirical work suggest that there is an escalation of coercive processes characterized by parents initially trying more controlling parenting techniques in response to the child’s disruptive behavior; however, when the child’s noncompliance and opposi- tionality escalate in response, parents of youth vulner- able to DBDs tend to acquiesce, unintentionally reinforcing and, subsequently, exacerbating the disrup- tive behavior (see McMahon & Forehand, 2003, for a review). With the aim of early intervention, the field of behavioral parent training (BPT) evolved via the development of programs targeting the coercive cycle and, in turn, decreasing vulnerability for protracted disturbances of behavior in adolescence and adulthood (see Forehand, Jones, & Parent, 2013; Garland, Hawley, Brookman-Frazee, & Hurlburt, 2008; Jones et al., 2013; Reitman & McMahon, 2012, for reviews).

BPT programs have a rich history and robust evidence base (see Chorpita et al., 2011, for a representative review); however, consistent with children’s mental health more broadly, ‘‘parenting interventions in general are less successful at engaging the most distressed and disadvantaged families’’ (Gardner et al., 2009, p. 545). Family stress theory highlights the indirect impact of financial strain on children through parental stress and associated compromises in parenting (see Conger & Donnellan, 2007, for a review). Accordingly, it is not sur- prising that low-income families are more vulnerable to the coercive cycle of parent–child interaction implicated in the development and exacerbation of DBDs and, in turn, more likely to have a child with an early onset DBD than relatively higher income families (see Dekovic

et al., 2011; Jones et al., 2013; Lundahl, Risser, & Lovejoy, 2006; Piquero, Farrington, Welsh, Tremblay, & Jennings, 2009, for reviews). Data suggest that if we successfully engage low-income families in BPT services, they benefit as much, if not more, than relatively higher income families, particularly at posttreatment and when the problem behaviors are in the clinical range (see Dekovic et al., 2011; Leijten, Raaijmakers, de Castro, & Matthys, 2013; Reyno & McGrath, 2006, for reviews). Financial strain and associated difficulties (e.g., poor mental=physical health, un=underemployment, lack of health insurance=underinsured), however, decrease the probability that low-income families will engage at a level necessary for BPT to be efficacious (e.g., 12 to 28 session hours, midweek telephone check-ins, daily home practice; see Eyberg, Nelson, & Boggs, 2008; McMahon & Fore- hand, 2003; Reyno & McGrath, 2006; Thomas & Zim- mer-Gembeck, 2007). Accordingly, a number of approaches have been tested to improve the engagement of families in BPT, including home-based and group- focused programs, as well as monetary incentives for attendance (Dumas, Begle, French, & Pearl, 2010; Gross et al., 2011; MacKenzie, Fite, & Bates, 2004; McGilloway et al., 2012). These approaches, however, have yielded similarly or more disappointing engagement rates than traditional clinic-based, individual family- focused BPT programs (also see Jones et al., 2013; Lundahl et al., 2006, for reviews), highlighting the critical public health importance of testing innovative approaches to engage low-income families.

As highlighted elsewhere (Aguilera & Muench, 2012; Kazdin & Blasé, 2011), it is time for a paradigm shift in the delivery of clinical services, and technology is at the forefront of this shift. Technology, albeit in its most basic forms (i.e., videotape modeling), is firmly rooted in the origins and history of BPT (e.g., Flanagan, Adams, & Forehand, 1979; Nay, 1976; O’Dell et al., 1982); how- ever, the potential exists for technology to advance the field further (see Jones et al., 2013, for a review). Of greatest relevance to our purposes, there is a relatively untapped potential for technology to better situate BPT programs at the forefront of low-income families’ daily lives, particularly between sessions and inspite of both acute and chronic family stressors associated with

TECHNOLOGY-ENHANCED BPT 89

D ow

nl oa

de d

by [

U ni

ve rs

it y

of V

er m

on t]

a t

08 :4

0 09

J an

ua ry

2 01

financial strain. Although some may worry that the uti- lization of technology to increase engagement may widen the digital divide, low-income homes are actually the most likely to forego landline service in favor of mobile phones, with rates of smartphone ownership in parti- cular on the rise due to the growing accessibility of a range of affordable smartphone options and providers, as well as the range of accessible platforms afforded by one handheld and portable device (e.g., Internet, video calls and conferencing, videotaping, e-mail, chat rooms, and social networking; see Anderson & Subramanyam, 2011; Davies, 2011; Snider, 2011, for reviews). It is precisely this accessibility, portability, and varied func- tionality of smartphones that have led some to refer to them as ‘‘therapeutic gold,’’ highlighting the relatively untapped potential to utilize smartphones to better con- nect clients with evidence-based services (Aguilera & Muench, 2012, p. 70). Accordingly, this study describes a line of research in which the authors developed and pilot tested the feasibility and outcomes of the Technology- Enhanced Helping the Noncompliant Child (TE-HNC) program, a smartphone-enhanced version of one evidence-based BPT program, Helping the Noncompliant Child (HNC; McMahon & Forehand, 2003).

HNC is one example of a clinic-based, individual family-focused, BPT program that evolved from the Hanf Model of BPT, a BPT framework that is synony- mous with a collection of efforts to promote improved behavior and healthier family functioning among young children with behavior problems (see McMahon & Forehand, 2003; Reitman & McMahon, 2012, for reviews). The HNC program in particular, however, may be exceptionally well suited to serve as a comparison group when evaluating the effects of smartphone enhancement with low-income families for two primary reasons. First, HNC is a criterion-driven program; that is, progression to each new skill is dependent upon meeting criterion (i.e., mastery) on the prior skill. As highlighted elsewhere, BPT programs requiring mastery before advancement are associated with better outcomes for low-income caregivers (Rogers, Forehand, Griest, Wells, & McMahon, 1981; also see Reyno & McGrath, 2006, for a review). Second, HNC materials are written at a sixth-grade reading level, making it ideal for low-income families with potential literacy issues (see McMahon & Forehand, 2003, for a review).

Building upon the relevance of HNC in particular for low-income families, we expected that the smartphone enhancements to the HNC program (i.e., skills video series, daily surveys of skill practice and progress, video recording home practice for review and feedback, text message reminders and alarms regarding sessions, calls, practice, and midweek video call check-ins) would pro- vide increased opportunities for feedback, support, and skill modeling to families both between and within

HNC sessions, increasing the likelihood that families would remain engaged in the HNC program (i.e., attend weekly sessions and midweek phone calls) and practice the HNC skills outside of session (i.e., daily home prac- tice; see Jones et al., 2013; Williams, Lynch, & Glasgow, 2007, for discussion of how technology has the potential to increase intrinsic motivation for behavior change). With regard to daily home practice in particular, we were interested in the extent to which the smartphone enhancements had the potential to increase not only the participating caregiver’s practice but nonparticipat- ing caregiver practice as well. Regardless of income, only one caregiver typically participates in BPT services (Cowan, Cowan, & Berry, 2011; also see McMahon & Forehand, 2003, for a review); however, this trend is even more pronounced and, in turn, may have more pronounced implications for treatment outcome in low-income families. That is, low-income families who are navigating challenges with balancing child care, shift or hourly employment, and transportation may simply not be able to afford or manage the participation of more than one caregiver, a circumstance that increases the potential for inconsistency between parents and decreases the likelihood that the program will lead to improvements in child behavior (see McMahon & Forehand, 2003, for a review). As such, we hypothesized that the smartphone enhancements may facilitate coparent involvement in home practice by providing participating caregivers a mechanism (e.g., skills videos series) and support (e.g., text message reminders and smartphone-assessment items regarding the importance of coparent involvement and practice) for involving the nonparticipating coparent in the BPT program.

HNC is an established, evidence-based BPT program; therefore, we expected both groups to evidence clinically significant improvement in child disruptive behaviors. However, we hypothesized that the smartphone enhance- ments would boost HNC’s impact on child behavior via enhanced engagement and skill generalization, as well as increased opportunity for skill modeling, practice, and feedback. Finally, by collecting both start-up and implementation costs, the study design allowed us to assess whether enhanced engagement and skill generalization resulted in greater efficiency (i.e., cost- effectiveness) of services for TE-HNC without compro- mising family satisfaction with the program.

METHOD

Overview

The line of research presented here has proceeded con- sistent with the recently proposed Web-Based Treatment Research Cycle, which aims to move the field beyond the

90 JONES ET AL.

D ow

nl oa

de d

by [

U ni

ve rs

it y

of V

er m

on t]

a t

08 :4

0 09

J an

ua ry

2 01

development of novel, mobile mental health intervention approaches (Stage 1) to theoretically informed RCTs (Stage 2), which are largely lacking in the literature on mobile and other technology-enhanced interventions (see Enock & McNally, 2013, for a review).

Participants

Families were included in the project if they met criteria for ‘‘low income’’ (i.e., adjusted gross income did not exceed 150% of the federal poverty limit, which takes into account both income and number of residents in the home), they had a child in the 3- to 8-year-old age range (age range for which HNC was developed and tested), and the child exhibited disruptive behaviors in the clinical range as evidenced by meeting or exceeding clinical cutoffs on the caregiver-report of the Eyberg Child Behavior Inventory (ECBI; Eyberg & Pincus, 1999) Severity or Intensity Subscales.

Exclusion criteria were (a) child developmental or physical disability that precluded use of HNC skills; (b) caregiver current diagnosis of substance abuse= dependence, mood, or psychotic disorder; and=or (c) family involvement with Department of Social Services related to abuse=neglect.

Smartphone ownership was not an inclusion= exclusion criterion as the project provided identical smartphone models to all participating families in order to ensure that the human interface (i.e., design, usability, interactivity) of the smartphone enhancements were identical for all families; however, the generalizability of this work rests on the premise that the technology is accessible and available to low-income families.

Forty-seven of the 48 families who completed a clinic- based eligibility interview (see Figure 1) owned a mobile phone and, of those, 30% (n ¼ 14) owned a smartphone.

Procedure

Low-income families in north central North Carolina (NC) were recruited via (a) advertisements targeting areas, work places, and retail outlets with an overrepre- sentation of low-income parents (48% of pilot families); (b) healthcare, social service, and other agencies that serve low-income families (28% of pilot families); (c) local schools (19% of pilot families); and (d) word-of-mouth (5% of pilot families). Of note, NC has the 10th-worst child poverty rate in the country, with 46% of children living in low-income families (National Center for Children in Poverty, 2010). As such, this area afforded a relevant recruitment infrastructure for our pilot research.

A brief (20-min) phone screen was conducted to determine interest and initial eligibility, which was then confirmed by a clinic-based interview, which also included consent and a more extensive caregiver assess- ment battery. During treatment, therapists collected data on engagement and skill generalization, as well as costs (e.g., number of sessions). Within 2 weeks of treatment termination, the caregiver assessment battery was read- ministered (postassessment). All assessments and ses- sions were completed at a community-based university training clinic in north central NC. Families were typi- cally seen in the late afternoon to evening (school-aged children) or morning to early afternoon (preschool chil- dren), with appointments chosen by families to enhance convenience. Caregivers were compensated $50 per

FIGURE 1 Participant-flow from baseline to trial completion.

TECHNOLOGY-ENHANCED BPT 91

D ow

nl oa

de d

by [

U ni

ve rs

it y

of V

er m

on t]

a t

08 :4

0 09

J an

ua ry

2 01

assessment and TE-HNC children received a ‘‘safe phone return bonus’’ ($100). All smartphones were returned.

Shared Intervention Features

All families received the standard, two-phase HNC pro- gram. HNC (McMahon & Forehand, 2003). Following an orientation session, caregiver–child dyads start Phase I, Differential Attention, in which caregivers learn to increase the frequency and range of social attention to the child and to reduce the frequency of competing verbal behavior. A primary goal is to reduce the coercive cycle of parent–child interaction by (re)establishing a positive and mutually reinforcing parent–child relationship. In the context of ‘‘Child’s Game’’ (i.e., child-directed play), the caregiver is taught to (a) increase the frequency and range of positive attention; (b) eliminate instructions, questions, and criticisms; and (c) ignore minor inappro- priate behavior. Parents are instructed to practice Child’s Game for at least 15 min per day and told that coparents also should practice. Parents progress to Phase II when the criteria for Phase I skills are met (McMahon & Forehand, 2003).

In Phase II, Compliance Training, caregivers are taught the difference between unclear and clear instruc- tions; to give the ‘‘Clear Instruction’’ sequence; and to use a nonphysical discipline procedure, ‘‘Time-Out,’’ for occasions of noncompliance and other inappropriate behavior that cannot be ignored. Phase II skills are taught within the context of ‘‘Parent’s Game’’ (i.e., parent- directed activities, such as a cleanup task), although care- givers are instructed to continue to practice Child’s Game at home to maintain mastery of Phase I skills. Program completion is based on caregivers meeting criteria for all Phase I and II skills, which requires an average of eight to 12 sessions (McMahon & Forehand, 2003).

Development of the TE-HNC Program

Prior health services research, as well as research on BPT in particular, suggests that some level of therapist involve- ment may be optimal for the most distressed and disadvan- taged families (e.g., Webster-Stratton, 1990, 1992; Webster-Stratton, Kolpakoff, & Hollinsworth, 1988; also see Mohr, Cujpers, & Lehman, 2011; Tate & Zabinski, 2004, for reviews). Accordingly, TE-HNC includes the HNC program, as well as smartphone components that were developed to enhance, rather than replace, clinic-based, therapist-guided services. Smartphone enhancements were developed via an interdisciplinary partnership including (a) researchers with expertise in BPT with underserved families; (b) a clinician advisory panel (20% male, 20% ethnic minority) that practices at least one BPT program; (c) an industry partner with experience developing sustainable technological

applications; and (d) health economists with expertise in health care efficiency, efficacy, and value.

Specifically, iterative feedback and modifications led to these smartphone enhancements (see Jones, Forehand, McKee, Cuellar, & Kincaid, 2010, for a review): (a) a 3-min skills video for each of the HNC skills, including psychoeducation, as well as modeling of the skill by parent–child dyads; (b) daily surveys of skill practice and progress that are used to guide mid- week calls and weekly sessions (e.g., problem solving more suitable home practice times if a parent indicates a failure to practice on a daily survey); (c) midweek video calls during which therapists reinforce caregivers for progress and problem solve obstacles to practice (e.g., helping the family to pick a time-out location in the home); (d) weekly videotaped home practice, which pro- vided a ‘‘window’’ for therapists to use during the session to provide feedback regarding skill development; and (e) text reminders regarding the relevance of home practice, the midweek call, and session attendance, as well as rein- forcing messages regarding progress. As such, TE-HNC capitalizes on the capacity for smartphones to push HNC content to the caregiver rather than relying on the caregiver to access the content, a proven strategy with other low-income clients (Aguilera & Muench, 2012; Aquilera & Munoz, 2011).

Therapist Training and Supervision

Therapists were M.A.-level graduate students. Training included didactic presentations and practice, one prac- tice case per therapist, and clinical emergency protocols. Therapists participated in weekly supervision, which included reviewing and discussing videotaped sessions. Therapist fidelity to program materials for both pro- grams was coded using the following procedures: (a) The critical material (e.g., rationale for program, expla- nation of a skill, practice of the skill with the child) to be covered in each session was delineated; (b) naı̈ve, trained coders watched video recordings of sessions; and (c) coders indicated whether each of the critical points was covered in the session. More than 50% of ses- sions were coded for fidelity by two coders, who achieved more than 90% reliability, yielding an average fidelity rating 90%. Finally, therapists treated families in both arms of study.

Measures

Engagement & skill generalization. Given the pilot nature of the study, engagement and skill generalization were assessed utilizing both between- and within-group measures. The between-group measures of engagement were operationalized as the following: (a) session attendance (i.e., family showed for scheduled weekly

92 JONES ET AL.

D ow

nl oa

de d

by [

U ni

ve rs

it y

of V

er m

on t]

a t

08 :4

0 09

J an

ua ry

2 01

appointment) and (b) midweek call availability (i.e., fam- ily available for the scheduled midweek call). In addition, the between-group measure of skill generalization was assessed utilizing a measure of caregiver report to the therapist regarding whether she or he completed their daily skill practice, as well as whether the caregiver completed assigned worksheets.

Several measures of engagement and skill generaliza- tion within the TE-HNC group were assessed. Caregivers randomized to the TE-HNC program were asked to do the following utilizing their smartphones: (a) complete brief daily surveys regarding their skill practice and (b) videotape at least one home practice. As noted earlier, five versions of the daily surveys were created, one version per HNC skill (e.g., caregivers were asked to complete the Attends Survey each day between sessions until they met behavioral criteria for Attends). Daily smartphone sur- veys were intended to be brief, with the surveys increasing slightly in length as the family progressed through the program and skills (e.g., Rewards Survey included ques- tions about Attends plus questions about Rewards). Each item was also intended to be brief, with one question (e.g., ‘‘Did you practice Child’s Game today’’) and the associa- ted response options (e.g., ‘‘Yes’’ or ‘‘No’’) appearing on the smartphone screen at a time. Caregiver response to each item determined the sequence of items (e.g., if the caregiver indicated that she practiced Child’s Game, the next item would ask, ‘‘How long did you practice Child’s Game?’’ with response options ranging less than 15 min- utes, 15 minutes, or more than 15 minutes). Responses were automatically uploaded to the therapist’s secure computer via identification code and were utilized by therapists to guide the midweek check-in and subsequent session.

TE-HNC families were also asked to utilize their smartphones and tripods to video record at least one home practice (i.e., Child’s Game) during the week to share with the therapist. During the session, therapists allowed caregivers to suggest parts of the video-recorded home practice that they thought went particularly well or parts where they may have had more difficulty, and therapists provided reinforcing and corrective feedback accordingly. If caregivers video recorded more than one home practice, they picked the one with which they had the most difficulty or the most questions.

Finally, we were interested in the extent to which smartphone enhancements facilitated nonparticipating coparent involvement in home practice. TE-HNC famil- ies were asked the following open-ended question at postassessment only: ‘‘Did you think sharing the parent- ing skills videos with your coparent was useful in terms of letting him=her know what you were learning and get- ting him=her to practice the skills too?’’ At the postas- sessment, HNC families were provided an overview of the smartphone enhancements and asked, ‘‘Now that

you know all of the things the families in the other group used the smartphones to do, do you think having copar- ents watch the skills videos everyday would have been helpful with getting coparents like yours involved in the parenting program?’’

Child disruptive behavior. Intensity and Problem subscales on the 36-item ECBI (Eyberg & Pincus, 1999) served as the dependent measure of behavior change due to the availability of normative data sensitive to age (2 to 16 years old; Burns & Patterson, 1991; Burns, Patterson, Nussbaum, & Parker, 1991) and established psychometrics with low-income samples (e.g., Fernandez et al., 2011).

For each item, caregivers rate the intensity of the behavior (0 ¼ never to 7 ¼ always) and whether each behavior is a problem (0 ¼ no; 1 ¼ yes). Clinically signifi- cant symptoms are defined by scores more than 2 stan- dard deviations above the normed mean for Intensity (clinical cutoff ¼ 127) and=or Problem (clinical cutoff ¼ 11) Scales. Alphas in this study were 0.86 (Inten- sity) and 0.72 (Problem).

Program costs and consumer satisfaction. Program start-up and implementation costs were collected and analyzed separately. Start-up costs include nonlabor costs for therapist manuals, toys, and handouts for both groups and, for the TE-HNC group only, smartphone- related costs (i.e., purchase of smartphones, service plans, enhancements, etc.). Implementation costs include the value of labor resources required to deliver the pro- gram (including value of therapist time). Therapists reported time spent both in and out of face-to-face ses- sions (including time spent on phone calls, review of ses- sion notes, supervision, and other activities). Therapist time was valued using 2010 median national hourly wage rates for M.A.-level mental health counselors from the Bureau of Labor Statistics.

To ensure that any cost savings of the TE-HNC program (e.g., through reductions in the number of intervention sessions) were not obtained at the expense of family satisfaction, HNC and TE-HNC families completed the HNC Consumer Satisfaction Scale (McMahon & Forehand, 2003).

The Consumer Satisfaction Scale is a 42-item measure that assesses caregiver satisfaction with the overall HNC program, the difficulty and usefulness of the HNC pro- gram format and skills, and the efficacy of the therapist. A total satisfaction score was calculated (a ¼ 0.82).

Data Analytic Approach

Enrollment and baseline characteristics of the sample are examined. Then, owing to the pilot nature of the study,

TECHNOLOGY-ENHANCED BPT 93

D ow

nl oa

de d

by [

U ni

ve rs

it y

of V

er m

on t]

a t

08 :4

0 09

J an

ua ry

2 01

pre–post analysis of primary study variables (engagement, skill generalization, and child behavior outcomes) are conducted with only those families who completed both the pre- and postassessment. First, between-group effect sizes (0.20–0.49 small, 0.50–0.79 medium, and �.80 large) were used to compare TE-HNC to HNC on engagement and skill generalization to the home (Cohen, 1988). Whereas significance testing conveys the likelihood that study results differ from chance expectations, effect-size calculations convey the relative magnitude of the experi- mental effect and, in turn, provide the opportunity to compare the magnitude of treatment effects within and across studies (see Thalheimer & Cook, 2002, for a review). Consistent with Cohen’s d and more recent mod- ifications (D’Amico, Neilands, & Zambarano, 2001), the difference between group means (e.g., Engagement MeanTE-HNC – EngagementMeanHNC) was divided by the average of each mean’s standard deviation (e.g., (SDTE-HNC þ SDHNC)=2). In addition, as one-per-week video recordings and daily surveys of Child’s Game home practice were part of the TE-HNC program, we also utilized within-group statistics only to examine whether TE-HNC caregivers completed these assignments.

Second, we compared TE-HNC and HNC groups on child behavior outcomes (ECBI Intensity and Problem Subscales) utilizing both within- and between-group effect sizes and measures of clinical significance. Within-group effect sizes were calculated utilizing Morris and DeShon’s (2002) Equation 8 to control for the correlation between pre- and postassessment means for each group. Given that small sample sizes can differ on preassessment scores, between-group effect sizes were calculated by first creating change scores for each family (e.g., preassessment to post- assessment). The between-group effect size then is calcu- lated by subtracting the average HNC group change score from the average TE-HNC change score and divid- ing by the standard deviation as previously described.

To test clinical significance of treatment effects (see Kendall, 1999, for an introduction to a special section on clinical significance), we utilized (a) Reliable Change Index (RCI; Jacobson & Truax, 1991), which indicates change attributable to treatment is most likely not due to chance (i.e., RCI �1.96; also see Abramowitz, 1988, for modifications for use with samples) and (b) Norma- tive Comparisons (Kendall & Grove, 1988), which assess if scores at posttreatment are distinguishable from indivi- duals in the normative range (i.e., is the group mean and upper limit [M þ 1 SD]).

Finally, we examined program costs and consumer satisfaction. Costs were quantified using a payer perspec- tive to value the time and resources required to deliver the program. Costs associated with program start-up (i.e., nonlabor start-up and development costs) and implementation (i.e., therapist time) were analyzed sep- arately. Start-up costs are considered investments that

are spread over all program participants, so we assigned an equal proportion of these costs to each program par- ticipant, regardless of how long they participated in HNC or TE-HNC (i.e., these analyses include both com- pleters and noncompleters). To avoid inadvertently and inaccurately biasing implementation costs downward by including labor costs for participants who dropped out before mastering a skill, implementation costs were collected at the participant level and include completers only. Consumer satisfaction was assessed using between group effect sizes just described.

RESULTS

Enrollment and Baseline Characteristics

Of the low-income families who completed a clinic-based eligibility interview (n ¼ 48), 24 were ineligible for the pilot RCT (e.g., pending Department of Social Services investigation) and two eligible families chose not to enroll. Using restricted random assignment to force equal sample sizes, Master’s-level therapists randomly assigned the remaining 22 eligible low-income families to the HNC or TE-HNC group. Of these 22 families, 96% (n ¼ 21) owned a mobile phone and, of these, 33% (n ¼ 7) owned a smartphone; however, as noted earlier all families were provided a project smartphone to ensure a common interface and access to materials among participants.

The initial three families randomized served as prac- tice cases for each of the three project therapists, and the remaining 19 families HNC (n ¼ 10) and TE-HNC (n ¼ 9) were considered for use in the current analyses.

Attrition and Missing Data

Overall rate of attrition (21%) was substantially lower than what has been reported in prior BPT work with low-income samples (i.e., as high as 56%; Fernandez et al., 2011; also see Lundahl et al., 2006; Reyno & McGrath, 2006, for reviews). The four families (two HNC, two TE-HNC) that dropped out of the study noti- fied project staff prior to discontinuing participation, and each cited a major health (e.g., organ transplant) or family (e.g., divorce) stressor necessitating dropout. Given the pilot nature of the project, we considered only complete data (i.e., data from participants available at pre- and posttreatment). On average (see Table 1), care- givers who completed both the pre- and postassessments were 37 years old, most were female (87%), and 80% worked at least part-time. More than half (53%) of the children were boys, with an average age of 5.67 years (range ¼ 3–8 years old).

94 JONES ET AL.

D ow

nl oa

de d

by [

U ni

ve rs

it y

of V

er m

on t]

a t

08 :4

0 09

J an

ua ry

2 01

Analysis of TE-HNC Versus HNC Condition at Baseline

As shown in Table 1, there were no differences between families randomized to TE-HNC or HNC on demo- graphics; however, randomization failed to yield equiva- lent groups on an established correlate of BPT dropout, baseline disruptive behaviors. Caregivers randomized to TE-HNC were more likely to report higher levels of child disruptive behaviors on the Intensity subscale in the TE-HNC group at baseline (MECBI Intensity: TE-HNC ¼ 148.86, HNC ¼ 131.50); however, both groups evidenced problem behaviors in the clinical range on both the Prob- lem and Intensity Scale scores.

Engagement and Skill Generalization

As shown in Table 2, findings demonstrated medium to large between-group effect sizes for engagement and skill generalization favoring TE-HNC relative to HNC. Effect sizes demonstrated that families in TE-HNC were more likely to attend weekly sessions (d ¼ 0.88), partici- pate in midweek calls (d ¼ 2.59), and complete their Child’s Game home practice (d ¼ 0.63) than families in HNC.

TE-HNC families also provided information regard- ing Child’s Game practice by completing scheduled daily smartphone surveys and a video recording of one home practice per week. Regarding the surveys, across

each of the five HNC skills, on average, TE-HNC fam- ilies completed on average a survey on 60% of possible between session days. The median time for survey com- pletion across families and skills was 2 to 3 min. During Phase I, TE-HNC families reported the following: (a) They practiced Child’s Game on the majority (69%) of the days for which they completed the surveys; (b) on 86% of those days, families reported that they practiced Child’s Game the recommended minimum of 15 min per day; and (c) the majority (95%) reported that their child enjoyed Child’s Game. Findings were similar for Phase II: (a) Caregivers reported practicing Child’s Game on 76% of the days for which they completed a survey; (b) on the majority (83%) of those days, caregivers reported practicing the minimum of 15 min per day; and (c) almost all (98%) reported that their child enjoyed Child’s Game. Regarding video recording, the data sug- gest this pattern: Two families video recorded at least one home practice per week between each HNC session, whereas the remaining families recorded at least one practice between 71% (n ¼ 1), 50% (n ¼ 3), and 33% (n ¼ 1) of sessions.

Finally, representative feedback from TE-HNC families regarding how smartphone enhancements facilitated coparent involvement in practice included the following: ‘‘It was nice to be able to not only tell him what I learned, but to show him [with the skills videos]’’; ‘‘[Without the smartphone enhancements], I don’t think he would have understood everything as

TABLE 1

Demographic Characteristics of the Low-Income Sample at Preassessment

Measure Possible Range

Treatment Groups

Total Sample TE-HNC HNC

% M SD % M SD % M SD

Child Demographics

Gender (% Male) 53 57 50

Age (Years) 3–8 5.67 1.72 5.57 1.27 5.75 2.12

Ethnicity=Race (% Minority) 57 63

Caregiver Demographics

Gender (% Female) 87 71 100

Age (Years) 36.73 8.81 35 5.92 38.25 10.95

Ethnicity=Race (% Minority) 29 50

Marital status

Married 33 43 38

Single 67 57 62

Employment Status

Unemployed 20 29 13

Part-Time 33 29 38

Full-Time 47 42 50

Child Behavior

Eyberg Child Behavior Inventory

Intensity 0–252 139.6 24.13 148.86 22.51 131.5 2.87

Problem 0–36 21.47 4.93 22.57 5.19 20.50 4.81

Note. N ¼ 15; TE-HNC n ¼ 7, HNC n ¼ 8. TE-HNC ¼ Technology-Enhanced Helping the Noncompliant Child.

TECHNOLOGY-ENHANCED BPT 95

D ow

nl oa

de d

by [

U ni

ve rs

it y

of V

er m

on t]

a t

08 :4

0 09

J an

ua ry

2 01

well’’; and ‘‘If my husband [did not have the smart- phone enhancements], I don’t think any of this would have made sense to him and he would have lost interest.’’ Representative feedback from caregivers in the HNC group regarding how the smartphone enhancements would have been helpful include ‘‘It would have allowed her to have a fuller sense [of the program] without being affected by my wording’’; ‘‘Having the [smartphone enhancements] would have made the program more real [for him], [he] would have taken the program more seriously’’; and ‘‘He would have gotten the information on his own terms and in his own time [with smartphone enhancements, which] would have eased the tension.’’

Child Disruptive Behavior

Findings reported in Table 2 reveal that HNC yields large effect sizes (Intensity d ¼ 1.20; Problem d ¼ 1.24) for change in child behavior from pre- to posttreatment, as well as clinically significant improvement (Intensity RCI ¼ 4.48; Problem RCI ¼ 4.71) that is within the nor- mative range at posttreatment for the Intensity but not the Problem subscale. However, all the indicators suggest that TE-HNC may boost child behavior outcomes: The

between-group effect size comparing the two interventions favored TE-HNC (Intensity d ¼ 0.99; Problem d ¼ 0.54;); the within-group TE-HNC effect size (Intensity d ¼ 3.71; Problem d ¼ 2.00) was larger than the HNC effect size; the RCIs for TE-HNC (Intensity RCI ¼ 7.40; Severity RCI ¼ 6.65) were greater than the RCIs for HNC; and normative comparisons resulted in TE-HNC being within normative range for both ECBI Subscales).

Program Costs and Consumer Satisfaction

As described in the Data Analytic Approach section, start-up costs were calculated for all enrolled families, whereas implementation costs were calculated for families who completed the programs only. Both interventions incurred start-up costs for therapist man- uals, toys, and handouts, resulting in program start-up costs of an average of $10 per enrolled family. The TE-HNC intervention had additional start-up costs for smartphone-related services and materials, yielding total program start-up costs for the TE-HNC intervention of $671 per enrolled family.

Consistent with our interests in the implementation costs of TE-HNC compared to HNC (i.e., therapist

TABLE 2

Means, Standard Deviations, Effect Sizes, and Clinically Significant Change for Primary Study Variables

Measure

Assessment

Pre %

M (SD)

Post %

M (SD)

ES a

Pre-Post (d)

CI b

95%

ES c TE-HNC

vs. HNC CId 95% RCIe Normative

Comparison

Engagement Session

Attendance

0.88 �0.29; 1.83

TE-HNC 97 (5)

HNC 90 (11)

Midweek Check-In 2.59 0.97; 3.92

TE-HNC 93 (8)

HNC 58 (19)

Skill Generalization

Home Skill Practice 0.63 �0.42; 1.68 TE-HNC 91 (13)

HNC 77 (23)

Child Behavior

ECBI Intensity 0.99 �0.13; 2.05 TE-HNC 148.86 (22.51) 83.00 (15.34) 3.71 2.00; 7.80 148.86 (22.51) 7.40� Yes HNC 131.50 (23.87) 91.63 (21.25) 1.20 0.36; 2.98 131.50 (23.87) 4.48� Yes

Problem 0.54 �0.51; 1.56 TE-HNC 22.57 (5.19) 6.14 (5.67) 2.00 1.59; 4.73 6.65� Yes HNC 20.50 (4.81) 8.88 (8.17) 1.24 0.87; 4.38 4.71� No

Note. Technology-Enhanced Helping the Noncompliant Child (TE-HNC) n ¼ 7; HNC n ¼ 8. ECBI ¼ Eyberg Child Behavior Inventory. aWithin group effect size at postassessment controlling for preassessment. b 95% confidence interval (CI) for within-group ES calculations.

c Between-group ES comparing TE-HNC versus HNC at postassessment controlling for preassessment. d95% CI for between-group ES calculations. eReliable Change Index (RCI) ¼ change attributable to treatment is clinically significant (�RCI �1.96). f Normative Comparison ¼ at posttreatment group mean and upper-limit (M þ 1 SD) is within normative range.

96 JONES ET AL.

D ow

nl oa

de d

by [

U ni

ve rs

it y

of V

er m

on t]

a t

08 :4

0 09

J an

ua ry

2 01

time cost savings, if any, associated with utilizing the smartphone enhancements, Table 3 reports mean implementation costs only (i.e., excludes program start-up costs) for each of the program components among program completers only for HNC and TE-HNC families. Skill 5 (Time Out) required the most therapist time (an average of 2.3 sessions among HNC families and 2.1 sessions among TE-HNC families) and, thus, was the most expensive skill, costing an average of $127 per family in HNC and $119 per family in TE-HNC to master. The average implementation cost to master a skill was similar for each intervention approach: $82 for HNC and $80 for TE-HNC. Total implementation costs were also similar for the two interventions, although HNC costs were slightly higher than costs for TE-HNC ($491 for HNC and $478 for TE-HNC) because of the additional number of sessions required for HNC families to complete the program. HNC families had an average of 10 therapist sessions (with a range of seven to 12 ses- sions) to complete the program, while TE-HNC families averaged eight therapy sessions to completion (range of seven to 10 sessions).

Our cost data analyses also allowed us to examine therapist time per completer family to consider whether enhanced engagement, skill generalization, and child behavior outcomes were simply a function of TE-HNC therapists spending more time in contact with families than HNC therapists. As demonstrated in Table 3, TE-HNC required slightly less, rather than more, total therapist time on average (M ¼ 1,302 total min per family) than HNC (M ¼ 1,318 total min per family).

Finally, consumer satisfaction analyses revealed that the increased efficiency (i.e., fewer sessions) did not compromise family satisfaction with the TE-HNC (M ¼ 59.83, SD ¼ 1.94) relative to HNC (M ¼ 56.56, SD ¼ 2.47) program. Rather, between-group effect size analysis favored TE-HNC (d ¼ 1.48).

DISCUSSION

Low-income youth are more likely to have DBDs than relatively higher income youth yet less likely to engage in BPT (see Jones et al., 2013; Lundahl et al., 2006; Reyno & McGrath, 2006, for reviews). The objective of this project was to develop and pilot a technology- enhanced version of one evidence-based BPT program, HNC (McMahon & Forehand, 2003), with the aim of increasing engagement and, in turn, BPT outcomes among low-income families of children with DBD. Findings suggest that TE-HNC shows promise as an innovative and efficacious approach to engaging low-income families in BPT with potential cost savings.

The current pilot data suggested medium (midweek call participation) to large (session attendance, home practice) effect sizes favoring TE-HNC relative to HNC on measures of engagement and skill generaliza- tion. The weekly demands of BPT are time intensive, and low-income families are less likely to effectively engage in BPT services due to financial strain and asso- ciated stressors (see Eyberg et al., 2008; McMahon & Forehand, 2003; Reyno & McGrath, 2006, for reviews). Our pilot findings suggest that smartphone enhance- ments have the potential to increase the therapist and program’s connection to and support for the family between sessions and, in turn, increase family’s auto- nomy with implementing the skills in the home setting.

As HNC is an evidence-based BPT program, it was expected, and did occur, that HNC would yield clinically significant change in child disruptive behavior; however, effect size and clinical significance findings suggest that TE-HNC may boost child treatment outcome. Although testing the mechanisms that account for the boost in child behavior outcomes is not possible due to limita- tions in statistical power, we hypothesize that enhanced engagement and skill generalization boost the impact of the program on child behavior outcomes.

TABLE 3

Mean Therapist Costs, Sessions, and Time per Program Completer

Program Components

TE-HNC TE-HNC

Cost No. of Sessions Total a Time (Minutes) Cost No. of Sessions Total

a Time (Minutes)

Orientation $50 1.0 138 $61 1.0 158

Attends $64 1.2 171 $83 1.4 228

Rewards $123 2.4 330 $72 1.3 200

Ignoring $57 1.1 157 $61 1.0 164

Clear Instruction $70 1.3 94 $81 1.4 224

Time-Out $127 2.3 328 $119 2.1 328

Mean per Component $82 1.6 $80 1.4

Total per Completer $491 10.0 1318 $478 8.3 1,302

Note. Technology-Enhanced Helping the Noncompliant Child (TE-HNC) n ¼ 7; HNC n ¼ 8. a Includes sessions, midweek calls, session notes, supervision, review of daily surveys (TE-HNC only), watching home practice video (TE-HNC

only), and other program-related activities.

TECHNOLOGY-ENHANCED BPT 97

D ow

nl oa

de d

by [

U ni

ve rs

it y

of V

er m

on t]

a t

08 :4

0 09

J an

ua ry

2 01

Although limited by within-group examination, trends in the daily smartphone surveys and video record- ing of home practice may further help to explain the mechanisms by which smartphone-enhancements boost child outcomes. First, daily survey data revealed that TE-HNC caregivers reportedly practiced Child’s Game for at least the recommended 15 min per day on the majority of days between sessions, suggesting that the smartphone reminders about skill practice is associated with skill generalization from the clinic to the home and, in turn, enhanced child behavior outcomes. Second, the majority of HNC families video recorded at least one home practice between the majority of their sessions, sug- gesting that families benefited from increased opport- unity for therapist review of and feedback on their skill practice with subsequent impact on child behavior. Third, caregiver responses to posttreatment interviews about the programs suggest that smartphone enhance- ments may have enhanced coparent practice of the skills at home by providing caregivers a mechanism (i.e., skills videos series) and support (e.g., text message reminders & smartphone-assessment items regarding the importance of coparent involvement and practice) for involving the nonparticipating coparent in the BPT program. This can increase the consistency of skill utilization between caregivers and, in turn, lead to greater improvements in child behavior (see McMahon & Forehand, 2003).

Tate, Finkelstein, Khavjou, and Gustufson (2009) recently highlighted that cost-effectiveness is a primary rationale for utilizing technology to enhance service delivery; nevertheless, they point out that few studies actually report data on economic indicators. Fortu- nately, some BPT programs have begun to report cost data (e.g., Foster, Prinz, Sanders, & Shapiro, 2008; O’Neil, McGilloway, Donnelly, Bywater, & Kelly, 2011; Sanders, 2008). Our pilot study was designed in part to move beyond these studies and address the question of whether technology-enhanced delivery of BPT had the potential to yield cost savings.

Not unexpectedly, the development of TE-HNC was costly; however, these start-up costs are viewed as one-time expenses for a service delivery agency. Furthermore, as smartphones continue to increase in use, it will not be necessary (as we did) for agencies to purchase smartphones and service plans for their clients. In essence, as the cost of smartphones continue to decline and the availability of contract-free carriers (e.g., no contract, $35=month unlimited data, messaging, web, & email plans, $39.99 smartphone model), increases the cost of facilitating smartphone-enhanced programs like TE-HNC will decline.

Our primary interest was not in program develop- ment (i.e., start-up) costs but rather implementation. The data from this study suggest that the number of ses- sions required to complete treatment (i.e., achieve HNC

criteria for skills) was fewer and overall therapist time from orientation through Time-Out was slightly less for TE-HNC than for HNC. We believe that as thera- pists became increasingly proficient with the technology, the therapist time to implement TE-HNC will decrease, boosting the minutes per client differential between TE-HNC and HNC.

Of course, this was a pilot study, and the limitations must be noted. First, as noted earlier, limitations in sample size and methods preclude significance testing, analysis of the mechanisms by which smartphone enhancements impact BPT outcomes, as well as quanti- tative comparison of groups on some study variables (i.e., TE-HNC-only variables). In addition, as noted ear- lier, randomization did not yield equivalent groups on children’s baseline level of disruptive behaviors, a variable typically associated with treatment outcome; however, it is notable that although TE-HNC children had higher levels of disruptive behaviors than HNC chil- dren at baseline, the former group had lower levels at posttreatment. Finally, there is much discussion in the broader treatment outcome literature regarding tension between internal and external validity, and our pilot study is no exception. Our exclusion criteria have the potential to limit the generalizability of our findings to real-world practice settings, which deal with complex families and constellations of presenting issues (e.g., families with maltreatment histories=Department of Social Services involvement). That said, BPT programs may require adaptations to effectively deal with specific populations, including those with maltreatment histories and others (see Wells, 2003, for a review). Given the pilot nature of our study, it was a necessity that we focus more narrowly on DBDs as the primary presenting issue and the standard HNC program.

This pilot study also has several strengths. First, this is the first study to our knowledge to examine the use of smartphone enhancements to improve the engagement and treatment outcomes of low-income families in BPT. Given that low-income families are among the most difficult to engage in BPT, smartphone enhancements may show promise across the broader socioeconomic spectrum as well. In addition, this study utilized the gold standard for evaluating treatment efficacy, a randomized control trial design. As such, we now have the research infrastructure and promising findings to guide a future, sufficiently powered trial. Third, in contrast to typical practice, the smartphone enhancements developed in this study represent the collaboration of researchers with expertise in BPT and underserved families, practicing clinicians with training in BPT, economists with expertise in tracking and calculating intervention costs, and an industry partner with experience in developing sustain- able technological applications. Even a cursory review of BPT-relevant ‘‘applications’’ (e.g., a time-out timer)

98 JONES ET AL.

D ow

nl oa

de d

by [

U ni

ve rs

it y

of V

er m

on t]

a t

08 :4

0 09

J an

ua ry

2 01

suggests that typical developers of service-related appli- cation likely have little understanding of BPT principles or the real challenges facing families (i.e., tracking elapsed time is unlikely to be the most challenging issue).

Fourth, although the integration of technology into services research has proceeded largely without theory (see Riley et al., 2011; Ritterband, Thorndike, Cox, Kovatchev, & Gonder-Frederick, 2009, for reviews), our aims and hypotheses were firmly grounded in a theor- etical framework. Specifically, we posited, and findings supported, that the portability, availability, and range of applications bundled into the smartphone had the potential to boost the reach of BPT program materials and skills to families between sessions, even in the face of the financial strain and stressors that compromise engagement in low-income families (see Aguilera et al., 2012; Anderson & Subramanyam, 2011; Davies, 2011; Jones et al., 2013; Snider, 2011; Williams et al., 2007, for reviews). Of importance, however, the functions (i.e., skill modeling, assessment of practice, session reminders, etc.) of the smartphone enhancements developed for this project could be delivered via a range of technologies available to therapists and families and can also be adapted as technology innovations evolve (see Ritterband et al., 2009, p. 22, for a review of the impor- tance of focusing on intervention content and users, rather than the ‘‘black box’’ of the technology).

Finally, HNC is but one of several theoretically and practically similar BPT programs evolving from the Hanf Model (Reitman & McMahon, 2012). Practicality and cost-effectiveness of technology depends in large part on the generalizability of approaches tested from one BPT program for DBDs to another. As such, we believe that the technology enhancements tested with HNC in this study could be used similarly with other Hanf-based programs with potential implications for the broader field of children’s mental health as well (see Jones et al., 2013, for review).

Before concluding, it is important to acknowledge the potential ethical issues associated with the intersection of technology and BPT, issues that are relevant for the broader field of children’s mental health as well. As with other aspects of the use of technology in services research, the field is progressing far more quickly than advances in relevant ethical guidelines (Novotney, 2011; also see Jones et al., 2013; Reed, McLaughlin, & Milholland, 2000; Richardson et al., 2009). In fact, lea- ders in the field highlight that ‘‘the tail is wagging the dog in some ways on this issue’’ (Novotney, 2011, p. 40), as advances in technology far outpace the rate at which practice guidelines are updated to deal with new challenges (please see work by Nelson et al., in this spe- cial section for advances on this front). Some of the potential ethical issues related to any telehealth approach include standard of care (e.g., emergency protocols when

a client is not physically in the same room as the provider or there is no ‘‘provider’’), privacy and security (e.g., use of secure networks), and feasibility (e.g., training both therapists and clients in the use of technology). The emerging field of technology-enhanced BPT is certainly not immune from any of these ethical issues (see Jones et al., 2013, for a review). For example, in our study we had standards in place to ensure that access to smart- phones was protected by programming only strong pass- words; that survey data were deidentified and linked only to a numerical code associated with the family; and that, if lost or misplaced, smartphones could be remotely ‘‘bricked’’ or deactivated to ensure protection of all fam- ily data, including identifiable information (e.g.,, home practice videos). Of note, none of our families reported their smartphones lost or misplaced during the course of the project, even temporarily, and all families returned their smartphones to the project staff at the end of the project. That said, we as a field must continue to give careful attention and discussion to how such issues will be handled as we consider deploying smartphone- enhanced and other technology-enhanced interventions beyond resource-intensive and carefully controlled university training clinics to real-world practice settings.

In conclusion, TE-HNC shows promise as an innova- tive approach to boosting BPT gains for low-income families. Future research with sufficient power is planned to examine whether the trends in this study are replicated and, if so, whether they reach statistical significance. Given our reliance on industry statistics for data regard- ing smartphone ownership, our goal is also to continue to collect data on emergent trends among low-income consumers in particular to ensure that our approach is one that remains viable. In the meantime, it is our hope that our pilot work will motivate collective efforts across the field to capitalize on advances in technology to best meet the needs of families.

REFERENCES

Abramowitz, J. S. (1998). Does cognitive-behavioral therapy cure

obsessive-compulsive disorder? A meta-analytic evaluation of clinical

significance. Behavior Therapy, 29, 339–355. doi:10.1016=S0005-

7894(98)80012-9

Aguilera, A., & Muench, F. (2012). There’s an App for that:

Information technology applications for cognitive behavioral

practitioners. The Behavior Therapist, 35, 65–73.

Aguilera, A., & Munoz, R. F. (2011). Text messaging as an adjunct to

CBT in low-income populations: A usability and feasibility study.

Professional Psychology: Research and Practice, 42, 472–478.

doi:10.1037=a0025499

Anderson, D., & Subramanyam, R. (2011). The new digital American

family: Understanding family dynamics, media, and purchasing

behavior trends. New York, NY: The Nielson Company.

Burns, G. L., & Patterson, D. R. (1991). Factor structure of the Eyberg

Child Behavior Inventory: Unidimensional or multidimensional

TECHNOLOGY-ENHANCED BPT 99

D ow

nl oa

de d

by [

U ni

ve rs

it y

of V

er m

on t]

a t

08 :4

0 09

J an

ua ry

2 01

measure of disruptive behavior? Journal of Clinical Child Psychology,

20, 439–444. doi:10.1207=s15374424jccp2004_13

Burns, G. L., Patterson, D. R., Nussbaum, B. R., & Parker, C. M.

(1991). Disruptive behaviors in an outpatient pediatric population:

Additional standardization data on the Eyberg Child Behavior

Inventory. Psychological Assessment, 3, 202–207. doi:10.1037=

1040-3590.3.2.202

Chorpita, B. F., Daleiden, E. L., Ebesutani, C., Young, J., Becker, K.

D., Nakamura, B. J., & Starace, N. (2011). Evidence-based treatments

for children and adolescents: An updated review of indicators of

efficacy and effectiveness. Clinical Psychology: Science and Practice,

18, 154–172. doi:10.1111=j.1468-2850.2011.01247.x

Cohen, J. (1988). Statistical power analysis for the behavioral sciences

(2nd ed.). Hillsdale, NJ: Erlbaum.

Conger, R. D., & Donnellan, M. B. (2007). An interactionist perspec-

tive on the socioeconomic context of human development. Annual

Review of Psychology, 58, 175–199. doi:10.1146=annurev.psych.58.

110405.085551

Cowan, C. P., Cowan, P. A., & Barry, J. (2011). Couples’ groups for

parents of preschoolers: Ten-year outcomes of a randomized trial.

Journal of Family Psychology, 25, 240–250. doi:10.1037=a0023003.

D’Amico, E. J., Neilands, T. B., & Zambarano, R. (2001). Power

analysis for multivariate and repeated measures designs: A flexible

approach using the SPSS MANOVA procedure. Behavior Research

Methods, Instruments, and Computers, 33, 479–484. doi:http://

dx.doi.org/10.3758/BF03195405

Davies, E. (2011). Learning from the digital jugglers: New trends in

smartphone adoption and usage patterns. New York, NY: Fjord.

Dekovic, M., Slagt, M. I., Asscher, J. J., Boendermaker, L.,

Eichelsheim, V. I., & Prinzie, P. (2011). Effects of early

prevention programs on adult criminal offending: A meta-analysis.

Clinical Psychology Review, 31, 532–544. doi:10.1016=

j.cpr.2010.12.003

Dodge, K. A., Malone, P. S., Lansford, J. E., Miller, S., Pettit, S., &

Bates, J. E. (2009). A dynamic cascade model of the development

of substance-use onset. Monographs of the Society for Research in

Child Development, 74, 1–31. doi:10.1111=j.1540-5834.2009.00528.x.

Dumas, J. E., Begle, A. M., French, B., & Pearl, A. (2010). Effects

of monetary incentives on engagement in the PACE parenting

program. Journal of Clinical Child and Adolescent Psychology, 39,

302–313. doi:10.1080=15374411003691792

Enock, P. M., & McNally, R. J. (2013). How mobile apps and other

web-based interventions can transform psychological treatment

and the treatment development cycle. The Behavior Therapist, 36,

56–66.

Eyberg, S. M., Nelson, M. M., & Boggs, S. R. (2008). Evidence-based

treatments for child and adolescent disruptive behavior disorders.

Journal of Clinical Child and Adolescent Psychology, 37, 213–235.

doi:10.1080=15374410701820117

Eyberg, S. M., & Pincus, D. (1999). Eyberg Child Behavior Inventory &

Sutter-Eyberg Student Behavior Inventory–Revised: Professional

manual. Odessa, FL: Psychological Assessment Resources.

Fernandez, M. A., Butler, A. M., & Eyberg, S. M. (2011). Treat-

ment outcome for low socioeconomic status African American

families in Parent–Child Interaction Therapy: A pilot study. Child

& Family Behavior Therapy, 33, 32–48. doi:10.1080= 07317107.

2011.545011

Flanagan, S., Adams, H. E., & Forehand, R. (1979). A comparison of

four instructional techniques for teaching parents to use time-out.

Behavior Therapy, 10, 94–102. doi:http://dx.doi.org/10.1016/S0005-

7894(79)80012-X

Forehand, R., Jones, D. J., & Parent, J. (2013). Behavioral parenting

interventions for child disruptive behaviors and anxiety: What’s

different and what’s the same? Clinical Psychology Review, 33,

133–145. doi:10.1016=j.cpr.2012.10.010

Foster, E., Prinz, R. J., Sanders, M. R., & Shapiro, C. J. (2008). The

costs of a public health infrastructure for delivering parenting and

family support. Children and Youth Services Review, 30, 493–501.

http://dx.doi.org/10.1016/j.childyouth.2007.11.002.

Gardner, F., Connell, A., Trentacosta, C. J., Shaw, D. S., Dishion,

T. J., & Wilson, M. N. (2009). Moderators of outcome in a brief

family-centered intervention for preventing early problem behavior.

Journal of Consulting and Clinical Psychology, 77, 543–553.

doi:10.1037=a0015622

Garland, A. F., Hawley, K. M., Brookman-Fraze, L., & Hurlburt,

M. S. (2008). Identifying common elements of evidence-based psy-

chosocial treatments for children’s disruptive behavior problems.

Journal of the Academy of Child and Adolescent Psychiatry, 47,

505–514. doi:10.1097=CHI.0b013e31816765c2

Gross, D., Johnson, T., Ridge, A., Garvey, C., Julion, W., Treysman,

A. B., . . . Fogg, L. (2011). Cost-effectiveness of childcare discounts

o parent participation in preventive parent training in low-income

communities. The Journal of Primary Prevention, 32, 283–298.

doi:10.1007=s10935-011-0255-7

Jacobson, N. S., & Truax, P. (1991). Clinical significance: A

statistical approach to defining meaningful change in psycho-

therapy research. Journal of Consulting and Clinical Psychology,

59, 12–19.

Jones, D. J., Forehand, R., Cuellar, J., Kincaid, C., Parent, J., Fenton,

N., & Goodrum, N. (2013). Harnessing innovative technologies to

advance children’s mental health: Behavioral parent training as an

example. Clinical Psychology Review, 32, 241–252. doi:10.1016=

j.cpr.2012.11.003

Jones, D. J., Forehand, R., McKee, L., Cuellar, J., & Kincaid, C.

(2010). Behavioral parent training: Is there an ‘app’ for that? The

Behavior Therapist, 33, 72–77.

Kazdin, A. E., & Blasé, S. L. (2011). Rebooting psychotherapy

research and practice to reduce the burden of mental illness.

Perspectives on Psychological Science, 6, 21–37. doi:10.1177=

1745691610393527

Kendall, P. C. (1999). Special section: Clinical significance. Journal of

Consulting and Clinical Psychology, 67, 283–339.

Kendall, P. C., & Grove, W. M. (1988). Normative comparisons in

therapy outcome. Behavioral Assessment, 10, 147–158.

Lee, S., Aos, S., Drake, E., Pennucci, A., Miller, M., & Anderson, L.

(2012). Return on investment: Evidence-based options to improve

statewide outcomes (Document No. 12–04-1201). Retrieved from

Washington State Institute for Public Policy: http://www.wsipp.

wa.gov/pub.asp?docid=12-04-1201.

Leijten, P., Raaijmakers, M. A., deCastro, B. O., & Matthys, W.

(2013). Does socioeconomic status matter? A meta-analysis on

parent training effectiveness for disruptive child behavior. Journal

of Clinical Child and Adolescent Psychology, 42, 384–392.

doi:10.1080=15374416.2013.769169.

Lundahl, B., Risser, H. J., & Lovejoy, C. (2006). A meta-analysis of par-

ent training: Moderators and follow-up effects. Clinical Psychology

Review, 26, 86–104. doi:10.1016= j.cpr.2005.07.004

MacKenzie, E. P., Fite, P. J., & Bates, J. E. (2004). Predicting outcome

in behavioral parent training: Expected and unexpected results.

Child & Family Behavior Therapy, 26, 37–53. doi:10.1300=

J019v26n02_03

McGilloway, S., Mhaille, G. N., Bywater, T., Furlong, M., Leckey, Y.,

Kelly, P., & Cominskey, C. (2012). A parenting intervention for

childhood behavioral problems: A randomized controlled trial in dis-

advantaged community-based settings. Journal of Consulting and

Clinical Psychology, 80, 116–127. doi:http://dx.doi.org/10.1037/

a0026304

McMahon, R. J., & Forehand, R. L. (2003). Helping the noncompliant

child: Family-based treatment for oppositional behavior. New York,

NY: Guilford.

100 JONES ET AL.

D ow

nl oa

de d

by [

U ni

ve rs

it y

of V

er m

on t]

a t

08 :4

0 09

J an

ua ry

2 01

Merikangas, K. R., Nakamura, E. F., & Kessler, R. C. (2009).

Epidemiology of mental disorders in children and adolescents.

Dialogues in Clinical Neuroscience, 11, 7–20.

Moffitt, T. E., Arseneault, L., Jaffee, S. R., Kim-Cohen, J., Koenen,

K. C., Odgers, C. L., & Viding, E. (2008). Research review:

DSM–V conduct disorder: Research needs for an evidence base.

Journal of Child Psychology and Psychiatry, 49, 3–33. doi:10.1111=

j.1469-7610.2007.01823.x

Mohr, D. C., Cujpers, P., & Lehman, K. (2011). Supportive account-

ability: A model for providing human support to enhance adherence

to eHealth interventions. Journal of Medical Internet Research, 13,

e30. doi:10.2196=jmir.1602

Morris, S. B., & DeShon, R. P. (2002). Combining effect size estimates

in meta-analysis with repeated measures and independent-groups

designs. Psychological Methods, 7, 105–125. doi:http://dx.doi.org/

10.1037//1082-989X.7.1.105

Nay, R. W. (1976). Behavioral interventions: Contemporary strategies.

New York, NY: Gardner.

Novotney, A. (2011, June). A new emphasis on telehealth: How can

psychologists stay ahead of the curve—and keep patients safe?

Monitor on Psychology, pp. 40–45.

O’Dell, S. L., O’Quin, J. A., Alford, B. A., O’Briant, A. L., Bradlyn, A.

S., & Giebenhain, J. E. (1982). Predicting the acquisition of parent-

ing skills via four training methods. Behavior Therapy, 13, 194–208.

doi:http://dx.doi.org/10.1016/S0005-7894(82)80063-4

O’Neill, D., McGilloway, S., Donnelly, M., Bywater, T., & Kelly, P.

(2011). A cost effectiveness analysis of the Incredible Years parent-

ing programme in reducing childhood health inequalities. The

European Journal of Health Economics, 14, 85–94. http://dx.doi.org/

10. 1007=s10198-011–0342-y.

Piquero, A. R., Farrington, D. P., Welsh, B. C., Tremblay, R., &

Jennings, W. G. (2009). Effects of early family=parent training

programs on antisocial behavior and delinquency. Journal of Experi-

mental Criminology, 5, 83–120. doi:10.1007=s11292-009-9072-x

Reed, G. M., McLaughlin, C. H., & Milholland, K. (2000). Ten

interdisciplinary principles for professional practice in telehealth:

Implications for psychology. Professional Psychology: Research &

Practice, 31, 170–178. doi:10.103==0735-7028.31.2.170

Reitman, D., & McMahon, R. J. (2012). Constance ‘‘Connie’’ Hanf

(1917–2002): The mentor and the model. Cognitive and Behavioral

Practice, 20, 106–116. doi:http://dx.doi.org/10.1016/j.cbpra.2012.

02.005

Reyno, S. M., & McGrath, P. J. (2006). Predictors of parent training

efficacy for child externalizing behavior problems—A meta-analytic

review. Journal of Clinical Child Psychology and Psychiatry, 47, 99–

111. doi:10.1111=j.1469-7610.2005.01544.x

Richardson, L. K., Frueh, B. C., Grubaugh, A. L., Johnston, R. H.,

Egede, L., Johnson, R. H., & Elhai, J. D. (2009). Current directions

in videoconferencing tele-mental health research. Clinical Psychology:

Research & Practice, 16, 323–338. doi:10.1111=j.1468-2850.2009.

01170.x

Riley, W. T., Rivera, D. E., Atienza, A. A., Nilsen, W., Allison, S. M.,

& Mermelstein, R. (2011). Health behavior models in the age

of mobile interventions: Are our theories up to task?

Translational Behavioral Medicine, 1, 53–71. doi:10.1007=

s13142-011-0021-7

Ritterband, L. M., Thorndike, F. P., Cox, D. J., Kovatchev, B. P., &

Gonder-Frederick, L. A. (2009). A behavior change model for

internet interventions. Annals of Behavioral Medicine, 38, 18–27.

doi:10.1007=s12160-009-9133-4

Rogers, T. R., Forehand, R., Griest, D. L., Wells, K. C., & McMahon,

R. J. (1981). Socioeconomic status: Effects on parent and child

behaviors and treatment outcome of parent training. Journal

of Clinical Child Psychology, 10, 98–101. doi:10.1080=

15374418109533024

Sanders, M. R. (2008). Triple P Positive Parenting Program as a

public health approach to strengthening parenting. Journal of

Family Psychology, 22, 506–517. http://dx.doi.org/10.1037/0893-

3200.22.3.506.

Shaw, D. S., & Bell, R. Q. (1993). Developmental theories of parental

contributors to antisocial behavior. Journal of Abnormal Child

Psychology, 21, 493–518. doi:10.1007=BF00916316

Snider, M. (2011, April 21). A quarter of American homes have hung

up on landlines. USA Today. Retrieved from http://www.usatoday.

com/tech/news/2011-04-20-cellphone-study.htm

Tate, D. F., Finkelstein, E. A., Khavjou, O., & Gustafson, A. (2009).

Cost effectiveness of internet interventions: Review and recommen-

dations. Annals of Behavioral Medicine, 38, 40–45. doi:10.1007=

s12160-009-9131-6

Tate, D. F., & Zabinski, M. F. (2004). Computer and internet

applications for psychological treatment: Update for clinicians.

Journal of Clinical Psychology, 60, 209–220. doi:10.1002=jclp.10247

Thalheimer, W., & Cook, S. (2002). How to calculate effect sizes from pub-

lished research articles: A simplified methodology. Retrieved from Work

Learning Research website: http://work-learning.com/effect_sizes.htm

Thomas, R., & Zimmer-Gembeck, M. (2007). Behavioral outcomes of

Parent-Child Interaction Therapy and Triple P-Positive Parenting

Program: A review and meta-analysis. Journal of Abnormal Child

Psychology, 35, 475–495.

Webster-Stratton, C. (1990). Long-term follow-up of families with

young conduct problem children: From preschool to grade school.

Journal of Clinical Child Psychology, 19, 144–149. doi:0.1207=

s15374424jccp1902_6

Webster-Stratton, C. (1992). Individually administered videotape

parent training: ‘‘Who benefits?’’ Cognitive Therapy and Research,

16, 31–35. doi:10.1007=BF01172955

Webster-Stratton, C., Kolpacoff, M., & Hollinsworth, T. (1988). Self-

administered videotape therapy for families with conduct-problem

children: Comparison with two cost-effective treatments and a

control group. Journal of Consulting and Clinical Psychology, 56,

558–566. doi:10.1037=0022-006X.56.4.558

Wells, K. (2003). Adaptations for specific populations. In R. McMahon

& R. Forehand, Helping the Noncompliant Child: Family Based

Treatment for Oppositional Defiant Disorder (pp. 182–200).

New York: Guildford Press.

Williams, G. C., Lynch, M. F., & Glasgow, R. E. (2007). Computer-

assisted intervention improves patient-centered diabetes care by

increasing autonomy support. Health Psychology, 26, 728–734.

TECHNOLOGY-ENHANCED BPT 101

D ow

nl oa

de d

by [

U ni

ve rs

it y

of V

er m

on t]

a t

08 :4

0 09

J an

ua ry

2 01

View publication statsView publication stats

https://www.researchgate.net/publication/255705360