discussion 4

Treatment Sequencing for Childhood ADHD: A Multiple-

Randomization Study of Adaptive Medication and Behavioral

Interventions

William E. Pelham Jr.1, Gregory A. Fabiano2, James G. Waxmonsky3, Andrew R. Greiner1,

Elizabeth M. Gnagy1, William E. Pelham III4, Stefany Coxe1, Jessica Verley2, Ira Bhatia2,

Katie Hart1, Kathryn Karch2, Evelien Konijnendijk2, Katy Tresco2, Inbal Nahum-Shani5, and

Susan A. Murphy5

1Florida International University

2State University of New York at Buffalo

3Pennsyvania State University

4Arizona State University

5University of Michigan

Abstract

Objective—Behavioral and pharmacological treatments for children with ADHD were

evaluated to address whether endpoint outcomes are better depending on which treatment is

initiated first, and, in case of insufficient response to initial treatment, whether increasing dose of

initial treatment or adding the other treatment modality is superior.

Methods—Children with ADHD (ages 5–12, N = 146, 76% male) were treated for one school

year. Children were randomized to initiate treatment with low doses of either (a) behavioral parent

training (8 group sessions) and brief teacher consultation to establish a Daily Report Card or (b)

extended-release methylphenidate (equivalent to .15 mg/kg/dose bid). After 8 weeks or at later

monthly intervals as necessary, insufficient responders were rerandomized to secondary

interventions that either increased the dose/intensity of the initial treatment or added the other

treatment modality, with adaptive adjustments monthly as needed to these secondary treatments.

Results—The group beginning with behavioral treatment displayed significantly lower rates of

observed classroom rule violations (the primary outcome) and parent/teacher ratings of

oppositional behavior at study endpoint and tended to have fewer out-of-class disciplinary events.

Further, adding medication secondary to initial behavior modification resulted in better outcomes

on the primary outcomes and other measures than adding behavior modification to initial

medication. Normalization rates on teacher and parent ratings were generally high. Parents who

began treatment with behavioral parent training had substantially better attendance than those

assigned to receive training following medication.

Correspondence should be sent to: William E. Pelham, Jr., Professor of Psychology, Florida International University, 11200 SW 8th St AHC1 146, Miami, FL 33199, 305-348-3002; fax 305-348-3646.

HHS Public Access Author manuscript J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

Published in final edited form as: J Clin Child Adolesc Psychol. 2016 ; 45(4): 396–415. doi:10.1080/15374416.2015.1105138.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

Conclusions—Beginning treatment with behavioral intervention produced better outcomes

overall than beginning treatment with medication.

Keywords

Behavioral treatment; pharmacological treatment; ADHD

It is well established that evidence-based treatment for attention-deficit/hyperactivity

disorder (ADHD) includes medication with psychostimulants (Conners, 2002; Greenhill,

Pliszka, Dulcan, & the Work Group on Quality Issues, 2002) and behavioral interventions

(Pelham & Fabiano, 2008; Evans, Owens & Bunford, 2014; Fabiano et al., 2009). These two

modalities of treatment have been studied for decades, both separately and in combination.

Even so, disagreements remain among professionals regarding which treatment modality is

preferable, as well as how treatment for ADHD should begin. Some recommend beginning

medication immediately and supplementing with additional medication when necessary

(AACAP Work Group on Quality Issues, 2007). Others recommend beginning with

psychosocial treatments and adding medication if those treatments are insufficient (APA

Working Group on Psychoactive Medications for Children and Adolescents, 2006). Others

recommend starting with both treatments simultaneously (http://www.chadd.org). Most

recently, the American Academy of Pediatrics recommended each of the above strategies for

different ages of children (Subcommittee on Attention-Deficit/Hyperactivity Disorder,

Steering Committee on Quality Improvement and Management, 2011). However, the

research base upon which these recommendations have been made is scant and limited in

important ways (see for example Fabiano, Schatz, Aloe, Chacko, & Chronis-Tuscano, 2015).

In contrast to the hundreds of studies evaluating stimulants and behavioral interventions

separately, only a handful of randomized controlled trials (RCT) have compared medication,

behavioral treatment, and their combination, and each of these trials has limitations. A

common feature in the existing studies is that they have used fixed doses—typically

relatively high doses—of each treatment. For example, the largest and best-known RCT of

comparative treatments for ADHD is the MTA (MTA Cooperative Group, 1999a), which

used “optimal” dosing of medication (e.g., medication at school, evenings, and weekends)

compared with a package of intensive behavioral treatments (parent training, summer

treatment program, extensive teacher consultation, a classroom aide in school), and a

combined condition that added the two high-dose treatments and began them

simultaneously. The high-dose behavioral treatment was complex and costly, whereas the

high-dose medication treatment had adverse effects on growth. The results of the MTA vary

considerably based on the measure, individual differences, setting, timing of assessments,

length of follow-up, and interpretation (e.g., MTA Cooperative Group, 1999a, 1999b, 2004;

Molina et al., 2009; Pelham, 1999; Pelham et al., 2000; Owens et al., 2003; Swanson et al.,

2007), suggesting that additional finely-tuned investigations with different doses and

sequences of treatments are necessary to clarify relative effects of the two major, evidence-

based treatment modalities.

More recent research has used both within-subject and RCT designs to evaluate multiple

doses of medication in different combinations with varying doses of behavioral treatments

Pelham et al. Page 2

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

(Fabiano et al. 2007; Pelham et al., 2005; Pelham et al., 2014; Pelham et al., under review).

These studies have consistently found that intensive behavior modification produces acute

effects similar to relatively high doses of medication, but that low doses of both treatments

also maximize response in some but not all children. Further, these studies show that

combining low-dose medication with low-intensity behavioral interventions produces

equivalent effects to those of high-dose/high-intensity unimodal treatments for the majority

of children but with lower side effects, high parental satisfaction, and less complex

behavioral interventions. Side effects of stimulants increase with escalating dose and

duration of exposure (Barkley et al. 1990; Pelham et al, 1999; Stein et al, 2003, Swanson et

al., 2007). Therefore, adding behavioral interventions that reduce medication dose should

improve the tolerability of medication treatments. These studies have provided much-needed

information regarding the relative effects of different doses of medication and behavior

modification. However, they were implemented in an analogue summer treatment program

setting and thus do not directly address whether low doses of either modality or their

combination would be sufficient for many children in community settings.

Moreover, no studies in the literature have systematically varied and compared the sequence in which the two evidence-based modalities for ADHD are implemented. Medication is the

most commonly employed intervention and often the only intervention used in practice

(Epstein, et al., 2014; Visser et al. 2014) even in young children where professional

guidelines recommend starting with behavioral treatments (Subcommittee on Attention-

Deficit Hyperactivity Disorder, 2011). Psychiatric guidelines endorse optimizing dose at

home and school and using multiple medications prior to adding behavioral treatments

(AACAP Work Group on Quality Issues, 2007). When medication is implemented at this

high intensity level, there is less need for behavioral interventions, so the opportunity does

not exist to discover whether some or most children would do well with behavioral

interventions alone. For example, 75% of the individuals in the MTA behavioral treatment

group remained without medication during the year of treatment, and, for the majority of

those, for years afterward (MTA Cooperative Group, 1999a, 2004, Molina et al., 2009). This

implies that many children might not need medication if behavioral treatments were

employed first. Further, the majority of children in the medication management group

needed additional treatment during the 14-month treatment period, but only medication

could be used in this condition, and maintaining the initial medication effect required a 25% increase in dose during the year of treatment (Vitiello et al., 2001). In the combined

treatment group, an adjustment to the classroom intervention—most often the Daily Report

Card (DRC)—had to be made before medication dose could be increased, and that

procedure reduced the need for increased doses of medication (Vitiello et al., 2001). The

simultaneous introduction of conditions in the combined treatment group in the MTA means

that it is not possible to evaluate whether a behavioral intervention employed before medication would have prevented the need for medication or reduced the dosage needed.

Thus, a significant limitation of existing ADHD treatment studies is that questions regarding

sequencing, dosing, and combining treatments in natural settings have not been

systematically explored. In contrast to this body of research, treatment decisions in practice

are ongoing, based on the child’s impairment and response to intervention, and typically

provided initially at low “doses” that are escalated only if necessary. There are two crucial

Pelham et al. Page 3

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

decision points in treating a child with ADHD: (1) which treatment should be implemented

first? and (2) what should be done if the child does not respond adequately to that initial

treatment? For example, if a child begins treatment with medication and fails to respond,

there are two possibilities – increase the medication dose or add behavioral treatment. These

decision points have many implications with regard to tolerability/side effects, treatment

cost, and treatment efficacy, yet no studies have systematically evaluated such treatment-

sequencing questions for ADHD.

Adaptive treatment strategies have been gaining recognition as a strategy for preventive

interventions and management of chronic disorders (Murphy, 2005; Collins, Murphy, &

Bierman, 2004). In an adaptive approach, different dosages of treatment are provided

differentially to individuals across time in response to decision rules that are based on

individual characteristics. The major advantage of adaptive treatment designs is that they

mimic what happens in typical practice where treatments are often modified or enhanced,

but they retain controlled procedures, dosages, and rules to ensure replicability. Adaptive

approaches have previously been used with comprehensive services of the type that are used

for children with ADHD (for examples see Conduct Problems Prevention Research Group,

1999a, 1999b). Thus, in the present investigation, we employed a research design that has

been recommended for developing and comparing adaptive strategies, a sequential multiple assignment randomized trial (SMART; Murphy, 2005; Lavori & Dawson, 2000). In such

trials, individuals are randomized at multiple decision points to produce each treatment

strategy, combinations of which can be analyzed because they have been assigned by

randomization.

The current study was undertaken to address the limitations in the existing treatment

literature for children with ADHD with regard to treatment decisions and sequencing. A

SMART design was used to compare the results of various treatment decisions that included

behavioral and/or pharmacological interventions and their combination that can be widely

applied in clinical practice. Starting with low doses, treatments were conducted over an

entire school year in children’s school and home settings and adapted monthly within setting

based on response and need for additional intervention. End-of-study outcomes were

measured on objective classroom observations of behavior and parent/teacher ratings to

determine the relative benefits of the treatments and their sequences.

Within this design we were able to examine three important clinical questions/aims. First

(Aim 1): does it produce better outcomes on endpoint objective classroom measures and

parent/teacher ratings to initiate treatment with a low dose of (a) pharmacological

intervention with a stimulant drug or (b) behavioral intervention (group parent training and a

DRC at school)? Second (Aim 2), what is the most effective treatment protocol, or pattern of

initial treatment and conditional secondary/adaptive treatment (e.g., BM: behavioral

followed by medication in the event of insufficient response) among the four that we

employed (BM, Behavioral-Behavioral (BB), Medication-Behavioral (MB), and

Medication-Medication (MM)? Third (Aim 3), in the event of insufficient response to one of

the initial treatments, are endpoint results improved more by increasing the dose of that

modality (e.g., adding secondary/adaptive B to initial B (B then B) when necessary) or

Pelham et al. Page 4

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

adding treatment with the other modality (e.g., adding secondary/adaptive M to initial B (B

then M)?

Methods

Participants

One hundred, fifty-two children with ADHD, between the ages of 5 and 12, participated.

Participants were recruited in three cohorts of approximately 50 each via radio

advertisement; direct mail; and referrals from schools, physicians and mental health

providers. Recruitment occurred during the spring and summer of 2006, 2007, and 2008,

with treatment commencing in September of each year and continuing throughout the school

year.

Exclusionary criteria included: (1) Full Scale IQ below 70; (2) history of seizures or other

neurological problems and/or medication to prevent seizures; (3) history of other medical

problems for which psychostimulant treatment may involve considerable risk; (4) childhood

history or concurrent diagnosis of pervasive developmental disorder, schizophrenia or other

psychotic disorders, sexual disorder, organic mental disorder, or eating disorder; (5) lack of

functional impairment; and (6) placement in special education classrooms.

After screening and informed consent, parents and teachers completed a number of

instruments to determine diagnosis and study eligibility. To determine ADHD diagnosis,

parents and teachers completed the Disruptive Behavior Disorders (DBD) Rating Scale

(Pelham, Gnagy, Greenslade, & Milich, 1992). The DBD RS is a list of the DSM symptoms

of ADHD, oppositional-defiant disorder (ODD) and conduct disorder (CD), updated for

DSM-IV, and rated as not at all, just a little, pretty much, or very much. In addition, parents

completed a semi-structured DBD interview consisting of DSM-IV symptoms of ADHD,

ODD, and CD with supplemental situational probes (available from the first author). Parents

and teachers completed the Impairment Rating Scale (IRS: Fabiano et al., 2006), which asks

parents and teachers to evaluate on a six point Likert scale the degree to which a child is like

a typical child and needs no treatment or has extreme problems that definitely require

treatment or special services in five areas of function—relationship with parents/teachers,

relationships with peers/siblings, academic progress, general classroom/family functioning,

and overall functioning. Two clinicians independently reviewed all screening instruments

and made diagnoses based on the DSM-IV rules, counting a symptom as present when

endorsed by either teacher or parent (pretty much or very much on the DBD or parent

interview). Impairment also had to be present in any domain, as indicated by cutpoints on

the Impairment Rating Scale (IRS; Fabiano et al., 2006). In case of disagreement, a third

clinician reviewed the file to determine final diagnosis. Eighty percent of the children met

criteria for ADHD-Combined Type, with 15% Predominately Inattentive and 5%

Predominately Hyperactive/Impulsive. Comorbid rates of ODD and CD are shown in Table

1, along with demographic and descriptive information. None of the ADHD diagnoses and

only 2% of the ODD and CD cases required a third reviewer to confirm diagnosis.

The sample size was determined using data from our previous study in a controlled setting

(Fabiano et al., 2007; Pelham et al., 2014) to estimate effect sizes for the first-stage

Pelham et al. Page 5

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

treatments used in this study. These calculations determined that a sample size of 150 would

result in at least 80% power for testing first-stage differences of at least 0.5 standard

deviations when testing at a 0.05 level of significance. Recruitment and participant flow are

illustrated in Figure 1. Six participants withdrew prior to the end of the study; 146 children

completed the study assessments (96%). Three families withdrew because they did not wish

to use medication and three withdrew because teachers refused participation upon initial

contact after the family had been randomized. In the context of this multiple-randomization

design, early withdrawal results in missing data on the group membership variable (i.e.,

responder versus non-responder). Although there are methods that can address this particular

challenge (e.g., Shortreed, Laber, Scott-Stroup, Pineau, & Murphy, 2014), the subsequent

analyses utilized only the 146 completers despite the use of multiple imputation to address

missing data.

Design

Figure 2 illustrates the study design. Participants were randomly assigned to one of two

initial treatments that were initiated at the beginning of the school year: low-dose medication

for school hours only—Medication First (MedFirst) —or low-intensity clinical behavioral

intervention consisting of weekly behavioral parent training groups (BPT) and a school

consultation to establish a Daily Report Card (DRC: Jacob & Pelham, 2000; available at

http://ccf.fiu.edu; Volpe & Fabiano, 2013)—Behavior First (BehFirst). Eight weeks of

treatment were then provided to allow for sufficient time to implement behavioral treatments

and medication and to measure their impact, after which each participant’s response was

measured according to the procedures described below. If a child experienced continued

impairment in the school and/or home setting—that is, insufficient response to the initial

treatment—then a second randomization occurred. At this point, one of two treatment

strategies was employed in the setting(s) where impairment was present: (1) increase the

dose/intensity of the initial treatment or (2) add the other treatment for a combined treatment

modality. Children who responded to the initial treatment condition were maintained on that

condition and monitored monthly; if their performance deteriorated at any time during the

school year, then the second treatment randomization occurred at that time. Children’s

progress continued to be monitored and their secondary treatment condition was tailored

adaptively (initial treatment was not tailored). For example, a child who began treatment

with a 10-mg dose of medication and was re-randomized to receive behavioral treatment

stayed on the initial 10-mg dose for the remainder of the school year, and subsequent

changes were made only to the adaptive behavioral part of the treatment. Treatments,

evaluations of response, and treatment adjustments were made independently for the home

and school settings, which afforded independent evaluations of need for treatments,

adherence, uptake, and effectiveness at home and at school.

Assessing need for additional treatment—Each month, parents and teachers

completed ratings on a study-specific version of the IRS. The IRS was modified to ask

whether, given the treatment currently in place, the child needed additional treatment, with

responses ranging from 1 (definitely not) to 5 (definitely yes). If a rater responded probably yes or definitely yes in any domain, a study staff member called the rater to ask follow-up

questions about the child’s impairment to ascertain whether the rating indicated true need for

Pelham et al. Page 6

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

additional services, and to ensure that the impairment could be addressed with the available

treatments (e.g., clinicians ruled out that a significant life event may have triggered a

temporary increase in problem behavior or that comorbid learning problems may have

accounted for impairment in academic progress).

As an objective measure of response to intervention in the school setting, teachers also kept

records from an individualized target behavior evaluation (ITBE; Pelham, Fabiano &

Massetti, 2005). The ITBE is sensitive to treatment effects, can be implemented by general

education classroom teachers, and is individualized to children’s areas of impairment

(Fabiano, Vujnovic, Naylor, Pariseau, & Robins, 2009; Fabiano et al., 2010). During the first

few weeks of school, a study case manager met with each child’s teacher to establish target

behaviors (e.g., work completion, complying with teacher directions, behavior toward peers)

and criteria for what the teacher considered success on that target behavior. ITBE goal

attainment percentages were computed across class periods each day, and weekly averages

were calculated for evaluation. ITBE results were not shared with children or parents for

children in the MedFirst group. For children in the BehFirst and adaptive behavioral groups,

the ITBE doubled as a DRC and was sent home to parents, who provided contingent

consequences at home.

At the 8-week point and monthly thereafter, the study team met to discuss each case. If

monthly IRS ratings indicated impairment, the study team ensured that the impairment was

related to an appropriate target of study treatments. Two clinicians who were not directly

involved in the child’s treatment and were unaware of the initial treatment condition were

required to agree that additional treatment was necessary based on the teacher or parent IRS

before the child could be rerandomized. In the school setting, ITBE performance was

simultaneously evaluated; if weekly averages consistently fell below 75%, and need for additional treatment was also indicated on the IRS, then additional treatment was

considered. Finally, if a child was in immediate danger of class failure or school suspension,

these factors were taken into account in treatment decisions.

For the children who were rerandomized, monthly treatment decisions were made regarding

additional dose increases or adjustments to the behavioral treatment. These decisions were

made using the same criteria as for initial response. Treatment recommendations were

tailored to specific domains of impairment as described below. Parents were able to decline

treatment recommendations for medication or additional behavioral services, but

recommendations were reoffered monthly if indicated. All treatment recommendations, the

reasons for them, and records of treatments received were documented.

Treatment Descriptions

Table 2 lists the components of the low and high dose medication and behavioral treatments.

Children were initially randomized to a dose of behavioral or medication treatment, with

additional treatment added, if indicated, based on a second randomization (see Figure 2).

Initial Treatments—For children assigned to the BehFirst condition, parents received an

8-session, group parent training program using the Community Parent Education Program

which has been extensively used with ADHD children (COPE; Cunningham, Bremner, &

Pelham et al. Page 7

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

Secord-Gilbert, 1998); children participated in concurrent group social skills training

sessions, modified after a recreational period in the Summer Treatment Program (STP;

Pelham et al., 2010). Prior to the first group parenting session, parents received an individual

session to establish a home reward system for the DRC. The case manager also conducted

three brief consultation visits with the child’s teacher regarding standard classroom

management strategies. This included an initial review of the teacher’s classroom

management practices, discussion of basic classroom management, including praising

appropriate behavior, planned ignoring, and appropriate commands, as well as procedures

related to implementing a DRC. DRCs were sent home each day and parents provided daily

and weekly rewards for good performance at school. Following the initial 8-week treatment

period, monthly parent-training booster sessions with a focus on maintenance and problem-

solving were offered for the remainder of the school year. The case manager also

communicated with the teacher each month regarding adjustments to the DRC and the basic

classroom management interventions that were in place.

For children assigned to the MedFirst condition, a dose equivalent to 0.15 mg/kg/dose b.i.d.

of immediate release methylphenidate was calculated. In order to separate home and school

settings for assessment and interventions, an 8-hour extended release preparation of

methylphenidate (MPH) was used for the school setting only. For most children (92%), this

was 10 mg per day of the extended release MPH preparation; for the remainder, their initial

dose was 20 mg daily. School doses were administered by parents in the morning prior to

school, and home meds were administered after school and on weekend mornings. The 0.15

mg/kg dose was selected based on data from controlled studies (Fabiano et al. 2007, Pelham

et al., 2005; Pelham et al., 2014) showing significant effects over placebo that are similar to

a low intensity behavioral intervention with very good tolerability. Side effects were

monitored weekly for the first two weeks of medication administration and monthly

thereafter; spontaneous reports of side effects were also collected. Any time a child

experienced moderate or severe side effects, the study physician made dosing adjustments if

necessary. The case manager also adjusted the ITBE for children in this group as needed

monthly based on teacher report.

Secondary (Adaptive) Treatments—For children who began with behavioral

treatment and were rerandomized to receive secondary/adaptive behavioral treatment (B-

then-B), more intensive standard behavior management procedures were implemented first

to address individualized areas of impairment. At school, these included school-based

rewards for DRC performance, classwide reward contingency systems, intensive classroom-

based contingency programs administered by the teacher or a paraprofessional, and time-out

procedures. Home-based DRCs and individual parent-training sessions were introduced in

the home setting. Other interventions were then added to address specific areas of child

impairment (See Table 2).

For MedFirst children who began with medication and then were rerandomized to

secondary/adaptive behavioral intervention (M-then-B), the standard initial behavioral

treatments were implemented first (i.e., group parent training, DRC consultation). After

eight weeks, the additional services described above were added according to the child’s

continued impairments and need for tailored behavioral treatments.

Pelham et al. Page 8

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

For children who began with medication treatment and were rerandomized to receive

secondary/adaptive medication (M-then-M), adjustments were made in two ways. First, the

morning dose of the extended release MPH preparation given on school days could be

increased if problems continued at school. Second, an after-school dose of immediate release

MPH could be added to the child’s regimen if home behavior or homework completion were

impaired (cf. Greenhill et al., 1996). In addition, MPH could be added for weekends. Parents

also had the option of switching to a 12-hour formulation if the criteria for additional

treatment were met in both settings.

For children who began with behavioral treatment and were rerandomized to receive

secondary/adaptive medication (B-then-M), medication could be added as above for school,

home, or both settings. Performance was evaluated monthly and adjustments were made

taking into account impairment level and side effects.

Primary Dependent Measure

Classroom rule violations—As it is commonly regarded as the gold standard in

assessments of treatment outcome for ADHD children in school settings, we used objective

observation of student behavior in the classroom context as our primary dependent measure

(Fabiano et al, 2009; Pelham, Fabiano, & Massetti, 2005). Every 4–6 weeks, independent

observers visited the children’s classrooms and conducted 40-min. direct observations

during academic tasks. Observers used the Student Behavior/Teacher Response code

(available from first author), which includes observations of children’s rule-breaking

behaviors (i.e., disrespect toward others, noncompliance with teacher requests, disrupting

others, leaving seat without permission, inappropriate use of materials, speaking out without

permission, and off-task behavior) and the teacher’s response to those behaviors (e.g.,

ignoring, providing a reprimand, providing a consequence; Vujnovic et al., 2014). Child

behaviors were coded independently of teacher responses and were coded even if the teacher

did not observe the behavior. Observers watched the entire class and coded behaviors

exhibited by the target child and classmates. Classmates were observed anonymously and

were not identified to the observer. The average number of behaviors exhibited by

classmates was computed to produce a classroom comparison rate used as a covariate in

analyses. The final observation of the school year was used in endpoint analyses because it

corresponded with the time interval during which parent and teacher endpoint ratings were

collected

In order to enhance reliability, observers were required to memorize operational definitions

of behavior categories and completed a training session consisting of role-plays, practice

observations, and classroom observations with an experienced observer. For 21% of the

classroom observations, a second trained observer accompanied the primary observer and

conducted an independent reliability observation of the same classroom. Reliability of the

observations was high, with a correlation of 0.91 (p<0.01), and a mean difference of 2.3 (SD = 2.8; range = 0–17) for the total classroom rule violations. These figures are consistent with

those from previous studies using the same observational code (e.g., Fabiano et al., 2010).

Pelham et al. Page 9

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

Secondary Dependent Measures

Number of out-of-class disciplinary events—Teachers kept daily records of out-

of-class disciplinary events (e.g., being sent to the principal’s office). The number of

reported events was summed over the length of the school year for subsequent analysis.

Parent and teacher ratings—At endpoint, parents and teachers completed the DBD

RS and the Social Skills Rating System (SSRS; Gresham & Elliott, 1989). These measures

have been widely used in studies of ADHD and have published psychometric information.

Tracking of Treatment Fidelity

Attendance records were kept for all treatment sessions, and clinicians recorded all meetings

and contacts with teachers and parents. Medication dispensing records were kept. Parents

returned all unused pills at each medication visit, and pill counts were conducted to

determine the number of pills used.

To ensure fidelity with the behavioral treatments, all treatment components were manualized

and procedural checklists were developed for all parent and teacher sessions. Clinicians met

weekly with supervisors to review records of their sessions, and supervisors provided

feedback as necessary. At each classroom observation, the observer recorded whether or not

the teacher implemented the prescribed behavioral management procedures during the

observation period.

Missing Data Handling

Missing values were minimal across all study variables and all participants. Outcomes

ranged from 0 to 14% missing: classroom rules violations (98% complete), out-of-class

disciplines (97%), teacher DBD rating (99%), teacher SSRS rating (99%), parent DBD

rating (90%), parent SSRS rating (86%), and final medication doses (100%). At the

participant level, 125 of 146 (86%) participants had complete data for all of the analyzed

outcomes. We used multiple imputation to ensure unbiased estimates, assuming the data to

be Missing at Random (MAR). Here MAR is a plausible assumption given the inclusion of a

large number of covariates, including baseline measures of outcome variables; measures’

values at earlier waves are typically the best predictors of missing values at later waves in a

longitudinal design.

Imputation—In order to accommodate the non-normal distributions of many relevant

variables, we implemented a chained equations approach in R 3.1.3 (R Core Team, 2015)

using the mice package (v2.22; van Buuren & Groothuis-Oudshoorn, 2011) extended by the

countimp package for imputing count variables (v1.0; Kleinke & Reinecke, 2013). As

methodologists recommend an inclusive strategy (Collins, Schafer, & Kam, 2001), the

imputation model included approximately 50 variables: all the variables in the subsequent

analyses, all the sample characteristics listed in Table 1, and baseline measures of outcomes

wherever available. Distributions of all imputed variables were inspected and each was

modeled using normal, predictive mean matching, negative binomial, logistic, and

multinomial regressions, as indicated. Due to the large number of items and their high

correlations (i.e., multiple items from the same measure), imputation occurred at the level of

Pelham et al. Page 10

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

the scale rather than at the level of the item. One hundred imputed data sets were created,

following recent recommendations that using larger number of imputations can minimize

simulation error (White, Royston, & Wood, 2011).

Analysis and pooling—All subsequent analyses were conducted in SAS 9.3. Analysis

was completed separately on each of the 100 imputations according to the procedures

described in subsequent sections. SAS 9.3 PROC MIANALYZE was used to combine

estimates across imputations; all reported estimates represent these combined (or pooled)

estimates.

Analytic Plan

Our analyses largely parallel those described by Nahum-Shani and colleagues (2012); we

direct readers to that article for more details about SMART design analyses. In the present

study, the analysis of treatment outcome data included a series of comparisons to test

different treatment decisions. Each comparison is described below.

Main effect of initial treatment assignment on endpoint outcomes (Aim 1)— End-of-treatment outcomes of those that started with medication (MedFirst group) and those

that started with behavioral treatment (BehFirst group) were compared using regressions

with group membership as a predictor in order to examine whether the initial treatment

modality impacted outcome. In addition, survival analyses were conducted to determine

whether the groups differed in the need for additional treatment and the length of time

before children needed additional treatment.

Pairwise comparisons among SMART-embedded treatment protocols on

endpoint outcomes (Aim 2)—Second, outcomes were compared across each of the four

treatment protocols naturally embedded in the SMART design—BB, BM, MB, and MM.

The first letter denotes that protocol’s initial treatment (first-stage treatment in Nahum-Shani

et al, 2012) and the second letter denotes that protocol’s secondary/adaptive treatment

(second stage treatment in Nahum-Shani et al, 2012), to be implemented in the event of

insufficient response to the initial treatment. For example, the BM protocol entailed starting

the participant with behavioral treatment and then adding medication if and only if there was

insufficient response. It is important to note that the protocols do not reflect the actual

treatment received, but rather the set of rules followed to assign treatment at both stages. For

example, a child who responded to Behavior First and is therefore never rerandomized to

receive secondary/adaptive treatment is included in analyses of the BM protocol even though

he did not receive medication. This idea of being consistent with a particular embedded

protocol is a subtle but important aspect of the SMART design that is discussed in detail

elsewhere (Nahum-Shani et al., 2012). In the present analyses, we used an effects coding

scheme and generalized estimating equations to achieve all the pairwise comparisons of

protocols in a single model using SAS PROC GENMOD with robust standard errors, as

described in the appendices of Nahum-Shani et al. (2012). We also gave weights of 2 to the

responders to first-stage treatment and weights of 4 to the insufficient responders in order to

account for the systematic undersampling of the latter in each protocol due to the second re-

randomization (Nahum-Shani et al., 2012).

Pelham et al. Page 11

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

Comparison of endpoint outcomes for secondary/adaptive treatments given

insufficient response to initial treatment (Aim 3)—Third, supplemental comparisons

were performed within each of the initial treatment arms to determine whether it is better to

augment (i.e., increase the dose of) that treatment or add the other treatment, given

insufficient response to an initial intervention. Thus, responders to the initial treatments were

excluded from these comparisons. These analyses consisted of regressions with group

membership as a predictor that compared (1) B-then-B with B-then-M and (2) M-then-M

with M-then-B.

Normalization rates—Finally, we used the procedure reported in Swanson et al. (2001)

to evaluate normalization of functioning on teacher and parent ratings at the study endpoint.

A score of 1.0 or lower on an aggregate of ADHD and ODD items from the DBD Rating

Scale was used to define normalization. Teacher and parent reports were examined

separately due to the separation of interventions across settings.

Count outcomes—Two dependent variables were counts: observed classroom rule

violations and number of out-of-class disciplinary events. Count outcomes often violate the

assumptions of linear (OLS) regression, so we adapted the SMART analysis procedure to

incorporate negative binomial regression, a robust approach to modeling count outcome

variables (Coxe, West, & Aiken, 2009). Negative binomial regression is related to the more

well-known Poisson regression, but relaxes some assumptions of Poisson regression that are

typically not met (i.e., equidispersion). As with the continuous outcomes, SAS PROC

GENMOD was used for these analyses, with the addition of the negative binomial modeling

and incorporating the OFFSET option to adjust for individual differences in the length of

observation intervals. In the observed classroom rule violations analyses, average peer rule

violations per hour was included as a covariate to control for the general level of disruptive

behavior in each classroom.

Results

Need for Additional Treatment

In the school setting, 67% of the children who began treatment with behavioral interventions

required additional treatment by the end of the school year compared with 47% of the

children who began the school year receiving a low dose of medication (odds ratio or

OR=2.23). Survival analyses indicated a significant group difference; Breslow χ2=7.4, p < .

01.

In the home setting, there was no difference in rate of rerandomization for BehFirst (82%)

and MedFirst (88%) groups; OR=0.63. Almost all children met criteria for additional

treatment in the home setting regardless of initial treatment.

Endpoint Classroom Observations

Tables 3–6 display the results of analyses for classroom rules violations as well as

subsequent outcomes. Comparisons of initial treatment strategy (BehFirst vs. MedFirst)

revealed a significant difference on classroom rule violations, as illustrated in Figure 3.

Pelham et al. Page 12

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

Children who began treatment with behavior management exhibited significantly fewer rule

violations per hour than children who received MedFirst (incidence rate ratio or IRR=0.66,

p<.01). Pairwise comparisons of the four treatment protocols revealed several significant

differences (Figure 3). The BB protocol resulted in fewer rule violations than the BM

protocol (IRR=0.78, p=.054), the MM protocol (IRR=0.56, p<.01), and the MB protocol

(IRR=0.50, p<.001). In addition, the BM protocol resulted in fewer rules violations than the

MB protocol (IRR=0.65, p<.01). For insufficient responders to first-stage behavioral

treatment, increasing the dose with second-stage behavioral treatment resulted in

significantly fewer violations than did adding medication (IRR=0.71, p<.05). For insufficient

responders to first-stage medication treatment, there were no significant differences between

second-stage treatments.

Out-of-Class Disciplinary Events

Comparisons of initial treatment strategy revealed a trend wherein the BehFirst group

displayed fewer out-of-class disciplinary events than the MedFirst group (IRR=0.52, p<.10,

Figure 3). Pairwise comparisons of the four treatment protocols indicated that (Figure 3): the

BM protocol resulted in significantly fewer events than the MB protocol (IRR= 0.16, p<.

001) and the BB protocol (IRR=0.34, p<.05), and the MM protocol resulted in significantly

fewer events than the MB protocol (IRR=0.34, p<.10). For insufficient responders to initial

behavioral treatment, adding medication trended toward resulting in significantly fewer

events than increasing the dose of behavioral treatment (IRR=0.30, p<.10). For insufficient

responders to first-stage medication treatment, increasing the dose with medication

treatment trended toward resulting in fewer events than did adding behavioral intervention

(IRR=0.27, p<.10).

Teacher Ratings

On teacher DBD ratings, no significant differences emerged for ADHD symptoms. For

ratings of oppositional/defiant behavior, the pairwise comparisons of the four treatment

protocols indicated a near significant advantage of the BM protocol over the MB protocol

(d=0.40, p=.06). The supplemental comparisons indicated that for insufficient responders to

first-stage medication, increasing the dose with second-stage medication trended toward

resulting in lower ratings of oppositional/defiant behavior than did adding behavioral

(d=0.61, p<.10).

For Total Social Skills score of the teacher SSRS, there was a trend toward advantage of the

BM protocol over the MB protocol (d=0.35, p<.10). Other comparisons were nonsignificant.

With regard to normalization of combined ADHD/ODD teacher ratings at endpoint, similar

numbers of the children assigned to MedFirst or BehFirst had mean DBD ratings of 1.0 or

less—69% and 78% respectively. Eighty-four percent of those who responded to first-stage

medication treatment met the normalization criterion, as did 92% of those who responded to

first-stage behavioral treatment. For those needing additional treatment, 63% of the M-then-

M group was normalized, compared to 61% of the B-then-B group, and 38% of the M-then-

B group compared to 80% of the B-then-M group.

Pelham et al. Page 13

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

Parent Ratings

As with teachers, there were no significant differences on ADHD ratings in any

comparisons. For ratings of oppositional/defiant behavior, pairwise comparisons of the four

treatment protocols revealed a significant advantage of the BM protocol over the MB

protocol (d=0.56, p<.05) and BB protocol (d=0.38, p<.10), as well as a trend advantage of

the MM protocol over the MB protocol (d=0.40, p<.10). The supplemental comparisons

indicated that for insufficient responders to first-stage behavioral, adding second-stage

medication trended toward resulting in lower ratings of oppositional/defiant behavior than

did increasing the intensity of behavioral treatment (d=0.45, p<.10). Likewise, for

insufficient responders to first-stage medication, increasing the dose with second-stage

medication trended toward resulting in lower ratings of oppositional/defiant behavior than

did adding behavioral (d=0.46, p<.10). There were no significant differences in any

comparisons of the Total Social Skills score of the parent SSRS.

With regard to normalization of ADHD/ODD parent ratings at endpoint, 31% of the

MedFirst and 39% of the BehFirst, groups met criteria for normalization. Two-thirds of

those who responded to first-stage medication treatment met the normalization criterion, as

did 54% of those who responded to first-stage behavioral treatment. For those needing

additional treatment, 34% of the M-then-M group was normalized, compared to 30% of the

B-then-B group, and 18% of the M-then-B group compared to 40% of the B-then-M group.

Treatment Received

For those who began with BehFirst, 3% of the families declined parent training. Remaining

parents attended an average of 6 of the 8 group sessions (median=7, mode=8), 69% attended

an adequate dose of parent training (cf. MTA Cooperative Group, 1999), and 31% attended

at least one booster session after the initial parent training (see Figure 4). All scheduled

teacher meetings were completed, and DRCs were established for all but one child in the

BehFirst condition. For children in B-then-M, 13 parents (21%) declined the initiation of

medication.

At the initial 8-week assessment point, 9% of the MedFirst families had declined

medication. Of those who accepted medication, they were medicated on 97% of their school

days. For those in MedFirst rerandomized to the M-then-B group, 60% of these parents did

not attend any of the assigned group parent training sessions (mean=1.9, median=0,

mode=0); only 11% of these families received an adequate dose of parent training, and only

11% attended at least one booster session (Figure 4). At school, all required teacher

meetings were completed and school DRCs were established.

In the adaptive behavioral treatment arms (either M-then-B or B-then-B), 11 children

attended Saturday Treatment Program sessions, 3 received extra academic tutoring, and 13

received additional intensive interventions at school (e.g., the good behavior game initiated

as a classwide intervention) administered by the teacher or a paraprofessional as part of the

adaptive behavioral treatment condition. The remainder received either additional standard

teacher consultations to establish higher-intensity teacher-delivered consequences such as

school-based rewards and class-wide contingencies or individual parent sessions to improve

Pelham et al. Page 14

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

parenting skills or establish more intensive parent-delivered interventions at home. Eleven

families assigned to behavioral treatment took medication outside of the protocol. Five of

these families used the medication for only 1–2 months before stopping the medication.

To determine whether beginning treatment with behavioral intervention would decrease the

dose of medication required for a child treated with medication in school, school-day dosing

in mg/kg/dose equivalent was compared for the two groups involving adaptive medication:

B-then-M and M-then-M at endpoint. At the end of the school year, 24% of the B-then-M

group was unmedicated at school compared with 5% of the M-then-M group (that is, parents

either did not start or elected to stop medication). Of those who were medicated, children in

B-then-M were taking significantly lower doses at school (M=0.21 mg/kg/dose, SD=0.10)

than M-then-M (M=0.28, SD=0.14), F(1, 70)=4.26, p<.05. At home, 39% of the M-then-M

group and 35% of the B-then-M group were unmedicated (parents either did not start or

elected to stop medication). For those who were medicated at home, doses were not

significantly different: (B-then-M: M=0.22, SD=0.10; M-then-M: M=0.21, SD=0.12).

Discussion

This study addressed three key questions: first (Aim 1), does it produce better outcomes on

objective classroom measures and teacher and parent ratings to begin treatment with a low

dose of (a) medication treatment or (b) behavioral treatment? Second (Aim 2), what is the

most effective treatment protocol, or pattern of first-stage treatment and conditional second-

stage treatment among the four imbedded SMART treatment protocols? Third (Aim 3), in

the event of insufficient response to a specific initial treatment, is it more effective to

increase the dose of that modality or add treatment with the other modality? All groups were

functioning relatively well at endpoint, as was expected given that two effective treatments

were compared. However, there were important differences in outcomes, as a result of the

initial treatment assignment and the protocol followed. Our findings provide the following

answers to the three questions/Aims noted above as follows:

1. Beginning treatment with a low dose of behavior modification resulted in

significantly lower rates of observed classroom rule violations and a trend for

out-of-class disciplinary events relative to beginning with a low dose of

medication.

2. The best of the four specific treatment protocols began with behavioral

treatment and then added medication in the event of insufficient response

(BM). The worst protocol began treatment with medication and added

behavioral treatment in the event of insufficient response (MB). The BB and

MM protocols produced outcomes in between these two and were often

comparable, though BB was superior to MB and MM on the primary outcome

variable.

3. In the event of insufficient response to initial behavioral treatment, increasing

the intensity of behavioral treatment (B-then-B) was significantly superior on

the primary outcome (classroom rule violations); adding medication (B-then-

M) had nominal advantages on several other outcome variables, two of which

Pelham et al. Page 15

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

were trends. In the event of insufficient response to medication, increasing

dose of medication (M-then-M) was nominally superior to adding behavior

modification (M-then-B) on every measure with small to moderate effect sizes,

three of which were trends.

These results have clear implications for treatment for children with ADHD in mental

health, primary care, and school settings, and we discuss each in turn.

With regard to our first question, beginning school-based treatment with a low dose of

behavior modification (eight sessions of group parent training plus establishing a DRC at

school with home rewards) resulted in functioning in the school setting on key outcome

measures that was comparable to or better than beginning school-based treatment with a low

dose of stimulant medication. Notably, the low dose of behavior modification was a superior

starting strategy on the primary outcome measure, direct observations of classroom behavior

(66% as many rule violations), as well as the frequency of out-of-class discipline (54% as

many incidents). Although teacher ratings did not differentiate BehFirst from MedFirst,

ratings of oppositional behavior decreased by more than 50% from baseline in the BehFirst

group, and teachers rated 78% of children in the BehFirst group as normalized. For 33% of

the BehFirst children, the low dose of behavioral intervention was sufficient treatment in

school for the entire school year.

There were no differences between the groups in the numbers of children who needed

additional treatment at home, with more than 80% of both groups meeting criteria for

rerandomization (see discussion below). Similarly, there were no significant differences

between initial treatments on parent ratings of symptoms, oppositional behavior, or social

skills.

Interestingly, compared to the 33% who did not need additional school-based treatment in

BehFirst, nearly two-thirds more, 53% of children in the MedFirst initial assignment did

sufficiently well with the low dose of medication that they did not require additional

treatment. Further, a substantial portion of the children assigned to the BM embedded

protocol (24%) were not taking medication at endpoint—far more than the MB protocol. In

other words, although the combined treatment protocol (MB) within the MedFirst arm

contained more medicated children than did the combined treatment protocol (BM) within

the BehFirst arm, and although there were more initial responders in the MedFirst group

than in the BehFirst arm, MedFirst remained inferior to BehFirst as an initial treatment

condition.

These differences in the effects of the initial intervention may be linked to differences in

treatment uptake of parent training, as shown in Figure 4. Engagement in the parent training

groups and the booster sessions was dramatically reduced in the MedFirst families relative to

the BehFirst families. Indeed, most BehFirst parents attended the majority of BPT sessions

and received an adequate “dose” of BPT, while only a small minority of MedFirst families

who were assigned to BPT as a secondary/adaptive intervention attended BPT. In other

words, the provision of medication before the initiation of parent training was associated

with greatly reduced rates of engagement in parent training, and presumably worse

functioning at school and home. This finding parallels those reported in the STAR*D study

Pelham et al. Page 16

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

of antidepressants for adults with major depression: 71% of adults that were insufficient

responders to SSRI treatment did not choose to pursue subsequent cognitive behavior

therapy (Wisniewski et al., 2007). Perhaps parents who began with behavioral treatment

were more motivated to engage because they had not already dealt with several of weeks of

problem behavior at school without having received the parenting toolkit provided in BPT.

Alternatively, perhaps parents who began with medication, which requires minimal effort

and time, were reluctant to participate in more effortful and time-consuming parent training,

a major portion of which is learning to provide home backup for the school DRC. It is

possible that the teachers were less engaged in second-stage behavioral interventions

following initial medication for the same reasons as parents. Other researchers have reported

difficulties with engagement and attendance in behavioral treatments for ADHD (see for

example Barkley et al., 2000). Additional research is necessary to elucidate the mechanisms

of this problem with treatment engagement. Whatever the mechanism, the clinical

implications are quite clear: if providers intend for the parents of ADHD children to receive parent training and for teachers to provide “extra” classroom management for the child (i.e., for the child to receive multimodal treatment) but start treatment with medication, they reduce the likelihood of engagement in behavioral treatment and thus negatively impact treatment outcome. Unfortunately, standard practice among physicians is to provide

medication immediately rather than delay it until the completion of parent training. The

results of the present study suggest this is a poor strategy.

With respect to our second question/Aim—which of the four embedded treatment protocols

produces the best outcomes?—it should be noted that each initial assignment is associated

with two embedded protocols, specifically those that began with that particular modality

(i.e., for BehFirst, the BB and BM protocols). With this in mind, the comparisons of

treatment protocols suggest that the primary driver of the first-stage treatment main effect

was the discrepancy between the two combined protocols, BM and MB. As Table 4 shows,

the two protocols involving increasing the initial intervention (BB and MM) produced

generally comparable results (though BB was superior to MM on classroom observations),

but the BM combined-treatment protocol was significantly more effective than the MB

protocol. The former produced the best outcomes on all but two variables, while the latter

produced the worst outcomes on all but two variables. Children following a combined

treatment protocol that began with behavioral treatment were superior on measures of

classroom observations, disciplinary actions, and teacher and parent ratings of ODD. Not

surprisingly, the normalization rates on teacher ratings in the school setting in the children

receiving multimodal treatment was double for the B-then-M (80%) versus M-then-B (38%)

groups, whereas normalization rates for the B-then-B and M-then-M protocols were nearly

identical (61% and 63%). This finding illustrating the superiority of BM over MB has

substantial clinical implications, shedding light on how sequencing of intervention can

enhance (or inhibit) engagement within an evidence-based intervention. For example, the

results of the present study suggest that the failure to begin behavioral treatment before

medication may have contributed to the relatively small advantage of combined treatment to

medication alone in the MTA study.

With respect to our third question/Aim, how to best augment treatment given insufficient

response to initial treatment, Table 5 shows that a low dose of medication is a useful

Pelham et al. Page 17

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

adjunctive intervention to add to initial behavioral treatment in the case of insufficient

response, as is increasing the intensity of behavioral intervention. Both additions were

helpful on a range of measures. In contrast, given insufficient response to medication,

increasing the dose of medication was superior across measures to adding behavioral

treatment (Table 6). This finding has important clinical implications. Once medication has

been employed, it appears that only a higher medication dose results in continued

improvement when more treatment is needed. As discussed above, the reasons for this may

be related to failure of parents (or teachers) to engage in behavioral treatments when they

follow medication. It should be noted that these comparisons involved small Ns, and power

to detect differences was limited. Further, a treatment regimen that includes only medication

is not a viable long-term treatment strategy for ADHD children, as it confers no long-term

benefit (Molina et al., 2009).

Furthermore, with regard to the secondary treatments involving adaptive medication,

children in the B-then-M group were taking significantly lower doses of medication at

school than children in the M-then-M condition. This finding indicates that beginning

treatment with behavior modification serves to decrease the necessary dose when medication

is used, which will result in lower levels of dose-related side effects (cf. Swanson et al,

2006).

Taken altogether, our results replicate and extend in the school-year environment what we

have reported in earlier studies conducted in analogue summer program settings (Fabiano et

al., 2007; Pelham et al., 2005; Pelham et al., 2014). Namely, a low dose of behavioral

treatment—in this study eight sessions of large group BPT and establishing a DRC at school

— is effective and sufficient for a substantial number of children with ADHD in school,

recreational, and home settings. Further, a low dose of medication (.15 mg/kg/dose b.i.d.)

was sufficient in the school setting for 53% of the children. Low doses of the two modalities

in combination were very effective for insufficient responders, but only when the behavioral treatment came first. Neither our previous studies nor the MTA sequenced interventions, but

these results provide clear guidance about which sequence should be followed when

implementing combined treatment—BM rather than MB.

This is the first study to our knowledge that has addressed the effectiveness of such low-dose

interventions as a starting treatment for ADHD implemented in a community/school/clinic

setting. Low dose medication was sufficient in the school setting for a year for nearly half of

the children, but providing it first limited the effectiveness of additional behavioral treatment

when necessary. Further, the cost of the MedFirst condition and its protocols in the study were far higher than BehFirst and its associated protocols (Page et al, this issue). Thus, the

study demonstrates that starting with a low dose of behavioral treatment and either

enhancing behavioral treatment or adding medication when necessary produces better

outcomes and is a far less costly approach to treatment for ADHD than starting with

medication (Page et al, this issue). Others have found increased side effects and reduced

tolerability as the dose and duration of medication increases (Barkley, McMurray,

Edelbrock, & Robbins., 1990; Stein et al., 2003; Pelham, et al, 1999; Swanson et al. 2006).

An adaptive approach in clinical practice that begins with low intensity behavior

modification and increases intensity or adds medication adaptively would appear to be the

Pelham et al. Page 18

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

treatment approach of choice for children with ADHD. The MTA had previously shown that

medication dose escalations over time are less necessary when multimodal treatment is

being implemented compared to medication alone (Vitiello et al., 2001), but the present

results extend that finding to considerably lower and less costly (Page et al., this issue) doses

of both medication and behavioral treatment than employed in the MTA and most other

studies in the ADHD treatment field.

Importantly, the adaptive nature of the approach to behavioral intervention was effective in

this study in producing very positive outcomes with relatively low intensity interventions for

most children and enhanced interventions for a small subset. For example, the adaptive

behavioral treatments employed in the school setting were carried out by the general

education teacher without additional intervention staff in two-thirds of the cases, and in only

six cases was an intensive paraprofessional-based program implemented. In contrast, in the

MTA a half-day paraprofessional—a very costly intervention—was provided for nearly a

full semester for all participants regardless of need. The present results suggest that that was

unnecessary for the vast majority of the children. These findings illustrate the utility of the

adaptive treatment approach, in which children only receive the types and levels of treatment

they require based on individual impairment. As discussed in the companion to this paper

(Page et al., this issue), the BB protocol was the least costly of the four protocols and the

BM protocol a close second. Thus, our effectiveness results and costs show a far different

picture than presented in the only other comparative study of cost-effectiveness in the

ADHD literature (cf. Jensen et al, 2005). These findings have important implications for the

public health system and insurance companies with regard to treatment costs for ADHD and

call for a reassessment of federal, insurer, and medical society recommendations on

treatments for ADHD, which currently prioritize medication and limit the extent to which

behavioral treatments can be utilized.

Limitations

It is important to note that this was an effectiveness study carried out in the natural

environment and therefore strict experimental control over the behavioral interventions

could not be exerted. Given the prevalence of classroom management training in schools,

teachers in the medication–only group were no doubt routinely implementing behavioral

strategies to manage their classrooms, and parents in the medication-only group may have

been implementing behavioral practices such as time out. Some of the lack of differences in

behavioral treatments may thus be due to the natural presence of behavioral treatments in

school and home settings. In addition, it was not possible to collect measures of parents’ in-

home implementation of procedures such as rewarding the Daily Report Card. Furthermore,

although observers completed checks of treatment integrity and fidelity during their

observations, they were often unable to observe teachers’ implementation of specific

procedures that were to be implemented (e.g., tracking DRC targets and giving feedback to

the child). We therefore were unable to calculate specific data for the fidelity of teacher-

delivered interventions. In contrast, medication was provided with greater experimental

control, with dosing practices varying some from what is done in routine clinical practice in

order to systematically assess sequencing effects across settings. For example, initial

Pelham et al. Page 19

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

medication treatments focused on school only, with evidence of objective impairment

required to be eligible for additional treatment at school or at home.

As noted above, many more children met criteria for additional treatment in the home setting

than in the school setting. In part this may be due to the initial treatment conditions. For

example, medication was initially provided only at school for MedFirst.. This approach

exposed those families to the impact of medication on their child at school and may have led

parents to rate their child as needing medication at home in order to obtain medication for

the home setting. The BPT program for BehFirst families was a brief, group-based

program--as opposed to the individually developed DRC for each child--and was sufficient

for only a subset of families. Others needed more individually-focused BPT, although the

amount required was relatively little except for a small subset of families. The lack of a

parallel, home-based DRC criterion (like the one used in the school setting) for additional

treatment also made it easier for children to meet criteria for additional treatment at home

relative to the criteria for allocating additional treatment at school. That is, parents simply

needed to indicate that their child was having problems and needed more treatment, whereas

teacher indication of need and ITBE target goal attainment rate below 75% were required at

school. This may also explain the somewhat contrary findings that BehFirst resulted in

superior outcomes relative to MedFirst, but the majority of children in the study met criteria

for rerandomization in the home setting. Future studies that utilize similar ITBE goal

attainment strategies in the home in addition to parent ratings may yield different outcomes

that are more similar to that obtained in the school setting in this study.

Another limitation relates to our study design, which included a maximum of two

randomizations per child. Ideally additional decision points might be included. For example,

in our protocol a child in need of adaptive second stage behavioral treatment could receive

ad lib treatment, as opposed to systematically limited, incrementally larger “doses” of

behavioral intervention, which might have been sufficient. Alternatively, for a child who is

still doing poorly after the first rerandomization, another opportunity to cross over to the

other treatment might be considered. For example, rather than a temporary classroom

paraprofessional, medication might have been considered at a third randomization for the

small number of children who require that level of assistance. This may be particularly true

in the case of nonadherence to the assigned treatments. The sample size required for

additional decision points precluded such considerations for the current study. Inclusion

criteria also required attendance within general education classrooms, so whether these

results generalize to self-contained special education classrooms is not known.

A final limitation was statistical power to address secondary aims. The study was fully

powered only for examination of the main effect of first-stage treatment (question/Aim 1).

We did not expect so many children to respond to first stage treatments and not need

additional treatment, so we did not plan for a larger N. Thus our power was reduced for

pairwise comparisons of the embedded treatment protocols (question/Aim 2), and then

further reduced for the comparison of treatments among insufficient responders

(question/Aim 3). Thus, numerous small to moderate effect sizes did not achieve statistical

significance but might have with a larger sample.

Pelham et al. Page 20

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

Future Research

Finally, the adaptive methodology employed is promising for future studies of interventions

for ADHD in the pursuit of treatment tailoring for individual differences in functional

deficits. Replication of these results with a larger sample would afford better power and the

opportunity to investigate mediators and moderators for the treatment results that we

reported herein, including individual differences in comorbidity and impairment. For

example, why did parents whose children received medication first have such dramatically

reduced rates of uptake of parent training and associated poor outcomes relative to the other

protocols? Why did so many more children meet criteria for rerandomization at home

compared to school? Since combined treatment starting with behavior modification was so

effective, a natural follow-up question is how psychologists and other psychosocial mental

health and school-based providers can collaborate with M.D. prescribers in practice settings

to implement the conjoint strategies that were shown in this paper to be effective. Finally,

how might these interventions and approaches have to change to be effective with samples of

ADHD children both younger and older than our elementary-aged sample?

Clinical Implications

The results have direct relevance for clinical practice. The relatively low-dose-treatment

strategies that we employed are implementable in community mental health, primary care,

and school settings. The results suggest that practitioners should initiate treatment with low

doses of intervention, increasing intensity only when indicated. Many children will respond

sufficiently to low doses of initial treatments. Further, practitioners who initiate treatment

with behavioral intervention (group parent training and a school DRC) will produce better

outcomes for their patients than those starting with medication. In the face of inadequate

response to initial behavioral treatment, either more intensive behavioral intervention or the

addition of a low dose of medication for school hours will produce improved patient and

family outcomes. In contrast, if medication is the first stage treatment and is insufficient,

adding behavioral treatment is not an effective treatment option--outcomes are worse than

other strategies and parent engagement in subsequent parent training is very poor. Physicians

in particular need to be aware of these facts—simply advising their patients who have started

medication to go to a psychologist for parent training will not result in the desired outcome

—that is, a multimodal intervention. This paper and our companion paper on the costs of

these interventions strongly suggest a reconsideration of the current practice of relying on

medication as first line and typically sole treatment for many ADHD children. These results

suggest that a stepwise approach to treatment, starting with low doses of behavioral

treatment and increasing in intensity or adding medication only if necessary, would be a

cost-effective public health strategy for treatment of childhood ADHD in school and

community settings.

Acknowledgments

Funding

This research was funded by a grant from the Institute of Education Sciences (R324B060045). Dr. Pelham was also supported in part by grants from the National Institute of Mental Health (MH069614, MH069434, MH092466, MH53554, MH065899, MH62988), the Institute of Education Sciences (R324J060024, LO30000665A), the

Pelham et al. Page 21

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

National Institute of Alcohol Abuse and Alcoholism (AA11873), and the National Institute on Drug Abuse (DA12414, DA12986).

References

AACAP Work Group on Quality Issues. Practice parameter for the assessment and treatment of children and adolescents with attention deficit/hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry. 2007; 46(7):994–921. DOI: 10.1097/chi. 0b013e318054e724

APA Working Group on Psychoactive Medications for Children and Adolescents. Report of the Working Group on Psychoactive Medications for Children and Adolescents. Psychopharmacological, psychosocial, and combined interventions for childhood disorders: Evidence base, contextual factors, and future directions. Washington, DC: American Psychological Association; 2006. Available at: http://www.apa.org/pi/cyf/childmeds/pdf

Barkley RA, McMurray MB, Edelbrock CS, Robbins K. Side effects of methylphenidate in children with attention deficit/hyperactivity disorder: A systematic, placebo-controlled evaluation. Pediatrics. 86:184–192. [PubMed: 2196520]

Barkley RA, Shelton TL, Crosswait C, Moorehouse M, Fletcher K, Barrett S, … Metevia L. Multi- method psycho-educational intervention for preschool children with disruptive behavior: Preliminary results at post-treatment. Journal of Child Psychology and Psychiatry and Allied Disciplines. 2000; 41:319–332.

Barrish HH, Saunders M, Wolf MM. Good Behavior Game: Effects of individual contingencies for group consequences on disruptive behavior in a classroom. Journal of Applied Behavior Analysis. 1969; 2:119–124. [PubMed: 16795208]

Briesch AM, Volpe RJ, Ferguson TD. The influence of student characteristics on the dependability of behavioral observation data. School Psychology Quarterly. 2014; 29:171–181. DOI: 10.1037/ spq0000042 [PubMed: 24274156]

Cohen, J. Statistical Power Analysis for the Behavioral Sciences. 2. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc; 1988.

Collins LM, Murphy SA, Bierman KL. A conceptual framework for adaptive preventive interventions. Prevention Science. 2004; 5:185–196. DOI: 10.1023/B:PREV.0000037641.26017.00 [PubMed: 15470938]

Collins LM, Schafer JL, Kam CM. A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods. 2001; 6(4):330–51. DOI: 10.1037/1082-989X. 6.4.330 [PubMed: 11778676]

Conduct Problems Prevention Research Group. Initial impact of the fast track prevention trial for conduct problems: I. The high-risk sample. Journal of Consulting and Clinical Psychology. 1999a; 67:631–647. DOI: 10.1037/0022-006X.67.5.631 [PubMed: 10535230]

Conduct Problems Prevention Research Group. Initial impact of the fast track prevention trial for conduct problems: II: Classroom effects. Journal of Consulting and Clinical Psychology. 1999b; 67:648–657. DOI: 10.1037/0022-006X.67.5.648 [PubMed: 10535231]

Conners CK. Forty years of methylphenidate treatment in Attention-Deficit/ Hyperactivity Disorder. Journal of Attention Disorders. 2002; 6(Suppl 1):S17–S30. [PubMed: 12685516]

Coxe S, West SG, Aiken LS. The analysis of count data: A gentle introduction to Poisson regression and its alternatives. Journal of Personality Assessment. 2009; 91(2):121–136. [PubMed: 19205933]

Cunningham, CE.; Bremner, R.; Secord-Gilbert, M. The community parent education (COPE) program: A school-based family systems oriented course for parents of children with disruptive behavior disorders. Chedoke-McMaster Hospitals and McMaster University; 1998.

Epstein JN, Kelleher KJ, Baum R, Brinkman WB, Peugh J, Gardner W, Langberg J. Variability in ADHD care in community-based pediatrics. Pediatrics. 2014; 134:1136–1143. DOI: 10.1542/peds. 2014-1500 [PubMed: 25367532]

Evans SW, Owens JS, Bunford N. Evidence-based psychosocial treatments for children and adolescents with attention-deficit/hyperactivity disorder. Journal of Clinical Child and Adolescent Psychology. 2014; 43:527–551. DOI: 10.1080/15374416.2013.850700 [PubMed: 24245813]

Pelham et al. Page 22

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

Fabiano GA, Pelham WE, Coles EK, Gnagy EM, Chronis AM, O’Connor B. A meta-analysis of behavioral treatments for attention-deficit/hyperactivity disorder. Clinical Psychology Review. 2009; 29(2):129–140. DOI: 10.1016/j.cpr.2008.11.001 [PubMed: 19131150]

Fabiano GA, Pelham WE, Gnagy EM, Burrows-MacLean L, Coles EK, Chacko A, … Robb JA. The single and combined effects of multiple intensities of behavior modification and methylphenidate for children with Attention Deficit Hyperactivity Disorder in a classroom setting. School Psychology Review. 2007; 36:195–216.

Fabiano GA, Pelham WE, Waschbusch DA, Gnagy EM, Lahey BB, Chronis AM, … Burrows- MacLean L. A practical measure of impairment: psychometric properties of the impairment rating scale in samples of children with attention deficit hyperactivity disorder and two school-based samples. Journal of Clinical Child and Adolescent Psychology. 2006; 35:369–385. DOI: 10.1207/ s15374424jccp3503_3 [PubMed: 16836475]

Fabiano GA, Schatz NK, Aloe AM, Chacko A, Chronis-Tuscano AM. A review of meta-analyses of psychosocial treatment for attention-deficit/hyperactivity disorder: systematic synthesis and interpretation. Clinical Child and Family Psychology Review. 2015; 18:77–97. [PubMed: 25691358]

Fabiano GA, Vujnovic R, Naylor J, Pariseau M, Robins ML. An investigation of the technical adequacy of a daily behavior report card (DBRC) for monitoring progress of students with attention-deficit/hyperactivity disorder in special education placements. Assessment for Effective Intervention. 2009; 34:231–241.

Fabiano GA, Vujnovic R, Pelham WE, Waschbusch DA, Massetti GM, Yu J, … Volker M. Enhancing the effectiveness of special education programming for children with ADHD using a daily report card. School Psychology Review. 2010; 39:219–239.

Greenhill LL, Pliszka S, Dulcan MK. the Work Group on Quality Issues. Practice parameter for the use of stimulant medications in the treatment of children, adolescents, and adults. Journal of the American Academy of Child & Adolescent Psychiatry. 2002; 41:26S–49S. DOI: 10.1097/00004583-200202001-00003 [PubMed: 11833633]

Greenhill LL, Abikoff HB, Arnold LE, Cantwell DP, Conners CK, Elliott G, … Wells K. Medication treatment strategies in the MTA study: Relevance to clinicians and researchers. Journal of the American Academy of Child & Adolescent Psychiatry. 1996; 35:1304–1313. DOI: 10.1097/00004583-199610000-00017 [PubMed: 8885584]

Gresham, FM.; Elliott, SN. Social Skills Rating System: Parent, teacher, and child forms. Circle Pines, MN: American Guidance Systems; 1989.

Jacob, R.; Pelham, WE. Behavior therapy. In: Sadock, B.; Sadock, V., editors. Comprehensive Textbook of Psychiatry. 7. New York: Williams & Wilkins; 1999. p. 2080-2127.

Jensen PS, Garcia JA, Glied S, Crowe M, Foster M, Schlander M, … Wells K. Cost-effectiveness of ADHD treatments: findings from the multimodal treatment study of children with ADHD. American Journal of Psychiatry. 2005; 162(9):1628–1636. [PubMed: 16135621]

Kleinke, K.; Reinecke, J. countimp 1.0 – A multiple imputation package for incomplete count data. University of Bielefeld, Faculty of Sociology; 2013. [Technical Report]available from www.uni- bielefeld.de/soz/kds/pdf/countimp.pdf

Lavori PW, Dawson R. A design for testing clinical strategies: biased individually tailored within- subject randomization. Journal of the Royal Statistical Society: Series A (Statistics in Society). 2000; 163:29–38. DOI: 10.1111/1467-985X.00154

Molina BSG, Hinshaw SP, Swanson JM, Arnold LE, Vitiello B, Jensen PS. … the MTA Cooperative Group. The MTA at 8 years: Prospective follow-up of children treated for combined-type ADHD in a multisite study. Journal of the American Academy of Child and Adolescent Psychiatry. 2009; 48:484–500. [PubMed: 19318991]

MTA Cooperative Group. 14-month randomized clinical trial of treatment strategies for attention deficit hyperactivity disorder. Archives of General Psychiatry. 1999a; 56:1073–1086. DOI: 10.1001/archpsyc.56.12.1073 [PubMed: 10591283]

MTA Cooperative Group. Moderators and mediators of treatment response for children with attention- deficit/hyperactivity disorder. Archives of General Psychiatry. 1999b; 56:1088–1096. DOI: 10.1001/archpsyc.56.12.1088 [PubMed: 10591284]

Pelham et al. Page 23

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

MTA Cooperative Group. National Institute of Mental Health multimodal treatment study of ADHD follow-up: 24-month outcomes of treatment strategies for attention-deficit/hyperactivity disorder (ADHD). Pediatrics. 2004a; 113(4):754–761. DOI: 10.1542/peds.113.4.754 [PubMed: 15060224]

Murphy SA. An experimental design for the development of adaptive treatment strategies. Statistics in Medicine. 2005; 24:1455–1481. DOI: 10.1002/sim.2022 [PubMed: 15586395]

Nahum-Shani I, Qian M, Almirall D, Pelham WE, Gnagy B, Fabiano GA, … Murphy S. Experimental design and primary data analysis methods for comparing adaptive interventions. Psychological Methods. 2012; 17:457–477. DOI: 10.1037/a0029372 [PubMed: 23025433]

Owens EB, Hinshaw SP, Kraemer HC, Arnold LE, Abikoff HB, Cantwell DP, … Wigal T. Which treatment for whom for ADHD? Moderators of treatment response in the MTA. Journal of Consulting and Clinical Psychology. 2003; 71(3):540–552. DOI: 10.1037/0022-006X.71.3.540 [PubMed: 12795577]

Page TF, Fabiano GA, Greiner AR, Gnagy EM, Pelham WE III, Hart K, Coxe S, Waxmonsky JG, Foster EM, Pelham WE Jr. Comparative cost analysis of sequential, adaptive, behavioral, pharmacological, and combined treatments for ADHD. Journal of Clinical Child & Adolescent Psychology. in press. this issue.

Pelham WE. The NIMH multimodal treatment study for ADHD: Just say yes to drugs alone? Canadian Journal of Psychiatry. 1999; 44:981–990. [PubMed: 10637677]

Pelham WE, Aronoff HA, Midlam J, Shapiro C, Gnagy E, Chronis A, … Waxmonsky J. A comparison of ritalin and adderall: Efficacy and time-course in children with attention-deficit/hyperactivity disorder. Pediatrics. 1999; 103(4):1–14. [PubMed: 9917431]

Pelham WE, Burrows-MacLean L, Gnagy EM, Fabiano GA, Coles EK, Tresco KE, … Hoffman MT. Transdermal methylphenidate, behavioral, and combined treatment for children with ADHD. Experimental and Clinical Psychopharmacology. 2005; 13(2):111–126. DOI: 10.1037/1064-1297.13.2.111 [PubMed: 15943544]

Pelham WE, Burrows-MacLean L, Gnagy EM, Fabiano GA, Coles EK, Wymbs BT, … Waschbusch DA. A dose-ranging study of behavioral and pharmacological treatment for children with ADHD. Journal of Abnormal Child Psychology. 2014; 42:1019–1031. DOI: 10.1007/s10802-013-9843-8 [PubMed: 24429997]

Pelham WE, Burrows-MacLean L, Gnagy EM, Fabiano GA, Coles EK, Wymbs BT, … Coxe S. Behavioral, pharmacological, and combined treatment for ADHD: A dose-ranging, between groups analysis. 2015 Manuscript under review.

Pelham WE, Fabiano GA. Evidence-based psychosocial treatment for attention deficit/hyperactivity disorder: An update. Journal of Clinical Child and Adolescent Psychology. 2008; 37(1):185–214. DOI: 10.1080/15374410701818681

Pelham WE, Fabiano GA, Massetti GM. Evidence-based assessment of attention-deficit/hyperactivity disorder in children and adolescents. Journal of Clinical Child and Adolescent Psychology. 2005; 34:449–476. DOI: 10.1207/s15374424jccp3403_5 [PubMed: 16026214]

Pelham WE, Gnagy EM, Greiner AR, Hoza B, Hinshaw SP, Swanson JM, … Baron-Myak C. Behavioral vs. behavioral and pharmacological treatment in ADHD children attending a summer treatment program. Journal of Abnormal Child Psychology. 2000; 28:507–526. DOI: 10.1023/A: 1005127030251 [PubMed: 11104314]

Pelham, WE.; Gnagy, EM.; Greiner, AR.; Waschbusch, DA.; Fabiano, GA.; Burrows-MacLean, L. Summer Treatment Programs for Attention Deficit/Hyperactivity Disorder. In: Kazdin, AE.; Weisz, JR., editors. Evidence-Based Psychotherapies for Children and Adolescents. 2. New York: The Guilford Press; 2010. p. 277-292.

Pelham WE, Gnagy EM, Greenslade KE, Milich R. Teacher ratings of DSM-III-R symptoms of the disruptive behavior disorders. Journal of the American Academy of Child and Adolescent Psychiatry. 1992; 31:210–218. DOI: 10.1097/00004583-199203000-00006 [PubMed: 1564021]

Power, TJ.; Habboushe, D.; Karustis, JL. Homework Success for Children with ADHD: A family- school intervention program. New York: Guilford Press; 2001.

R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing; Vienna, Austria: 2015. URL http://www.R-project.org/

Pelham et al. Page 24

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

Schultz, BK.; Evans, SW. A Practical Guide for Implementing School-Based Interventions for Adolescents with ADHD. New York, NY: Springer; 2015.

Shortreed SM, Laber E, Scott Stroup T, Pineau J, Murphy SA. A multiple imputation strategy for sequential multiple assignment randomized trials. Statistics in Medicine. 2014; 33(24):4202–4214. [PubMed: 24919867]

Stein MA, Sarampote CS, Waldman ID, Robb AS, Conlon C, Pearl PL, … Newcorn JH. A dose- response ostudy of OROS methylphenidate in children with attention deficit/hyperactivity disorder. Pediatrics. 2003; 112:e404.doi: 10.1542/peds.112.5.e404 [PubMed: 14595084]

Subcommittee on Attention-Deficit/Hyperactivity Disorder, Steering Committee on Quality Improvement and Management. ADHD: Clinical practice guideline for the diagnosis, evaluation, and treatment of Attention-Deficit/Hyperactivity Disorder in children and adolescents. Pediatrics. 2011; 128:2011–2654. DOI: 10.1542/peds.2011-2654

Swanson J, Greenhill L, Wigal T, Kollins S, Stehli A, Davies M, … Wigal S. Stimulant-related reductions of growth rates in the PATS. Journal of the American Academy of Child & Adolescent Psychiatry. 2006; 45:1304–1313. DOI: 10.1097/01.chi.0000235075.25038.5a [PubMed: 17023868]

Swanson JM, Hinshaw SP, Arnold LE, Gibbons R, Marcus S, Hur K. … the MTA Cooperative Group. Secondary evaluations of MTA 36-month outcomes: Propensity score and growth mixture model analyses. Journal of the American Academy of Child and Adolescent Psychiatry. 2007; 46:1003– 1014. DOI: 10.1097/CHI.0b013e3180686d63 [PubMed: 17667479]

Swanson JM, Kraemer H, Hinshaw SP, Arnold LE, Conners CK, Abikoff H, … Wu M. Clinical relevance of the primary findings of the MTA: Success rates based on severity of ADHD and ODD symptoms at the end of Treatment. Journal of the American Academy of Child and Adolescent Psychiatry. 2001; 40:168–179. [PubMed: 11211365]

Van Buuren S, Groothuis-Oudshoorn K. mice: Multivariate imputation by chained equations in R. Journal of Statistical Software. 2011; 45(3)

Visser SN, Danielson ML, Bitsko BH, Holbrook JR, Kogan MD, Ghandour RM, … Blumberg SJ. Trends in the parent-report of health care provider-diagnosed and medicated attention-deficit/ hyperactivity disorder: United States, 2003-2011. Journal of the American Academy of Child & Adolescent Psychiatry. 2014; 53:34–46. DOI: 10.1016/j.jaac.2013.09.001 [PubMed: 24342384]

Vitiello B, Severe JB, Greenhill LL, Arnold LE, Abikoff HB, Bukstein OG, … Cantwell DP. Methylphenidate dosage for children with ADHD over time under controlled conditions: Lessons from the MTA. Journal of the American Academy of Child & Adolescent Psychiatry. 2001; 40:188–196. DOI: 10.1097/00004583-200102000-00013 [PubMed: 11211367]

Volpe, R.; Fabiano, GA. Daily Behavior Report Cards: An Evidence-Based System of Assessment and Intervention. New York: The Guilford Press; 2013.

Vujnovic RK, Fabiano GA, Waschbusch DA, Pelham WE, Greiner A, Gera S, … Buck M. Preliminary psychometric properties of an observation system to assess teachers/ use of effective behavior support strategies in preschool classrooms. Education and Treatment of Children. 2014; 37:323– 346. DOI: 10.1353/etc.2014.0020

White IR, Royston P, Wood AM. Multiple imputation using chained equations: Issues and guidance for practice. Statistics in Medicine. 2011; 30(4):377–399. [PubMed: 21225900]

Wisniewski SR, Fava M, Trivedi MH, Thase ME, Warden D, Niederehe G, … John Rush A MD. Acceptability of second-step treatments to depressed outpatients: a STAR* D report. The American Journal of Psychiatry. 2007; 164(5):753–760. [PubMed: 17475734]

Pelham et al. Page 25

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

Figure 1. Participant Flow

Pelham et al. Page 26

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

Figure 2. Study Design

Pelham et al. Page 27

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

Figure 3. Means on Observed Classroom Rules Violations and Out-of-Class Disciplinary Events as a

Function of Treatment Decisions

Pelham et al. Page 28

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

Figure 4. Parent Training Attendance by Initial Treatment Assignment

Note. Figures for the Medication First families consider only those that were rerandomized

to behavioral treatment (M-then-B, N=35).

Pelham et al. Page 29

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

Pelham et al. Page 30

Table 1

Sample Characteristics

Variable Medication First Behavioral First

Number of participants 74 72

Child Age in Years 8.3 (2.0) 8.5 (1.8)

Child Gender (% Male) 77% 75%

Child Race

White 76% 84%

Black/African American 17% 7%

Other 7% 8%

Child IQ 99.9 (16.2) 99.2 (12.5)

Other Diagnoses

Oppositional/Defiant Disorder 60% 54%

Conduct Disorder 17% 14%

ADHD Symptoms Endorsed

Inattention 7.6 (1.9) 8.1 (1.5)

Hyperactivity/Impulsivity 7.1 (2.2) 6.8 (2.1)

Parent Disruptive Behavior Disorders Rating

ADHD 1.89 (0.61) 1.99 (0.50)

ODD 1.32 (0.67) 1.29 (0.57)

CD 0.26 (0.28) 0.21 (0.20)

Teacher Disruptive Behavior Disorder Rating

ADHD 1.84 (0.62) 1.78 (0.60)

ODD 1.17 (0.84) 0.95 (0.73)

CD 0.45 (0.59) 0.31 (0.44)

Parental Marital Status (% Single Parent) 11% 7%

Highest Parental Education Level

High School Diploma or Less 10% 10%

Partial college or technical training 17% 14%

2-year degree 25% 19%

4-year degree 24% 31%

Graduate training 25% 26%

Previous Medication Treatment 27% 31%

Note. Groups did not differ significantly on any demographic measure.

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

Pelham et al. Page 31

Table 2

Intervention Components

Modality Initial Treatment Secondary/Adaptive Treatment

Medication • 8-hour stimulant equivalent to 0.15 mg/kg methylphenidate b.i.d.

• Increased school dose

• Added evening/weekend doses

Behavioral Treatment

• 8 weekly sessions of group behavioral parent training (concurrent group social skills training for children)

• Monthly booster parent training sessions

• 3 consultation meetings with primary teacher to establish a school-home daily report card

• One individual parent training session to establish home rewards for daily report card

• Group or individual classroom contingency management systems (Barrish, Sauders, & Wolf, 1969)

• Time-out in school

• Tutoring

• Organizational skills training (Schultz & Evans, 2015)

• School-based rewards

• Weekly Saturday social skills sessions (Pelham et al., 2008)

• Homework skills training (Power et al., 2001)

• Paraprofessional-implemented school rewards programs

• Home-based daily report card

Note. The adaptive components listed represent those offered or recommended as-needed based on individual areas of impairment. Not every child received every component of the adaptive treatment.

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

Pelham et al. Page 32

Table 3

Outcomes at Endpoint by Initial Treatment Assignment

Outcome Medication First Behavioral First Effect Size

Classroom rules violations per hour** 12.6 [10.5, 15.3] 8.4 [6.8, 10.3] IRR = 0.66

Out-of-class disciplinary events per school year† 3.1 [1.8, 5.2] 1.6 [0.9, 2.7] IRR = 0.52

Teacher DBD—ADHD 0.98 (.67) 1.00 (.64) d = −0.02

Teacher DBD—ODD 0.59 (.66) 0.45 (.51) d = 0.24

Teacher SSRS Social Skills Total Score 33.9 (9.5) 36.0 (10.5) d = 0.21

Parent DBD—ADHD 1.44 (.64) 1.45 (.62) d = −0.01

Parent DBD—ODD 1.09 (.71) 0.98 (.65) d = 0.16

Parent SSRS Social Skills Total Score 45.2 (10.8) 45.3 (10.7) d = 0.01

Note. IRR=incidence rate ratio, DBD=Disruptive Behavior Disorders Rating Scale (scores are average scale scores, range 0–3), ADHD=attention deficit hyperactivity disorder, ODD=Oppositional defiant disorder, SSRS=Social Skills Rating Scale. Values are means with standard deviations in parentheses (for continuous outcomes) or asymmetric 95% confidence intervals about the mean (for count outcomes). The IRR is the ratio of the event (e.g., rule violation) incidence rate in one group (here, Behavioral First) to the incidence rate in another group (here, Medication First). The other effect sizes are Cohen’s D with pooled standard deviation (equations 2.5.1 and 2.5.2, pp. 66–67, Cohen, 1988), and are listed such that a positive d reflects an advantage of Behavioral First.

† p<0.10,

** p<0.01.

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

Pelham et al. Page 33

Table 4

Outcomes at Endpoint by Treatment Protocol Followed

Outcome BB protocol BM protocol MB protocol MM protocol

Classroom rules violations per hour 7.2† [5.8, 9.0] 9.3a† [7.6, 11.4] 14.3b [11.1, 18.5] 12.7ab [9.0, 18.0]

Out-of-class disciplinary events per school year 2.6ab [1.1, 6.1] 0.9c [0.5, 1.7] 5.5a† [2.4, 12.9] 1.9bc† [0.9, 4.2]

Teacher DBD— ADHD 1.09 (.65)a 0.91 (.61)a 1.02 (.71)a 0.94 (.63)a

Teacher DBD— ODD 0.48 (.55)ab 0.42 (.46)a† 0.69 (.79)b† 0.50 (.50)ab

Teacher SSRS Social Skills Total Score 35.0 (10.8)ab 36.8 (10.0)a† 33.2 (10.7)b† 34.5 (8.2)ab

Parent DBD— ADHD 1.52 (.63)a 1.37 (.59)a 1.54 (.65)a 1.34 (.60)a

Parent DBD—ODD 1.10 (.69)ab† 0.86 (.58)c† 1.23 (.74)a‡ 0.95 (.64)bc‡

Parent SSRS Social Skills Total Score 44.2 (10.0)a 46.4 (11.2)a 45.0 (10.1)a 45.4 (11.3)a

Note. DBD=Disruptive Behavior Disorders Rating Scale (scores are average scale scores, range 0–3), ADHD=attention deficit hyperactivity disorder, ODD=Oppositional defiant disorder, SSRS=Social Skills Rating Scale. The first letter of each protocol indicates its first-stage treatment and the second letter indicates its second-stage treatment, to be implemented in the event of insufficient response (‘B’ for behavioral, ‘M’ for medication). Values are means with standard deviations in parentheses (for continuous outcomes) or asymmetric 95% confidence intervals about the mean (for count outcomes). They were calculated using the weighting method as described in Nahum-Shani et al. (2012). Within each row, means that have no superscript in common are significantly different from each other, p<.05. Cross or doublecross next to a pair of means indicates difference was only marginal, p<.10.

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

Pelham et al. Page 34

Table 5

Outcomes at Endpoint by Secondary/adaptive Treatment Given Insufficient Response to Initial Behavioral

Treatment

Outcome B-then-B B-then-M Effect Size

Classroom rule violations per hour* 6.6 [5.1, 8.6] 9.4 [7.5, 11.7] IRR = 1.41

Out-of-class disciplinary events per school year† 3.2 [1.2, 8.3] 1.0 [0.4, 2.7] IRR = 0.30

Teacher DBD—ADHD 1.28 (.65) 1.00 (.65) d = 0.44

Teacher DBD—ODD 0.63 (.60) 0.52 (.49) d = 0.19

Teacher SSRS Social Skills Total Score 32.0 (9.6) 35.0 (9.1) d = 0.31

Parent DBD—ADHD 1.60 (.66) 1.43 (.63) d = 0.26

Parent DBD—ODD† 1.20 (.69) 0.90 (.59) d = 0.45

Parent SSRS Social Skills Total Score 41.8 (9.1) 44.4 (11.2) d = 0.26

Note. B-then-B=began with behavioral treatment and then received higher dose behavioral treatment, B-then-M=began with behavioral treatment then added medication treatment, IRR=incidence rate ratio, DBD=Disruptive Behavior Disorders Rating Scale (scores are average scale scores, range 0–3), ADHD=attention deficit hyperactivity disorder, ODD=Oppositional defiant disorder, SSRS=Social Skills Rating Scale. Values are means with standard deviations in parentheses (for continuous outcomes) or asymmetric 95% confidence intervals about the mean (for count outcomes). The IRR is the ratio of the event (e.g., rule violation) incidence rate in one group (here, B-then-M) to the incidence rate in another group (here, B-then-B). The other effect sizes are Cohen’s D with pooled standard deviation (equations 2.5.1 and 2.5.2, pp. 66–67, Cohen, 1988), and are listed such that a positive d reflects an advantage of B-then-M.

† p<0.10,

* p<0.05.

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.

A utho

r M anuscrip

t A

utho r M

anuscript A

utho r M

anuscrip t

A utho

r M anuscript

Pelham et al. Page 35

Table 6

Outcomes at Endpoint by Secondary/adaptive Treatment Given Insufficient Response to Initial Medication

Treatment

Outcome M-then-M M-then-B Effect Size

Classroom rule violations per hour 14.5 [9.5, 22.1] 17.1 [10.9, 26.9] IRR = 1.18

Out-of-class disciplinary events per school year† 2.2 [0.8, 6.6] 8.2 [3.5, 19.6] IRR = 3.66

Teacher DBD—ADHD 1.21 (.63) 1.43 (.71) d = −0.34

Teacher DBD—ODD† 0.70 (.52) 1.15 (.91) d = −0.61

Teacher SSRS Social Skills Total Score 32.2 (6.2) 28.8 (11.0) d = −0.39

Parent DBD—ADHD 1.38 (.60) 1.62 (.63) d = −0.38

Parent DBD—ODD† 1.02 (.65) 1.33 (.73) d = −0.46

Parent SSRS Social Skill Total Score 44.5 (11.2) 44.0 (9.6) d = −0.05

Note. M-then-M=began with medication treatment and then received higher dose medication treatment, M-then-B=began with medication treatment and then added behavioral treatment, IRR=incidence rate ratio, DBD=Disruptive Behavior Disorders Rating Scale (scores are average scale scores, range 0–3), ADHD=attention deficit hyperactivity disorder, ODD=Oppositional defiant disorder, SSRS=Social Skills Rating Scale. Values are means with standard deviations in parentheses (for continuous outcomes) or asymmetric 95% confidence intervals about the mean (for count outcomes). The IRR is the ratio of the event (e.g., rule violation) incidence rate in one group (here, M-then-B) to the incidence rate in another group (here, M-then-M). The other effect sizes are Cohen’s D with pooled standard deviation (equations 2.5.1 and 2.5.2, pp. 66–67, Cohen, 1988), and are listed such that a positive d reflects an advantage of M-then-B.

† p<0.10.

J Clin Child Adolesc Psychol. Author manuscript; available in PMC 2017 July 01.