accesare_resursa.pdf

8/10/2019 accesare_resursa.pdf

1/17

Multimodal Treatments for Childhood Attention-deficit/Hyperactivity Disorder: Interpreting Outcomes in the Context

of Study Designs

Betsy Hoza

Nina M. Kaiser

Elizabeth Hurt

Published online: 21 August 2007

Springer Science+Business Media, LLC 2007

Abstract The goal of this article was to outline issues

critical to evaluating the literature on incremental benefit ofmultiple effective treatments used together, vs. a single

effective treatment, for childhood ADHD. These issues

include: (1) sequencing and dosage of treatments being

combined and compared; (2) difficulty drawing valid

conclusions about individual components of treatment

when treatment packages are employed; (3) differing

results emerging from measurement tools that purportedly

measure the same domain; and (4) the resultant difficulty in

reaching a summary conclusion when multiple outcome

measures yielding conflicting results are used. The impli-

cations of these issues for the design and conduct of future

studies are discussed, and recommendations are made forfuture research.

Keywords Attention-deficit/hyperactivity disorder

ADHD Multimodal treatment Combined treatment

Behavior therapy

In recent years, a number of multimodal treatment studies

have emerged in the empirical treatment literature forchildhood attention-deficit/hyperactivity disorder (ADHD).

These studies are important both for the level of design

sophistication they contribute to the literature on ADHD

and also because their results have influenced important

documents, such as pediatric treatment guidelines and

insurance coverage policies. The importance of these

studies is indisputable, as is their impact on the way in

which pediatricians, mental health professionals, schools,

and parents approach treatment of childhood ADHD. We

propose, however, that greater knowledge can be gained

from these important studies by evaluating their outcomes

in the context of the studies designs. This article repre-sents an attempt to focus greater attention on this issue.

Our view is that no single study design can address all

issues pertinent to treatment of a specific disorder. Indeed,

there is no penultimate design, but rather, different designs

address specific issues in a better or worse fashion. Hence,

results of studies may be most informative to the field when

interpreted in the context of their designs and greatest

weight given to the questions best addressed by each

design. Our purposes in writing this article are first, to

illustrate the differential conclusions that may be reached

as a function of whether design factors are emphasized in

interpreting outcomes and second, to outline key issues forfuture research.

In this article, we review multimodal treatment studies

published roughly in the past decade (1995 to the present).

We choose this time frame in order to restrict our studies to

those using diagnostic criteria from the DSM-IV (Diag-

nostic and Statistical Manual of Mental Disorders, Fourth

Edition; American Psychiatric Association 1994). This

strategy allows us to eliminate confounds that may arise,

when considering varying definitions of ADHD across

Present Address:

B. Hoza (&)

E. HurtDepartment of Psychology, University of Vermont,

2 Colchester Ave., Burlington, VT 05405-0134, USA

e-mail: [email protected]

N. M. Kaiser

Department of Psychiatry, University of California, San

Francisco, 401 Parnassus Ave., CPT, San Francisco,

CA 94143-0984, USA

e-mail: [email protected]

E. Hurt

Purdue University, West Lafayette, IN, USA

1 3

Clin Child Fam Psychol Rev (2007) 10:318334

DOI 10.1007/s10567-007-0025-5


2/17

multiple versions of the DSM. Within this time frame, we

discuss issues pertaining to the way in which the studies

designs may have affected their outcomes and conclusions.

For purposes of this review, we limit our discussion to

multimodal treatment studies examining incremental ben-

efit, a subset of which have the added strength of also being

multisite studies (Abikoff et al. 2004a,b; Hechtman et al.

2004a,b; Hoza et al.2005; MTA Cooperative Group1999;Pelham et al. 2000; Wells et al. 2000, 2006). As in our

prior work, we define incremental benefit studies as those

wherein two distinct forms of active intervention [were]

included (e.g., medication, behavior modification) and the

use of one active treatment [was] compared to the use of

two or more active treatments (Hoza et al. in press, p. 6).

Of course the need for incremental benefit studies

implies that single modality treatments for ADHD are

insufficient to yield desired outcomes, a premise that

applies better to some domains than to others. For example,

whereas medication often controls core symptoms of

ADHD reasonably well, when used alone, non-symptomdomains, such as parentchild relationships, peer relation-

ships, aggression, and internalizing symptoms may fare

better with a multimodal approach (MTA Cooperative

Group1999). Furthermore, neither medication nor behav-

ior therapy are effective for all children (Hoza et al.1999),

and, even in children who may be responders to these

treatments, non-treatment factors play a role in whether or

not either treatment is utilized (Hoza et al. 2006). For

instance, parental concerns regarding safety and accept-

ability of medication use may interfere with initial

implementation of treatment or sustained treatment

adherence (Hoza et al.2006). Importantly, after decades of

debate, recent evidence documents that ongoing use of

stimulant medications may reduce childrens growth for up

to three years (Swanson et al. 2007) and recent concerns

have been raised based on animal research regarding the

long-term effects of stimulants on the developing brain

(Volkow and Insel2003). Similarly, behavioral treatments

have drawbacks with criticisms centering on difficulty of

implementation (Mrug et al.2001) and lack of evidence for

generalization outside the treatment setting (Richters et al.

1995). These limitations of pharmacological and behav-

ioral treatments, however, can be viewed as indicating an

even greater need for consideration of incremental benefit,

as combined approaches often permit lower doses of

medications and less complex behavioral strategies to be

used (Pelham and Hinshaw 1992). For these reasons, we

choose to focus herein on incremental benefit.

Aside from this key requirement of considering incre-

mental benefit, other requirements for inclusion in this

review, also consistent with Hoza et al. (in press), were:

First, studies were selected if investigators applied DSM-

IV diagnostic criteria for ADHD, using either DSM-IV-

based clinical assessments or diagnostic cutoffs on parent

and/or teacher-completed dimensional (i.e., rating) mea-

sures of ADHD symptoms. Importantly, prior work

examining comparability of samples selected according to

these two approaches provide evidence that they are rea-

sonably comparable (Owens and Hoza 2003), although

impairment and age of onset criteria are not necessarily

captured to the same degree by the latter approach. Second,researchers included a multimodal intervention, at least one

component of which could be classified as a psychosocial

treatment based on well-established behavioral principles.

Third, researchers employed some type of control condi-

tion (this control could be pill or psychological placebo, an

alternative treatment, a wait-list or no-treatment control, or

treatment reversal), although we focus on evaluation of

incremental benefit rather than comparisons to these con-

trols in the current article. Fourth, researchers examined

outcomes in at least one of six outcome domains of primary

interest to us: ADHD symptoms (assessed via ADHD

symptom rating scales completed by adult informants orbehavior counts evaluating frequencies of behaviors that

are DSM-IV ADHD symptoms); behavior (oppositional

and aggressive behavior, general behavioral or conduct

problems); social/peer functioning (parent- and teacher-

rated social function/dysfunction, peer nominations/ratings

of friendship/liking/disliking, as well as adult evaluations

of social skills), academic functioning, parenting practices/

parentchild relationship measures, and child internalizing

symptoms. Importantly, we include child self-report mea-

sures only in those domains in which children generally are

accepted as valid reporters (e.g., internalizing problems but

not ADHD symptoms). Fifth, the participants were children

of preschool- or school-age (with the goal of ensuring

comparability of interventions across studies). Sixth, data

necessary to compute effect sizes for post-treatment dif-

ferences between treatment conditions were provided in the

original article and/or contact information for the authors

was available and the authors provided this information

upon request.

In the context of these parameters, we identified 13

different studies (listed in Table1) described in 18 articles,

as more than one article was published on some studies.

Specifically, there were four articles based on the study by

Abikoff and colleagues (hereafter collectively referred to

as the Abikoff Group; Abikoff et al. 2004a, b; Hechtman

et al. 2004a,b). There were three articles from the Multi-

modal Treatment Study of Children with ADHD (hereafter

collectively referred to as the MTA Group; MTA Coop-

erative Group1999; Wells et al.2000,2006). Ten of these

studies employed between-subjects designs; two employed

within-subjects designs, and one employed a single-subject

design (see Table1for a listing of all studies according to

these design classifications). In many instances, data were

Clin Child Fam Psychol Rev (2007) 10:318334 319

1 3


3/17

Table1

StudiesEmployingMultimodalInterventions

A1:Multimodalinterventionstudie

semployingbetween-groupsdesigns:symp

tomandbehaviordomains

Study

N

Diagnostic

procedures

**

Treatment

length

Treatmentcomponents*

Controls*

Symptom

measure**

MeanES(range)

Behavior

measure**

MeanES(range)

Abikoff

Group

103s

a,b,k

24months

1,2/4,6,8,10,11

1

c,d,k,l,m

0.03(0.28,0.21)

c,d,k,m

0.03(0.39,0.45)

1+21

c,d,k,l,m

0.29(0.53,0.00)

c,d,k,m

0.03(0.45,1.26)

Barkleyetal.

(2000)

158s

d

12months

2,6,7,11,1

3,14,

15,16,18

2

d,k,l

0.18(0.12,0.41)

d,i,k,l,m

0.23(0.12,0.57)

6,7,11,13,14,

15,16,18

d,k,l

0.01(0.32,0.59)

d,i,k,l,m

0.05(0.22,0.37)

24

d,k,l

0.04(0.16,0.28)

d,i,k,l,m

0.09(0.12,0.34)

2

6,7,11,13,14,

15,16,18

d,k,l

0.21(0.90,0.59)

d,i,k,l,m

0.27(0.63,0.18)

24

d,k,l

0.15(0.51,0.28)

d,i,k,l,m

0.14(0.36,0.13)

6,7,11,13,

14,

15,16,18

24

d,k,l

0.03(0.74,0.31)

d,i,k,l,m

0.13(0.16,0.47)

Ercanetal.

(2005)

83s

b,d,k

6monthsmedication;

eightsessions

ofparenttraining

1,2

1

d,k

0.05(0.34,0.17)

d,k

0.05(0.15,0.26)

Kleinand

Abikoff

(1997)

89s

b,k

8weeks

1,2,13,14

1

c,d,k,l,m

0.38(0.10,0.99)

c,d,k,l,m

0.24(0.50,0.77)

2,13,14

c,d,k,l,m

0.91(0.42,1.82)

c,d,k,l,m

0.82(0.00,2.14)

2,13,14

1

c,d,k,l,m

0.55(0.88,0.13)

c,d,k,l,m

0.55(1.13,0.00)

MTA

Coop.

Group

579s

a,b,d,k

14months

1,2,9,12,1

3,14

1

d,k,l

0.30(1.47,0.15)

d,i,k,l

0.03(0.23,0.26)

2,9,12,13,14

d,k,l

0.19(0.90,0.57)

d,i,k,l

0.29(0.07,0.48)

22

d,k,l

0.18(0.74,0.71)

d,i,k,l

0.31(0.07,0.53)

2,9,12,13,

14

1

d,k,l

0.47(0.61,0.38)

d,i,k,l

0.26(0.43,0.05)

22

d,k,l

0.01(0.52,0.19)

d,i,k,l

0.02(0.25,0.21)

Pelhametal.

(2000)

117s

b,k

14months

1,2,9(6,11,15,16,18,

25,26),12,13,14

2,9

j,k

0.45(0.31,0.58)

j,k,t

0.21(0.03,0.72)

Pfiffnerand

McBurnett

(1997)

27s

a,b,d

8wks

2,6(15,19)

,14

6

NA

d,k

0.04(0.04,0.11)

20

d,k

0.99(0.42,1.56)

6

20

d,k

1.07(0.60,1.53)

Tuttyetal.

(2003)

100s

a,b,d,k

8weeklysessions

1,2,6,7

1,22

d,k

0.44(0.11,0.77)

NA

vanderOord

etal.(2007)

45s

b

10weeks

1,2,6/7/10,

14

1

d,k

0.10(0.22,0.44)

d,k

0.24(0.06,0.58)

320 Clin Child Fam Psychol Rev (2007) 10:318334

1 3


4/17

Table1

continued


semployingwithin-groupsdesigns:sympto

mandbehaviordomains

Study

N

Diagnostic

procedures*

*

Treatment

length

Treatmentcomponents*

Controls*

Symptom

measure

**

MeanES

(range)

Behavior

measure**

MeanES(range)

Kolkoetal.

(1999)

16s

a,b,d,k

8weeks;b-modevery

otherwk,2med

dosesadminrandomly

1x/wkduringwks38

1,9(15,16,18,25)

1

j

0.05(0.23,0.33)j,t

0.13(0.41,0.72)

9

j

0.51(0.22,0.79)

j,t

0.27(0.26,0.85)

23

j

1.92(1.72,2.12)

j,t

0.77(0.41,1.47)

9

1

j

0.44(0.98,0.10)j,t

0.13(0.69,0.68)

23

j

1.14(0.56,1.71)

j,t

0.50(0.47,1.09)

Pelhametal.

(2005a)

21s

a,b,d,k

8weeks;b-modevery

otherweekfor4wks;

3meddoseseach

randomlyadmin

1x/wkfor6wks

1,9(2,14,15,16,

18,25)

1

j,k

0.93(0.70,1.15)

j,k,t

0.77(0.53,1.31)

9

j,k

1.55(1.31,1.78)

j,k,t

1.19(0.72,1.86)

23

j,k

3.02(1.89,4.15)

j,k,t

1.83(0.65,3.97)

9

1

j,k

0.61

(0.65,0.56)

j,k,t

0.40(0.79,0.01)

23

j,k

0.96 (0.63,1.29)

j,k,t

0.77 (0.25,1.8

1)


semployingsinglecasedesigns:symptom

andbehaviordomains

Study

N

Diagnostic

procedures**

Treatmentlength

Treatment

components*

Controls*

Symptom

measure**

Effect

Behavior

measure*

*

Effect

Reitman

etal.

(2001)

3s

a,b,d

20days;3040

observations

1,15

23

m

IM

t

IM

1,15

1

m

IM

t

IM

1,15

15

m

IM

t

IM

15

1

m

IM

t

IM

15

23

m

IM

t

IM

1

23

m

IM

t

IM

B1:Multimodalinterventionstudie

semployingbetween-groupsdesigns:parenting/parentchildrelationshipandinterna

lizingdomains

Study

N

Diagnostic

procedures**

Treatmentlength

Treatment

components*

Controls*

Parentin

g

measure**

MeanES

(range)

Internalizing

measure**

MeanES

(range)

Abikoff

Group

103sa,b,k

24months

1,2/4,6,

8,10,11

1

e,f

0.12(0.35,0.25)s

0.29

1+21

e,f

0.31(0.57,0.13)s

0.3

Barkleyetal.

(2000)

158sd

12months

2,6,7,11,13,14,

15,16,

18

2

e,i

0.09(0.03,0.29)

d,k

0.28(0.03,0.35)

6,7,11,13,14,

15,16,18

e,i

0.04(0.09,0.03)d,k

0.07(0.39,0.10)

24

e,i

0.03(0.09,0.19)d,k

0.13(0.00,0.21)


1 3


5/17

Table1

continued

B1:Multimodalinterventionstudiesemployingbetween-groupsdesigns:parenting/parentchildrelationshipandinterna

lizingdomains

Study

N

Diagnostic

procedures**

Treatment

length

Treatment

components*

Controls*

Symptom

measure*

*

MeanES

(range)

Behavior

measure**

MeanES(range)

2

6,7,11,13,14,

15,16,18

e,i

0.15(0.42,0.06)

d,k

0.35(0.50,0.20)

24

e,i

0.06(0.10,0.02)

d,k

0.16(0.31,0.09)

6,7,11,13,14,

15,16,18

24

e,i

0.09(0.02,0.30)

d,k

0.20(0.04,0.55)

Ercanetal.

(2005)

83s

b,d,k

6monthsmedication;

eightsessionsparent

training

1,2

1

g

0.02

NA

Kleinand

Abikoff

(1997)

89s

b,k

8weeks

1,2,13,14

1

NA

d,k

0.10(0.28,0.32)

2,13,14

d,k

0.09(0.57,0.20)

2,13,14

1

d,k

0.19(0.00,0.32)

MTA

Group

579sa,b,d,k

14months

1,2,9,12,13,14

1

e,f,g,h,i

0.22(0.01,0.42)

d,k,s

0.11(0.23,0.00)

2,9,12,13,14

e,f,g,h,i

0.18(0.05,0.37)

d,k,s

0.03(0.24,0.26)

22

e,f,g,h,i

0.18(0.07,0.50)

d,k,s

0.09(0.13,0.37)

2,9,12,13,14

1

e,f,g,h,i

0.05(0.15,0.12)

d,k,s

0.08(0.25,0.11)

22

e,f,g,h,i

0.002(0.14,0.23)d,k,s

0.13(0.00,0.26)

Pfiffnerand

McBurnett

(1997)

27s

a,b,d

8wks

2,6(15,1

9),14

6

NA

d,k

0.32(0.76,0.13)

20

d,k

0.31(0.21,0.40)

6

20

d,k

0.67(0.26,1.08)

Tuttyetal.

(2003)

100sa,b,d,k

8weeklysessions

1,2,6,7

1,22

e

0.45

NA

vanderOord

etal.(2007)

45s

b

10weeks

1,2,6/7/10,14

1

NA

s

0.03

B2:Multimodalinterventionstudiesemployingwithin-groupsdesigns:parenting/parentchildrelationshipandinternalizingdomains:none

B3:Multimodalinterventionstudiesemployingsinglecasedesigns:parenting

/parentchildrelationshipandinternalizingdomains:none

C1:Multimodalinterventionstudie

semployingbetween-groupsdesigns:acad

emicandsocial/peerdomains

Study

N

Diagnostic

procedures**

Treatment

length

Treatment

components*

Controls*

Academic

measure**

MeanES

(range)

Social/peer

measure**

MeanES

(range)

Abikoff

Group

103sa,b,k

24months

1,2/4,6,8

,10,11

1

d,q

0.15(0.64,0.31)

d,k,l,s

0.01(0.31,0.42)

1+21

d,q

0.18(0.82,0.15)

d,k,l,s

0.10(0.46,0.16)

Barkleyetal.

(2000)

158sd

12months

2,6,7,11,13,14,

15,16,18

2

k,q

0.12(0.29,0.17)

d,k

0.48(0.20,0.71)


1 3


6/17

Table1

continued

C1:Multimodalinterventionstudiesemployingbetween-groupsdesigns:academicandsocial/peerdomains

6,7,11,13,14,

15,16,18

k,q0.25(0.56,0.04)d,k

0.17(0.31,0.02)

24

k,q0.08(0.37,0.17)

d,k

0.03(0.30,0.33)

2

6,7,11,13,14,15,16,18k,q0.12(0.42,0.04)

d,k

0.58(0.76,0.45)

24

k,q

0.04(0.28,0.26)

d,k

0.42(0.48,0.34)

6,7,11,13,14,15,16,1824

k,q

0.16(0.07,0.45)

d,k

0.18(0.04,0.45)

Hozaetal.

(2005)

285sb,k

14months

1,2,9,12,13,14

1

NA

r

0.06(0.21,0.06)

2,9,12,13,14

r

0.21(0.12,0.37)

22

r

0.05(0.01,0.11)

2,9,12,13,14

1

r

0.27(0.41,0.08)

22

r

0.17(0.27,0.09)

Kleinand

Abikoff

(1997)

89s

b,k

8weeks

1,2,13,14

1

p,q0.16(0.27,0.08)c,d,k,m

0.06(0.18,0.00)

2,13,14

p,q

0.11(0.02,0.33)

c,d,k,m

0.18(0.72,0.96)

2,13,14

1

p,q0.35(0.92,0.09)c,d,k,m

0.23(0.91,0.50)

MTAGroup

579sa,b,d

,k14months

1,2,9,12,13,14

1

q

0.07(0.05,0.10)

d,k,r

0.09(0.05,0.19)

2,9,12,13,14

q

0.15(0.01,0.23)

d,k,r

0.35(0.27,0.42)

22

q

0.16(0.01,0.27)

d,k,r

0.30(0.18,0.46)

2,9,12,13,14

1

q

0.08(0.16,0.04)

d,k,r

0.25(0.40,0.08)

22

q

0.00(0.04,0.05)

d,k,r

0.05(0.19,0.03)

Pelhametal.

(2000)

117sb,k

14months

1,2,9(6,11,15,16,18,

25,26),

12,13,14

2,9

n,o

0.15(0.09,0.25)

j,r,t

0.39(0.18,0.77)

Pfiffnerand

McBurnett(1997)

27s

a,b,d

8wks

2,6(15,19),14

6

NA

d,k,s

0.37(0.52,0.27)

20

d,k,s

1.53(0.42,2.41)

6

20

d,k,s

2.11(0.68,3.41)

vanderOord

etal.

(2007)

45s

b

10weeks

behavioral

treatment

1,2,6/7/10,14

1

NA

d,k

0.14(0.08,0.19)

C2:Multimodalinterventionstudiesemployingwithin-groupsdesigns:academicandsocial/peerdomains

Kolkoetal.(1999)

16s

a,b,d

,k8weeks;b-modevery

otherwk,2meddoses

adminrandomly1x/wk

duringwks38

1,9(15,16,18,25)

1

NA

j

0.15(0.37,0.66)

9

j

0.46(0.39,0.53)

23

j

0.98(0.65,1.31)

9

1

j

0.13(0.56,0.31)

23

j

0.60(0.21,0.98)


1 3


7/17

Table1

continued

C2:Multimodalinterventionstudie

semployingwithin-groupsdesigns:academ

icandsocial/peerdomains

Pelhametal.(2005a)

21s

a,b,d,k

8weeks;b-modeve

ry

otherweekfor4wks;

3meddoseseach

randomlyadmin

1x/wkfor6wks

1,9

1

n,o

0.01(0.44,0.46)

j,k

0.35(0.48,0.22)

9

n,o

1.10(0.71,1.49)

j,k

0.49(0.37,0.61)

23

n,o

1.22(0.60,1.83)

j,k

2.32(1.11,3.53)

9

1

n,o

0.88(0.94,0.82)

j,k

0.76(0.99,0.52)

23

n,o

0.15(0.05,0.35)

j,k

1.28(0.72,1.84)

C3:Multimodalinterventionstudiesemployingsinglecasedesigns:academic

andsocial/peerdomains:none

Note:NA=notapplicable.UnderN:p

=preschool;s

=school-age.*Treatme

ntcomponentsandcontrolconditions:1=

stimulantmedicationormedicationassess

ment/referral;2=parent

training;3=parent-targetedpartnersupportand/orcopingskilltraining;4=familyproblem-solvingorcommunicationtherapy;5=supportgroup;6=child-targe

tedsocialskillstraining;

7=child-targetedself-control/angermanagementtraining;8=individualchild-targetedpsychotherapy;9=behavioralsum

mertreatmentprogram;10=childfocused

academicorganizational

skillstraining;11=individualized

academicassistance/remediation/tutoring;12=behaviorallytrainedindividualparapr

ofessionalsupport;13=schoolconsultatio

nand/orteacherpsycho-

education;14=dailyreportcard;15=tokenreinforcement;16=tokenrespon

se-cost;17=delayedreward;18=timeout;19=peerattention/feedback/positiverein

forcement;20=wait-list

control;21=attentioncontrol;22

=communitycare;23=treatmentwithdrawal;24=nointervention;25=adultadm

inisteredpositivereinforcement/praise;26=sportsskillstraining.

**Diagnosticandoutcomemeasure

s:a=clinicaldiagnosisbypediatricianormentalhealthprofessional;b=clinicianad

ministeredinterviewwithparent,child,and

/orteacher;c=clinician

ratings;d=parentratings;e=parentratingsofparentingpractices/strategies;f

=childratingsofparentingpractices/strategies;g=parentratingsofparentchildrelationship;h=childratings

ofparentchildrelationship;i=parentchildobservations;j=counselorratin

gs;k=teacherratings;l=classroomobse

rvations;m=observerratings;n=academ

icproductivity;o=aca-

demicperformance;p=cognitivetests;q=achievementtests;r=peerratingsornominations;s=childself-ratings;t=

behaviorcounts.Forcasestudies,IM=improved;NC=nochange;

DT=deteriorated.Therangeofeffectsizesisdesignatedby:(lowerbound,upperbound)

AbikoffGroupstudiesincluded:A

bikoffetal.2004a,b;Hechtmanetal.2004a,b

MTAGroupstudiesincluded:M

TACooperativeGroup1999;Wellsetal.2000,2006

Hozaetal.(2005)andPelhametal.(2000)areconsideredseparatelyfrom

theMTAGroupstudiesbecausethesestudiesarebasedononlyasubsetofMTAparticipants


1 3


8/17

available in the articles to make other comparisons in

addition to examining incremental benefit (e.g., compari-

son of a multimodal or unimodal treatment to a no

intervention group). Although not explicitly discussed here,

these comparisons also are included in the table for the

interested reader.

In evaluating incremental benefit, we provide data

regarding the magnitude of incremental benefit treatmenteffects to the best of our ability, given the constraints of

original articles. Specifically, within each relevant outcome

domain, we report Cohens deffect sizes that assess post-

treatment differences between the multimodal treatment vs.

the initial or baseline treatment condition. Importantly,

because treatment packages often were used as either the

baseline or incremental treatment, a single treatment con-

dition could include multiple treatment components; in

interpreting results from these studies, effects sizes repor-

ted pertain to the effects of the packages as a whole and

conclusions may not be drawn about the individual com-

ponent interventions. In computing effect sizes for within-subjects studies, we averaged data across all repetitions of

the same treatment condition. Further, and regardless of

type of study design, we averaged effect sizes across all

dependent measures within each outcome domain of any

given study in order to generate a single, overall effect size

for the treatment effect in that domain and study. We note

that results for different individual measures within a single

domain did not always produce consistent results; conse-

quently, our averaged results do not necessarily produce

the same conclusions as those reported by the original

authors based on individual measures. Finally, the reader

should note that we altered the direction of effect sizes with

the goal of improving ease of comparison across outcome

domains. Thus, positive effect sizes denote improved

functioning on the part of the multimodal treatment group

or condition (i.e., incremental benefit) relative to the uni-

modal treatment group or condition (e.g., lower ADHD

symptoms, better academic performance). Effect sizes

were interpreted according to the guidelines of Cohen

(1988), who categorized values below 0.20 as non-impor-

tant, values from 0.20 to\0.50 as small in magnitude,

values from 0.50 to\0.80 as moderate in magnitude,

and values at or above 0.80 as large. Where we discuss

case studies that employed visual inspection of graphs to

draw conclusions, we limit our discussion to descriptive

comments and report for each domain, whether improve-

ment, deterioration, or no change was observed (in the

absence of any better method with which to quantify these

effects). We recognize the limited generalizability of

results from single-subjects designs to the ADHD popula-

tion at large, yet include them for purposes of

comprehensiveness.

Using these parameters to structure our discussion, we

organize our review around five key issues that we view as

particularly relevant in examining evidence for incremental

benefit of treatment for childhood ADHD: (1) How does

choice of baseline or initial treatment(s), and particularly

initial treatment dosage, affect conclusions about incre-

mental benefit?; (2) How do we interpret studies involving

treatment packages?; (3) How does choice of outcomemeasure(s) affect conclusions drawn?; (4) How do we

reach a summary conclusion when multiple outcome

measures are used?; and (5) What are the implications of

these issues for the design and conduct of future studies? In

discussing these issues, due to space limitations, we choose

specific exemplars from the literature that we find partic-

ularly useful in illustrating our points rather than

attempting a discussion of each and every study indexed in

Table1. However, we include a listing of all studies in the

table for the interested reader.

Issue 1: How does Choice of Baseline or Initial

Treatment, Timing of the Addition of Treatments, and/

or their Dosages, Affect Conclusions about Incremental

Benefit?

This issue lies at the crux of attempts to evaluate incre-

mental benefit of multiple treatments over the effects of

an initial treatment or treatment package. Yet, this issue

has received little attention in the literature. First,

regarding choice of initial treatment, because there is a

finite amount of improvement that can occur in treatment

studies in certain domains, there may be an inverse

relationship between the amount of variance accounted

for by an initial treatment, and the remaining variance

that can be accounted for by a second treatment (Cunn-

ingham 1999). In such a circumstance, the initial

treatment may produce a large improvement, whereas the

added treatment produces a lesser effect or no significant

effect. In domains where no known treatment comes close

to normalizing functioning (e.g., peer relationships), this

is less of a concern. Nonetheless, the timing of when the

treatments are added relative to one another is very

important, especially in domains in which one of the

treatments does come close to normalizing functioning.

There is greater likelihood of the second treatment

showing improvement either if both treatments are started

together, if the second treatment is added to a lower dose

of the initial treatment, or if the outcome domain is one in

which neither treatment alone comes close to normalizing

functioning.

Examining the studies in Table 1, we see several illus-

trations of this point. For example, the Abikoff Group


1 3


9/17

(Abikoff et al. 2004a,b; Hechtman et al. 2004a,b) inclu-

ded only medication responders in their study and titrated

these children to their maximum tolerable effective dose of

methylphenidate (up to 50 mg/day) prior to beginning the

psychosocial intervention (Klein et al. 2004). These

researchers found no incremental effects of the psychoso-

cial intervention, as reflected in our averaged effect sizes,

in any of the domains summarized in the table. Similarly,researchers studying a sample of Turkish children with

ADHD (Ercan et al. 2005), did not begin parent training

until children had been on medication for a full month,

with upward medication dose adjustments occurring during

this month (and subsequently); these researchers found no

evidence of incremental benefit of parent training. In

contrast, Klein and Abikoff (1997) began medication and

psychosocial treatments concurrently and found modest

evidence for incremental benefit of combined treatment

(medication plus a behavioral intervention) over the effects

of medication alone in a subset of domains. Hence, timing

of the initial treatment relative to the second treatment maybe a contributing factor in a studys outcome.

Perhaps even more illustrative of this point are results

from Pelham et al. (2005a), who crossed placebo and three

doses of methylphenidate (MPH) with two levels of

behavior modification (present vs. absent) in their study

design. Since means and standard deviations were reported

for all conditions, we were able to examine evidence for

incremental benefit of combined treatment (intensive

summer treatment program plus medication) as a function

of the dose of medication in both medication-only and

combined treatment conditions (12.5, 25, 37.5 mg

transdermal MPH). As Fig.1 depicts, higher doses of

medication in the medication-only condition were related

to lower incremental benefit of combined treatment relative

to medication-only treatment (averaged across 13 outcome

variables in the behavior domain); this was true regardless

of medication dose in the combined treatment condition.

More specifically, incremental benefit of combined treat-

ment was greatest when combined treatment was compared

to the lowest dose in the medication-only condition (Co-

hens ds averaged across the 13 outcome measures ranged

from 0.9 to 1.2). Incremental benefit of combined treatment

was lowest when the dose of medication in the medication-

only treatment condition was highest (averaged Cohens ds

ranged from 0.2 to 0.8). Hence, it is critically important

that issues such as these be routinely discussed in inter-

preting the results of clinical trials.

Thus far, we have discussed the incremental benefit of

adding behavioral treatment to medication as being largely

dependent on the dose of medication being employed when

behavioral treatment is added. However, the reverse phe-

nomenon holds as well, as Pelham et al. (2000) aptly

demonstrated. Specifically, using a subset of subjects from

the MTA study, Pelham and colleagues compared children

receiving a maximum intensity multi-component

behavioral intervention package in the context of a com-

prehensive summer treatment program to those receiving

this same intensive behavioral treatment package combined

with medication. Importantly, they reported no incremental

benefit resulting from also receiving medication on 30 of

35 dependent variables reported, despite the fact that

medication is an intervention of known efficacy for ADHD.

This study, together with those summarized immediately

above, nicely illustrates the potential impact of dosage

of initial treatments, whether pharmacological or behav-

ioral, on comparisons evaluating incremental benefit.

Fortunately, discussions of the effects of treatment

sequencing and dosages on outcomes are emerging in the

literature with increasing frequency. For example, in regard

to the primary analyses from the Multimodal Treatment

Study of Children with ADHD (MTA; MTA Cooperative

Group1999), Pelham (1999) argued that the superiority of

Medication Management over Behavioral Treatment at

14 months was attributable to differences in dose or

intensity of treatments at the 14-month assessment point,

specifically, intensive medication vs. faded (i.e., lower

intensity) behavioral treatment. Similar arguments have

been made to explain the limited differences reported

between Combined Treatment and Medication Manage-

ment at 14 months in the MTA study (Cunningham1999).

Follow-up articles directly examining these issues using

data from the MTA study, however, have yielded seem-

ingly conflicting findings. As just noted, Pelham et al.

(2000) argue that comparisons between a subgroup of

MTA participants in an early phase of treatment who were

receiving Behavior Therapy during its most active and

0

0.2

0.4

0.6

0.8

1

1.2

1.4

MED 12.5 MED 25 MED 37.5

Medication Dose in Medication Only Condition (in mg)

EffectSizeInde

xingIncrementalBenefit

COMB 12.5

COMB 25

COMB 37.5

Fig. 1 Cohensdeffect sizes (averaged across 13 outcome variables

in the behavioral domain) indicating incremental benefit of multi-

modal treatment relative to medication only (transdermal

methylphenidate), according to medication dose, in both combined

and medication-only conditions. (Data taken from: Pelham et al.

(2005a) )


1 3


10/17

intensive phase (i.e., during the summer treatment pro-

gram), as compared to children receiving Combined

Treatment, yielded little incremental benefit of Combined

Treatment over very intensive Behavior Therapy, arguing

that behavior therapy is a sufficient treatment used alone if

it is intensive enough. However, Arnold et al. (2004),

compared the MTA groups a number of months later at the

9 month assessment point, when subjects were no longerinvolved in the intensive summer program, but parent

training and the school-based behavior therapy intervention

were only moderately intensive, and found results conso-

nant with the 14 month findingsspecifically, superiority

of Medication Management over Behavior Therapy alone

for symptom measures.

How do we make sense of these results? Our view is

that these results are not necessarily contradictory if

viewed within the context of the dosages of medication

and behavior therapy being administered, since each of

these treatments can vary widely in dosage levels. How-

ever, dosages of behavior therapy have not been widelyconsidered in the psychosocial treatment literature. Yet, it

is well known that direct contingency management (such

as was delivered in Pelham et al. 2000) is more intensive

and generally more effective than clinical behavior ther-

apy, or behavior modification delivered by other agents

parents, teacherswho have been taught the skills by

clinicians (Pelham et al. 1998). Hence, comparisons of

behavior therapy to other treatments in the MTA should

yield more favorable results for behavior therapy when

maximum intensity direct contingency management is

employed (as in Pelham et al. 2000) than when less

intensive clinical behavior therapy is employed, albeit

with the support of paraprofessional classroom aides (as

in Arnold et al. 2004). This indeed is the pattern that was

obtained.1

From our perspective, it is not problematic that results

diverge within the contexts of these differing treatment

designs; indeed, as we just have demonstrated, these

divergences generally can be reconciled through careful

contemplation of methodological and design issues. How-

ever, parents, teachers, health care professionals, and other

consumers of the results may interpret and utilize the

results without careful consideration of these importantdesign issues, as they may not be immediately apparent to

some consumers, especially those with little or no research

training. Investigators can assist in ameliorating this con-

cern by emphasizing these factors in research reports; and

striving through a variety of media (e.g., question and

answer sheets for parents and teachers) to make results of

complex studies accessible to audiences of all levels of

methodological sophistication. This is a very important

point to which we will return in the discussion.

Issue 2: How do we Interpret Studies InvolvingTreatment Packages?

Most of the studies listed in Table 1 involved packages

of psychosocial treatments. For example, in the Abikoff

Groups study (Abikoff et al. 2004a, b; Hechtman et al.

2004a, b; Klein et al. 2004) the multi-component psycho-

social treatments employed included parent training/family

therapy, academic skills training, individual psychotherapy

for the child, and social skills training, as part of a psy-

chosocial treatment package. Similarly, Pelham et al.

(2000) employed a summer treatment program intervention

involving multiple components including an extensive

point system, social skills training and other peer inter-

ventions (cooperative tasks with peers, a friendship

intervention), sports skills training, a classroom interven-

tion modeled after a special education classroom, a daily

report card, and parent training. Indeed, multi-component

treatment packages have, to some extent, become the norm

in psychosocial treatment for childhood ADHD, largely

due to the absence of a single successful treatment that

effectively normalizes the many deficits associated with

ADHD. From an historical perspective, then, the emer-

gence of treatment packages makes sense, and represents a

logical response to a disorder comprising impairments in

multiple domains across the lifespan. From a research

perspective, however, available data do not permit ade-

quate evaluation of the individual components of the

packages employed. Whereas evaluating individual com-

ponents was unnecessary to the stated purposes of these

studies, and, as we noted earlier, no single study can

address all pertinent issues, this presents a quandary for

those trying to provide evidence-based, cost-effective ser-

vices, as well as for researchers trying to design future

1 Of course, intensive behavioral therapies, similarly to high doses of

medication, have limitations. Specifically, aside from the limited

availability and high cost of intensive, complex, direct contingency

management programs (see Barkley 2000, for a discussion of this

issue in regard to the MTA study), an added difficulty for

psychosocial treatment generally is that gains are difficult to maintain

outside the treatment setting (Richters et al. 1995). Indeed, general-

ization of gains and persistence of effects following participation inan intensive behavioral treatment program (e.g., Pelham and Hoza

1996; Pelham et al. 2005b) has not been adequately studied. In

addition, follow-up studies into adolescence and young adulthood of

children who participated in intensive behavioral treatments as

children are rare, and have only recently begun to appear in the

literature. Consistent with other long-term follow-up studies of

ADHD samples (for a brief review, see Richters et al. 1995), these

studies document continuing adjustment problems and risky behav-

iors into adolescence and adulthood relative to controls (e.g., Flory

et al.2006; Molina et al.2007; Thompson et al. 2007). This suggests

that short-term intensive behavioral treatment, like medication, has

limitations.


1 3


11/17

studies. It is simply unclear exactly which components of

treatment are most critical to include, or in what manner

critical components are most effectively delivered. Of

course, one could easily argue that until a comprehensive

treatment package that successfully manages the chronic,

varied symptoms and impairments of ADHD across mul-

tiple developmental periods is identified, a dismantling

strategy may be premature. We recognize the value in sucha position, yet, wish to point out the need for such an

approach at the time that our state of knowledge reaches a

point where it becomes feasible.

In a related fashion, another issue that arises in the

context of discussing treatment packages is whether con-

sumers of such treatments view receiving more

components as necessarily more desirable than fewer

components. Indeed, relatively little published work has

examined consumer satisfaction with regard to ADHD

treatments. As the number of components increases, so too

does the implementation burden on parents, teachers, and

others involved in the intervention, consequently perhapsdecreasing compliance with the interventions and limiting

potential gains. Further, costs of these interventions also

generally increase as intervention packages become more

complex. Studies are needed to determine whether and at

what point consumer satisfaction and effectiveness may

begin to diminish as additional components are added or as

treatments are made more complex.

Interestingly, available consumer satisfaction data sug-

gest that parents report greater satisfaction with treatments

involving behavioral components than with those involving

medication alone (Pelham et al. 2007). Teachers in this

same study similarly indicated that they felt better equip-

ped to address ADHD in their classrooms when behavioral

treatments were used, as compared to medication man-

agement alone (Pelham et al. 2007). These results are

impressive given the greater demands of the behavioral

intervention in terms of time required from parents and

teachers, as well as the level of improvement that was

obtained for families receiving medication management.

Yet, the point at which satisfaction begins to drop as

demands increase on participants is an unknown area in

need of further study.

Issue 3: How does Choice of Outcome Measure(s) Affect

Conclusions Drawn?

In performing this review, we were struck by a simple fact:

different outcome measures purported to measure the same

domain often yielded drastically different results for the

same comparisons within a study. The reader is encouraged

to peruse the ranges of effect sizes obtained in each domain

for each study, as presented in Table 1. Often effect sizes

within the same domain were of widely varying magnitude

and frequently also in opposite directions. For example, in

the Klein and Abikoff (1997) study, for the ADHD

symptoms domain, effect sizes for comparisons of com-

bined psychosocial and psychostimulant treatment to

stimulant treatment alone ranged from .10 to +.99,

depending on the measure. In the MTA study (MTA

Cooperative Group1999), also for the symptoms domain,comparisons of combined treatment to behavior therapy

alone were similarly variable, with effect sizes ranging

from .90 to +.57. Of importance, this pattern was not

unique to specific studies, but for the most part, charac-

terized this group of studies overall. This simple fact was

surprising to us in preparing this article and may not be

evident to the field at large, as results are generally not

grouped by domain across sources, as we have done. This

indicates the possibility that our methods, rather than our

purported constructs being measured, may be driving the

sizes of our effects, and relatedly, that the treatment effects

that we obtain reflect, to a non-trivial degree, our chosenmeasurement methods and analysis strategies.

We had difficulty in discerning any pattern in the vari-

ation of effect sizes. This task was complicated by

methodological differences among studies, such as those

already discussed, as well as by the fact that each different

study generally employed different outcome measures and/

or different raters; for example, summer program-based

studies (e.g., Kolko et al.1999) report ratings from summer

program counselors rather than classroom teachers. No

single study employed all possible types of outcome vari-

ables within any given domain, making comparisons of

effect sizes generated by data collected from one type of

rater vs. another quite difficult.

Nonetheless, there indeed appeared to be considerable

variance both among and within raters. More specifically,

in studies examining incremental benefit of multimodal

psychosocial and stimulant treatment upon ADHD symp-

toms or behavior vs. either psychosocial or stimulant

treatment alone, averaged effect sizes based on teacher

ratings generally were somewhat larger than were averaged

effect sizes based on parent ratings.

Further, even effect sizes from the same rater within the

same study and purportedly assessing the same construct

(yet based on administration of different measures) pro-

duced differing results. As an example, Klein and Abikoff

(1997) administered three separate measures of parent-rated

hyperactivity, providing a unique opportunity to compare

results of multiple measures of parent-rated hyperactivity.

These measures (and effect sizes indexing incremental

benefit of combined psychosocial and psychostimulant

treatment over psychostimulant treatment alone) were as

follows: Parent Home Hyperactivity Scale (Cohens

d = 0.62); Parent CPRS Hyperactivity subscale (Cohens


1 3


12/17

d= 0.24); and Mother Hillside Behavior Scale, Gross

Motor Activity subscale (Cohensd= 0.12). Based on these

results, it seems likely that differences in items or in

wording among these measures contributed to the discrep-

ancies in effect size. Had these researchers not employed

this comprehensive multi-measure approach, results would

have depended upon the specific measure chosen.

As mentioned earlier, our strategy for dealing with thesewidely varying results even within the same domain was to

average effect sizes across all measures within a domain to

yield one overarching effect size. But, we remind the

reader that using this strategy, our conclusions often dif-

fered from those of the original authors who interpreted

measures individually and may have weighted one measure

more heavily over another. We do not advocate for one

approach (e.g., averaging effect sizes) over another (e.g.,

interpreting individual effect sizes). Instead, we caution

consumers of the treatment literature that choice of out-

come measure will heavily influenceboth size and possibly

also direction of effects in treatment outcome studies.Hence, results cannot and should not be interpreted outside

the context of these factors.

Issue 4: How do we Reach a Summary Conclusion

When Multiple Outcome Measures are Used?

In light of the pressing issue just described, a question of

enormous importance for intervention researchers becomes:

How do we reach a summary conclusion when multiple

outcome measures are used? This is a dicey issue because

use of multiple outcome measures, though consistent with

generally recommended multi-method, multi-informant

approaches, opens the door for pre-selection by investiga-

tors based on any of a variety of factors. These may include

practical issues such as how quickly or easily one measure

vs. another can be made ready for analysis, or likelihood of

publication based on size or strength of obtained effects.

Unfortunately, null findings remain hard to publish (Ro-

senthal1979). This appears to be true even when studies are

well-designed and adequately powered; we are hopeful that

this is beginning to change, as it is just as important to know

what doesnotwork as it is to know what doeswork. These

larger issues fall beyond the scope of this article, but are

raised merely as a backdrop for the ensuing discussion.

In this era of evidence-based treatments, with both best

practices guidelines and treatment manuals widely avail-

able, one might assume that similar guidelines exist to

guide us in choosing assessment instruments, both for

selection of study participants and in reporting and inter-

preting outcomes from treatment studies. Unfortunately,

few guidelines exist either for researchers who are report-

ing research results in scientific journals, or for consumers

of scientific journals, to assist in deciding what to conclude

under these often confusing circumstances. As noted

recently by Mash and Hunsley (2005), even though evi-

dence-based treatments have advanced significantly over

the past decade, it generally is the case that evidence-based

assessment has not progressed in the same manner in

regards to most childhood disorders. Yet, conclusions

regarding treatment outcomes only are as valid as theprocedures used to select study participants and the reli-

ability and validity of treatment outcome measures

employed. Hence, the status of evidence-based assessment

bears directly on the level of evidence for evidence-based

treatments (EBTs). For example, practices vary widely

across studies in terms of whether parent informants alone,

or parent and teacher informants together, are used to

gather diagnostic information. Despite this variability in

assessment practices, however, and as noted by Mash and

Hunsley (2005): For the most part, treatment-focused task

force statements and guidelines have been silent on the use

of assessment in developing and evaluating EBTs, not-withstanding general recommendations to use a reliable

and valid core assessment batteryrecommendations that

are presented with little guidance as to what makes an

assessment method or process reliable or valid in a par-

ticular context or for a particular purpose (pp. 363364).

This measurement quandary, however, is not necessarily

cause to throw the metaphorical baby out with the bath

water. Rather, this dilemma begs for the development of

measurement models that are mindful of these challenges,

even if completely eliminating these challenges may be too

ambitious a goal at the present time. Nonetheless, a key

first step would be to outline the most important domains to

be assessed and the types of measures to be included in

treatment outcome studies for ADHD. Measures should

include not just those most easily obtained, such as parent

and teacher ratings, as ratings such as these are more prone

to various types of bias (particularly if the informants are

not blind to treatment conditions). Instead, these measures

are best balanced to a reasonable degree by more objective

measurese.g., peer sociometrics, achievement testing,

behavior countseven though more objective measures

such as these often are more expensive to obtain in terms of

time and resources. Finally, given our current knowledge

that ADHD is a chronic disorder that emerges early in

childhood and persists into adulthood (for a discussion of

long-term prognosis, see Richters et al. 1995), the appli-

cability of available measures, as well as their reliability

and validity across key developmental periods, is a par-

ticularly important consideration.

In addition to developmental sensitivity, importantly, the

inclusion of measurements thought to be sensitive to the

change produced byeachtype of treatment being compared

seems important to producing a balanced comparison among


1 3


13/17

treatments (Cunningham 1999). For example, whereas

medication might be expected to have a large impact on

symptom reduction, behavioral treatments such as parent

training might be expected to have the largest impact on the

parentchild relationship. If this is the case, results may be

most informative when these diverse measures can be

examined and presented simultaneously. Publishing data in

the order in which it becomes available based on ease ofpreparation for analysis ought to be discouraged, in the

interest of presenting the most balanced picture.

In sum, measurement models need to be developed that

guide researchers in the selection of measures and that

promote consistent choice of domains and measurement

strategies across studies and across developmental periods.

Such models do not currently exist in the ADHD literature

or in the child psychological treatment literature at large,

but the field would benefit greatly from their development.

Attempts to produce such models should go hand-in-hand

with developments in evidence-based assessment and

necessarily should proceed in tandem. Indeed, the fact thatevidence-based treatment has marched ahead despite lim-

itations in our evidence-based assessments is a

complicating factor, as evidence-based assessment is a

necessary prerequisite for measuring outcomes of evi-

dence-based treatments.

Issue 5: What are the Implications of these Issues for

the Design and Conduct of Future Studies?

We began this article by indicating that outcomes of

treatment studies may be at least partially a function of

their designs. To this simple premise, we added the notion

that it is impossible for any given study to single-handedly

answer most key treatment questions. These are points that

we wish to reiterate as we discuss the implications of past

work for future studies. Our recommendations for future

investigators conducting research in this area follow from

each of our discussion points. Admittedly, a number of our

recommendations are applicable to research design gener-

ally and may not be unique to ADHD treatment outcome

research. Nonetheless, we believe a specific discussion of

how these points apply in this context is useful, particularly

since few such discussions appear in the ADHD literature

specifically, or the child treatment literature more

generally.

Recommendation 1

Use care to insure that your choice of baseline or initial

treatment, the timing of when additional treatments are

added, and the dosages of treatments employed match your

research question. In other words, be clear regarding

whether your goal is to study the benefit of adding a second

treatment to one that is already administered at full inten-

sity and considered primary, or whether you wish to

consider dosage effects of both treatments simultaneously

in order to evaluate the best dosage combination of treat-

ments from each modality; these are very different

questions. When the course of action chosen is the former,be cautious not to draw conclusions about the latter as part

of the same study. In other words, for example, if a child is

titrated to their maximum effective dose of medication and

behavior therapy is thereafter added as a secondary treat-

ment, avoid drawing conclusions about the relative

effectiveness of any other dosage combinations of these

two treatments. For example, based on this design, you

cannot rule out the possibility that the child might respond

just as well to a high intensity behavioral intervention

coupled with a minimal medication dose; such a compar-

ison must be made directly.

Recommendation 2

Use care not to draw conclusions about which compo-

nent(s) are responsible for the success or failure of an

intervention when you employ multiple treatments used

together as a package. Similarly, avoid concluding that

each of the components of a successful intervention are

necessary to create that successful outcome. In like fashion,

you cannot conclude that each of the components of an

unsuccessful intervention are not valuable, as you cannot

rule out the possibility that the various treatments under-

mined one another. In other words, empirical studies

employing a treatment package do not provide an evidence

base for the components of the treatment package. Hence,

if you design a study involving a treatment package, use

care that it is only the package about which you draw

conclusions. All other conclusions require a different

design.

Recommendation 3

This recommendation, derived from issues 3 and 4, has to

do with choice of outcome measures and how to reach a

summary conclusion involving multiple (and often con-

flicting) outcome measures. We believe that this is the area

in need of the greatest work, and with the greatest potential

to shape the field. As previously noted (Pelham et al.

2005c), studies comparing assessment strategies are badly

needed and few have been conducted to date. We see a

particularly great need for development of standardized

approaches to assessing treatment outcome that are


1 3


14/17

applicable across diverse developmental periods (e.g., from

preschool-age to adulthood). This is not to say that each

study or each developmental period should employ the

exact same measures, but rather, common principles

regarding domains assessed and types of measures required

for adequate coverage of a domain should guide the field,

with the goal of increasing comparability across studies

and across the lifespan. A logical approach might includecovering 45 key domains of functioning, and including

several established sources of information for each. These

measures might include both pragmatic measures such as

parent and teacher ratings (despite limitations such as rater

bias), as well as more objective strategies such as behavior

counts, peer nominations, and achievement testing. Based

on the wide range of effect sizes displayed in Table 1, even

for measures that purportedly tap the same domain, we

advise extreme caution in relying on a single measure or

type of measure and suggest that researchers err on the side

of comprehensiveness rather than parsimony.

In making this recommendation, however, we note thatwe would hope that any future evidence-based and stan-

dardized approach to outcome assessment does aim for

brevity; as any researcher or clinician knows, administering

fewer measures is logistically easier and more cost-effec-

tive than is administering more. Indeed, an important

recent review (Pelham et al. 2005c) summarizes the state

of the field in regard to assessment of ADHD and identifies

this as one of numerous pressing issues for future research.

Summarizing their conclusions is beyond the scope of this

article and, hence, the reader is referred directly to this

article. Among their key recommendations, however, is an

appeal for simplicity, parsimony, a focus on functional

problems as opposed to symptoms, and a return to greater

use of functional behavior analysis in both initial assess-

ment and in evaluating outcomes. These authors also

suggest the possibility of a briefer assessment approach to

ADHD based on a combination of simple strategies that

have heretofore been used in a variety of separate studies;

for example, a small number of items rated by parents and

teachers, behavior counts of an individualized target

behavior, teacher records of academic work completion

and accuracy, and whether or not a child has their school

supplies in their desk (Pelham et al. 2005c). These

researchers acknowledge: The kind of large-scale

research needed to validate this approach to assessment has

not yet been conducted, but clinicians can be confident

based on extant research that it has some empirical sup-

port (Pelham et al. 2005c, p. 468).

We agree with Pelham et al. (2005c) recommendation

that assessment of functional difficulties (as opposed to

DSM-IV symptoms) is most appropriate for clinical

intervention, and we similarly find their arguments for

brevity in assessment quite compelling. Still, we believe

that precisely the type of large-scale research to which

Pelham et al. refer is a necessary prerequisite that should

occur prior to adoption of this type of assessment strat-

egy. Thus, we do not endorse (at least not at the present

time) combining several simple assessment strategies,

each with some supportive evidence, in lieu of the cur-

rently accepted comprehensive assessment approach. In

our view, taking this approach to assessment in treatmentoutcome research may be premature in light of our find-

ings that choice of outcome measure can dramatically

affect the results of between-treatment comparisons.

Instead, we encourage researchers to pursue validation of

measurement models comprised of carefully selected

combinations of measures, and to directly compare dif-

fering combinations of measures in terms of their

usefulness, reliability and validity, incremental predictive

utility, applicability across diverse developmental periods,

sensitivity to treatment response, and cost. In other words,

we recommend an approach similar to our recommenda-

tion in regard to examination of treatment packages,whereby we argued that evidence for the package does

not imply evidence for the components and vice versa.

Although not without drawbacks, we believe this

approach has the greatest chance of encouraging

researchers and clinicians to select measures and treat-

ment components carefully, and to avoid adding measures

or treatments in the absence of evidence for their incre-

mental usefulness.

Recommendation 4

Finally, and at the risk of sounding prescriptive, we

encourage researchers to be explicit and emphatic about

what questions their studies are and are not equipped to

answer. No study is perfect, and no study answers all

questions; hence, we encourage open acknowledgement of

design limitations that influence interpretation of study

results. There are no penultimate studiesonly studies

designed to answer specific questions. Researchers are

encouraged to resist the temptation to go beyond their data

in drawing conclusions and to correct others, including

media representatives, who may portray their study results

in an overly zealous fashion. Most often such misrepre-

sentations are unintentional and a result of

misunderstanding of the nuances of the study designsan

understandable problem among lay consumers. We

encourage researchers to correct these misunderstandings,

even if doing so requires great effort and additional work.

This is critical as our work is read not only by our col-

leagues, but also guides policy issues, affects insurance

coverage and directs best practices guidelines. Interpret

each and every study cautiously, as if your own childs or


1 3


15/17

grandchilds future may be affected by it; in fact, it likely

will be.

Limitations

Just as no research study answers all study questions, so

too, no review covers all pertinent issues. Our focus was onseveral specific issues, at the expense of others that,

admittedly, may be equally or even more important. Before

closing, we wish to make some of these limitations explicit,

in the hope that other researchers may tackle some of these

issues that we leave unaddressed.

A key unaddressed issue has to do with the effects of

comorbidity on outcomes in incremental benefit studies.

Given the overwhelming number of comparisons already

summarized in Table1, we chose not to include comor-

bidity comparisons in the subset of studies that reported

them. However, as work by Jensen et al. (2001) has dem-

onstrated, different comorbid subgroups of children withADHD may respond better or worse to different treatments.

Hence, some of the lack of clarity of the big picture we

describe herein, might be clarified by such an analysis of

comorbid subgroups. We leave this task to a future team of

investigators.

Similarly, we did not evaluate the evidence separately

by age, gender, length of treatment, or differences in

sample characteristics (e.g., chosen by comprehensive

clinical assessments vs. DSM-based rating scales).

Although we include descriptive data in regard to some of

these characteristics in Table1, an in-depth analysis is

needed to clearly elucidate their effects. Again, this

remains a task for future work.

Finally, the issues we chose to highlight are admittedly

controversial ones. Many viewpoints are possible, and

considering differing perspectives is an important part of

the scientific process. We are certain that we have not

captured all possible issues, and even in regard to the issues

we chose to address, additional viewpoints were left un-

explicated. We hope that other researchers will assist in

filling the gaps we leave unaddressed and the perspectives

that we leave unexplicated. Nonetheless, if this article

serves to stimulate discussion regarding the importance of

design issues to study outcomes, and if it promotes con-

structive discussion about how to improve future work,

then we have accomplished our goals.

Summary

The goal of this article was to outline issues critical to

evaluating the literature on incremental benefit of multiple

effective treatments used together, vs. a single effective

treatment, for childhood ADHD. These issues included: (1)

sequencing and dosage of treatments being combined and

compared; (2) difficulty drawing valid conclusions about

individual components of treatment when treatment pack-

ages are employed; (3) differing results emerging from

measurement tools that purportedly measure the same

domain; and (4) the resultant difficulty in reaching a

summary conclusion when multiple outcome measuresyielding conflicting results are used. The implications of

these issues for the design and conduct of future studies

were discussed, with a particular emphasis on the need to

develop and validate measurement models that can guide

research.

Acknowledgments During the preparation of this manuscript, all

authors were supported in part by a grant from the National Institute

of Mental Health to the first author: R01 MH065899. The first author

also was supported in part by N01 MH12010 to B.S.G. Molina; and

the first and third authors were supported in part by R43 MH076359

to A. Terrazas. The views represented in this article are solely those of

the authors.

References

Abikoff, H., Hechtman, L., Klein, R. G., Gallagher, R., Fleiss, K.,

Etcovitch, J., et al. (2004a). Social functioning in children with

ADHD treated with long-term methylphenidate and multimodal

psychosocial treatment. Journal of the American Academy of

Child and Adolescent Psychiatry, 43, 820829.

Abikoff, H., Hechtman, L., Klein, R. G., Weiss, G., Fleiss, K.,

Etcovitch, J., et al. (2004b). Symptomatic improvement in

children with ADHD treated with long-term methylphenidate

and multimodal psychosocial treatment.Journal of the American

Academy of Child and Adolescent Psychiatry, 43, 802811.

American Psychiatric Association (1994). Diagnostic and statistical

manual of mental disorders(4th ed.). Washington, DC: Author.

Arnold, L. E., Chuang, S., Davies, M., Abikoff, H. B., Conners, C. K.,

Elliott, G. R., et al. (2004). Nine months of multicomponent

behavioral treatment for ADHD and effectiveness of MTA

fading procedures. Journal of Abnormal Child Psychology, 32,

3951.

Barkley, R. A. (2000). Commentary on the multimodal treatment

study of children with ADHD. Journal of Abnormal Child

Psychology, 28, 595599.

Barkley, R. A., Shelton, T. L., Crosswait, C., Moorehouse, M.,

Fletcher, K., Barrett, S., et al. (2000). Multi-method psycho-

educational intervention for preschool children with disruptive

behavior: Preliminary results at post-treatment. Journal of Child

Psychology and Psychiatry, 41, 319332.Cohen, J. (1988). Statistical power analysis for the behavioral

sciences(2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.

Cunningham, C. E. (1999). In the wake of the MTA: Charting a new

course for the study and treatment of children with attention-

deficit hyperactivity disorder. Canadian Journal of Psychiatry,

44, 9991006.

Ercan, E. S., Varan, A., & Deniz, U. (2005). Effects of combined

treatment on Turkish children diagnosed with attention-deficit/

hyperactivity disorder: A preliminary report. Journal of Child

and Adolescent Psychopharmacology, 15, 203219.

Flory, K., Molina, B. S. G., Pelham, W. E., Gnagy, E., & Smith, B.

(2006). Childhood ADHD predicts risky sexual behavior in


1 3


16/17

young adulthood. Journal of Clinical Child & Adolescent


Hechtman, L., Abikoff, H., Klein, R. G., Greenfield, B., Etcovitch, J.,

Cousins, L., et al. (2004a). Children with ADHD treated with

long-term methylphenidate and multimodal psychosocial treat-

ment: Impact on parental practices. Journal of the American

Academy of Child and Adolescent Psychiatry, 43, 830838.

Hechtman, L., Abikoff, H., Klein, R. G., Weiss, G., Respitz, C.,

Kouri, J., et al (2004b). Academic achievement and emotional

status of children with ADHD treated with long-term methyl-

phenidate and multimodal psychosocial treatment.Journal of the

American Academy of Child and Adolescent Psychiatry, 43,

812819.

Hoza, B., Gerdes, A. C., Mrug, S., Hinshaw, S. P., Bukowski, W. M.,

Gold, J. A., et al. (2005). Peer-assessed outcomes in the

multimodal treatment study of children with attention deficit

hyperactivity disorder.Journal of Clinical Child and Adolescent


Hoza, B., Johnston, C., Pillow, D. R., & Ascough, J. C. (2006).

Predicting treatment response for childhood attention-deficit/

hyperactivity disorder: Introduction of a heuristic model to guide

research.Applied and Preventive Psychology, 11, 215229.

Hoza, B., Kaiser, N. M., & Hurt, E. (in press). Evidence-based

treatments for attention-deficit/hyperactivity disorder (ADHD).

In M. Roberts, D. Elkin, & R. Steele (Eds.), Handbook of

evidence based therapies for children and adolescents. New

York: Springer.

Hoza, B., Owens, J. S., & Pelham, W. E. (1999). Attention-deficit/

hyperactivity disorder. In R.T. Ammerman, M. Hersen, & C. G.

Last (Eds.), Handbook of prescriptive treatments for children

and adolescents (2nd ed., pp. 6383). Boston: Allyn & Bacon.

Jensen, P. S., Hinshaw, S. P., Kraemer, H. C., Lenora N., Newcorn J.

H., Abikoff, H. B., et al. (2001). ADHD comorbidity findings

from the MTA study: Comparing comorbid subgroups. Journal

of the American Academy of Child & Adolescent Psychiatry, 40,

147158.

Klein, R. G., & Abikoff, H. (1997). Behavior therapy and methyl-

phenidate in the treatment of children with ADHD. Journal of

Attention Disorders, 2, 89114.

Klein, R. G., Abikoff, H., Hechtman, L., & Weiss, G. (2004). Design

and rationale of controlled study of long-term methylphenidate

and multimodal psychosocial treatment in children with ADHD.

Journal of the American Academy of Child and Adolescent

Psychiatry, 43, 792801.

Kolko, D. J., Bukstein, O. G., & Barron, J. (1999). Methylphenidate

and behavior modification in children with ADHD and comorbid

ODD or CD: Main and incremental effects across settings.

Journal of the American Academy of Child and Adolescent

Psychiatry, 38, 578586.

Mash, E. J., & Hunsley, J. (2005). Evidence-based assessment of

child and adolescent disorders: Issues and challenges.Journal of

Clinical Child & Adolescent Psychology, 34, 362379.

Molina, B. S. G., Pelham, W. E., Gnagy, E. M., Thompson, A. L., &

Marshal, M. P. (2007). Attention-deficit/hyperactivity disorderrisk for heavy drinking and alcohol use disorder is age specific.

Alcoholism: Clinical and Experimental Research, 31, 643654.

Mrug, S., Hoza, B., & Gerdes, A. C. (2001). Children with attention-

deficit/hyperactivity disorder: Peer relationships and peer-ori-

ented interventions. In W. Damon (Series Ed.), D. W. Nangle, &

C. A. Erdley (Eds.), New directions for child and adolescent

development: No. 91. The role of friendship in psychological

adjustment(pp. 5177). San Francisco: Jossey-Bass.

MTA Cooperative Group (1999). A 14-month randomized clinical

trial of treatment strategies for attention-deficit/hyperactivity

disorder. Archives of General Psychiatry, 56, 10731086.

Owens, J. S., & Hoza, B. (2003). The role of inattention and

hyperactivity/impulsivity in the positive illusory bias. Journal of

Consulting and Clinical Psychology, 71, 680691.

Pelham, W. E. (1999). The NIMH multimodal treatment study for

attention-deficit hyperactivity disorder: Just say yes to drugs

alone? Canadian Journal of Psychiatry, 44, 981990.

Pelham, W. E., Burrows-MacLean, L., Gnagy, E. M., Fabiano, G. A.,

Coles, E. K., Tresco, K. E., et al. (2005a). Transdermal

methylphenidate, behavioral, and combined treatment for chil-

dren with ADHD. Experimental and Clinical

Psychopharmacology, 13, 111126.

Pelham, W. E., Erhardt, D., Gnagy, E. M., Greiner, A. R., Arnold, L.

E., Abikoff, H. B., et al. (2007). Parent and teacher evaluation

of treatment in the MTA: Consumer satisfaction and perceived

effectiveness.University at Buffalo (submitted).

Pelham, W. E., Fabiano, G. A., Gnagy, E. M., Greiner, A. R., & Hoza,

B. (2005b). The role of summer treatment programs in the

context of comprehensive treatment for attention-deficit/hyper-

activity disorder. In E. D. Hibbs & P. S. Jensen (Eds.),

Psychosocial treatments for child and adolescent disorders:

Empirically based strategies for clinical practice (2nd ed., pp.

377409). Washington, DC: American Psychological

Association.

Pelham, W. E., Fabiano, G. A., & Massetti, G. M. (2005c). Evidence-

based assessment of attention deficit hyperactivity disorder in

children and adolescents. Journal of Clinical Child and Adoles-

cent Psychology, 34, 449476.

Pelham, W. E., Gnagy, E. M., Greiner, A. R., Hoza, B., Hinshaw, S.

P., Swanson, J. M., et al. (2000). Behavioral versus behavioral

and pharmacological treatment in ADHD children attending a

summer treatment program. Journal of Abnormal Child Psy-

chology, 28, 507525.

Pelham, W. E., & Hinshaw, S. P. (1992). Behavioral intervention for

attention deficit-hyperactivity disorder. In S. M. Turner, K. D.

Calhoun, & H. E. Adams (Eds.), Handbook of clinical behavior

therapy (Vol. 2, pp. 259283). New York: Wiley.

Pelham, W. E., & Hoza, B. (1996). Intensive treatment: A summer

treatment program for children with ADHD. In E. D. Hibbs & P.

S. Jensen (Eds.), Psychosocial treatments for child and adoles-

cent disorders: Empirically based strategies for clinical practice

(pp. 311340). Washington, DC: American Psychological

Association.

Pelham, W. E., Wheeler, T., & Chronis, A. (1998). Empirically

supported psychosocial treatments for attention deficit hyperac-

tivity disorder. Journal of Clinical Child Psychology, 27, 190

205.

Pfiffner, L. J., & McBurnett, K. (1997). Social skills training with

parent generalization: Treatment effects for children with

attention deficit disorder. Journal of Consulting and Clinical


Reitman, D., Hupp, S. D. A., OCallaghan, P. M., Gulley, V., &

Northup, J. (2001). The influence of a token economy and

methylphenidate on attentive and disruptive behavior during

sports with ADHD-diagnosed children. Behavior Modification,25, 305323.

Richters, J. E., Arnold, L. E., Jensen, P. S., Abikoff, H., Conners, C.

K., Greenhill, L. L., et al. (1995). NIMH collaborative multisite

multimodal treatment study of children with ADHD: I. Back-

ground and rationale. Journal of the American Academy of Child

& Adolescent Psychiatry, 34, 9871000.

Rosenthal, R. (1979). The file drawer problem and tolerance for

null results. Psychological Bulletin, 86, 638641.

Swanson, J. M., Elliott, G. R., Greenhill, L. L., Wigal, T., Arnold, L.

E., Vitiello, B., et al. (2007). Effects of stimulant medication on

growth rates across 3 years in the MTA follow-up. Journal of the


1 3


17/17

American Academy of Child and Adolescent Psychiatry, 46,

10151027.

Thompson, A. L., Molina, B. S. G., Pelham, W., & Gnagy, E. M.

(2007). Risky driving in adolescents and young adults with

childhood ADHD.Journal of Pediatric Psychology, 32, 745759.

Tutty, S., Gephart, H., & Wurzbacher, K. (2003). Enhancing

behavioral and social skill functioning in children newly diag-

nosed with attention-deficit hyperactivity disorder in a pediatric

setting.Developmental and Behavioral Pediatrics, 24, 5157.

van der Oord, S., Prins, P. J. M., Oosterlaan, J., & Emmelkamp, P. M.

G. (2007). Does brief, clinically based, intensive multimodal

behavior therapy enhance the effects of methylphenidate in

children with ADHD? European Child and Adolescent Psychi-

atry, 16, 4857.

Volkow, N. D., & Insel, T. R. (2003). What are the long-term effects

of methylphenidate treatment? Biological Psychiatry, 54,

13071309.

Wells, K. C., Chi, T. C., Hinshaw, S. P., Epstein, J. N., Pfiffner, L.,

Nebel-Schwalm, M., et al. (2006). Treatment-related changes in

objectively measured parenting behaviors in the multimodal

treatment study of children with attention-deficit/hyperactivity

disorder. Journal of Consulting and Clinical Psychology, 74,

649657.

Wells, K. C., Epstein, J. N., Hinshaw, S. P., Conners, C. K., Klaric, J.,

Abikoff, H. B., et al. (2000). Parenting and family stress

treatment outcomes in attention deficit hyperactivity disorder

(ADHD): An empirical analysis in the MTA study. Journal of

Abnormal Child Psychology, 28, 543553.


1 3

accesare_resursa.pdf

Documents

Transcript of accesare_resursa.pdf