Critique quality of the literature reviews conducted for two different types of studies- a quantitative and qualitative research.

Yoe1980

Critiquequalityoftheliteraturereviewsconductedfortwodifferenttypesofstudies.docx

Home >Nursing homework help >Critique quality of the literature reviews conducted for two different types of studies- a quantitative and qualitative research.

Quantitative research:

Qualitative research:

Quantitative Research on Critical Thinking and Predicting Nursing Students’ NCLEX-RN Performance Elizabeth M. Romeo, MS, CRNP, FNP

ABSTRACT The concept of critical thinking has been influential in

several disciplines. Both education and nursing in general have been attempting to define, teach, and measure this concept for decades. Nurse educators realize that critical thinking is the cornerstone of the objectives and goals for nursing students. The purpose of this article is to review and analyze quantitative research findings relevant to the measurement of critical thinking abilities and skills in undergraduate nursing students and the usefulness of critical thinking as a predictor of National Council Licen- sure Examination-Registered Nurse (NCLEX-RN) perfor- mance. The specific issues that this integrative review ex- amined include assessment and analysis of the theoretical and operational definitions of critical thinking, theoretical frameworks used to guide the studies, instruments used to evaluate critical thinking skills and abilities, and the role of critical thinking as a predictor of NCLEX-RN out- comes. A list of key assumptions related to critical think- ing was formulated. The limitations and gaps in the litera- ture were identified, as well as the types of future research needed in this arena.

Higher education has attempted to change the way in which the nursing curriculum is structured (Ad-ams, 1999). It is no longer acceptable to teach only knowledge-based facts and skills; instead, the emphasis has shifted toward guiding students to become lifelong, in- dependent critical thinkers (Lee, 2007). Starting in the late 1980s, colleges of nursing moved from evaluation of the cur- riculum to assessment of student outcomes (Riddell, 2007). Nursing faculty must continue this shift from an emphasis on teaching nursing content to one focusing on the applica- tion of nursing knowledge (Adams, 1999; Del Bueno, 2005). The application of nursing knowledge within the nurs- ing process is enhanced through utilization of the process of critical thinking (National League for Nursing [NLN], 2007). Nurse educators remain accountable for creating and implementing curricula that produce graduate nurses who are able to use critical thinking skills to formulate ap- propriate clinical and nursing judgments (Henriques, 2002; Hoffman, 2006; NLN, 2007; Youssef & Goodrich, 1996).

The National Council of State Boards of Nursing (NCSBN) is an organization through which boards of nursing act collaboratively on matters of public health, safety, and welfare (NCSBN, 2007). This organization is also responsible for the development of the licensing ex- aminations in nursing. Because the NCSBN considers critical thinking to be an important component in nursing education and the National Council Licensure Examina- tion-Registered Nurse (NCLEX-RN) measures graduates’ competency for entry into practice, it is presumed that critical thinking is one element of nursing that is tested by the NCLEX-RN (Giddens & Gloeckner, 2005). According to the NCSBN, the majority of items on the NCLEX-RN are written at the application and analysis level of Bloom’s taxonomy. Successful performance on the NCLEX-RN re- quires the use of critical thinking to correctly answer ques- tions at this cognitive level (Wacks, 2005; Wendt, 2003; Wendt & Brown, 2000).

Received: October 30, 2008 Accepted: October 27, 2009 Posted: March 31, 2010 Ms. Romeo is Clinical Instructor, Gwynedd-Mercy College, School

of Nursing, Gwynedd Valley, Pennsylvania. The author has no financial or proprietary interest in the material

presented herein. Address correspondence to Elizabeth M. Romeo, MS, CRNP,

FNP, Clinical Instructor, 405 Stonebridge Road, Perkasie, PA 18944; e-mail: [email protected].

doi:10.3928/01484834-20100331-05

ROMEO

METHOD FOR INTEGRATIVE REVIEW

The process that was used for this integrative review was the template proposed by Whittemore and Knafl (2005). Their method for conducting an integrative review includes the following five stages: a problem identification stage, with the definition of relevant terms; a literature search stage, to identify all significant literature on the topic; a data evaluation stage, with the recommendation to include similar designs; a data analysis stage that en- compasses data reduction, data display, data comparison, conclusion drawing, and verification; and a final presenta- tion stage that is generally reported in a table or diagram- matic form (Whittemore & Knafl, 2005).

Problem Identification Stage and Identification of Relevant Terms

Critical thinking is an attribute that enhances one’s skill in problem solving and decision making. The purpose of this article is to review and analyze recent quantitative research findings relevant to measuring critical thinking abilities and skills in undergraduate nursing students and critical thinking’s role as a predictor of NCLEX-RN performance. The specific issues that this integrative re- view will examine include assessment and analysis of the theoretical and operational definitions of critical thinking, theoretical frameworks used to guide the studies, tools to evaluate critical thinking skills and abilities, and the role of critical thinking as a predictor of NCLEX-RN outcomes. A list of key assumptions related to critical thinking will be formulated. The limitations and gaps in the literature will be identified, as well as the types of future research needed in this area.

Theoretical Definitions of Critical Thinking Nursing has evolved from a simplistic occupation to a

complex and highly technical profession; therefore, nurses have changed from being task-oriented team members to becoming autonomous health care providers (Allen, Ru- benfeld, & Scheffer, 2004). Due to the increasing complex- ity of the clinical setting, the need to develop and nur- ture critical thinking skills is paramount (Frye, Alfred, & Campbell, 1999). A concise definition of the concept of critical thinking is one that various disciplines continue to struggle with today (Akerson, 2001; Allen et al., 2004; Frye et al., 1999; Kataoka-Yahiro & Saylor, 1994).

The studies reviewed in this integrative review in- cluded a variety of definitions of critical thinking. The definition offered by Paul (1993) is that it is a method of examining one’s thinking with the goal of improving the thought process to make it clearer and more accu- rate (Frye et al., 1999). In the study conducted by Wacks (2005), the definition of critical thinking used was from the work of Watson and Glaser (1980), who defined it as a combination of one’s attitudes, level of knowledge, and skills. Akerson (2001) offered yet another definition of critical thinking that is specific to nursing and is based on the work of Kataoka-Yahiro and Saylor (1994). Accord-

ing to Akerson, critical thinking is a process involving critical, reflective, and reasonable thinking about prob- lems specific to nursing practice that do not have a single answer and is centered on deciding what to do or believe. It also has been defined as a cognitive procedure that drives problem solving and decision making (Henriques, 2002). Frost (2000) offered another definition of critical thinking that is specific to nursing: critical thinking is a process involving critical, reflective practice with a basis in sound reasoning of intelligent minds that are commit- ted to safe and effective patient care. The process of criti- cal thinking is best measured from a holistic perspective encompassing both abstract thinking and the practice skills that are unique to the nursing environment (Hoff- man, 2006).

One of the first intensive research studies done to de- velop a widely accepted theoretical definition for critical thinking was undertaken in 1990 by means of the Delphi method. A panel of 46 experts from the United States and Canada from different scholarly disciplines participated in this 2-year project in an effort to globally define critical thinking. Peter Facione, PhD, is internationally known for his work on the definition and measurement of critical thinking. His research on teaching and assessing critical thinking has been ongoing for the past 40 years. Dr. Fa- cione was a prominent member of The Delphi Report in 1990, which defined critical thinking as a decisive, self- regulated judgment that promotes a forum in which to ad- dress clinical and professional nursing issues in an effec- tive method. According to Facione (1990), “critical thinking is essential as a tool of inquiry” (p. 3). The ideal critical thinker continually draws on past experiences and one’s knowledge base to honestly and openly assess and resolve complex issues in an orderly fashion (Facione, 1990; Mor- ris, 1999; Morris, 1998).

Stewart and Dempsey (2005) defined critical thinking from the specific standpoint of the American Philosophi- cal Association (1990), which describes critical thinking dispositions. These dispositions include attributes such as “habits of the mind, intellectual virtues, a character- ological profile, and a set of attitudes toward thinking pro- cesses” (Stewart & Dempsey, 2005, p. 81). Critical think- ing has also been defined as a vibrant, purposeful, logical process that results in articulate decisions and judgments. The process of critical thinking in this definition encom- passes interpretation, analysis, evaluation, inference, explanation, and self-regulation (Whitehead, 2006). The final study reviewed did not provide a definition of critical thinking (Youssef & Goodrich, 1996).

One definition of critical thinking that is important to nursing and was not found in any of the reviewed studies is the definition based on the work of Scheffer and Ruben- feld (2000). They also used the Delphi technique with five rounds of input to achieve a definition of critical thinking specifically for the discipline of nursing. The participants in that study consisted of an international panel of expert nurses representing 9 countries and 23 states in the Unit- ed States who worked from 1995 to 1998 to develop the fol-

Journal of Nursing Education • Vol. 49, No. 7, 2010 379

QUANTITATIVE RESEARCH ON CRITICAL THINKING

lowing consensus definition of critical thinking in nursing (Scheffer & Rubenfeld, 2000):

Critical thinking in nursing is an essential component of professional accountability and quality nursing care. Criti- cal thinkers in nursing exhibit these habits of the mind: confidence, contextual perspective, creativity, flexibility, in- quisitiveness, intellectual integrity, intuition, open-mind- edness, perseverance, and reflection. Critical thinkers in nursing practice the cognitive skills of analyzing, applying standards, discriminating, information seeking logical rea- soning, predicting and transforming knowledge. (p. 357)

LITERATURE SEARCH

Data Collection The literature review was conducted via hand, Inter-

net, and database searches from January to April 2008. Computer searches used the Cumulative Index for Nursing and Allied Health Literature, ProQuest Nurs- ing Journals, and PubMed. Search words that were used in various combinations in- cluded “critical thinking,” “nursing, students,” “research instruments,” “quantitative research,” “pretest-posttest,” and “NCLEX-RN results.” The Cumulative Index for Nursing and Allied Health Literature, ProQuest, and PubMed data- bases produced 20, 10, and 23 articles, respectively, which were selected based on specific inclusion and exclusion cri- teria. The inclusion criteria were studies that were quanti- tative in design, research articles published in professional journals, and dissertations written in English, and they in- volved undergraduate nursing students, described the use of a critical thinking assessment tool, and were published between 1988 and 2008.

Undergraduate nursing students were the population selected because the majority of studies pertaining to critical thinking are based on this group. The inclusion of studies that contain a quantitative critical thinking tool was necessary to measure the independent variable. The rationale for the dates of the search was to coincide with the implementation of the current format for the NCLEX- RN as a pass/fail examination. Exclusion criteria included qualitative research, graduate nursing students compris- ing the population, editorials and articles published prior to 1988, studies involving a specific intervention to en- hance critical thinking with no correlation to NCLEX-RN outcomes, abstracts without full text, case studies, letters, or secondary sources.

There were 8 articles that met the criteria for inclu- sion in this project. An ancestry search and citation-in- dex search were conducted on the 8 articles, which were initially found in the 3 different Internet databases. An

additional 4 articles that met the criteria were found, for a total of 12 relevant articles for this integrative re- view.

Data Analysis and Interpretation This integrative review revealed four different meth-

ods that have been used to study the critical thinking abilities of nursing students in comparison to NCLEX-RN outcomes. The first method used was the measurement of critical thinking abilities at the beginning and at the end of the nursing curriculum, which was used as a predictor of NCLEX-RN success. A second method measured criti- cal thinking abilities once at either the beginning or the end of the nursing program and examined students’ criti- cal thinking abilities as a predictor of NCLEX-RN suc- cess. The third method used in one study measured criti- cal thinking by means of repeated measurements taken

five times over the course of the students’ education. The final method that was used in three studies examined the differences between two distinct types of students in regard to their critical think- ing abilities and NCLEX-RN scores. The analysis and in- terpretation of the articles are categorized by these methods. A summary table of the studies is located in the Table.

Key Assumptions from Integrative Review Several assumptions of critical thinking that were ei-

ther implied or explicitly stated became apparent in the course of this review (Hall, 1996; Stewart & Dempsey, 2005; Wacks, 2005):

Critical thinking is widely accepted as a skill that is necessary in the practice of professional nursing.

Critical thinking skills can be taught, learned, and measured.

Students in the studies make an honest effort to per- form well on the tests.

Data Comparison The state of the science in regard to studies that have

been conducted on critical thinking in nursing education and practice is expansive, as found during the process of this integrative review. However, few studies have exam- ined critical thinking as a predictor of NCLEX-RN perfor- mance. Several limitations in these studies were identified in the process of this review. Some of the major limitations identified were related to the lack of a theoretical or con- ceptual framework, sampling issues, the definition of criti- cal thinking, and measurement tools. The lack of quanti- tative studies available that use critical thinking as an independent variable and NCLEX-RN as the dependent variable substantiate the need for further research.

The lack of quantitative studies

available that use critical thinking as an

independent variable and NCLEX-RN as

the dependent variable substantiate the

need for further research.

ROMEO

Theoretical Frameworks Most quantitative research has the common underly-

ing goal of testing the relationships that are suggested by theories. Of the 12 studies that were reviewed, only 5 in- cluded a theoretical framework that guided the research (Akerson, 2001; Hoffman, 2006; Morris, 1999; Wacks, 2005; Whitehead, 2006). Because there is currently not one accepted theory of critical thinking, each of the 5 stud- ies used a different theory, which further demonstrated the infancy of the state of the science in studying critical thinking. The remaining research studies did not include any framework or theory as a guide for the study.

Theoretical Definition of Critical Thinking Because there is currently a lack of an accepted frame-

work for critical thinking, there is not a widely acknowl- edged and accepted theoretical definition. As previously noted, there is a plethora of both theoretical and opera- tional definitions for critical thinking. As a discipline, nursing is in need of agreement on and acceptance of one of the current, concise theoretical definitions of critical thinking for use in education and clinical practice (Adams, 1999; Frye et al., 1999; Hall, 1996; Stewart & Dempsey, 2005). Interestingly, neither of the current nursing- specific definitions of critical thinking—the one developed by the NLN for clinical nursing practice or the definition from the work of Scheffer and Rubenfeld (2000)—were used as the theoretical definition for critical thinking in any of the studies that were analyzed. Because the theo- retical definition of critical thinking is directly related to a critical thinking theory, the exploration of these two fac- tors needs be conducted simultaneously. The operational definitions of critical thinking will remain tool specific, with a need to continue studying the reliability and valid- ity of the measures of critical thinking.

Critical Thinking Measurement Tools The California Critical Thinking Skills Test. The

majority of the valid and reliable tools available to mea- sure critical thinking skills and abilities are not specific for use with nursing students. The measurement tool that was used most frequently was the California Critical Thinking Skills Test, which yielded mixed findings in the reviewed studies. The California Critical Thinking Skills Test is a 34-item multiple choice test that measures over- all critical thinking skills, in addition to 5 subscales that specifically assess analysis, evaluation, inference, and deductive and inductive reasoning (Brunt, 2005; Phillips, Chestnut, & Rospond, 2004). This is a timed test of 45 minutes, with a maximum score of 34 on the skills test (Brunt, 2005).

Positive relationships of critical thinking skills and NCLEX-RN success were found in half of the studies that used this tool (Giddens & Gloeckner, 2005; Hall, 1996; Morris, 1999), although no statistical significance was found in the remainder of the studies with the California Critical Thinking Skills Test (Akerson, 2001; Henriques, 2002; Morris, 1998).

The California Critical Thinking Disposition In- ventory. The California Critical Thinking Disposition In- ventory (CCTDI) is designed to measure a student’s procliv- ity to think critically (Phillips et al., 2004). There are seven identified dispositions or “habits of the mind” that are the elements of one’s character that move one toward using critical thinking skills. Each section has several questions that evaluate specific thought habits; there is a subscale score for each of the 7 subscales, in addition to a total score for the test. For each subscale, a score below 40 indicates a general weakness in that area, and a score above 50 rep- resents a consistent strength. A total CCTDI score below 280 represents serious overall deficiencies in the student’s critical thinking dispositions and a score above 350 shows overall strengths (Giddens & Gloeckner, 2005; Phillips et al., 2004). The 7 dispositions are as follows:

Truth-seeking is a desire to gain the best knowledge, even if it undermines one’s preconceptions or beliefs.

Open-mindedness is a tolerance of views other than your own, as well monitoring oneself for possible bias.

Analyticity is demonstrated by the demand for ap- plication of reason and evidence, awareness of problem situations, and an inclination to anticipate consequences.

Systematicity relates to one’s focus on approaching all levels of problems and valuing organization.

Critical thinking self-confidence is the trusting of one’s own reasoning skills.

Inquisitiveness relates to one’s curiosity and eager- ness to acquire knowledge and learn explanations.

Cognitive maturity is indicative of prudence in mak- ing, suspending, or revising judgment. It is awareness that multiple solutions can be acceptable (Facione, 2007; Phillips et al., 2004).

Construct validity of the CCTDI was reported by Stew- art and Dempsey (2005) as being established by Facione and Facione (1992). Validity for this instrument was not reported in the remainder of the studies (Giddens & Gloeckner, 2005; Henriques, 2002). Giddens and Gloeck- ner (2005) were the only ones who reported statistically significant findings when the total mean CCTDI exit scores were found to be higher in the first-time NCLEX- RN pass group than in the failure group.

The Watson-Glaser Critical Thinking Appraisal Tool. The Watson-Glaser Critical Thinking Appraisal (WGCTA) is a tool available to measure critical think- ing abilities with three different forms of the instrument available. Form A and Form B are parallel tools available for use as pretest and posttest measurements, and Form S is the latest, shortest version of the instrument (Wacks, 2005). The WGCTA Form S is composed of 16 scenarios and 40 items to be answered. This instrument has been found to be appropriate for assessing critical thinking ability in individuals with at least a ninth grade education. The as- sessment is divided into five subsets: inference, recogni- tion of assumptions, deduction, interpretation, and evalu- ation of arguments. In the liberal arts setting, upper-level students have higher scores than do entry-level students on the WGCTA (Frye et al., 1999).

Journal of Nursing Education • Vol. 49, No. 7, 2010 381

QUANTITATIVE RESEARCH ON CRITICAL THINKING

The WGCTA total score and various subsets of the score were found to be significant predictors of NCLEX-RN per- formance (Frye et al., 1999; Wacks, 2005). In contrast, Hall’s (1996) literature review suggested that WGCTA may not be a valid measure of critical thinking in nurs- ing students because it does not measure the cognitive process underlying the nursing process. The WGCTA may not be an accurate measure of the reasoning needed for critical thinking and, ultimately, for making reasonable clinical decisions. The question that arises both implicitly and explicitly in these studies is the appropriateness of non-nursing-specific tools as reliable and valid measures of critical thinking abilities in nursing students.

Other Critical Thinking Tools There were five lesser known tools specifically de-

signed to measure critical thinking in nursing students that were used by different researchers. These include the Assessment Technologies Institute’s (ATI) instru- ment, the Critical Thinking Assessment (CTA) test, and three tests by the Educational Resources Incorporated (ERI): the Nurse Entrance Test (NET), the Critical Think- ing Process Test (CTPT), and the RN Assessment Test. A final critical thinking mea- surement tool is the Scale of Judgment Abilities in Nurs- ing (SJAN).

The CTA is a 40-item ob- jective instrument that was developed to determine stu- dents’ performance on spe- cific critical thinking skills at the beginning and end of the nursing program. There are six to eight questions for each critical thinking skill on the test. The goal of the diagnostic pretest is for development of instructional strategies by nursing faculty to enhance higher thinking skills. The posttest measures the student outcomes after the instructional intervention and, as such, readiness for the NCLEX- RN. The six competencies that are incorporated into the ATI CTA are interpretation, analysis, evaluation, inference, explanation, and self-regulation (Whitehead, 2006). This particular nursing-specific tool used to mea- sure critical thinking has only been available since 2000 (Whitehead, 2006).

ERI designed three different tests to measure criti- cal thinking at various times during the nursing cur- riculum; however, only two of the tests will be described in this integrated review. The first test that was used to measure critical thinking skills in nursing students is the CTPT. This tool assesses the following five levels of abstract thinking: prioritizing/discriminating, inferen- tial reasoning, goal setting, application of knowledge, and evaluation of predicted outcomes (Hoffman, 2006).

The second test provided by ERI is the RN Assessment Test, used by both Stewart and Dempsey (2005) and Hoffman (2006) to assess the critical thinking abilities of nursing students prior to graduation. This assess- ment test provides a total score plus 32 subscale scores. Five of the subscales are related to critical thinking and are labeled as prioritizing/discriminating, infer- ential reasoning, main idea and predicting outcomes, application and knowledge, and evaluating predicted outcomes.

The SJAN is another instrument that examines the professional judgment of nursing students in four areas: legal/ethical, problem solving and decision making, com- munication, and leadership functioning (Youssef & Go- odrich, 1996). The purpose of the test is to examine an in- dividual’s ability to understand and apply knowledge. The tool is composed of 28 multiple-choice questions and has an equal number of questions in the four content areas. Reliability and validity of the SJAN were not addressed by the authors.

The tools that were developed to be used specifically with nursing students also had differing results. The ERI

tests results were found to have no statistical signifi- cance in relation to critical thinking and were even deemed a measure of safe practice rather than a direct measure of critical thinking (Stewart & Dempsey, 2005). Hoffman (2006) reported the exact opposite findings in her study using the three ERI tests that revealed a predic- tive relationship of critical thinking skills and NCLEX- RN pass rates. The ATI CTA and SJAN were used in stud- ies that compared two diverse

groups of nursing students and their critical thinking abil- ities. Neither study found significant differences between the groups’ critical thinking skills; however, no attempt was made in either study to correlate critical thinking to NCLEX-RN scores (Whitehead, 2006; Youssef & Goodrich, 1996) (Table).

Sampling Issues The majority of the studies used nonrandomized,

convenience samples due to the nature of the indepen- dent variable of critical thinking as not being conducive to manipulation. Sample sizes ranged from 24 (Akerson, 2001) to 437 nursing students (Hoffman, 2006). In gen- eral, the majority of studies included primarily nursing students at the BSN level with two studies that included only ADN students (Wacks, 2005; Youssef & Goodrich, 1996) and one study that compared the critical think- ing of BSN and ADN students (Whitehead, 2006). The majority of the authors stated that their studies were

The current state of science on

measurement of critical thinking in nursing

students reflects a lack of adequate

sample sizes to effectively determine the

relationship of critical thinking scores and

NCLEX-RN outcomes.

ROMEO

TA B

C ri

tic al

T hi

nk in

g in

N ur

si ng

E du

ca tio

S tu

dy C

on ce

pt s

Th eo

ry D

es ig

n an

d S

am pl

e To

ol R

el ia

bi lit

y an

d Va

lid ity

R es

ul ts

A ke

rs on

00 1)

C T,

N C

LE X

- R

N K

at ao

ka -Y

ah iro

d S

ay lo

r A

p ilo

t s tu

dy . A

c on

ve ni

en t

sa m

pl e

of 2

4 B

S N

s tu

de nt

s C

C T

S T

F or

m A

( K

ud ar

-R ic

ha rd

so n

in te

rn al

c on

si st

en cy

r el

ia bi

lit y

co ef

fic ie

fr om

0 .6

8 to

0 .7

0) . C

ro nb

ac h’

s al

ph a

= 0.

71 . C

on st

ru ct

v al

id ity

a cc

or di

ng to

T d

efi ni

tio n

by T

he D

el ph

i R ep

or t.

C on

cu rr

en t v

al id

ity b

y co

rr el

at io

n of

th e

C C

T S

T to

c ol

le ge

G PA

a nd

S AT

v er

ba l

sc or

es . N

C LE

X -R

N o

si gn

ifi ca

nt c

or re

la tio

n co

ul d

be te

st ed

b et

w ee

n C

sc or

es (

to ta

l a nd

s ub

sc al

e) a

nd N

C LE

X a

nd a

ls o

no te

st o

f co

rr el

at io

n be

tw ee

n th

e nu

m be

r of

q ue

st io

ns to

p as

s N

C LE

du e

to a

ll su

bj ec

ts p

as si

ng . T

im e

fo r

ex am

in at

io n

an d

C T

ill s

ha d

si gn

ifi ca

nt ly

n eg

at iv

e co

rr el

at io

Fr ye

, A lfr

ed ,

& C

am pb

el l

(1 99

C T,

N C

LE X

- R

N N

on e

id en

tifi ed

C ro

ss -s

ec tio

na l d

es ig

n an

d a

lo ng

itu di

na l d

es ig

n; a

nv en

ie nc

e sa

m pl

e of

1 32

es hm

an a

nd 7

7 se

ni or

s B

S N

s tu

de nt

s. L

on gi

tu di

na l

de si

gn w

ith 2

7 fr

es hm

an .

W G

C TA

( Fo

rm A

) pr

et es

t- po

st te

(C ro

nb ac

h’ s

al ph

a co

ef fic

ie nt

= 0

.6 9

to 0

.8 5)

. C on

te nt

v al

id ity

e xa

m in

ed in

ve ra

l s et

tin gs

a nd

c or

re la

te s

w ith

ec ifi

c ob

je ct

iv es

o f e

du ca

tio na

l pr

og ra

m s)

. N C

LE X

-R N

S ta

tis tic

al s

ig ni

fic an

t d iff

er en

ce b

et w

ee n

th e

un re

la te

d fr

es hm

an a

nd s

en io

r C

T s

co re

s. P

os iti

ve c

or re

la tio

n w

fo un

d be

tw ee

n C

T s

co re

s an

d N

C LE

X -R

N s

uc ce

ss a

m on

g 27

s en

io rs

. N o

st at

is tic

al ly

s ig

ni fic

an t d

iff er

en ce

b et

w ee

n th

e C

T s

co re

s of

th e

27 a

s fr

es hm

an a

nd a

ga in

a s

se ni

or s.

G id

de ns

G lo

ec kn

(2 00

C T,

N C

LE X

- R

N N

on e

id en

tifi ed

N on

ex pe

rim en

ta l e

x po

s fa

ct o

re se

ar ch

a pp

ro ac

h. A

nv en

ie nc

e sa

m pl

e of

2 18

S N

s tu

de nt

C C

T S

T a

nd C

C T

D I b

ot h

m ea

su re

d tw

ic e.

N C

LE X

-R N

C C

T D

I ( C

ro nb

ac h’

s al

ph a

= 0.

90 o

n to

ta l s

co re

a nd

ra ng

e of

0 .6

0 to

0 .7

8 on

s ub

sc al

es . C

on st

ru ct

lid ity

e st

ab lis

he d

by F

ac io

ne a

Fa ci

on e.

)

P ar

tic ip

an ts

w ho

p as

se d

th e

N C

LE X

-R N

h ad

h ig

he r

C C

T S

sc or

es th

an th

os e

w ho

fa ile

d. T

he re

w as

n o

st at

is tic

al ly

gn ifi

ca nt

c ha

ng e

in e

ith er

C T

s ki

lls o

r di

sp os

iti on

s ov

er th

e co

ur se

o f t

he n

ur si

ng c

ur ric

ul um

H al

l ( 19

96 )

C T,

N C

LE X

- R

N , C

og ni

tiv e

D ev

el op

m en

N on

e id

en tifi

ed C

or re

la tio

na l,

no n-

ex pe

rim en

ta l d

es ig

n 10

5 se

ni or

B S

N s

tu de

nt s

fr om

co nv

en ie

nc e

sa m

pl e

C C

T S

T a

nd L

E P

te st

s. N

C LE

X -R

N A

s a

gr ou

p, d

id n

ot d

em on

st ra

te C

T a

bi lit

y. C

T a

nd fi

na nc

ia l

st at

us w

er e

si gn

ifi ca

nt p

re di

ct or

s of

N C

LE X

-R N

s uc

ce ss

H en

riq ue

s (2

00 2)

C T,

N C

LE X

- R

N on

e id

en tifi

ed R

et ro

sp ec

tiv e

co rr

el at

io na

l de

si gn

. C on

ve ni

en ce

m pl

e of

1 52

B S

st ud

en ts

C C

T D

I, C

C T

S T,

N C

LE X

-R N

, P re

- N

ur si

ng G

ui da

nc e

te st

s co

re s,

N D

R T

or es

, D R

T s

co re

T he

C C

T D

I a nd

C C

T S

T d

id n

ot r

ev ea

l a s

ta tis

tic al

gn ifi

ca nc

e in

p re

di ct

in g

N C

LE X

-R N

s uc

ce ss

. T he

b es

t pr

ed ic

to rs

fo r

N C

LE X

-R N

s uc

ce ss

w er

e th

e N

D R

T a

nd D

R T

or es

H of

fm an

00 6)

C T,

N C

LE X

- R

N P

er ry

’s E

th ic

an d

In te

lle ct

ua l

D ev

el op

m en

t S

ch em

D es

cr ip

tiv e

co rr

el at

io na

l an

al ys

is o

f s ec

on da

da ta

fr om

a c

on ve

ni en

sa m

pl e

at a

B S

N p

ro gr

co m

pa rin

g 43

7 st

ud en

(t ra

di tio

na l,

n =

31 8

an d

ac ce

le ra

te d,

n =

1 19

)

E R

I: N

E T,

C T

P T,

a nd

R N

A ss

es sm

en t

Te st

. C T

P T

m ea

su re

d tw

ic e.

( C

T P

C ro

nb ac

h’ s

al ph

a =

0. 93

. V al

id ity

dr es

se d

w he

n st

ud en

ts ’ s

co re

s im

pr ov

ed fr

om p

re te

st to

p os

tte st

. R

N A

ss es

sm en

t t es

t: C

ro nb

ac h’

s al

ph a

= 0.

90 to

0 .9

4. V

al id

ity a

dd re

ss ed

d ev

el op

er w

ith in

cr ea

se in

s co

re s

fr om

p re

te st

to p

os tte

st .)

N E

T r

ea di

ng c

om pr

eh en

si on

s co

re s

(a fte

r co

m pl

et in

g 1s

t nu

rs in

g co

ur se

) w

er e

hi gh

ly p

re di

ct iv

e on

R N

A ss

es sm

en t

an d

N C

LE X

-R N

. C T

P T

a nd

R N

A ss

es sm

en t s

co re

s (a

t e nd

p ro

gr am

) w

er e

pr ed

ic tiv

e of

N C

LE X

-R N

M or

ris (

19 99

) C

T, N

C LE

X -

R N

B en

ne r’s

N ov

ic e

to E

xp er

t W

ilk in

so n’

s de

sc rip

tio n

th e

nu rs

in g

pr oc

es s

D es

cr ip

tiv e

co rr

el at

io na

l st

ud y

w ith

8 3

se ni

or B

S N

C T

S T

g iv

en o

nc e

C D

M N

S -c

lin ic

al d

ec is

io n

m ak

in g

Lo w

p os

iti ve

s ta

tis tic

al s

ig ni

fic an

ce b

et w

ee n

C T

a bi

lit y

an d

N C

LE X

-R N

p as

s ra

te s.

N o

si gn

ifi ca

nc e

be tw

ee n

th e

fir st

a nd

co nd

s en

io r

se m

es te

r C

T s

co re

M or

ris (

19 98

) C

T, N

C LE

X -

R N

N on

e id

en tifi

ed E

x po

st fa

ct o

co rr

el at

io na

l w

ith B

S N

8 2

st ud

en ts

C C

T S

T F

or m

s A

( ju

ni or

y ea

r) a

nd B

en io

r ye

ar );

N LN

P A

X -R

N N

o si

gn ifi

ca nt

d iff

er en

ce b

et w

ee n

pr et

es t a

nd p

os tte

st C

ab ili

ty . T

he v

ar ia

bl e

se t a

s a

w ho

le w

as r

el at

ed to

e xi

t C T

or es

, b ut

n o

on e

va ria

bl e

de m

on st

ra te

d an

in de

pe nd

en t

re la

tio ns

hi p

to C

T. N

o co

m bi

na tio

n of

v ar

ia bl

es w

as p

re di

ct iv

e of

N C

LE X

-R N

o ut

co m

es .

Journal of Nursing Education • Vol. 49, No. 7, 2010 383

QUANTITATIVE RESEARCH ON CRITICAL THINKING

TA B

C ri

tic al

T hi

nk in

g in

N ur

si ng

E du

ca tio

S tu

dy C

on ce

pt s

Th eo

ry D

es ig

n an

d S

am pl

e To

ol R

el ia

bi lit

y an

d Va

lid ity

R es

ul ts

S te

w ar

t &

D em

ps ey

00 5)

C T,

N C

LE X

- R

N N

on e

id en

tifi ed

Lo ng

itu di

na l d

es cr

ip tiv

e de

si gn

. A c

on ve

ni en

sa m

pl e

of 5

5 B

S N

ud en

C C

T D

I, E

R I,

an d

N C

LE X

-R N

C C

T D

I s ub

sc al

es a

nd to

ta l s

co re

w er

e bo

th s

ta tis

tic al

si gn

ifi ca

nt w

ith th

e C

T s

ub sc

al e

of in

fe re

nt ia

l r ea

so ni

ha vi

ng a

c or

re la

tio n

to G

PA . W

ilc ox

on s

ig ne

d ra

nk s

sh ow

no s

ig ni

fic an

t d iff

er en

ce s

be tw

ee n

C T

d is

po si

tio ns

o f

th os

e w

ho p

as se

d an

d th

os e

w ho

fa ile

d N

C LE

X -R

N . E

R I

R N

a ss

es sm

en t s

co re

s di

d no

t c or

re la

te w

ith a

ny C

C T

D I

su bs

co re

s— in

te rp

re te

d to

m ea

n th

at E

R I d

oe s

no t r

efl ec

t a

pr op

en si

ty to

C T.

W ac

ks (

20 05

) C

T, N

C LE

X -

R N

B an

du ra

R et

ro sp

ec tiv

e, d

es cr

ip tiv

e co

rr el

at io

na l d

es ig

C on

ve ni

en ce

s am

pl e

26 6

st ud

en ts

fr om

a n

A D

N p

ro gr

W G

C TA

( Fo

rm S

) A

C T,

G PA

, a nd

C LE

X -R

N . A

ll m

ea su

re d

on ce

on a

dm is

si on

. ( C

ro nb

ac h’

s al

ph a

co ef

fic ie

nt =

0 .8

1. C

on te

nt v

al id

ity

ex am

in ed

in s

ev er

al s

et tin

gs a

co rr

el at

es w

ith s

pe ci

fic o

bj ec

tiv es

o f

ed uc

at io

na l p

ro gr

am s.

)

A C

T to

C T

( W

G C

TA )

(p =

0 .0

00 6)

a nd

W G

C TA

to N

C LE

X -

R N

( p

< 0.

00 2)

w er

e si

gn ifi

ca nt

. N o

ot he

r co

m bi

na tio

ns w

er e

si gn

ifi ca

nt ; W

G C

TA w

as s

tr on

ge st

p re

di ct

or o

f N C

LE X

-R N

rfo rm

an ce

W hi

te he

(2 00

6) C

T, N

C LE

X -

R N

N ot

id en

tifi ed

n am

e— 6

co m

pe te

nc ie

P ilo

t s tu

dy w

ith a

nv en

ie nc

e sa

m pl

e of

9 st

ud en

ts (

N at

iv e

E ng

lis h,

n =

1 92

, N

on na

tiv e

E ng

lis h

sp ea

ki ng

, n =

1 7)

AT I C

TA -p

re te

st -p

os tte

st . (

C ro

nb ac

h’ s

al ph

a =

0. 69

4. V

al id

ity b

as ed

o n

th e

w or

k of

F ac

io ne

a nd

F ac

io ne

a nd

r es

ea rc

h fr

om T

he D

el ph

i P ro

je ct

N o

st at

is tic

al ly

s ig

ni fic

an t d

iff er

en ce

s w

ith N

C LE

X -R

pa ss

ag e

ra te

s an

d C

T s

co re

s be

tw ee

n gr

ou ps

Yo us

se f &

oo dr

ic h

(1 99

C T,

N C

LE X

- R

N N

on e

id en

tifi ed

A p

ro sp

ec tiv

e st

ud y.

on ve

ni en

ce s

am pl

e of

9 4

A D

N s

tu de

nt s

(t ra

di tio

na l,

n =

46 a

ac ce

le ra

te d,

n =

4 8)

S JA

N to

a ss

es s

C T

g iv

en d

ur in

g se

co nd

m es

te r

an d

18 m

on th

s la

te r

du rin

g th

e la

st w

ee k

of th

e pr

og ra

m . S

TA I

w as

g iv

en tw

ic e

to m

ea su

re a

nx ie

ty a

t th

e be

gi nn

in g

of s

em es

te r

an d

1 w

ee k

be fo

re fi

na ls

. N C

LE X

-R N

. R el

ia bi

lit y

an d

va lid

ity n

ot a

dd re

ss ed

. S JA

N s

ho w

ed n

o st

at is

tic al

s ig

ni fic

an t d

iff er

en ce

s be

tw ee

n th

e 2

gr ou

ps .

S TA

I r es

ul ts

a t t

he b

eg in

ni ng

o f t

he s

em es

te r

bo th

g ro

up s

ha d

hi gh

er s

tr es

s th

an a

t t he

e nd

, w ith

th e

ac ce

le ra

te d

gr ou

p ha

vi ng

h ig

he r

st re

ss le

ve ls

th an

th e

tra di

tio na

l g ro

up . A

t t he

d of

th e

se m

es te

r, bo

th g

ro up

s ha

d lo

w er

s tr

es s

w ith

ou t

an y

si gn

ifi ca

nt d

iff er

en ce

s be

tw ee

n th

e 2

gr ou

ps . N

C LE

X -R

pa ss

ag e

ra te

s be

tw ee

n th

e 2

gr ou

ps w

er e

no t s

ig ni

fic an

C T

= cr

iti ca

l t hi

nk in

g; N

C LE

X -R

N =

N at

io na

l C ou

nc il

Li ce

ns ur

e E

xa m

in at

io n

fo r

R eg

is te

re d

N ur

se s;

B S

N =

b ac

he lo

r of

s ci

en ce

in n

ur si

ng ; C

C T

S T

= C

al ifo

rn ia

C rit

ic al

T hi

nk in

g S

ki lls

T es

t; G

PA =

g ra

po in

t a ve

ra ge

; S AT

= S

ch ol

as tic

A pt

itu de

T es

t; W

G C

TA =

W at

so n

G la

se r

C rit

ic al

T hi

nk in

g A

pp ra

is al

; C C

T D

I = C

al ifo

rn ia

C rit

ic al

T hi

nk in

g D

is po

si tio

n In

ve nt

or y;

L E

P =

L ea

rn in

g E

nv iro

nm en

t P re

fe re

nc e;

D R

T =

N el

so n-

D en

ny R

ea di

ng T

es t;

D R

T =

D ia

gn os

tic R

ea di

ne ss

T es

t; E

R I =

E du

ca tio

na l R

es ou

rc e

In co

rp or

at ed

; N E

T =

th e

N ur

se E

nt ra

nc e

Te st

; C T

P T

= C

rit ic

al T

hi nk

in g

P ro

ce ss

T es

t; C

D M

N S

C lin

ic al

D ec

is io

n- M

ak in

g in

N ur

si ng

S ca

le ; N

LN P

A X

-R N

= T

he N

at io

na l L

ea gu

e fo

r N

ur si

ng P

re -A

dm is

si on

R N

e xa

m in

at io

n; A

D N

= A

ss oc

ia te

D eg

re e

in N

ur si

ng ; A

C T

= a

s ta

nd ar

di ze

d ac

hi ev

em en

t ex

am in

at io

n fo

r co

lle ge

a dm

is si

on s;

A T

I = A

ss es

sm en

t T ec

hn ol

og ie

s In

st itu

te ; C

TA =

C rit

ic al

T hi

nk in

g A

ss es

sm en

t; S

JA N

= S

ca le

o f J

ud gm

en t A

bi lit

ie s

in N

ur si

ng ; S

TA I =

T he

S ta

te -T

ra it

A nx

ie ty

ve nt

or y.

(C on

tin ue

ROMEO

conducted in one school of nursing and in a limited geo- graphic area, which greatly reduced the generalizability of the findings beyond their specific populations. In ad- dition, the small sample sizes were identified as prob- lematic by several researchers (Akerson, 2001; Frye et al., 1999; Hall, 1996; Morris, 1999; Morris, 1998; Stew- art & Dempsey, 2005) and the authors of only two of the studies reported that a power analysis was calcu- lated to determine the number of subjects needed to ad- dress statistical and practical significance (Henriques, 2002; Hoffman, 2006). The current state of science on measurement of critical thinking in nursing students reflects a lack of adequate sample sizes to effectively determine the relationship of critical thinking scores and NCLEX-RN outcomes. A final limitation related to sample size is the high degree of attrition that is found with longitudinal studies in this population (Stewart & Dempsey, 2005).

IMPLICATIONS FOR NURSING EDUCATION AND NURSING RESEARCH

The quest for identifying variables that are related to NCLEX-RN outcomes continues as programs of nurs- ing strive to adequately prepare students for success on the licensure examination and in their future nursing careers (Giddens & Gloecker, 2005; Morris, 1999). Ad- equate critical thinking abilities and skills have been identified as a major component of the NCLEX-RN and one’s ultimate success on this examination (Giddens & Glockner; Hall, 1996; Hennriques, 2002; Hoffman, 2006). There is a need to create quantitative nursing- specific tools that are rigorous, reliable, and valid to ad- equately assess critical thinking abilities and disposi- tions (Morris, 1999; Morris, 1998; Stewart & Dempsey, 2005; Youssef & Goodrich, 1996). Further research is needed to develop a nursing-focused critical thinking instrument to adequately measure critical thinking skills and abilities in nursing students (Hall, 1996). These measures of critical thinking abilities need to be designed for specific use with both beginning and gradu- ating nursing students. Utilization of these measures is an area that requires immediate attention in nursing research to identify those individuals who need to im- prove their skills. Additional exploration and research are then necessary to determine whether these quantifi- able measures can be used to examine the role of critical thinking skills and NCLEX-RN results.

NCLEX test-takers who failed the examination and those who passed on their first attempt need to be studied in greater detail to determine whether there is a statisti- cally significant difference between the critical thinking skills of those who failed and those who passed (Akerson, 2001; Morris, 1998). In addition, further quantitative stud- ies are needed to determine whether other factors, such as attention to the setting, timing, and consequences of ad- equate effort in the testing situation might also influence critical thinking outcomes and success rates of first-time

NCLEX test-takers (Morris, 1999; Morris, 1998; Wacks, 2005; Whitehead, 2006).

CONCLUSION

Critical thinking is an essential component in the competency of nurses in today’s health care environment (Henriques, 2002). The NLN Accrediting Commission (2007) and the American Association of Colleges of Nurs- ing (2008) have both mandated that nursing education in- clude content and activities specifically designed for the development and measurement of critical thinking skills. Continued empirical research needs to be conducted to better understand the role of critical thinking as a pre- dictor of NCLEX-RN outcomes. To meet these demands, nursing’s regulatory bodies must first agree on a definition of critical thinking for the discipline and then, after deter- mining a clear definition, settle on an applicable theory of critical thinking for nursing educators to use as a frame- work in curriculum development. Until nursing comes to accept a consensus definition of critical thinking, the disci- pline will continue to be disjointed in describing, teaching, and evaluating this skill in the profession. Once critical thinking is adequately defined, nursing faculty and stu- dents will have a clearer understanding of the role and degree of critical thinking that is used in the NCLEX-RN. This research related to critical thinking and NCLEX-RN will aid nursing faculty in meeting one of their many re- sponsibilities as an educator.

REFERENCES Adams, B. (1999). Nursing education for critical thinking: An in-

tegrated review. Journal of Nursing Education, 38, 111-119. Akerson, D. (2001). The relationship between critical thinking,

work experience, and study. Unpublished doctoral disserta- tion, Saint Louis University, Saint Louis, Missouri.

Allen, G., Rubenfeld, M., & Scheffer, B. (2004). Reliability of as- sessment of critical thinking. Journal of Professional Nursing, 20(1), 15-22.

American Association of Colleges of Nursing. (2008). The essen- tials of baccalaureate education. Retrieved from http://www. aacn.nche.edu/Education/pdf/BEdraft.pdf

American Philosophical Association. (1990). Critical thinking: A statement of expert consensus for purposes of educational as- sessment and instrument. The Delphi Report: Research find- ings and recommendations prepared for the committee on pre- college philosophy. (ERIC Document Reproduction Service No. ED 315-412.)

Assessment Technologies Institute, LLC. (2000). Retrieved from http://www.atitesting.com/

Brunt, B. (2005). Models, measurement, and strategies in devel- oping critical thinking skills. The Journal of Continuing Edu- cation in Nursing, 36, 255-261.

Del Bueno, D. (2005). A crisis in critical thinking. Nursing Educa- tion Perspectives, 26, 278-282.

Facione, P.A. (1990). Critical thinking: A statement of expert con- sensus for purposes of educational assessment and instruction: The Delphi Report. Millbrae, CA: The California Academic Press.

Facione, P.A. (2007). Critical thinking: What it is and why it counts. [Electronic Version]. Insight Assessment, 1-23. Re- trieved from http://www.insightassessment.com/t.html

Facione, P.A., & Facione, N.C. (1992). Test manual: The Cali-

Journal of Nursing Education • Vol. 49, No. 7, 2010 385

QUANTITATIVE RESEARCH ON CRITICAL THINKING

fornia Critical Thinking Disposition Inventory. Milbrae, CA: California Academic Press.

Frost, M.D. (2000). The critical thinking process test technical and developmental report. Shawnee, KS: Educational Resources, Inc.

Frye, B., Alfred, N., & Campbell, M. (1999). Use of the Watson- Glaser Critical Thinking Appraisal with BSN students. Nurs- ing & Health Care Perspectives, 20, 253-255.

Giddens, J., & Gloeckner, G.W. (2005). The relationship of critical thinking to performance on the NCLEX-RN. Journal of Nurs- ing Education, 44, 85-89.

Hall, G.H. (1996). Critical thinking in nursing education. Un- published doctoral dissertation, The University of Minnesota, Minneapolis.

Henriques, M.A.M. (2002). Predictors of performance on the Na- tional Council Licensure Examination for Registered Nurses. Unpublished doctoral dissertation, The University of South- ern Mississippi, Hattiesburg.

Hoffman, J.J. (2006). The relationships between critical thinking, program outcomes, and NCLEX-RN performance in tradition- al and accelerated nursing students. Unpublished doctoral dis- sertation, University of Maryland, College Park.

Kataoka-Yahiro, M., & Saylor, C. (1994). A critical thinking model for nursing judgment. Journal of Nursing Education, 33, 351- 356.

Lee, K. (2007). Online collaborative case study learning. Journal of College Reading and Learning 37, 82-100.

Morris, B.C. (1999). Relationship among academic achievement, clinical decision making, critical thinking, work experience and NCLEX-RN pass status. Unpublished doctoral disserta- tion. Arizona State University, United States, Arizona.

Morris, M.C. (1998). The relationship between critical thinking ability and selected educational variables in baccalaureate nursing students. Unpublished doctoral dissertation, Univer- sity of Southern Mississippi, Hattiesburg.

National Council Licensure Examination for Registered Nurses. (2007). NCLEX-RN Examination: Test plan for the National Council Licensure Examination for Registered Nurses. Re- trieved from https://www.ncsbn.org/index.htm

National League for Nursing. (2007). Critical thinking in clinical

nursing practice/RN examination. Retrieved from http://www. nln.org/testproducts/pdf/CTinfobulletin.pdf

Paul, R. (1993). Critical thinking: What every person needs to sur- vive in a rapidly changing world. Santa Rosa, CA: Foundation for Critical Thinking.

Phillips, C.R., Chestnut, R.J., & Rospond, R.M. (2004). The Cali- fornia Critical Thinking Instruments for benchmarking, pro- gram assessment, and directing curricular change. American Journal of Pharmaceutical Education, 68(4), 1-8.

Riddell, T. (2007). Critical assumptions: Thinking critically about critical thinking. Journal of Nursing Education, 46, 121-126.

Scheffer, B.K., & Rubenfeld, M.G. (2000). A consensus statement on critical thinking in nursing. Journal of Nursing Education, 39, 352-359.

Stewart, S., & Dempsey, L.F. (2005). A longitudinal study of bac- calaureate nursing students’ critical thinking dispositions. Journal of Nursing Education, 44, 81-84.

Wacks, G.J. (2005). Relationships among pre-admission charac- teristics in associate degree nursing programs as predictors of NCLEX-RN success. Unpublished doctoral dissertation, The University of Alabama, Birmingham.

Watson, G., & Glaser, E. (1980). Watson-Glaser Critical Thinking Appraisal manual. San Antonio, TX: Psychological Corpora- tion.

Wendt, A. (2003). The NCLEX-RN examination charting the course of nursing practice. Nurse Educator, 28, 276-280.

Wendt, A., & Brown, P. (2000). The NCLEX examination. Prepar- ing for future nursing practice. Nurse Educator, 25, 297-300.

Whitehead, T. D. (2006). Comparison of native versus nonnative English-speaking nurses on critical thinking assessments at entry and exit. Nursing Administration Quarterly, 30, 285- 290.

Whittemore, R., & Knafl, K. (2005). The integrative review: Up- dated methodology. Journal of Advanced Nursing, 52, 546- 553.

Youssef, F.A., & Goodrich, N. (1996). Accelerated versus tradi- tional nursing students: A comparison of stress, critical think- ing ability and performance. International Journal of Nursing Studies, 33, 76-82.

Copyright of Journal of Nursing Education is the property of SLACK Incorporated and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.

ARTICLE

Mental Health of Transgender and Gender Nonconforming Youth Compared With Their Peers Tracy A. Becerra-Culqui, PhD, MPH, OT/L, a Yuan Liu, PhD, b Rebecca Nash, MPH, c Lee Cromwell, MS, d W. Dana Flanders, MD, DSc, c Darios Getahun, MD, PhD, MPH, a Shawn V. Giammattei, PhD, e Enid M. Hunkeler, MA, f Timothy L. Lash, DSc, c Andrea Millman, MA, f Virginia P. Quinn, PhD, MPH, a Brandi Robinson, MPH, d Douglas Roblin, PhD, g David E. Sandberg, PhD, h Michael J. Silverberg, PhD, MPH, f Vin Tangpricha, MD, PhD, i, j Michael Goodman, MD, MPHc

BACKGROUND: Understanding the magnitude of mental health problems, particularly life- threatening ones, experienced by transgender and/or gender nonconforming (TGNC) youth can lead to improved management of these conditions. METHODS: Electronic medical records were used to identify a cohort of 588 transfeminine and 745 transmasculine children (3–9 years old) and adolescents (10–17 years old) enrolled in integrated health care systems in California and Georgia. Ten male and 10 female referent cisgender enrollees were matched to each TGNC individual on year of birth, race and/ or ethnicity, study site, and membership year of the index date (first evidence of gender nonconforming status). Prevalence ratios were calculated by dividing the proportion of TGNC individuals with a specific mental health diagnosis or diagnostic category by the corresponding proportion in each reference group by transfeminine and/or transmasculine status, age group, and time period before the index date. RESULTS: Common diagnoses for children and adolescents were attention deficit disorders (transfeminine 15%; transmasculine 16%) and depressive disorders (transfeminine 49%; transmasculine 62%), respectively. For all diagnostic categories, prevalence was severalfold higher among TGNC youth than in matched reference groups. Prevalence ratios (95% confidence intervals [CIs]) for history of self-inflicted injury in adolescents 6 months before the index date ranged from 18 (95% CI 4.4–82) to 144 (95% CI 36–1248). The corresponding range for suicidal ideation was 25 (95% CI 14–45) to 54 (95% CI 18–218). CONCLUSIONS: TGNC youth may present with mental health conditions requiring immediate evaluation and implementation of clinical, social, and educational gender identity support measures.

abstract

NIH

aDepartment of Research and Evaluation, Kaiser Permanente Southern California, Pasadena, California; Departments of bBiostatistics and Bioinformatics and cEpidemiology, Rollins School of Public Health, and iEmory School of Medicine, Emory University, Atlanta, Georgia; dCenter for Clinical and Outcomes Research, Kaiser Permanente Georgia, Atlanta, Georgia; eRockway Institute, Alliant International University, San Francisco, California; fDivision of Research, Kaiser Permanente, Northern California, Oakland, California; gMid-Atlantic Permanente Research Institute, Kaiser Permanente Mid-Atlantic States, Rockville, Maryland; hDepartment of Pediatrics, Medical School, University of Michigan, Ann Arbor, Michigan; and jAtlanta Veterans Affairs Medical Center, Atlanta, Georgia

Drs Becerra-Culqui and Goodman conceptualized and designed the study, contributed to the acquisition of data, conceptualized the analysis plan, coordinated the interpretation of results (including contributing expertise in epidemiologic methods and childhood developmental and/ or psychological outcomes), and drafted and finalized the manuscript; Drs Getahun, Nash, Quinn, Roblin, and Silverberg and Ms Hunkeler conceptualized and designed the study, contributed to the acquisition of data, critically reviewed the manuscript for important intellectual content

PEDIATRICS Volume 141, number 5, May 2018:e20173845

WHAT’S KNOWN ON THIS SUBJECT: Small, specialized, clinic- based studies reveal a high prevalence of mental health diagnoses and self-reported emotional and behavioral problems among transgender and/or gender nonconforming youth.

WHAT THIS STUDY ADDS: In this large cohort study of an unselected transgender and/or gender nonconforming group, youth experienced a high relative prevalence of mental health conditions such as anxiety, depression, and attention deficit disorders compared with their cisgender counterparts.

To cite: Becerra-Culqui TA, Liu Y, Nash R, et al. Mental Health of Transgender and Gender Nonconforming Youth Compared With Their Peers. Pediatrics. 2018;141(5):e20173845

by guest on January 12, 2021www.aappublications.org/newsDownloaded from

As gender identity develops, it may not match the gender of rearing or gender assigned at birth, which are typically based on the appearance of external genitalia.1 – 3 When gender identity differs from the one assigned at birth, the terms gender diverse or gender nonconforming may apply.4, 5 Although the natural history of gender nonconformity presented in early childhood remains an area of ongoing research, some gender nonconforming children may go on to adopt an identity that is different from their assigned gender (10%–30%, according to reported estimates).6, 7 Researchers in studies of gender development in the general population support that gender- typed behavior is noticeable and stable between 3 and 8 years of age, especially in children with relatively high or low gender-typical behavior.8, 9 Individuals may identify as transgender, a term that refers more narrowly to those whose identity is “opposite” of their assigned gender.10 Conversely, individuals who identify with the gender assigned to them at birth are sometimes referred to as cisgender.10

An important priority for the health of transgender and/or gender nonconforming (TGNC) children and adolescents is the identification and management of mental health conditions.11 – 13 These conditions may be related to gender dysphoria, which is defined as a feeling of distress when one’s assigned gender does not match their identity.14 In addition, children with gender nonconforming behavior may experience stress from prejudice and discrimination because of being part of a minority group, which can create or exacerbate emotional and behavioral problems.15

The literature on TGNC youth consistently reveals a high prevalence of self-reported emotional and behavioral problems and mental health diagnoses.16 – 23 Most of the available data used to address

the mental health status of TGNC youth come from specialized clinics providing care to this population.24 Although researchers in clinic- based studies offer detailed and high-quality data, 25, 26 they often lack information on individuals who have not sought or had no access to specialized care. Moreover, a reliance on specialized clinics to identify study participants may yield relatively small sample sizes, making it difficult to select comparable reference groups from the same underlying population.27

These issues motivated the researchers in the Study of Transition, Outcomes, and Gender (STRONG), which was designed to assess morbidity among TGNC people overall and in the transfeminine and transmasculine subgroups of different ages, and captured in any care setting. However, this study communication is focused on cohort members who first presented as TGNC before their 18th birthday. Our objectives in this study were to estimate the prevalence of mental health diagnoses among transfeminine and transmasculine children and adolescents at the time of their initial presentation (index date) and compare their mental health status to that of their cisgender counterparts.

METHODS

The STRONG was designed as an electronic medical record (EMR)– based retrospective and prospective cohort study of members at 3 Kaiser Permanente (KP) sites (Georgia, Northern California, and Southern California) in partnership with the coordinating center at the Emory University Rollins School of Public Health. These KP sites provide comprehensive health services to >8.8 million members who are sociodemographically diverse and representative of their respective communities.28, 29 In the clinical

setting, the identification of TGNC youth may begin at age 13 years during physical examinations as part of the psychosocial and/or behavioral assessment recommended by the American Academy of Pediatrics30; however, some TGNC children may be identified earlier or later in life. All activities were reviewed and approved by the institutional review boards of the 4 participating institutions.

As described previously, 31, 32 persons with first evidence of TGNC status between January 1, 2006, and December 31, 2014, were identified on the basis of International Classification of Diseases, Ninth Edition (ICD-9) codes and the presence of specific keywords in free-text clinical notes (Supplemental Table 5). TGNC status was then verified (Supplemental Fig 1). A second free-text program was developed with additional anatomy- related or gender-affirmation keywords, which were reviewed and adjudicated for transfeminine or transmasculine status (Supplemental Table 6). Transfeminine and/or transmasculine status was assigned by using demographic information from the EMRs of 220 children whose gender assignment could not be determined from text strings because a validation revealed that the demographic variable accurately reflected assigned gender in 96% of youth. Subjects with evidence of disorders of sex development (eg, variation of chromosomal, gonadal, and/or anatomic sex development) were excluded because they may have distinct medical histories and gender identity trajectories.33

Ten male and 10 female cisgender KP enrollees were matched to each member of the final validated TGNC cohort on the basis of year of birth, race and/or ethnicity, site, and membership year of the index date. Because reference group enrollees had not been identified as TGNC by the methods described above, they

BECERRA-CULQUI et al2 by guest on January 12, 2021www.aappublications.org/newsDownloaded from

http://pediatrics.aappublications.org/lookup/suppl/doi:10.1542/peds.2017-3845/-/DCSupplemental

were assumed to be cisgender (ie, no evidence that gender identity does not correspond to assigned gender at birth). The race and/or ethnicity categories used were non- Hispanic white, non-Hispanic African American, Asian American and/or Pacific Islander, Hispanic, and other races. Index date was defined on the basis of the first recorded evidence of TGNC status. For some TGNC cohort members, <10 matched reference cisgender males or females were available; no TGNC individual was matched to <7 referents of either sex.

Subjects 3 through 17 years old at the index date were included in this study. Children <3 years old were excluded to reduce possible instability in gender identification and mental health diagnoses among the cohort.8 The ICD-9 codes for mental health diagnoses were grouped into

categories of conditions according to recommendations from the Mental Health Research Network34: anxiety disorders, attention deficit disorders, autism spectrum disorders, bipolar disorders, conduct and/ or disruptive disorders, depressive disorders, eating disorders, other psychoses, personality disorders, schizophrenia spectrum disorders, self-inflicted injuries (including poisonings), substance use disorders, and suicidal ideation (Supplemental Table 7). People could be represented more than once if they had multiple diagnoses and were thus counted in each category for which they had a diagnosis.

The prevalence of mental health conditions in each of these categories was calculated for 2 time windows: any time (ever) and within 6 months before the index date. These 2 time

windows were selected to examine mental health status just proximal to TGNC identification and to capture longer-standing conditions diagnosed at earlier ages (eg, autism spectrum disorders). In these calculations, the numerator for each disorder or group of disorders included persons with at least 1 relevant diagnostic code recorded during the time interval of interest. All prevalence estimates were calculated separately for transfeminine and transmasculine subjects within 2 age groups: 3 to 9 years (children) and 10 to 17 years (adolescents). Age categorization was selected to separately represent young school- aged children and adolescents by using the adolescent starting age of 10 years, corresponding to the World Health Organization’s definition.35 To assess differences in the severity of the conditions of interest, additional

PEDIATRICS Volume 141, number 5, May 2018 3

TABLE 1 Characteristics of the TGNC Children and Adolescents Enrolled in the STRONG

Child and Adolescent Characteristics Transfeminine Cohort Transmasculine Cohort

Transfeminine Cohort, n (%)

Reference Males, n (%)

Reference Females, n (%)

Transmasculine Cohort, n (%)

Reference Males, n (%)

Reference Females, n (%)

Age, a y 3–9 161 (27) 1605 (28) 1598 (28) 90 (12) 892 (12) 888 (12) 10–17 427 (73) 4206 (72) 4204 (72) 655 (88) 6448 (88) 6459 (88) Health plan site KPNC 344 (59) 3392 (58) 3378 (58) 431 (58) 4238 (58) 4245 (58) KPSC 227 (39) 2249 (39) 2254 (39) 295 (40) 2915 (40) 2912 (40) KPGA 17 (2.9) 170 (2.9) 170 (2.9) 19 (2.6) 187 (2.6) 190 (2.6) Race and/or ethnicity Non-Hispanic white 268 (46) 2633 (45) 2629 (45) 374 (50) 3663 (50) 3671 (50) Non-Hispanic African American 53 (9.0) 523 (9.0) 521 (9.0) 57 (7.7) 564 (7.7) 564 (7.7) Asian American and/or Pacific Islander 37 (6.3) 367 (6.3) 370 (6.4) 60 (8.1) 596 (8.1) 594 (8.1) Hispanic 179 (30) 1779 (31) 1772 (31) 204 (27) 2018 (27) 2020 (27) Other and/or unknown 51 (8.7) 509 (8.8) 510 (8.8) 50 (6.7) 499 (6.8) 498 (6.8) Use ever before index date (average visits per y) 0 22 (3.7) 325 (5.6) 321 (5.5) 23 (3.1) 367 (5.0) 350 (4.8) <1 15 (2.6) 568 (9.8) 530 (9.1) 25 (3.4) 763 (10) 746 (10) 1–3 127 (22) 1815 (31) 1879 (32) 154 (21) 2583 (35) 2562 (35) >3–6 171 (29) 1712 (29) 1781 (31) 240 (32) 2200 (30) 2235 (30) >6–9 115 (20) 757 (13) 763 (13) 107 (14) 795 (11) 777 (11) >9 137 (23) 631 (11) 528 (9.1) 196 (26) 630 (8.6) 675 (9.2) Unknown 1 (0.2) 3 (0.1) 0 (0.0) 0 (0.0) 2 (0.0) 2 (0.0) Use 6 mo before index date (average visits per mo) 0 101 (17) 2225 (38) 2062 (36) 111 (15) 2825 (39) 2502 (34) <1 258 (44) 2846 (49) 2903 (50) 277 (37) 3531 (48) 3640 (50) 1–2 141 (24) 552 (9.5) 625 (11) 195 (26) 757 (10) 901 (12) >2 87 (15) 186 (3.2) 212 (3.7) 162 (22) 225 (3.1) 302 (4.1) Unknown 1 (0.2) 2 (0.0) 0 (0.0) 0 (0.0) 2 (0.0) 2 (0.0) Total 588 (2.1) 5811 (21) 5802 (21) 745 (2.7) 7340 (27) 7347 (27)

KPGA, Kaiser Permanente Georgia; KPNC, Kaiser Permanente Northern California; KPSC, Kaiser Permanente Southern California. a Assessed at index date (date of first evidence of transgender status in EMRs).

by guest on January 12, 2021www.aappublications.org/newsDownloaded from

http://pediatrics.aappublications.org/lookup/suppl/doi:10.1542/peds.2017-3845/-/DCSupplemental

analyses were conducted for admittance or most serious diagnoses associated with hospitalizations.

Each prevalence estimate in the TGNC cohort was compared with corresponding estimates among matched cisgender male and female referents. For ease in presenting results, cisgender males and females will be referred to as male or female referents. Referents were assigned the same index date as the matched TGNC cohort member. For rare events (prevalence ≤10% in both the TGNC and referent cohorts), the prevalence ratios (PRs) were approximated by calculating the odds ratios with exact 95% confidence intervals (CIs). For events with >10% prevalence in either group, PRs and CIs were calculated by using logistic regression with the log link option.

In addition to the primary analysis that captures the true prevalence of mental health conditions, we

conducted sensitivity analyses to address possible differences in the prevalence of mental health conditions because of differences in health care visit frequency (use) between TGNC cohort members and those in the referent groups. We excluded the index date from the time window, and when the sample size was sufficient (>5 cases in each group), the PR estimates were adjusted for use of care. Average health care use was calculated for each individual by dividing the total number of visits by the cumulative duration of enrollment; this was expressed as the number of visits per year of enrollment in the analyses of “ever” prevalence and as the number of visits per month of enrollment in the analyses within 6 months before the index date. In adjusted analyses, average use was dichotomized for each time interval as above (high) or below (low) the median by using cutoffs for the overall population.

Analyses were conducted by using SAS version 9.4 (SAS Institute, Inc, Cary, NC) with custom macros developed at the Biostatistics and Bioinformatics Shared Resource at the Winship Cancer Institute of Emory University.36

RESULTS

A total of 2164 cohort candidates 3 to 17 years of age at the index date were initially identified in the EMR. After validation, 1347 (62%) were confirmed as TGNC. People excluded from the TGNC cohort were most often those with keywords referring to family or partners, standard disclaimers not related to care (eg, listing indications for hormone use), or evidence of disorders of sex development. After excluding subjects with unknown gender assigned at birth (N = 14), the final analysis data set was based on a cohort of 1333 subjects matched with

BECERRA-CULQUI et al4

TABLE 2 Prevalence of Mental Health Diagnoses in TGNC Children Ages 3–9 Years Relative to Those in Referent Groups

Categories of Mental Health Disorders

Prevalence in Transfeminine Subjects, N (%)

PR (95% CI)a in Transfeminine Subjects

Prevalence in Transmasculine Subjects, N (%)

PR (95% CI)a in Transmasculine Subjects

Versus Reference

Males

Versus Reference Females

Versus Reference Males

Versus Reference Females

All diagnoses of interest Ever before index date 51 (31.7) 3.0 (2.3–3.9) 5.7 (4.2–7.7) 31 (34.4) 3.3 (2.3–4.6) 5.4 (3.7–7.8) 6 mo before index date 46 (28.6) 6.0 (4.3–8.4) 13.0 (8.7–19.6) 26 (28.9) 5.9 (3.8–9.0) 10.7 (6.4–17.8) Anxiety disorders Ever before index date 19 (11.8) 4.4 (2.6–7.4) 6.3 (3.6–10.9) 14 (15.6) 6.3 (3.3–11.9) 6.0 (3.2–11.3) 6 mo before index date 15 (9.3) 16.3 (6.7–41.4) 23.3 (8.8–68.5) 9 (10.0) 9.8 (3.4–27.6) 12.2 (4.0–37.3) Attention deficit disorders Ever before index date 24 (14.9) 3.3 (2.1–5.0) 6.1 (3.8–9.9) 14 (15.6) 2.8 (1.6–4.9) 6.9 (3.6–13.2) 6 mo before index date 22 (13.7) 5.0 (3.1–8.1) 10.9 (6.1–19.6) 14 (15.6) 4.2 (2.3–7.6) 12.6 (5.9–26.8) Autism spectrum disorders Ever before index date 8 (5.0) 2.2 (0.9–4.9) 11.8 (3.7–38.9) 0 NC NC 6 mo before index date 8 (5.0) 3.9 (1.5–9.5) 20.8 (5.5–95.3) 0 NC NC Conduct and/or disruptive

disorders Ever before index date 12 (7.5) 3.3 (1.5–6.7) 14.2 (5.4–38.8) 7 (7.8) 6.2 (2.0–17.5) 12.3 (3.5–45.5) 6 mo before index date 8 (5.0) 8.3 (2.8–23.8) 83.0 (11.0–3707.3) b 13.7 (2.3–95.1) 20.5 (2.9–229.2) Depressive disorders Ever before index date 9 (5.6) 6.7 (2.5–17.0) 7.8 (2.9–20.6) 10 (11.1) 27.5 (7.7–123.0) 12.3 (5.0–30.5) 6 mo before index date 6 (3.7) 8.8 (2.4–31.1) 12.3 (3.1–51.5) 8 (8.9) 43.0 (8.4–422.3) 28.5 (6.7–170.2) Eating disorders Ever before index date b 3.7 (0.8–12.6) 6.7 (1.4–28.8) 0 NC NC 6 mo before index date b 10.0 (0.1–787.4) 10.0 (0.1–784.0) 0 NC NC

NC, not calculated because there were 0 cases among transgender subjects. a Logistic regression with exact 95% CIs for rare events (prevalence ≤10% in both the transgender and referent cohorts) or logistic regression with log link and approximate 95% CIs for not-rare events (>10% prevalence in either the transgender or referent cohorts). b Fewer than 5 cases were not reported.

by guest on January 12, 2021www.aappublications.org/newsDownloaded from

PEDIATRICS Volume 141, number 5, May 2018 5

TA BL

E 3

Pr ev

al en

ce o

f M en

ta l H

ea lth

D ia

gn os

es in

T GN

C Ad

ol es

ce nt

s Ag

es 1

0– 17

Y ea

rs R

el at

iv e

to T

ho se

in R

ef er

en t G

ro up

Ca te

go ri

es o

f M en

ta l H

ea lth

D is

or de

rs Pr

ev al

en ce

Tr an

sf em

in in

e Su

bj ec

ts , N

( %

)

PR (

95 %

C I)a

in T

ra ns

fe m

in in

e Su

bj ec

ts Pr

ev al

en ce

Tr an

sm as

cu lin

e Su

bj ec

ts , N

( %

)

PR (

95 %

C I)a

in T

ra ns

m as

cu lin

e Su

bj ec

Ve rs

us R

ef er

en ce

M al

es Ve

rs us

R ef

er en

ce F

em al

es Ve

rs us

R ef

er en

ce M

al es

Ve rs

us R

ef er

en ce

F em

al es

Al l d

ia gn

os es

o f i

nt er

es t

er b

ef or

e in

de x

da te

30 3

(7 1.

0) 3.

0 (2

.8 –3

.3 )

3. 6

(3 .3

–3 .9

) 48

8 (7

4. 5)

3. 0

(2 .8

–3 .2

) 3.

7 (3

.4 –3

.9 )

m o

be fo

re in

de x

da te

25 3

(5 9.

3) 7.

8 (6

.8 –8

.9 )

8. 6

(7 .5

–9 .8

) 42

9 (6

5. 5)

7. 8

(7 .1

–8 .7

) 8.

7 (7

.8 –9

.6 )

An xi

et y

di so

rd er

Ev er

b ef

or e

in de

x da

te 15

9 (3

7. 2)

5. 0

(4 .2

–5 .8

) 4.

2 (3

.6 –4

.9 )

25 5

(3 8.

9) 4.

9 (4

.3 –5

.6 )

4. 0

(3 .5

–4 .5

)

6 m

o be

fo re

in de

x da

te 10

6 (2

4. 8)

18 .0

( 13

.3 –2

4. 4)

9. 2

(7 .2

–1 1.

8) 17

5 (2

6. 7)

15 .7

( 12

.5 –1

9. 6)

8. 7

(7 .2

–1 0.

5) At

te nt

io n

de fic

it di

so rd

er s

er b

ef or

e in

de x

da te

10 7

(2 5.

1) 2.

0 (1

.7 –2

.4 )

5. 3

(4 .3

–6 .5

) 10

6 (1

6. 2)

1. 3

(1 .0

–1 .5

) 3.

3 (2

.7 –4

.1 )

m o

be fo

re in

de x

da te

68 (

15 .9

) 3.

8 (2

.9 –4

.9 )

9. 0

(6 .6

–1 2.

4) 69

( 10

.5 )

2. 5

(1 .9

–3 .3

) 6.

4 (4

.6 –8

.8 )

Au tis

m s

pe ct

ru m

d is

or de

Ev er

b ef

or e

in de

x da

te 31

( 7.

3) 4.

1 (2

.6 –6

.4 )

25 .2

( 12

.7 –5

2. 9)

24 (

3. 7)

1. 8

(1 .1

–2 .8

) 7.

6 (4

.3 –1

3. 5)

m o

be fo

re in

de x

da te

25 (

5. 9)

8. 1

(4 .6

–1 4.

3) 26

0. 8

(4 2.

5– 10

73 3.

9) 17

( 2.

6) 3.

4 (1

.8 –6

.1 )

17 .2

( 7.

4– 42

.1 )

Bi po

la r

di so

rd er

Ev er

b ef

or e

in de

x da

te 23

( 5.

4) 9.

9 (5

.3 –1

8. 5)

10 .3

( 5.

5– 19

.5 )

34 (

5. 2)

8. 6

(5 .2

–1 3.

9) 8.

0 (4

.9 –1

2. 9)

m o

be fo

re in

de x

da te

16 (

3. 8)

18 .1

( 7.

5– 46

.9 )

14 .8

( 6.

4– 35

.6 )

19 (

2. 9)

11 .3

( 5.

5– 23

.3 )

14 .8

( 6.

9– 32

.8 )

Co nd

uc t a

nd /o

r di

sr up

tiv e

di so

rd er

Ev er

b ef

or e

in de

x da

te 60

( 14

.1 )

2. 8

(2 .1

–3 .7

) 6.

6 (4

.8 –9

.0 )

59 (

9. 0)

1. 7

(1 .3

–2 .3

) 4.

5 (3

.2 –6

.2 )

m o

be fo

re in

de x

da te

22 (

5. 2)

5. 3

(3 .0

–9 .1

) 12

.6 (

6. 4–

25 .2

) 27

( 4.

1) 5.

5 (3

.3 –9

.0 )

10 .6

( 5.

9– 19

.1 )

De pr

es si

ve d

is or

de rs

er b

ef or

e in

de x

da te

20 7

(4 8.

5) 5.

8 (5

.1 –6

.7 )

4. 4

(3 .9

–5 .0

) 40

3 (6

1. 5)

7. 0

(6 .4

–7 .8

) 5.

7 (5

.2 –6

.2 )

m o

be fo

re in

de x

da te

17 2

(4 0.

3) 23

.5 (

18 .2

–3 0.

4) 10

.1 (

8. 4–

12 .2

) 32

6 (4

9. 8)

22 .8

( 19

.0 –2

7. 3)

13 .3

( 11

.5 –1

5. 4)

Ea tin

g di

so rd

er s

er b

ef or

e in

de x

da te

18 (

4. 2)

7. 7

(3 .9

–1 4.

9) 3.

3 (1

.8 –5

.7 )

28 (

4. 3)

6. 0

(3 .6

–9 .8

) 3.

2 (2

.0 –5

.0 )

m o

be fo

re in

de x

da te

11 (

2. 6)

18 .5

( 6.

2– 61

.2 )

6. 1

(2 .6

–1 3.

8) 19

( 2.

9) 27

.5 (

11 .0

–7 7.

6) 8.

7 (4

.4 –1

7. 0)

Ps yc

ho se

Ev er

b ef

or e

in de

x da

te 19

( 4.

5) 19

.5 (

8. 6 –

47 .3

) 12

.2 (

5. 9–

25 .5

) 32

( 4.

9) 12

.2 (

7. 0–

21 .3

) 14

.4 (

8. 1–

25 .9

)

6 m

o be

fo re

in de

x da

te 10

( 2.

3) 20

.1 (

6. 2–

75 .4

) 10

0. 6

(1 4.

2– 43

75 .0

) 18

( 2.

8) 22

.7 (

9. 4–

60 .6

) 30

.4 (

11 .5

–9 3.

8) Pe

rs on

al ity

d is

or de

Ev er

b ef

or e

in de

x da

te 10

( 2.

3) 14

.4 (

4. 9 –

44 .7

) 11

.2 (

4. 1–

31 .2

) 15

( 2.

3) 11

.6 (

5. 1–

26 .6

) 7.

9 (3

.7 –1

6. 6)

m o

be fo

re in

de x

da te

b 19

.8 (

2. 8–

22 0.

0) 19

.8 (

2. 8–

21 9.

9) 9

(1 .4

) 29

.9 (

7. 4–

17 2.

1) 29

.9 (

7. 4–

17 2.

4) Sc

hi zo

ph re

ni a

sp ec

tr um

d is

or de

Ev er

b ef

or e

in de

x da

te 5

(1 .2

) 49

.7 (

5. 5–

23 57

.0 )

24 .9

( 4.

1– 26

1. 7)

13 (

2. 0)

21 .7

( 7.

7– 69

.9 )

32 .6

( 10

.0 –1

37 .8

)

6 m

o be

fo re

in de

x da

te b

c 14

.8 (

1. 7–

17 8.

2) 10

( 1.

5) 99

.8 (

14 .2

–4 33

8. 3)

50 .0

( 10

.6 –4

70 .2

) Se

lf- in

fli ct

ed in

ju ri

Ev er

b ef

or e

in de

x da

te 11

( 2.

6) 3.

9 (1

.8 –8

.2 )

4. 1

(1 .8

–8 .6

) 54

( 8.

2) 14

.0 (

9. 1–

21 .8

) 8.

7 (5

.9 –1

2. 8)

m o

be fo

re in

de x

da te

7 (1

.6 )

69 .9

( 9.

0– 31

59 .2

) 17

.5 (

4. 4–

81 .7

) 28

( 4.

3) 14

3. 7

(3 6.

1– 12

47 .8

) 20

.5 (

10 .4

–4 2.

4) Su

bs ta

nc e

us e

di so

rd er

Ev er

b ef

or e

in de

x da

te 33

( 7.

7) 3.

0 (1

.9 –4

.5 )

3. 7

(2 .4

–5 .6

) 46

( 7.

0) 2.

4 (1

.7 –3

.4 )

3. 3

(2 .3

–4 .7

)

6 m

o be

fo re

in de

x da

te 24

( 5.

6) 5.

8 (3

.3 –9

.8 )

8. 9

(4 .9

–1 6.

0) 34

( 5.

2) 4.

5 (2

.9 –6

.9 )

8. 2

(5 .0

–1 3.

2) Su

ic id

al id

ea tio

Ev er

b ef

or e

in de

x da

te 32

( 7.

5) 17

.8 (

9. 7–

33 .6

) 11

.3 (

6. 5–

19 .4

) 68

( 10

.4 )

21 .2

( 13

.8 –3

3. 2)

11 .0

( 7.

7– 15

.9 )

m o

be fo

re in

de x

da te

21 (

4. 9)

54 .2

( 18

.2 –2

18 .3

) 31

.0 (

12 .6

–8 6.

7) 47

( 7.

2) 45

.2 (

22 .9

–9 7.

1) 24

.9 (

14 .3

–4 4.

6) a L

og is

tic r

eg re

ss io

n w

ith e

xa ct

9 5%

C Is

fo r

ra re

e ve

nt s

(p re

va le

nc e

≤ 10

% in

b ot

h th

e tr

an sg

en de

r an

d re

fe re

nt c

oh or

ts )

or lo

gi st

ic r

eg re

ss io

n w

ith lo

g lin

k an

d ap

pr ox

im at

e 95

% C

Is fo

r no

t-r ar

e ev

en ts

( >1

0% p

re va

le nc

e in

e ith

er th

e tr

an sg

en de

r or

r ef

er en

t c oh

or ts

). b

Fe w

er th

an 5

c as

es w

er e

no t r

ep or

te d.

c N ot

c al

cu la

te d

be ca

us e

th er

e w

er e

0 ca

se s

am on

g re

fe re

nt s.

by guest on January 12, 2021www.aappublications.org/newsDownloaded from

BECERRA-CULQUI et al6

TA BL

E 4

Pr ev

al en

ce o

f H os

pi ta

liz at

io n

fo r

M en

ta l H

ea lth

D ia

gn os

es in

T GN

C Ch

ild re

n Ag

es 1

0– 17

Y ea

rs R

el at

iv e

to T

ho se

in R

ef er

en t G

ro up

Ca te

go ri

es o

f M en

ta l H

ea lth

D is

or de

rs Pr

ev al

en ce

Tr an

sf em

in in

e Su

bj ec

ts , N

( %

)

PR (

95 %

C I)a

in T

ra ns

fe m

in in

e Su

bj ec

ts Pr

ev al

en ce

Tr an

sm as

cu lin

e Su

bj ec

ts , N

( %

)

PR (

95 %

C I)a

in T

ra ns

m as

cu lin

e Su

bj ec

Ve rs

us R

ef er

en ce

M al

es Ve

rs us

R ef

er en

ce F

em al

es Ve

rs us

R ef

er en

ce M

al es

Ve rs

us R

ef er

en ce

F em

al es

Al l d

ia gn

os es

o f i

nt er

es t

er b

ef or

e in

de x

da te

58 (

13 .6

) 9.

9 (6

.9 –1

4. 0)

8. 9

(6 .3

–1 2.

5) 99

( 15

.1 )

10 .2

( 7.

8– 13

.3 )

7. 8

(6 .1

–1 0.

6 m

o be

fo re

in de

x da

te 33

( 7.

7) 43

.9 (

19 .7

–1 10

.7 )

35 .1

( 16

.7 –8

0. 4)

59 (

9. 0)

35 .3

( 20

.4 –6

4. 1)

21 .9

( 13

.7 –3

5. 8)

An xi

et y

di so

rd er

Ev er

b ef

or e

in de

x da

te 9

(2 .1

) 9.

0 (3

.2 –2

4. 9)

6. 0

(2 .3

–1 4.

8) 25

( 3.

8) 15

.0 (

7. 7–

29 .8

) 13

.4 (

7. 1–

26 .0

)

6 m

o be

fo re

in de

x da

te 6

(1 .4

) 59

.8 (

7. 2–

27 57

.2 )

29 .9

( 5.

3– 30

3. 7)

13 (

2. 0)

43 .4

( 11

.9 –2

38 .3

) 32

.6 (

10 .0

–1 37

.8 )

At te

nt io

n de

fic it

di so

rd er

Ev er

b ef

or e

in de

x da

te 20

( 4.

7) 9.

8 (5

.0 –1

9. 1)

14 .7

( 7.

0– 31

.7 )

14 (

2. 1)

4. 0

(2 .0

–7 .7

) 9.

4 (4

.2 –2

0. 9)

m o

be fo

re in

de x

da te

6 (1

.4 )

19 .9

( 4.

2– 12

3. 6)

59 .8

( 7.

2– 27

55 .8

) b

c 9.

9 (1

.3 –7

4. 0)

Au tis

m s

pe ct

ru m

d is

or de

Ev er

b ef

or e

in de

x da

te b

6. 6

(1 .4

–2 8.

0) 39

.7 (

3. 9–

19 57

.7 )

5 (0

.8 )

5. 0

(1 .3

–1 6.

0) 8.

3 (2

.0 –3

2. 6)

m o

be fo

re in

de x

da te

b c

c b

3. 3

(0 .1

–4 1.

0) c

Bi po

la r

di so

rd er

Ev er

b ef

or e

in de

x da

te 12

( 2.

8) 30

.3 (

9. 1–

12 9.

6) 13

.5 (

5. 2–

36 .4

) 18

( 2.

8) 10

.1 (

4. 9–

20 .7

) 9.

1 (4

.5 –1

8. 2)

m o

be fo

re in

de x

da te

b 39

.7 (

3. 9–

19 58

.7 )

19 .8

( 2.

8– 21

9. 9)

8 (1

.2 )

15 .9

( 4.

6– 62

.0 )

19 .9

( 5.

3– 90

.7 )

Co nd

uc t a

nd /o

r di

sr up

tiv e

di so

rd er

Ev er

b ef

or e

in de

x da

te 10

( 2.

3) 16

.8 (

5. 5–

56 .4

) 16

.8 (

5. 5–

56 .4

) 10

( 1.

5) 5.

9 (2

.4 –1

3. 6)

6. 7

(2 .7

–1 5.

6 m

o be

fo re

in de

x da

te 5

(1 .2

) 49

.7 (

5. 5–

23 57

.0 )

c b

39 .6

( 3.

9– 19

51 .1

) 39

.6 (

3. 9–

19 54

.4 )

De pr

es si

ve d

is or

de rs

er b

ef or

e in

de x

da te

41 (

9. 6)

21 .1

( 12

.1 –3

8. 1)

10 .3

( 6.

4– 16

.4 )

83 (

12 .7

) 17

.8 (

12 .5

–2 5.

2) 9.

1 (6

.8 –1

2. 1)

m o

be fo

re in

de x

da te

26 (

6. 1)

68 .0

( 23

.4 –2

69 .3

) 54

.3 (

20 .4

–1 82

.2 )

49 (

7. 5)

10 4.

0 (4

1. 5–

33 5.

9) 21

.7 (

12 .9

–3 7.

2) Ea

tin g

di so

rd er

Ev er

b ef

or e

in de

x da

te b

6. 6

(0 .5

–5 7.

7) 2.

8 (0

.3 –1

4. 9)

6 (0

.9 )

29 .8

( 5.

3– 30

2. 1)

4. 0

(1 .3

–1 0.

6 m

o be

fo re

in de

x da

te b

c 3.

3 (0

.1 –4

1. 0)

b 29

.8 (

5. 3–

30 2.

1) 4.

0 (1

.3 –1

0. 9)

Ps yc

ho se

Ev er

b ef

or e

in de

x da

te 8

(1 .9

) 26

.7 (

6. 4–

15 6.

9) 11

.4 (

3. 6–

37 .2

) 10

( 1.

5) 20

.0 (

6. 2–

74 .7

) 12

.5 (

4. 4–

36 .6

)

6 m

o be

fo re

in de

x da

te 5

(1 .2

) 49

.7 (

5. 5–

23 57

.0 )

49 .7

( 5.

5– 23

55 .8

) b

19 .8

( 2.

8– 21

9. 1)

9. 9

(1 .8

–5 3.

3) Pe

rs on

al ity

d is

or de

Ev er

b ef

or e

in de

x da

te b

39 .7

( 3.

9– 19

58 .7

) 19

.8 (

2. 8–

21 9.

9) 8

(1 .2

) 26

.5 (

6. 3–

15 5.

7) 8.

9 (3

.0 –2

6. 0)

m o

be fo

re in

de x

da te

0 (0

) NC

NC 7

(1 .1

) c

69 .7

( 8.

9– 31

44 .6

) Sc

hi zo

ph re

ni a

sp ec

tr um

d is

or de

Ev er

b ef

or e

in de

x da

te b

c 14

.8 (

1. 7–

17 8.

2) b

9. 9

(1 .8

–5 3.

2) 13

.2 (

2. 2–

90 .4

)

6 m

o be

fo re

in de

x da

te 0

(0 )

NC NC

b 9.

9 (0

.1 –7

73 .9

) 9.

9 (0

.1 –7

75 .2

) Se

lf- in

fli ct

ed in

ju ri

Ev er

b ef

or e

in de

x da

te b

6. 6

(0 .5

–5 7.

7) 4.

9 (0

.4 –3

4. 6)

13 (

2. 0)

65 .2

( 14

.7 –5

96 .2

) 13

.0 (

5. 3–

33 .4

)

6 m

o be

fo re

in de

x da

te b

c c

7 (1

.1 )

c 34

.8 (

6. 6–

34 4.

4) Su

bs ta

nc e

us e

di so

rd er

Ev er

b ef

or e

in de

x da

te 8

(1 .9

) 5.

7 (2

.1 –1

4. 7)

4. 2

(1 .6

–1 0.

1) 16

( 2.

4) 6.

2 (3

.1 – 1

2. 0)

6. 2

(3 .1

–1 2.

6 m

o be

fo re

in de

x da

te 6

(1 .4

) 29

.9 (

5. 3–

30 3.

9) 12

.0 (

3. 0–

49 .7

) 9

(1 .4

) 11

.2 (

3. 8–

33 .5

) 11

.2 (

3. 8–

33 .6

) Su

ic id

al id

ea tio

Ev er

b ef

or e

in de

x da

te 9

(2 .1

) 15

.1 (

4. 8–

51 .7

) 5.

6 (2

.2 –1

3. 6)

29 (

4. 4)

19 .8

( 10

.2 –4

0. 1)

11 .5

( 6.

5– 20

.4 )

m o

be fo

re in

de x

da te

7 (1

.6 )

c 23

.3 (

5. 3–

14 0.

2) 17

( 2.

6) 24

.5 (

9. 6–

70 .1

) 21

.5 (

8. 7–

57 .7

)

NC , n

ot c

al cu

la te

d be

ca us

e th

er e

w er

e 0

ca se

s am

on g

tr an

sg en

de r

su bj

ec ts

. a L

og is

tic r

eg re

ss io

n w

ith e

xa ct

9 5%

C Is

fo r

ra re

e ve

nt s

(p re

va le

nc e

≤ 10

% in

b ot

h th

e tr

an sg

en de

r an

d re

fe re

nt c

oh or

ts )

or lo

gi st

ic r

eg re

ss io

n w

ith lo

g lin

k an

d ap

pr ox

im at

e 95

% C

Is fo

r no

t-r ar

e ev

en ts

( >1

0% p

re va

le nc

e in

e ith

er th

e tr

an sg

en de

r or

r ef

er en

t c oh

or ts

). b

Fe w

er th

an 5

c as

es w

er e

no t r

ep or

te d.

c N ot

c al

cu la

te d

be ca

us e

th er

e w

er e

0 ca

se s

am on

g re

fe re

nt s.

by guest on January 12, 2021www.aappublications.org/newsDownloaded from

13 151 reference males and 13 149 reference females.

The cohort included 588 (44%) transfeminine and 745 (56%) transmasculine children and adolescents (Table 1). Children <10 years old represented 27% of the transfeminine cohort and 12% of the transmasculine cohort. Compared with TGNC children (n = 251), in which 36% (n = 90) were transfeminine, 61% (n = 655) of adolescents (n = 1082) were transmasculine. More than 45% of subjects in both groups were white; Hispanics represented 30% of transfeminine and 27% of transmasculine subjects, whereas the remainder of the study population was approximately equally distributed among African Americans, Asian Americans and/ or Pacific Islanders, and persons whose race and/or ethnicity was characterized as other or unknown. Health care use levels were much higher in both transfeminine and transmasculine subjects than in those in the corresponding reference groups.

The most common diagnostic categories among TGNC children 3 to 9 years of age were attention deficit disorders (15% transfeminine; 16% transmasculine) and anxiety disorders (12% transfeminine; 16% transmasculine; Table 2). The PR (95% CI) estimates for attention deficit disorders ranged from 2.8 (95% CI 1.6–4.9) to 13 (95% CI 5.9–27). The PR (95% CI) estimates for anxiety disorders ranged from 4.4 (95% CI 2.6–7.4) to 23 (95% CI 8.8–69) depending on the time window before the index date and the reference group. Among transfeminine children, 5% had an autism spectrum disorder diagnosis; however, no cases were observed in transmasculine children. For all the diagnostic categories, the most pronounced PR estimates were observed within the 6-month

period before the index date. Among transfeminine children, the highest PR (95% CI) estimate was for conduct and/or disruptive disorders relative to reference females (83 [95% CI 11–3707]). Among transmasculine children, the highest PR (95% CI) estimate was for depressive disorders relative to reference males (43 [95% CI 8.4–422]). Additional analyses of the prevalence of hospitalizations by mental health diagnostic category were not possible in this age group because of small sample sizes.

In the adolescent group (age 10–17 years), like in the younger age group, attention deficit disorders and anxiety disorders remained common (“ever” prevalence: 25% transfeminine and 16% transmasculine; 40% both transfeminine and transmasculine, respectively; Table 3). The diagnostic category with the highest prevalence in this age group was depressive disorders, which were found in 49% of transfeminine and 62% of transmasculine subjects. For all diagnostic categories, PR estimates used to compare STRONG adolescents to matched reference groups were highest within 6 months before the index date. Compared with reference females, transfeminine and transmasculine adolescents experienced particularly pronounced increased prevalence in psychoses (PR 101 and 95% CI 14–4375; PR 30 and 95% CI 12–94, respectively). Additionally, the PR estimates among transfeminine subjects were particularly elevated for autism spectrum disorders (PR 261; 95% CI 43–10 734) and among transmasculine subjects for schizophrenia spectrum disorders (PR 50; 95% CI 11–470) compared with reference females. Compared with reference males, PR estimates for suicidal ideation and self-inflicted injuries for transfeminine subjects were 54 (95% CI 18–218) and 70

(95% CI 9.0–159), respectively, which were also high among transmasculine subjects, (45 [95% CI 23–97] and 144 [95% CI 14–4338], respectively).

When prevalence estimates were limited to mental health conditions recorded during hospitalizations, the patterns among adolescents generally remained the same. In several instances, however, the PR estimates could not be calculated because of the absence of cases in the reference groups (Table 4).

The median cutoff values used for adjusted analyses were 3.2 average visits per year for the “ever” analyses and 0.2 average visits per month for the 6-month analyses. The prevalence estimates were slightly attenuated or remained approximately the same for most diagnostic categories. However, some estimates changed appreciably. For children 3 to 9 years, adjusting for use 6 months before and excluding the index date produced the largest decrease in the PR (95% CI) for anxiety disorders, from 23 (95% CI 8.8–69) to 9.0 (95% CI 2.9–29) when transfeminine children were compared with reference males (Supplemental Table 8). The PR (95% CI) for suicidal ideation among transfeminine adolescents compared with reference males within 6 months of the index date decreased from 54 (95% CI 18–218) to 38 (95% CI 12–159; Supplemental Table 9).

DISCUSSION

The results of this study reveal that among TGNC youth, mental health conditions, specifically anxiety and depression, are common and often severe among adolescents, as evidenced by diagnoses associated with hospitalizations. Gender nonconforming children (3–9 years of age) have a higher prevalence of anxiety and attention deficit disorders compared with their cisgender counterparts. In nearly all

PEDIATRICS Volume 141, number 5, May 2018 7 by guest on January 12, 2021www.aappublications.org/newsDownloaded from

http://pediatrics.aappublications.org/lookup/suppl/doi:10.1542/peds.2017-3845/-/DCSupplemental

instances, mental health diagnoses were more common in the TGNC youth than in referent children and adolescents.

These results support findings from previous research in which the sample sizes were much smaller.17, 19 –22, 37 – 42 Researchers in a survey of 101 transfeminine and transmasculine patients ages 12 to 24 years in a transgender youth clinic in Los Angeles found that 35% had symptoms of depression and >50% had suicidal thoughts.19 In comparison, we found that adolescents had a higher prevalence (40%–60%) of depression but a lower prevalence of suicidal ideation (5%–10%). In a medical record abstraction study of 97 transfeminine and transmasculine patients ages 4 to 20 years presenting to the Gender Management Service Clinic at Boston Children’s Hospital, 44% presented with a significant psychiatric history, 21% had a history of self-mutilation, and 9% had documentation of suicide attempts.22 In a UK study, a baseline chart review of children 5 to 11 years old referred to a national specialty clinic revealed that 17% had symptoms of anxiety, and 15% had a history of suicidal ideation, self-harm, and/or a diagnosis of attention-deficit/hyperactivity disorder recorded before entering services.39 Our results for children were similar for demonstrated anxiety (9%–16%) and attention deficit disorders (14%–16%). Direct comparisons to the current study are challenging because there are methodological differences. Two important differences are the way in which mental health conditions were ascertained and presentations of age. In addition, we included a broader population of children and adolescents who were not necessarily seeking treatment for gender-related issues.

In recent years, researchers in several studies have suggested that gender dysphoria may be associated

with autism spectrum disorders.43 – 45 The most widely cited evidence supporting this hypothesis comes from a study of 204 children and/or adolescents referred to the Gender Identity Clinic in Amsterdam.46 The presence of an autism spectrum disorder was established via a standardized diagnostic interview, 47 yielding a prevalence of 10% among transfeminine patients and 4% among transmasculine patients, which was reported by the authors to be higher than the 1% estimate reported in the general population. The prevalence of autism spectrum disorders in our study was somewhat lower (7% in transfeminine and 3% in transmasculine subjects across both age groups), but our case ascertainment was based on documented diagnostic codes, and the denominator in our calculations was not limited to children with established gender dysphoria. With these differences in mind, our results are generally comparable to those reported in the Dutch study.

The gender ratio in this TGNC cohort reveals that transfeminine youth may present earlier in age than transmasculine individuals, which may pose a unique challenge to the early identification of mental health needs in transmasculine children and adolescents. Historically, researchers in studies of TGNC adolescents have reported a greater proportion of transfeminine than transmasculine subjects, but in recent years, the direction of the transmasculine: transfeminine ratio appears to have changed.48 For example, researchers in 1 recent study observed that transmasculine youth with gender dysphoria (aged 12–24 years) presented in significantly higher numbers than their transfeminine counterparts.19 Our data, which were based on EMRs, were used to confirm this observation. Therefore, providers should also be aware of the growing transmasculine population

needing timely and appropriate medical and psychosocial services.

An important contribution of the STRONG to the extant literature is its relatively large cohort, which allowed for focusing on previously understudied groups (such as young children), and an evaluation of relatively rare events (such as hospitalizations). In addition, the current study was based on children and adolescents who were not necessarily in specialized care and enrolled in a large health care system; and we did not require participant opt-in. The availability of a well-defined source population allowed for matching transfeminine and transmasculine study subjects to male and female referents of the same age, race and/ or ethnicity, and geographic region. This design feature permitted direct comparisons of prevalence estimates among transfeminine, transmasculine, and cisgender referent groups.

A limitation of this study is its cross- sectional design. Although we were able to retrospectively ascertain mental health conditions before the index date and we matched on the basis of membership year, a differential ascertainment of diagnoses could have occurred. The identification of the TGNC cohort was based on health care use, which is different from the matched referent groups. Results from sensitivity analyses adjusting for use and excluding the index date revealed a similar or slight attenuation of the PR results for most diagnostic categories. However, when adjusting for use 6 months before the index date, a more notable attenuation of PRs was seen in anxiety disorders in transfeminine children and suicidal ideation in transfeminine adolescents compared with reference males, indicating possible higher surveillance of mental health conditions in the several months

BECERRA-CULQUI et al8 by guest on January 12, 2021www.aappublications.org/newsDownloaded from

before cohort identification. Nevertheless, this baseline study reveals that TGNC youth experience a multitude of mental health problems before initial presentation. However, there is indication that TGNC children who receive meaningful gender identity support do not necessarily experience elevated rates of depression and anxiety.49 As the STRONG cohort follow-up extends, it will be possible to examine temporal changes in the frequency and severity of mental health problems, particularly in relation to the age of gender affirmation, which is an area of considerable uncertainty, and the impact of interventions to treat gender dypshoria.50 –53

CONCLUSIONS The most important finding is the high frequency of mental health conditions that TGNC children and adolescents experience. Especially worrisome are the results for suicidal ideation and self-inflicted injuries with prevalence estimates orders of a magnitude that is higher in TGNC children and adolescents than in matched cisgender reference groups. For nearly all mental health disorders, the PRs increased during the time window closest to the index date. Overall, these data reveal that children and adolescents presenting as TGNC to health care providers may require not only thorough and immediate evaluation

of mental health needs but also urgent implementation of social and educational measures of gender identity support.

PEDIATRICS Volume 141, number 5, May 2018 9

within their areas of expertise (such as epidemiologic methods, bias, health care access and health service use interpretation, and the broad messaging of the manuscript), and revised the manuscript; Drs Liu, Flanders, and Nash provided substantial statistical analysis consultation, conducted the analyses, and critically reviewed and revised the manuscript for important statistical interpretation of the data; Ms Cromwell substantially contributed to the design of multisite data collection, critically reviewed the manuscript for appropriate interpretation of the data variables with respect to the results, and revised the manuscript; Ms Millman and Ms Robinson conceptualized the study and substantially contributed to the acquisition of data by coordinating site data collection, critically reviewed and revised the manuscript by providing and ensuring the interpretation of results with respect to site-specific patient populations, and revised the manuscript; Drs Giammattei, Sandberg, and Tangpricha provided clinical consultation regarding the interpretation of results, revised the manuscript, and critically reviewed the manuscript for important intellectual content specific to transgender and/or gender nonconforming youth, gender transitioning, and the mental health outcomes discussed in the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

DOI: https:// doi. org/ 10. 1542/ peds. 2017- 3845

Accepted for publication Feb 22, 2018

Address correspondence to Michael Goodman, MD, MPH, Department of Epidemiology, Emory University School of Public Health, 1518 Clifton Rd, NE, CNR 3021, Atlanta, GA 30322. E-mail: [email protected]

PEDIATRICS (ISSN Numbers: Print, 0031-4005; Online, 1098-4275).

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

FUNDING: Funded by contract AD-12-11-4532 from the Patient-Centered Outcomes Research Institute and grant R21HD076387 from the Eunice Kennedy Shriver National Institute of Child Health and Human Development. Funded by the National Institutes of Health (NIH).

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

COMPANION PAPER: Companions to this article can be found online at www. pediatrics. org/ cgi/ doi/ 10. 1542/ peds. 2017- 3004 and www. pediatrics. org/ cgi/ doi/ 10. 1542/ peds. 2018- 0361.

ABBREVIATIONS

CI: confidence interval EMR: electronic medical record ICD-9: International Classification

of Diseases, Ninth Edition KP: Kaiser Permanente PR: prevalence ratio STRONG: Study of Transition,

Outcomes, and Gender TGNC: transgender and/or

gender nonconforming

REFERENCES

1. Lombardi E. Enhancing transgender health care. Am J Public Health. 2001;91(6):869–872

2. Wallien MS, Cohen-Kettenis PT. Psychosexual outcome of gender-dysphoric children. J Am Acad Child Adolesc Psychiatry. 2008;47(12):1413–1423

3. Steensma TD, van der Ende J, Verhulst FC, Cohen-Kettenis PT.

Gender variance in childhood and sexual orientation in adulthood: a prospective study. J Sex Med. 2013;10(11):2723–2733

4. Leibowitz SF, Spack NP. The development of a gender identity psychosocial clinic: treatment issues, logistical considerations, interdisciplinary cooperation, and future initiatives. Child

Adolesc Psychiatr Clin N Am. 2011;20(4):701–724

5. Chen D, Hidalgo MA, Leibowitz S, et al. Multidisciplinary care for gender-diverse youth: a narrative review and unique model of gender- affirming care. Transgend Health. 2016;1(1):117–123

6. Costa R, Carmichael P, Colizzi M. To treat or not to treat: puberty

by guest on January 12, 2021www.aappublications.org/newsDownloaded from

https://doi.org/10.1542/peds.2017-3845

mailto:

http://www.pediatrics.org/cgi/doi/10.1542/peds.2017-3004

http://www.pediatrics.org/cgi/doi/10.1542/peds.2018-0361

suppression in childhood-onset gender dysphoria. Nat Rev Urol. 2016;13(8):456–462

7. Fast AA, Olson KR. Gender development in transgender preschool children [published online ahead of print April 25, 2017]. Child Dev. doi: 10. 1111/ cdev. 12758

8. Golombok S, Rust J, Zervoulis K, Croudace T, Golding J, Hines M. Developmental trajectories of sex- typed behavior in boys and girls: a longitudinal general population study of children aged 2.5-8 years. Child Dev. 2008;79(5):1583–1593

9. Martin CL, Ruble DN. Patterns of gender development. Annu Rev Psychol. 2010;61:353–381

10. Rosenthal SM. Transgender youth: current concepts. Ann Pediatr Endocrinol Metab. 2016;21(4):185–192

11. Coleman E, Bockting WO, Botzer M, et al. Standards of care for the health of transsexual, transgender, and gender- nonconforming people, version 7. Int J Transgenderism. 2012;13(4):165–232

12. Wilczynski C, Emanuele MA. Treating a transgender patient: overview of the guidelines. Postgrad Med. 2014;126(7):121–128

13. Levine DA; Committee on Adolescence. Office-based care for lesbian, gay, bisexual, transgender, and questioning youth. Pediatrics. 2013;132(1). Available at: www. pediatrics. org/ cgi/ content/ full/ 132/ 1/ e297

14. Schneider C, Cerwenka S, Nieder TO, et al. Measuring gender dysphoria: a multicenter examination and comparison of the Utrecht Gender Dysphoria Scale and the Gender Identity/ Gender Dysphoria Questionnaire for adolescents and adults. Arch Sex Behav. 2016;45(3):551–558

15. Grossman AH, D’Augelli AR. Transgender youth and life-threatening behaviors. Suicide Life Threat Behav. 2007;37(5):527–537

16. Cohen-Kettenis PT, Steensma TD, de Vries AL. Treatment of adolescents with gender dysphoria in the Netherlands. Child Adolesc Psychiatr Clin N Am. 2011;20(4):689–700

17. de Vries AL, Doreleijers TA, Steensma TD, Cohen-Kettenis PT. Psychiatric comorbidity in gender dysphoric adolescents. J Child Psychol Psychiatry. 2011;52(11):1195–1202

18. Nahata L, Quinn GP, Caltabellotta NM, Tishelman AC. Mental health concerns and insurance denials among transgender adolescents. LGBT Health. 2017;4(3):188–193

19. Olson J, Schrager SM, Belzer M, Simons LK, Clark LF. Baseline physiologic and psychosocial characteristics of transgender youth seeking care for gender dysphoria. J Adolesc Health. 2015;57(4):374–380

20. Reisner SL, Biello KB, White Hughto JM, et al. Psychiatric diagnoses and comorbidities in a diverse, multicity cohort of young transgender women: baseline findings from project LifeSkills. JAMA Pediatr. 2016;170(5):481–486

21. Reisner SL, Vetters R, Leclerc M, et al. Mental health of transgender youth in care at an adolescent urban community health center: a matched retrospective cohort study. J Adolesc Health. 2015;56(3):274–279

22. Spack NP, Edwards-Leeper L, Feldman HA, et al. Children and adolescents with gender identity disorder referred to a pediatric medical center. Pediatrics. 2012;129(3):418–425

23. de Vries AL, Kreukels BP, Steensma TD, Doreleijers TA, Cohen-Kettenis PT. Comparing adult and adolescent transsexuals: an MMPI-2 and MMPI-A study. Psychiatry Res. 2011;186(2–3):414–418

24. Olson-Kennedy J, Cohen-Kettenis PT, Kreukels BP, et al. Research priorities for gender nonconforming/ transgender youth: gender identity development and biopsychosocial outcomes. Curr Opin Endocrinol Diabetes Obes. 2016;23(2):172–179

25. Dekker MJ, Wierckx K, Van Caenegem E, et al. A European network for the investigation of gender incongruence: endocrine part. J Sex Med. 2016;13(6):994–999

26. Kreukels BP, Haraldsen IR, De Cuypere G, Richter-Appelt H, Gijs L, Cohen- Kettenis PT. A European network for the investigation of gender incongruence: the ENIGI initiative. Eur Psychiatry. 2012;27(6):445–450

27. Reisner SL, Deutsch MB, Bhasin S, et al. Advancing methods for US transgender health research. Curr Opin Endocrinol Diabetes Obes. 2016;23(2):198–207

28. Koebnick C, Langer-Gould AM, Gould MK, et al. Sociodemographic characteristics of members of a large, integrated health care system: comparison with US Census Bureau data. Perm J. 2012;16(3):37–41

29. Gordon NP. How Does the Adult Kaiser Permanente Membership in Northern California Compare With the Larger Community? Oakland, CA: Kaiser Permanente Division of Research; 2006

30. American Academy of Pediatrics. Periodicity schedule. 2017. Available at: https:// www. aap. org/ en- us/ Documents/ periodicity_ schedule. pdf. Accessed July 12, 2017

31. Roblin D, Barzilay J, Tolsma D, et al. A novel method for estimating transgender status using electronic medical records. Ann Epidemiol. 2016;26(3):198–203

32. Quinn VP, Nash R, Hunkeler E, et al. Cohort profile: Study of Transition, Outcomes and Gender (STRONG) to assess health status of transgender people. BMJ Open. 2017;7(12):e018121

33. Lee PA, Houk CP, Ahmed SF, Hughes IA; International Consensus Conference on Intersex Organized by the Lawson Wilkins Pediatric Endocrine Society and the European Society for Paediatric Endocrinology. Consensus statement on management of intersex disorders. International Consensus Conference on Intersex. Pediatrics. 2006;118(2). Available at: www. pediatrics. org/ cgi/ content/ full/ 118/ 2/ e488

34. Coleman KJ, Stewart C, Waitzfelder BE, et al. Racial-ethnic differences in psychiatric diagnoses and treatment across 11 health care systems in the mental health research network. Psychiatr Serv. 2016;67(7):749–757

35. World Health Organization. Recognizing adolescence. 2014. Available at: http:// apps. who. int/ adolescent/ second- decade/ section2/ page1/ recognizing- adolescence. html. Accessed November 7, 2017

36. Nickleach D, Liu Y, Shrewsberry A, Ogan K, Kim S, Wang Z. SAS macros to conduct common biostatistical analyses and generate reports. In: SouthEast SAS User Group; October 20-23, 2013; St. Pete Beach, FL

37. Arcelus J, Claes L, Witcomb GL, Marshall E, Bouman WP. Risk factors for non-suicidal self-injury

BECERRA-CULQUI et al10 by guest on January 12, 2021www.aappublications.org/newsDownloaded from

www.pediatrics.org/cgi/content/full/132/1/e297

https://www.aap.org/en-us/Documents/periodicity_schedule.pdf

www.pediatrics.org/cgi/content/full/118/2/e488

http://apps.who.int/adolescent/second-decade/section2/page1/recognizing-adolescence.html

PEDIATRICS Volume 141, number 5, May 2018 11

among trans youth. J Sex Med. 2016;13(3):402–412

38. Kaltiala-Heino R, Sumia M, Työläjärvi M, Lindberg N. Two years of gender identity service for minors: overrepresentation of natal girls with severe problems in adolescent development. Child Adolesc Psychiatry Ment Health. 2015;9:9

39. Holt V, Skagerberg E, Dunsford M. Young people with features of gender dysphoria: demographics and associated difficulties. Clin Child Psychol Psychiatry. 2016;21(1):108–118

40. Shields JP, Cohen R, Glassman JR, Whitaker K, Franks H, Bertolini I. Estimating population size and demographic characteristics of lesbian, gay, bisexual, and transgender youth in middle school. J Adolesc Health. 2013;52(2):248–250

41. Diemer EW, Grant JD, Munn-Chernoff MA, Patterson DA, Duncan AE. Gender identity, sexual orientation, and eating- related pathology in a national sample of college students. J Adolesc Health. 2015;57(2):144–149

42. Chen M, Fuqua J, Eugster EA. Characteristics of referrals for gender dysphoria over a 13-year period. J Adolesc Health. 2016;58(3):369–371

43. Glidden D, Bouman WP, Jones BA, Arcelus J. Gender dysphoria and autism spectrum disorder: a systematic review of the literature. Sex Med Rev. 2016;4(1):3–14

44. Jacobs LA, Rachlin K, Erickson-Schroth L, Janssen A. Gender dysphoria and co-occurring autism spectrum disorders: review, case examples, and treatment considerations. LGBT Health. 2014;1(4):277–282

45. Van Der Miesen AI, Hurley H, De Vries AL. Gender dysphoria and autism spectrum disorder: a narrative review. Int Rev Psychiatry. 2016;28(1):70–80

46. de Vries AL, Noens IL, Cohen- Kettenis PT, van Berckelaer-Onnes IA, Doreleijers TA. Autism spectrum disorders in gender dysphoric children and adolescents. J Autism Dev Disord. 2010;40(8):930–936

47. Wing L, Leekam SR, Libby SJ, Gould J, Larcombe M. The diagnostic interview for social and communication disorders: background, inter-rater reliability and clinical use. J Child Psychol Psychiatry. 2002;43(3): 307–325

48. Aitken M, Steensma TD, Blanchard R, et al. Evidence for an altered sex

ratio in clinic-referred adolescents with gender dysphoria. J Sex Med. 2015;12(3):756–763

49. Olson KR, Durwood L, DeMeules M, McLaughlin KA. Mental health of transgender children who are supported in their identities. Pediatrics. 2016;137(3): e20153223

50. Hembree WC. Guidelines for pubertal suspension and gender reassignment for transgender adolescents. Child Adolesc Psychiatr Clin N Am. 2011;20(4):725–732

51. Milrod C. How young is too young: ethical concerns in genital surgery of the transgender MTF adolescent. J Sex Med. 2014;11(2):338–346

52. Milrod C, Karasic DH. Age is just a number: WPATH-affiliated surgeons’ experiences and attitudes toward vaginoplasty in transgender females under 18 years of age in the United States. J Sex Med. 2017;14(4):624–634

53. Shumer DE, Spack NP. Current management of gender identity disorder in childhood and adolescence: guidelines, barriers and areas of controversy. Curr Opin Endocrinol Diabetes Obes. 2013;20(1):69–73

by guest on January 12, 2021www.aappublications.org/newsDownloaded from

DOI: 10.1542/peds.2017-3845 originally published online April 16, 2018; 2018;141;Pediatrics

Michael J. Silverberg, Vin Tangpricha and Michael Goodman Millman, Virginia P. Quinn, Brandi Robinson, Douglas Roblin, David E. Sandberg,

Darios Getahun, Shawn V. Giammattei, Enid M. Hunkeler, Timothy L. Lash, Andrea Tracy A. Becerra-Culqui, Yuan Liu, Rebecca Nash, Lee Cromwell, W. Dana Flanders,

With Their Peers Mental Health of Transgender and Gender Nonconforming Youth Compared

Services Updated Information &

http://pediatrics.aappublications.org/content/141/5/e20173845 including high resolution figures, can be found at:

References http://pediatrics.aappublications.org/content/141/5/e20173845#BIBL This article cites 49 articles, 4 of which you can access for free at:

Subspecialty Collections

http://www.aappublications.org/cgi/collection/public_health_sub Public Health y_sub http://www.aappublications.org/cgi/collection/psychiatry_psycholog Psychiatry/Psychology following collection(s): This article, along with others on similar topics, appears in the

Permissions & Licensing

http://www.aappublications.org/site/misc/Permissions.xhtml in its entirety can be found online at: Information about reproducing this article in parts (figures, tables) or

Reprints http://www.aappublications.org/site/misc/reprints.xhtml Information about ordering reprints can be found online:

by guest on January 12, 2021www.aappublications.org/newsDownloaded from

http://http://pediatrics.aappublications.org/content/141/5/e20173845

http://pediatrics.aappublications.org/content/141/5/e20173845#BIBL

http://www.aappublications.org/cgi/collection/psychiatry_psychology_sub

http://www.aappublications.org/cgi/collection/public_health_sub

http://www.aappublications.org/site/misc/Permissions.xhtml

http://www.aappublications.org/site/misc/reprints.xhtml

DOI: 10.1542/peds.2017-3845 originally published online April 16, 2018; 2018;141;Pediatrics

Michael J. Silverberg, Vin Tangpricha and Michael Goodman Millman, Virginia P. Quinn, Brandi Robinson, Douglas Roblin, David E. Sandberg,

Darios Getahun, Shawn V. Giammattei, Enid M. Hunkeler, Timothy L. Lash, Andrea Tracy A. Becerra-Culqui, Yuan Liu, Rebecca Nash, Lee Cromwell, W. Dana Flanders,

With Their Peers Mental Health of Transgender and Gender Nonconforming Youth Compared

http://pediatrics.aappublications.org/content/141/5/e20173845 located on the World Wide Web at:

The online version of this article, along with updated information and services, is

http://pediatrics.aappublications.org/content/suppl/2018/04/12/peds.2017-3845.DCSupplemental Data Supplement at:

by the American Academy of Pediatrics. All rights reserved. Print ISSN: 1073-0397. the American Academy of Pediatrics, 345 Park Avenue, Itasca, Illinois, 60143. Copyright © 2018 has been published continuously since 1948. Pediatrics is owned, published, and trademarked by Pediatrics is the official journal of the American Academy of Pediatrics. A monthly publication, it

by guest on January 12, 2021www.aappublications.org/newsDownloaded from

http://pediatrics.aappublications.org/content/141/5/e20173845

http://pediatrics.aappublications.org/content/suppl/2018/04/12/peds.2017-3845.DCSupplemental