Psychology assignment 2

Development of Gender Identity Implicit Association Tests to Assess Attitudes Toward Transmen and Transwomen

Tiffani “Tie” S. Wang-Jones California School of Professional Psychology, Alliant

International University

Omar M. Alhassoon California School of Professional Psychology and University of

California, San Diego

Kate Hattrup San Diego State University, San Diego

Bernardo M. Ferdman and Rodney L. Lowman California School of Professional Psychology, Alliant

International University

The purpose of this study was to develop and validate 2 gender identity implicit association tests (GI-IATs) designed to assess attitudes toward transsexual men (Transmen-IAT) and transsexual women (Transwomen-IAT). A sample of 344 Mechanical Turk participants from the United States (173 women, 129 men, 43 transgender) completed the following: GI-IATs, Genderism and Transphobia Scale, Allophilia Toward Transsexual Individuals Scale, Social Desirability Scale-17, feelings thermometers, and ratings of intention to support transgender workplace policies. Results indicate that people who are cisgender (non-transgender), heterosexual, politically conservative, or who reported no personal contact with transgender individuals showed cisgender preferences on both GI-IATs. Additionally, both mea- sures correlated as predicted with the explicit measures (feeling thermometers) of attitude toward transgender individuals. As expected, the explicit attitude measures, but not the GI-IATs, correlated with social desirability. Further, confirmatory factor analyses supported the model comprising 4 distinct latent variables: implicit attitudes toward transmen, explicit attitudes toward transmen, implicit attitudes toward transwomen, and explicit attitudes toward transwomen. Finally, hierarchical multiple regressions showed that both explicit and implicit measures predicted support for transgender workplace policies. Additional analyses showed that both the Transmen-IAT and the Transwomen-IAT accounted for incremental variance above and beyond the relative feelings thermometers in predicting policy support intentions. These findings provide significant psychometric support for both GI-IATs. They also highlight the importance of incorporating implicit measures in studying attitudes toward transgender individuals, and of distinguishing attitudes toward transmen versus transwomen.

Public Significance Statement This study created and validated the first implicit tests of attitude toward transsexual men and transsexual women. Current measures are limited because they depend exclusively on self-report methods and treat the transgender community as one homogenous group. Our tests examine attitudes toward transsexual men and women separately using a method that is less affected by people’s own judgement of how they feel and are more likely to pick up unconscious bias.

Keywords: attitudes toward transsexual men, attitudes toward transsexual women, implicit bias, implicit measure, test development

Supplemental materials: http://dx.doi.org/10.1037/sgd0000218.supp

This article was published Online First January 12, 2017. Tiffani “Tie” S. Wang-Jones, Dual Clinical Psychology/Industrial-

Organizational Psychology PhD Program, California School of Professional Psychology, Alliant International University; Omar M. Alhassoon, Clinical Psychology PhD Program, California School of Professional Psychology and Department of Psychiatry, University of California, San Diego; Kate Hattrup, Department of Psychology, San Diego State University, San Diego; Bernardo

M. Ferdman and Rodney L. Lowman, Industrial/Organizational Psychology PhD Program, California School of Professional Psychology, Alliant Interna- tional University.

Correspondence concerning this article should be addressed to Omar M. Alhassoon, California School of Professional Psychology, Associate Project Scientist, UCSD, Daley Hall 112C, 10455 Pomerado Road, San Diego, CA 92131. E-mail: [email protected]

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

Psychology of Sexual Orientation and Gender Diversity © 2017 American Psychological Association 2017, Vol. 4, No. 2, 169 –183 2329-0382/17/$12.00 http://dx.doi.org/10.1037/sgd0000218

169

Transgender1 issues are becoming more visible in the United States through mainstream media and politics. Reality TV series like I Am Cait and I Am Jazz, as well as recent controversies regarding state laws about restroom access, have captured the public’s attention. A national survey by the Public Religion Re- search Institute showed that 9 out of 10 Americans supported equal rights for transgender people (Cox & Jones, 2011). Even religious and political conservatives were largely supportive of transgender equality (Cox & Jones, 2011). Despite positively espoused atti- tudes, people’s support for specific protections against transgender discrimination is divided (Newport, 2016). Furthermore, the over- whelming support for transgender rights belies the fact that trans- gender Americans report significant discrimination (Grant et al., 2011). These concerns highlight the need to improve on current transphobia measures, which are limited because they depend exclusively on self-report and treat the transgender community as one homogenous group. Implicit cognition research shows that self-report is a poor method for uncovering attitudes that people may not be fully aware of or may not be willing to disclose (Gawronski & Payne, 2010; Greenwald & Banaji, 1995). There- fore, it is possible that current reliance on surveys to measure transphobia does not yield a complete picture of the underlying biases. In addition, data indicate that certain transgender subgroups face more bias than others. According to crime data, male-to- female transsexual2 individuals (transwomen) face the most severe consequences of bias-motivated actions such as assault and homi- cide (Human Rights Campaign, 2015; Human Rights Council, 2011; Schilt & Westbrook, 2009). Considering these devastating consequences of discrimination, and the limitations in current transphobia measures, this study was aimed at developing implicit measures to indirectly assess attitudes toward transwomen and transmen independently to allow necessary comparisons so that these phenomena can be better understood and managed.

Implicit and Explicit Definitions

The terms implicit and explicit have been used in bias research to describe both constructs and measures (Fazio & Olson, 2003; Greenwald & Banaji, 1995). Implicit, when used to describe a measure, pertains to the procedure of obtaining data on something indirectly; that is without having overtly asked the participants about the focal topic (Fazio & Olson, 2003). Conversely, explicit measures gather information by directly querying participants on the topic of interest. When these terms are used in reference to a construct (e.g., implicit cognition), they describe whether or not the mental process occurs automatically (Greenwald & Banaji, 1995). Some researchers (e.g., Greenwald & Banaji, 1995) argue that, because implicit and explicit cognitions differ from one another, measurement methods also need to differ to capture these concepts appropriately.

Implicit Association Test

The Implicit Association Test (IAT) is the most widely re- searched implicit procedure used to assess automatic preferences based on the concept of reaction time (RT) differentials (Gawron- ski & Payne, 2010). Developed by Greenwald, McGhee, and Schwartz (1998), the IAT is considered a relative measure of attitudes because the respondent is evaluating their preferences

between two target groups. For an intergroup IAT, the procedure compares preferences for one target group versus another based on people’s response-time latency in associating positively versus negatively valenced stimuli with each of the two target groups. The IAT yields consistent patterns of results that point to attitudinal preferences favoring the privileged social group on identity dimen- sions such as race (Amodio & Devine, 2006), disability (Pruett & Chan, 2006), sexual orientation (Breen & Karpinski, 2013), weight (Agerström & Rooth, 2011), and gender (Nier & Gaertner, 2012). Research has also shown that the IAT procedure can predict discriminatory behavior. Greenwald et al. (2009) conducted a meta-analysis of 122 studies that used the IAT with 184 indepen- dent samples across various domains such as race, gender, sexual orientation, consumer behaviors, and political preferences. Results showed that RTs on the IAT predicted behaviors such as social interactions, medical decisions, and voting preferences. Employ- ment studies that used IATs showed that scores on these measures predicted hiring outcomes for Middle Eastern (Rooth, 2010) and obese applicants (Agerström & Rooth, 2011). Therefore, a large and growing body of research has demonstrated the potential usefulness of the IAT for measuring implicit attitudes.

Currently, there are no implicit measures of attitudes toward transgender individuals. However, there are IAT studies focused on sexual orientation that have found attitudinal preferences within the heterosexual population for their own group (Banse, Seise, & Zerbes, 2001; Jellison, McConnell, & Gabriel, 2004; Steffens, 2005). In fact, Steffens (2005) showed that heterosexual people tend to report positive attitudes toward gay men and lesbians, but nonetheless show some automatic preferences for straight individ- uals. Furthermore, Jellison et al. (2004) found that the sexuality IAT predicted the degree to which gay men were active in the lesbian, gay, bisexual, transgender, plus people of all other sexual orientations and gender identities (LGBT�) community, whereas the explicit measures were more predictive of self-disclosure of one’s sexual orientation. The differences in the types of behaviors predicted by explicit and implicit measures, and the divergence between these measurement scores, suggests that explicit and implicit measures capture related yet disparate variance in these attitudes. Not surprisingly, homophobia and transphobia are re- lated (Hill & Willoughby, 2005; Nagoshi et al., 2008; Walch, Ngamake, Francisco, Stitt, & Shingler, 2012); hence, research that has used IATs to assess homophobia provides insight to inform the development of implicit measures to assess attitudes toward trans- men and transwomen.

1 The American Psychological Association (2011) defines transgender as “an umbrella term for persons whose gender identity, gender expression or behavior does not conform to what is typically associated with the sex to which they were assigned at birth” (p. 1). Sex refers to people’s physiology at birth in terms of genes and anatomy. Gender identity is one’s inner sense of self as woman, man, or transgender. Gender expression refers to the ways in which people choose to communicate their gender identity through dress, hairstyle, mannerisms, and other means of self- expression.

2 Transsexual individuals, “often referred to as either male-to-female (MtF) or female-to-male (FtM), are biological men or biological women, respectively, who seek hormonal, surgical, and/or other procedures to make their bodies conform to their desired gender” (Gerhardstein & Anderson, 2010, p. 361). The terms transwomen and transmen are sometimes used as synonyms for MtF and FtM transsexuals, respectively.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

170 WANG-JONES ET AL.

Research Aims and Hypotheses

The primary aim of this study was to create two gender identity IATs (GI-IATs): one for assessing relative preferences toward transsexual men versus biological men (Transmen-IAT), and an- other for assessing relative preferences toward transsexual women versus biological women (Transwomen-IAT). Evaluation of the instruments’ reliability and validity was based on internal consis- tency, stability, and various types of validity (known-groups, con- vergent, discriminant, and predictive).

Known-Groups Validity

Known-groups validity is a method of assessing initial construct validity of measurements by evaluating the basic assumption that a test will capture differences between groups that should logically or empirically differ on the construct of interest (Cronbach & Meehl, 1955). First conceptualized by Cronbach and Meehl (1955), known-groups validity has been widely used in scale development (Mackenzie, Podsakoff, & Podsakoff, 2011; Rubin & Babbie, 2009a, 2009b), and has been deemed critical evidence for evaluating meaningful differences among groups to augment other methods of assessing measurement validity (Hattie & Cooksey, 1984). When used for IAT development, this common approach tests whether IAT scores diverge between two groups known to differ in their bias against the target (Gawronski & Payne, 2010). For example, results from race and ethnicity IATs show expected known-group differences between Blacks, Whites, and Latinos (Blair, Judd, Havranek, & Steiner, 2010), Japanese and Korean Americans (Greenwald et al., 1998), and East and West Germans (Kuhnen et al., 2001). Because group divergence in test scores does not by itself provide sufficient evidence of measurement validity, the current study did not rely exclusively on the known- groups method, but incorporated other lines of evidence to support inferences about the GI-IATs.

The basic assumption that the GI-IATs can differentiate be- tween groups that are expected to differ on attitudes toward trans- men and transwomen was tested in several ways. First, because these measures aim to assess preferences about gender identity, and research points to the prevalence of transphobia within the American cisgender (non-transgender) population (Flores, 2015; Grant et al., 2011; Norton & Herek, 2013), it was expected that cisgender individuals will show greater preference for biological versus transsexual targets than will transgender individuals.

Hypothesis 1a: Cisgender individuals will show greater cis- gender preference on both GI-IATs compared with transgen- der individuals.

Studies of homophobia provide evidence that heterosexual men, compared with heterosexual women, show greater negativity to- ward sexual minorities on implicit and explicit measures of bias (Banse et al., 2001; Hill & Willoughby, 2005; Nagoshi et al., 2008; Steffens, 2005). Heterosexual men have also shown greater bias than heterosexual women toward transgender persons in studies using explicit measures of transphobia (Cragun & Sumerau, 2015; Warriner, Nagoshi, & Nagoshi, 2013; Woodford, Silverschanz, Swank, Scherrer, & Raiz, 2012). Thus, it was expected that cis- gender heterosexual men and women will differ in their responses to implicit measures of bias toward transgender persons.

Hypothesis 1b: Cisgender heterosexual men will show greater cisgender preference on both GI-IATs than cisgender hetero- sexual women.

Other demographic factors empirically related to bias toward transgender individuals are sexual orientation (Case & Stewart, 2013; Cragun & Sumerau, 2015; Warriner et al., 2013), personal contact with transgender people (King, Winter, & Webster, 2009; Walch et al., 2012), political conservatism (Warriner et al., 2013; Woodford et al., 2012), and degree of religiosity (Cragun & Sumerau, 2015; Woodford et al., 2012). These variables were also included to test known-groups validity.

Hypothesis 1c: Heterosexual individuals will show greater cisgender preference on both GI-IATs compared with non- heterosexual individuals.

Hypothesis 1d: People without any personal contact with transgender individuals will show greater cisgender prefer- ence on both GI-IATs than those who personally know at least one transgender person.

Hypothesis 1e: Degree of political conservatism and religios- ity will be related to both GI-IATs.

Convergent Validity

Meta-analyses of IAT studies show consistent relationships be- tween parallel implicit and explicit bias measures (Hofmann, Gawronski, Gschwendner, Le, & Schmitt, 2005; Nosek, 2005). Nosek (2005) examined 57 Internet IAT studies in various attitude domains, and reported a correlation between implicit and explicit measures of r � .36 (Nosek, 2005). Hofmann et al. (2005) exam- ined 126 IAT studies on a range of topics, such as intergroup attitudes, consumer preferences, and clinical applications, and observed that correlations between the IAT and its associated explicit measures for intergroup attitudes (r � .25) were somewhat lower than those found for consumer attitudes (r � .34), but were still consistent across studies. Thus, evidence from the literature suggests that implicit and explicit measures are weakly to moder- ately related to one another on issues pertaining to intergroup attitudes. Therefore, similar results were expected in the present study; we predicted weak to moderate correlations between ex- plicit measures toward transgender persons and the GI-IATs.

Hypothesis 2a: Both GI-IATs will show weak to moderate positive correlations (r � .10 to .30) with an explicit measure of negative attitudes toward transgender people.

Hypothesis 2b: Both GI-IATs will show weak to moderate negative correlations (r � �.10 to �.30) with an explicit measure of positive attitudes toward transgender people.

Discriminant Validity

Evidence from explicit measures has also been used to support discriminant validity of IATs. Nosek and Smyth (2007) used multitrait-multimethod procedures to show strong evidence of convergent and discriminant validity of the IAT across seven target attitude pairs (e.g., straight/gay, White/Black, or flower/ insect). Convergent validity was supported by findings of signif-

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

171DEVELOPMENT OF GENDER IDENTITY IATS

icant correlations between explicit and implicit attitudes for five of the seven target pairs. To demonstrate discriminant validity, con- firmatory factory analysis (CFA) was used; results indicated that a dual-construct model with distinct but related explicit and implicit factors provided the best model fit. Thus, Nosek and Smyth’s study supported the idea that explicit and implicit attitudes were distinct but related constructs. Similar findings were expected in the present study, with scores on explicit and implicit measures of bias against transmen and transwomen loading on four different but related latent attitude factors.

Hypothesis 3a: CFA will support a four-factor model with factors representing implicit attitude towards transmen, im- plicit attitude towards transwomen, explicit attitude towards transmen, and explicit attitude towards transwomen.

Research has also shown that implicit measures are less affected by social desirability than are explicit measures (Gawronski & Payne, 2010; Sherman et al., 2014). This is especially true in studies of the sensitive social topic of intergroup bias (Greenwald et al., 2009; Nosek et al., 2007). Because implicit measures are expected to be less affected by conscious self-presentation, some researchers have used social desirability to assess the construct validity of their IATs, and have observed support for the hypoth- esized results (LaBouff, Rowatt, Johnson, Thedford, & Tsang, 2010; Pruett & Chan, 2006). This study assessed the relationship of the GI-IATs with a measure of social desirability to gather additional evidence of discriminant validity.

Hypothesis 3b: Explicit measures, but not implicit measures of attitudes toward transgender individuals, will be related to social desirability.

Predictive Validity

Predictive validity of the GI-IATs was assessed by correlating them with ratings of behavioral intentions to support transgender human resource policies. Studies have shown that IATs are pre- dictive of policy attitudes, voting behavior, and political judgments (Hanson, 2012; Pérez, 2010; Roccato & Zogmaister, 2010). For example, Pérez (2010) found that a race IAT predicted people’s attitudes about illegal immigration policies affecting Latinos. Wang-Jones, Allen, Budzyn, and Ferdman (2013) found that feel- ings of threat predicted people’s level of support for transgender workplace policies. However, overall ratings of threat were low, whereas ratings of support were high, so people generally reported positive explicit attitudes toward transgender workplace policies. The fact that threat was a predictor of policy support even though people explicitly denied feeling threat suggests that people may not be aware of, or may be unwilling to disclose, their negative explicit attitudes. Considering these findings, both implicit and explicit measures of attitudes toward transgender people were expected to be significant predictors of support for transgender workplace policies.

Hypothesis 4: Both implicit (GI-IATs) and explicit measures (feelings thermometers) of attitude toward transgender indi- viduals will be significant predictors of intention to support transgender workplace policies.

Method

Participants

After Institutional Review Board approval, data from two sam- ples, a pilot sample (N � 113) and a validation sample (N � 344), were collected using Mechanical Turk (MTurk). The pilot sample consisted of 72 women and 41 men living in the United States who were between the ages of 18 and 65 (see Table 1 for demographic data). Participants in the validation sample were also adults living in the United States who were between the ages of 18 and 65. Targeted sampling was used to obtain transgender participants by administering a three-item self-report screener to determine sex, gender, and sexual orientation. Transgender-identified individuals were invited to participate in the validation study through MTurk Prime. The initial recruitment sample consisted of 459 partici- pants. Of this number, 17 individuals dropped out, 68 had tech- nological issues that prohibited access to the GI-IATs, 8 cases were deleted because of duplicate data obtained from the same IP address, and 22 were removed for having greater than 25% error on either of the GI-IATs (Rudman, 2011). The final validation sample included 344 participants with a mean age of 34.31 (SD � 10.63) and 14.82 (SD � 2.08) mean years of education (see Table 2).

The use of MTurk and other online sampling services has flourished recently because of their efficiency for recruiting di- verse samples that tend to be more representative than those obtained with typical sampling methods (Berinsky, Huber, & Lenz, 2012; Buhrmester, Kwang, & Gosling, 2011; Gosling, Vazire, Srivastava, & John, 2004). Prior research evidence pro- vides ample support for the reliability and validity of data collected through MTurk sampling (Buhrmester et al., 2011; Crump, Mc- Donnell, & Gureckis, 2013; Steelman, Hammer, & Limayem, 2014), especially for the U.S. population (Steelman et al., 2014).

Measures

Transmen-IAT and Transwomen-IAT. The two GI-IATs were coded and administered online through Inquisit Millisecond (2015). The Transmen-IAT was designed to assess automatic preferences for transsexual men versus biological men, and the Transwomen-IAT was designed to assess automatic preferences for transsexual women versus biological women. The block se- quence schematic for both GI-IATs is depicted in Table 3, and the specific stimuli used in each test are shown in Table 4. Scoring of the GI-IATs followed the improved algorithm recommended by Greenwald, Nosek, and Banaji, (2003). The built-in penalty for

Table 1 Pilot Study Participant Demographics

Demographics n

Total 118 Female birth sex 76 Male birth sex 42 Asian/Pacific Islander 6 Black/Caribbean/African-American 10 Hispanic 5 White/Caucasian 93 Multi-racial 4

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

172 WANG-JONES ET AL.

extreme scores eliminated all trials with �10,000 ms latency. Data were used from test and practice tasks of the four dual- categorization blocks to calculate the IAT scores. The resulting D score is based on latency differences obtained by subtracting the compatible blocks (bias-congruent responses) from the incompat- ible blocks (bias-incongruent responses). The directionality of the D scores was coded such that larger positive scores indicate greater relative preference toward cisgender over transgender targets, and larger negative scores suggest greater preference for transgender over cisgender targets. Reliability and validity evidence of these measures is presented in more detail in the results section.

Genderism and Transphobia Scale (GTS). The GTS (Hill & Willoughby, 2005) is a 32-item, 7-point Likert scale measuring self-reported attitudes and behaviors suggestive of transphobia. Cronbach’s � of the entire scale ranged from .79 to .96 in previous studies (Hill & Willoughby, 2005; Tebbe, Moradi, & Ege, 2014). In the present sample of cisgender participants, Cronbach’s � of the GTS was .96 for the entire scale.

Allophilia Toward Transsexual Individuals Scale (AlloTrans). The Allophilia Scale (Pittinsky, Rosenthal, & Montoya, 2010), a 7-point Likert-like scale of 17-items, was adapted by the current authors to create the AlloTrans scale with higher scores indicating positive attitudes toward transsexual peo-

ple. The original Allophilia Scale was constructed as a measure meant to be easily adapted for assessing attitudes toward different groups by simply inserting the name of the group as the subject. An adaptation of this scale that assessed attitudes toward lesbian, gay, and bisexual people showed a Cronbach’s � of .97 for the composite score (Fingerhut, 2011). Because the Allophilia Scale had not been adapted for the transgender target group before, a separate MTurk sample of 138 participants (90 women, 65 men) was used to determine its overall internal consistency (� � .99). Evidence of validity suggest that the AlloTrans is related to polit- ical conservatism (r � �.58), religiosity (r � �.44), and support for transgender workplace policies (r � .86). For the current study, Cronbach’s � for the AlloTrans was .98, and the 1-week test–retest reliability coefficient was .94.

Transmen policy support and transwomen policy support. Eight human resource policies were adapted from Wang-Jones et al. (2013) to separately assess support for employment rights and benefits for transsexual men and transsexual women. Participants were asked to rate their level of support for each policy on a 6-point Likert scale. Policy topics include general nondiscrimina- tion clauses, health insurance, restroom access, and dress codes. Items were reverse-scored such that larger numbers indicated less support, to parallel the directionality of the GI-IAT scores. Cron- bach’s � for Transmen Policy Support was .94 and Transwomen Policy Support was .94.

Transmen relative feelings & transwomen relative feelings thermometers. Three feelings thermometer items were admin- istered for each of the four target groups (transsexual men, trans- sexual women, biological men, and biological women). Each feel- ings thermometer was rated on a scale of 0 to 100 with the anchors of cold to warm (Renfro, Duran, Stephan, & Clason, 2006), neg- ative to positive (Breen & Karpinski, 2013), and unfavorable to favorable (Donakowski & Esses, 1996). The decision to use three thermometers was based on the desire to establish potential con- vergence of these items. To mirror the relative-scoring method of the IAT procedure, the following formulas were used to obtain relative explicit feelings scores: Transmen Relative Feelings � (Absolute Feelings Toward Biological Men�Absolute Feelings Toward Transsexual Men); Transwomen Relative Feelings � (Ab- solute Feelings Toward Biological Women�Absolute Feelings Toward Transsexual Women). Similar scoring procedures have been used to create relative preference measures for IAT research (Amodio & Devine, 2006; Blair et al., 2010; Nosek & Smyth, 2007). The valence of these relative scores matched the direction- ality of the IATs such that larger positive scores suggest more

Table 2 Validation Study Participant Demographics

Demographics Transgender (n) Cisgender (n)

Total 42 302 Female birth sex 33 173 Male birth sex 9 129 Asian/Pacific Islander 0 17 Black/Caribbean/African-American 6 23 Hispanic 2 16 Native American 1 3 White/Caucasian 31 234 Multi-racial 2 7 Other race 0 2 Asexual 7 13 Bisexual 8 49 Heterosexual 7 194 Homosexual 6 35 Pansexual 14 11 Female-to-male transsexual 6 — Male-to-female transsexual 4 — Other transgender (e.g., agender,

gender-fluid, gender-queer, and non-binary) 32 —

Table 3 Block Sequence Schematic of the Gender-Identity Implicit Association Tests

Block Number of trials Task Stimuli assigned to E-key response Stimuli assigned to I-key response

1 20 Practice Transsexual stimuli Biological stimuli 2 20 Practice Good stimuli Bad stimuli 3 20 Practice dual categorization Transsexual � Good stimuli Biological � Bad stimuli 4 40 Test dual categorization Transsexual � Good stimuli Biological � Bad stimuli 5 40 Practice Biological stimuli Transsexual stimuli 6 20 Practice dual categorization Biological � Good stimuli Transsexual � Bad stimuli 7 40 Test dual categorization Biological � Good stimuli Transsexual � Bad stimuli

Note. Blocks 1, 3, 4, and Blocks 5, 6, and 7 were switched for half the participants.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

173DEVELOPMENT OF GENDER IDENTITY IATS

cisgender preference, and larger negative scores suggest more transgender preference. After establishing that the three thermom- eters for each of the four target groups were highly intercorrelated (Cronbach’s � range: .96 –.98), two summated relative feelings scores were created for transmen and transwomen, respectively, by averaging the three relative formula calculations.

Social Desirability Scale–17 (SDS-17). The SDS-17 (Blake, Valdiserri, Neuendorf, & Nemeth, 2006; Stöber, 2001) was used to assess social desirability in responding. The scale consisted of 16 true/false items. Estimates of its internal consistency with U.S. samples is between the range of .64 to .70 (Blake et al., 2006). In the present sample of cisgender participants, reliability was .74 for the entire scale.

Political conservatism and religiosity. Conservatism and re- ligiosity were measured with self-report single items. The question for political stance asked participants “How would you rate your political stance in general?” The scale ranged from one to seven indicative of extremely conservative to extremely liberal. The religiosity question asked “How important is organized religion in your life?” with the range of one to seven indicative of extremely important to extremely unimportant.

Procedure

IAT stimuli pilot study. A pilot study was conducted before the validation study to determine the optimal stimuli for the GI-IATs. Two criteria were established to select specific items for the GI-IATs: (a) 95% accuracy in matching the item with the correct target group by pilot study participants, and (b) selection by six subject matter experts (SMEs). Lists of synonyms and words associated with transsexual women, transsexual men, biological women, and biological men were initially generated based on literature review and best practices for transgender inclusion. This word list was given to SMEs for review and further expanded upon through an iterative process. SMEs were licensed psychologists, some of whom were also transgender, with research and clinical expertise in LGBT�, ethnic, and gender issues. Participants for the pilot were recruited online through MTurk (Woo, Keith, & Thorn- ton, 2015) and were paid 25 cents to do the 10-min pilot study. Upon consent, participants were presented with the list of words one at a time, and were asked to match the target group to which the stimuli best belonged (biological women vs. transsexual wom- en; biological men vs. transsexual men). An initial item pool was selected based on 95% sorting accuracy. Thereafter, SMEs were consulted again and instructed to pick items from the accurately

sorted pool that were synonyms for transwomen and transmen, familiar to the general public, and not offensive to the transgender community. Three target stimuli per target group were chosen based on the selection criteria. Attribute stimuli for the targets of good and bad were the same for both measures and were the pre-validated items employed in the sexuality IAT of Nosek et al. (2007). Items from the sexuality IAT were chosen based on re- search showing the relationship between homophobia and trans- phobia (Nagoshi et al., 2008; Tebbe & Moradi, 2012), and because these specific stimuli were not identified as stereotypes associated with transmen and transwomen (Gazzola & Morrison, 2014). Additionally, the chosen exemplars are common attribute stimuli included in typical intergroup IATs (Rudman, 2011). Table 4 lists the stimuli used in the two GI-IATs.

The GI-IATs were modeled after a variant of the IAT procedure that uses target category groups and their synonyms as stimuli (Stef- fens, Kirschbaum, & Glados, 2008). For example, if the target cate- gories were African American and Caucasian, then the category labels of African American and Caucasian, and/or synonyms of Black and White are used rather than an associated concept of race-typical names like Diamond and Tyrone versus Emily and Connor. Steffens et al. (2008) found that using the actual target category labels as IAT stimuli resulted in higher correlations with attitudes toward out- groups than using ethnic names. They suggested that this is because using the actual target labels made the instrument more specifically tuned to the superordinate target constructs than using more distally associated concepts. This variant of the procedure was used for the GI-IATs because the pilot study showed that associated gender iden- tity words, such as transgender, gender-queer, and two-spirit, were less familiar to the general public than the actual target labels of transsexual women and transsexual men.

Validation study. Participants for the validation study were recruited from MTurk and, upon consent, completed demographic items, and then read information about the differences between sex and gender, as well as definitions of transsexual men and transsexual women. They then completed the implicit and explicit measures in a counterbalanced order, such that half were presented with the two GI-IATs first, and the other half responded to the explicit measures first. The two GI-IATs were also counterbalanced in the order of presentation (Transmen-IAT vs. Transwomen-IAT), as well as the presentation order of congruent versus incongruent pairings for target and attribute categories. One week later, participants were invited to the test–retest portion of the study in which they completed the two GI-IATs and AlloTrans measures in counterbalanced fashion. Trans-

Table 4 Stimuli Used for the Two Gender-Identity Implicit Association Tests

GI-IAT Target categories Target stimuli Attribute categories Attribute stimuli

Transmen-IAT Transsexual men Transsexual men, transsexual males, male transsexuals

Good Marvelous, superb, pleasure, beautiful, joyful, glorious, lovely, wonderful

Biological men Biological men, biological males, biological guys

Bad Tragic, horrible, agony, painful, terrible, awful, humiliate, nasty

Transwomen-IAT Transsexual women Transsexual women, transsexual females, female transsexuals

Good Marvelous, superb, pleasure, beautiful, joyful, glorious, lovely, wonderful

Biological women Biological women, biological females, biological gals

Bad Tragic, horrible, agony, painful, terrible, awful, humiliate, nasty

Note. GI-IAT � Gender-Identity Implicit Association Tests; IAT � Implicit Association Test.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

174 WANG-JONES ET AL.

gender participants were only asked to complete demographics, the two GI-IATs, and the feelings thermometers for the four target groups. These participants were not invited for the retest because the measures are ultimately intended to assess cisgender attitudes toward transgender groups. Each participant was paid $5.00 for completion of the entire study. Those who did not complete the test–retest round (N � 195) were paid $3.00.

Results

Reliability

Test–retest and internal consistency reliabilities were obtained for both GI-IATs. Cronbach’s � for the Transmen-IAT was .95, and retest reliability at one week was .45. For the Transwomen- IAT, Cronbach’s � was .95, and retest reliability was .48. This is comparable with the psychometric properties of other IAT proce- dures, which typically have internal consistencies that range be- tween .70 and .90 (Nosek et al., 2007), and retest reliabilities that vary between .25 and .69 (Lane, Banaji, Nosek, & Greenwald, 2007).

Additional analyses were conducted to assess for potential se- lective attrition considering that only a third (107/302) of the cisgender participants completed the retest a week later. Results showed that no differences were found in any of the measurement scores, nor in the demographic characteristics of those who com- pleted the retest compared to those who did not.

Known-Groups Validity

Responses to the GI-IATs showed group differences that were largely in the predicted directions when comparing respondents who differed in gender identity, sex, sexual orientation, and con- tact with transgender individuals. Figures 1 and 2 show the mean D scores and standard errors of the tested known-groups for the Transmen-IAT and the Transwomen-IAT, respectively. Neither GI-IATs violated assumptions of normality nor homogeneity of variance; thus, analyses were robust to differences in group size.

Cisgender versus transgender. Independent samples t-tests were used to examine group differences in scores on the GI-IATs between transgender and cisgender respondents. Both t-tests were statistically significant, Transmen-IAT t(342) � 2.10, p � .04, d � 0.35 and Transwomen-IAT t(342) � 2.51, p � .03, d � 0.46. Hypothesis 1a was supported; cisgender individuals had greater cisgender preference on both GI-IATs.

Cisgender heterosexual men versus women. Independent samples t-tests were used to examine group differences between cisgender heterosexual women versus men for scores on the GI- IATs. Contrary to the hypotheses, neither t-test was statistically significant, Transmen-IAT t(192) � �.70, p � .48 and Transwomen-IAT t(192) � �1.34, p � .18. This suggests that cisgender heterosexual women and men did not reliably show differences on the GI-IATs as predicted by Hypothesis 1b. How- ever, subsequent analyses also showed an absence of group dif- ferences on the explicit measures.

Heterosexual cisgender versus non-heterosexual cisgender. Independent-samples t-tests compared the scores on the GI-IATs between heterosexual and non-heterosexual groups. Group differ- ences based on sexuality were significant for both the Transmen- IAT t(300) � 4.68, p � .01, d � .56 and the Transwomen-IAT t(300) � 4.13, p � .01, d � .52. Hypothesis 1c was supported; heterosexual people showed greater cisgender preference than non-heterosexual people for both GI-IATs.

No contact versus contact. As hypothesized, independent- samples t-tests showed significant differences in GI-IAT scores between participants who differed in contact with transgender individuals for both the Transmen-IAT t(300) � 3.86, p � .001, d � 0.44 and the Transwomen-IAT t(300) � 4.46, p � .001, d � 0.51. As Hypothesis 1d predicted, participants who said they personally knew at least one transgender individual showed less cisgender preference on the GI-IATs compared with those who had no personal contact.

Political conservatism and religiosity. Bivariate correlations showed that while the GI-IATs were significantly correlated with political conservatism (Transmen-IAT r � .14, p � .02;

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

T ra

ns m

en -I

A T

D -s

co re

Figure 1. Transmen-IAT known-group differences in D score means and SEs. Displays the Transmen-IAT scores to illustrate known-group differences. Cis Het Fem � Cisgender Heterosexual Female; Cis Het Male � Cisgender Heterosexual Male; Non-Het Cis � Non-Heterosexual Cisgender; Het Cis � Heterosexual Cisgender.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

175DEVELOPMENT OF GENDER IDENTITY IATS

Transwomen-IAT r � .12, p � .04), they were not significantly correlated with greater degrees of religiosity (Transmen-IAT r � .11, p � .06; Transwomen-IAT r � .08, p � .18). Thus, Hypoth- esis 1e was partially supported.

Correlations Among Explicit and Implicit Measures

Table 5 shows correlations between the GI-IATs, explicit mea- sures of transphobia, and social desirability. As predicted by Hypotheses 2a and 2b, both GI-IATs were significantly correlated with the explicit measure of transphobia (GTS), and with the explicit measure of positive attitudes toward transsexual individ- uals (AlloTrans). Hypothesis 3b was also supported in that explicit measurement scores were significantly correlated with social de- sirability, but scores on the GI-IATs were not.

Note that measures sharing similar methods of measurement were more related with one another than measures using a different method of assessment. However, relationships between explicit and implicit measures are not typically high, with mean correla- tions between .24 (Hofmann et al., 2005) and .36 (Nosek, 2005), thus suggesting they are related but distinct constructs (Hofmann et al., 2005; Nosek, 2005; Nosek & Smyth, 2007).

Confirmatory Factor Analysis

Confirmatory Factory Analysis (CFA) was used to test the hypothesis that the latent variables underlying responses to the

explicit and implicit measures were related but distinct constructs. The standard procedure to derive IAT D scores averages the Da and Db scores calculated from the four dual categorization blocks of the practice and test tasks, respectively. These Da and Db scores of each GI-IAT were used in the CFA models as separate implicit bias indicators as opposed to creating testlets from the raw data to preserve the integrity of the D score. The relative feelings mea- sures were used as explicit attitude indicators because they allowed for differentiation between transwomen and transmen. Four nested models were tested to assess the fit of competing models, as depicted in Figure 3. List-wise deletion was used to address the 12 cases containing missing items for feelings thermometers for these analyses using the cisgender sample (n � 290).

Model 1 consisted of a single factor for attitudes toward trans- sexual individuals, with all indicators of the explicit and implicit measures loading on a single common factor. Model 2 had two factors, one from the IAT scores and the second factor comprised of the feelings thermometer items. Model 3 also had two factors, but they represented overall attitudes toward transmen and overall attitudes toward transwomen, with the explicit and implicit items loading together for each transgender group. Finally, Model 4 was the hypothesized four-factor model, with separate factors for im- plicit attitudes toward transmen, implicit attitudes toward trans- women, explicit attitudes toward transmen, and explicit attitudes toward transwomen. AMOS was used to test the competing models with maximum likelihood estimation, and many fit indices were obtained for each model (see Table 6): 2, 2/df, Akaike Informa- tion Criterion (AIC), comparative fit index (CFI), incremental fit index (IFI), normed fit index (NFI) and root mean square error of approximation (RMSEA). Values of the CFI, IFI, and NFI above .95 indicate good fit, while values of the RMSEA lower than .06 indicate good fit (Hancock & Freeman, 2001; Hu & Bentler, 1999). Consistent with the hypothesis, Model 4 yielded substan- tially better fit than the competing models (see Table 6). The AIC is comparative among models, and results showed that Model 4 is the best for minimizing information loss. Finally, the 2 analyses showed that only Model 4 had non-significant fit estimates be- tween expected and observed covariance matrices. Thus, absolute,

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

T ra

ns w

om en

-I A

T D

-s co

re

Figure 2. Transwomen IAT known-group differences in D score means and SEs. Displays the Transwomen- IAT scores to illustrate known-group differences. Cis Het Fem � Cisgender Heterosexual Female; Cis Het Male � Cisgender Heterosexual Male; Non-Het Cis � Non-Heterosexual Cisgender; Het Cis � Heterosexual Cisgender.

Table 5 Correlations of Measures

Measure Transmen-

IAT Transwomen-

IAT GTS AlloTrans

Transwomen-IAT .45�

GTS .31� .29�

AlloTrans �.31� �.23� �.86�

SDS-17 �.05 �.02 �.13� .18�

Note. GTS � Genderism and Transphobia Scale; AlloTrans � Allophilia Toward Transsexual Individuals; SDS-17 � Social Desirability Scale-17. � p � .05.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

176 WANG-JONES ET AL.

relative, and parsimonious indices all support Hypothesis 3a that implicit and explicit attitudes toward transmen and transwomen are related, but separate and unique latent attitude variables.

Predictive Validity

Two hierarchical multiple regressions (HMRs) were per- formed to test the hypotheses that implicit and explicit mea- sures predict intentions to support transgender workplace pol-

icies. Again, the relative feelings scales were the only explicit measures used because they assessed attitudes toward transmen and transwomen independently, as opposed to treating them as one homogenous group like the GTS and the AlloTrans. Table 7 presents results of these tests for transmen and Table 8 presents results for transwomen. In Step 1 of these analyses, social desirability, political conservatism, religiosity, contact, and sexual orientation (heterosexual or nonheterosexual) were entered as covariates. The GI-IATs and relative-feelings ther- mometer scores were entered at Step 2. The two analyses supported Hypothesis 4 that both implicit and explicit measures were significant predictors of support for transmen and trans- women workplace policies.

Two additional HMRs were performed post hoc, each with three steps to determine whether or not the implicit measures contribute incremental variance in predicting transgender policy support above and beyond the relative feelings thermometers. These anal- yses were similar to those conducted to evaluate predictive valid- ity, except that the GI-IATs were entered in the third step. Results showed that both the Transmen-IAT ( R2 � .02, F(1, 284) � 9.91, p � .01) and the Transwomen-IAT ( R2 � .01, F(1, 282) � 7.32, p � .01) explained additional variance in policy support intentions above and beyond the parallel explicit measures.

Table 6 Confirmatory Factory Analysis Fit Indices

Model 2 df 2/df AIC CFI IFI NFI RMSEA

Model 1 958.13 35 27.38 998.13 .68 .68 .67 .30 Model 2 789.71 34 23.23 831.71 .74 .74 .73 .28 Model 3 223.19 34 6.56 265.19 .93 .93 .92 .14 Model 4 38.47a 29 1.33a 90.47 1.00a 1.00a .99a .03a

Note. AIC � Akaike Information Criterion, lowest value indicates best among models tested; CFI � comparative fit index; IFI � incremental fit index; NFI � Normed Fit Index; RMSEA � root mean square error of approximation. a Good fit. CFA n � 290 cisgender participants used with 12 list-wise deletions for cases with missing data for feelings thermometer items.

Figure 3. Confirmatory factory analysis models. Illustrates the four nested models tested for model fit. TM-RFpositive � Transmen Relative Feelings with positive/negative anchors; TM-RFfavor � Transmen Relative Feelings with favorable/unfavorable anchors; TM-RFwarm � Transmen Relative Feelings with warm/cold anchors; TW-RFpositive � Transwomen Relative Feelings with positive/negative anchors; TW- RFfavor � Transwomen Relative Feelings with favorable/unfavorable anchors; TW-RFwarm � Transwomen Relative Feelings with warm/cold anchors. Each feelings thermometer was rated on a scale of 0 – 100 with a possible relative score range of �100 to 100. Larger positive scores indicated greater explicit bias against transsexual individuals.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

177DEVELOPMENT OF GENDER IDENTITY IATS

Discussion

The goal of this study was to develop implicit measures of attitudes toward transsexual individuals, and to gather initial evi- dence of reliability and validity of the new GI-IATs. Two IAT measures were created, one to assess attitude toward transsexual women versus biological women (Transwomen-IAT), and another to assess attitude toward transsexual men versus biological men (Transmen-IAT). Various methods of assessing test reliability (stability, internal consistency) and validity (known-groups, con- vergent, discriminant, and predictive) were examined, and results showed strong psychometric evidence for both GI-IATs.

Reliability

Consistent with previous IAT research (Greenwald et al., 2009; Nosek et al., 2007; Lane et al., 2007), test–retest reliability of the GI-IATs were modest in size, while the internal consistencies were

high. Modest stability and high internal consistency are common with IATs, perhaps suggesting that IATs capture some state- dependent constructs (Nosek et al., 2007). In fact, research has shown that IATs with these patterns of reliability coefficients are indeed capturing both state and trait constructs (Egloff, Schwerdt- feger, & Schmukle, 2005; Schmukle & Egloff, 2004). Further- more, some researchers question the existence of orthogonal trait and state attitudes because many studies have shown the moder- ating effects of context (Conrey & Smith, 2007; Steyer, Schmitt, & Eid, 1999; Tisak & Tisak, 2000). Thus, it is possible that attitudes toward transsexual people captured by the GI-IATs are state and trait dependent, and therefore, might not exhibit high stability.

Known-Groups Validity

Both GI-IATs successfully distinguished between different groups of respondents that should hypothetically differ in their attitudes toward transsexual individuals. Cisgender, heterosexual,

Table 7 Summary of Hierarchical Regression for Support of Transmen Workplace Policies

Step Variable B SE B � t p sr2

Step 1 Social desirability �.61 .35 �.09 �1.75 .08 .01 Conservatism .31 .05 .46 6.32 .001 .10 Religiosity .07 .04 .11 1.89 .06 .01 Contact �.28 .15 �.10 �1.82 .07 .01 Sexuality .53 .16 .18 3.31 .00 .03

Step 2 Social desirability �.08 .29 �.01 �.29 .77 .00 Conservatism .13 .04 .15 2.98 .003 .01 Religiosity .03 .03 .04 .95 .34 .00 Contact �.08 .13 �.03 �.64 .52 .00 Sexuality .17 .13 .06 1.28 .20 .00 Transmen-IAT .54 .17 .14 3.15 .002 .02 Transmen relative feelings .03 .00 .55 11.45 .001 .22

Note. Step 1: R2 � .27, adjusted R2 � .26, F(5, 286) � 21.49, p � .01; Step 2 R2 � .25, R2 � .52, adjusted R2 � .51, F(7, 284) � 44.52, p � .011. Analysis conducted with cisgender sample (n � 292) using list-wise deletion for 10 cases with missing data for Transmen Relative Feelings.

Table 8 Summary of Hierarchical Regression for Support of Transwomen Workplace Policies

Step Variable B SE B � t p sr2

Step 1 Social desirability �.51 .35 �.08 �1.46 .15 .01 Conservatism .33 .05 .37 6.61 .00 .11 Religiosity .07 .04 .11 1.97 .05 .01 Contact �.39 .15 �.13 �2.52 .01 .02 Sexuality .54 .16 .18 3.36 .00 .03

Step 2 Social desirability �.11 .27 �.02 �.42 .68 .00 Conservatism .17 .04 .19 4.21 .00 .03 Religiosity .02 .03 .04 .85 .40 .00 Contact �.05 .119 �.02 �.42 .68 .00 Sexuality .24 .13 .08 1.91 .06 .01 Transwomen-IAT .38 .14 .11 2.71 .01 .01 Transwomen relative feelings .03 .00 .60 13.74 .001 .27

Note. Step 1: R2 � .30, adjusted R2 � .28, F(5, 284) � 23.80, p � .01; Step 2 R2 � .25, R2 � .30, adjusted R2 � .59, F(7, 282) � 52.94, p � .01. Analysis conducted with cisgender sample (n � 290) using list-wise deletion for 12 cases with missing data for Transwomen Relative Feelings.

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

178 WANG-JONES ET AL.

politically conservative people, and those with no personal contact with transgender individuals showed cisgender preferences on both of the GI-IATs. These findings mirrored patterns seen in research with explicit measures of transphobia and homophobia (Case & Stewart, 2013; Cragun & Sumerau, 2015; Warriner et al., 2013; Woodford et al., 2012). Unexpectedly, cisgender heterosex- ual women and men did not differ in their GI-IAT scores. How- ever, subsequent analyses showed that sex differences were also absent on the explicit measures in this study, suggesting an ab- sence of male–female differences among cisgender participants in attitudes toward transgender people rather than an inability of the GI-IATs to detect such differences.

Convergent and Discriminant Validity

The GI-IATs showed convergent validity with explicit measures of attitude toward transgender individuals in the expected direc- tions. Both implicit measures were positively related with the GTS, a measure of negative attitudes toward transgender individ- uals. Likewise, both GI-IATs were inversely related with the AlloTrans scale, a measure of positive attitudes toward transsexual individuals. The relationships were moderate in size, which is consistent with previous research showing that implicit and ex- plicit attitudes are related, but distinct constructs (Nosek, 2007; Nosek et al., 2007; Nosek & Smyth, 2007). Therefore, the size and directionality of the relationships observed in the present study suggest that the GI-IATs are related to conceptually similar mea- sures.

Evidence of discriminant validity was established by comparing the fit of four different nested models with CFA. Results showed that the hypothesized 4-factor attitude model (Model 4) yielded superior fit to the data compared to the three competing models. Thus, results of the CFA supported the notion that implicit atti- tudes toward transmen, implicit attitudes toward transwomen, ex- plicit attitudes toward transmen, and explicit attitude toward trans- women are related but differentiable constructs. In other words, people have different attitudes pertaining to implicit and explicit attitudes toward transsexual men and women, and thus, it is necessary to have different measures to assess these distinct but associated constructs. Relying solely on self-reports and treating the transgender community as a homogenous group will not cap- ture attitudes that are not overtly reported and will not capture the differences in how people feel about transwomen versus transmen.

Additional discriminant validity evidence was supported by the non-significant correlations between the GI-IATs and social desir- ability. These findings suggest that the GI-IATs are more robust to self-presentation effects than the GTS, a popular explicit measure of transphobia. This highlights another value of using implicit measures to study attitudes toward transgender people by reducing the impact of response demands, and thus, underscoring the po- tential for the GI-IATs to reveal new insights about bias toward transgender individuals.

Predictive Validity

The fact that both implicit and explicit measures were signifi- cant predictors of policy support is consistent with literature show- ing that these types of measures assess both distinct and overlap- ping attitudinal concepts (Nosek, 2007; Nosek et al., 2007; Nosek

& Smyth, 2007). Furthermore analyses showed that the implicit measures explained incremental variance in policy support inten- tions above and beyond the explicit feelings thermometers. The unique variance in policy support intentions explained by the Transmen-IAT (2%) and the Transwomen-IAT (1%) were small, but research has shown that small effect sizes can translate to meaningful consequences (Greenwald, Banaji, & Nosek, 2015; Rosenthal & Rubin, 1982). The effect sizes of the GI-IATs were close to, but still below the 2.2 to 5.6% range of IAT predictive validity estimates for explaining unique attitude variance accord- ing to meta-analyses (Greenwald et al., 2009; Oswald, Mitchell, Blanton, Jaccard, & Tetlock, 2013). Even at the high end of this range, some researchers would dismiss the magnitude as practi- cally irrelevant (Oswald et al., 2013).

Greenwald et al. (2015) recently published an article on the practical meaning of small effect sizes from IAT meta-analyses regarding predictive validity. They cited Rosenthal (1990), who illustrated that miniscule effect sizes in medical research have been associated with “breakthrough” (p. 776) treatments with dramatic benefits. For example, research that initially identified aspirin as an effective treatment in reducing risk of heart attack only obtained r � .03. The correlation of azidothymidine and treatment out- comes for AIDS was r � .23. In relating this to implicit attitudes research, Greenwald et al. (2015) showed that the average 4% variance in anti-Black bias explained by the race IAT “represents potential for discriminatory impacts with very substantial societal significance” (p. 560). They cited discrimination studies of actual hiring practices by Rooth (2010) and Agerström and Rooth (2011) showing that the IAT can predict hiring disparities between Arab- Muslin versus Swedish job applicants (r � .11 to .18) and weight bias against women (r � .08) and men (r � .07). Thus the small effect sizes obtained from intergroup IATs do translate into mean- ingful real-world situations.

Greenwald et al. (2015) also examined the cumulative effects of recurrent experiences of intergroup bias, showing that even a 1% difference in the probability of successful outcomes between a privileged and disadvantaged group can lead to the latter experi- encing 15% discriminatory impact after 20 repeated occurrences. A simulation study by Martell, Lane, and Emrich (1996) showed that a 1% difference in performance evaluations favoring men over women led to differential sex representation throughout each level of the organization after 20 iterations. Where the absence of sex bias would have maintained 50/50% representation of women and men staffed throughout all eight levels of the hypothetical organi- zation, introduction of the 1% bias resulted in a ratio of 35% women and 65% men at the highest level. Considering that small effect sizes can translate into substantial practical consequences, and that the IAT can predict these societal disparities, the GI-IATs developed in the present research pave the road toward advancing the understanding of biases toward transgender persons.

Limitations

There are several limitations to this study. First, the research design was not experimental, limiting the ability to establish causal relationships. Second, the transgender sample in the present re- search consisted predominantly of birth-sex women who reported non-binary gender identities. This is a limitation because it is possible that these individuals’ experiences and attitudes toward

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

179DEVELOPMENT OF GENDER IDENTITY IATS

gender identity are different from those who specifically identify as transmen or transwomen. Third, the attitudes about workplace policies that were investigated in this study represent behavioral intentions, not actual behaviors. Although behavioral intentions are predictive of actual behaviors, the relationship is far from perfect (Maio & Haddock, 2009). Thus, additional research is needed to investigate behaviors directly. Another potential limita- tion is that the GI-IAT target categories and items were marked by the word “biological” when describing cisgender persons, which may have primed biases connected to the ideas about what is “natural” or “normal.” However, transphobia itself may be, at least in part, influenced by such biases, given that literature about transphobia describes cisgender people’s discomfort and ste- reotypes of this group as being partly driven by prejudices that the transgender experience is somehow unnatural or abnormal (Bettcher, 2012; Gazzola & Morrison, 2014; Schilt & Westbrook, 2009; Winter et al., 2009). Additionally, it is unlikely that the evidence of convergent, discriminant, and incremental validity found on both GI-IATs was all resulting from a natural bias toward what is biological. Another limitation is that the definitions of sex, gender, and transsexual given at the beginning of the survey may have induced priming effects. The reason these definitions were included was to ensure that people understood these terms accu- rately, given that these concepts are not part of current vernacular. As transgender terms and issues become more familiar to lay people, researchers can omit these definitions to obviate potential priming effects.

Finally, caution around the use of MTurk for data collection is another limitation that stems from concerns about sampling bias, accuracy of results, and generalizability. Certainly samples from MTurk are not representative of the population, but neither are traditional sampling methods. However, research by Berinsky, Huber, and Lenz (2012) comparing MTurk to representative U.S. samples suggest that although demographic differences were found, these variations are “substantively small” (p. 361), and do not present grossly distorted observations of the population. In fact, many studies show that MTurk samples are quite diverse, and much more representative than traditional convenience and student samples (Berinsky et al., 2012; Buhrmester et al., 2011; Gosling et al., 2004; Mason & Suri, 2012). Additionally, replication studies show that data from MTurk samples are accurate (Berinsky et al., 2012), psychometrically sound (Buhrmester et al., 2011; Steelman et al., 2014), and are generalizable to other formats and settings (Berinsky et al., 2012; Crump et al., 2013; Goodman, Cryder, & Cheema, 2013).

Future Research

Recent research has found that door-to-door canvassing by a transgender person can change explicit attitudes toward transgen- der individuals (Broockman & Kalla, 2016). It would be interest- ing to examine whether interventions designed to change attitudes are as effective in changing implicit attitudes measured by the GI-IATs as they are in changing explicit attitudes. Moreover, where resources are available to conduct studies in person, exper- iments can test the ability of the GI-IATs to predict actual behav- iors toward transgender people. Intergroup IATs have been shown to predict subtle social behaviors such as sitting distance, speech errors, and talk time (Amodio & Devine, 2006; Greenwald et al.,

2009; McConnell & Leibold, 2001). Future studies can use exper- imental designs to assess whether the GI-IATs can predict similar subtle intergroup behaviors toward transgender individuals. An- other more nuanced known-groups study can examine differences between sexual orientation beyond the binary division of straight versus not-straight that was used in this study. For example, people with more fluid sexualities may differ in their attitudes toward transgender individuals more than people whose sexualities follow the more rigid dichotomy of straight versus gay. Finally, additional research should be conducted to evaluate responses on the GI- IATs in other cultural, national, and demographic contexts.

Conclusions

Bias and unfair discrimination toward transgender individuals continue to occupy significant attention in the popular press and in the academic literature. Relatively less is known about implicit attitudes toward transgender persons, partly because implicit as- sociation measures have historically not been available for these groups. Therefore, two GI-IATs were developed in this study, one for transmen and one for transwomen, and overall, data strongly supported their reliability and validity in assessing implicit biases against transgender people in U.S. adults. The hope is that the present research will contribute to future studies aimed at measur- ing transphobia, understanding the causes and consequences of biases toward transgender persons, and the development of inter- ventions to reduce these biases.

References

Agerström, J., & Rooth, D. O. (2011). The role of automatic obesity stereotypes in real hiring discrimination. Journal of Applied Psychology, 96, 790 – 805. http://dx.doi.org/10.1037/a0021594

American Psychological Association. (2011). Answers to your questions about transgender people, gender identity, and gender expression. Re- trieved from http://www.apa.org/topics/lgbt/transgender.aspx

Amodio, D. M., & Devine, P. G. (2006). Stereotyping and evaluation in implicit race bias: Evidence for independent constructs and unique effects on behavior. Journal of Personality and Social Psychology, 91, 652– 661. http://dx.doi.org/10.1037/0022-3514.91.4.652

Banse, R., Seise, J., & Zerbes, N. (2001). Implicit attitudes towards homosexuality: Reliability, validity, and controllability of the IAT. Zeitschrift Für Experimentelle Psychologie: Organ Der Deutschen Ge- sellschaft Für Psychologie, 48, 145–160.

Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012). Evaluating online labor markets for experimental research: Amazon.com’s mechanical turk. Political Analysis, 20, 351–368. http://dx.doi.org/10.1093/pan/ mpr057

Bettcher, T. M. (2012). Full-frontal morality: The naked truth about gen- der. Hypatia, 27, 319 –337. http://dx.doi.org/10.1111/j.1527-2001.2011 .01184.x

Blair, I. V., Judd, C. M., Havranek, E. P., & Steiner, J. F. (2010). Using community data to test the discriminant validity of ethnic/racial group IATs. Zeitschrift Für Psychologie, 218, 36 – 43.

Blake, B. F., Valdiserri, J., Neuendorf, K. A., & Nemeth, J. (2006). Validity of the SDS-17 measure of social desirability in the American context. Personality and Individual Differences, 40, 1625–1636. http:// dx.doi.org/10.1016/j.paid.2005.12.007

Breen, A. B., & Karpinski, A. (2013). Implicit and explicit attitudes toward gay males and lesbians among heterosexual males and females. The Journal of Social Psychology, 153, 351–374. http://dx.doi.org/10.1080/ 00224545.2012.739581

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

180 WANG-JONES ET AL.

Broockman, D., & Kalla, J. (2016). Durably reducing transphobia: A field experiment on door-to-door canvassing. Science, 352, 220 –224. http:// dx.doi.org/10.1126/science.aad9713

Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechan- ical Turk: A new source of inexpensive, yet high-quality, data? Per- spectives on Psychological Science, 6, 3–5. http://dx.doi.org/10.1177/ 1745691610393980

Case, K., & Stewart, B. (2013). Intervention effectiveness in reducing prejudice against transsexuals. Journal of LGBT Youth, 10, 140 –158. http://dx.doi.org/10.1080/19361653.2012.718549

Conrey, F. R., & Smith, E. R. (2007). Attitude representation: Attitudes as patterns in a distributed, connectionist representational system. Social Cognition, 25, 718 –735. http://dx.doi.org/10.1521/soco.2007.25.5.718

Cox, D., & Jones, R. P. (2011). Most Americans favor rights and legal protections for transgender people. Retrieved from http://www.prri.org/ research/american-attitudes-towards-transgender-people/

Cragun, R. T., & Sumerau, J. E. (2015). The last bastion of sexual and gender prejudice? Sexualities, race, gender, religiosity, and spirituality in the examination of prejudice toward sexual and gender minorities. Journal of Sex Research, 52, 821– 834. http://dx.doi.org/10.1080/ 00224499.2014.925534

Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302. http://dx.doi.org/10.1037/ h0040957

Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013). Evaluating Amazon’s Mechanical Turk as a tool for experimental behavioral re- search. PLoS ONE, 8, e57410. http://dx.doi.org/10.1371/journal.pone .0057410

Donakowski, D. W., & Esses, V. M. (1996). Native Canadians, First Nations, or Aboriginals: The effect of labels on attitudes toward native peoples. Canadian Journal of Behavioural Science, 28, 86 –91. http:// dx.doi.org/10.1037/0008-400X.28.2.86

Egloff, B., Schwerdtfeger, A., & Schmukle, S. C. (2005). Temporal sta- bility of the implicit association test-anxiety. Journal of Personality Assessment, 84, 82– 88. http://dx.doi.org/10.1207/s15327752jpa8401_14

Fazio, R. H., & Olson, M. A. (2003). Implicit measures in social cognition. research: Their meaning and use. Annual Review of Psychology, 54, 297–327. http://dx.doi.org/10.1146/annurev.psych.54.101601.145225

Fingerhut, A. W. (2011). Straight Allies: What Predicts Heterosexuals’ Alliance With the LGBT Community? Journal of Applied Social Psy- chology, 41, 2230 –2248.

Flores, A. R. (2015). Attitudes toward transgender rights: Perceived knowledge and secondary interpersonal contact. Los Angeles, CA: The Williams Institute. Retrieved from http://williamsinstitute.law.ucla.edu/ wp-content/uploads/Attitudes-Transgender-Rights-USA-September- 2015.pdf

Gawronski, B., & Payne, B. K. (Eds.). (2010). Handbook of implicit social cognition: Measurement, theory, and applications. New York, NY: Guilford Press.

Gazzola, S. B., & Morrison, M. A. (2014). Cultural and personally en- dorsed stereotypes of transgender mena and transgender women: Nota- ble correspondence of disjunction? International Journal of Transgen- derism, 15, 76 –99. http://dx.doi.org/10.1080/15532739.2014.937041

Gerhardstein, K. R., & Anderson, V. N. (2010). There’s more than meets the eye: Facial appearance and evaluations of transsexual people. Sex Roles, 62(5– 6), 361–373. http://dx.doi.org/10.1007/s11199-010-9746-x

Goodman, J. K., Cryder, C. E., & Cheema, A. (2013). Data collection in a flat world: The strengths and weaknesses of Mechanical Turk samples. Journal of Behavioral Decision Making, 26, 213–224. http://dx.doi.org/ 10.1002/bdm.1753

Gosling, S. D., Vazire, S., Srivastava, S., & John, O. P. (2004). Should we trust web-based studies? A comparative analysis of six preconceptions about internet questionnaires. American Psychologist, 59, 93–104. http:// dx.doi.org/10.1037/0003-066X.59.2.93

Grant, J. M., Mottet, L. A., Tanis, J., Min, D., Harrison, J., Herman, J. L., & Keisling, M. (2011). Injustice at every turn: A report of the National Transgender Discrimination Survey. Washington, DC. Retrieved from http://www.thetaskforce.org/reports_and_research/ntds

Greenwald, A. G., & Banaji, M. R. (1995). Implicit social cognition: Attitudes, self-esteem, and stereotypes. Psychological Review, 102, 4 –27. http://dx.doi.org/10.1037/0033-295X.102.1.4

Greenwald, A. G., Banaji, M. R., & Nosek, B. A. (2015). Statistically small effects of the Implicit Association Test can have societally large effects. Journal of Personality and Social Psychology, 108, 553–561. http://dx .doi.org/10.1037/pspa0000016

Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. (1998). Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology, 74, 1464 –1480. http:// dx.doi.org/10.1037/0022-3514.74.6.1464

Greenwald, A. G., Nosek, B. A., & Banaji, M. R. (2003). Understanding and using the Implicit Association Test: I. An improved scoring algo- rithm. Journal of Personality and Social Psychology, 85, 197–216. http://dx.doi.org/10.1037/0022-3514.85.2.197

Greenwald, A. G., Poehlman, T. A., Uhlmann, E. L., & Banaji, M. R. (2009). Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity. Journal of Personality and Social Psychology, 97, 17– 41. http://dx.doi.org/10.1037/a0015575

Hancock, G. R., & Freeman, M. J. (2001). Power and sample sizes for the root mean square error of approximation test of not close fit in structural equation modeling. Educational and Psychological Measurement, 61, 741–758.

Hanson, J. (Ed.), (2012). Ideology, psychology, and law. Oxford, UK: Oxford University Press. http://dx.doi.org/10.1093/acprof:oso/ 9780199737512.001.0001

Hattie, J., & Cooksey, R. W. (1984). Procedures for assessing the validities of tests using the “Known-Groups” method. Applied Psychological Measurement, 8, 295–305. http://dx.doi.org/10.1177/01466216 8400800306

Hill, D. B., & Willoughby, B. L. B. (2005). The development and valida- tion of the Genderism and Transphobia Scale. Sex Roles, 53, 531–544. http://dx.doi.org/10.1007/s11199-005-7140-x

Hofmann, W., Gawronski, B., Gschwendner, T., Le, H., & Schmitt, M. (2005). A meta-analysis on the correlation between the implicit associ- ation test and explicit self-report measures. Personality and Social Psychology Bulletin, 31, 1369 –1385. http://dx.doi.org/10.1177/ 0146167205275613

Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alterna- tives. Structural Equation Modeling, 6, 1–55.

Human Rights Campaign. (2015). Addressing anti-transgender violence: Exploring realities, challenges, and solutions for policymakers and community advocates. Retrieved from http://hrc-assets.s3-website-us- east-1.amazonaws.com//files/assets/resources/HRC-AntiTransgender Violence-0519.pdf

Human Rights Council. (2011). Discriminatory laws and practices and acts of violence against individuals based on their sexual orientation and gender identity. United Nations. Retrieved from http://www.ohchr.org/ Documents/Issues/Discrimination/A.HRC.19.41_English.pdf

Inquisit Millisecond. (2015). Inquisit 4. Retrieved from http://www .millisecond.com

Jellison, W. A., McConnell, A. R., & Gabriel, S. (2004). Implicit and explicit measures of sexual orientation attitudes: In group preferences and related behaviors and beliefs among gay and straight men. Person- ality and Social Psychology Bulletin, 30, 629 – 642. http://dx.doi.org/10 .1177/0146167203262076

King, M. E., Winter, S., & Webster, B. (2009). Contact reduces transpreju- dice: A study on attitudes towards transgenderism and transgender civil

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

181DEVELOPMENT OF GENDER IDENTITY IATS

rights in Hong Kong. International Journal of Sexual Health, 21, 17–34. http://dx.doi.org/10.1080/19317610802434609

Kühnen, U., Schiessl, M., Bauer, N., Paulig, N., Pöhlmann, C., & Schmidthals, K. (2001). How robust is the IAT? Measuring and manip- ulating implicit attitudes of East- and West-Germans. Zeitschrift für Experimentelle Psychologie, 48, 135–144.

LaBouff, J. P., Rowatt, W. C., Johnson, M. K., Thedford, M., & Tsang, J. A. (2010). Development and initial validation of an implicit measure of religiousness-spirituality. Journal for the Scientific Study of Religion, 49, 439 – 455. http://dx.doi.org/10.1111/j.1468-5906.2010.01521.x

Lane, K. A., Banaji, M. R., Nosek, B. A., & Greenwald, A. G. (2007). Understanding and using the Implicit Association Test: IV: What we know (so far) about the method. In B. Wittenbrink & N. Schwarz (Eds.), Implicit measures of attitudes (pp. 59 –102). New York, NY: Guilford Press.

Mackenzie, S. B., Podsakoff, P. M., & Podsakoff, N. P. (2011). Construct measurement and validation procedures in MIS and behavioral research: Integrating new and existing techniques. MIS Quarterly, 35, 293–334. Retrieved from http://dl.acm.org/citation.cfm?id�2017510-npapers3:// publication/uuid/E4D3717C-7F3F-4791-8835-141D4309976B

Maio, G. R., & Haddock, G. (Eds.). (2009). The psychology of attitudes and attitude change. London, UK: SAGE.

Martell, R. F., Lane, D. M., & Emrich, C. (1996). Male-female differences: A computer simulation. American Psychologist, 51, 157–158. http://dx .doi.org/10.1037/0003-066X.51.2.157

Mason, W., & Suri, S. (2012). Conducting behavioral research on Ama- zon’s Mechanical Turk. Behavior Research Methods, 44, 1–23. http:// dx.doi.org/10.3758/s13428-011-0124-6

McConnell, A. R., & Leibold, J. M. (2001). Relations among the Implicit Association Test, discriminatory behavior, and explicit measures of racial attitudes. Journal of Experimental Social Psychology, 37, 435– 442. http://dx.doi.org/10.1006/jesp.2000.1470

Nagoshi, J. L., Adams, K. A., Terrell, H. K., Hill, E. D., Brzuzy, S., & Nagoshi, C. T. (2008). Gender differences in correlates of homophobia and transphobia. Sex Roles, 59, 521–531. http://dx.doi.org/10.1007/ s11199-008-9458-7

Newport, F. (2016). Disentangling attitudes toward transgender bathroom use. Retrieved from http://www.gallup.com/opinion/polling-matters/ 191774/disentangling-attitudes-toward-transgender-bathroom.aspx

Nier, J. A., & Gaertner, S. L. (2012). The challenge of detecting contem- porary forms of discrimination. Journal of Social Issues, 68, 207–220. http://dx.doi.org/10.1111/j.1540-4560.2012.01745.x

Norton, A. T., & Herek, G. M. (2013). Heterosexuals’ attitudes toward transgender people: Findings from a national probability sample of U.S. adults. Sex Roles, 68, 738 –753. http://doi.org/10.1007/s11199-011- 0110-6

Nosek, B. A. (2005). Moderators of the relationship between implicit and explicit evaluation. Journal of Experimental Psychology: General, 134, 565–584. http://dx.doi.org/10.1037/0096-3445.134.4.565

Nosek, B. A. (2007). Implicit-explicit relations. Current Directions in Psychological Science, 16, 65– 69. http://dx.doi.org/10.1111/j.1467- 8721.2007.00477.x

Nosek, B. A., Greenwald, A. G., & Banaji, M. R. (2007). The Implicit Association Test at age 7: A methodological and conceptual review. In J. A. Bargh (Ed.), Automatic processes in social thinking and behavior (pp. 265–292). New York, NY: Psychology Press.

Nosek, B. A., & Smyth, F. L. (2007). A multitrait-multimethod validation of the Implicit Association Test: Implicit and explicit attitudes are related but distinct constructs. Experimental Psychology, 54, 14 –29. http://dx.doi.org/10.1027/1618-3169.54.1.14

Nosek, B. A., Smyth, F. L., Hansen, J. J., Devos, T., Lindner, N. M., Ranganath, K. A., . . . Banaji, M. R. (2007). Pervasiveness and correlates of implicit attitudes and stereotypes. European Review of Social Psy- chology, 18, 36 – 88. http://dx.doi.org/10.1080/10463280701489053

Oswald, F. L., Mitchell, G., Blanton, H., Jaccard, J., & Tetlock, P. E. (2013). Predicting ethnic and racial discrimination: A meta-analysis of IAT criterion studies. Journal of Personality and Social Psychology, 105, 171–192. http://dx.doi.org/10.1037/a0032734

Pérez, E. O. (2010). Explicit evidence on the import of implicit attitudes: The IAT and immigration policy judgments. Political Behavior, 32, 517–545. http://dx.doi.org/10.1007/s11109-010-9115-z

Pittinsky, T. L., Rosenthal, S. A., & Montoya, R. M. (2010). Measuring positive attitudes toward outgroups: Development and validation of the Allophilia Scale. In L. R. Tropp & R. K. Mallett (Eds.), Moving beyond prejudice reduction: Pathways to positive intergroup relations (pp. 41– 60). Washington, DC: American Psychological Association.

Pruett, S. R., & Chan, F. (2006). The development and psychometric validation of the Disability Attitude Implicit Association Test. Rehabil- itation Psychology, 51, 202–213. http://dx.doi.org/10.1037/0090-5550 .51.3.202

Renfro, C. L., Duran, A., Stephan, W. G., & Clason, D. L. (2006). The role of threat in attitudes toward Affirmative Action and its beneficiaries. Journal of Applied Social Psychology, 36, 41–74. http://dx.doi.org/10 .1111/j.0021-9029.2006.00003.x

Roccato, M., & Zogmaister, C. (2010). Predicting the vote through implicit and explicit attitudes: A field research. Political Psychology, 31, 249 – 274. http://dx.doi.org/10.1111/j.1467-9221.2009.00751.x

Rooth, D. O. (2010). Automatic associations and discrimination in hiring: Real world evidence. Labour Economics, 17, 523–534. http://dx.doi.org/ 10.1016/j.labeco.2009.04.005

Rosenthal, R. (1990). How are we doing in soft psychology? American Psychologist, 45, 775–777. http://dx.doi.org/10.1037/0003-066X.45.6 .775

Rosenthal, R., & Rubin, D. B. (1982). A simple, general purpose display of magnitude of experimental effect. Journal of Educational Psychology, 74, 166 –169. http://dx.doi.org/10.1037/0022-0663.74.2.166

Rubin, A., & Babbie, E. (2009a). Research methods for social work (7th ed.). Belmont, CA: Brooks/Cole. http://dx.doi.org/10.1093/OBO_datas- et_home

Rubin, A., & Babbie, R. (2009b). Essential research methods for social work. Belmont, CA: Thomson/Brooks/Cole. Retrieved from https:// books.google.nl/books?id�cO8lh0omJtMC

Rudman, L. A. (2011). Implicit measures for social and personality psy- chology. Los Angeles, CA: SAGE. http://dx.doi.org/10.4135/ 9781473914797

Schilt, K., & Westbrook, L. (2009). Doing gender, doing heteronormativ- ity: “Gender normals,” transgender people, and the social maintenance of heterosexuality. Gender & Society, 23, 440 – 464. http://dx.doi.org/10 .1177/0891243209340034

Schmukle, S. C., & Egloff, B. (2004). Does the Implicit Association Test for assessing anxiety measure trait and state variance? European Journal of Personality, 18, 483– 494. http://doi.org/10.1002/per.525

Sherman, J. W., Gawronski, B., & Trope, Y. (Eds.). (2014). Dual-process theories of the social mind. New York, NY: Guilford Press.

Steelman, Z. R., Hammer, B. I., & Limayem, M. (2014). Data collection in the digital age: Innovation alternatives to student samples. Management Information Systems Quarterly, 38, 355–A20.

Steffens, M. C. (2005). Implicit and explicit attitudes towards lesbians and gay men. Journal of Homosexuality, 49, 39 – 66. http://dx.doi.org/10 .1300/J082v49n02_03

Steffens, M. C., Kirschbaum, M., & Glados, P. (2008). Avoiding stimulus confounds in Implicit Association Tests by using the concepts as stimuli. The British Journal of Social Psychology/the British Psychological Society, 47, 217–243. http://doi.org/10.1348/014466607X226998

Steyer, R., Schmitt, M., & Eid, M. (1999). Latent State-Trait Theory and research in personality and individual differences. European Journal of Personality, 13, 389 – 408. http://dx.doi.org/10.1002/(SICI)1099- 0984(199909/10)13:5�389::AID-PER361�3.0.CO;2-A

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

182 WANG-JONES ET AL.

Stöber, J. (2001). The Social Desirability Scale-17 (SDS-17): Convergent validity, discriminant validity, and relationship with age. European Journal of Psychological Assessment, 17, 222–232. http://dx.doi.org/10 .1027//1015-5759.17.3.222

Tebbe, E. N., & Moradi, B. (2012). Anti-transgender prejudice: A struc- tural equation model of associated constructs. Journal of Counseling Psychology, 59, 251–261. http://dx.doi.org/10.1037/a0026990

Tebbe, E., Moradi, B., & Ege, E. (2014). Revised and abbreviated forms of the Genderism and Transphobia Scale: Tools for assessing anti- transgender prejudice, 61, 581–592.

Tisak, J., & Tisak, M. S. (2000). Permanency and ephemerality of psy- chological measures with application to organizational commitment. Psychological Methods, 5, 175–198. http://dx.doi.org/10.1037/1082- 989X.5.2.175

Walch, S. E., Ngamake, S. T., Francisco, J., Stitt, R. L., & Shingler, K. A. (2012). The attitudes toward transgendered individuals scale: Psycho- metric properties. Archives of Sexual Behavior, 41, 1283–1291. http:// dx.doi.org/10.1007/s10508-012-9995-6

Walch, S. E., Sinkkanen, K. A., Swain, E. M., Francisco, J., Breaux, C. A., & Sjoberg, M. D. (2012). Using Intergroup Contact Theory to reduce stigma against transgender individuals: Impact of a transgender speaker panel presentation. Journal of Applied Social Psychology, 42, 2583– 2605. http://dx.doi.org/10.1111/j.1559-1816.2012.00955.x

Wang-Jones, T. S., Allen, R. M., Budzyn, N. J., & Ferdman, B. M. (2013). Support and perceived threat of transgender workplace policies: Effects

of rationales and labels. Honolulu, HI: Poster presentation at the Amer- ican Psychological Association Annual Conference.

Warriner, K., Nagoshi, C. T., & Nagoshi, J. L. (2013). Correlates of homophobia, transphobia, and internalized homophobia in gay or lesbian and heterosexual samples. Journal of Homosexuality, 60, 1297–1314. http://dx.doi.org/10.1080/00918369.2013.806177

Winter, S., Chalungsooth, P., Teh, Y. K., Rojanalert, N., Maneerat, K., Wong, Y. W., . . . Macapagal, R. A. (2009). Transpeople, transprejudice and pathologization: A seven-country factor analytic study. Interna- tional Journal of Sexual Health, 21, 96 –118. http://dx.doi.org/10.1080/ 19317610902922537

Woo, S. E., Keith, M., & Thornton, M. A. (2015). Amazon Mechanical Turk for industrial and organizational psychology: Advantages, chal- lenges, and practical recommendations. Industrial and Organizational Psychology, 8, 171–179. http://dx.doi.org/10.1017/iop.2015.21

Woodford, M. R., Silverschanz, P., Swank, E., Scherrer, K. S., & Raiz, L. (2012). Predictors of heterosexual college students’ attitudes toward LGBT people. Journal of LGBT Youth, 9, 297–320. http://dx.doi.org/10 .1080/19361653.2012.716697

Received May 17, 2016 Revision received December 5, 2016

Accepted December 5, 2016 �

E-Mail Notification of Your Latest Issue Online!

Would you like to know when the next issue of your favorite APA journal will be available online? This service is now available to you. Sign up at https://my.apa.org/portal/alerts/ and you will be notified by e-mail when issues of interest to you become available!

T hi

s do

cu m

en t

is co

py ri

gh te

d by

th e

A m

er ic

an P

sy ch

ol og

ic al

A ss

oc ia

ti on

or on

e of

it s

al li

ed pu

bl is

he rs

. T

hi s

ar ti

cl e

is in

te nd

ed so

le ly

fo r

th e

pe rs

on al

us e

of th

e in

di vi

du al

us er

an d

is no

t to

be di

ss em

in at

ed br

oa dl

y.

183DEVELOPMENT OF GENDER IDENTITY IATS