Psychology of Personality Research and Well-Being Research Matrix

Cognitive Behaviour therapy, 2017 voL. 46, no. 3, 224–238 http://dx.doi.org/10.1080/16506073.2016.1238503

Cognitive risk factors explain the relations between neuroticism and social anxiety for males and females

Nicholas P. Allan, Mary E. Oglesby, Aubree Uhl and Norman B. Schmidt

Department of psychology, Florida State university, tallahasssee, FL, uSa

ABSTRACT The hierarchical model of vulnerabilities to emotional distress contextualizes the relation between neuroticism and social anxiety as occurring indirectly through cognitive risk factors. In particular, inhibitory intolerance of uncertainty (IU; difficulty in uncertain circumstances), fear of negative evaluation (FNE; fear of being judged negatively), and anxiety sensitivity (AS) social concerns (fear of outwardly observable anxiety) are related to social anxiety. It is unclear whether these risk factors uniquely relate to social anxiety, and whether they account for the relations between neuroticism and social anxiety. The indirect relations between neuroticism and social anxiety through these and other risk factors were examined using structural equation modeling in a sample of 462 individuals (M age  =  36.56, SD  =  12.93; 64.3% female). Results indicated that the relations between neuroticism and social anxiety could be explained through inhibitory IU, FNE, and AS social concerns. No gender differences were found. These findings provide support for the hierarchical model of vulnerabilities to emotional distress disorders, although the cognitive risk factors accounted for variance beyond their contribution to the relation between neuroticism and social anxiety, suggesting a more complex model than that expressed in the hierarchical model of vulnerabilities.

Social anxiety disorder (SAD) is characterized by excessive fear and avoidance of social and/or performance situations in which the individual is fearful of evaluation or rejec- tion by others (APA, 2013). The lifetime prevalence rate of SAD is approximately 11%, making it one of the most common psychiatric conditions in the United States (Kessler, Petukhova, Sampson, & Zaslavsky, 2012). Further, SAD, even at the subsyndromal level, can lead to substantial impairment in social, occupational, and familial domains (Merikangas, Avenevoli, Acharyya, Zhang, & Angst, 2002), making SAD a considerable public health burden (Kessler et al., 2011; Olatunji, Cisler, & Tolin, 2007). Given the burden associated with SAD, identifying constructs that contribute to development and/or maintenance of this disorder would be important.

Neuroticism, broadly defined as the tendency to experience heightened levels of negative emotionality (Barlow et al., 2014; Watson, Clark, & Tellegen, 1988), has been consistently

KEYWORDS Social anxiety; neuroticism; intolerance of uncertainty; fear of negative evaluation; anxiety sensitivity

ARTICLE HISTORY received 5 March 2016 accepted 14 September 2016

© 2016 Swedish association for Behaviour therapy

CONTACT nicholas p. allan [email protected] Department of psychology, Florida State university, tallahasssee, FL 32306-4301, uSa.

COgNITIvE BEhAvIOUR ThERAPy 225

linked to SAD. Neuroticism has also been broadly linked with other anxiety-related and mood disorders (Brown, Chorpita, & Barlow, 1998; Kashdan, 2007), leading researchers to conceptualize neuroticism as a non-specific vulnerability factor for the development of SAD (Barlow, 2002; Mineka, Watson, & Clark, 1998). Given the lack of specificity of neuroticism as a risk factor for mood and anxiety disorders, researchers have hypothesized that the relation between neuroticism and mood and anxiety disorders may be indirect, through more specific cognitive risk factors (Norton & Mehta, 2007; Sexton, Norton, Walker, & Norton, 2003).

Several cognitive risk factors have formed strong associations with social anxiety. Among these are intolerance of uncertainty (IU), fear of negative evaluation (FNE), and anxiety sen- sitivity (AS) social concerns. IU is defined as an incapacity to endure an aversive response triggered by the perception of a lack of certainty about salient unknowns in the environment (Carleton, 2016; Carleton, Norton, & Asmundson, 2007). IU comprises two lower order dimensions, prospective IU and inhibitory IU (McEvoy & Mahoney, 2011). Prospective IU is described as a desire for predictability (e.g. “Unforeseen events upset me greatly”), whereas inhibitory IU is described as difficulty reacting to uncertain situations (e.g. “When it’s time to act, uncertainty paralyses me”; (Carleton et al., 2007). Although the global IU construct is associated with social anxiety, even controlling for other risk factors, including FNE and AS (Boelen & Reijntjes, 2009; Carleton et al., 2012), there appears to be some specificity within the lower order dimensions. Across multiple studies, when the IU dimensions were examined together as correlates of social anxiety, inhibitory IU demonstrated a robust, unique relation with social anxiety, including social interaction and performance anxiety as well as social avoidance and distress (Carleton, Collimore, & Asmundson, 2010; McEvoy & Mahoney, 2012). In contrast, no relation was found between prospective IU and social anxiety (Carleton et al., 2010; McEvoy & Mahoney, 2012). Therefore, the relation between IU and social anxiety may be primarily due to the inhibitory IU dimension and not the prospective IU dimension.

Additionally, FNE, apprehension about being negatively evaluated, is one of the most widely recognized risk factors for social anxiety (Heimberg, Brozovich, & Rapee, 2010; Rapee & Heimberg, 1997). Recent studies on FNE have not only continued to support its relevance in social anxiety, but have also demonstrated that this relation is robust, even considering other risk factors. For example, FNE is uniquely associated with social anx- iety, accounting for IU, AS, injury/illness sensitivity (IIS), as well as pain-related anxiety (Carleton, Abrams, Asmundson, Antony, & McCabe, 2009; Carleton, Thibodeau, Osborne, Taylor, & Asmundson, 2014; Carleton et al., 2007). Of particular relevance to the current study, Carleton et al. (2010) found an independent association between FNE and social anxiety, controlling for both IU and AS. Therefore, FNE also appears to be a robust, unique correlate of social anxiety.

AS is another risk factor associated with social anxiety. AS refers to the fear of anxiety-re- lated physical sensations due to the belief that these sensations have potentially harmful consequences (Reiss & McNally, 1985; Rodriguez, Bruce, Pagano, Spencer, & Keller, 2004). AS is best characterized as consisting of three lower order dimensions: physical concerns (e.g. “Its scares me when my heart beats rapidly”), cognitive concerns (e.g. “When I am nervous, I worry that I might be mentally ill”), and social concerns (e.g. “It is important for me not to appear nervous”; Zinbarg, Barlow, & Brown, 1997). A recent meta-analysis revealed that, of the AS dimensions, AS social concerns were most strongly related to social anxiety (Naragon-Gainey, 2010). Further, Zinbarg et al. (1997) reported that individuals

226 N. P. AllAN ET Al.

with social phobia had the highest average score on AS social concerns in comparison to other patient groups. Moreover, the AS social concerns dimension has been shown to have unique relations with social anxiety, controlling for IU and FNE (Carleton et al., 2010), whereas the AS physical and cognitive concerns dimensions do not have unique relations with social anxiety, controlling for AS social concerns (Allan, Capron, Raines, & Schmidt, 2014). Together, these findings suggest that AS social concerns also appears to operate as a unique predictor of social anxiety.

Given these associations, neuroticism may be related to social anxiety symptoms as well as symptoms of other anxiety-related disorders indirectly through cognitive risk fac- tors such as IU, FNE, and AS (i.e. the hierarchical model of vulnerabilities for emotional distress disorders; Norton, Sexton, Walker, & Norton, 2005; Paulus, Talkovsky, Heggeness, & Norton, 2015; Sexton et al., 2003). Specific to social anxiety, Norton and Mehta (2007) examined the indirect relations between neuroticism and social anxiety in a sample of undergraduate students using structural equation modeling (SEM). They found neurot- icism was indirectly related to social anxiety, primarily through IU. In a replication and extension of this study, conducted in a clinical sample, and again using SEM, Paulus et al. (2015) again found neuroticism was indirectly related to social anxiety through IU, but not AS. In a study exploring the mediating role of the lower order IU dimensions, McEvoy and Mahoney (2012) found that inhibitory IU, but not prospective IU, mediated the relation between neuroticism and social anxiety. Although these prior findings suggest that IU, and not AS, mediates the relations between neuroticism and social anxiety, these studies did not include a direct comparison between the lower order dimensions of IU and AS, which is especially relevant for social anxiety given prior studies indicating that IU is associated with social anxiety through inhibitory IU (e.g. McEvoy & Mahoney, 2012) and AS is associated with social anxiety through the AS social concerns dimension (e.g. Carleton et al., 2010)

Norton and colleagues championed the use of SEM in their studies examining the hier- archical mode of vulnerabilities because of the benefits this approach has over traditional regression-based approaches, such as decreased measurement error and the ability to model complex mediation models (Norton & Mehta, 2007). Another benefit of SEM is the ability to incorporate theoretically relevant moderator variables such as gender to examine meas- urement (differences in individual constructs) and structural (relations between constructs) differences. This is especially relevant for social anxiety given that lifetime prevalence rates of SAD diagnoses are significantly higher for females than they are for males (Kessler et al., 2012). Further, there are also gender differences in (1) social anxiety presentation, such that women report a greater number of social fears whereas men are more likely to report a fear of dating, (2) functional impairment, such that men are more likely to resort to drugs and alcohol and women are more likely to utilize pharmacological interventions, and (3) comorbidity, such that women are more likely to have a comorbid internalizing disorder while men are more likely to have a comorbid externalizing disorder (Turk et al., 1998; Xu et al., 2012). These differences might be the result of different mechanisms from neuroticism to the development of social anxiety in men and women.

The current study builds on prior work explicating the indirect relations between neurot- icism and social anxiety, which included higher order AS and IU intervening variables (e.g. Norton & Mehta, 2007), by including lower order intervening variables reflecting differential relations with social anxiety. Based on prior studies, it was expected that neuroticism would be indirectly related to social anxiety through inhibitory IU, FNE, and AS social concerns,

COgNITIvE BEhAvIOUR ThERAPy 227

but not the other AS dimensions or prospective IU (e.g. Allan et al., 2014; Carleton et al., 2010, 2012; Naragon-Gainey, 2010). Finally, although gender differences have been found such that females tend to have elevated levels of social anxiety symptoms and to be diag- nosed with SAD more so than males (Xu et al., 2012), no studies have considered whether the indirect relations between neuroticism and social anxiety differ by gender. Therefore, the moderating role of gender was examined to determine whether these complex rela- tions were different for males and females. Whereas it was expected that differences would emerge in levels of social anxiety, neuroticism, and cognitive risk factors, no differences in the magnitude of the relations (i.e. structural differences) between social anxiety and neuroticism and the cognitive risk factors were expected.

Method

Participants

The current sample consisted of 462 individuals recruited through an online crowd sourcing marketplace. Participants were primarily female (64.3%) with ages ranging from 18 to 77 (M = 36.56 years, SD = 12.93). A majority of participants identified as Caucasian (87.2%), followed by African American (5.6%), Asian (3.9%), American Indian or Alaskan Native (.4%), Native Hawaiian or Other Pacific Islander (.2%), and other (e.g. biracial; 2.7%). Furthermore, 6.9% of the sample identified as Hispanic or Latino. In regard to highest education achieved, 2.1% completed some high school, 13.2% completed high school or the equivalent, 3.7% completed a business, trade or technical school, 29.7% completed some college, 37% earned a 4-year college degree, and 14.3% a graduate degree.

Procedure

The current investigation recruited participants through Amazon’s Mechanical Turk (Mturk) to complete an online survey advertised as examining risk factors for anxiety and related pathol- ogy. Social anxiety was not specifically targeted. Participation took approximately one hour and individuals were paid $1.00 for their participation. Previous research has found that data collected through Mturk are of high quality and provides a diverse sample (Buhrmester, Kwang, & Gosling, 2011; Paolacci & Chandler, 2014). The following were the eligibility requirements for the current study: living in the United States, being 18 years of age or older, and having demonstrated high quality work on previous Mturk tasks as indicated by a Human Intelligence Task rating greater than 90% (i.e. greater than 90% of a participant’s prior submissions have been accepted as valid by those collecting their responses). Additionally, two validity check items were included in the battery of questionnaires as an indicator of random responding (e.g. “Are you reading this questionnaire?”). Individuals who missed at least one validity check item were excluded from the current analyses (19.5%). Informed consent was obtained prior to data collection and all procedures were approved by the university’s institutional review board.

Measures

Social Interaction Anxiety Scale (SIAS) The SIAS was used as an index of social anxiety symptoms (Mattick & Clarke, 1998). The SIAS is a 20-item self-report questionnaire designed to measure an individual’s cognitive, affective, and behavioral reactions to a social interaction situation (e.g. Meeting people at

228 N. P. AllAN ET Al.

parties). Respondents rate how characteristic of them each item is on a 5-point scale ranging from not at all to extremely. Previous research has found the SIAS to be psychometrically sound (Cox, Ross, Swinson, & Direnfeld, 1998). In the present analyses, the reverse-coded items were removed (i.e. items 5, 9, 11; Rodebaugh et al., 2011; Rodebaugh, Woods, & Heimberg, 2007). Given that the current investigation utilized a SEM approach, reliability was calculated using coefficient rho (ρ), according to Raykov (1997). The SIAS demonstrated good internal consistency (ρ = .97, 95% CI [.96, .97]).

Intolerance of Uncertainty Scale – Short Form (IUS-12) IU was indexed by the IUS-12 (Carleton et al., 2007). The IUS-12 is a 12-item questionnaire adapted from the original 27-item IUS (Freeston, Rhéaume, Letarte, Dugas, & Ladouceur, 1994) to assess an individual’s ability to tolerate the uncertainty of ambiguous situations, cognitive and behavioral responses to uncertainty, perceived implications of uncertainty, and attempts to control the future. Items are rated on a 5-point Likert scale ranging from 1 (Not at all characteristic of me) to 5 (Entirely characteristic of me). The IUS-12 is composed of two subscales: Prospective IU (7 items; e.g. “I can’t stand being taken by surprise”) and inhibitory IU (5 items; e.g. “When I am uncertain, I can’t function very well”). Previous research has found the IUS-12 to be psychometrically sound (Carleton et al., 2007; McEvoy & Mahoney, 2011). Within the current study, the Inhibitory IU (ρ = .94, 95% CI [.92, .95]) and Prospective IU (ρ = .90, 95% CI [.88, .92]) factors demonstrated good reliability.

Anxiety sensitivity index-3 (ASI-3) The ASI-3 is an 18-item self-report questionnaire designed to assess feared consequence of sensations associated with anxious arousal (Taylor et al., 2007). The ASI-3 is a modification of the original ASI (Reiss, Peterson, Gursky, & McNally, 1986). In addition to a total score, the multidimensional scale is composed of three subscales: physical, cognitive, and social concerns. Respondents are asked to rate the degree to which they agreed with each statement using a 5-point Likert-type scale ranging from 0 (Very little) to 4 (Very much). Research has demonstrated that the ASI-3 is a reliable and valid measure of anxiety sensitivity (Taylor et al., 2007). For the current sample, the ASI-3 Physical Concerns (ρ = .91, 95% CI [.90, .93]), Cognitive Concerns (ρ = .94, 95% CI [.93, .95]) and Social Concerns (ρ = .88, 95% CI [.86, .90]) factors demonstrated good internal consistency.

Big five inventory (BFI)—Neuroticism Neuroticism was indexed using the BFI. The BFI is a 44-item self-report questionnaire assessing the Big Five personality domains: Extraversion, Agreeableness, Conscientiousness, Neuroticism, and Openness (John, Donahue, & Kentle, 1991). Participants were asked to read a number of characteristic and rate how much each item applied to them on a 5-point Likert-type scale (1 = Strongly disagree, 5 = Agree strongly). Previous research has demon- strated strong psychometrics for the BFI scales (John & Srivastava, 1999). In the present investigation, only the neuroticism scale was used. In addition, to be consistent with other measures used, the reverse-coded items (i.e. items 9, 24, 34) were removed. The Neuroticism factor demonstrated good reliability (ρ = .89, 95% CI [.87, .91]).

Brief fear of negative evaluation, straightforward items (BFNE-S) Fears of negative evaluation (FNE) were indexed utilizing the BFNE-S (Rodebaugh, Holaway, & Heimberg, 2004; Weeks et al., 2005). The BFNE-S is a modified version of the original

COgNITIvE BEhAvIOUR ThERAPy 229

BFNE (Leary, 1983) comprising only the eight straightforwardly worded items (i.e. items 1, 3, 5, 6, 8, 9, 11, 12). Participants rate each item on a 5-point Likert scale ranging from 0 (Not at all characteristic of me) to 4 (Extremely characteristic of me). Previous research has found the 8-item measure to be a more reliable index of FNE than the reverse-coded items found in the original BFNE (Rodebaugh et al., 2004; Weeks et al., 2005). The BFNE-S has shown sound psychometric properties in both clinical and non-clinical samples (Carleton et al., 2007; Rodebaugh et al., 2004, 2011; Weeks et al., 2005). In the current sample, the FNE factor (composed of BFNE-S items) demonstrated good internal consistency (ρ = .97, 95% CI [.97, .98]).

Data analytic procedure

Structural equation modeling (SEM) was conducted in Mplus version 7.31 (Muthén & Muthén, 1998–2012). All data were modeled using full information maximum likelihood with the Yuan-Bentler scaled chi-square (Y-B χ2) to correct for any nonnormally distrib- uted data through robust standard errors, treating Likert-like data as continuous. Although there is some debate regarding whether Likert-like items can be modeled as continuous, a recent simulation study provided evidence that Likert-like items with five or more response options can be modeled accurately using MLR (Rhemtulla, Brosseau-Liard, & Savalei, 2012). Further, Stark, Chernyshenko, and Drasgow (2006) reported that treating Likert-like items as continuous is an effective approach for determining measurement invariance and Schmitt and Kuljanin (2008) reported that there are not well-validated approaches to examining measurement invariance when treating Likert-like items as categorical. Across all models, model fit was assessed using the Y-B χ2. A non-significant Y-B χ2 indicates that exact model fit cannot be dismissed. However, because the χ2 might suggest rejecting a model even when differences are minor, especially when scales with many items are modeled at the item-level (Browne, MacCallum, Kim, Andersen, & Glaser, 2002; Moshagen, 2012; Mulaik, 2007), the comparative fit index (CFI), the root mean square error of approximation (RMSEA) with accompanying 90% confidence interval (CI), and the squared root mean residual (SRMR), were used in conjunction to determine whether a model fit adequately. Although there are no “golden rules” for these fit indices, and any model producing a significant Y-B χ2 indi- cates some degree of model misfit, agreement among several fit indices provides evidence of acceptable model fit (Brown, 2015). CFI and TLI values greater than .90 indicate adequate fit and values greater than .95 indicate good fit. RMSEA values below .08 indicate adequate fit and RMSEA and SRMR values below .05 indicate good fit. Finally, a lower bound 90% CI less than .05 indicates that good fit cannot be rejected and an upper bound 90% CI greater than .10 indicates that poor fit cannot be rejected (Bentler, 1990; Browne & Cudeck, 1992; Hu & Bentler, 1999).

Confirmatory factor analysis (CFA) models were first constructed to examine fit and measurement invariance for males and females for each construct independently. Measurement invariance was examined within each construct according to procedures recommended by Meredith (1993). First, weak invariance was tested by comparing a model in which factor loadings were free to a model in which factor loadings were held to equality. Then, strong invariance was tested by comparing a model in which the intercepts were free to a model in which the intercepts were held to equality. Models were scaled by setting the first indicator to one by convention. Models were compared using the Y-B adjusted χ2 test,

230 N. P. AllAN ET Al.

with a non-significant value indicating that invariance held. If invariance was not achieved, the model was examined to determine if partial measurement invariance could be achieved (Steenkamp & Baumgartner, 1998). Items were systematically allowed to be free by first selecting items with the greatest difference from the mean of all items. If at least partial weak invariance was achieved the factor variance and factor covariances (if more than one factor comprised a construct) were examined for equality. If partial strong measurement invariance was achieved, latent mean differences were compared, treating males as the reference group by setting factor means for males to zero.

Once all CFA models were examined, a SEM model was conducted to examine the effects of the Neuroticism factor on the risk factors and the Social Anxiety factor. A final SEM model was conducted to examine multiple mediation pathways from the Neuroticism factor to the Social Anxiety factor. Bias-corrected bootstrapped CIs were calculated with 5000 bootstrap estimates to examine mediation pathways. Structural pathways within these models were tested for invariance across males and females.

Results

Descriptive statistics and correlations

Descriptive statistics and correlations, for males and females separately, are reported in Table 1. All variables were significantly correlated for both males and females. Examining missing data patterns revealed that five participants had not completed the SIAS and three participants had not completed the BFI neuroticism scale. These participants were included because the model estimator was robust to missing data. Scores on the SIAS (using straightforward items were elevated compared to a community sample of 18–59 year-olds (M = 16.30, SD = 12.48), but not to the range of a clinical sample (M = 43.93, SD = 11.84; Rodebaugh et al., 2011).

Confirmatory factor analyses models and invariance testing

All CFA model fit statistics and indices are provided in Table 2. Invariance was achieved across factor loadings and intercepts for the BFI Neuroticism factor with all items loading

Table 1. Descriptive statistics and correlations for neuroticism, risk factors, and social anxiety by gender.

notes: Correlations for females are on the lower diagonal and correlations for males are on the upper diagonal. Fne = Fear of negative evaluation. iu = intolerance of uncertainty. all correlations significant at p < .001.

*indicates that the factor means were significant at p < .05.

1 2 3 4 5 6 7 8 1. neuroticism – .78 .68 .72 .57 .58 .69 .73 2. Fne .71 – .73 .76 .50 .46 .69 .78 3. prospective iu .61 .60 – .87 .61 .58 .69 .71 4. inhibitory iu .72 .70 .88 – .62 .70 .71 .77 5. aS physical .58 .53 .50 .58 – .91 .79 .59 6. aS Cognitive .60 .60 .52 .68 .76 – .75 .59 7. aS Social .71 .75 .60 .68 .79 .75 – .71 8. Social anxiety .70 .76 .59 .70 .57 .66 .79 – Males Mean 2.58* 19.91* 20.12* 10.85 6.29 4.70 8.75 21.18 SD 1.00 10.09 6.82 5.77 6.36 6.12 6.40 17.54 Females Mean 2.97* 22.41* 21.20* 11.99 6.46 5.19 8.83 24.21 SD 1.05 10.96 7.37 6.18 6.52 6.54 6.45 18.45

COgNITIvE BEhAvIOUR ThERAPy 231

significantly (λs  ≥  .70). Factor variances were equal across males and females (∆ Y-B χ2 = .004, df = 1, p = .95). The latent mean was significantly higher for females (M = .38, standard error [SE] = .08) as compared to the latent mean for males (fixed to zero by default; ∆ Y-B χ2 = 12.75, df = 1, p < .001).

Invariance was also achieved across factor loadings and intercepts for the FNE factor, with all items loading significantly (λs  ≥  .87). Factor variances were equal across males and females (∆ Y-B χ2 = 3.50, df = 1, p = .06). The latent mean was significantly higher for females (M = .30, standard error [SE] = .12; ∆ Y-B χ2 = 6.06, df = 1, p < .05).

Partial weak invariance was achieved for the IU factors by allowing loadings for items 9 and 10 to freely vary across males and females. Several items had to be freed to obtain partial strong invariance (i.e. items 7, 10, 11, and 15). All items loaded significantly on their respective factor within the female and male models (λs  ≥  .60). Factor variances (∆ Y-B χ2 = 1.58, df = 2, p = .45) and covariances (r = .88, p < .001; ∆ Y-B χ2 = .04, df = 1, p = .84) were invariant. Whereas no difference in latent means was found for Inhibitory IU (∆ Y-B χ2 = 2.79, df = 1, p = .09), a significant difference was found for Prospective IU (∆ Y-B χ2 = 6.07, df = 1, p = .01) such that females had elevated mean factor scores (M = .27, SE = .11).

Partial weak and partial strong invariance were achieved by allowing the loading for item 16 and the intercept for item 9 to freely vary across males and females. All items loaded significantly on their respective factors (λs ≥ .57). Factor variances (∆ Y-B χ

2 = .53, df = 3, p  =  .91) were equal across males and females. Factor covariates between the AS Social Concerns factor and the AS Physical and Cognitive Concerns factors were equal (rs = .79 and .75, respectively; ∆ Y-B χ2  =  .04, df  =  2, p  =  .83), although the covariance between

Table 2. Model fit statistics and indices for confirmatory factor analyses of neuroticism, risk factors, and social anxiety factors.

notes: Weak invariance = equal factor loadings. Strong invariance = equal intercepts. y-B = yuan-Bentler. CFi = comparative fit index. rMSea = root mean square error of approximation. Ci = confidence interval. SrMr = square root mean square residual. LL = lower limit. uL = upper limit.

***p < .001; **p < .01; *p < .05

90% CI CFA Models y-B χ2 df ∆ y-B χ2 CFI RMSEA LL uL SRMR Neuroticism Baseline Configural 24.35** 10 – .98 .08 .04 .12 .02 Weak invariance 33.35** 14 8.49 .98 .08 .04 .11 .04 Strong invariance 40.78** 18 7.17 .97 .07 .05 .11 .05 FNE Baseline Configural 109.28 40 – .97 .09 .07 .11 .02 Weak invariance 120.87 47 3.28 .97 .08 .07 .10 .02 Strong invariance 132.02 54 7.90 .97 .08 .06 .10 .02 IU Baseline Configural 406.36 106 – .91 .11 .10 .12 .05 Weak invariance 432.90 116 19.48* .91 .11 .10 .12 .06 partial Weak invariance 423.68 114 9.34 .91 .11 .10 .12 .06 Strong invariance 473.14 126 50.22 .90 .11 .10 .12 .07 partial Strong invariance 439.70 122 14.15 .91 .11 .10 .12 .06 AS Baseline Configural 592.19 264 – .93 .07 .07 .08 .05 Weak invariance 620.66 279 26.76* .93 .07 .07 .08 .06 partial Weak invariance 615.16 278 20.31 .93 .07 .07 .08 .05 Strong invariance 645.70 293 29.63* .92 .07 .07 .08 .06 partial Strong invariance 637.19 292 18.87 .92 .07 .06 .08 .06 Social Anxiety Baseline Configural 617.41 238 – .92 .08 .08 .09 .04 Weak invariance 645.88 254 22.36 .92 .08 .07 .09 .04 Strong invariance 673.87 270 23.86 .92 .08 .07 .09 .04

232 N. P. AllAN ET Al.

the AS Physical and Cognitive Concerns factor was higher in males (r = .92, p < .001; ∆ Y-B χ2 = 28.57, df = 1, p < .001) than it was in females (r = .75, p < .001). There was no significant mean difference between males and females for AS Physical Concerns (∆ Y-B χ2 = .64, df = 1, p = .42), AS Cognitive Concerns (∆ Y-B χ2 = 1.87, df = 1, p = .17), or AS Social Concerns (∆ Y-B χ2 = .001, df = 1, p = .98).

Finally, invariance was achieved across factor loadings and intercepts for the Social Anxiety factor with all items loading significantly (λs ≥ .57). Factor variances were equal across males and females (∆ Y-B χ2 = 1.46, df = 1, p = .23). The latent mean was significantly higher for females (M = .29, standard error [SE] = .10; ∆ Y-B χ2 = 7.88, df = 1, p < .01).

Structural equation model examining the relations between neuroticism and social anxiety factors

A SEM was conducted examining the relations between the Neuroticism factor and all potential mediator variables as well as the Social Anxiety factor. The relations between the Neuroticism factor and the outcome variables did not differ by gender (∆ Y-B χ2  =  .65, df = 7, p = 1.00). A model with these pathways held to equality (with all factors modeled using their best-fitting CFA models) provided marginal to adequate fit to the data (Y-B χ2 = 6024.73, CFI = .89 RMSEA = .06, 90% CI [.05, .06], SRMR = .06). Model parameters are provided in Table 3. To account for multiple parameter tests, Bonferroni-corrected p values (.05/7 = .007) were used. Neuroticism was significantly associated with all risk factors as well as Social Anxiety, with estimates of proportion of variance accounted ranging from 30% (AS Physical Concerns) to 50% (FNE).

Mediation model examining the direct and indirect relations between neuroticism and social anxiety through risk factors

The mediation model including indirect pathways from the Neuroticism factor to the Social Anxiety factor did not differ by gender in pathways from the risk factors to the Social Anxiety factor (∆ Y-B χ2 = 5.89, df = 5, p = .32). A model with all direct and indirect path- ways held to equality across males and females provided marginal to adequate fit to the data (Y-B χ2 = 6073.10, CFI = .89 RMSEA = .06, 90% CI [.05, .06], SRMR = .07). Direct and indirect pathway estimates are provided in Figure 1. All results are reported as unstandard- ized (B) by convention, using Bonferroni-corrected p values below .004 (.05/13) and 99% CIs for indirect effect to approximate this correction. Indirect effects were found between the Neuroticism factor and the Social Anxiety factor through the FNE factor (B = .33, 99%

Table 3. Structural equation model parameters for the relations neuroticism shares with social anxiety risk factors and social anxiety.

note: Se = Standard error. ***p < .001.

Social Anxiety variables B SE β R2

Fear of negative evaluation .99*** .07 .70 .50 prospective iu .70*** .07 .60 .36 inhibitory iu .88*** .06 .68 .47 aS physical Concerns .66*** .06 .55 .30 aS Cognitive Concerns .60*** .05 .57 .32 aS Social Concerns .54*** .05 .66 .43 Social anxiety .84*** .06 .64 .41

COgNITIvE BEhAvIOUR ThERAPy 233

CI [.17, .51]) and the AS Social Concerns factor (B = .28, 99% CI [.10, .49]) and when not accounting for the Bonferroni correction, the Inhibitory IU factor (B = .19, 99% CI [.01, .38]), Accounting for these effects, the relation between the Neuroticism factor and the Social Anxiety factor was no longer significant (B = .17, p = .02). Further, 57% of the variance in the Social Anxiety factor was accounted for by this model.

Discussion

Results of the current study provide support for the hierarchical model of vulnerabilities for emotional distress disorders, in regards to social anxiety (Norton & Mehta, 2007; Norton et al., 2005; Sexton et al., 2003). The relations between neuroticism and social anxiety were accounted for indirectly through inhibitory IU (marginally), FNE, and AS social concerns. However, given that 41% of the variance in social anxiety was accounted for when only neuroticism was included as a predictor of social anxiety and 57% of the vari- ance was accounted for when the cognitive risk factors were also included, these cognitive risk factors appear to be uniquely related to social anxiety beyond their role in explicat- ing the relations between neuroticism and social anxiety. The finding that these cognitive risk factors uniquely contribute to variance in social anxiety provides some support for a competing model for emotional distress disorders, the triple vulnerability model (Barlow, 2000, 2002). Within this model, general biological vulnerabilities (i.e. neuroticism), general psychological vulnerabilities (i.e. perceived control over life stressors), and disorder-specific vulnerabilities are each purported to be uniquely related to emotional distress disorders. However, only partial support is provided for this model as well, given that neuroticism and the cognitive risk factors are purported to be uniquely related to anxiety, in contrast to the finding that neuroticism is indirectly related to social anxiety. Therefore, given that

Figure 1.  Structural equation mediation model examining the direct and indirect relations between neuroticism and Social anxiety through Several risk Factors. neur = neuroticism. Fne = Fear of negative evaluation. pros iu = prospective intolerance of uncertainty. inhib iu = inhibitory intolerance of uncertainty. aS phy = anxiety Sensitivity physical Concerns. aS Cog = anxiety Sensitivity Cognitive Concerns. aS Soc = anxiety Sensitivity Social Concerns. non-significant effects are represented by dashed lines. Significance was determined by Bonferroni-corrected p value of .004. all item-level data and residual variances are omitted for clarity. Model effects were not different across males and females.

234 N. P. AllAN ET Al.

neither model fully accounts for the relations between neuroticism, social anxiety, and the cognitive risk factors, it might be that some combination of the hierarchical model and the triple vulnerability model best explains the complex interplay between general biological vulnerabilities, cognitive risk factors, and social anxiety.

Within the current study, AS social concerns, inhibitory IU, and FNE were all uniquely related to social anxiety, although inhibitory IU was not under the most conservative of analytical approaches. In contrast, AS physical and cognitive concerns and prospective IU were not. These findings are consistent with prior findings also demonstrating specificity in the lower order dimensions of AS and IU (e.g. Allan et al., 2014; Carleton et al., 2010; Naragon-Gainey, 2010). Although it would seem that results of the current study are incon- sistent with prior studies by Norton and colleagues (e.g. Norton & Mehta, 2007; Paulus et al., 2015) in which AS was not associated with social anxiety, the current study actually clarifies the relations between AS and social anxiety. AS is associated with social anxiety through fear of observable anxiety symptoms and not simply through a fear of these anxiety symptoms per se.

Examination of gender differences revealed mean-level differences such that women reported higher levels of neuroticism, social anxiety, FNE, and prospective IU. These mean- level differences did not extend to reflect different relations between neuroticism, the cog- nitive risk factors, and social anxiety. These mean-level differences are consistent with prior studies reporting mean-level differences between men and women across neuroticism, cognitive risk factors, and social anxiety (e.g. McLean & Anderson, 2009). Given neurot- icism’s conceptualization as a biological vulnerability, and likely temporal precedence in the development of cognitive risk factors and social anxiety, it is likely that the differences in the cognitive risk factors and social anxiety are a result of differences in neuroticism. Therefore, to understand differences in social anxiety, it might be most fruitful to focus on differences in the etiology of neuroticism. Indeed, McLean and Anderson (2009) note that in very young children, neuroticism differences tend to be reversed, such that boys seem to possess higher levels of neuroticism. It is only around age two that these differ- ences reverse, which led the authors to suggest that these differences might reflect gender socialization or a complex gender socialization by gene interaction. The current findings seem to confirm that gender differences are not necessarily a function of differences in the mechanisms leading from neuroticism to social anxiety, but rather a function of differences in neuroticism, more generally.

There are several limitations to consider when interpreting the results of this study. First, cross-sectional data were used, and as a result, the direction of these effects cannot be assured. Second, only self-report measures were utilized. Multi-method and longitu- dinal studies are needed to address both of these limitations and to more firmly establish the validity of the hierarchical model of vulnerabilities for emotional distress disorders or directly compare this model to the triple vulnerability model. Third, it is possible that these risk factors might mediate the relation between social anxiety other general vulnerability factors such as extraversion, especially given studies linking extraversion to both IU and social anxiety (e.g. Fergus & Rowatt, 2014; Norton & Mehta, 2007). Finally, two related limitations are the use of data collected via online crowdsourcing techniques (i.e. Mturk) and the use of a nonclinical sample. However, whereas there are several differences between Amazon Mturk survey participants and the general population (Berinsky, Huber, & Lenz, 2012; Paolacci & Chandler, 2014), there is also evidence that Mturk participants are more

COgNITIvE BEhAvIOUR ThERAPy 235

attentive to instructions than are participants from convenience samples (Hauser & Schwarz, 2015). However, given that Mturk samples may be more socially anxious than community samples (e.g. Shapiro, Chandler, & Mueller, 2013), these results should be replicated in community and clinical samples. A similar approach that was undertaken to test measure- ment and structural invariance for gender could be used to establish invariance between an Mturk sample and community and clinical samples.

Regardless of these limitations, this study adds to the literature in several important ways. Support for the hierarchical model of vulnerabilities for emotional distress disorders was provided for social anxiety although some limitations of this model were also noted. In addition, the indirect pathways from neuroticism to social anxiety were clarified. That is, not only does neuroticism relate to social anxiety through IU, but also through AS and FNE, and the role of IU and AS is at the subdimension level (specifically inhibitory IU and AS social concerns). Finally, the current study demonstrated that whereas levels of social anxiety may differ between males and females, these differences do not extend to how social anxiety relates to risk factors.

Disclosure statement

No potential conflict of interest was reported by the authors.

References

Allan, N. P., Capron, D. W., Raines, A. M., & Schmidt, N. B. (2014). Unique relations among anxiety sensitivity factors and anxiety, depression, and suicidal ideation. Journal of Anxiety Disorders, 28, 266–275. http://dx.doi.org/10.1016/j.janxdis.2013.12.004

American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author.

Barlow, D. H. (2000). Unraveling the mysteries of anxiety and its disorders from the perspective of emotion theory. American Psychologist, 55, 1247–1263. http://dx.doi.org/10.1037/0003- 066X.55.11.1247

Barlow, D. H. (2002). Anxiety and its disorders: The nature and treatment of anxiety and panic (2nd ed.). New York, NY: Guilford Press.

Barlow, D. H., Sauer-Zavala, S., Carl, J. R., Bullis, J. R., & Ellard, K. K. (2014). The nature, diagnosis, and treatment of neuroticism back to the future. Clinical Psychological Science, 2, 344–365.

Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238– 246. http://dx.doi.org/10.1037/0033-2909.107.2.238

Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012). Evaluating online labor markets for experimental research: Amazon.com’s mechanical turk. Political Analysis, 20, 351–368. http://dx.doi.org/10.1093/ pan/mpr057

Boelen, P. A., & Reijntjes, A. (2009). Intolerance of uncertainty and social anxiety. Journal of Anxiety Disorders, 23, 130–135. http://dx.doi.org/10.1016/j.janxdis.2008.04.007

Brown, T. A. (2015). Confirmatory factor analysis for applied research. New York, NY: Guilford Publications.

Brown, T. A., Chorpita, B. F., & Barlow, D. H. (1998). Structural relationships among dimensions of the DSM-IV anxiety and mood disorders and dimensions of negative affect, positive affect, and autonomic arousal. Journal of Abnormal Psychology, 107, 179–192. http://dx.doi.org/10.1037/0021- 843X.107.2.179

Browne, M. W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21, 230–258. http://dx.doi.org/10.1177/0049124192021002005

236 N. P. AllAN ET Al.

Browne, M. W., MacCallum, R. C., Kim, C.-T., Andersen, B. L., & Glaser, R. (2002). When fit indices and residuals are incompatible. Psychological Methods, 7, 403–421. http://dx.doi.org/10.1037/1082- 989X.7.4.403

Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon's mechanical turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6, 3–5. http://dx.doi. org/10.1177/1745691610393980

Carleton, R. N. (2016). Fear of the unknown: One fear to rule them all? Journal of Anxiety Disorders, 41, 5–21. http://dx.doi.org/10.1016/j.janxdis.2016.03.011

Carleton, R. N., Abrams, M. P., Asmundson, G. J. G., Antony, M. M., & McCabe, R. E. (2009). Pain- related anxiety and anxiety sensitivity across anxiety and depressive disorders. Journal of Anxiety Disorders, 23, 791–798. http://dx.doi.org/10.1016/j.janxdis.2009.03.003

Carleton, R. N., Collimore, K. C., & Asmundson, G. J. G. (2010). “It’s not just the judgements-It’s that I don’t know”: Intolerance of uncertainty as a predictor of social anxiety. Journal of Anxiety Disorders, 24, 189–195. http://dx.doi.org/10.1016/j.janxdis.2009.10.007

Carleton, R. N., Mulvogue, M. K., Thibodeau, M. A., McCabe, R. E., Antony, M. M., & Asmundson, G. J. G. (2012). Increasingly certain about uncertainty: Intolerance of uncertainty across anxiety and depression. Journal of Anxiety Disorders, 26, 468–479. http://dx.doi.org/10.1016/j. janxdis.2012.01.011

Carleton, R. N., Norton, M. A. P. J., & Asmundson, G. J. G. (2007). Fearing the unknown: A short version of the Intolerance of Uncertainty Scale. Journal of Anxiety Disorders, 21, 105–117. http:// dx.doi.org/10.1016/j.janxdis.2006.03.014

Carleton, R. N., Thibodeau, M. A., Osborne, J. W., Taylor, S., & Asmundson, G. J. G. (2014). Revisiting the fundamental fears: Towards establishing construct independence. Personality and Individual Differences, 63, 94–99. http://dx.doi.org/10.1016/j.paid.2014.01.040

Cox, B. J., Ross, L., Swinson, R. P., & Direnfeld, D. M. (1998). A comparison of social phobia outcome measures in cognitive-behavioral group therapy. Behavior Modification, 22, 285–297. http://dx.doi. org/10.1177/01454455980223004

Fergus, T. A., & Rowatt, W. C. (2014). Intolerance of uncertainty and personality: Experiential permeability is associated with difficulties tolerating uncertainty. Personality and Individual Differences, 58, 128–131. http://dx.doi.org/10.1016/j.paid.2013.10.017

Freeston, M. H., Rhéaume, J., Letarte, H., Dugas, M. J., & Ladouceur, R. (1994). Why do people worry? Personality and Individual Differences, 17, 791–802. http://dx.doi.org/10.1016/0191- 8869(94)90048-5

Hauser, D. J., & Schwarz, N. (2015). Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior Research Methods, 48, 400–407. http:// dx.doi.org/10.3758/s13428-015-0578-z

Heimberg, R. G., Brozovich, F. A., & Rapee, R. M. (2010). A cognitive-behavioral model of social anxiety disorder: Update and extension. Social Anxiety: Clinical, Developmental, and Social Perspectives, 2, 395–422. http://dx.doi.org/10.1016/B978-0-12-375096-9.00015-8

Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55. http://dx.doi.org/10.1080/10705519909540118

John, O. P., & Srivastava, S. (1999). The big five trait taxonomy: History, measurement, and theoretical perspectives. In Handbook of personality: Theory and research (pp. 102–138). New York, NY: Guilford Press.

John, O. P., Donahue, E. M., & Kentle, R. L. (1991). The Big Five Inventory-Versions 4a and 54. Berkeley, CA: University of California. Berkeley, Institute of Personality and Social Research.

Kashdan, T. B. (2007). Social anxiety spectrum and diminished positive experiences: Theoretical synthesis and meta-analysis. Clinical Psychology Review, 27, 348–365. http://dx.doi.org/10.1016/j. cpr.2006.12.003

Kessler, R. C., Aguilar-Gaxiola, S., Alonso, J., Chatterji, S., Lee, S., Ormel, J., … Wang, P. S. (2011). The global burden of mental disorders: An update from the WHO World Mental Health (WMH) Surveys, Epidemiologia e psichiatria sociale, 18, 23–33. http://dx.doi.org/10.1017/S1121189X00001421

COgNITIvE BEhAvIOUR ThERAPy 237

Kessler, R. C., Petukhova, M., Sampson, N. A., & Zaslavsky, A. M. (2012). Twelve-month and lifetime prevalence and lifetime morbid risk of anxiety and mood disorders in the United States. International Journal of Methods in Psychiatric Research, 21, 169–184. http://dx.doi.org/10.1002/ mpr

Leary, M. R. (1983). A brief version of the fear of negative evaluation scale. Personality and Social Psychology Bulletin, 9, 371–375. http://dx.doi.org/10.1177/0146167283093007

Mattick, R. P., & Clarke, J. C. (1998). Development and validation of measures of social phobia scrutiny fear and social interaction anxiety11Editor’s note: This article was written before the development of some contemporary measures of social phobia, such as the Social Phobia and Anxiety Inventory (Turner et al., 1989). We have invited this article for publication because of the growing interest in the scales described therein. S.T. Behaviour Research and Therapy, 36, 455–470. http://dx.doi. org/10.1016/S0005-7967(97)10031-6

McEvoy, P. M., & Mahoney, A. E. J. (2011). Achieving certainty about the structure of intolerance of uncertainty in a treatment-seeking sample with anxiety and depression. Journal of Anxiety Disorders, 25, 112–122. http://dx.doi.org/10.1016/j.janxdis.2010.08.010

McEvoy, P. M. & Mahoney, A. E. (2012). To be sure, to be sure: Intolerance of uncertainty mediates symptoms of various anxiety disorders and depression. Behavior therapy, 43, 533–545. http:// dx.doi.org/10.1016/j.beth.2011.02.007

McLean, C. P., & Anderson, E. R. (2009). Brave men and timid women? A review of the gender differences in fear and anxiety. Clinical Psychology Review, 29, 496–505. http://dx.doi.org/10.1016/j. cpr.2009.05.003

Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58, 525–543. http://dx.doi.org/10.1007/BF02294825

Merikangas, K. R., Avenevoli, S., Acharyya, S., Zhang, H., & Angst, J. (2002). The spectrum of social phobia in the zurich cohort study of young adults. Biological Psychiatry, 51, 81–91. http://dx.doi. org/10.1016/S0006-3223(01)01309-9

Mineka, S., Watson, D., & Clark, L. A. (1998). Comorbidity of anxiety and unipolar mood disorders. Annual Review of Psychology, 49, 377–412. http://dx.doi.org/10.1146/annurev.psych.49.1.377

Moshagen, M. (2012). The model size effect in SEM: Inflated goodness-of-fit statistics are due to the size of the covariance matrix. Structural Equation Modeling: A Multidisciplinary Journal, 19, 86–98. http://dx.doi.org/10.1080/10705511.2012.634724

Mulaik, S. (2007). There is a place for approximate fit in structural equation modelling. Personality and Individual Differences, 42, 883–891. http://dx.doi.org/10.1016/j.paid.2006.10.024

Muthén, L. K., & Muthén, B. O. (1998–2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Author. Naragon-Gainey, K. (2010). Meta-analysis of the relations of anxiety sensitivity to the depressive

and anxiety disorders. Psychological Bulletin, 136, 128–150. http://dx.doi.org/10.1037/a0018055 Norton, P. J., & Mehta, P. D. (2007). Hierarchical model of vulnerabilities for emotional disorders.

Cognitive Behaviour Therapy, 36, 240–254. http://dx.doi.org/10.1080/16506070701628065 Norton, P. J., Sexton, K. A., Walker, J. R., & Norton, G. R. (2005). Hierarchical model of vulnerabilities

for anxiety: Replication and extension with a clinical sample. Cognitive Behaviour Therapy, 34, 50–63. http://dx.doi.org/10.1080/16506070410005401

Olatunji, B. O., Cisler, J. M., & Tolin, D. F. (2007). Quality of life in the anxiety disorders: A meta- analytic review. Clinical Psychology Review, 27, 572–581. http://dx.doi.org/10.1016/j.cpr.2007.01.015

Paolacci, G., & Chandler, J. (2014). Inside the Turk: Understanding mechanical turk as a participant pool. Current Directions in Psychological Science, 23, 184–188. http://dx.doi. org/10.1177/0963721414531598

Paulus, D. J., Talkovsky, A. M., Heggeness, L. F., & Norton, P. J. (2015). Beyond negative affectivity: A hierarchical model of global and transdiagnostic vulnerabilities for emotional disorders. Cognitive Behavior Therapy, 44, 389–405. http://dx.doi.org/10.1080/16506073.2015.1017529

Rapee, R. M., & Heimberg, R. G. (1997). A cognitive-behavioral model of anxiety in social phobia. Behaviour Research and Therapy, 35, 741–756. http://dx.doi.org/10.1016/S0005-7967(97)00022-3

Raykov, T. (1997). Estimation of composite reliability for congeneric measures. Applied Psychological Measurement, 21, 173–184. http://dx.doi.org/10.1177/01466216970212006

238 N. P. AllAN ET Al.

Reiss, S., & McNally, R. J. (1985). Expectancy model of fear. In S. Reiss, & R. Bootzin (Eds.), Theoretical Issues in Behavior Therapy (pp. 107–121). San Diego, CA: Academic Press.

Reiss, S., Peterson, R. A., Gursky, D. M., & McNally, R. J. (1986). Anxiety sensitivity, anxiety frequency and the prediction of fearfulness. Behaviour Research and Therapy, 24, 1–8. http:// dx.doi.org/10.1016/0005-7967(86)90143-9

Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17, 354–373. http://dx.doi.org/10.1037/a0029315

Rodebaugh, T. L., Heimberg, R. G., Brown, P. J., Fernandez, K. C., Blanco, C., Schneier, F. R., & Liebowitz, M. R. (2011). More reasons to be straightforward: Findings and norms for two scales relevant to social anxiety. Journal of Anxiety Disorders, 25, 623–630. http://dx.doi.org/10.1016/j. janxdis.2011.02.002

Rodebaugh, T. L., Holaway, R. M., & Heimberg, R. G. (2004). The treatment of social anxiety disorder. Clinical Psychology Review, 24, 883–908. http://dx.doi.org/10.1016/j.cpr.2004.07.007

Rodebaugh, T. L., Woods, C. M., & Heimberg, R. G. (2007). The reverse of social anxiety is not always the opposite: The reverse-scored items of the social interaction anxiety scale do not belong. Behavior Therapy, 38, 192–206. http://dx.doi.org/10.1016/j.beth.2006.08.001

Rodriguez, B. F., Bruce, S. E., Pagano, M. E., Spencer, M. A., & Keller, M. B. (2004). Factor structure and stability of the anxiety sensitivity index in a longitudinal study of anxiety disorder patients. Behaviour Research and Therapy, 42, 79–91. http://dx.doi.org/10.1016/S0005-7967(03)00074-3

Schmitt, N., & Kuljanin, G. (2008). Measurement invariance: Review of practice and implications. Human Resource Management Review, 18, 210–222. http://dx.doi.org/10.1016/j.hrmr.2008.03.003

Sexton, K. A., Norton, P. J., Walker, J. R., & Norton, G. R. (2003). Hierarchical model of generalized and specific vulnerabilities in anxiety. Cognitive Behaviour Therapy, 32, 82–94. http://dx.doi. org/10.1080/16506070302321

Shapiro, D. N., Chandler, J., & Mueller, P. A. (2013). Using mechanical turk to study clinical populations. Clinical Psychological Science, 1, 213–220. http://dx.doi.org/10.1177/2167702612469015

Stark, S., Chernyshenko, O. S., & Drasgow, F. (2006). Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy. Journal of Applied Psychology, 91, 1292–1306. http://dx.doi.org/10.1037/0021-9010.91.6.1292

Steenkamp, J., & Baumgartner, H. (1998). Assessing measurement invariance in cross‐national consumer research. Journal of Consumer Research, 25, 78–107. http://dx.doi.org/10.1086/ jcr.1998.25.issue-1

Taylor, S., Zvolensky, M. J., Cox, B. J., Deacon, B., Heimberg, R. G., Ledley, D. R., … Cardenas, S. J. (2007). Robust dimensions of anxiety sensitivity: Development and initial validation of the Anxiety Sensitivity Index-3. Psychological Assessment, 19, 176–188. http://dx.doi.org/10.1037/1040- 3590.19.2.176

Turk, C. L., Heimberg, R. G., Orsillo, S. M., Holt, C. S., Gitow, A., Street, L. L., … Liebowitz, M. R. (1998). An investigation of gender differences in social phobia. Journal of Anxiety Disorders, 12, 209–223. http://dx.doi.org/10.1057/palgrave.rm.8250002

Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54, 1063–1070. http://dx.doi.org/10.1037/0022-3514.54.6.1063

Weeks, J. W., Heimberg, R. G., Fresco, D. M., Hart, T. A., Turk, C. L., Schneier, F. R., & Liebowitz, M. R. (2005). Empirical validation and psychometric evaluation of the brief fear of negative evaluation scale in patients with social anxiety disorder. Psychological Assessment, 17, 179–190. http://dx.doi. org/10.1037/1040-3590.17.2.179

Xu, Y., Schneier, F., Heimberg, R. G., Princisvalle, K., Liebowitz, M. R., Wang, S., & Blanco, C. (2012). Gender differences in social anxiety disorder: Results from the national epidemiologic sample on alcohol and related conditions. Journal of Anxiety Disorders, 26, 12–19. http://dx.doi.org/10.1016/j. janxdis.2011.08.006

Zinbarg, R. E., Barlow, D. H., & Brown, T. A. (1997). Hierarchical structure and general factor saturation of the anxiety sensitivity index: Evidence and implications. Psychological Assessment, 9, 277–284. http://dx.doi.org/10.1037//1040-3590.9.3.277

Copyright of Cognitive Behaviour Therapy is the property of Routledge and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.