W2-DQ

Contents lists available at ScienceDirect

Personality and Individual Differences

journal homepage: www.elsevier.com/locate/paid

Self- and other-focused emotional intelligence: Development and validation of the Rotterdam Emotional Intelligence Scale (REIS)

Keri A. Pekaar⁎, Arnold B. Bakker, Dimitri van der Linden, &Marise Ph. Born Center of Excellence for Positive Organizational Psychology, Erasmus University Rotterdam, The Netherlands

A R T I C L E I N F O

Keywords: Emotions Emotional intelligence Scale development

A B S T R A C T

The present study aimed to develop an instrument to measure emotional intelligence (EI). This novel scale distinguishes between four factors, namely, self- and other-focused emotion appraisal and emotion regulation. In Study 1, the Rotterdam Emotional Intelligence Scale (REIS) was developed and examined with respect to its factorial structure and reliability (N = 383). In Study 2, the factorial structure of the REIS was validated in two new samples (N = 2728 and N = 590). Study 3 examined convergent and discriminant validity by comparing the REIS dimensions with other EI instruments, cognitive intelligence, and personality (N= 108 and N = 105). The criterion validity of the REIS was examined in Study 4 (N = 73, N= 95, and N = 103). The results indicate that the REIS follows a four-factorial structure and can be reliably measured with 28 items. The REIS was strongly correlated with other self-reported EI instruments and weakly to moderately correlated with an ability EI test, cognitive intelligence, and personality. Moreover, self-focused emotion regulation was negatively as- sociated with tutors' perceived stress, whereas other-focused emotion regulation was positively associated with tutors' work engagement, jobseekers' other-rated interview performance, and leaders' transformational leader- ship style.

Scientific interest in the role of emotional intelligence (EI) in dif- ferent life domains is flourishing (Joseph & Newman, 2010; Martins, Ramalho, &Morin, 2010). EI can be broadly defined as the knowledge and/or competencies to effectively deal with emotions to regulate so- cial and emotional behaviors (Petrides, 2011; Salovey &Mayer, 1990; Zeidner, Roberts, &Matthews, 2008). In previous studies, EI has been associated with both intrapersonal (i.e., health) and interpersonal (i.e., being social) benefits. Specifically, EI was positively associated with mental and physical health, work performance, and the quality of social interactions (Joseph &Newman, 2010; Lopes et al., 2004; Martins et al., 2010). As the field is moving forward, researchers are becoming in- terested in the processes that underlie the positive effects of EI (e.g., Lievens & Chan, 2010). Accordingly, an important question is whether dealing with one's own emotions or the emotions of other individuals are of equal importance for the prediction of criteria (Brasseur, Grégoire, Bourdu, &Mikolajczak, 2013; Zeidner et al., 2008). We pro- pose that both EI dimensions (i.e., dealing with one's own emotions and dealing with others' emotions) may have a positive impact; however, this impact may occur in different life domains. To illustrate, effectively dealing with the emotions of the self presumably plays a major role in staying (mentally and physically) healthy, whereas effectively dealing with the emotions of others may be more important to facilitate smooth

social interactions. As the positive effects of EI may thus reflect dif- ferent processes, it may be relevant to differentiate self- from other- focused EI.

The rise of EI to a prominent research topic has stimulated the de- velopment of various EI instruments. Although there has been sub- stantial debate on the format of these instruments (i.e., ability tests or self-reported questionnaires; Roberts, Matthews, & Zeidner, 2010), to date, the question of whether they should involve both self- and other- focused EI dimensions has received relatively little attention. Accord- ingly, most EI instruments do not explicitly distinguish self- from other- focused EI. Therefore, it remains largely unclear which EI dimension contributes to which criterion. We consider this a limitation in the field because self-focused EI dimensions may not always reconcile with their other-focused counterparts (Niven, Totterdell, Stride, & Holman, 2011) and may have differential effects. In the related, yet somewhat separate, research field of emotion regulation, the distinction between dealing with one's own emotions or the emotions of others is well acknowl- edged. Instruments have been developed that measure both self and other-focused emotion regulation (e.g., Emotion Regulation of Others and Self Scale; Niven et al., 2011) or one of these factors (e.g., Mana- ging the Emotions of Others Scale; Austin & O'Donnell, 2013). By combining these measures with EI measures, scholars have attempted

http://dx.doi.org/10.1016/j.paid.2017.08.045 Received 13 July 2017; Received in revised form 29 August 2017; Accepted 30 August 2017

⁎ Corresponding author at: Erasmus University Rotterdam, Center of Excellence for Positive Organizational Psychology, P.O. Box 1738, 3000 DR Rotterdam, The Netherlands. E-mail address: [email protected] (K.A. Pekaar).

Personality and Individual Differences 120 (2018) 222–233

Available online 09 September 2017 0191-8869/ © 2017 Elsevier Ltd. All rights reserved.

MARK

to balance the focus on the ways individuals deal with self- and other- emotions (Austin, Saklofske, Smith, & Tohver, 2014). In a first attempt to develop an instrument that distinguishes self- from other-focused EI, the Profile of Emotional Competence (PEC) was developed (Brasseur et al., 2013). Although the theoretical approach of the PEC is pro- mising, its distinction in ten highly correlated facets did not enable a meaningful differentiation between self- and other-focused EI. Thus, as the facets of the PEC are relatively narrow and fine-grained, it remains difficult to disentangle which facet is responsible for a specific effect. We therefore argue for a more parsimonious alternative. Consequently, the major aim of the current paper is to develop and validate a short and simple scale to explicitly measure self- and other-focused EI. We believe that this type of scale is vital in unraveling the processes that underlie EI.

1. Theoretical background

Although the EI literature is abundant, there is no consensus re- garding the definition and measurement of the construct. Efforts con- tinue to refine the models and measurements of EI (Keefer, 2015). The two major and overarching perspectives are the ability- and trait-po- sitions of EI (Siegling, Saklofske, & Petrides, 2015). The ability-position defines EI as a set of emotion-related abilities akin to cognitive abilities (Salovey &Mayer, 1990; Zeidner et al., 2008). By contrast, the trait- position defines EI as a set of emotion-related traits more akin to per- sonality (Petrides, Pita, & Kokkinaki, 2007). At the core of the debate between these two positions lies the way in which EI is measured, i.e., with an ability test similar to the way cognitive intelligence is measured or a self-reported instrument that resembles the way personality is measured. The current research follows this latter tradition by con- structing a self-reported instrument to examine self- and other-focused EI. Self-reported EI instruments appear more straightforward for a construct that addresses subjective emotional experiences than ability EI tests (Siegling et al., 2015). Furthermore, self-reported EI instru- ments have demonstrated superior explanatory power over cognitive intelligence and personality in predicting criteria such as job perfor- mance (O'Boyle, Humphrey, Pollack, Hawver, & Story, 2011).

1.1. Self- and other-focused emotional intelligence

The introduction of EI in the scientific literature was partially based on the work of Gardner (1983), who differentiated the concept of in- telligence in multiple dimensions. Specifically, Gardner proposed that the emotional aspect of intelligence consists of two dimensions: in- trapersonal and interpersonal intelligence. Accordingly, Salovey and Mayer (1990) distinguished emotion appraisal in the self from emotion appraisal in others, as well as emotion regulation in the self from emotion regulation in others. However, in their Four-Branch Model, they revised this previous definition and added the components of emotion use and emotion understanding to their conceptualization. Although this resulted in a richer pallet of EI dimensions, the distinction between self- and other-focused EI dimensions was pushed into the background because “each branch applied to emotions internally and in others” (Mayer & Salovey, 1997, p. 10). The Four-Branch Model became an influential model in the literature, and whether one's capacity to deal with one's own emotions can be considered to be similar to one's capacity to deal with the emotions of others is still a conceptual issue (Brasseur et al., 2013; Zeidner et al., 2008). Furthermore, merging self- and other-focused EI dimensions may mask their unique effects. In an attempt to address these theoretical issues, we argue to reinstate the explicit and meaningful distinction between self- and other-focused EI.

To illustrate, some individuals are more competent in the regulation of their own emotions than in the regulation of the emotions of others (Niven et al., 2011). This finding implies that when the source of emotions is not specified in EI instruments, incorrect conclusions may be drawn. Furthermore, self- and other-focused EI dimensions may not

always reconcile. Psychotherapists who are overly involved with their clients' emotions are at risk for burnout because they may take their clients' difficulties home (Lee, Lim, Yang, & Lee, 2011). Thus, compe- tence in other-focused EI may, in some contexts, mean being in- competent in self-focused EI and vice versa. Based on the above men- tioned reasons, positive associations of EI with health criteria (Martins et al., 2010) may be reflective of self-focused EI because this directly addresses one's own mood state. By contrast, the positive associations of EI with social criteria (Joseph &Newman, 2010; Lopes et al., 2004) may be more reflective of other-focused EI because this directly addresses the mood state of other individuals.

1.2. Emotion appraisal and emotion regulation

As we aim for a short and practical instrument to reliably differ- entiate between emotional processes, EI will be captured by two main dimensions that are theoretically relevant and consistently appear in every conceptual model of EI, namely, emotion appraisal and emotion regulation (e.g., Davies, Stankov, & Roberts, 1998; Mayer & Salovey, 1997; Petrides et al., 2007). We argue that emotion appraisal and regulation play crucial roles in the way individuals deal with emotions. In the first part of the process, emotion appraisal may draw one's at- tention to the emotion without altering its impact. In the second part of the process, the emotion is regulated to facilitate mood or social in- teraction. Thus, one could infer that emotion appraisal functions as a precondition for emotion regulation (cf. Joseph &Newman, 2010); however, emotion appraisal does not always have to result in emotion regulation. Based on an individual's capacity, motivation, and the context, different reactions might follow.

EI models and instruments vary considerably in the precise com- position of the EI dimensions included (Siegling et al., 2015). However, the different interpretations of the construct complement rather than contradict each other (Ciarrochi, Chan, & Caputi, 2000). Moreover, the distinction between emotion appraisal and emotion regulation maps well onto the distinction between emotion generation and emotion regulation in the basic emotion regulation literature (Gross, Sheppes, & Urry, 2011), which suggests that it might function as an appealing framework for conceptualizing the process of dealing with emotions.

1.3. The present studies

The aim of the present studies was to develop and validate a self- reported EI instrument that captures emotion appraisal and emotion regulation. When combining these EI dimensions with a focus on either the self or the other, four dimensions emerged. We suggest that this simple yet intuitive distinction can help gain additional insights into emotional processes. Although several validated instruments that dis- tinguish self- from other-focused EI dimensions have previously been developed, these tests have their limitations. They lack an explicit other-focused emotion regulation dimension (Wong and Law Emotional Intelligence Scale (WLEIS): Wong & Law, 2002) or their items and subscales can empirically and statistically only be differentiated in two defendable factors (PEC; Brasseur et al., 2013). In developing a scale that is balanced in its focus on self- and other-emotions and that com- prises the two main dimensions of EI, we aim to facilitate empirical research on the working mechanisms that underlie the manifestation of EI.

2. Study 1: scale development and factorial structure

In study 1, the factorial validity of a new scale was examined to measure self- and other-focused EI: the Rotterdam Emotional Intelligence Scale (REIS). In line with its theoretical background, the hypothesis was that the REIS follows a four-factorial structure that consists of self-focused emotion appraisal, other-focused emotion

K.A. Pekaar et al. Personality and Individual Differences 120 (2018) 222–233

223

appraisal, self-focused emotion regulation, and other-focused emotion regulation (Hypothesis 1).

2.1. Methods

2.1.1. Procedure and participants Data were obtained using a convenience sample of Dutch employees

who were invited to participate in the study. Emails were sent via social media and professional network sites.1 The emails included a link to the online questionnaire with the newly developed EI items. For their participation, employees could win a cinema voucher.

Three hundred eighty-three employees participated in the study, including 129 males (33.7%). The mean age was 39.84 (SD= 13.96) years, and the majority had finished higher vocational education (44.1%) or held an advanced degree (46.5%). Most participants worked in education (21.7%), healthcare (18.5%), the research and develop- ment industry (9.7%), the marketing and communication sector (9.4%), or business management (9.4%). On average, the participants worked 34.32 (SD = 11.05) hours per week, and 53.0% worked fulltime (> 36 h a week).

2.1.2. Construction of the REIS Together with two PhD students who study emotion-related topics,

the authors constructed a pool of 63 candidate items to capture the four proposed dimensions. The contributors were provided with general definitions of the EI dimensions, and they were asked to come up with understandable, concrete, self-referent, neutral, and unambiguous items to measure them (Angleitner, John, & Lohr, 1986). Specifically,

the definitions used in the construction of the emotion appraisal di- mensions of the WLEIS (Davies et al., 1998; Wong & Law, 2002) were used for the emotion appraisal dimensions of the REIS:

• Self-focused emotion appraisal: The extent to which individuals per- ceive and understand their own emotions.

• Other-focused emotion appraisal: The extent to which individuals perceive and understand other individuals' emotions.

To construct items for the self- and other-focused emotion regula- tion dimensions, definitions were formulated that could involve both affect-improving and affect-worsening strategies depending on an in- dividual's regulatory goal (cf. Niven et al., 2011). We intentionally avoided the inclusion of the direction of emotion or the motivation that underlies emotion regulation efforts because it has been shown that EI may facilitate social and antisocial behavior depending on individuals' interests (Côté, DeCelles, McCarthy, Van Kleef, & Hideg, 2011). Thus, the definitions emphasized that emotions are regulated to attain (social) behavioral goals (Petrides, 2011; Salovey &Mayer, 1990; Zeidner et al., 2008):

• Self-focused emotion regulation: The extent to which individuals regulate their own emotions to reach a goal.

• Other-focused emotion regulation: The extent to which individuals regulate other individuals' emotions to reach a goal.

The total item pool was initially reviewed in terms of the clarity and fit with the proposed dimensions. All authors and collaborating experts indicated the ten candidate items that were most reflective of each dimension. Following a comparison of these ratings and extensive dis- cussions between the authors, 27 items were excluded. The excluded

Table 1 Items, means, standard deviations, internal consistencies, and factor loadings of the REIS in study 1 (N = 383).

Factor

Item wording M SD α 1 2 3 4

Self-focused emotion appraisal 0.82 1 I always know how I feel. 3.51 0.93 0.74 2 I can distinguish my own emotions well. 3.71 0.75 0.68 3 I am aware of my own emotions. 3.97 0.62 0.67 4 I understand why I feel the way I feel. 3.71 0.72 0.60 5 I know which emotions I experience. 4.10 0.70 0.58 6 Mostly, I am able to explain exactly how I feel. 3.55 0.89 0.57 7 I can judge well if events touch me emotionally. 3.87 0.74 0.47

Other-focused emotion appraisal 0.85 8 I am aware of the emotions of the people around me. 3.90 0.62 0.75 9 I know which feelings others experience. 3.55 0.73 0.70 10 When I look at other people, I can see how they feel. 3.66 0.64 0.70 11 I can empathize with the people around me. 4.04 0.66 0.64 12 I understand why other people feel the way they feel. 3.63 0.80 0.61 13 I can distinguish well between other people's emotions. 3.84 0.70 0.61 14 I can judge well if events touch others emotionally. 3.66 0.64 0.47

Self-focused emotion regulation 0.80 15 I am in control of my own emotions. 3.36 0.86 0.73 16 I can suppress my emotions easily. 3.03 1.00 0.73 17 I do not let my emotions take over. 3.47 0.92 0.72 18 I only show my emotions when it is appropriate. 3.26 1.00 0.68 19 Even when I am angry, I can stay calm. 3.39 0.98 0.54 20 If I want to, I put on my poker face. 3.24 1.10 0.50 21 I adjust my emotions when necessary. 3.54 0.82 0.33

Other-focused emotion regulation 0.82 22 I can make someone else feel differently. 3.48 0.69 0.82 23 I can alter another person's emotional state. 3.25 0.70 0.80 24 I can boost or temper the emotions of others. 3.64 0.68 0.60 25 I have great influence on how others feel. 3.03 0.78 0.58 26 I know what to do to improve people's mood. 3.64 0.63 0.56 27 I know how to influence people. 3.73 0.72 0.42 28 I am able to calm others down. 3.93 0.53 0.37

Notes. Factor loadings > 0.32 are shown. Items were translated in English.

1 Informed consent was obtained for all participants in the current studies.

K.A. Pekaar et al. Personality and Individual Differences 120 (2018) 222–233

224

items were ambiguous, too similar to the other items, or referred to specific emotions (vs. no specific emotions). Specifically, to avoid biased responses caused by individual differences in emotional re- sponsivity to specific emotions (Gray, 1987), we decided to delete items that referred to specific emotions. We subsequently examined whether the 36 retained candidate items followed the four proposed dimensions. To this end, the participants were instructed to indicate the extent to which they agreed with each item on a 5-point Likert scale that ranged from 1 (totally disagree) to 5 (totally agree).

2.2. Results

To explore the factorial structure of the REIS, factor analysis (maximum likelihood) with oblique rotation in SPSS was used. As a criterion, factors with eigenvalues> 1 were retained. When a factor included fewer than three items, this factor (and its items) was deleted (cf. Costello & Osborne, 2005). Within the extracted factors, items that loaded at least 0.32 on the intended factor were retained (Tabachnick & Fidell, 2001). We thus excluded items that had cross loadings> 0.32 or that did not load at least 0.32 on a factor. Following these criteria, we deleted four items in a first factor analysis and an additional four items in a second and third factor analysis, until all criteria were met. This iterative process resulted in 28 items loading on four factors that were identical to the hypothesized dimensions in hy- pothesis 1 (Table 1).

The four factors, which consisted of seven items each, explained a cumulative 43.3% of the variance in the data. Specifically, the first factor consisted of other-focused emotion appraisal (Eigenvalue = 6.53) and explained 21.3% of the variance. The second factor, self-focused emotion regulation (Eigenvalue = 2.93), explained 8.7%. The third factor, other-focused emotion regulation (Eigenvalue = 2.89), explained 8.3%. The fourth and final factor, self- focused emotion appraisal (Eigenvalue = 1.93), explained an addi- tional 5.0% of the variance. The internal consistencies (alphas) of all dimensions were satisfactory (Table 1), and the intercorrelations ranged between r= 0.19 and r = 0.45 (Table 2).

2.3. Discussion

This first study provided initial support for the four proposed di- mensions of the REIS. Good reliabilities and weak to moderate inter- correlations between the subscales were identified, which suggest that

the subscales appear to reliably capture different EI dimensions. To examine whether the proposed structure of the REIS is independent of the sample used, the subsequent step was to cross-validate the findings.

3. Study 2: cross-validation

The aim of study 2 was to examine the four-factorial structure of the REIS in new samples using confirmatory factor analysis. A four-factor model, including a higher order EI factor, was predicted to fit the data best compared with alternative models (Hypothesis 2). More specifi- cally, this hierarchical four-factor model was tested against a hier- archical three-factor model that is comparable to the WLEIS (Wong & Law, 2002). That is, a higher order EI factor that is dis- tinguished in self- and other-focused emotion appraisal and a general emotion regulation factor. In addition, a hierarchical two-factor model with a higher order EI factor and two lower order factors that re- presented self- and other-focused EI was tested. Alternatively, we ex- amined a hierarchical two-factor model with a higher order EI factor and two lower order factors that represented emotion appraisal and emotion regulation. Moreover, we determined how the data fit to a one- factor model in which all items loaded on one general EI factor. Finally, we examined the robustness of the REIS across employees and students, gender, and age groups using invariance tests.

3.1. Methods

3.1.1. Procedure and participants The samples of study 2 were convenience samples that consisted of

Dutch employees (sample 1) and students (sample 2). To recruit par- ticipants, a link to the online questionnaire was distributed via a pop- ular scientific website (i.e., Quest) that provides personality and other intellectual tests. Participation was voluntary, and participants received immediate feedback on their score. Participants without a job or younger than 18 were excluded.

Sample 1 included 2728 employees, including 900 males (33.0%). The mean age was 36.60 (SD= 12.36) years. Most participants had completed vocational education (31.5%), higher vocational education (40.1%), or held an advanced degree (19.6%). All types of professions were represented in the sample, with a majority working in healthcare (29.8%), education (11.4%), marketing and communication (11.1%), and the industrial sector (8.3%). In total, 36.1% of the participants worked fulltime (> 36 h per week), whereas the majority worked

Table 2 Items, means, standard deviations, internal consistencies (between brackets), and correlations of the REIS dimensions in studies 1 and 2.

Study M SD 1 2 3 4 5

1 Self-focused emotion appraisal 1 3.77 0.54 (0.82) 2a 3.89 0.59 (0.81) 2b 3.75 0.68 (0.84)

2 Other-focused emotion appraisal 1 3.76 0.50 0.45⁎⁎⁎ (0.85) 2a 3.94 0.56 0.50⁎⁎⁎ (0.86) 2b 3.96 0.58 0.48⁎⁎⁎ (0.86)

3 Self-focused emotion regulation 1 3.33 0.65 0.19⁎⁎⁎ 0.21⁎⁎⁎ (0.80) 2a 3.41 0.68 0.21⁎⁎⁎ 0.15⁎⁎⁎ (0.79) 2b 3.52 0.68 0.14⁎⁎ 0.10⁎ (0.79)

4 Other-focused emotion regulation 1 3.53 0.47 0.23⁎⁎⁎ 0.37⁎⁎⁎ 0.20⁎⁎⁎ (0.82) 2a 3.74 0.52 0.38⁎⁎⁎ 0.54⁎⁎⁎ 0.27⁎⁎⁎ (0.84) 2b 3.71 0.55 0.37⁎⁎⁎ 0.58⁎⁎⁎ 0.21⁎⁎⁎ (0.84)

5 Total REIS score 1 3.60 0.36 0.69⁎⁎⁎ 0.73⁎⁎⁎ 0.66⁎⁎⁎ 0.63⁎⁎⁎ (0.86) 2a 3.74 0.41 0.73⁎⁎⁎ 0.75⁎⁎⁎ 0.62⁎⁎⁎ 0.74⁎⁎⁎ (0.88) 2b 3.74 0.43 0.73⁎⁎⁎ 0.76⁎⁎⁎ 0.55⁎⁎⁎ 0.75⁎⁎⁎ (0.88)

Notes. Study 2a refers to Sample 1: N = 2728; Study 2b refers to sample 2: N = 590. ⁎ p < 0.05. ⁎⁎ p < 0.01. ⁎⁎⁎ p < 0.001.

K.A. Pekaar et al. Personality and Individual Differences 120 (2018) 222–233

225

between 17 and 36 h per week (52.4%). With the exception of a larger proportion of women, Sample 1 is comparable to the general Dutch working population (CBS, 2016). Sample 2 consisted of 590 students, including 191 males (32.4%). The mean age was 21.43 (SD= 3.70) years. Most participants were attending higher vocational education (30.0%) or pursuing their Bachelor's degree (28.8%).

3.1.2. Measures Self- and other-focused EI was measured with the 28 REIS items.

3.2. Results

3.2.1. Confirmatory factor analysis Table 2 presents the means, standard deviations, internal con-

sistencies (alphas), and correlations of the REIS dimensions in both samples. Confirmatory factor analysis was used to determine whether a hierarchical four-factor solution fitted the total data set best compared with alternative models (hypothesis 2) using AMOS. The fit of the proposed models was assessed with five indices: the comparative fit index (CFI), the Tucker-Lewis index (TLI), the incremental fit index (IFI), the root mean squared error of approximation (RMSEA), and the standardized root mean squared residual (SRMR). The fit indices were interpreted using Hu and Bentler's (1999) suggested values, which

should be close to 0.95 for CFI, TLI, and IFI, close to 0.06 for RMSEA, or close to 0.08 for SRMR.

The results of the confirmatory factor analysis are reported in Table 3. The CFI, TLI, and IFI indices of the hierarchical four-factor model were all 0.91 and the RMSEA and SRMR were small (0.05), which indicates that this model showed an acceptable fit to the data. All items significantly loaded on their proposed latent factors (coefficients ranged between 0.48 and 0.77, all p's > 0.001). Supporting hypothesis 2, the fit of the proposed hierarchical four-factor model to the data was significantly and substantially better compared with a hierarchical three-factor model (Δχ2 = 4373.93, Δdf= 1, p < 0.001), a hier- archical two-factor model with two lower order factors that represented emotion appraisal and emotion regulation (Δχ2 = 7639.67, Δdf= 3, p < 0.001), and a one-factor model (Δχ2 = 10,842.42, Δdf= 4, p < 0.001). Furthermore, the analyses showed that fitting the data to the alternative hierarchical two-factor model with a general EI factor and two lower order factors that represented self- or other-focused items produced several Heywood cases as a result of negative variances, which indicates the inappropriateness of this alternative. Fig. 1 displays the hierarchical four-factor model.

3.2.2. Invariance tests The invariance of the REIS across employees and students was

Table 3 Confirmatory factor analysis and invariance tests of the REIS in study 2 (N = 3318).

Model χ2 df CFI TLI IFI RMSEA SRMR

Four-factor model 3178.68 346 0.91 0.91 0.91 0.05 0.05 Three-factor model 7552.61 347 0.78 0.76 0.78 0.08 0.08

Two-factor model (self-focused - other-focused)a

Two-factor model (emotion appraisal - emotion regulation) 10818.35 349 0.68 0.65 0.68 0.10 0.09 One-factor model 14021.10 350 0.58 0.55 0.58 0.11 0.11

Invariance test among students (N = 590) and employees (N = 2728) Model 1 (four-factor model - unconstrained) 3620.02 692 0.91 0.91 0.91 0.04 0.06 Model 2 (four-factor model - factor loadings constrained) 3644.70 716 0.91 0.91 0.91 0.04 0.06

Invariance test among men (N = 1091) and women (N = 2227) Model 3 (four-factor model - unconstrained) 3443.48 692 0.91 0.91 0.91 0.04 0.06 Model 4 (four-factor model - factor loadings constrained) 3478.21 716 0.91 0.91 0.91 0.03 0.06

Invariance test among young (N = 1121), middle (N = 1200) and older (N = 984) adults

Model 5 (four-factor model - unconstrained) 4240.49 1098 0.90 0.90 0.90 0.03 0.06 Model 6 (four-factor model - factor loadings constrained) 4278.16 1122 0.90 0.90 0.90 0.03 0.06

Notes. CFI = comparative fit index; TLI = Tucker-Lewis index; IFI = incremental fit index; RMSEA = root mean squared error of the mean; SRMR = standardized root mean squared residual.

a As a result of several Heywood cases, this model did not lead to a permissible solution.

Fig. 1. Hierarchical four-factor model used in the confirmatory factor analysis of study 2.

K.A. Pekaar et al. Personality and Individual Differences 120 (2018) 222–233

226

tested using a multi-group analysis in AMOS (Byrne, 2004). Specifi- cally, we initially ran a model (model 1 in Table 3) in which all para- meters were simultaneously estimated without cross-group constraints. We subsequently ran a model in which we constrained the factor loadings (model 2) and compared the fit with the unconstrained model. This comparison produced a non-significant chi-square difference test value (Δχ2 = 24.68, Δdf = 24, p = 0.423), which implied that the factor loadings were invariant across the samples.

A similar procedure was performed to test for invariance across men and women. The student and employee samples were initially merged and subsequently split in terms of gender. We then compared the fit of a model without equality constraints (model 3) with the fit of a model in which we constrained the factor loadings (model 4). This comparison produced a non-significant chi-square difference test value (Δχ2 = 34.73, Δdf= 24, p = 0.073), which implied that the factor loadings were invariant across gender groups.

Finally, we tested for invariance across different age groups. The total data (N = 3318) were split into three age groups (18–25 years; 26–40 years; and> 40 years). We subsequently compared the fit of a model without equality constraints (model 5) with the fit of a model in which we constrained the factor loadings to be equal across the age groups (model 6). This comparison yielded a significant chi-square difference test value (Δχ2 = 37.67, Δdf = 24, p = 0.037), which im- plied that the factor loadings slightly differed between the age groups. Inspection of these loadings indicated that the factor loadings in the younger age group were relatively lower than those in the older age groups. Despite these differences, the model fit values of this con- strained model were acceptable.

3.3. Discussion

The results of study 2 indicated that the proposed hierarchical four- factorial structure showed a substantially better fit to the data than alternative structures in two new samples. Furthermore, the invariance tests indicated that the factor loadings of the REIS were invariant across employees, students, and gender groups, which implies that these dif- ferent groups respond to the items in the same way. The invariance test for age indicated that the factor loadings in the younger age group were relatively lower than in the older age groups (however, they were ac- ceptable in terms of model fit). This finding might be related to the phenomenon that some EI facets become more crystallized among older adults (Doerwald, Scheibe, Zacher, & van Yperen, 2016). Together, the results of Study 2 established the measurement properties of the new scale. Consequently, a logical next step was to further examine the convergent and discriminant validity of the REIS.

4. Study 3: convergent and discriminant validity

Study 3 examined the convergent and discriminant validity of the REIS by relating its dimensions to other EI instruments, cognitive in- telligence, and personality measures. To examine convergent validity in a first sample (study 3a), two different self-reported EI questionnaires were used: the WLEIS (Wong & Law, 2002) and the TEIQue (Petrides, 2009). We expected that the total score of the REIS is strongly and positively correlated with the total scores of the WLEIS and the TEIQue (Hypothesis 3). In addition, it was predicted that three of the four REIS dimensions relate strongly and positively to three comparable WLEIS dimensions. Specifically, the hypothesis was that self-focused emotion appraisal, other-focused emotion appraisal, and self-focused emotion regulation exhibit strong positive correlations with self-emotion ap- praisal, other-emotion appraisal, and regulation of emotions of the WLEIS, respectively (Hypothesis 4). The newly included REIS dimension other-focused emotion regulation was not expected to show a strong relationship with a specific WLEIS dimension. For the TEIQue, there were no specific expectations regarding the dimensional level because the REIS and TEIQue are composed of different EI dimensions.

To examine discriminant validity, the REIS dimensions were related to cognitive intelligence and personality measures. Previous research has indicated that ability EI tests tend to correlate particularly with cognitive intelligence, whereas self-reported EI questionnaires tend to correlate with personality measures (O'Boyle et al., 2011; Van der Linden et al., 2017). As the REIS is a self-reported questionnaire, its dimensions were hypothesized to correlate weakly or non-significantly with cognitive intelligence (Hypothesis 5) and weakly with personality measures (Hypothesis 6).

In a second sample (study 3b), we included an ability test of EI (MSCEIT; Mayer, Salovey, & Caruso, 2002) and another self-reported EI questionnaire (PEC; Brasseur et al., 2013). Self-reported EI ques- tionnaires and ability EI tests tend to correlate weakly to moderately (Joseph &Newman, 2010; Petrides, 2011). Therefore, we expected that the total score of the REIS is weakly to moderately positively correlated with the total score of the MSCEIT (Hypothesis 7). We had no specific expectations regarding the dimensional level because the REIS dimen- sions are differentiated in terms of self- versus other-emotions in con- trast to the MSCEIT branches. Regarding the PEC, we expected the total score of the REIS to exhibit a strong and positive correlation with the total score of the PEC (Hypothesis 8). Furthermore, we expected that the self-focused REIS dimensions exhibit a stronger correlation with the intrapersonal PEC factor than the interpersonal PEC factor, and the other-focused REIS dimensions exhibit a stronger correlation with the interpersonal PEC factor than the intrapersonal PEC factor (Hypothesis 9).

4.1. Study 3a

4.1.1. Methods 4.1.1.1. Procedure and participants. One hundred eight Dutch university students participated in the study in exchange for course credits. Forty students were male (37.0%). The participants were instructed to complete the EI instruments and a personality questionnaire and were subsequently given 10 min to solve as many items as possible of a well- established IQ-test (subsequently described). The mean age of the participants was 21.93 (SD = 2.87) years. The majority of the participants studied psychology (63.9%) or economics (13.9%). In addition to their studies, most participants (69.4%) had a part-time job.

4.1.1.2. Measures. REIS. Self- and other-focused EI was measured with the 28 REIS items.

WLEIS was included as another self-reported EI instrument (Wong & Law, 2002). This 16-item scale measures self-emotion ap- praisal, others-emotion appraisal, use of emotions, and regulation of emotions. A sample item is “I am a self-motivated person” (1 = totally disagree, 5 = totally agree).

TEIQue was used as another self-reported EI instrument. We ad- ministered the 30-item TEIQue-SF (Petrides, 2009), which measures emotionality, sociability, self-control, and wellbeing. A sample item is “Others admire me for being relaxed” (1 = totally disagree, 7 = totally agree).

Personality was measured with a 21-item Dutch version of the Big Five Inventory (Denissen, Geenen, Van Aken, Gosling, & Potter, 2008), which measure openness (α = 0.75), conscientiousness (α = 0.70), extraversion (α= 0.78), agreeableness (α = 0.56), and neuroticism (α= 0.57). A sample item is “I am someone who is depressed” (1 = strongly disagree, 5 = strongly agree).

Because the internal consistencies of agreeableness and neuroticism fell below the recommended cut-off value of 0.70, we identified the items that caused this problem. Deletion of the items “I am someone who is generally trusting” (agreeableness; new α= 0.65) and “I am someone who is relaxed, handles stress well” (neuroticism; new α= 0.80) considerably increased the respective internal consistencies. In the analyses, we thus used the original subscales (4 items each) and the subscales without the problematic items (3 items each; Table 5).

K.A. Pekaar et al. Personality and Individual Differences 120 (2018) 222–233

227

Cognitive intelligence was measured using Raven's Advanced Progressive Matrices (RPM; Raven, 1962). The complete RPM consists of 48 multiple-choice questions of abstract reasoning.

4.1.2. Results Table 4 presents the means, standard deviations, internal con-

sistencies (alphas), and correlations of the REIS, WLEIS, and TEIQue dimensions. Confirming hypothesis 3, the results showed that the total score of the REIS was strongly and positively correlated with the total score of the WLEIS (r = 0.64, p < 0.001) and the total score of the TEIQue (r = 0.58, p < 0.001). On the dimensional level, the results showed that self-focused emotion appraisal, other-focused emotion appraisal, and self-focused emotion regulation were strongly and posi- tively correlated with self-emotion appraisal (r = 0.77, p < 0.001), other-emotion appraisal (r = 0.69, p < 0.001), and regulation of emotions (r = 0.59, p < 0.001) of the WLEIS, respectively. These predicted correlations were significantly larger than the correlations of the respective REIS dimensions with the other WLEIS or TEIQue di- mensions (all Z's > 2.32, all p's < 0.05). Together, these results sup- ported hypothesis 4.

Table 5 presents the correlations of the REIS dimensions with

personality and cognitive intelligence. It was predicted that the REIS dimensions would correlate weakly or non-significantly with cognitive intelligence (hypothesis 5). The results confirmed that only self-focused emotion regulation was moderately correlated with cognitive in- telligence (r = 0.21, p= 0.034), whereas the other dimensions and the total REIS score were unrelated to cognitive intelligence. Furthermore, the REIS was predicted to weakly correlate with the Big Five personality factors (hypothesis 6). The results indicated that conscientiousness (which ranged between r = 0.22 and r= 0.33) and neuroticism (which ranged between r = − 0.31 and r = −0.36) showed moderate corre- lations with several REIS dimensions. However, the majority of the correlations between the REIS and the Big Five personality factors were non-significant, which supports hypothesis 6.

4.2. Study 3b

4.2.1. Methods 4.2.1.1. Procedure and participants. One hundred five Dutch psychology students participated for course credits. The mean age was 19.98 (SD = 2.28) years, and 9.5% of the participants were male. The participants were instructed to complete the MSCEIT before they

Table 4 Means, standard deviations, internal consistencies (between brackets), and correlations of the REIS dimensions and indicators of convergent validity in study 3a (N = 108).

M SD 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

REIS 1 Self-focused emotion

appraisal 3.83 0.58 (0.90)

2 Other-focused emotion appraisal

3.90 0.49 0.19 (0.87)

3 Self-focused emotion regulation

3.66 0.58 0.24⁎ −0.00 (0.72)

4 Other-focused emotion regulation

3.63 0.59 0.31⁎⁎ 0.45⁎⁎⁎ 0.22⁎ (0.89)

5 Total REIS score 3.76 0.37 0.68⁎⁎⁎ 0.59⁎⁎⁎ 0.58⁎⁎⁎ 0.76⁎⁎⁎ (0.87)

WLEIS 6 Self-emotion

appraisal 3.80 0.68 0.77⁎⁎⁎ 0.08 0.15 0.26⁎⁎ 0.50⁎⁎⁎ (0.83)

7 Others-emotion appraisal

3.90 0.59 0.15 0.69⁎⁎⁎ −0.11 0.31⁎⁎ 0.37⁎⁎⁎ 0.13 (0.72)

8 Use of emotions 3.66 0.65 0.12 0.11 0.21⁎ 0.10 0.21⁎ 0.27⁎⁎ −0.01 (0.74) 9 Regulation of

emotions 3.65 0.77 0.42⁎⁎⁎ 0.16 0.59⁎⁎⁎ 0.15 0.51⁎⁎⁎ 0.35⁎⁎⁎ 0.09 0.28⁎⁎ (0.81)

10 Total WLEIS score 3.75 0.43 0.60⁎⁎⁎ 0.39⁎⁎⁎ 0.37⁎⁎⁎ 0.32⁎⁎ 0.64⁎⁎⁎ 0.71⁎⁎⁎ 0.44⁎⁎⁎ 0.61⁎⁎⁎ 0.73⁎⁎⁎ (0.79)

TEIQue 11 Wellbeing 5.56 0.81 0.22⁎ 0.04 0.06 0.14 0.18 0.36⁎⁎⁎ 0.07 0.46⁎⁎⁎ 0.27⁎⁎ 0.47⁎⁎⁎ (0.75) 12 Self-control 4.76 0.92 0.59⁎⁎⁎ 0.09 0.42⁎⁎⁎ 0.28⁎⁎ 0.54⁎⁎⁎ 0.55⁎⁎⁎ −0.04 0.32⁎⁎ 0.57⁎⁎⁎ 0.59⁎⁎⁎ 0.43⁎⁎⁎ (0.70) 13 Emotionality 5.30 0.74 0.59⁎⁎⁎ 0.23⁎ 0.06 0.29⁎⁎ 0.45⁎⁎⁎ 0.57⁎⁎⁎ 0.26⁎⁎ 0.18 0.28⁎⁎ 0.51⁎⁎⁎ 0.29⁎⁎ 0.45⁎⁎⁎ (0.63) 14 Sociability 5.14 0.82 0.40⁎⁎⁎ 0.25⁎⁎ 0.15 0.57⁎⁎⁎ 0.53⁎⁎⁎ 0.30⁎⁎ 0.15 0.25⁎⁎ 0.25⁎ 0.38⁎⁎⁎ 0.31⁎⁎ 0.43⁎⁎⁎ 0.32⁎⁎ (0.68) 15 Total TEIQue score 5.19 0.59 0.61⁎⁎⁎ 0.19⁎ 0.27⁎⁎ 0.41⁎⁎⁎ 0.58⁎⁎⁎ 0.61⁎⁎⁎ 0.14 0.49⁎⁎⁎ 0.49⁎⁎⁎ 0.70⁎⁎⁎ 0.70⁎⁎⁎ 0.80⁎⁎⁎ 0.71⁎⁎⁎ 0.66⁎⁎⁎ (0.85)

⁎ p < 0.05. ⁎⁎ p < 0.01. ⁎⁎⁎ p < 0.001.

Table 5 Correlations of the REIS dimensions with indicators of discriminant validity in study 3a (N = 108).

O C E A (A) N (N) IQ

1 Self-focused emotion appraisal −0.17 0.28⁎⁎ 0.10 0.15 (0.12) −0.35⁎⁎⁎ (−0.39⁎⁎⁎) −0.09 2 Other-focused emotion appraisal 0.14 −0.00 0.11 −0.03 (−0.06) −0.12 (−0.06) −0.05 3 Self-focused emotion regulation 0.21⁎ 0.32⁎⁎ −0.07 −0.10 (0.01) −0.13 (−0.17) 0.21⁎

4 Other-focused emotion regulation 0.12 0.22⁎ 0.32⁎⁎ −0.11 (−0.12) −0.31⁎⁎ (−0.24⁎) 0.15 5 Total REIS score 0.11 0.33⁎⁎ 0.18 −0.03 (−0.02) −0.36⁎⁎⁎ (−0.34⁎⁎⁎) 0.09

Notes. O = openness; C = conscientiousness; E = extraversion; A = agreeableness; N = neuroticism; IQ = cognitive intelligence. Correlations between brackets are based on the subscales agreeableness and neuroticism without the problematic items.

⁎ p < 0.05. ⁎⁎ p < 0.01. ⁎⁎⁎ p < 0.001.

K.A. Pekaar et al. Personality and Individual Differences 120 (2018) 222–233

228

were presented with the REIS and the PEC in a randomized order.

4.2.1.2. Measures. REIS. Self- and other-focused EI was measured with the 28 REIS items.

Ability EI was measured with the Dutch 141-item MSCEIT (Mayer et al., 2002). The MSCEIT is an ability EI test designed to measure the branches perceiving emotions, facilitating thought, understanding emotions, and managing emotions using emotional problems (often in the scenario format) or tasks in which emotions are central.

PEC was used as another self-reported EI instrument (Brasseur et al., 2013). The 50-item PEC consists of ten facets (i.e., identification, expres- sion, comprehension, regulation, and utilization of self- and other-emo- tions) that load on two factors: intrapersonal emotional competence and interpersonal emotional competence. A sample item is “When I am sad, I often don't know why” (1 = strongly disagree, 5 = strongly agree).

4.2.2. Results Table 6 presents the means, standard deviations, internal con-

sistencies, and correlations of the REIS, MSCEIT, and PEC. Noteworthy, although we used the recommended Spearman Brown corrected split- half approach of equivalent forms to estimate the internal consistency of the MSCEIT (Mayer, Salovey, & Caruso, 2012), three of the four branches had a relatively low reliability. Confirming hypothesis 7, the results showed that the total score of the REIS was weakly and posi- tively correlated with the total score of the MSCEIT (r = 0.19, p = 0.049). In particular, other-focused emotion appraisal was asso- ciated with two MSCEIT branches (i.e., perceiving emotions and using emotions). Supporting hypothesis 8, the total score of the REIS was strongly and positively correlated with the total score of the PEC (r = 0.73, p < 0.001). At the dimensional level, self-focused emotion appraisal was more strongly associated with the intrapersonal PEC factor than the interpersonal PEC factor (r = 0.80, p < 0.001, Z = 6.03, p < 0.001). Self-focused emotion regulation was moderately and positively associated with the intrapersonal PEC factor (r = 0.33, p < 0.001); however, this correlation did not differ from its correlation with the interpersonal PEC factor (Z = 1.22, ns). The other-focused REIS dimensions were strongly and positively associated with the in- terpersonal PEC factor (r = 0.56, p < 0.001 and r = 0.65, p < 0.001 for emotion appraisal and emotion regulation, respectively). These correlations were stronger than their correlation with the intrapersonal PEC factor (Z's > 2.79, p's < 0.01). These results partially supported hypothesis 9.

4.3. Discussion studies 3a and 3b

The first goal of study 3 was to examine the convergent validity of the REIS. We determined that the total score of the REIS was strongly and positively associated with the total scores of three other self-re- ported EI questionnaires and weakly and positively associated with the total score of an ability EI test. The convergence with scores on other EI instruments is in accordance with the overlap of EI measures as dis- cussed in the literature (Joseph & Newman, 2010; Petrides, 2011). Furthermore, the individual REIS dimensions were strongly and posi- tively correlated with their designated WLEIS and PEC dimensions (i.e., self- or other-emotions). These findings suggest that the REIS shows adequate convergent validity.

The second goal of study 3 was to examine the discriminant validity of the REIS. The results confirmed that the total REIS score and three of its four dimensions were unrelated to cognitive intelligence. Furthermore, the majority of the correlations between the REIS di- mensions and the Big Five personality factors were non-significant, which confirms their discriminating value. Moreover, the personality factors that moderately correlated with the REIS (conscientiousness and neuroticism) tend to correlate similarly with other self-reported EI in- struments (Law, Wong, & Song, 2004). Thus, it may be concluded that the REIS shows adequate discriminant validity.

5. Study 4: criterion validity

The aim of the final study was to examine the relation of the REIS dimensions with criteria that are expected to be the result of self- and other-focused EI. Following the reasoning that the appraisal of an emotion will mainly draw attention to the presence of an emotion, whereas the regulation of an emotion will change its impact, we ex- pected that mainly self- and other-focused emotion regulation are asso- ciated with external criteria. This idea is in accordance with Joseph and Newman's meta-analysis (2010), in which emotion regulation was considered key to EI's association with job performance. In the present study, we aimed to contribute to their understanding by explicitly in- vestigating the differential criterion validity of self- and other-focused emotion regulation using both health- and work-related criteria.

For this purpose, we initially investigated the work experience of a sample of tutors (i.e., university teachers who guide small groups of students) in study 4a. It was predicted that tutors' self-focused emotion regulation is negatively associated with their perceived stress

Table 6 Means, standard deviations, internal consistencies (between brackets), and correlations of the REIS dimensions and indicators of convergent validity in study 3b (N = 105).

M SD 1 2 3 4 5 6 7 8 9 10 11 12 13

REIS 1. Self-focused emotion appraisal 3.57 0.61 (0.83) 2. Other-focused emotion appraisal 3.93 0.38 0.34⁎⁎⁎ (0.76) 3. Self-focused emotion regulation 3.36 0.65 0.31⁎⁎ 0.11 (0.75) 4. Other-focused emotion regulation 3.55 0.50 0.10 0.33⁎⁎ 0.17 (0.80) 5. Total REIS score 3.61 0.35 0.71⁎⁎⁎ 0.59⁎⁎⁎ 0.69⁎⁎⁎ 0.57⁎⁎⁎ (0.83)

MSCEIT 6. Perceiving emotions 96.73 13.65 0.16 0.26⁎⁎ 0.12 0.11 0.23⁎ (0.89)a

7. Facilitating thought 98.52 12.94 0.04 0.33⁎⁎ −0.02 0.07 0.13 0.64⁎⁎⁎ (0.66)a

8. Understanding emotions 95.58 8.66 −0.03 0.17 −0.04 0.01 0.02 0.31⁎⁎ 0.31⁎⁎ (0.49)a

9. Managing emotions 94.85 7.77 0.13 0.16 −0.05 0.03 0.09 0.44⁎⁎⁎ 0.48⁎⁎⁎ 0.24⁎ (0.58)a

10. Total MSCEIT score 95.88 11.35 0.13 0.33⁎⁎ 0.03 0.08 0.19⁎ 0.86⁎⁎⁎ 0.82⁎⁎⁎ 0.56⁎⁎⁎ 0.69⁎⁎⁎ (0.89)a

PEC 11. Intrapersonal emotional competence 3.42 0.50 0.80⁎⁎⁎ 0.29⁎⁎ 0.33⁎⁎ 0.09 0.61⁎⁎⁎ 0.11 −0.00 −0.01 0.14 0.11 (0.87) 12. Interpersonal emotional competence 3.79 0.36 0.32⁎⁎ 0.56⁎⁎⁎ 0.20⁎ 0.65⁎⁎⁎ 0.62⁎⁎⁎ 0.35⁎⁎⁎ 0.24⁎ 0.14 0.30⁎⁎ 0.36⁎⁎⁎ 0.38⁎⁎⁎ (0.83) 13. Total PEC score 3.60 0.36 0.71⁎⁎⁎ 0.48⁎⁎⁎ 0.33⁎⁎ 0.38⁎⁎⁎ 0.73⁎⁎⁎ 0.25⁎⁎ 0.12 0.06 0.25⁎ 0.26⁎⁎ 0.89⁎⁎⁎ 0.76⁎⁎⁎ (0.89)

a The internal consistencies of the MSCEIT branches are corrected Spearman-Brown split-half estimates of equivalent forms. ⁎ p < 0.05. ⁎⁎ p < 0.01. ⁎⁎⁎ p < 0.001.

K.A. Pekaar et al. Personality and Individual Differences 120 (2018) 222–233

229

(Hypothesis 10), as effectively dealing with one's own emotions reduces stress within the work context (Jordan, Ashkanasy, & Hartel, 2002). Furthermore, other-focused emotion regulation was predicted to be associated with tutors' work engagement based on findings in which the allocation of personal resources (such as other-focused emotion reg- ulation) when needed boosts work engagement (Xanthopoulou, Bakker, & Fischbach, 2013). To illustrate, in the job of a tutor, other- focused emotion regulation may be demanded to effectively guide students through their learning process. Consequently, we hypothesized that tutors with a high level of other-focused emotion regulation be- come work engaged from using this quality (Hypothesis 11).

In addition, we investigated the interview performance of a sample of jobseekers during a selection interview at an employment agency in study 4b. In this context, jobseekers are expected to present themselves positively to convince the employment agent that they are suitable candidates for the available vacancies. Therefore, it was hypothesized that their other-focused emotion regulation determines their interview performance, as rated by the employment agent (Hypothesis 12).

Finally, we examined a sample of leaders to determine whether their leadership style and their leadership effectiveness are associated with their EI. A meta-analysis has shown that high-EI leaders are in- clined to employ a leadership style in which followers are encouraged to learn and achieve, as well as to develop themselves individually (Harms & Credé, 2010). This so-called transformational leadership style (Bass, 1985) had a corrected meta-analytic correlation of ρ= 0.56 (based on same-source data). The corrected meta-analytic correlation with transactional leadership, a style characterized by a focus on re- wards and mistakes, was substantially lower. Furthermore, EI was po- sitively associated with leadership effectiveness (Harms & Credé, 2010). We expected that mainly other-focused emotion regulation is important in the EI‑leadership association. Leaders who can adequately manage the emotions of their followers will translate this knowledge or ability into a leadership style that involves encouragement or emotional sup- port. In turn, this practice will increase their effectiveness as a leader. Thus, we hypothesized that mainly other-focused emotion regulation is positively associated with transformational leadership (Hypothesis 13) and leadership effectiveness (Hypothesis 14).

5.1. Study 4a

5.1.1. Methods 5.1.1.1. Procedure and participants. Seventy-three tutors, including 18 males (24.7%), voluntarily participated in the study. The mean age was 28.87 (SD= 6.60) years. Fifty-three tutors were employed at the Law institute, and 20 tutors were employed at the Psychology institute of a Dutch university. On average, tutors had 16.78 (SD= 14.02) months of work experience and worked for approximately 20.43 (SD= 5.85) h per week as a tutor. A majority of the participants (64.4%) combined their work as a tutor with another part-time job. The tutors were instructed to complete an online questionnaire that assessed EI, perceived stress, and work engagement.

5.1.1.2. Measures. Self- and other-focused EI was measured with the 28 REIS items.

Perceived stress was measured with 13 items of the subscales fatigue, worries, and tension of the Perceived Stress Questionnaire (Levenstein et al., 1993). We adjusted the timeframe of the items so that they were reflective of the previous months. A sample item is “During the past months, I felt tired” (1 = never, 7 = always).

Work engagement was measured with the 9-item Utrecht Work Engagement Scale (Schaufeli, Bakker, & Salanova, 2006). We adjusted the timeframe of the items so that they were reflective of the previous months. A sample item is “During the past months, I was enthusiastic about my job” (1 = never, 7 = always).

5.1.2. Results Table 7 presents the means, standard deviations, internal con-

sistencies (alphas), and correlations between the study variables. Con- firming hypothesis 10, the correlations showed that self-focused emo- tion regulation was the only REIS dimension that exhibited a significant and negative correlation with perceived stress (r = −0.42, p < 0.001). Other-focused emotion regulation was the only REIS di- mension that showed a significant and positive correlation with work engagement (r = 0.49, p < 0.001). Thus, hypothesis 11 was sup- ported.

5.2. Study 4b

5.2.1. Methods 5.2.1.1. Procedure and participants. Ninety-five Dutch jobseekers, including 42 males (44.2%), participated in the study. The mean age was 31.06 (SD = 8.42) years, and most participants had finished vocational education (44.2%) or higher vocational education (31.6%). The participants completed an EI questionnaire prior to engaging in a selection interview at an employment agency. After this one-hour interview, the respective employment agent assessed the jobseekers' interview performance.

5.2.1.2. Measures. Self- and other-focused EI was measured with the 28 REIS items.

Other-rated interview performance was measured with three items constructed to assess interview performance within a selection inter- view. Specifically, the employment agent was instructed to rate the extent to which the jobseeker was a good presenter of oneself / col- legiate / easy to employ at a company with a school mark (1−10).

5.2.2. Results Table 7 presents the means, standard deviations, internal con-

sistencies (alphas), and correlations between the study variables. Con- firming hypothesis 12, the correlations indicated that other-focused emotion regulation was the only REIS dimension that showed a sig- nificant and positive correlation with other-rated interview perfor- mance (r = 0.23, p = 0.027).

5.3. Study 4c

5.3.1. Methods 5.3.1.1. Procedure and participants. A convenience sample of 103 leaders, including 49 males (47.6%), voluntarily participated in the study. The mean age was 42.93 (SD = 12.21) years, and the majority had completed higher vocational education (36.9%) or held an advanced degree (52.4%). The leaders worked in different sectors, including healthcare (22.3%), education (21.7%), sales (11.7%), and HRM (11.7%). On average, the leaders had 34 (SD= 113) followers and 7.89 (SD = 6.94) years of leadership experience.

5.3.1.2. Measures. Self- and other-focused EI was measured with the 28 REIS items.

Transactional leadership was measured with the 9-item Dutch translation (Stuart, 2005) of the Multifactor Leadership Questionnaire (MLQ; Bass & Avolio, 1990). A sample item is “I direct attention toward failures to meet standards” (1 = never, 5 = always).

Transformational leadership was measured with the 15-item Dutch MLQ (Bass & Avolio, 1990; Stuart, 2005). A sample item is “I display a sense of power and confidence” (1 = never, 5 = always).

Leadership effectiveness was measured with the 4-item effectiveness subscale of the MLQ (Bass & Avolio, 1990). A sample item is “I lead a group that is effective” (1 = never, 5 = always).

5.3.2. Results Table 7 presents the means, standard deviations, internal

K.A. Pekaar et al. Personality and Individual Differences 120 (2018) 222–233

230

consistencies (alphas), and correlations between the study variables. Confirming hypothesis 13, the only REIS dimension that showed a significant and positive correlation with transformational leadership was other-focused emotion regulation (r = 0.33, p < 0.001). In addi- tion, the results indicated that none of the REIS dimensions were as- sociated with transactional leadership or leadership effectiveness. De- spite the lack of a direct effect of EI on leadership effectiveness, an exploratory mediation analysis using the bootstrapping method (MacKinnon, Lockwood, &Williams, 2004) indicated that other-focused emotion regulation was indirectly related to leadership effectiveness through transformational leadership (indirect effect = 0.265, 95% CI = 0.104 to 0.534).

5.4. Discussion studies 4a, 4b, and 4c

In study 4, the associations of the REIS dimensions with different work-related outcomes were examined. In general, the results suggest that emotion regulation (vs. emotion appraisal) is responsible for the lion share of EI's associations with health- and work-related criteria, which is in accordance with previous findings in the literature (Joseph &Newman, 2010). Furthermore, self-focused emotion regula- tion appears important to maintain well-being, whereas other-focused emotion regulation appears important to perform well during a selec- tion interview and engage in transformational leadership. This pattern of findings confirms the proposed differential roles of self- and other- focused EI dimensions.

Study 4a showed that perceived stress and work engagement had moderate correlations with the total REIS score (r =−0.34 and r = 0.29, respectively) and strong correlations with self- or other-fo- cused emotion regulation (r = −0.42 and r= 0.49, respectively). These results not only suggest that self- and other-focused emotion regulation predict different types of criteria but that total EI scores may partially mask these effects. In addition, the association of other-fo- cused emotion regulation with interview performance in study 4b not only replicated the role of other-focused emotion regulation for effec- tive functioning in the work place but also strengthened this previous finding using a more objective (i.e., other-rated) criterion. Interestingly, self-focused emotion regulation was not associated with interview performance. Although we expected other-focused emotion regulation

to play a more important role than self-focused emotion regulation, it appears counterintuitive that the effective regulation of feelings of stress does not contribute to the evaluation of an interviewer. The current findings suggest that self-focused emotion regulation may not be noticed or valued by the interviewer, which may be because a level of nervousness is typical in this type of setting. Study 4c indicated that other-focused emotion regulation was positively associated with a transformational leadership style but not with a transactional leader- ship style. Transformational leadership, in turn, was positively asso- ciated with leadership effectiveness, which suggests that other-focused emotion regulation may contribute to leadership performance (cf. Harms & Credé, 2010).

6. General discussion

The present paper introduced a new self-reported instrument to measure self- and other-focused EI. The REIS comprises four con- ceptually distinct EI dimensions: self-focused emotion appraisal, self- focused emotion regulation, other-focused emotion appraisal, and other-focused emotion regulation. These EI dimensions have been shown to be reliable and factorially distinct across eight different samples. The convergent and discriminant validity of the REIS was established by showing its strong associations with other self-reported EI instruments and its weak to moderate associations with an ability EI test, cognitive intelligence, and personality measures. Finally, the cri- terion validity of the REIS was demonstrated by a negative association of self-focused emotion regulation with tutors' perceived stress and positive associations of other-focused emotion regulation with tutors' work engagement, jobseekers' other-rated interview performance, and leaders' transformational leadership style.

This novel scale contributes to the literature in two main ways. First, the REIS is among the first EI instruments that systematically capture self- and other-focused EI (cf. Brasseur et al., 2013); it thus provides a wider scope of EI dimensions than most existing measures. In parti- cular, the inclusion of other-focused emotion regulation fills a gap in the conventional EI instruments. Among the most currently well-known EI instruments, only the full-length TEIQue has a unique subscale that covers other-focused emotion regulation (Petrides, 2009). Our data show that this specific dimension was the only REIS dimension that

Table 7 Means, standard deviations, internal consistencies (between brackets), and correlations of the REIS dimensions and indicators of criterion validity in studies 4a (N = 73), 4b (N = 95), and 4c (N = 103).

Study M SD 1 2 3 4 5 6 7 8 9 10 11

1 Self-focused emotion appraisal 4a 3.87 0.48 (0.83) 4b 3.99 0.42 (0.79) 4c 3.84 0.43 (0.76)

2 Other-focused emotion appraisal 4a 3.74 0.52 0.49⁎⁎⁎ (0.85) 4b 3.78 0.45 0.47⁎⁎⁎ (0.84) 4c 3.85 0.46 0.28⁎⁎ (0.85)

3 Self-focused emotion regulation 4a 3.65 0.62 0.25⁎ 0.18 (0.85) 4b 3.72 0.50 0.26⁎ 0.47⁎⁎⁎ (0.74) 4c 3.54 0.53 0.18 0.19 (0.72)

4 Other-focused emotion regulation 4a 3.63 0.50 0.24⁎ 0.37⁎⁎ 0.14 (0.83) 4b 3.51 0.51 0.34⁎⁎ 0.47⁎⁎⁎ 0.44⁎⁎⁎ (0.82) 4c 3.76 0.40 0.32⁎⁎ 0.56⁎⁎⁎ 0.14 (0.72)

5 Total REIS score 4a 3.72 0.36 0.71⁎⁎⁎ 0.74⁎⁎⁎ 0.63⁎⁎⁎ 0.63⁎⁎⁎ (0.87) 4b 3.75 0.35 0.67⁎⁎⁎ 0.80⁎⁎⁎ 0.74⁎⁎⁎ 0.77⁎⁎⁎ (0.89) 4c 3.75 0.30 0.64⁎⁎⁎ 0.74⁎⁎⁎ 0.61⁎⁎⁎ 0.71⁎⁎⁎ (0.84)

6 Perceived stress 4a 2.76 0.85 −0.19 −0.19 −0.42⁎⁎⁎ −0.08 −0.34⁎⁎ (0.91) 7 Work engagement 4a 4.78 0.92 0.16 0.18 −0.01 0.49⁎⁎⁎ 0.29⁎ −0.35⁎⁎ (0.90) 8 Other-rated interview performance 4b 7.20 1.37 0.13 0.08 0.03 0.23⁎ 0.16 (0.89) 9 Transactional leadership 4c 2.55 0.61 0.05 −0.12 0.01 0.12 0.02 (0.74) 10 Transformational leadership 4c 3.74 0.46 0.14 0.19 −0.05 0.33⁎⁎ 0.20⁎ 0.30⁎⁎ (0.84) 11 Leadership effectiveness 4c 3.90 0.48 0.17 0.06 0.16 0.10 0.18 0.24⁎ 0.62⁎⁎⁎ (0.65)

⁎ p < 0.05. ⁎⁎ p < 0.01. ⁎⁎⁎ p < 0.001.

K.A. Pekaar et al. Personality and Individual Differences 120 (2018) 222–233

231

could predict job performance related outcomes, such as leadership and interview performance. This predictive value suggests that other-fo- cused emotion regulation is a valid and important aspect of EI.

Second, the divide in EI's key dimensions, emotion appraisal and emotion regulation, in the REIS enables a reliable differentiation in two conceptually distinct EI dimensions. Specifically, the emotion appraisal and emotion regulation dimensions showed only moderate inter-cor- relations across eight samples, which suggest their ability to capture different emotional processes. For example, the current data showed that (self-focused) emotion appraisal is negatively associated with neuroticism, whereas (self-focused) emotion regulation decreased em- ployees' perceived stress in real work situations. These findings may indicate that different EI dimensions play a role in social or work-re- lated constructs. In turn, these associations may thus be reflective of different steps in the process of dealing with emotions.

6.1. Limitations

The present studies are not without limitations. First, we choose to develop a self-reported instrument of self- and other-focused EI and not an ability test. Self-reported measures of EI have been criticized in the literature (e.g., Mayer & Salovey, 1997; Roberts et al., 2010) because of the potential influence of a social desirability bias. On the positive side, self-reported EI instruments have demonstrated good incremental va- lidity over cognitive intelligence and personality compared with ability EI tests (O'Boyle et al., 2011). Moreover, compared with ability tests, self-reported instruments can be more easily employed in field studies. A second limitation is the use of cross-sectional data, which prevents us from making causal inferences. Nevertheless, in the present studies, we did not aim to establish causal relationships between self- and other- focused EI and other constructs; we aimed to develop a reliable scale to measure these factors and examine how they are associated with the- oretically related constructs. A third limitation may be the generic format of the REIS items. This limitation was based on literature that indicates specific emotions may trigger extreme responses among cer- tain respondents (Gray, 1987), as well as a practical inability to include all types of emotions proportionally in a short scale. Finally, the rela- tively small samples of studies 3 and 4 may limit the generalizability of the specific identified relations. Future research using larger samples must examine whether the associations with specific outcomes can be replicated.

6.2. Implications and conclusion

By developing a reliable scale to measure self- and other-focused EI, we would like to encourage researchers to delve deeper into the pro- cesses that underlie the manifestation of EI. Many studies have pre- viously shown that EI is positively associated with performance and health. However, few studies have examined these outcomes in tandem or zoomed in on the role of specific EI dimensions in the processes that underlie these associations. A certain combination (i.e., a balance) of self- and other-focused EI dimensions may work best to remain a healthy and effective employee. The REIS could be used to answer these important questions.

Practically, the REIS could be used to construct an individual's profile of EI dimensions for selection purposes. For example, several popular intelligence measures (e.g., Wechsler, 2008) deliver unique score profiles to diagnose or select respondents. In the usage of these profiles, a critical yet often overlooked precondition is the reliability of the difference scores between the dimensions (Drenth & Sijtsma, 1990). In the current studies, these reliabilities were satisfactory (e.g., ranged between 0.70 and 0.75 in study 1), which can be considered a strength for the differential prediction of EI dimensions that we aimed to cap- ture.

To conclude, the current studies have resulted in a novel and psy- chometrically sound instrument to measure self- and other-focused EI,

which may be used in future research to build on our current under- standing of EI. Hopefully, the REIS will facilitate the undertaking of further empirical research regarding the role of EI in various domains. This research is necessary to understand the specific effects of emo- tional processes on the lives of individuals.

Acknowledgement

We wish to thank Omar Lamkadmi, Joris de Vries, and Loeki Lauw for their help in collecting the data.

References

Angleitner, A., John, O. E., & Lohr, E. J. (1986). It's what you ask and how you ask it: An itemmetric analysis of personality questionnaires. In A. Angleitner, & J. S. Wiggins (Eds.). Personality assessment via questionnaires (pp. 61–107). Berlin: Springer-Verlag.

Austin, E. J., & O'Donnell, M. M. (2013). Development and preliminary validation of a scale to assess managing the emotions of others. Personality and Individual Differences, 55, 834–839.

Austin, E. J., Saklofske, D. H., Smith, M., & Tohver, G. (2014). Associations of the managing the emotions of others (MEOS) scale with personality, the dark triad and trait EI. Personality and Individual Differences, 65, 8–13.

Bass, B. M. (1985). Leadership and performance beyond expectations. New York: Free Press. Bass, B. M., & Avolio, B. J. (1990). Improving organizational effectiveness through trans-

formational leadership. Thousand Oaks: Sage Publications. Brasseur, S., Grégoire, J., Bourdu, R., & Mikolajczak, M. (2013). The profile of emotional

competence (PEC): Development and validation of a measure that fits dimensions of emotional competence theory. PloS One, 8, e62635.

Byrne, B. M. (2004). Testing for multigroup invariance using AMOS graphics: A road less traveled. Structural Equation Modeling: A Multidisciplinary Journal, 11, 272–300.

Centraal Bureau voor de Statistiek (2016). Barometer beroepsbevolking (Retrieved from) http://www.cbs.nl/nl-NL/menu/themas/arbeid-sociale-zekerheid/publicaties/ barometer-beroepsbevolking/barometer-beroepsbevolking-art.htm.

Ciarrochi, J. V., Chan, A. Y. C., & Caputi, P. (2000). A critical evaluation of the emotional intelligence construct. Personality and Individual Differences, 28, 539–561.

Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting most form your analysis. Practical Assessment, Research & Evaluation, 10, 1–9.

Côté, S., DeCelles, K. A., McCarthy, J. M., Van Kleef, G. A., & Hideg, I. (2011). The Jekyll and Hyde of emotional intelligence emotion-regulation knowledge facilitates both prosocial and interpersonally deviant behavior. Psychological Science, 22, 1073–1080.

Davies, M., Stankov, L., & Roberts, R. D. (1998). Emotional intelligence: In search of an elusive construct. Journal of Personality and Social Psychology, 75, 989–1015.

Denissen, J. A., Geenen, R., van Aken, M. A. G., Gosling, S. D., & Potter, J. (2008). Development and validation of a Dutch translation of the big five inventory (BFI). Journal of Personality Assessment, 90, 152–157.

Doerwald, F., Scheibe, S., Zacher, H., & Van Yperen, N. W. (2016). Emotional compe- tencies across adulthood: State of knowledge and implications for the work context. Work, Aging and Retirement, 2, 159–216.

Drenth, P. J. D., & Sijtsma, K. (1990). Testtheorie. Houten: Bohn, Stafleu, Van Loghum bv. Gardner, H. (1983). Frames of mind: The theory of multiple intelligences. New York: Basic

Books. Gray, J. A. (1987). Perspectives on anxiety and impulsivity: A commentary. Journal of

Research in Personality, 21, 493–509. Gross, J. J., Sheppes, G., & Urry, H. L. (2011). Emotion generation and emotion regula-

tion: A distinction we should make (carefully). Cognition and Emotion, 25, 765–781. Harms, P. D., & Credé, M. (2010). Emotional intelligence and transformational and

transactional leadership: A meta-analysis. Journal of Leadership and Organizational Studies, 17, 5–17.

Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55.

Jordan, P. J., Ashkanasy, N. M., & Hartel, C. E. (2002). Emotional intelligence as a moderator of emotional and behavioral reactions to job insecurity. Academy of Management Review, 27, 361–372.

Joseph, D. L., & Newman, D. A. (2010). Emotional intelligence: An integrative meta- analysis and cascading model. Journal of Applied Psychology, 95, 54–78.

Keefer, K. V. (2015). Self-report assessments of emotional competencies: A critical look at methods and meanings. Journal of Psychoeducational Assessment, 33, 3–23.

Law, K. S., Wong, C. S., & Song, L. J. (2004). The construct and criterion validity of emotional intelligence and its potential utility for management studies. Journal of Applied Psychology, 89, 483–496.

Lee, J., Lim, N., Yang, E., & Lee, S. M. (2011). Antecedents and consequences of three dimensions of burnout in psychotherapists: A meta-analysis. Professional Psychology: Research and Practice, 42, 252–258.

Levenstein, S., Prantera, C., Varvo, V., Scribano, M. L., Berto, E., Luzi, C., & Andreoli, A. (1993). Development of the perceived stress questionnaire: A new tool for psycho- somatic research. Journal of Psychosomatic Research, 37, 19–32.

Lievens, F., & Chan, D. (2010). Practical intelligence, emotional intelligence, and social intelligence. In J. L. Farr, & N. T. Tippins (Eds.). Handbook of employee selection (pp. 339–360). New York: Routledge.

Lopes, P. N., Brackett, M. A., Nezlek, J. B., Schutz, A., Sellin, I., & Salovey, P. (2004).

K.A. Pekaar et al. Personality and Individual Differences 120 (2018) 222–233

232

Emotional intelligence and social interaction. Personality and Social Psychology Bulletin, 30, 1018–1034.

MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004). Confidence limits for the indirect effect: Distribution of the product and resampling methods. Multivariate Behavioral Research, 39, 99–128.

Martins, A., Ramalho, N., & Morin, E. (2010). A comprehensive meta-analysis of the re- lationship between emotional intelligence and health. Personality and Individual Differences, 49, 554–564.

Mayer, J. D., & Salovey, P. (1997). What is emotional intelligence? In P. Salovey, & D. J. Sluyter (Eds.). Emotional development and emotional intelligence: Educational implica- tions (pp. 3–34). New York: Harper Collins.

Mayer, J. D., Salovey, P., & Caruso, D. R. (2002). Mayer– Salovey–Caruso emotional in- telligence test (MSCEIT) item booklet. Toronto, Ontario, Canada: MHS Publishers.

Mayer, J. D., Salovey, P., & Caruso, D. R. (2012). The validity of the MSCEIT: Additional analyses and evidence. Emotion Review, 4, 403–408.

Niven, K., Totterdell, P., Stride, C., & Holman, D. (2011). Emotion regulation of others and self (EROS): The development and validation of a new individual difference measure. Current Psychology, 30, 53–73.

O'Boyle, E. H., Jr., Humphrey, R. H., Pollack, J. M., Hawver, T. H., & Story, P. A. (2011). The relation between emotional intelligence and job performance: A meta-analysis. Journal of Organizational Behavior, 31, 788–818.

Petrides, K. V. (2009). Psychometric properties of the trait emotional intelligence ques- tionnaire. In C. Stough, D. H. Saklofske, & J. D. A. Parker (Eds.). Assessing emotional intelligence: Theory, research, and applications (pp. 85–101). New York: Springer.

Petrides, K. V. (2011). Ability and trait emotional intelligence. In T. Chamorro-Premuzic, A. Furnham, & S. von Stumm (Eds.). The Blackwell-Wiley handbook of individual dif- ferences (pp. 656–678). New York: Wiley.

Petrides, K. V., Pita, R., & Kokkinaki, F. (2007). The location of trait emotional in- telligence in personality factor space. British Journal of Psychology, 98, 273–289.

Raven, J. C. (1962). Advanced progressive matrices, set II. London, UK: H.K. Lewis.

Roberts, R. D., Matthews, G., & Zeidner, M. (2010). Emotional intelligence: Muddling through theory and measurement. Industrial and Organizational Psychology, 3, 140–144.

Salovey, P., & Mayer, J. D. (1990). Emotional intelligence. Imagination, Cognition, and Personality, 9, 185–211.

Schaufeli, W. B., Bakker, A. B., & Salanova, M. (2006). The measurement of work en- gagement with a short questionnaire a cross-national study. Educational and Psychological Measurement, 66, 701–716.

Siegling, A. B., Saklofske, D. H., & Petrides, K. V. (2015). Measures of ability and trait emotional intelligence. In G. J. Boyle, G. Matthews, & D. H. Saklofske (Eds.).Measures of personality and social psychological constructs. San Diego: Academic Press.

Stuart, E. (2005). Willingness to change because of leadership. Is leadership the key to successful change? An investigation of the influence of leadership style on the willingness to change by teachers in secondary schoolsThe Netherlands: Vrije Universiteit van Amsterdam (Unpublished doctoral dissertation).

Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics. Boston: Allyn and Bacon.

Van der Linden, D., Pekaar, K. A., Bakker, A. B., Schermer, J. A., Vernon, P. A., Dunkel, C. S., & Petrides, K. V. (2017). Overlap between the general factor of personality and emotional intelligence: A meta-analysis. Psychological Bulletin, 143, 36–52.

Wechsler, D. (2008). Wechsler Adult Intelligence Scale (Fourth edition). San Antonio, TX: Pearson Assessment.

Wong, C. S., & Law, K. S. (2002). The effects of leader and follower emotional intelligence on performance and attitude: An exploratory study. The Leadership Quarterly, 13, 243–274.

Xanthopoulou, D., Bakker, A. B., & Fischbach, A. (2013). Work engagement among em- ployees facing emotional demands. Journal of Personnel Psychology, 12, 74–84.

Zeidner, M., Roberts, R. D., & Matthews, G. (2008). The science of emotional intelligence: Current consensus and controversies. European Psychologist, 13, 64–78.

K.A. Pekaar et al. Personality and Individual Differences 120 (2018) 222–233

233