Discussion

Workplace Bullying and Suicidal Ideation: A 3-Wave Longitudinal Norwegian Study Morten Birkeland Nielsen, PhD, Geir Høstmark Nielsen, PhD, Guy Notelaers, PhD, and Ståle Einarsen, PhD

Suicide is a leading cause of death around the globe. Estimates show that more than 800 000 people take their own lives every year.1 In Norway (with a population of 5165 802) there are about 530 reported suicides every year.2

In the United States, 12 suicide deaths per 100 000 people were reported in 2010, mak- ing suicide the 10th leading cause of death among Americans.3 Altogether, 1719 em- ployees committed suicide in US workplaces between 2003 and 2010.4

Although psychiatric disorders are involved in the majority of suicide attempts,5 most psychiatric patients do not commit suicide. A psychiatric disorder alone is, therefore, an insufficient condition for suicide.6 To identify other risk factors, we must look beyond the presence of a psychiatric syndrome and un- derstand the underlying factors of suicide and suicidal ideation. Among many potential causes, exposure to workplace bullying has been proposed to be an important predictor of both suicidal ideation and actual suicide.7,8

To date, bullying as an antecedent to suicide has been examined only with anecdotal evidence9,10 and cross-sectional research designs.11,12 Einarsen et al.13 established that severely bullied workers were 6 times more likely than nonbullied workers to report sui- cidal ideations. Sterud et al.8 found that work- place bullying was positively associated with suicidal ideation in a nationwide sample of 1022 Norwegian ambulance personnel. Bully- ing was more strongly associated with suicidal ideation than were gender, neuroticism, anxi- ety, somatic complaints, depersonalization, and job dissatisfaction.

Cross-sectional research cannot provide ad- equate evidence for anything more than that suicidal ideation is a correlate of bullying. To understand the directional nature of the asso- ciation, longitudinal research is needed. In this representative, longitudinal study, we contribute to the literature by examining whether victimization from bullying is related

to increased risk of suicidal ideation over time and whether suicidal ideation is related to subsequent bullying.

Workplace bullying refers to a situation in which 1 or several individuals persistently perceive themselves to be on the receiving end of negative actions from superiors or co- workers and in which the targets find it difficult to defend themselves against these actions.14,15

Following this definition, there are 3 main characteristics of workplace bullying: (1) an employee becomes the target of systematic negative and unwanted social behavior in the workplace; (2) the exposure occurs over a pro- longed period, often with ever more escalating intensity and frequency in the attacks; and (3) targets feel they cannot easily escape the situation or stop the unwanted treatment. This third characteristic, the feeling of being vic- timized by the harassment, distinguishes bul- lying from other forms of mistreatment in the workplace.15 Globally, about 11% of workers perceive themselves as victims of bullying,16

and 5% of the Norwegian working population is victimized by bullying at any time.17

The interpersonal theory of suicide (ITS)5

provides a theoretical foundation for how

exposure to such bullying may be related to suicide. The theory posits that fundamental to suicidal ideation and behavior is that an indi- vidual has both the desire and the ability to die by suicide.18 With regard to the desire to die, displayed through suicidal ideation, the ITS asserts that when people over a prolonged period perceive themselves to be socially alienated from others and simultaneously feel that they are a burden on others, they develop a desire for death.19 As for the ability to commit suicide, displayed through suicidal behavior, the ITS proposes that people who are repeat- edly exposed to painful and provocative events will lose any fear of pain, injury, and death and thereby be able to overcome the instinct of self-preservation.

Because of its focus on persistent exposure to painful events and social alienation, the ITS strongly suggests that repeated and long-term exposure to negative treatment and social exclusion from one’s peers or supervisors at work constitutes a risk factor for suicidal ideation and behavior.

Although previous research has assumed bullying to be an antecedent to suicidal ideation, it is possible that the established cross-sectional

Objectives. We examined whether victimization from bullying is related to an

increased risk of suicidal ideation over time and whether suicidal ideation is

related to subsequent bullying.

Methods. In a longitudinal study (2005–2010), we used well-established single-

item measures to assess victimization from bullying and suicidal ideation. We

used latent Markov models to determine forward and reverse relationships

between variables at 3 time points with 2 or 3 years between the measurement

points among a randomized nationwide sample of 1846 employees in Norway.

Results. Victimization from bullying was associated with subsequent suicidal

ideation (odds ratio = 2.05; 95% confidence interval = 1.08, 3.89). Suicidal ideation

at baseline was not related to subsequent victimization from workplace bullying.

Conclusions. Workplace bullying may be a precursor to suicidal ideation,

whereas suicidal ideation seems to have no impact on subsequent risk of being

bullied. Regulations against bullying should be integrated into work-related

legislation and public health policies. (Am J Public Health. 2015;105:e23–e28.

doi:10.2105/AJPH.2015.302855)

RESEARCH AND PRACTICE

November 2015, Vol 105, No. 11 | American Journal of Public Health Nielsen et al. | Peer Reviewed | Victimization and Violence | e23

association reflects a relationship in which suicidal ideation is a precursor to bullying. Two different mechanisms can explain such a re- verse association. First, employees with suicidal ideation may report less favorable work char- acteristics because their distress makes them evaluate their work environment increasingly more negatively.20 Second, employees with suicidal ideations may elicit aggressive behav- ior in others because their psychological state creates aversive feelings among co-workers and supervisors.21,22

To provide better indications of how work- place bullying is related to suicidal ideation, we investigated direct forward and reverse asso- ciations with longitudinal data. We tested the following hypotheses:

Hypothesis 1: Victimization from bullying is associated with an increased risk of later suicidal ideation. Hypothesis 2: Suicidal ideation is associated with an increased risk of later victimization from bullying.

METHODS

A nationwide sample of the Norwegian workforce participated in 3 surveys with a time lag of approximately 2 years between baseline (T1; 2005) and the second measurement point (T2; 2007), and 3 years between the second and third measurement points (T3; 2010). These time lags represent the typical length of a bullying episode15 and are in line with the majority of previous prospective studies on bullying.20 They have also been found to be optimal for detecting long-term stressor---strain relationships.23 We have extended the litera- ture24---25 from the same data collection by being the first to include 3 time points and examine associations between bullying and suicidal ideation.

In 2005, we drew a random sample of 4500 employees from the Norwegian Central Em- ployee Register by Statistics Norway. Sampling criteria were being aged between 18 and 65 years, being employed during the last 6 months before the survey in an enterprise with a staff of 5 or more, and having a mean work week of more than 15 hours. We distributed question- naires through the Norwegian Postal Service to the respondents’ home addresses. Altogether

2539 questionnaires were returned (response rate = 57%).

We conducted the second and third surveys using the same procedure. We asked all respon- dents from the baseline survey to participate in both follow-ups. Of the T1 respondents, 1775 (70%) participated at T2, whereas 1613 (64%) participated at T3. Altogether 1291 persons participated at all 3 time points (overall response rate = 51%). The final sample is representative of the Norwegian working population with regard to demographic characteristics.26---28

Cohort

We included respondents who participated in the survey at least twice and who answered the questions about bullying and suicidal ide- ation (n = 1846). At baseline, the mean age was 44.3 years, with 96.1% employed in a full-time (77.3%) or part-time (18.8%) position and 3.9% on temporary sick leave, paid leave, or vocational rehabilitation. About 22.0% of the respondents had changed jobs between base- line and the third follow-up measurement.

Attrition analyses showed no age differences between the final cohort and dropouts (t = 0.38; df = 2537; P = .71). Systematic gen- der differences (v2 = 10.78; df = 1; P < .001) were revealed between cohort (54% women) and dropouts (47% women) at baseline. We found no differences between cohort and dropouts for bullying (v2 = 0.96; df = 4; P = .92) or suicidal ideation (v2 = 1.97; df = 3; P = .58) at baseline. Analyses of attrition from T1 to T2 and T3 and from T2 to T3 revealed similar patterns of attrition. The only significant differences between stayers and dropouts were those related to gender. This suggests that, with the exception of gender distribution, the follow-up samples are representative of the baseline sample and thereby the general Nor- wegian working population.

Instruments

We measured workplace bullying with the well-established self-labeling method.29---32 We presented respondents with the following def- inition:

Bullying (harassment, badgering, niggling, freez- ing out, offending someone) is a problem in some workplaces and for some workers. To label something bullying it has to occur repeatedly over a period of time, and those confronted have

to have difficulties defending themselves. It is not bullying if 2 parties of approximately equal “strength” are in conflict or the incident is an isolated event.

We then asked them, “Have you been subjected to bullying at the workplace during the last 6 months?” The response categories were “no,” “rarely,” “now and then,” “once a week,” and “several times a week.”

We examined suicidal ideation with a single item from the 25-item version of the Hopkins Symptoms Checklist,30 which asked respon- dents if they had experienced “thoughts about ending your life” during the past 7 days. Respondents provided answers on a 4-point severity scale ranging from “not at all” through “sometimes” and “very often” to “extreme.” This single-item method has previously been established as a valid approach to the mea- surement of suicidal ideation.33---35

We included gender, age, and whether the respondents had changed jobs or workplaces as covariates in the analyses.

Statistical Analysis

Because we measured suicidal ideation and bullying with categorical single-item measures, we employed a dual-process latent Markov model (LMM) in Latent Gold 5.036 to analyze the data. A description of this statistical method is available as a supplement to the online version of this article at http://www.ajph.org. We followed a model comparison procedure for nested models to determine the direction of the relationship between bullying and suicidal ideation. Compared with approaches that ex- amine between-person changes, the LMM method assesses within-person changes over time.

The LMM procedure compares models on the basis of their log-likelihood squared (L2) and degree of freedom (df ). In the first model, labeled “the stability model,” previous latent states of bullying (WBt---1) predicted subsequent latent states of bullying (WBt), and previous latent states of suicidal ideation (SIt---1) predict subsequent states of suicidal ideation (SIt). Extending the associations included in the stability model, the second model added re- lationships between previous latent states of bullying (WBt---1) and subsequent latent states of suicidal ideation (SIt; forward relationships). In the third model, we added a cross-lagged

RESEARCH AND PRACTICE

e24 | Victimization and Violence | Peer Reviewed | Nielsen et al. American Journal of Public Health | November 2015, Vol 105, No. 11

effect from SIt---1 to WBt (reverse relationships) to the stability model. Finally, we tested a model with bidirectional associations, from SIt---1 to WBt and from WBt---1 to SIt, to de- termine reciprocal relationships.

We calculated the change parameter esti- mates of the examined relationships as odds ratios (ORs). These estimates show the calcu- lated odds for changing from a previous state (i.e., state at a previous measurement occasion; labeled “T---1”) to a later state (i.e., state at a later measurement occasion; labeled “T”). The stability logits associated with the change in bullying states and in suicidal ideation states served as reference categories for estimating model parameters.

RESULTS

The prevalence of workplace bullying and suicidal ideation and the intercorrelations

between study variables are presented in Table 1. At T1, 3.9% of the respondents reported suicidal ideation. The corresponding numbers at T2 2 years, and T3 5 years later were 4.9% and 4.0%, respectively. Altogether, 4.3% at T1, 4.6% at T2, and 4.2% at T3 reported bullying. Cross-tabulation analyses showed no significant differences between men and women with regard to bullying (v2 = 0.72; df = 4; P = .95) and suicidal ideation (v2 = 3.80; df = 1; P = .053). No significant differences were established between non- bullied and bullied respondents (t = 0.16; df = 2367; P = .88) or between nonsuicidal ideation and suicidal ideation respondents (t = 0.66; df = 2363; P = .51) with regard to age.

Moderate stability was found for bullying (Spearman r = 0.24---0.31; P < .001), whereas moderate to high stability was found for sui- cidal ideation (Spearman r = 0.35---0.47; P < .001). In support of hypothesis 1, bullying

at T1 was positively associated with suicidal ideation at both T2 (Spearman r = 0.10; P < .001) and T3 (Spearman r = 0.09; P < .001). In support of hypothesis 2, suicidal ideation at T1 was positively related to sub- sequent bullying at T2 (Spearman r = 0.13; P < .001) and T3 (Spearman r = 0.10; P < .001).

Identification of Latent States

The Bayesian information criterion for the different models (Table 2) suggested that a dual-process LMM with 2 latent states for bullying and 2 latent states for suicidal ideation had the best fit to the data. Distinguishing more states leads to a deterioration of fit. In the first bullying state, the probability of responding “no” to the question about bullying varied from 98.4% to 99.8% over the 3 time points, indicating that the respondents in this latent state were not bullied at any time point.

In the second latent bullying state, the probability of responding “no” varied between 36.5% and 55.5% over time. Hence, the re- spondents in the second latent state perceived themselves as bullied to some extent over the 3 time points. We allowed the relationship between time and the self-labeling item be- cause of the high bivariate residuals between time and the bullying item. The estimates of the measurement model demonstrated that this partial variance entails that over time targets were more likely to agree with all response categories except “no,” indicating a higher fre- quency or intensity of bullying. On the basis of these findings, the states were labeled “not bullied” and “bullied.” As for the latent states of suicidal ideation, the respondents in the first state had a 99.3% probability of not reporting suicidal ideation over the time points. The second latent state was characterized by a 56.0% probability of experiencing some suicidal ideation over time. The states were labeled “no suicidal ideation” and “suicidal ideation.”

Cross-Lagged Associations Between

Bullying and Suicidal Ideation

The model fit for and comparisons of dif- ferent causal models are presented in Table 3. Our findings show that only model 2, in which bullying predicted later suicidal ideation, gave a significant improvement fit at the 0.05

TABLE 1—Prevalence Estimates (%) and the Spearman r Intercorrelations for Bullying and

Suicidal Ideation: Norway, 2005–2010

Variable Prevalence Estimate, %a 1 2 3 4 5 6

1. Bullying, 2005 4.3 . . .

2. Bullying, 2007 4.6 0.31*** . . .

3. Bullying, 2010 4.2 0.24*** 0.24*** . . .

4. Suicidal ideation, 2005 3.9 0.12*** 0.13*** 0.10*** . . .

5. Suicidal ideation, 2007 4.9 0.10*** 0.16*** 0.06 0.45*** . . .

6. Suicidal ideation, 2010 4.0 0.09*** 0.17*** 0.10*** 0.35*** 0.47*** . . .

a Percentage of respondents with positive responses to the questions about bullying (“Yes, rarely,” “Yes, now and then,” “Yes, once a week,” and “Yes, several times a week”) and suicidal ideation (from “sometimes” and “very often” to “extreme”). ***P < .001 (2-tailed).

TABLE 2—Fit Statistics to Determine Number of Latent States for Bullying and Suicidal

Ideation: Norway, 2005–2010

Model of LS Suicidal Ideation LL BIC(LL) Npar L 2

1: LS bullying 1 –2158.68 4434.305 15 2993.68

2: LS bullying 2 –1973.12 4187.948 31 2622.57***

3: LS bullying 2 –1964.32 4263.903 43 2604.97***

2: LS bullying 3 –1966.51 4260.480 42 2609.34***

3: LS bullying 3 –1953.58 4351.579 57 2583.49**

Note. BIC = Bayesian information criterion; LL = log likelihood; L 2 = log likelihood squared; LS = latent states; Npar = number

of parameters. **P < .01; ***P < .001.

RESEARCH AND PRACTICE

November 2015, Vol 105, No. 11 | American Journal of Public Health Nielsen et al. | Peer Reviewed | Victimization and Violence | e25

level over the stability model. Neither the reverse model, in which suicidal ideation pre- dicted subsequent victimization from bullying, nor the reciprocal model, which specified a bi- directional association between the variables, improved the model fit compared with the stability model. As the fit for model 2 was significantly better compared with the reverse and reciprocal models, our findings support hypothesis 1 by showing that existing bullying

predicted later suicidal ideation. Hypothesis 2 was not supported, as suicidal ideation was not associated with subsequent bullying.

Change parameter estimates of the relation- ships in model 2 are displayed in Figure 1. The ORs show that respondents who were not bullied at previous measurement occasions were unlikely to be bullied at later measure- ment points (OR = 0.005; confidence interval [CI] = 0.001, 0.015; P < .001). Respondents

who previously reported no suicidal ideation were unlikely to report later suicidal ideation (OR = 0.028; CI = 0.027, 0.029; P < .001). As the odds for reporting suicidal ideation were twice those for reporting bullying at a previous time point compared with reporting no bullying (OR=2.050; CI=1.08, 3.89; P<.05), bullying was significantly associated with later suicidal ideation. Detailed calculations from the LMM analyses are available as a supplement to the online version of this article at http://www.ajph.org.

We repeated the analyses and adjusted them for the potential impact of gender, age, and changing job or workplace on the initial states, the transition probabilities, and the cross- lagged associations. Our findings showed that the adjusted models were all associated with a higher Bayesian information criterion and thereby a deterioration of fit. In addition, the covariates were not significant related to the model parameters.

DISCUSSION

Supporting hypothesis 1, the findings revealed that victimization from bullying was positively related to later suicidal ideation. Hypothesis 2 was not supported, as suicidal ideation was unrelated to subsequent reports of bullying. Our results are in line with the main assumption of the ITS18 by showing the im- portance of negative life events as antecedents of suicidal ideation. The positive association between bullying and suicidal ideation is con- sistent with previous cross-sectional findings among adults8,11,12 and children and adoles- cents.37---39 As bullying is a form of social exclusion from work, the results are also in line with findings on social isolation as a predictor of suicidal ideation.5

The main theoretical contribution of our study is that bullying is established as a risk factor for later suicidal ideation. The odds for suicidal ideation at a later time point were 2.05 higher among those bullied than among those not bullied. Although the ITS provides a theoretical rationale for how bullying is related to suicidal ideation, it should be em- phasized that we adjusted the findings only for demographic control variables, as we did not examine any other variables.

Further studies are needed to discern whether bullying contributes to the variance in

TABLE 3—Fit Statistics and Multiple Comparison for Establishing Direction of Relationships

Between Bullying and Suicidal Ideation: Norway, 2005–2010

Model BIC(LL) L2 df P DL2 (Ddf) DL2 (Ddf)

0: Baseline 4434.3 2993.68 2418 .001

1: Stability 4176.7 2626.96 2404 .001

2: Normal. Bullying →

suicidal ideation

4180.2 2622.66 2403 .001 model 1 vs

model 2: 4.30*

3: Reversed. Suicidal

ideation → bullying

4184.1 2626.52 2403 .001 model 1 vs

model 3: 0.44(1)

model 2 vs

model 3: 3.86(0)

4: Reciprocal 4187.9 2622.57 2402 .001 model 1 vs

model 4: 4.39(2)

Note. BIC = Bayesian information criterion; df = degrees of freedom; LL = log likelihood; L 2 = log likelihood squared;

D = changes in L2/df. *P < .05.

0.005*** (0.001, 0.015) Not bullied

Bullied

No suicidal ideation

Suicidal ideation

Not bullied

Bullied

No suicidal ideation

Suicidal ideation

T-1 T

2.050* (1.08, 3.89)

0.028*** (0.027, 0.029)

Note. T-1 = previous measurement/state; T = current measurement/state. Numbers indicate odds ratios with 95% confidence

intervals in parentheses. Stability parameters served as reference categories for estimates. Odds Ratios were calculated on

the basis of standard errors of logit paratmeters. Only significant paths are shown.

*P < .05; ***P < .001.

FIGURE 1—Parameter estimates of change expressed as odds ratios in the forward

relationships model between bullying and suicidal ideation: Norway, 2005–2010.

RESEARCH AND PRACTICE

e26 | Victimization and Violence | Peer Reviewed | Nielsen et al. American Journal of Public Health | November 2015, Vol 105, No. 11

suicidal ideation over and above other work- related factors. There is also a need for re- search that determines mediating and moder- ating factors explaining how, when, and for whom bullying is related to suicidal ideation. On the basis of the ITS, psychological distress, hopelessness, and alienations are potential mediators, whereas opportunity for interaction with others is an important moderator, which should be examined in future studies.

Contrasting research that shows reciprocal relationships between bullying and mental distress,20,40 we found no associations between suicidal ideation and the subsequent risk of being bullied. An explanation for this may be that previous research has investigated general symptoms of mental distress,41---43 whereas we focused specifically on suicidal ideation. Con- sequently, it may be that the risk of being exposed to bullying is more strongly influenced by other symptoms of mental health problems. To add to the further understanding of how mental distress relates to later bullying, re- search should investigate specific aspects of mental distress that predict bullying.

Strengths and Limitations

Strengths of this study are the longitudinal survey design, the nationwide and representa- tive sample, and response rates that are above average for survey research.44 Hence, the findings may be considered more reliable and valid than are findings from nonrandom cross-sectional studies. As analyses were lon- gitudinal, we can be relatively sure of the direction of the association from exposure to response. Because of its latent variable ap- proach to changes, the use of latent Markov methodology enables the isolation of the true states from measurement error.45

We assessed both bullying and suicidal ideation with single-item questions. It has pre- viously been shown that single-item measures have high content and criterion validity with regard to assessing job stressors.46 Further- more, single-item measures are reliable, as estimated by test---retest correlations; they also correlate strongly with multiple-item scales and predict outcomes effectively.47 Other advan- tages are greater cost effectiveness, greater face validity, and reduced response burden. The use of a single suicide item derived from a de- pression scale is a valid approach to assessing

suicidal ideation.31 The single-item question we used is considered the state-of-the-art method for measuring victimization from bullying.29

As for limitations, we collected data using self-report questionnaires. This may increase the risk of common method variance and response set tendencies. Yet, the time lags between measurement points should reduce the impact of these biases. Although the time lags we used are adequate for detecting stressor---strain relationships,23 other results may have been obtained with longer or shorter lags. Relying on self-report methodology may be problematic with regard to assessing bully- ing and suicidal ideation because of feelings of shame and guilt among respondents,25 and although anonymity is ensured, individuals may underreport both bullying and suicidal ideation. Underreporting may attenuate corre- lations between the variables because error is introduced in the observed relationship.48 We controlled whether demographic factors af- fected the initial states, the transition probabil- ities, and the cross-lagged effects. There are, however, other unaccounted covariates and confounders that may have affected our results. The findings should therefore be interpreted with caution.

We assessed workplace bullying with a 6-month time frame and examined suicidal ideation with a week as a reference. Although this difference may have influenced the re- sults, the time frames we employed are in line with previous studies using the same items. Hence, the findings should be comparable to other studies on these variables.

Conclusions

We have advanced existing theory and re- search on work factors and suicide by clarifying the time-ordered relationships between bully- ing and suicidal ideation. Having established a significant association between victimization from bullying and subsequent suicidal ideation, our findings suggest that bullying may be a cause of suicidal ideation. The findings pinpoint the importance of effective preventive measures against bullying because employees who are exposed to bullying may have an increased probability of considering ending their lives.

At the primary level, our findings suggest that regulations against bullying should be

included in work-related legislation and public health policies. If bullying is allowed to escalate in organizations, it is vital to provide help and support to those targeted. Following the ITS, it may be especially important to help victims rebuild their experience of self-worth and self-esteem to help them establish more positive basic assumptions about life. From a public health perspective, it is essential to develop forms of rehabilitation that can help victims readjust their view of the world, others, and themselves to make them better prepared to meet the requirements of a demanding worklife.49 Fair management of, and inter- vention in, specific cases may also reassure victims of bullying that the world is still comprehensible, manageable, and meaningful and that they are an integral and valued part of a well-functioning social group. j

About the Authors Morten Birkeland Nielsen is with the National Institute of Occupational Health, Oslo, Norway, and the Department of Psychosocial Science, University of Bergen, Bergen, Norway. Geir Høstmark Nielsen is with the Department of Clinical Psychology, University of Bergen. Guy Notelaers and Ståle Einarsen are with the Department of Psychosocial Science, University of Bergen. Correspondence should be sent to Morten Birkeland

Nielsen, National Institute of Occupational Health, PB 8149 Dep, 0033 Oslo, Norway (e-mail: morten.nielsen@ stami.no). Reprints can be ordered at http://www.ajph.org by clicking the “Reprints” link. This article was accepted July 28, 2015.

Contributors M. B. Nielsen participated in data collection, initiated the study, and was responsible for hypotheses development, preliminary data analyses, and the writing of the article. G. H. Nielsen contributed to the idea development and writing of the article. G. Notelaers was responsible for data analyses and participated in all parts of study development. S. Einarsen was responsible for the data collection and the overall project and participated in the writing of the article.

Acknowledgments This study was grounded on data from a collaborative project between the University of Bergen and Statistics Norway (Statistisk Sentralbyrå); Statistics Norway drew the sample and collected the data. The project was made possible by joint grants from 2 Norwegian employers’ associations (Næringslivets Hovedorganisasjon, grant S-2413 and Kommunenes Sentralforbund, grant 044034) and the Norwegian government (Rikstrygde- verket, grant 2004/04102-013/2095).

We would like to thank Jeroen K. Vermunt, Depart- ment of Methodology, University of Tilburg, Netherlands, for his advice and kind help with the syntax of Latent Gold 5. Thanks to Bengt Oscar Lagerstrøm and Maria Høstmark of Statistics Norway and to Anders Skogstad and Stig Berge Matthiesen at the Faculty of Psychology at

RESEARCH AND PRACTICE

November 2015, Vol 105, No. 11 | American Journal of Public Health Nielsen et al. | Peer Reviewed | Victimization and Violence | e27

the University of Bergen for their contribution to the data collection.

Human Participant Protection The overall project, including this study, was approved by the Regional Committee for Medical Research Ethics in Western Norway.

References 1. World Health Organization. Suicide data. Available at: http://www.who.int/mental_health/prevention/ suicide/suicideprevent/en. Accessed March 17, 2015.

2. Norwegian Institute of Public Health. Selvmord og selvmordsforsøk—faktaark med helsestatistikk. [Suicide and suicide attempts—factsheet with health statistics]. Available at: http://www.fhi.no/artikler/?id=70808. Accessed March 17, 2015.

3. Centers for Disease Control and Prevention. Injury Prevention & Control: Data & Statistics (WISQARS). Available at: http://www.cdc.gov/injury/wisqars/index. html. Accessed March 17, 2015.

4. Tiesman HM, Konda S, Hartley D, Menendez CC, Ridenour M, Hendricks S. Suicide in U.S. workplaces, 2003---2010: a comparison with non-workplace suicides. Am J Prev Med. 2015;48(6):674---682.

5. Van Orden KA, Witte TK, Cukrowicz KC, Braithwaite SR, Selby EA, Joiner TE. The interpersonal theory of suicide. Psychol Rev. 2010;117(2):575---600.

6. Mann JJ, Waternaux C, Haas GL, Malone KM. Toward a clinical model of suicidal behavior in psychi- atric patients. Am J Psychiatry. 1999;156(2):181---189.

7. Leymann H. Sjalvmord till følgjd av psykiskt våld i arbetslivet [Suicide as a consequence of psychological terror in the workplace]. Arbete, människa, miljö. 1987;3:155---160.

8. Sterud T, Hem E, Lau B, Ekeberg O. Suicidal ideation and suicide attempts in a nationwide sample of operational Norwegian ambulance personnel. J Occup Health. 2008;50(5):406---414.

9. Leymann H. Självmord till följd av förhålland en i arbetsmiljön [Suicide as a result of conditions in the work environment]. Arbete, människa, miljö. 1987(3):155--- 160.

10. Leymann H. Ingen annan utväg. Om utslagning och självmord som följd av mobbning i arbetslivet [No Other Way Out. About Expulsion and Suicide as a Consequence of Workplace Bullying]. Stockholm, Sweden: Wahlström & Widstrand; 1988.

11. Balducci C, Alfano V, Fraccaroli F. Relationships between mobbing at work and MMPI-2 personality profile, posttraumatic stress symptoms, and suicidal ideation and behavior. Violence Vict. 2009;24(1): 52---67.

12. Soares A. When darkness comes. Workplace bul- lying and suicidal ideation. In: Tehrani N, ed. Workplace Bullying: Symptoms and Solutions. London, UK: Rout- ledge; 2012:67---80.

13. Einarsen S, Raknes BI, Matthiesen SB, Hellesøy OH. Mobbing og harde personkonflikter. Helsefarlig samspill på arbeidsplassen [Bullying and Severe Interpersonal Con- flicts. Unhealthy Interaction at Work]. Bergen, Norway: Sigma Forlag; 1994.

14. Olweus D. Bullying at Schools: What We Know and What We Can Do. Hoboken, NJ: Blackwell; 1993.

15. Einarsen S, Hoel H, Zapf D, Cooper CL. The concept of bullying and harassment at work: the European tradition. In: Einarsen S, Hoel H, Zapf D, Cooper CL, eds. Bullying and Harassment in the Workplace: Developments in Theory, Research and Practice. 2nd ed. Boca Raton, FL: CRC Press; 2011:3---40.

16. Nielsen MB, Matthiesen SB, Einarsen S. The impact of methodological moderators on prevalence rates of workplace bullying. A meta-analysis. J Occup Organ Psychol. 2010;83(4):955---979.

17. Nielsen MB, Skogstad A, Matthiesen SB, et al. Prevalence of workplace bullying in Norway: compari- sons across time and estimation methods. Eur J Work Organ Psyl. 2009;18(1):81---101.

18. Ribeiro JD, Joiner TE. The interpersonal---psychological theory of suicidal behavior: current status and future directions. J Clin Psychol. 2009;65(12):1291---1299.

19. Joiner TE. Why People Die by Suicide. Cambridge, MA: Harvard University Press; 2005.

20. Nielsen MB, Einarsen S. Outcomes of workplace bullying: a meta-analytic review. Work Stress. 2012; 26(4):309---332.

21. Felson RB. “Kick ’em when they’re down”: expla- nations of the relationships between stress and interper- sonal aggression and violence. Sociol Q. 1992;33(1): 1---16.

22. Einarsen S. The nature and causes of bullying at work. Int J Manpow. 1999;20:16---27.

23. Ford MT, Matthews RA, Wooldridge JD, Mishra V, Kakar UM, Strahan SR. How do occupational stressor--- strain effects vary with time? A review and meta-analysis of the relevance of time lags in longitudinal studies. Work Stress. 2014;28(1):9---30.

24. Einarsen S, Nielsen MB. Workplace bullying as an antecedent of mental health problems: a five-year pro- spective and representative study. Int Arch Occup Environ Health. 2015;88(2):131---142.

25. Nielsen MB, Hetland J, Matthiesen SB, Einarsen S. Longitudinal relationships between workplace bullying and psychological distress. Scand J Work Environ Health. 2012;38(1):38---46.

26. Nielsen MB, Einarsen S. Prospective relationships between workplace sexual harassment and psycho- logical distress. Occup Med (Lond). 2012;62(3): 226---228.

27. Høstmark M, Lagerstrøm BO. Undersøkelse om arbeidsmiljø: Destruktiv atferd i arbeidslivet. Dokumentas- jonsrapport [A Study of Work Environments: Destructive Behaviours in Working Life. Documentation Report]. Oslo: Statistics Norway; 2006. Report No. 44.

28. Holmøy A. Undersøkelse om Arbeidsmiljø 2010. Destruktiv atferd i arbeidslivet. Dokumentasjonsrapport. Oslo, Norway: Statistics Norway; 2013. Report No. 39/ 2013.

29. Olweus D. Bully/victim problems among school- children: basic facts and effects of a school-based in- tervention program. In: Pepler DJ, Rubin KH, eds. The Development and Treatment of Childhood Aggression. Hillsdale, NJ: Erlbaum; 1991:411---488.

30. Einarsen S, Skogstad A. Bullying at work: epidemi- ological findings in public and private organizations. Eur J Work Organ Psy. 1996;5(2):185---201.

31. Solberg ME, Olweus D. Prevalence estimation of school bullying with the Olweus Bully/Victim Question- naire. Aggress Behav. 2003;29(3):239---268.

32. Nielsen MB, Notelaers G, Einarsen S. Measuring exposure to workplace bullying. In: Einarsen S, Hoel H, Zapf D, Cooper CL, eds. Bullying and Harassment in the Workplace: Developments in Theory, Research and Prac- tice. Boca Raton, FL: CRC Press; 2010:149---174.

33. Derogatis LR, Lipman RS, Rickels K, Uhlenhuth EH, Covi L. The Hopkins Symptom Checklist (HSCL): a self- report symptom inventory. Behav Sci. 1974;19(1):1---15.

34. Desseilles M, Perroud N, Guillaume S, et al. Is it valid to measure suicidal ideation by depression rating scales? J Affect Disord. 2012;136(3):398---404.

35. Fialko L, Freeman D, Bebbington PE, et al. Un- derstanding suicidal ideation in psychosis: findings from the Psychological Prevention of Relapse in Psychosis (PRP) trial. Acta Psychiatr Scand. 2006;114(3):177---186.

36. Vermunt JK, Magidson J. Latent GOLD 5.0. Upgrade Manual. Belmont, MA: Statistical Innovations Inc; 2013.

37. Bauman S, Toomey RB, Walker JL. Associations among bullying, cyberbullying, and suicide in high school students. J Adolesc. 2013;36(2):341---350.

38. Meltzer H, Vostanis P, Ford T, Bebbington P, Dennis MS. Victims of bullying in childhood and suicide attempts in adulthood. Eur Psychiatry. 2011;26(8):498---503.

39. Rivers I, Noret N. Potential suicide ideation and its association with observing bullying at school. J Adolesc Health. 2013;53(1 suppl):S32---S36.

40. Nielsen MB, Magerøy N, Gjerstad J, Einarsen S. Workplace bullying and subsequent health problems. Tidsskr Nor Laegeforen. 2014;134(12---13):1233---1238.

41. Finne LB, Knardahl S, Lau B. Workplace bullying and mental distress—a prospective study of Norwegian employees. Scand J Work Environ Health. 2011;37 (4):276---287.

42. Hoobler JM, Rospenda KM, Lemmon G, Rosa JA. A within-subject longitudinal study of the effects of positive job experiences and generalized workplace harassment on well-being. J Occup Health Psychol. 2010;15(4): 434---451.

43. Kivimäki M, Virtanen M, Vartia M, Elovainio M, Vathera J, Keltikangas-Järvinen L. Workplace bullying and the risk of cardiovascular disease and depression. Occup Environ Med. 2003;60(10):779---783.

44. Baruch Y, Holtom BC. Survey response rate levels and trends in organizational research. Hum Relat. 2008;61(8):1139---1160.

45. Hagenaars JA. Categorical Longitudinal Data---Log- linear Analysis of Panel, Trend and Cohort Data. Newbury Park, CA: Sage; 1990.

46. Gilbert S, Kelloway EK. Using single items to measure job stressors. Int J Workplace Health Manag. 2014;7(3):186---199.

47. Fisher GG, Matthews RA, Gibbons AM. Developing and investigating the use of single-item measures in organizational research. J Occup Health Psychol. April 20, 2015 [Epub ahead of print].

48. Spector PE, Fox S. The stressor---emotion model of counterproductive work behavior. In: Fox S, Spector PE, eds. Counterproductive Behavior: Investigations of Actors and Targets. Washington, DC: American Psychological Association; 2005:151---174.

49. Nielsen MB, Matthiesen SB, Einarsen S. Sense of coherence as a protective mechanism among targets of workplace bullying. J Occup Health Psychol. 2008;13 (2):128---136.

RESEARCH AND PRACTICE

e28 | Victimization and Violence | Peer Reviewed | Nielsen et al. American Journal of Public Health | November 2015, Vol 105, No. 11

Copyright of American Journal of Public Health is the property of American Public Health Association and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.