research

International Journal of Nursing Studies 137 (2023) 104385

Contents lists available at ScienceDirect

International Journal of Nursing Studies

j ourna l homepage: www.e lsev ie r .com/ locate /ns

Association of burnout and intention-to-leave the profession with work environment: A nationwide cross-sectional study among Belgian intensive care nurses after two years of pandemic

Arnaud Bruyneel a,⁎, Nicolas Bouckaert b, Charline Maertens de Noordhout b, Jens Detollenaere b, Laurence Kohn b, Magali Pirson a, Walter Sermeus c, Koen Van den Heede b,c

a Health Economics, Hospital Management and Nursing Research Dept, School of Public Health, Université Libre de Bruxelles, Belgium b Belgian Health Care Knowledge Centre (KCE), Brussels, Belgium c KU Leuven Institute for Healthcare Policy, Leuven, Belgium

⁎ Corresponding author at: École de Santé Publique Cam de Lennik 808, CP591 1070, Bruxelles, Belgium.

E-mail address: arnaud.bruyneel@ulb.be (A. Bruyneel)

https://doi.org/10.1016/j.ijnurstu.2022.104385 0020-7489/© 2022 Elsevier Ltd. All rights reserved.

a b s t r a c t

a r t i c l e i n f o

Article history: Received 25 August 2022 Received in revised form 27 October 2022 Accepted 30 October 2022

Background: Intensive care unit (ICU) nurses are at an increased risk of burnout and may have an intention-to- leave their jobs. The COVID-19 pandemic may increase this risk. Objective: The objective of this study was to describe the prevalence of burnout risk and intention-to-leave the job and nursing profession among ICU nurses and to analyse the relationships between these variables and the work environment after two years of the COVID-19 pandemic. Design: A national cross-sectional survey of all nurses working in Belgian ICUs was conducted between December 2021 and January 2022 during the 4th and 5th waves of the COVID-19 pandemic in Belgium. The Practice Environ- ment Scale of the Nursing Work Index (PES-NWI) was used to measure the work environment, intention-to-leave the hospital and/or the professionwas assessed. The risk of burnoutwas assessed using theMaslach Burnout Inven- tory scale including emotional exhaustion, depersonalisation, and reduced personal accomplishment. Setting: Nurses in 78 out of 123 Belgian hospital sites with an ICU participated in the survey. Participants: 2321 out of 4851 nurses (47.8%) completed the entire online survey. Results: The median overall risk of burnout per hospital site (high risk in all three subdimensions) was 17.6% [P25: 10.0 - P75: 28.8] and the median proportion of nurses with a high risk in at least one subdimension of burnout in Belgian ICUs was 71.6% [56.7–82.7]. A median of 42.9% [32.1–57.1] of ICU nurses stated that they intended-to- leave the job and 23.8% [15.4–36.8] stated an intent-to-leave the profession. Themedian overall score of agreement with the presence of positive aspects in thework environmentwas 49.0% [44.8–55.8]. Overall, nursesworking in the top 25% of best-performing hospital sites with regard to work environment had a statistically significant lower risk of burnout and intention-to-leave the job andprofession compared to those in the lowest performing25%of hospital sites. Patient-to-nurse ratio in theworst performing quartilewas associatedwith a higher risk for emotional exhaus- tion (OR=1.53, 95% CI:1.04–2.26) and depersonalisation (OR=1.48, 95% CI:1.03–2.13) and intention-to-leave the job (OR= 1.46, 95% CI:1.03–2.05). Conclusions: In this study, a high prevalence of burnout risk and intention-to-leave the job and nursing profession was observed after two years of the COVID-19 pandemic. Nevertheless, there was substantial variation across hos- pital sites which was associated with the quality of the work environment. Tweetable abstract: “Burnout & intention to leave was high for Belgian ICU nurses after 2 years of COVID, but wellbeing was better with high quality work environments and more favourable patient to nurse ratios”.

© 2022 Elsevier Ltd. All rights reserved.

Keywords: Burnout Intensive care units Nurses COVID-19 Adults Workload

pus Erasme, Bâtiment A Route

.

What is already known

• The COVID-19 pandemic has had a strong impact on ICU nurses' wellbeing and increases the risk of burnout

• A large proportion of ICU nurses in Belgium are at high risk of burnout • A better work environment is associatedwith better quality of care, em- ployee satisfaction, turnover, and average job tenure among ICU nurses

2 A. Bruyneel, N. Bouckaert, C. Maertens de Noordhout et al. / International Journal of Nursing Studies 137 (2023) 104385

What this paper adds

• Even after two years of the COVID-19 pandemic, ICU nurses working in hospitals with a better working environment had less risk of burn- out and intention-to-leave.

• Nurse participation in hospital affairs and staffing, and resource ade- quacy are the work environment dimensions with the lowest score and require particular attention.

• ICU nurses working in hospitals with a lower patient-to-nurse ratio have a lower risk of emotional exhaustion, depersonalisation, and intention-to-leave the job.

1. Background

Burnout is a syndrome that is conceptualised as chronic workplace stress that has not been successfully managed and has resulted in feel- ings of emotional exhaustion, depersonalization, and a lack of personal accomplishment (Maslach et al., 1996; Atroszko et al., 2020). Nurses are particularly prone to burnout due to various professional practice circumstances and working conditions that can lead to physical and emotional exhaustion (Aiken et al., 2012a, 2012b; Woo et al., 2020). Nurse burnout is associated with reduced safety and quality of care, de- creased patient satisfaction, and reduced organisational commitment and productivity of nurses (Jun et al., 2021; Van Gerven et al., 2016). Some studies have reported a high prevalence of burnout among nurses in Belgium compared to other European countries (Woo et al., 2020). One study reported a prevalence of 30% of high risk of emotional ex- haustion among nurses working in hospitals in 2012 (Vandenbroeck et al., 2017) while another reported a prevalence of 36% among nurses in general medical and surgical units in 2019 (Van den Heede et al., 2019). One of the main potential explanations for this high prevalence is the high patient-to-nurse ratio in Belgium compared to other coun- tries (Aiken et al., 2012a, 2012b), which has been shown to be associ- ated with high risk of burnout (Pérez-Francisco et al., 2020; Welp et al., 2019). Nevertheless, a study published in 2018 reported a lower risk of burnout among intensive care unit (ICU) nurses: 11% had a high risk of emotional exhaustion. This study also showed that 7% of ICU nurses intended to leave the nursing profession (Vermeir et al., 2018).

Since March 2020, Europe has been facing the COVID-19 pandemic, caused by the SARS-CoV-2 virus. In Belgium, the Hospital Emergency Plan was launched on March 13, 2020. Intensive care units in Belgium have been heavily impacted by COVID-19 and mortality has been high (Taccone et al., 2020). As of this time, the country has experienced seven waves, mainly in the autumn and winter of December 2021 (Dellicour et al., 2021; Molenberghs et al., 2020; Natalia et al., 2022). During the COVID-19 pandemic, frontline nurses were exposed to higher levels of stress and traumatic events that could increase the risk of burnout (Guixia and Hui, 2020; Hu et al., 2020; LeClaire et al., 2022; Moradi et al., 2021). A study conducted after the first wave of COVID-19 showed a high risk of emotional exhaustion (38%) among French-speaking ICU nurses in Belgium (Bruyneel et al., 2021b). This high risk of emotional exhaustion was more pronounced in ICU nurses compared to non-ICU nurses (Odds Ratio [OR] = 1.23, 95% Confidence Interval [CI]: 1.07–1.42) (Bruyneel and Smith, 2021). The main risk fac- tors for burnout among ICUnurses during the COVID-19 pandemicwere a higher than standard patient-to-nurse ratio, a perceived increase in workload, having symptoms of COVID-19 without being tested, and shortage of COVID-19 protective equipment. Another study in Belgium showed an increase in the risk of burnout and the high risk of emotional exhaustion in French-speaking ICU nurses compared to before the COVID-19 pandemic (Butera et al., 2021). The nurse-to-patient ratio in the ICU in Belgium (about 1:3) is the most unfavourable in Europe, which could explain this high prevalence (Bruyneel et al., 2019; Van Den Heede et al., 2022).

There is a large body of evidence that shows that nurse workload, the quality of the work environment, and patient care demands influ- ence patient outcomes (Cho et al., 2015; Lake et al., 2020; Simonetti et al., 2021). A study in the United States (US) found that interventions that aim to provide better work environment standards (e.g., true col- laboration, authentic leadership, collaboration, effective decision mak- ing, appropriate staffing, meaningful recognition) are associated with improved employee satisfaction, turnover, and average job tenure among ICU nurses (Kester et al., 2021).

The association between work environment and nurse well-being is very clear in the literature. However, it is unclear whether this associa- tion was still valid during a public health crisis that induced a lot of ad- ditional stress in healthcare workers: was a better hospital work environment associated with the well-being of ICU nurses during the COVID-19 pandemic? The objectives of this study were to describe the prevalence of high burnout risk, and intention-to-leave the job and the nursing profession among ICU nurses and to analyse the relation- ships between these variables and the work environment after two years of the COVID-19 pandemic.

2. Methods

2.1. Study design

A national cross-sectional study was conducted between December 2021 and January 2022 during the 4th and 5th waves of the COVID-19 pandemic in Belgium (supplementary material Fig. 1). For the recruit- ment of hospitals and intensive care units, all general acute hospitals (including university hospitals) were invited to participate (invitation to the Chief Nursing Officer - CNO and the Chief Executive Officer (CEO)/Chief Medical Officer – CMO) via an email invitation sent in the firstweekof September 2021. After 14 days, a reminderwas sent to hos- pitals that did not respond to the initial invitation. The call to participate was open until 1 October, 2021. All hospitals that wanted to participate were allowed to participate with all the intensive care units of the hos- pital. The hospitalmanagers could limit the participation to one hospital site if a hospital consisted of multiple sites. Each hospital had to appoint a field manager (e.g., chief nursing officer, middle management, ward manager) to be the single point of contact with the research team. Dur- ing an information session (virtual or in person), the field manager was given additional explanation about the data collection process or could ask for more details about the study.

The survey target population included all registered staff nurses pro- viding direct care to patients in the participating ICU units. Nurses on leave (sick leave, maternity leave, or holiday), agency nurses, ward managers, healthcare assistants, and other non-nursing staff were ex- cluded. Per participating ICU, the number of eligible nurses (applying the inclusion and exclusion criteria on 21 September, 2021) was used to prepare unique participation IDs (QR code giving access to the online survey). All nurses received an envelope from thefieldmanagerwith in- formation about the study and an invitation to participate. This invita- tion included an individualised but anonymized link (QR-code) to an online survey (online platform, Check Market). The QR code allowed (with consent of the respondents) the identification of the nursing unit of the respondent.

2.2. Instruments and measures

The survey was largely based on the RN4CAST survey that was de- signed to study how features of the work environment and staffing levels (number and qualification) impact nurse retention, burnout among nurses, and patient outcomes (Sermeus et al., 2011). Except for some minor modifications in scoping review and professional nursing organisation (e.g., SIZ Nursing) to better match the ICU site, the same survey that was used during a previous nationwide study conducted

A. Bruyneel, N. Bouckaert, C. Maertens de Noordhout et al. / International Journal of Nursing Studies 137 (2023) 104385 3

on general hospitalisation units in 2009 and 2019 was used. The survey is available in French, Dutch, and German (Van den Heede et al., 2019).

The instrument includes the following measures and questions (Sermeus et al., 2011):

(1) Demographic variables, such as education level, age, gender and years of expertise within the hospital/profession/ICU.

(2) Information on nurse staffing. Each nurse was asked to provide information on their qualification level, and to report the number of nurses and patients present on the nursing ward as well as the number of patients cared for during the last shift or workday. Based on the pro- vided information, patient-to-nurse ratios were calculated overall and per type of shift (Sermeus et al., 2011).

(3) The risk of burnout. The risk of burnout was assessed using the validated Maslach Burnout Inventory (MBI) scale (Dion and Tessier, 1994; Maslach et al., 1986). The 22 items of this scale are scored on a 7-point Likert scale ranging from 0=Never to 6= Every day capturing 3 dimensions of burnout: emotional exhaustion (EE), depersonalisation (DP), and reduced personal achievement (PA). The cut-off points for high riskwere >26 for emotional exhaustion, >9 for depersonalisation, and <33 for reduced personal achievement (Maslach et al., 1986). To estimate the overall prevalence of burnout, two definitions were used to compare with other studies. The first is the high risk of burnout in at least one of the three dimensions (broad definition) (Bruyneel et al., 2021b; Butera et al., 2021; Embriaco et al., 2007; Shah et al., 2019). The second is a single score according to the method that is most frequently used in the literature: a combination of high emotional exhaustion, high depersonalisation, and low personal achievement (strict definition) (Boutou et al., 2019; Rotenstein et al., 2018).

(4) The intention-to-leave the hospital and/or the profession in the next year was assessed dichotomously (yes or no). For those who an- swered yes, the type of work sought was requested.

(5) The Practice Environment Scale of theNursingWork Index (PES- NWI) (Lake, 2002) was used to measure the nursing practice environ- ment as is commonly done (Swiger et al., 2017). It includes 32 items with an ordinal score (from 1 = strongly disagree to 4 = strongly agree) to assess 5 components of the environment: Nurse participation in hospital affairs (1), nursing foundations for quality of care (2), nurse manager ability, leadership, and support of nurses (3), staffing and re- source adequacy (4), and collegial nurse-physician relations (5). For each sub-dimension, we calculated the average percentage of nurses at the hospital site answering that they agree or strongly agree with the items of the Practice Environment Scale of the Nursing Work Index (overall score) of the items in the sub-dimension. We also categorised the nurse working environment into quartiles, based on a composite score calculated as themean of the item scores for each hos- pital site. Hence, the lowest quartile consists of the 25% of hospital sites with the worst performance in nurse working environment and the highest quartile includes the 25% best performing hospital sites.

2.3. Statistical analyses

Nurses are considered key sources of information for the assessment of performance at the hospital site level. Hospital site descriptive statis- tics were computed by averaging answers from nurses within the same hospital site. Arbitrarily, at the level of a hospital site, a minimum of 10 respondents or a response rate of 75% was required to include the hos- pital site in the final sample, being sufficiently representative. Results at the national level were calculated as weighted average percentages by hospital site, the weight being the number of respondents at the hospi- tal site. Median (Mdn) variables and InterQuartile Ranges (p25-p75) were used to describe all variables.

Multiple logistic regression analyses were performed to test the association between the nurse working environment and patient-to- nurse ratio at the hospital site level on the one hand (independent var- iables), and high risk of burnout and intention-to-leave the profession on the other hand (dependent variables). All analyses included

variables for region and type of hospital site, and the sex, age, and qual- ification of the nurse. Moreover, the clustering of nurses within hospital siteswas taken into account using a generalised estimation equation ap- proach. Odds ratios (OR) with 95% confidence intervals (95% CI) and p- values are used to describe the results. A p-value <0.05was considered statistically significant.

The statistical analyses were performed using SAS 9.4 and R 4.1.2.

2.4. Ethical considerations

The ethics committee of University Hospital Leuven approved this study (S65994) on 12 November, 2021. The data were processed ac- cording to the Belgian law of 30 July, 2018 and with Regulation (EU) 2016/679 of 27April, 2016, concerning the protection of natural persons in respect of processing activities and to ensure the free flow of personal data.

All nurses were informed via the information letter and electroni- cally signed a Declaration of Consent via the software platform.

3. Results

3.1. Description of the participating hospitals and nurse characteristics

Eighty-six out of 100 Belgian acute hospitals (78 out of 123 hospital sites) with ICUs agreed to participate in the survey. Three hospitals that initially agreed to participate decided not to participate at the time the data collection started due to the extra burden caused by the 4th and 5th waves of the COVID-19 pandemic. The sample, therefore, included 83 hospitals, including six out of seven university hospitals, accounting for 1721 operational beds or 86.4% of the estimated operational ICU beds in Belgium. In total, 2321 of 4851 invited nurses completed the questionnaire, corresponding to a response rate of 47.8% (supplemen- tary material Fig. 2). For the data analysis, 24 hospital sites with 138 re- spondents (5.9%) were further excluded because they did not reach the minimum of 10 respondents nor a response rate of 75%. Hence, the final sample included 68 hospitals, 78 hospital sites, and 2183 respondents (Fig. 1).

The median age [p25-p75] of nurses per hospital site was 35 years [28–47] and the proportion of women was 74.5%. A total of 59.8% of the nurses worked full time and 72.2% of the sample were ICU specialist nurses and 10.6% had a university degree with a specialisation. Theme- dian patient-to-nurse ratio for the day shift was 2.1 [1.9–2.4] and 2.6 [2.3–2.8] for the night shift. The median ratio of healthcare assistant- to-patient and logistics support assistant-to-patient was 0.21 [0.10– 0.43] and 0.43 [0.26–0.69], respectively (Table 1).

3.2. Prevalence of high risk of burnout and intention-to-leave the job and profession

Regarding the high risk of burnout, the median prevalence at the hospital site level was 17.6% [10.0–28.8] using the strict definition and 71.6% [56.7–82.7] using the broad definition. The analysis of high risk of burnout by subdimension showed a median prevalence of 42.9% [30.8–55.8] for reduced personal achievement, 45.0% [27.9–57.7] for depersonalisation, and 46.0% [34.3–56.4] for emotional exhaustion. A median of 42.9% [32.1–57.1] of nurses at a given hospital site indicated an intention-to-leave their job and 23.8% [15.4–36.8] stated an intention-to-leave the nursing profession (Fig. 2).

3.3. Work environment

Themedian percentage of nurses at a given hospital site who agreed or strongly agreed with all Practice Environment Scale of the Nursing Work Index items was 49.0% [44.8–55.8]. There were three subscales with a higher median score (nurse-physician collegial relationship – 70.4% [62.3–77.8]; foundations for nursing quality of care – 64.0%

Fig. 1. Hospital sites and respondent selection process.

4 A. Bruyneel, N. Bouckaert, C. Maertens de Noordhout et al. / International Journal of Nursing Studies 137 (2023) 104385

[57.5–70.4]; manager ability, leadership, and support – 58.3% [46.2–70.0]) and two subscales with a lower median score (staffing and resource adequacy – 28.5% [23.2–37.5]; participation in hospital affairs – 26.7% [20.3–29.3]) (Fig. 3).

3.4. Factors correlating burnout risk and intention-to-leave with work environment

In themultivariate analysis, overall, nursesworking in hospitalswith a better work environment had a statistically significant lower risk of burnout based on both definitions and all three dimensions as well as a lower risk of intention-to-leave the job and the nursing profession when comparing quartile 4 hospitals to quartile 1 (the lowest score). The higher patient-to-nurse ratio in the 4th quartile was associated with a higher risk for emotional exhaustion (OR = 1.53, 95% CI:1.04–

Table 1 Description of the participating hospitals and nurse characteristics at the hospital site level.

Variables Results

Number ICU beds, n 1721 Number hospital academic, n (%) 6 (8.8%) Number bed academic, n (%) 284 (16.5%) Age (year), median [p25-p75] 35 [28–47] Gender, female, n (%) 1626 (74.5%) Experience in ICU (years), median [p25-p75] 11 [4–22] Full-time employed, n (%) 1305 (59.8%) Diploma, n (%) Bachelor's degree 303 (13.9%) Specialised nurses 1577 (72.2%) University degree 34 (1.6%) Specialised nurses and university degree 232 (10.6%) Nurses w/o bachelor's 37 (1.7%)

Patient-to-nurse ratio, median [p25-p75] Overall ratio 2.3 [2.0–2.5] Day shifts (morning to evening) 2.1 [1.9–2.4] Night shift 2.6 [2.3–2.8]

Healthcare assistants-to-patient ratio (all shifts and days), median [p25-p75]

0.21 [0.10–0.43]

Logistic support assistants-to-patient ratio (all shifts and days), median [p25-p75]

0.42 [0.26–0.69]

Legend: ICU: Intensive Care Unit.

2.26) and depersonalisation (OR = 1.48, 95% CI:1.03–2.13) and intention-to-leave the nursing profession (OR = 1.46, 95% CI:1.03– 2.05) compared to the first quartile.

The nurses working in Brussels and Wallonia were statistically sig- nificant more at risk for the dimensions of burnout, for both definitions of burnout and intention-to-leave job as well as for the nursing profes- sion than nurses working in Flanders. Master-level trained nurses had a higher risk of intention-to-leave the nursing profession (OR=3.13, 95% CI:1.38–7.12) compared to bachelor-level nurses. Nurses in academic hospitals were more at risk for emotional exhaustion (OR = 1.42, 95% CI:1.11–1.82) and intention-to-leave the job (OR = 1.32, 95% CI:1.00– 1.75) than those in non-academic hospitals. With regard to age, overall, older nurses were less likely to experience depersonalisation but more likely to have reduced personal achievement and nurses aged 25–40 weremore likely to intend to leave their jobs and the nursing profession than nurses aged under 25 (Table 2).

4. Discussion

This study assessed burnout and intention-to-leave the job and the nursing profession and analysed the relationships between these vari- ables and the work environment among ICU nurses. Based on our study findings we can make some important observations.

First, this study confirms that, after two years of the COVID-19 pan- demic, the rates of ICU nurses in Belgium at high risk of burnout and with the intention-to-leave the job and the profession are higher com- pared to similar data dating from before the pandemic (Bruyneel et al., 2021b; Van den Heede et al., 2019). Compared to studies in other countries, our results are also higher for risk of burnout and intention-to-leave the job and profession among ICU nurses (Gualano et al., 2021; Ramírez-Elvira et al., 2021; Xu et al., 2021). The impact of COVID-19 on the risk of burnout and intention-to-leave the job and the profession seems to be increased among ICUnurses compared to be- fore the pandemic and after the first wave (Bruyneel et al., 2021b; Butera et al., 2021; Vermeir et al., 2018). This can be explained by sev- eral elements including the lack of personal protective equipment, fear of being contaminated, increased mortality rates compared to nor- mal, successive waves, societal expectations, and accumulation over time (Bruyneel et al., 2021b; Khan et al., 2022; LeClaire et al., 2022; Moradi et al., 2021). Hospitals have implemented, in a variable and

Fig. 2. Prevalence of risk of burnout and intention-to-leave the job.

A. Bruyneel, N. Bouckaert, C. Maertens de Noordhout et al. / International Journal of Nursing Studies 137 (2023) 104385 5

heterogeneous way, several internal measures (e.g., psychologists, debriefing, meditation) which may have influenced the results with ICU nurses in Belgium (Paquay et al., 2022; Servotte et al., 2020; Van Den Heede et al., 2022).

Second, hospitals with a better working environment, even in times of pandemic, have better results for the risk of burnout and intention- to-leave the job and profession than hospitals with a worse environ- ment. This association has already been demonstrated in other studies and it seems that hospitals with a better work environment are better able to protect the wellbeing of nurses even in times of high stress (Alharbi et al., 2020; Kester et al., 2021; Lake et al., 2022; Sermeus et al., 2011; Van denHeede et al., 2019;Wei et al., 2018). Given these re- sults by subscale, hospital managers and health-related policies can prioritise their interventions to improve the nursingwork environment. Initiatives such as the “Magnet hospital accreditation” concept could be interesting to implement to foster a better working environment that includes factors such as structural empowerment of clinical staff, trans- formational leadership, exemplary and evidence-based professional practice, new knowledge, innovations and improvements (Lake et al., 2022; Sermeus et al., 2022a, 2022b). This accreditation requires, among other things, flat and decentralised organisational structures, empowerment of the frontline staff in decision-making and transforma- tional leadership (Sermeus et al., 2022a, 2022b). The analysis of the working environment in this study showed that the worst subscales were “participation in hospital affairs” and “adequate staffing and

resources”. If we compare the results on the work environment with a study conducted in general hospitalisation units in 2019, the results on the dimension “participation in hospital affairs” were much worse in this study (Van den Heede et al., 2019). It is likely that many factors related to the pandemic are responsible for this, such as the integration of head nurses into care, the difficulties of management present in ICUs, and an expectation of structural changes in the hospital's organisation. The dimension “foundations for quality of care” also decreased com- pared to 2019, as COVID-19 patients may contribute to a high workload in ICUs (Bruyneel et al., 2021a; Dellicour et al., 2021). In addition, Belgium experienced two initial major waves in COVID-19 hospital ad- missions to the ICU and high occupancy rates impacted themortality of COVID-19 and non-COVID-19 patients (Brinkman et al., 2022; Guillon et al., 2021; Payet et al., 2022; Taccone et al., 2020). On the other hand, the results for the dimension “physician-nurse relationships” were better, this can be explained by the close and permanent collabo- ration in the ICU, especially during the COVID-19 period (Chapman et al., 2021; Yan et al., 2021). The results for the “leadership” and “man- agement” dimensions are somewhat ambiguous with relatively good scores for some items and relatively poor scores on items like “praise” and “recognition”. Lack of praise and recognition from management and supervisors are likely to be priorities for investment in thework en- vironment, alongside the general lack of recognition of their role throughout the pandemic (e.g., from the general public, policy makers, the media) ensuring that people receive recognition in line with their

Fig. 3. General overview of nurses (strongly) agreeing with presence of positive work environment aspects at hospital site.

6 A. Bruyneel, N. Bouckaert, C. Maertens de Noordhout et al. / International Journal of Nursing Studies 137 (2023) 104385

personal definition of meaning, fulfilment, development, and advance- ment at every stage of their professional career.

Third, concerning the patient-to-nurse ratio, we found an independent association between only emotional exhaustion and depersonalisation with intention-to-leave the job. This observation had already been made for the risk of burnout and the patient-to-nurse ratio (Bruyneel et al., 2021b). In addition, the increased workload due to COVID-19 and the de- ployment of non-ICU nurses to compensate for it may also have increased thehigh risk of burnout among ICUnurses (Bruyneel et al., 2021a; Lucchini et al., 2021). A better patient-to-nurse ratio could therefore also influence some of the outcomes of intensive care nurses.

Finally, we found other independent factors for the risk of burnout and the intention-to-leave the job and the nursing profession. Nurses with a university degree had a higher risk of intention-to-leave the nursing profession, which can be explained by other professional op- portunities, butwill complicate the ability to hire and retain nurseman- agers who are required to have a university degree in Belgium (Phillips et al., 2021). ICUnurses between 25 and 40 years old had a higher risk of intention-to-leave their job and the nursing profession. Retention seems to be a real challenge in these age categories. Given these results, it seems essential to invest in retention policies to minimise turnover (such as improved job satisfaction and organisational commitments), especially as the expertise required to work in critical care and provide high quality care is scarce and requires a long period of education and on-the-job training (Butera et al., 2021; Lee et al., 2018; Xu et al., 2021). A specific strategy per age group seems to be useful with mea- sures to improve work-life balance (e.g., family-friendly working hours, adapting shifts with childcare needs, more financial incentives),

to stimulate younger nurses (e.g., professional development and project work), and to divide and adjust work circumstances to adapt the work to account for generational differences (e.g., less frequency of night shifts, physically demanding work) and paying attention to intergener- ational conflicts for older nurses (Clendon andWalker, 2012; Khan et al., 2019;Wieck et al., 2010). In addition, the COVID-19 pandemic certainly worsened the relationships between colleagues and traumatic/stressful experiences in the workplace, important factors for the intention-to- leave the job among ICU nurses (Khan et al., 2019).

4.1. Limits and strengths

This study has some limitations. The current study is cross-sectional in nature, which means that it allows us to describe the variation in nurse working environments and nurse outcomes across hospital sites and study the relationships between these concepts but the interpreta- tion of the relationships is limited because many other factors may in- fluence the results. Moreover, the results are self-reported and need to be further explored with additional, more objective measures (e.g., what proportion of nurses that report an intention-to-leave actu- ally leave their job, what is the link between high risk of burnout and clinically-diagnosed burnout, does the nurse-perceived quality of care correspond with objectively measured quality indicators). Although the response rate of 47.8% is relatively good, it is not at the level of sim- ilar studies conducted prior to the pandemic (from 40% to 80% depend- ing on the country) (Aiken et al., 2014; Van den Heede et al., 2019). Nevertheless, the response rate (both at the level of the hospitals and the nurses) can, given the specific context (e.g., the COVID crisis had

Table 2 Factors associated with burnout risk, intention-to-leave the profession, and work environment.

At high risk of emotional exhaustion

At high risk of depersonalization

At high risk of reduced personal accomplishment

At high risk of burnout (strict definition)

At high risk of burnout (broad definition)

Intention-to-leave job

Intention-to-leave nursing profession

OR [95%CI] OR [95%CI] OR [95%CI] OR [95%CI] OR [95%CI] OR [95%CI] OR [95%CI]

Region Flanders [REF] – – – – – – – Brussels 1.74 [1.28–2.37]*** 2.51 [1.74–3.61]*** 2.27 [1.57–3.29]*** 2.20 [1.37–3.52]** 2.88 [1.61–5.13]*** 1.79 [1.36–2.35]*** 1.98 [1.24–3.18]** Wallonia 1.99 [1.50–2.62]*** 2.62 [1.93–3.56]*** 2.42 [1.93–3.04]*** 2.61 [1.77–3.85]*** 2.93 [2.16–3.98]*** 1.71 [1.25–2.34]*** 2.26 [1.67–3.08]***

Hospital type General hospital [REF]

– – – – – – –

University hospital 1.42 [1.11–1.82]** 1.09 [0.76–1.55] 0.90 [0.64–1.26] 1.37 [0.83–2.26] 0.96 [0.64–1.44] 1.32 [1.00–1.75]* 1.34 [0.88–2.04] Hospital-level patient-to-nurse ratio Quartile 1 - lowest/best PTN [REF]

– – – – – – –

Quartile 2 1.18 [0.93–1.50] 1.35 [0.93–1.94] 0.90 [0.67–1.22] 1.20 [0.78–1.85] 1.15 [0.77–1.70] 1.37 [1.07–1.77]* 1.61 [1.14–2.27]** Quartile 3 0.90 [0.67–1.21] 1.09 [0.75–1.57] 0.83 [0.62–1.10] 1.08 [0.67–1.74] 0.92 [0.64–1.31] 0.94 [0.65–1.37] 1.07 [0.72–1.60] Quartile 4 - highest/worst PTN

1.53 [1.04–2.26]* 1.48 [1.03–2.13]* 0.80 [0.56–1.14] 1.56 [0.83–2.97] 1.10 [0.71–1.69] 1.46 [1.03–2.05]* 1.40 [0.85–2.32]

Nurse Working Environment Quartile 1 - lowest/worst score [REF]

– – – – – – –

Quartile 2 0.76 [0.54–1.07] 0.94 [0.67–1.31] 0.97 [0.75–1.27] 0.73 [0.46–1.15] 1.03 [0.73–1.47] 0.59 [0.43–0.81]** 0.83 [0.57–1.21] Quartile 3 0.53 [0.38–0.73]*** 0.75 [0.52–1.09] 0.83 [0.62–1.10] 0.62 [0.39–0.99]* 0.71 [0.50–1.01] 0.56 [0.40–0.79]*** 0.74 [0.53–1.02] Quartile 4 - highest/best score

0.28 [0.19–0.42]*** 0.41 [0.26–0.66]*** 0.64 [0.43–0.95]* 0.24 [0.11–0.54]*** 0.45 [0.30–0.68]*** 0.33 [0.23–0.46]*** 0.35 [0.23–0.55]***

Education Bachelor's degree [REF]

– – – – – – –

Specialised nurses 1.00 [0.81–1.25] 1.16 [0.88–1.53] 1.30 [0.99–1.73] 1.01 [0.69–1.47] 1.08 [0.85–1.38] 1.17 [0.91–1.51] 1.23 [0.92–1.65] University degree 1.34 [0.61–2.94] 1.09 [0.56–2.15] 1.31 [0.56–3.06] 1.14 [0.43–2.99] 0.94 [0.39–2.30] 2.12 [0.99–4.50] 3.13 [1.38–7.12]** Specialised nurses and university degree

1.12 [0.75–1.68] 1.34 [0.96–1.86] 1.20 [0.86–1.69] 1.10 [0.71–1.72] 1.03 [0.71–1.49] 1.45 [0.99–2.12] 1.95 [1.29–2.95]**

Nurses no bachelor 1.02 [0.45–2.30] 1.38 [0.71–2.66] 1.37 [0.72–2.57] 0.99 [0.41–2.36] 1.18 [0.59–2.37] 1.16 [0.57–2.35] 0.94 [0.38–2.31] Sex Female [REF] – – – – – – – Male 0.83 [0.69–0.99]* 1.68 [1.36–2.07]*** 0.86 [0.69–1.08] 1.11 [0.85–1.45] 0.95 [0.77–1.16] 1.14 [0.97–1.35] 1.26 [1.04–1.54]* Non-binary 0.67 [0.14–3.10] 1.11 [0.38–3.27] 0.40 [0.10–1.59] 0.39 [0.07–2.13] 0.65 [0.21–2.01] 3.80 [1.52–9.55]** 2.65 [0.94–7.49]

Age Under 25 [REF] – – – – – – – 25–29 1.24 [0.81–1.90] 1.27 [0.88–1.82] 1.63 [1.17–2.29]** 1.51 [0.95–2.41] 1.62 [1.09–2.40]* 2.12 [1.51–2.99]*** 1.86 [1.18–2.93]** 30–34 1.42 [0.94–2.15] 1.05 [0.68–1.63] 1.61 [1.07–2.40]* 1.26 [0.78–2.03] 1.79 [1.18–2.70]** 3.31 [2.25–4.85]*** 3.26 [1.90–5.59]*** 35–39 1.14 [0.80–1.63] 0.94 [0.62–1.44] 1.82 [1.27–2.62]** 1.49 [0.98–2.25] 1.28 [0.80–2.06] 2.48 [1.68–3.67]*** 2.91 [1.73–4.88]*** 40–44 1.01 [0.69–1.48] 0.63 [0.40–1.00]* 1.92 [1.22–3.03]** 1.48 [0.87–2.51] 0.99 [0.61–1.61] 2.21 [1.52–3.21]*** 2.27 [1.28–4.01]** 45–49 1.34 [0.94–1.92] 0.71 [0.47–1.08] 1.86 [1.23–2.80]** 1.34 [0.87–2.09] 1.43 [0.90–2.28] 1.56 [1.03–2.35]* 1.61 [0.89–2.92] 50–54 1.28 [0.81–2.01] 0.62 [0.39–0.97]* 1.49 [1.04–2.13]* 1.36 [0.79–2.37] 1.00 [0.64–1.56] 1.50 [1.00–2.23]* 1.56 [0.84–2.91] 55–59 0.76 [0.46–1.25] 0.35 [0.22–0.56]*** 1.73 [1.12–2.68]* 0.63 [0.34–1.14] 0.99 [0.59–1.65] 1.33 [0.81–2.19] 1.79 [0.96–3.31] 60 or over 0.73 [0.46–1.15] 0.20 [0.10–0.39]*** 1.20 [0.65–2.22] 0.30 [0.12–0.76]* 0.77 [0.38–1.55] 0.66 [0.36–1.23] 1.27 [0.70–2.28]

Legend: * p-value <0.05, ** p-value <0.01, *** p-value <0.001; value in bold is statistically significant; [ref] = reference.

A. Bruyneel, N. Bouckaert, C. Maertens de Noordhout et al. / International Journal of Nursing Studies 137 (2023) 104385 7

led to increased workload and higher absenteeism among ICU nurses), be considered relatively high. Note that nurses on sick leave were ex- cluded from the analyses, whichmay have influenced the results. In ad- dition, to allow hospital site-level analyses, we have excluded hospital sites based on arbitrary criteria (i.e., >75% response rate of ICU nursing staff per hospital site or at least 10 ICU nurses responding). Moreover, it is possible that nurses included in the sample were biased (e.g., more nurses with complaints responded) compared to non-responders. Yet, it is also possible that nurses not included in the survey were ab- sent because of problems such as burnout. The study period was de- fined by the commissioner of the study (the Belgian Health Care Knowledge Centre) to inform the Belgian Minister of Public Health about the state of wellbeing of ICU nurses after two years of pan- demic. After all, ICU nurses were identified as an important bottle- neck during the pandemic to ensure sufficient hospital capacity.

The timing has an important potential impact on the results. A fur- ther follow-up of the study will be required to evaluate how this wellbeing evolved over time (e.g., in periods with lower ICU capacity demands). Finally, other psychological factors (e.g., depression, anx- iety) and several internal measures of psychological support that were not analysed in this study could also have influenced the risk of burnout (Fischer et al., 2020).

5. Conclusions

This study found a high prevalence of high burnout risk and intention-to-leave the job and nursing profession after two years of the COVID-19 pandemic. However, there was statistically significant variability in our results by hospital site, which can be largely explained by a better work environment and more favourable patient-to-nurse

8 A. Bruyneel, N. Bouckaert, C. Maertens de Noordhout et al. / International Journal of Nursing Studies 137 (2023) 104385

ratios at some sites. This study highlighted the importance of monitor- ing the risk of burnout and turnover among ICU nurses during the COVID-19 pandemic or other stressful events and implementing inter- ventions to prevent and manage it, taking into account the risk factors identified in this study.

CRediT authorship contribution statement

Arnaud Bruyneel: Conceptualization, Visualisation, Writing - Review & Editing; Nicolas Bouckaert: Methodology, Software, Data Curation, Formal analysis; Charline Maertens de Noordhout: Valida- tion; Jens Detollenaere: Validation; Laurence Kohn: Validation; Magali Pirson: Validation and Supervision; Walter Sermeus: Valida- tion and Supervision; Koen Van den Heede: Validation, Supervision, Writing- Reviewing and Editing, Project administration.

Data availability statement

The authors prefer not to share the data from our study publicly. However, they are available from the authors.

Funding

This studywas funded by the BelgianHealth Care Knowledge Centre (KCE). The KCE is a federal institution which is financed by the National Institute for Health and Disability Insurance (NIHDI, RIZIV – INAMI), the Federal Public Service of health, food chain safety and environment, and the Federal Public Service of social security. The development of health services research studies is part of the legalmission of theKCE. Although the development of the studies is paid by the KCE budget, the sole mis- sion of the KCE is providing scientifically valid information.

Declaration of Competing Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

The authors acknowledge the contribution of a medical writer, Sandy Field, PhD, in the preparation of this manuscript.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi. org/10.1016/j.ijnurstu.2022.104385.

References

Aiken, L.H., Sermeus, W., Van den Heede, K., Sloane, D.M., Busse, R., McKee, M., Bruyneel, L., Rafferty, A.M., Griffiths, P., Moreno-Casbas, M.T., Tishelman, C., Scott, A., Brzostek, T., Kinnunen, J., Schwendimann, R., Heinen, M., Zikos, D., Sjetne, I.S., Smith, H.L., Kutney-Lee, A., 2012. Patient safety, satisfaction, and quality of hospital care: cross sectional surveys of nurses and patients in 12 countries in Europe and the United States. BMJ 344, e1717. https://doi.org/10.1136/bmj.e1717.

Aiken, L.H., Sloane, D.M., Bruyneel, L., Van den Heede, K., Griffiths, P., Busse, R., Diomidous, M., Kinnunen, J., Kózka, M., Lesaffre, E., McHugh, M.D., Moreno-Casbas, M.T., Rafferty, A.M., Schwendimann, R., Scott, P.A., Tishelman, C., van Achterberg, T., Sermeus, W., 2014. Nurse staffing and education and hospital mortality in nine European coun- tries: a retrospective observational study. Lancet 383, 1824–1830. https://doi.org/ 10.1016/S0140-6736(13)62631-8.

Aiken, L.H., Sermeus, W., Van den Heede, K., Sloane, D.M., Busse, R., McKee, M., Bruyneel, L., Rafferty, A.M., Griffiths, P., Moreno-Casbas, M.T., Tishelman, C., Scott, A., Brzostek, T., Kinnunen, J., Schwendimann, R., Heinen, M., Zikos, D., Sjetne, I.S., Smith, H.L., Kutney-Lee, A., 2012. Patient safety, satisfaction, and quality of hospital care: cross sectional surveys of nurses and patients in 12 countries in Europe and the United States. BMJ 344, e1717. https://doi.org/10.1136/bmj.e1717.

Alharbi, A.A., Dahinten, V.S., MacPhee, M., 2020. The relationships between nurses’ work environments and emotional exhaustion, job satisfaction, and intent to leave among nurses in Saudi Arabia. J. Adv. Nurs. 76, 3026–3038. https://doi.org/10.1111/jan. 14512.

Atroszko, P.A., Demetrovics, Z., Griffiths, M.D., 2020. Work addiction, obsessive- compulsive personality disorder, burn-out, and global burden of disease: implications from the ICD-11. IJERPH 17, 660. https://doi.org/10.3390/ijerph17020660.

Boutou, A., Pitsiou, G., Sourla, E., Kioumis, I., 2019. Burnout syndrome among emergency medicine physicians: an update on its prevalence and risk factors. Eur. Rev. Med. Pharmacol. Sci. 23, 9058–9065. https://doi.org/10.26355/eurrev_201910_19308.

Brinkman, S., Termorshuizen, F., Dongelmans, D.A., Bakhshi-Raiez, F., Arbous, M.S., de Lange, D.W., de Keizer, N.F., Verbiest, D.P., te Velde, L.F., van Driel, E.M., Rijpstra, T., Elbers, P.H.J., Houwink, A.P.I., Georgieva, L., Verweij, E., de Jong, R.M., van Iersel, F.M., Koning, T.J.J., Rengers, E., Kusadasi, N., Erkamp, M.L., van den Berg, R., Jacobs, C.J.M.G., Epker, J.L., Rijkeboer, A.A., de Bruin, M.T., Spronk, P., Draisma, A., Versluis, D.J., van den Berg, A.E., Mos, M.V., Lens, J.A., Pruijsten, R.V., Kieft, H., Rozendaal, J., Nooteboom, F., Boer, D.P., Janssen, I.T.A., van Gulik, L., Koetsier, M.P., Silderhuis, V.M., Schnabel, R.M., Drogt, I., de Ruijter, W., Bosman, R.J., Frenzel, T., Urlings-Strop, L.C., Dijkhuizen, A., Hené, I.Z., de Meijer, A.R., Holtkamp, J.W.M., Postma, N., Bindels, A.J.G.H., Wesselink, R.M.J., van Slobbe-Bijlsma, E.R., van der Voort, P.H.J., Eikemans, B.J.W., Mehagnoul-Schipper, D.J., Gommers, D., Lutisan, J.G., Hoeksema, M., Barnas, M.G.W., Festen-Spanjer, B., van Lieshout, M., Gritters, N.C., van Tellingen, M., Brunnekreef, G.B., Vandeputte, J., Dormans, T.P.J., Hoogendoorn, M.E., de Graaff, M., Moolenaar, D., Reidinga, A.C., Spijkstra, J.J., de Waal, R., 2022. Comparison of outcome and characteristics between 6343 COVID-19 patients and 2256 other community- acquired viral pneumonia patients admitted to Dutch ICUs. J. Crit. Care 68, 76–82. https://doi.org/10.1016/j.jcrc.2021.12.007.

Bruyneel, A., Smith, P., 2021. Comparison of the prevalence of burnout risk between ICU and non-ICU nurses during the COVID-19 outbreak in French-speaking Belgium. In- tensive Crit. Care Nurs. 66, 103086. https://doi.org/10.1016/j.iccn.2021.103086.

Bruyneel, A., Tack, J., Droguet, M., Maes, J., Wittebole, X., Miranda, D.R., Pierdomenico, L.D., 2019. Measuring the nursing workload in intensive care with the Nursing Activities Score (NAS): a prospective study in 16 hospitals in Belgium. J. Crit. Care 54, 205–211. https://doi.org/10.1016/j.jcrc.2019.08.032.

Bruyneel, A., Gallani, M.-C., Tack, J., d’Hondt, A., Canipel, S., Franck, S., Reper, P., Pirson, M., 2021a. Impact of COVID-19 on nursing time in intensive care units in Belgium. Inten- sive Crit. Care Nurs. 62, 102967. https://doi.org/10.1016/j.iccn.2020.102967.

Bruyneel, A., Smith, P., Tack, J., Pirson, M., 2021b. Prevalence of burnout risk and factors associated with burnout risk among ICU nurses during the COVID-19 outbreak in French speaking Belgium. Intensive Crit. Care Nurs. 103059. https://doi.org/10. 1016/j.iccn.2021.103059.

Butera, S., Brasseur, N., Filion, N., Bruyneel, A., Smith, P., 2021. Prevalence and associated factors of burnout risk among intensive care and emergency nurses before and dur- ing the coronavirus disease 2019 pandemic: a cross-sectional study in Belgium. J. Emerg. Nurs. 47, 879–891. https://doi.org/10.1016/j.jen.2021.08.007.

Chapman, L.B., Kopp, K.E., Petty, M.G., Hartwig, J.L.A., Pendleton, K.M., Langer, K., Meiers, S.J., 2021. Benefits of collaborative patient care rounds in the intensive care unit. In- tensive Crit. Care Nurs. 63, 102974. https://doi.org/10.1016/j.iccn.2020.102974.

Cho, E., Sloane, D.M., Kim, E.-Y., Kim, S., Choi, M., Yoo, I.Y., Lee, H.S., Aiken, L.H., 2015. Ef- fects of nurse staffing, work environments, and education on patient mortality: an observational study. Int. J. Nurs. Stud. 52, 535–542. https://doi.org/10.1016/j. ijnurstu.2014.08.006.

Clendon, J., Walker, L., 2012. ‘Being young’: a qualitative study of younger nurses’ experi- ences in the workplace: younger nurses’ experiences in the workplace. Int. Nurs. Rev. 59, 555–561. https://doi.org/10.1111/j.1466-7657.2012.01005.x.

Dellicour, S., Linard, C., Van Goethem, N., Da Re, D., Artois, J., Bihin, J., Schaus, P., Massonnet, F., Van Oyen, H., Vanwambeke, S.O., Speybroeck, N., Gilbert, M., 2021. In- vestigating the drivers of the spatio-temporal heterogeneity in COVID-19 hospital in- cidence—Belgium as a study case. Int. J. Health Geogr. 20, 29. https://doi.org/10.1186/ s12942-021-00281-1.

Dion, G., Tessier, R., 1994. Validation de la traduction de l’Inventaire d’épuisement professionnel de maslach et Jackson. Can. J. Behav. Sci. 26, 210–227. https://doi.org/ 10.1037/0008-400X.26.2.210.

Embriaco, N., Papazian, L., Kentish-Barnes, N., Pochard, F., Azoulay, E., 2007. Burnout syn- drome among critical care healthcare workers. Curr. Opin. Crit. Care 13, 482–488. https://doi.org/10.1097/MCC.0b013e3282efd28a.

Fischer, R., Mattos, P., Teixeira, C., Ganzerla, D.S., Rosa, R.G., Bozza, F.A., 2020. Association of Burnout with Depression and Anxiety in critical care clinicians in Brazil. JAMA Netw. Open 3, e2030898. https://doi.org/10.1001/jamanetworkopen.2020.30898.

Gualano, M.R., Sinigaglia, T., Lo Moro, G., Rousset, S., Cremona, A., Bert, F., Siliquini, R., 2021. The burden of burnout among healthcare professionals of intensive care units and emergency departments during the COVID-19 pandemic: a systematic review. IJERPH 18, 8172. https://doi.org/10.3390/ijerph18158172.

Guillon, A., Laurent, E., Duclos, A., Godillon, L., Dequin, P.-F., Agrinier, N., Kimmoun, A., Grammatico-Guillon, L., 2021. Case fatality inequalities of critically ill COVID-19 pa- tients according to patient-, hospital- and region-related factors: a French nationwide study. Ann. Intensive Care 11, 127. https://doi.org/10.1186/s13613-021-00915-4.

Guixia, L., Hui, Z., 2020. A study on burnout of nurses in the period of COVID-19. PBS 9, 31. https://doi.org/10.11648/j.pbs.20200903.12.

Hu, D., Kong, Y., Li, W., Han, Q., Zhang, X., Zhu, L.X., Wan, S.W., Liu, Z., Shen, Q., Yang, J., He, H.-G., Zhu, J., 2020. Frontline nurses’ burnout, anxiety, depression, and fear statuses and their associated factors during the COVID-19 outbreak in Wuhan, China: a large-scale cross-sectional study. EClinicalMedicine 100424. https://doi.org/10. 1016/j.eclinm.2020.100424.

Jun, J., Ojemeni, M.M., Kalamani, R., Tong, J., Crecelius, M.L., 2021. Relationship between nurse burnout, patient and organizational outcomes: systematic review. Int. J. Nurs. Stud. 119, 103933. https://doi.org/10.1016/j.ijnurstu.2021.103933.

Kester, K., Pena, H., Shuford, C., Hansen, C., Stokes, J., Brooks, K., Bolton, T., Ornell, A., Parker, P., Febre, J., Andrews, K., Flynn, G., Ruiz, R., Evans, T., Kettle, M., Minter, J., Granger, B., 2021. Implementing AACN’s healthy work environment framework in

A. Bruyneel, N. Bouckaert, C. Maertens de Noordhout et al. / International Journal of Nursing Studies 137 (2023) 104385 9

an intensive care unit. Am. J. Crit. Care 30, 426–433. https://doi.org/10.4037/ ajcc2021108.

Khan, N., Jackson, D., Stayt, L., Walthall, H., 2019. Factors influencing nurses’ intentions to leave adult critical care settings: what factors influence nurses’ intentions to leave the adult critical care areas? Nurs. Crit. Care 24, 24–32. https://doi.org/10.1111/nicc. 12348.

Khan, Y., Bruyneel, A., Smith, P., 2022. Determinants of the risk of burnout among nurses during the first wave of the COVID-19 pandemic in Belgium: a cross-sectional study. J. Nurs. Manag. 30, 1125–1135. https://doi.org/10.1111/jonm.13624.

Lake, E.T., 2002. Development of the practice environment scale of the nursing work index. Res. Nurs. Health 25, 176–188. https://doi.org/10.1002/nur.10032.

Lake, E.T., Riman, K.A., Lee, C., 2022. The association between hospital nursing resource profiles and nurse and patient outcomes. J. Nurs. Manag., jonm.13553. https://doi. org/10.1111/jonm.13553.

Lake, E.T., Riman, K.A., Sloane, D.M., 2020. Improved work environments and staffing lead to less missed nursing care: a panel study. J. Nurs. Manag. 28, 2157–2165. https://doi. org/10.1111/jonm.12970.

LeClaire, M., Poplau, S., Linzer, M., Brown, R., Sinsky, C., 2022. Compromised integrity, burnout, and intent to leave the job in critical care nurses and physicians. Crit. Care Explor. 4, e0629. https://doi.org/10.1097/CCE.0000000000000629.

Lee, S.E., Vincent, C., Dahinten, V.S., Scott, L.D., Park, C.G., Dunn Lopez, K., 2018. Effects of individual nurse and hospital characteristics on patient adverse events and quality of care: a multilevel analysis: patient safety quality of care. J. Nurs. Scholarsh. 50, 432–440. https://doi.org/10.1111/jnu.12396.

Lucchini, A., Bambi, S., Bruyneel, A., 2021. Redefining “Critical care”: fromwhere intensive care unit beds are located to patients’ status. Intensive Crit. Care Nurs. 103188. https://doi.org/10.1016/j.iccn.2021.103188.

Maslach, C., Jackson, S.E., Leiter, M.P., Schaufeli, W., Schwab, R., 1986. Maslach Burnout In- ventory Manual, General Survey, Human Services Survey, Educators Survey Ad Scor- ing Guides. Menlo Park, CA Mind Garden.

Maslach, C., Jackson, S.E., Leiter, M.P., 1996. Maslach Burnout Inventory Consulting Psy- chologists Press.

Molenberghs, G., Faes, C., Verbeeck, J., Deboosere, P., Abrams, S., Willem, L., Aerts, J., Theeten, H., Devleesschauwer, B., Sierra, N.B., Renard, F., Herzog, S., Lusyne, P., Van der Heyden, J., Van Oyen, H., Van Damme, P., Hens, N., 2020. Belgian COVID-19 mor- tality, excess deaths, number of deaths per million, and infection fatality rates (9 March — 28 June 2020) (preprint). Epidemiology. https://doi.org/10.1101/2020.06. 20.20136234.

Moradi, Y., Baghaei, R., Hosseingholipour, K., Mollazadeh, F., 2021. Challenges experienced by ICU nurses throughout the provision of care for COVID-19 patients: a qualitative study. J. Nurs. Manag. 29, 1159–1168. https://doi.org/10.1111/jonm.13254.

Natalia, Y.A., Faes, C., Neyens, T., Molenberghs, G., 2022. The COVID-19 wave in Belgium during the fall of 2020 and its association with higher education. PLoS ONE 17, e0264516. https://doi.org/10.1371/journal.pone.0264516.

Paquay, M., Dubois, N., Diep, A.N., Graas, G., Sassel, T., Piazza, J., Servotte, J.-C., Ghuysen, A., 2022. “Debriefing and organizational lessons learned” (DOLL): a qualitative study to develop a classification framework for reporting clinical debriefing results. Front. Med. 9, 882326. https://doi.org/10.3389/fmed.2022.882326.

Payet, C., Polazzi, S., Rimmelé, T., Duclos, A., 2022. Mortality among noncoronavirus dis- ease 2019 critically ill patients attributable to the pandemic in France. Crit. Care Med. 50, 138–143. https://doi.org/10.1097/CCM.0000000000005215.

Pérez-Francisco, D.H., Duarte-Clíments, G., del Rosario-Melián, J.M., Gómez-Salgado, J., Romero-Martín, M., Sánchez-Gómez, M.B., 2020. Influence of workload on primary care nurses’ health and burnout, patients’ safety, and quality of care: integrative re- view. Healthcare 8, 12. https://doi.org/10.3390/healthcare8010012.

Phillips, J., Palokas, M., Christian, R., Harris, J., Hinton, E., 2021. Emotional intelligence in nurse managers as it relates to staff nurse job satisfaction and retention: a scoping re- view. JBI Evid. Synth. 19, 3238–3267. https://doi.org/10.11124/JBIES-20-00299.

Ramírez-Elvira, S., Romero-Béjar, J.L., Suleiman-Martos, N., Gómez-Urquiza, J.L., Monsalve-Reyes, C., Cañadas-De la Fuente, G.A., Albendín-García, L., 2021. Prevalence, risk factors and burnout levels in intensive care unit nurses: a systematic review and meta-analysis. IJERPH 18, 11432. https://doi.org/10.3390/ijerph182111432.

Rotenstein, L.S., Torre, M., Ramos, M.A., Rosales, R.C., Guille, C., Sen, S., Mata, D.A., 2018. Prevalence of burnout among physicians: a systematic review. JAMA 320, 1131. https://doi.org/10.1001/jama.2018.12777.

Sermeus, W., Aiken, L.H., Ball, J., Bridges, J., Bruyneel, L., Busse, R., De Witte, H., Dello, S., Drennan, J., Eriksson, L.E., Griffiths, P., Kohnen, D., Köppen, J., Lindqvist, R., Maier, C.B., McHugh, M.D., McKee, M., Rafferty, A.M., Schaufeli, W.B., Sloane, D.M., Alenius, L.S., Smith, H., 2022. A workplace organisational intervention to improve hospital nurses’ and physicians’ mental health: study protocol for the Magnet4Europe wait list cluster randomised controlled trial. BMJ Open 12, e059159. https://doi.org/10. 1136/bmjopen-2021-059159.

Sermeus, W., Aiken, L.H., Van den Heede, K., Rafferty, A.M., Griffiths, P., Moreno-Casbas, M.T., Busse, R., Lindqvist, R., Scott, A.P., Bruyneel, L., Brzostek, T., Kinnunen, J., Schubert, M., Schoonhoven, L., Zikos, D., RN4CAST consortium, 2011. Nurse forecast- ing in Europe (RN4CAST): rationale, design and methodology. BMC Nurs. 10 (6). https://doi.org/10.1186/1472-6955-10-6.

Sermeus, W., Aiken, L.H., Ball, J., Bridges, J., Bruyneel, L., Busse, R., De Witte, H., Dello, S., Drennan, J., Eriksson, L.E., Griffiths, P., Kohnen, D., Köppen, J., Lindqvist, R., Maier, C.B., McHugh, M.D., McKee, M., Rafferty, A.M., Schaufeli, W.B., Sloane, D.M., Alenius, L.S., Smith, H., 2022. A workplace organisational intervention to improve hospital nurses’ and physicians’ mental health: study protocol for the Magnet4Europe wait list cluster randomised controlled trial. BMJ Open 12, e059159. https://doi.org/10. 1136/bmjopen-2021-059159.

Servotte, J.-C., Welch-Horan, T.B., Mullan, P., Piazza, J., Ghuysen, A., Szyld, D., 2020. Devel- opment and implementation of an end-of-shift clinical debriefing method for emer- gency departments during COVID-19. Adv. Simul. 5, 32. https://doi.org/10.1186/ s41077-020-00150-0.

Shah, A., Wyatt, M., Gourneau, B., Shih, G., De Ruyter, M., 2019. Emotional exhaustion among anesthesia providers at a tertiary care center assessed using the MBI burnout survey. Psychol. Health Med. 24, 620–624. https://doi.org/10.1080/13548506.2018. 1546019.

Simonetti, M., Cerón, C., Galiano, A., Lake, E.T., Aiken, L.H., 2021. Hospital work environ- ment, nurse staffing and missed care in Chile: a cross-sectional observational study. J. Clin. Nurs. jocn.16068. https://doi.org/10.1111/jocn.16068.

Swiger, P.A., Patrician, P.A., Raju, D., Breckenridge-Sproat, S., Loan, L.A., Miltner, R.S.Susie, 2017. The Practice Environment Scale of the NursingWork Index: an updated review and recommendations for use. Int. J. Nurs. Stud. 74, 76–84. https://doi.org/10.1016/j. ijnurstu.2017.06.003.

Taccone, F.S., Vangoethem, N., Depauw, R., Wittebole, X., Blot, K., Vanoyen, H., Lernout, T., Montourcy, M., Meyfroidt, G., Vanbeckhoven, D., 2020. The role of organizational characteristics on the outcome of COVID-19 patients admitted to the ICU in Belgium. Lancet Reg. Health Eur. 100019. https://doi.org/10.1016/j.lanepe.2020. 100019.

Van den Heede, K., Bruyneel, L., Beeckmans, D., Boon, N., Bouckaert, N., Cornelis, J., Dossche, D., Van de Voorde, C., Sermeus, W., 2019. Safe nurse staffing levels in acute hospitals. Health Services Research (HSR)A]KCE Reports 325, D/2019/10.273/ 75.

Van den Heede, K., Bouckaert, N., Detollenaere, J., Kohn, K., Maertens de Noordhout, C., Vanhooreweghe, J., Bruyneel, A., Sermeus, W., 2022. Nurse staffing on Belgian inten- sive care units: the impact of two years of COVID-19 pandemic. Health Services Re- search (HSR) Brussels: Belgian Health Care Knowledge Centre (KCE). (No. KCE Reports 353. D/2022/10.273/24.).

Vandenbroeck, S., Van Gerven, E., De Witte, H., Vanhaecht, K., Godderis, L., 2017. Burnout in Belgian physicians and nurses. Occup. Med. 67, 546–554. https://doi.org/10.1093/ occmed/kqx126.

Van Gerven, E., Vander Elst, T., Vandenbroeck, S., Dierickx, S., Euwema, M., Sermeus, W., De Witte, H., Godderis, L., Vanhaecht, K., 2016. Increased risk of burnout for physi- cians and nurses involved in a patient safety incident. Med. Care 54, 937–943. https://doi.org/10.1097/MLR.0000000000000582.

Vermeir, P., Blot, S., Degroote, S., Vandijck, D., Mariman, A., Vanacker, T., Peleman, R., Verhaeghe, R., Vogelaers, D., 2018. Communication satisfaction and job satisfaction among critical care nurses and their impact on burnout and intention to leave: a questionnaire study. Intensive Crit. Care Nurs. 48, 21–27. https://doi.org/10.1016/j. iccn.2018.07.001.

Welp, A., Rothen, H.U., Massarotto, P., Manser, T., 2019. Teamwork and clinician burnout in Swiss intensive care: the predictive role of workload, and demographic and unit characteristics. Swiss Med. Wkly. https://doi.org/10.4414/smw.2019.20033.

Wei, H., Sewell, K.A., Woody, G., Rose, M.A., 2018. The state of the science of nurse work environments in the United States: a systematic review. Int. J. Nurs. Sci. 5, 287–300. https://doi.org/10.1016/j.ijnss.2018.04.010.

Wieck, K.L., Dols, J., Landrum, P., 2010. Retention priorities for the intergenerational nurse workforce. Nurs. Forum 45, 7–17. https://doi.org/10.1111/j.1744-6198.2009.00159.x.

Woo, T., Ho, R., Tang, A., Tam, W., 2020. Global prevalence of burnout symptoms among nurses: a systematic review and meta-analysis. J. Psychiatr. Res. 123, 9–20. https:// doi.org/10.1016/j.jpsychires.2019.12.015.

Xu, G., Zeng, X., Wu, X., 2021. Global prevalence of turnover intention among intensive care nurses: a meta-analysis. Nurs. Crit. Care nicc.12679. https://doi.org/10.1111/ nicc.12679.

Yan, C., Zhang, X., Gao, C., Wilfong, E., Casey, J., France, D., Gong, Y., Patel, M., Malin, B., Chen, Y., 2021. Collaboration structures in COVID-19 critical care: retrospective net- work analysis study. JMIR Hum. Factors 8, e25724. https://doi.org/10.2196/25724.