Research Assignment

International Journal of Nursing Studies 91 (2019) 47–59

Characteristics of successful interventions to reduce turnover and increase retention of early career nurses: A systematic review

Judy Brooka,*, Leanne Aitkena, Rebecca Webba,b, Julie MacLarena, Debra Salmona

a School of Health Sciences City, University of London, Northampton Square, London, EC1V 0HB, UK b Faculty of Health and Medical Sciences, Section of Clinical Medicine and Ageing, Leggett Building, Daphne Jackson Road, University of Surrey, Guildford, Surrey, GU2 7XH, UK

A R T I C L E I N F O

Article history: Received 12 June 2018 Received in revised form 5 November 2018 Accepted 6 November 2018

Keywords: Attrition Intervention Nurse retention Nurse turnover Nurse workforce Transition Preceptorship Residency Internship Mentorship

A B S T R A C T

Background: Nurse shortages have been identified as central to workforce issues in healthcare systems globally and although interventions to increase the nursing workforce have been implemented, nurses leaving their roles, particularly in the first year after qualification, present a significant barrier to building the nurse workforce. Objective: To evaluate the characteristics of successful interventions to promote retention and reduce turnover of early career nurses. Design: This is a systematic review Data sources: Online databases including Academic Search Complete, Medline, Health Policy reference Centre, EMBASE, Psychinfo, CINAHL and the Cochran Library were searched to identify relevant publications in English published between 2001 and April 2018. Studies included evaluated an intervention to increase retention or reduce turnover and used turnover or retention figures as a measure. Review methods: The review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines. Studies were quality-assessed using the Joanna Briggs Institute Critical Appraisal tools for Quasi Experimental and Randomised Controlled Trials. Retention/ turnover data were used to guide the comparison between studies and appropriate measures of central tendency and dispersion were calculated and presented, based on the normality of the data. Results: A total of 11, 656 papers were identified, of which 53 were eligible studies. A wide variety of interventions and components within those interventions were identified to improve nurse retention. Promising interventions appear to be either internship/residency programmes or orientation/transition to practice programmes, lasting between 27–52 weeks, with a teaching and preceptor and mentor component. Conclusions: Methodological issues impacted on the extent to which conclusions could be drawn, even though a large number of studies were identified. Future research should focus on standardising the reporting of interventions and outcome measures used to evaluate these interventions and carrying out further research with rigorous methodology. Clinical practice areas are recommended to assess their current interventions against the identified criteria to guide development of their effectiveness. Evaluations of cost-effectiveness are considered an important next step to maximise return on investment.

© 2019 Elsevier Ltd. All rights reserved.

Contents lists available at ScienceDirect

International Journal of Nursing Studies

journal homepage: www.elsevier.com/ijns

* Corresponding author. E-mail addresses: [email protected] (J. Brook), [email protected]

(L. Aitken), [email protected] (R. Webb), [email protected] (J. MacLaren), [email protected] (D. Salmon).

https://doi.org/10.1016/j.ijnurstu.2018.11.003 0020-7489/© 2019 Elsevier Ltd. All rights reserved.

What is already known about the topic?

� Nurse turnover and retention impact on healthcare workforce development and delivery of clinical care

� A critical time for turnover of nurses is during the first year after qualification

� Interventions to reduce turnover and increase retention have been implemented but there is little understanding of which characteristics of these interventions contribute to their success

48 J. Brook et al. / International Journal of Nursing Studies 91 (2019) 47–59

What this paper adds

� Promising interventions appear to be either internship/residen- cy programmes or orientation/transition to practice pro- grammes, lasting between 27–52 weeks, with a teaching and preceptor and mentor component.

� These characteristics can be used as a foundation for developing or refining transition programmes for early career nurses so that maximum return on investment is achieved.

1. Introduction and background

The sustainability of the health workforce is a global imperative (World Health Organisation (WHO), 2017) and nurse shortages have been identified as central to workforce issues in healthcare systems in both high and low income countries (Van der Heed and Aiken, 2013; Li et al., 2010). Although campaigns to increase the nursing workforce have been implemented in several countries such as Australia, United Kingdom and China (Health Workforce Australia, 2014a; Department of Health, 2017; Ministry of Health, 2005), nurses leaving their roles, particularly in the first year after qualification (Zhang et al., 2016), present a significant barrier to building the nurse workforce. This paper aims to systematically review interventions to reduce attrition and promote retention of early career nurses, with particular exploration of the character- istics of effective interventions.

Turnover of nurses is a widely but inconsistently used measure for healthcare organisation analysis. It can encompass nurses who leave a particular post, an organisation or the profession entirely. Turnover may also be voluntary, or involuntary for example due to performance management. These inconsistencies result in varia- tion in reporting of turnover rates (Kovner et al., 2014; Li and Jones, 2013). Turnover in any form draws on healthcare organisation funding as it is expensive to recruit new nurses into the global or organisational workforce. It is therefore valuable to consider how turnover can be reduced and retention of nurses increased, particularly voluntary turnover where nurses are lost to an organisation or the profession.

Although less is known about the adequacy of the nursing workforce in developing economies, surveys indicate that it is a global issue (WHO, 2010) and particularly acute in rural and remote areas where it has major consequences for accessibility to healthcare (Mbemba et al., 2013). In countries with developed healthcare systems, the extent and impact of nurse turnover has also been noted. Examples include the USA, where a survey of 136 organisations in 2017 indicated bedside registered nurse turnover averaged 14.9%, with an annual hospital financial loss of $5.13 M - $7.86 M and 80% of hospitals reporting a registered nurse vacancy rate greater than 5% (Nursing Solutions Inc, 2017). Health Workforce Australia (2014a) predicted a shortfall of approxi- mately 85,000 nurses by 2025, and 123,000 nurses by 2030 due to demand significantly exceeding supply in the short to medium term. In the UK, the picture is similar, with a shortfall of 22,000 nurses in England in 2015 and a projected shortfall of 10–15% of the nursing workforce by 2020 (Buchan et al., 2017). Within the UK in Scotland specifically, the shortfall of nurses has resulted in greater spend on temporary nurses, impacting on budgets (Audit Scotland, 2017) and a similar picture is emerging in Wales and Northern Ireland (Royal College of Nursing, 2018). In Europe the workforce crisis was extensively explored by the European NEXT (Nurses Early Exit) -study (Hasselhorn et al., 2003). In addition, the Registered Nurse Forecasting (RN4CAST) project indicated that 20–50% of nurses intended to leave their jobs across a sample of eight countries including Belgium, France, Germany, Greece, Ireland, Poland, Spain and Switzerland. This has significant

impacts on the provision of safe patient care and patient mortality rates (Zander et al., 2016).

The nursing workforce has consequently become central to national and global health policy (WHO, 2017) and recommenda- tions focus on leadership and productivity (Health workforce Australia, 2014b), education (Zander et al., 2016) and retention (Department of Health, 2017; Good Governance Institute, 2015) to mitigate risk of the forecast nursing shortage. There has been particular scrutiny of the effectiveness of interventions to both reduce turnover of nurses between organisations and increase retention of nurses in the profession (Mbemba et al., 2013; The Association of UK University Hospitals, 2017). In England, this has led to a national nurse retention initiative to secure a sustainable NHS workforce for the future (NHS Improvement, 2017).

Although current nurse workforce issues are particularly acute, this is not a new concern and the nursing literature is replete with examples of initiatives to reduce turnover and increase retention in high and medium income economies. There has been a particular focus on those in the early stages of their careers (Whitehead et al., 2013; Rush et al., 21013; Al-Dossary et al., 2014) as the transition from student to registered nurse is seen as a critical time as turnover is high in the first year following qualification. Broadly speaking, these initiatives fall in to six categories (McDonald and Smith, 2012; Edwards et al., 2015): preceptorship, mentoring programmes, residency programmes and internships, externships, transition or orientation to practice programmes and clinical ladder programmes (Table 1).

In reality, there is substantial overlap between different types of interventions to support transition from student nurse to qualified practitioner and encourage retention of nurses; individual pro- grammes may not fit neatly into one category. In addition, there is significant variation between the content and quality of the programmes within each category (Van Camp and Chappy, 2017; McDonald and Ward-Smith, 2012). Edwardset al. (2015) suggest that by offering a transition to practice programme of any description, the organisationisindicatingtheimportanceattachedtonewlyqualified nurses and this alone is enough to positively influence recruitment and retention, especially if the organisation is perceived to be investing in the workforce to a greater extent than competitors.

There is a growing body of literature that explores the effectiveness of strategies to support such transitions (Al-Dossary et al., 2014; Edwards et al., 2015; Irwin et al., 2018; Ke et al., 2017). Effectiveness may manifest as an increase in job satisfaction, confidence, or competence, and alternatively a decrease in voluntary turnover. However, there is a significant knowledge gap in the literature about the characteristics that influence the successful outcomesofinterventions.Healthcareorganisationsare implement- ing interventions in an attempt to reduce turnover and increase retention but the evidence base does not offer sufficient guidance about which elements of an intervention are the most influential in their success. Ten years ago, Salt et al. (2008) conducted a systematic review of healthcare organisation retention interventions, however this did not explore the components or characteristics of the interventions. This systematic review is therefore the first to identify the characteristicsofsuccessful interventions to reduceturnoverand increase retention of early career nurses.

2. Method

Initial scoping searches were conducted, including Google Scholar and the Cochrane Library to identify similar and relevant systematic reviews that had already been conducted. Subsequent literature searches and study selection for this systematic review were conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines (Moher et al., 2009; http://prisma-statement.org/).

Table 1 Interventions to increase retention of early career nurses.

Intervention Description Characteristics

Preceptorship (Irwin et al., 2018; Ke et al., 2017; Whitehead et al., 2013)

� Designed to support transition from student to qualified practi- tioner, originated in the USA

� In the UK seen as a way of socializing the new nurse in to the profession

� Newly qualified nurse is allocated a qualified nurse preceptor and regular meetings are arranged

� May involve formal teaching, supervision and competency assessment

� Usually of defined length but can vary from a few weeks to up to two years in duration

� Relationship between preceptor and preceptee is fundamental for psychological support, knowledge exchange and role modelling

Mentoring Programmes (Zhang et al., 2016)

� Draws on a mentoring relationship to support transition into the workplace

� Relationship between mentor and mentee may be enduring and nurturing

� Vary widely in format and structure � May include dyad models, peer mentoring, group mentoring,

constellation models or online distance mentoring � Mentors may be allocated to mentees or may rely on self-selection

or incentives to motivate tenacity in the relationship

Residency Programmes and Internships (Al-Dossary et al., 2014; Anderson et al., 2009)

� Predominantly established in the USA, Canada and Australia in response to retention challenges in the 1980s

� Aim to develop and consolidate clinical reasoning and leadership skills of new graduates

� Contractual arrangement between the new graduate nurse and the employer

� Usually lasts between six months and one year � Delivered in partnership between academia and practice � Involve structured teaching linked to clinical immersion and a

mentoring relationship

Externships (Friday et al., 2015)

� Opportunities for student nurses to undertake experiential learning in a new clinical area on a short placement

� Usually lasts no longer than a semester � Opportunity for student to apply theoretical knowledge, experience

the clinical environment and understand workplace opportunities � Valuable in nursing programmes that are weighted in favour of

theory rather than practical placements

Orientation and Transition to Practice Programmes (Rush et al., 2013)

� Designed to support the socialisation of new nurses in to the workplace

� May be generalised to promote transition in to the organisation or more specialised, to facilitate development of skills specific to a particular clinical area

� Involve formal teaching and a mentor or preceptor relationship � Often shorter in duration

Clinical Ladder or Advancement Programmes (Drenkard and Swartwout, 2005)

� Organisational structures that allow nurses to gain promotion and recognition for their developing skill and achievement

� Implemented as a means of enhancing the quality of nursing care and as a mechanism for enhancing nurse satisfaction

� Offer clarity about professional development and promotion � May involve competency assessment � Usually involve remuneration for achievement

J. Brook et al. / International Journal of Nursing Studies 91 (2019) 47–59 49

2.1. Eligibility criteria

Studies were required to meet the following criteria to be eligible for inclusion in the systematic review: peer-reviewed journal articles published in the English language between 2001 and 2017. A 2001 parameter was applied to ensure currency of findings. Studies were included in the review if they reported primary research on interventions designed to reduce the attrition or increase retention of the nursing workforce, with data on attrition/retention/turnover rates reported, on at least two occasions (e.g. pre-post data, time series design, RCT, equivalent or non-equivalent control) for comparison.

2.2. Search strategy

Searches were conducted to identify peer reviewed pub- lications using key search terms. Search terms were con- structed in collaboration with an academic librarian and were based on a PICO structured question. Only P (population) and O (outcome) were used to construct the searches as C (compari- son) was not relevant and I (intervention) proved too limiting. Three groups of search terms were used and then combined using the Boolean operators AND and OR including: (retain*, retention, attrition, leav*, turnover, quit, loyalty) AND (staff, personnel, employee, workforce) AND nurs*. All terms were used in each database. Limits were imposed around date of publication and publication language in accordance with the eligibility criteria. Detail of a full search is presented in Supplementary table S7.

2.3. Data sources

Online databases searched included Academic Search Com- plete, Medline, Health Policy reference Centre, EMBASE, Psychinfo, and CINAHL. The reference lists of relevant systematic reviews were used to identify further studies. Initial searches were conducted between January 2017 and March 2017 and repeat searches conducted in April 2018.

2.4. Study selection and data extraction

Duplicate papers were removed and studies were screened against the inclusion and exclusion criteria for eligibility, based on the abstract and title. This was undertaken by two researchers (JB and either LA, DS or JM). Where the two researchers were not in agreement, the paper was downloaded and read in full and a mutual decision made about eligibility. Full papers for all tentatively included studies were downloaded and scrutinised by one researcher using a screening tool to confirm that the studies met the inclusion criteria. Any ambiguous studies were discussed with a second researcher and assessed as either eligible or ineligible.

2.5. Quality appraisal

All included studies were quality appraised using the relevant Joanna Briggs Institute checklist for quasi-experimental studies, which asks 9 questions or the checklist for Randomised Control Trials, which asks 13 questions (Tufanaru et al., 2017). Each paper

Fig. 1. PRISMA Flow diagram (Moher et al., 2009) of literature search and results.

50 J. Brook et al. / International Journal of Nursing Studies 91 (2019) 47–59

was awarded a quality assessment score by adding the total number of positive responses.

Initially 6 (12%) of the papers were double coded by two researchers with a 75% agreement about eligibility. The criteria were subsequently refined and a further 6 (12%) were double coded with at least 95% agreement. The remaining papers were assessed by both researchers and any clarity issues were resolved in discussion with the research team. This process aligns with guidance offered by the Cochrane Library (Higgins and Green, 2011).

2.6. Data extraction

A data extraction sheet was designed and used to extract the relevant information from the eligible studies. This included: 1) author & year; 2) country of study; 3) study design; 4) intervention used including specific characteristics of the intervention; 5) participant sample size and characteristics; 6) outcome measures; 7) retention and/or turnover data; 8) other outcomes.

2.7. Data synthesis

Following data extraction and quality analysis, key features and findings of included studies were reviewed and iteratively discussed by three authors (JB, RW, LA) to establish the structure of the synthesis of the study findings. Due to the heterogeneity of the studies and the nature of the question, meta-analysis was not performed. Instead a narrative summary of the characteristics of the different interventions was conducted. Specific detail such as sample size, outcome measures and intervention design are presented in Tables S1–S6, which are organised according to intervention type.

Retention/turnover data were used to guide the comparison between studies. The change in retention/turnover was calculated by subtracting the pre-intervention/comparison group data from the post-intervention/target group data. Therefore, negative numbers reflect a decrease in retention and/or an increase in turnover. To identify the benefits achieved by each of the interventions in regard to our outcomes of retention and turnover, data were summarised using appropriate measures of central tendency and dispersion, based on the normality of the data. Using these data, identification of the intervention elements that were successful was possible.

3. Results

3.1. Identification of papers

A total of 11,656 papers were identified through database searches, and an additional 8 papers were found from the reference lists of identified studies (Fig. 1). After duplicates were removed, 3252 papers remained which were screened by abstract and title. After applying eligibility criteria to abstract and title, 3033 papers were excluded leaving a total of 108 papers for full text assessment. A further 55 papers were excluded, leaving 53 papers for inclusion in the review. Included studies are outlined in Table 2; turnover and retention data are detailed in Table 3.

3.2. Critical appraisal

No studies were excluded on the basis of quality appraisal and the implications of this are noted in the discussion section. Fifty- two papers were appraised using the adapted JBI checklist for Quasi-experimental studies and 1 was appraised using the checklist for Randomised Controlled Trials (RCTs). Out of a total potential score of 9 for quasi-experimental studies, 8 studies

scored 7 or above and 32 scored 4-6. Twelve studies scored 3 or below (Leigh et al., 2005; Shermont and Krepcio, 2006; Zucker et al., 2006; Pine and Tart, 2007; Trice et al., 2007; Keahey, 2008; Mills and Mullins, 2008; Scott and Smith, 2008; Morris et al., 2009; Hillman and Foster, 2011; Kooker and Kamikawa, 2011; Vergara, 2017). The one RCT study scored of 7 out of a possible 13. Low appraisal scores were frequently influenced by an absence of information. Nine of the studies were unclear whether participants in the comparison and intervention groups were similar or received similar treatment other than exposure to the interven- tion. Nine of the studies were unclear or did not include multiple measurements of retention or turnover pre and post intervention. None of the 12 studies reported how turnover or retention was calculated, and 11 of the 12 studies were unclear whether the outcomes of the participants in comparison groups were measured in the same way as the intervention group.

3.3. Study characteristics

Fifty-three studies were included in this review. Fifty-two of the studies were conducted in English-speaking countries (United States of America: n = 48; Australia: n = 2; Canada: n = 1; United Kingdom: (n = 1). The remaining study was carried out in Taiwan but published in English. The majority of the interventions were designed to target newly graduated nurses (n = 30); 10 interven- tions were designed for nurses at any stage of their career; 5 for newly hired nurses; 4 for newly licensed/qualified nurses who had taken the national nursing exam in order to become a licenced practitioner; and 4 for nurses with no experience in a particular speciality.

The hospitals size in which the interventions took place ranged from 12 to 1800 beds. Eleven studies reported interventions in children’s hospitals and 8 were multi-site studies including a mixture of specialities.

Thirty-nine studies provided data on the number of nurses taking part in the intervention. Sample sizes ranged from 3 to 6000 with a median of 90 (IQR 163). Turnover was reported in 35 studies, retention in 16 and both retention and turnover data were reported in 2 of the studies.

Table 2 Included studies (n=53).

Author, Year & Country Design Sample Size for retention /turnover

Quality Score

Intervention Duration of intervention

Main Findings in relation to retention and turnover

Allen et al., 2010, USA Time series non-equivalent control group

1499 nurses 5/9 Nursing advancement programme

Variable Turnover (2006) was 9.4% less for the intervention group

Almada et al., 2004, USA Nonequivalent control group Pretest posttest

All nurses hired during two specific time periods

4/9 Preceptorship programme

8 weeks with support to 1 year

Overall hospital retention increased by 29% from pre (April 2000) to post (May 2001) intervention; newly graduated nurse retention increased by 68%

Anderson et al., 2009, USA Nonequivalent control group pretest posttest

120 nurses 6/9 Residency programme

1 year 25% improvement in retention pre (2001) to post (2006) intervention and 11% improvement in retention from pre (2001) to post (2007) intervention

Baggot et al., 2005, USA Time series design

All registered nurses 7/9 Preceptor action days

< 4 weeks 3.9% reduction in turnover from pre (2001) to post (2004) intervention.

Beecroft et al., 2001, USA Nonequivalent control group posttest only

68 nurses 7/9 Internship in paediatrics

26 weeks 22% reduction in turnover compared to control group, measured over 12 month period

Beyea et al., 2010, USA Time series nonequivalent control group

260 newly graduated nurses

5/9 Nurse residency programme

12 weeks 7.8% decrease in turnover at 1 year post intervention and 9.3% at 2 years post intervention

Blegen et al., 2015, USA Time series nonequivalent control group

405 newly licensed nurses

5/9 Transition to practice programme

26 weeks 6% increase in retention in interventions with high levels of preceptor support compared to interventions with low levels of preceptor support.

Bonczek et al., 2016, USA Pretest posttest

399 Newly hired nurses 4/9 Holistic approach workshops

4 weeks 1.5% increase in retention from pre (one year prior to the study) to post (during the study period) intervention

Berube et al., 2012, Canada Time series 67 nurses 6/9 Critical care nurse residency programme

52 weeks 30% increase in retention pre (June 2005-March 2008; 3-year average) and post (June 2008-March 2011; 3 year average)

Bullock et al., 2011, USA Pretest- posttest Quasi experimental

651 newly hired nurses 5/9 Orientation programme

52 weeks 31.1% decrease in turnover from pre (2007) to post (2009) intervention.

Cantrell and Browne, 2005, USA

Time series 193 newly graduated nurses

6/9 Externship 10 weeks 8.8% increase in turnover for intervention group; of the 193 nurses who completed the externship, 153 accepted a graduate nurse position at the institution

DiMeglio et al., 2005, USA Time series design

All nurses on 6 inpatient units

5/9 Team building intervention

>4 weeks 3% decrease in turnover from pre (April 2002- September 2002) to post (October 2003-March 2004) intervention

Drenkard and Swartwout, 2005, USA

Nonequivalent control group pretest posttest

268 clinical ladder promoted nurses

5/9 Clinical Ladder programme

Variable 10.8% decrease in turnover pre intervention to post intervention; turnover was 8.9% less for those on the clinical ladder programme

Faron and Poeltler, 2007, USA

Pretest- posttest

77 new graduate nurses 6/9 Orientation Programme

52 weeks 13% decrease in turnover pre (July 2003-June 2004) to post (October 2004 – July 2005) intervention.

Figuerola et al., 2015, USA Nonequivalent control group posttest

343 nurses 4/9 Preceptorship programme

18 weeks 9.5% improvement between turnover in intervention group and cohort 1; 23.1% improvement between intervention group and cohort 2

Fox, 2010, USA Time series non-equivalent control group

All nurses in hospital 4/9 Mentorship programme

52 weeks 15.4% decrease in turnover pre (2005) to post (2006) intervention; 18.23% decrease in turnover pre (2005) to post (2007) intervention; 10.7% decrease in turnover pre (2005) to post (2008) intervention

Friday et al., 2015, USA Time series design

60 newly graduated nurses

4/9 Externship and residency programme

52 weeks 4% decrease in retention at 1 year for externship and internship; 14% decrease at 2 years.

Friedman et al., 2013, USA Time series nonequivalent control group

49 nurses 8/9 Paediatric orientation programme

52 weeks 12% improvement in retention across all paediatric nurses pre and post intervention.

Friedman et al., 2011, USA Time series nonequivalent control group

90 nurses 9/9 Critical care orientation programme

52 weeks 20% increase in retention in intervention group (critical care orientation) versus control group (standard orientation)

Gokenbach, 2004, USA Time series Open to all nurses in the Emergency Department

5/9 Empowerment model

Ongoing Pre intervention: 2.96 nurses leaving per month; Post intervention: 1.64 nurses leaving per month � Mean difference 0.132

Halfer, 2007, USA Time series 296 newly graduated nurses

4/9 Internship 52 weeks 17.2% decrease in turnover pre (2002) to post (12 months after) intervention

Halfer et al., 2008, USA Nonequivalent control group pretest posttest

212 newly graduated nurses

6/9 Internship 52 weeks 8% decrease in turnover pre (2001-2002) to post (2003-2005) intervention

Harrison and Ledbetter, 2014, USA

Nonequivalent control group pretest posttest

461 newly licenced nurses

5/9 Nurse residency programme

52 weeks 2% increased turnover for intervention site at 1 year post intervention compared to control sites.

J. Brook et al. / International Journal of Nursing Studies 91 (2019) 47–59 51

Table 2 (Continued)

Author, Year & Country Design Sample Size for retention /turnover

Quality Score

Intervention Duration of intervention

Main Findings in relation to retention and turnover

Hillman and Foster, 2011, USA

Time series design

251 newly graduated nurses

2/9 Transition to practice programme

16 weeks 22.5% increase in retention pre (1 year before intervention) to post (5 years after adoption of intervention).

Hinson and Spatz, 2011, USA

Pretest posttest

All newly employed nurses

5/9 Retention collaborative

Ongoing 13.36% reduction in turnover from pre intervention (2007) to post intervention (2008); 7.69% reduction in turnover from pre intervention (2007) to post intervention (2009).

Keahey, 2008, USA Nonequivalent control group posttest

5 nurse participants 2/9 Nurse residency programme

36 weeks 40% increase in retention from pre intervention to post (2 years after intervention).

Kooker and Kamikawa, 2011, USA

Time series design

All nurses 1/9 New nurse fellowship programme

24 weeks 12.23% increase in retention from pre (2005) to post (2009) intervention

Kozub et al., 2015, USA Nonequivalent control group posttest

42 newly hired nurses 7/9 Orientation programme

6-16 weeks Improvement of 20% from pre (previous 2-year retention) to post (current 2-year retention) intervention.

Latham et al., 2011, USA Nonequivalent pretest posttest non control group

All nurses in hospitals 1 and 2

5/9 Mentor programme

>4 weeks 21% increase in retention for Hospital 1 pre intervention to 3-4 years after; 17.07% increase in retention for Hospital 2 pre intervention to 3-4 years after

Lee et al., 2009, Taiwan Pretest posttest

224 newly graduated nurses

6/9 Preceptorship programme

12 weeks Reduction in turnover of 17.7% from pre- intervention (March 2006) to during 6-month study period (August 2006)

Leigh et al., 2005, UK Time series design

201 newly graduated nurses

3/9 Preceptorship programme

30 weeks 9% decrease in turnover at 12 months post intervention; 4% at 24 months post intervention

Marcum and West, 2004, USA

Pretest posttest

20 nurses 4/9 Structured orientation

26 weeks 11% decrease in turnover from pre to post (18 months after intervention)

Meraviglia et al., 2008, USA Time series 1150 nurses 4/9 Nurse friendly criteria and policies

Ongoing 2.7% decrease in turnover from pre (2004) to post (2006) intervention

Mills and Mullins, 2008, USA

Nonequivalent control posttest

All newly hired nurses 1/9 Mentor programme

Not clear 15% lower turnover for intervention group compared to nurses who did and did not take part in the intervention

Morphet et al., 2008, Australia

Nonequivalent control posttest

87 newly hired nurses 5/9 Orientation programme

52 weeks Short term retention (12 months) increased by 37% for intervention group versus control group; long term retention (nurses who stayed in emergency nursing, regardless of location) increased by 49.7%

Morris et al., 2009, USA Time series non-equivalent control group

All ICU nurses (control); 197 newly hired nurse (intervention)

3/9 Orientation programme

7 weeks 1.9% increase in retention from pre (May 2005) to post intervention (August 2006); 2.5% decrease in turnover

Newhouse et al., 2007, USA Nonequivalent control group posttest only

377 newly graduated nurses (intervention) 115 newly graduated nurses (control)

7/9 Internship 52 weeks 8% improvement in retention for intervention group versus comparison group at 18 months post intervention; 12% improvement in retention at 24 months

Olson-Sitki et al., 2012, USA Time series All newly graduated nurses

6/9 Nurse residency programme

12 weeks 5% decrease in turnover from 1 year pre to 1 year post intervention; 8% decrease in turnover 2 years pre to 1 year post intervention; 4% decrease in turnover 2 years pre to 2 years post intervention

Orsini, 2005, USA Non- equivalent control group pretest posttest

All nurses in hospital (control) 3 newly graduated nurse participants (intervention)

5/9 Orthopaedic transition to practice programme

12 weeks 14.9% reduction in turnover for intervention group compared to comparison group (whole hospital)

Pine and Tart, 2007, USA Pretest posttest

All newly graduated nurses

2/9 Nurse residency programme

52 weeks 37% decrease in turnover from pre (2003) to post (2005) intervention

Porter and Kolcaba, 2010, USA

pretest posttest

1443 nurses (pretest) 1695 nurses (posttest)

7/9 Nursing labour management partnership intervention

Ongoing 3.1% decrease in turnover from pre (2005) to post (2008) intervention

Rickard et al., 2012, Australia

Pretest posttest

Total nursing workforce in two hospitals

4/9 Workload intervention

Ongoing Hospital 1: 17% decrease in turnover from pre (2004) to post (2010) intervention; Hospital 2: 15% decrease in turnover from pre (2004) to post (2010) intervention

Schoessler and Waldo, 2006, USA

Time series All newly graduated nurses

4/9 Orientation programme

104 weeks 28% decrease in turnover from pre (1998) to post (2003) intervention

Scott and Smith, 2008, USA Time series 87 newly graduated nurses

3/9 Transition to practice programme

52 weeks 10.7% decrease in turnover from pre (2002) to post (2005) intervention

Shermont and Krepcio, 2006, USA

Pretest posttest

All newly hired nurses 3/9 Orientation programme

Variable 50% reduction in turnover from pre (2001/2002) to post (2002/2004) intervention

Silvestre et al., 2017, USA Randomised controlled multi-site

1032 nurses 7/13 Transition to practice programme

26 weeks Intervention group turnover rate 15.5%, control group turnover rate 26.8%

Trepanier et al., 2012, USA Pretest posttest

524 newly graduated nurses

6/9 Nurse residency programme

Unknown Data averaged across 15 hospitals: 29.63% mean decrease in turnover from pre (12 months) to post (12 months) intervention

52 J. Brook et al. / International Journal of Nursing Studies 91 (2019) 47–59

Table 2 (Continued)

Author, Year & Country Design Sample Size for retention /turnover

Quality Score

Intervention Duration of intervention

Main Findings in relation to retention and turnover

Trice et al., 2007, USA Nonequivalent control group posttest only

38 newly graduated nurses (intervention) 28 Newly graduated nurses (control)

3/9 Externship and internship

30 weeks 21% improvement in retention from 2 years pre intervention to post intervention.

Ulrich et al., 2010, USA Time series 6000+ newly graduated nurses

7/9 Nurse residency programme

52 weeks 19.9% decrease in turnover from pre intervention compared to post (12 months post recruitment); 29.4% decrease in turnover from pre intervention compared to post (24 months post recruitment)

Vergara, 2017, USA Pretest posttest

All staff in one critical care unit

2/9 Critical care mentor programme

26 weeks 4% decrease in turnover from 2014 to 2015

Williams et al., 2002, USA Non- equivalent control group pretest posttest

nurses in 9 intensive care units

5/9 Nurse residency programme

30 weeks For intervention group: Mean change in turnover = decrease of 9.3%; for non-intervention group: Mean change in turnover = increase of 14.5%.

Wright et al., 2017, USA Pretest posttest

3000+ nurses 4/9 Self-scheduling programme

Ongoing 1.9% mean decrease in turnover across 4 hospitals. Hospitals A and D increase in turnover, hospitals B and C, decrease in turnover.

Zucker et al., 2006, USA Pretest posttest

260 newly graduated nurses across 5 hospitals

1/9 Mentor programme

78 weeks 13% increase in retention 6 months post intervention compared to pre intervention; 12.4% decrease in turnover

Table 3 Turnover and retention data summary.

Turnover (% improvement/difference) Retention (% improvement/difference)

Sample size

Effect Mean Median IQR Min- Max

Effect Mean Median IQR Min-Max

Length of intervention < 4 weeks 300-400+ 2 positive 3.4 3.4 0.4 3-3.9 – – – – – 4-26 weeks 3-1032 11 positive / 1

negative 10.8 9.3 7.1 1-23.1 5 positive 8.8 6 10.3 1.5-22.5

27-52 weeks 5-6000+ 9 positive / 1 negative 16.5 13.3 8.4 4-37 11 positive / 2 negative

25.4 21 21.5 5-68

> 52 weeks NR 1 positive – – – 12.4 1 positive – – – 13 Variable 42- 70 1 positive – – – 54 1 positive – – – 20 Ongoing 1856-5400 6 positive 12.7 10.8 5.67 2.7-38 – – – – – Type of intervention Preceptorships 123-400 4 positive 11.2 9.2 10.4 4-23.1 – – – – – Mentorships 12-450 5 positive 12.6 13.7 4.2 4-18.2 2 positive 17.0 17.1 4 13-21 Residency programmes and Internships

31-6000+ 8 positive 1 negative

18.1 18.6 19.2 1-37 4 positive 1 negative

24.2 19.5 18.5 5-68

Externship/ Externship + nurse residency

100-153 1 negative – – – – 1 positive / 1 negative – – – 21

Orientation & Transition to Practice 3-1032 9 positive 17.6 13.6 10.6 2.5-50 7 positive 13.3 12 11.5 1.9-22.5 Clinical ladder/advancement 33-478 2 positive 11.4 11.4 2 9.4-13.4 – – – – – Other 89-1856 7 positive 7.4 5.4 7.4 1.9-17 2 positive 6.9 6.9 5.4 1.5-12.2 Characteristics of intervention Preceptor component 3-1032 13 positive 14.3 9.5 8.7 1-50 9 positive / 1 negative 22.7 20.5 18.2 1.9-68 Mentor component 12-524 7 positive / 1 negative 14.0 13 4.7 4-29.6 2 positive 17.0 17.1 4 13-21 Both preceptor and mentor component

50-6000+ 5 positive 19.8 19.9 4.8 10.7- 29.4

3 positive / 1 negative 14.6 12 9 5-25

Incentives 20-2400 8 positive 11.8 11 4.2 2.7-18.2 3 positive / 1 negative 15.4 15.0 8.5 5-25 Teaching/training 3-6000+ 21 positive / 1

negative 15.1 11.85 10.8 1-50 16 positive / 2

negative 23.8 20 18 1.5-80

Assessment 3-6000+ 14 positive 12.0 10.1 8.4 1-29.4 9 positive / 1 negative 19.7 14 14 1.9-68 Speciality tracks 3-260 7 positive / 1 negative 9.1 9 9 2.5-17.2 5 positive 16.3 17 7.5 1.9-30

IQR – Interquartile Range. *Sample size was not reported precisely in the original paper.

J. Brook et al. / International Journal of Nursing Studies 91 (2019) 47–59 53

The narrative summary presents two approaches. First, studies were analysed according to the assignation of the authors to the intervention (descriptions of the different types of interventions are provided in Table 1). Due to considerable overlap between different types of interventions, the second analysis categorised studies based on the description of the components of the intervention. This was conducted to ascertain the influence of specific components on the outcome. For example, an intervention may have been labelled as a residency programme but involved mentorship of participants. What follows is a systematic report of

the impact of both the interventions and their constituent components on nurse retention and turnover.

3.4. Type of components

Components that were detailed in the most studies (i.e. over 10 studies evaluating them) include intervention lengths between 4– 26 weeks (n = 16) and 27–52 weeks (n = 21); interventions with a preceptor component (n = 30), incentives (n = 10), a teaching component (n = 37) and an assessment component (n = 21).

54 J. Brook et al. / International Journal of Nursing Studies 91 (2019) 47–59

3.4.1. Preceptor component A larger number of studies reported interventions with a

preceptor component (n = 22) and of these, 13 evaluated turnover with an average 14% decrease in turnover. The 9 studies that measured retention found an average 23% increase in retention. Friday et al. (2015) was the notable exception, with a 4% decrease in retention at 1 year after the intervention and a 14% decrease at 2 years post intervention.

3.4.2. Mentor component Nine studies reported that the interventions included a mentor

component. Seven of these provided turnover data, with a 14% average decrease in turnover across the studies, and one reporting a 2% increase in turnover (Harrison and Ledbetter, 2014). Retention data was reported by 2 studies and retention improved by an average of 17% across these studies.

3.4.3. Both preceptor and mentor component Eight studies described interventions that offered both a

mentor and preceptor component, 4 scoring 7 or above in the quality assessment (Beecroft et al., 2001; Friedman et al., 2011, 2011; Ulrich et al., 2010). Five of these studies provided turnover data and found that the interventions led to an average decrease in turnover of 20%. Three studies looked at retention and found an average of 15% improved retention. However, Anderson et al. (2009) reported a 1% increase in turnover but this was after changes were made to an established intervention programme.

3.4.4. Teaching/training component Thirty seven of the 53 interventions had a teaching component,

often including a combination of classroom based teaching, simulated practice and clinical immersion. Twenty-one studies found an improvement in turnover, with 15% average decrease across studies. Across the 16 studies that measure retention, there was an average of 24% increase in retention. However, 3 studies with a teaching/training component found negative effects of turnover and retention (Anderson et al., 2009; Friday et al., 2015; Harrison and Ledbetter, 2014).

3.4.5. Assessment component Twenty-one studies incorporated a formal assessment compo-

nent and 4 scored 3 or less in the quality assessment (Leigh et al., 2005; Morris et al., 2009; Trice et al., 2007; Zucker et al., 2006). Fourteen of the studies found an improvement in turnover (M = 12.0; SD = 7.2). Across 9 studies an average of 20% increase in retention was found.

3.4.6. Incentives component Incentives were used to attract nurses to a position or tempt

them stay in the role or organisation. Incentives included financial benefits such as payment to attend certain meetings or training (Faron and Poeltler, 2007; Lee et al., 2009) promotion or pay rise (Allen et al., 2010; Drenkard and Swartwout, 2005) and bonus payments to reward loyalty (Fox 2010, Marcum and West, 2004; Zucker et al., 2006). Ten studies found a positive effect of incentives on turnover and retention and all but 1 (Zucker et al., 2006) scored above 3 in the quality assessment. On average, turnover decreased by 12% and retention increased by 15%. However, Anderson et al. (2009) reported the positive effect of offering a financial reward for nurses staying for more than 2 years but found an increase in turnover after 2 years.

3.4.7. Intervention length The length of the interventions varied from 1 day to

interventions that were ongoing. Analysis of interventions length was grouped as follows: 1) less than 4 weeks 2) 4–26 weeks; 3) 27–

52 weeks; 4) over a year; 5) ongoing; 6) varies depending on the nurse. Studies reporting negative effects were Cantrell and Browne (2005) reporting increased turnover of 8.8% following a summer externship lasting 10 weeks and three studies of 27–52 weeks. Harrison and Ledbetter (2014) found an increased turnover of 2% for the hospital that had a nurse residency programme, compared to two control hospitals. Friday et al. (2015) found that nurses who did an externship and internship were less likely to stay at the hospital (4% decrease in retention at 1 year; 14% decrease at 2 years). Furthermore, Anderson et al. (2009) found that a residency programme of this length led to improved retention from 2001 to 2007, however, after the residency programme was revised, retention fell by 1%.

The largest number of studies lasted between 27 and 52 weeks (n = 21). Only two studies evaluated interventions that lasted over 52 weeks and in these, turnover decreased by 12.4%, retention improved by 13%.

3.5. Types of intervention

Types of interventions that were most frequently evaluated in this review are residency programmes/internships (n = 16), orien- tation and transition to practice programmes (n = 15), and interventions with a speciality track (n = 12).

3.5.1. Preceptorship programmes Four studies evaluated the impact of preceptorships on

turnover and found turnover decreased an average of 11% across studies (SD = 7.7) (Table S1). All but one study scored above 3 in the quality assessment process.

3.5.2. Mentorship programmes Five studies provided turnover data for mentorships, and

turnover was reduced by an average of 12%. Two studies looked at retention data and found a 17% increase in retention on average. However, three of these studies scored 3 or less in the quality assessment (Mills and Mullins, 2008; Vergara, 2017 and Zucker et al., 2006) (Table S2).

3.5.3. Residency programmes and internships Sixteen studies reported findings from residency programmes

and internships, with 3 scoring 3 or less in the quality assessment. Ten studies looked at turnover and found an average 18% decrease. Six studies examined retention and found an average increase of 24%. However, two of these studies found a negative impact on turnover and retention: Anderson et al. (2009) found a 1% decrease in retention and Harrison and Ledbetter (2014) found an increased turnover of 2%.

3.5.4. Externships Three studies examined the impact of externships, either alone

(Cantrell and Browne, 2005), in combination with a nurse residency programme (Friday et al., 2015), or with an internship (Trice et al., 2007). All but 1 study (Trice et al., 2007) scored between 4 and 6 in the quality assessment. The two studies that included an externship in combination with other interventions found a positive effect. Friday et al. (2015) found that an externship and nurse residency programme, in comparison to a standalone nurse residency, led to decreased turnover at year 1 (4%) and year 2 (14%) but found no differences in scores on the Casey-Fink nurse experience survey (Casey et al., 2004) between the two groups, indicating that the decrease in turnover was not related to perception of the experience of the externship and residency programme. Trice et al. (2007) found that of those nurses who had completed a nursing internship and externship between 1999 and 2005 had a retention rate of 89% at 2 years, compared to those who

J. Brook et al. / International Journal of Nursing Studies 91 (2019) 47–59 55

just did the internship in 2007, who had a retention rate of 68%. However, nine of these interns had not reached their 2-year mark yet, making comparisons difficult. The one study that looked at the impact of an externship as a single intervention on turnover found a negative outcome: turnover in the externship group was 9% higher than the control group (Cantrell and Browne, 2005) (Table S6).

3.5.5. Orientation and transition to practice programmes Fifteen studies examined turnover and retention in relation to

orientation and transition to practice programmes and found an average decrease of 18% in turnover, and a 13% increase in retention. Four of these studies scored 3 or less and 5 scored 7 or above in the quality assessment.

3.5.6. Speciality tracks Twelve studies evaluated interventions that offered speciality

training. Seven were interventions designed for specific clinical specialities, such as critical care (n = 5); specialist care (n = 1); or orthopaedic care (n = 1). Five described specialty tracks which could be adapted depending on the clinical area; all interventions included pairing with mentors who trained them in their clinical specialism. Specialty tracks were used to ease the transition from one clinical area to another (Morris et al., 2009; Vergara, 2017), or support the integration of newly qualified nurses in to areas of specialism. Speciality tracks were components in residency programmes, transition programmes, orientation programmes, preceptorship programmes and internships.

Three of these studies scored 7 or above and 3 scored 3 or less in the quality assessment. Across seven studies an average of 9% decrease in turnover was found. Retention increased by an average of 16% across five studies. One study found a negative effect of having a speciality track component on turnover (Harrison and Ledbetter, 2014).

3.5.7. Clinical ladder or advancement programmes Two studies looked at clinical ladder/advancement pro-

grammes and found an average of 11% decrease in turnover (Table S5).

A range of interventions were examined in the remaining studies, including a team building approach (DiMeglio et al., 2005), a nurse friendly hospital project (Meraviglia et al., 2008) and a new nurse fellowship (Kooker and Kamikawa, 2011), however there was insufficient similarity in intervention characteristics to enable comparisons (Table S6).

4. Discussion

This systematic review was conducted to identify the character- istics of interventions that improved the retention of nurses within clinical practice. A wide variety of interventions and components within those interventions have been identified to improve nurse retention. In this review, the number of studies that examined each component of an intervention was variable.

Two key characteristics of effective interventions appear to be mentorship and preceptorship. Friday et al. (2015) found a negative effect for turnover, but this was for nurses who took part in both an externship (no preceptor) and an internship (with a preceptor) versus those who just took part in the internship, and therefore it is unlikely the preceptor component influenced the results.

The effects of preceptorship on new nurses has previously been explored systematically, concluding that there was insufficient evidence to support the beneficial effects of preceptorship on professional socialisation, job satisfaction or retention, but significant positive effects on new nurse competence (Ke et al., 2017). A period of support following qualification has also been

recognised as important (Whitehead et al., 2013) identifying that key to success is the quality of the preceptorship/preceptee relationship. This relationship can be developed if preceptors are supported through structured training and peer support (Adlam et al., 2009). The fact that newly qualified nurses feel that a period of preceptorship is essential (Higgins et al., 2010) suggests that nurses will seek to be employed by an organisation that provides preceptorship as it indicates commitment to their career development (Edwards et al., 2015). This in turn may lead to positive effects on retention and turnover such as we have found. Increases in competence may also lead to increases in confidence and greater job satisfaction (Edwards et al., 2015), which is again linked with lower turnover (Twigg and McCullough, 2014).

Interventions with both a preceptor and mentor component appear to have merit, although the results are not consistent. Inclusion of both mentorship and preceptorship appears to have a larger impact on turnover, with an average decrease of 20% across the 5 studies that measured turnover, compared to a decrease of 14% across the 13 studies that described standalone preceptorship components. Conversely, this review found an average 15% increase in retention for those interventions with both preceptor and mentor components and an average 23% increase for those with preceptors only.

Mentorship is an established mechanism for support in early career nursing, with the mentor/mentee relationship again central to the success of the intervention. Mentorship programmes can take many forms and there are debates between advocates of random matching of mentor and mentee and those that favour pairing based on the rigorous consideration of specific character- istics of the participants (Zhang et al., 2016). Appropriate fit is clearly considered important and, although there is no consensus in the literature on how this should be achieved, similarities between personality, experience and opportunity to interact will influence the quality of the relationship (Nowell et al., 2016). Given the importance of the relationship to the outcome, it would seem prudent to consider the characteristics of the individuals involved. In this review three studies employed organised pairing and two allocated with random or unspecified pairing methods; all reported a positive effect. In addition, the contribution to successful maintenance of the relationship made by the regularity of the mentor/mentee meetings and clarity of mentoring objectives should not be underestimated (Zhang et al., 2016).

The diversity of the mentoring and preceptorship components considered in this review make conclusions difficult to draw but the key characteristics of many programmes such as networking and socialisation in the profession, together with a nurturing and developmental relationship, are consistently linked to positive practice environments and increased retention (Twigg and McCullough, 2014; Lartey et al., 2014). As many preregistration nursing courses also include elements of mentorship, newly qualified nurses may be familiar with the dynamic of a mentoring relationship and be able to use it to maximum effect.

A promising component of an intervention appears to be teaching/training. The taught element was evaluated in 37 studies with a generally positive effect on retention and turnover. These findings concur with those of other reviews where increases in competence, knowledge and confidence were identified as important effects of interventions to support newly qualified nurses (Edwards et al., 2015; Ke et al., 2017).

Although there is acknowledgement that newly qualified nurses need to continue to develop competence and knowledge (Berkow et al., 2009), there is no agreement on the best approach for acquiring these capabilities (Al-Dossary et al., 2014). The interventions studied in this review utilised a variety of approaches to teaching nurses, either formally or informally, in the classroom or via clinical immersion in simulated or live

56 J. Brook et al. / International Journal of Nursing Studies 91 (2019) 47–59

settings, making comparisons and clear conclusions difficult. Furthermore, three studies found a negative impact of these interventions on turnover and retention. It is unclear whether the negative impact was linked to the teaching element of the intervention because empirical research in this area to demon- strate causation is weak (Irving et al., 2018). What may be important is a defined structure to the teaching and a managerial support framework to underpin the delivery (Irving et al., 2018). Success will be dependent on a system of line management that recognises the time commitment and support required for endeavours such as teaching of newly qualified nurses to take place effectively (Whitehead et al., 2013).

Whilst it may be assumed that the longer the intervention, the more likely it is to be effective, the results of this and previous reviews indicate that even shorter interventions have an effect (Salt et al., 2008). Interventions lasting between 4–26 weeks and 27–52 weeks were the most highly evaluated but those lasting between 27–52 weeks had the most beneficial effect on retention and turnover.

Newly qualified nurses often have unrealistic expectations of their new role (Higgins et al., 2010), leading to frustration and demoralisation, potentially linked to short-lived lack of confidence (Maben et al., 2007). In this review, Anderson et al., (2009) found that job satisfaction scores for newly qualified nurses dropped at one year after qualification, despite increases in perception that they could perform more effectively and had developed effective work relationships. At two years post qualification the nurses’ job satisfaction had increased again. Newly graduated nurses often feel that they still need further support and mentoring after their transition or orientation programme has been completed and may leave the organisation in anticipation of greater support elsewhere (Almada et al., 2004).

This may be related to unrealistic expectations of their ability (Mooney, 2007) and so preparation for transition to registration could involve further contextualisation to meet their specific needs. Potentially, there is a critical time span in which newly qualified nurses reassess the reality of the role, move out of the stage of conscious incompetence and acquire confidence. If confidence is something that cannot be taught but must be developed over time (Whitehead, 2001), the timeframe is particular to the individual nurse and as such the optimum intervention length for greatest effect is an area that warrants further research.

Salt et al., (2008) indicated that interventions with more than one component were most effective. Strategies that incorporated several methods of education, including classroom instruction, observational experience, coaching and computer-based learning, together with a strong preceptor element designed to provide one to one guidance for the newly qualified nurse were optimal. In the last ten years, developments in e-learning have resulted in greater understanding of the benefit for novice nurses from this component when used as an adjunct to traditional methods (Feng et al., 2013). More recently, a combination of both didactic and clinical taught elements in combination with well prepared and supported mentors or preceptors is advocated to ease the transition from student to qualified nurse (Edwards et al., 2015; Rush et al., 2012). Research exploring the perspective of newly qualified nurses indicates that a combination of opportunities to connect with peers with healthy work environments is also influential to the success of the intervention (Rush et al., 2012; Mooney, 2007). Although there is no consensus in the literature about the formulation for optimal effectiveness it would seem that, in keeping with the findings from this review, multi-faceted interventions that combine elements that focus on building competence and confidence, such as teaching, with those that focus on socialisation and embedding the new nurse into the work

environment, such as mentoring or preceptorship, may be effective at retaining nurses.

4.1. Recommendations

Despite concerns about the quality of study reporting and the distribution of the data, some suggestions about potentially successful interventions can be made. Interventions with the highest benefit appear to be an internship/residency programme or an orientation/transition to practice programme that incorpo- rates formal teaching, a preceptorship element and possibly the addition of a mentorship element, lasting 27–52 weeks in duration. Consistent with the fact that most of the included studies originated in high income economies, these findings align with support that is already offered in USA, Canada and Australia. The elements of preceptorship and mentorship are also embedded in UK nursing culture (Irwin et al., 2018). What is of interest is not the novelty of these interventions but rather the understanding that to develop and refine already established transition programmes, focus could be on the elements of teaching, preceptorship and mentorship and considering how these can constitute a combined offer. As variation in the quality of mentors, preceptors or teaching will affect the outcome of the programme of support and in turn impact on retention or turnover, we suggest review of these practices may be beneficial. When refining established provision, inclusion of all elements where feasible may be prudent.

Healthcare systems internationally are subject to funding restraints and astute fiscal decisions are paramount; these findings help to inform policy and practice related to retention of early career nurses. Given that even the shortest interventions had a positive effect, further cost-benefit analyses to determine where the point of maximum benefit for minimal financial outlay lies would be beneficial, although the complexity of the benefit of these interventions in terms of increased confidence, competence and patient satisfaction may not be easily captured using economic analysis tools.

It is clear that support for early career nurses is important to encourage retention and if the requirement for this support were to be enshrined in either local or national policy, early career nurses would feel more confident about their transition from student to qualified practitioner. The results of this review offer policy makers guidance for the development of such strategy.

Future research should focus on evaluating the intervention components identified as promising with high quality research methodology. First, standardised outcome measures should be used so that comparisons can be made across studies. This would include identifying an algorithm to calculate turnover and retention for parity and using statistical analyses to identify if the differences between groups are statistically significant. Second, future studies should report participant characteristics of both intervention and control groups and examine the importance of baseline differences. Detail of the reasons for participants leaving their post would also enable more effective comparison. Third, consideration could be given to identifying sub-groups or strata in larger studies to determine the differential effect of the interven- tion on groups with particular characteristics. This may help explain or mitigate the highly dispersed results found in this review. Carrying out this research with high quality methodology will allow for conclusions to be drawn about the effectiveness of an intervention, and comparisons to be made across studies.

4.2. Strengths

The strength of this systematic review lies in the rigour of the process, which was conducted according to PRISMA guidelines. Secondary scrutiny of all papers prior to inclusion mitigated

J. Brook et al. / International Journal of Nursing Studies 91 (2019) 47–59 57

subjectivity and clear criteria for quality assessment and data extraction reinforced this objective perspective. This is a large systematic review with consideration of over 11,000 papers and inclusion of 53. Subsequently the review has captured the characteristics of a wide range of interventions and the multiple reporting of some types of interventions in particular such as residency programmes or preceptorship, enhances the credibility of the findings.

4.3. Limitations, including methodological quality

Limitations that should be taken into account when considering the implications of the results include inconsistent and incomplete description of the interventions, missing detail of some compo- nents of the intervention, variations in sample size and the method of evaluation. As found in previous reviews (Ke et al., 2017; Edwards et al., 2015; Salt et al., 2008; Rush et al., 2012), the quality of the review is necessarily limited by the quality of the study reports that are available. Many of the studies were not conducted using rigorous research methods but we have chosen to include all studies that met the eligibility criteria to provide a complete description of the body of work. In addition, comparisons across studies were difficult to make due to variation in outcome measures (retention or turnover or both), the absence or presence of measurement of statistical significance and use of a range of validated or other measurement tools to assess additional outcomes. Even though many of the studies had large numbers of participants, measures of dispersion were also frequently large, suggesting highly heterogenous characteristics. Very few of the papers reported negative findings, which may indicate that this topic is subject to publication bias, with those studies that found nil or negative effect not being submitted for or accepted for publication. Consideration of unpublished reports may have addressed this bias to some extent.

5. Conclusion

The characteristics of interventions that successfully influence retention of nurses within practice have been identified. Methodological issues impacted on the extent to which con- clusions could be drawn, even though a large number of studies were identified. Despite this, promising interventions appear to be either internship/residency programmes or orientation/transition to practice programmes, lasting between 27–52 weeks, with a teaching and preceptor and mentor component. Future research should focus on standardising the reporting of interventions and outcome measures used to evaluate these interventions and carrying out further research with rigorous methodology. Clinical practice areas are recommended to assess their current inter- ventions against these criteria to guide development of their effectiveness and evaluations of cost-effectiveness are considered an important next step.

Conflict of interest

None.

Funding

This work was supported by the Burdett Trust for Nursing [Grant number: SBZA101010662253815]

Appendix A. Supplementary data

Supplementary material related to this article can be found, in the online version, at doi:https://doi.org/10.1016/j.ijnurstu.2018.11.003.

References

Adlam, K.A., Dotchin, M., Hayward, S., 2009. Nursing first year of practice, past, present and future: documenting the journey in New Zealand. J. Nurs. Manag. 17 (5), 570–575.

Al-Dossary, R., Kitsantas, P., Maddox, P.J., 2014. The impact of residency programmes on new nurse graduates’ clinical decision-making and leadership skills: a systematic review. Nurse Educ. Today 14, 1024–1028.

Allen, S.R., Fiorini, P., Dickey, M., 2010. A streamlined clinical advancement program improves RN participation and retention. JONA 40 (7/8), 316–322.

Almada, P., et al., 2004. Improving the retention rate of newly graduated nurses. J. Nurses Staff. Dev. 20 (6), 268–273.

Anderson, T., Allen, M., Linden, L., Gibbs, E., 2009. New graduate RN work satisfaction after completing an interactive nurse residency. J. Nurs. Manag. 39 (4), 165–169.

Association of UK University Hospitals, 2017. Nurse Retention Best Practice Guide [Online] Available at:. . . (Accessed 3 December 17) https://www.medschools.ac. uk/media/2326/aukuh-nurse-retention-guide.pdf.

Audit Scotland, 2017. The NHS in Scotland 2017 Available online at. . . (Accessed 5 March 18) http://www.audit-scotland.gov.uk/report/nhs-in-scotland-2017.

Baggot, D.M., Hensinger, B., Parry, J., Valdes, M.S., Zaim, S., 2005. The new hire/ preceptor experience: cost-benefit analysis of one retention strategy. J. Nurs. Manag. (3), 138–145.

Beecroft, P.C., Kunzman, L., Krozek, C., 2001. RN Internship: outcomes of a one-year pilot program. JONA 31 (12), 575–582.

Berkow, S., Virkstis, K., Stewart, J., Conway, L., 2009. Assessing new graduate nurse performance. J. Nurs. Adm. 38, 468–474.

Berube, M., Valiquette, M.P., Laplante, E., Lepage, I., Belmonte, A., Tanguay, N., Lepine, I., Lalone, L., Touchette, S., 2012. Nursing residency program: a solution to introducing new grads into critical care more safely while improving accessibility to services. Nurs. Leadersh. 25 (1), 50–67.

Beyea, S.C., Slattery, M.J., von Reyn, L.J., 2010. Outcomes of a simulation-based nurse residency program. Clin. Simul. Nurs. 6, e169–e175.

Blegen, M.A., Spector, N., Ulrich, B.T., Lynn, M.R., Barnsteiner, J., Silvestre, J., 2015. Preceptor support in hospital transition to practice programs. J. Nurs. Manag. 45 (12), 642–649.

Bonczek, M.E., Quinlan-Colwell, A., Tran, S., Wines, K., 2016. A holistic approach to improve nursing retention during the first year of employment. Nurs. Adm. Q. 40 (3), 269–275.

Buchan, J., Seccombe, I., Gershlick, B., Charlesworth, A., 2017. In Short Supply: a Pay Policy and Nurse Numbers: Workforce Profile and Trends in the English NHS [Online] Available at:. . . (Accessed 13 December 17) http://www.health. org.uk/sites/health/files/Workforce%20pressure%20points%202017%20FI- NAL_0.pdf.

Bullock, L.M., Groff Paris, L., Terhaar, M., 2011. Designing an outcome-focused model for orienting new graduate nurses. J. Nurses Staff. Dev. 27 (6), 252–258.

Cantrell, M.A., Browne, A.M., 2005. The impact of a nurse externship program on the transition process from graduate to registered nurse. J. Nurses Staff. Dev. 6 (249– 256), 21.

Casey, K., Fink, R., Krugman, M., Propst, J., 2004. The graduate nurse experience. JONA 34 (6), 303–311.

Department of Health, 2017. Next Steps on the NHS Five Year Forward View [Online] Available at:. . . (Accessed 13 December 17) https://www.england.nhs.uk/wp- content/uploads/2017/03/NEXT-STEPS-ON-THE-NHS-FIVE-YEAR-FORWARD- VIEW.pdf.

DiMeglio, K., Padula, C., Piatek, C., Korber, S., Barrett, A., Ducharme, M., Lucas, S.R.N., Piermont, N., Joyal, E., DeNicola, V., Corry, K., 2005. Group cohesion and nurse satisfaction: examination of a Team-Building approach. J. Nurs. Adm. 35 (3), 110–120.

Drenkard, K., Swartwout, E., 2005. Effectiveness of a clinical ladder program. J. Nurs. Adm. 35 (11), 502–506.

Edwards, D., Hawker, C., Carrier, J., Rees, C., 2015. A systematic review of the effectiveness of strategies and interventions to improve the transition from student to newly qualified nurse. Int. J. Nurs. Stud. 52, 1254–1268.

Faron, S., Poeltler, D., 2007. Growing our own: inspiring growth and increasing retention through mentoring. Nurs. Womens Health 11 (2), 139–143.

Feng, J.-Y., Chang, Y.-T., Chang, H.-Y., Erdley, W.S., Lin, C.-H., Chang, Y.-J., 2013. Systematic review of effectiveness of situated E-learning on medical and nursing education. Worldviews Evid. Nurs. 10 (3), 174–183 2013.

Figuerola, S., Bulos, M., Forges, E., Judkins-Cohn, T., 2015. Stabilizing and retaining a quality nursing work force through the use of the married state preceptorship model. J. Contin. Educ. Nurs. 44 (8), 365–373.

Fox, C., 2010. Mentor program boosts new nurses’ satisfaction and lowers turnover rate. J. Contin. Educ. Nurs. 41 (7), 312–316.

Friday, L., Zoller, J.S., Hollerbach, A.D., Jones, K., Knofczynskiet, G., 2015. The effects of a prelicensure extern program and nurse residency program on new graduate outcomes and retention. J. Nurses Prof. Dev. 31 (3), 151–157.

Friedman, M.I., Cooper, A.H., Click, E., Fitzpatrick, J.J., 2011. Specialized new graduate RN critical care orientation: retention and financial impact. Nurs. Econ. 29 (1), 7–14.

Friedman, M.I., Deleaney, M.M., Scmidt, K., Quinn, C., Macyk, I., 2013. Specialized new graduate RN pediatric orientation: a strategy for nursing retention and its financial impact. Nurs. Econ. 31 (4), 162–170.

Gokenbach, V., 2004. The Effects of an Empowerment Model Intervention on Registered Nurse Turnover and Absenteeism. University of Phoenix, Phonenix.

58 J. Brook et al. / International Journal of Nursing Studies 91 (2019) 47–59

Good Governance Institute, 2015. https://www.good-governance.org.uk/services/ the-nursing-journey-recruitment-and-retention/,. [Online] Available at: https://www.good-governance.org.uk/services/the-nursing-journey-recruit- ment-and-retention/, (Accessed 13 December 17).

Halfer, D., 2007. A magnetic strategy for new graduate nurses. Nursing Economics 25 (1), 6–11.

Halfer, D., Graf, E., Sullivan, C., 2008. The organizational impact of a new graduate pediatric nurse mentoring program. Nurs. Econ. 26 (4), 243–249.

Harrison, D., Ledbetter, C., 2014. Nurse residency program: outcome comparisons to best practices. J. Nurses Prof. Dev. 30 (2), 76–82.

Hasselhorn, H.-M., Mueller, H.B., Tackenberg, P., 2003. Investigating Premature Departure From the Nursing Professions in Europe – The European NEXT-Study ISSBN:1404-790X, available at:. . . (Accessed 16 August 18) http://www.next. uni-wuppertal.de/EN/index.php?book-publication.

Health Workforce Australia, 2014a. Australia’s Future Health Workforce- Nurses Overview [Online] Available at:. . . (Accessed 13 December 17) http://www. health.gov.au/internet/main/publishing.nsf/content/australias-future-health- workforce-nurses.

Health Workforce Australia, 2014b. Nursing Workforce Sustainability, Improving Nurse Retention and Productivity [Online] Available at:. . . (Accessed 13 December 17) http://www.health.gov.au/internet/main/publishing.nsf/Content/nursing- workforce-sustainability-improving-nurse-retention-and-productivity.

Higgins, G., Spencer, R.L., Kane, R., 2010. A systematic review of the experiences and perceptions of the newly qualified nurse in the United Kingdom. Nurse Educ. Today 30, 499–508.

Higgins, J.P.T., Green, S. (Eds.), 2011. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration. . Available from http://handbook.cochrane.org (Accessed 5 March 18).

Hillman, L., Foster, R.R., 2011. The impact of a nursing transitions programme on retention and cost savings. J. Nurs. Manag. 19, 50–56.

Hinson, T.D., Spatz, D.L., 2011. Improving nurse retention in a large tertiary acute- care hospital. JONA 41 (3), 103–108.

Irwin, C., Bliss, J., Poole, K., 2018. Does preceptorship improve confidence and competence in newly qualified nurses: a systematic literature review. Nurse Educ. Today 60, 35–46.

Ke, Y.T., Kuo, C.C., Hung, C.H., 2017. The effects of nursing preceptorship on new nurses’ competence, professional socialization, job Satisfaction and retention: a systematic review. J. Adv. Nurs. 73, 2296–2305.

Keahey, S., 2008. Against the odds: orienting and retaining rural nurses. J. Nurses Staff. Dev. 24 (2), E15–E20.

Kooker, B.M., Kamikawa, C., 2011. Successful strategies to improve RN retention and patient outcomes in a large medical centre in Hawaii. J. Clin. Nurs. 20, 34–39.

Kovner, C.T., Brewer, C.S., Fatehi, F., Jun, J., 2014. What does nurse turnover rate mean and what is the rate? Policy Polit. Nurs. Pract. 15 (3–4), 64–71. doi:http://dx.doi. org/10.1177/1527154414547953.

Kozub, E., Hibanada-Laserna, M., Harget, G., Ecoff, L., 2015. Redesigning orientation in an intensive care unit using 2 theoretical models. AACN Adv. Crit. Care 26 (3), 204–214.

Lartey, S., Cummings, G., Profetto-McGrath, J., 2014. Interventions that promote retention of experienced registered nurses in health care settings: a systematic review. J. Nurs. Manag. 22, 1027–1041.

Latham, C.L., Ringl, K., Hogan, M., 2011. Professionalization and retention outcomes of a university-service mentoring program partnership. J. Prof. Nurs. 27 (6), 344–353.

Lee, T.-Y., Tzeng, W.-C., Ling, C.-H., Yeh, M.-L., 2009. Effects of a preceptorship programme on turnover rate, cost, quality and professional development. J. Clin. Nurs. 18, 1217–1225.

Leigh, J.A., Douglas, C.H., Lee, K., Douglas, M.R., 2005. A case study of a preceptorship programme in an acute NHS Trust - using the European Foundation for Quality Management tool to support clinical practice development. J. Nurs. Manag. 13, 508–518.

Li, Y., Jones, C., 2013. A literature review of nursing turnover costs. J. Nurs. Manag. 21 (3), 405–418.

Maben, J., Latter, S., Macleod Clark, J., 2007. The sustainability of ideals, values and nursing mandate: evidence from a longitudinal qualitative study. Nurs. Inq. 14 (2), 99–113.

Marcum, E.H., West, R.D., 2004. Structured orientation for new graduates: a retention strategy. J. Nurses Staff. Dev. 20 (3), 118–124.

Mbemba, G., Gagnon, M.-P., Pare, G., Cote, J., 2013. Interventions for supporting nurse retention in rural and remote areas: an umbrella review. Hum. Resour. Health 11 (44) .

McDonald, A.W., Ward-Smith, P., 2012. A review of evidence-based strategies to retain graduate nurses in the profession. J. Nurses Staff. Dev. 28 (1), E16–20.

Meraviglia, M., et al., 2008. Nurse-Friendly Hospital Project: enhancing nurse retention and quality of care. J. Nurs. Care Qual. 23 (4), 305–313.

Mills, J.F., Mullins, A.C., 2008. The California nurse mentor project: every nurse deserves a mentor. Nurs. Econ. 26 (5), 310–315.

Ministry of Health, 2005. The nursing development plan in China (2005–2010) [in Chinese]. Chin. Hospitals 9 (9), 26–28 cited in Li, J., Fu, H., Hu, Y., Shang, L., Wu, Y., Sønderga ̊rd Kristensen, T., Mueller, B.H. and Hasselhorn, H.M., 2010. Psychosocial work environment and intention to leave the nursing profession: Results from the longitudinal Chinese NEXT study, Scandinavian Journal of Public Health, 2010; 38(Suppl 3): 69–80.

Moher, D., Liberati, A., Tetzlaff, J., Altman, D., 2009. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement Available at:. . . (Accessed 5 March 18) http://www.prisma-statement.org/.

Mooney, M., 2007. Facing registration: the expectations and the unexpected. Nurse Educ. Today 27, 840–847.

Morphet, J., McKenna, L., Considine, J., 2008. The career development year: responding to the emergency nursing shortage in Australia. Australasian Emerg. Nurs. 11, 32–38.

Morris, L.M., Pfeifer, P., Catalano, R., Fortney, R., Nelson, G., Rabito, R., Harap, R., 2009. Outcome evaluation of a new model of critical care orientation. Am. J. Crit. Care 18 (3), 252–262.

Newhouse, R.P., Hoffman, J.J., Suflita, J., Hairston, D.P., 2007. Evaluating an innovative program to improve new nurse graduate socialization into the acute healthcare setting. Ser. Nurs. Adm. 31 (1), 50–60.

NHS Improvement, 2017. Securing a Sustainable NHS Workforce for the Future [Online] Available at:. . . (Accessed 13 December 17). https://improvement.nhs. uk/news-alerts/securing-sustainable-nhs-workforce-future/.

Nowell, L., Norris, J.M., Mrklas, K., White, D.E., 2016. A literature review of mentorship programs in academic nursing. J. Prof. Nurs. 33, 334–344.

Nursing Solutions Inc, 2017. 2017 National Health Care Retention & RN Staffing Report [Online] Available at:. . . (Accessed 13 December 17) http://www. nsinursingsolutions.com/administrators/adminresearch/StaffingandWorkfor- ceStudies.aspx.

Olson-Sitki, K., Wendler, M.C., Forbes, G., 2012. Evaluating the impact of a nurse residency program for newly graduated registered nurses. J. Nurses Staff. Dev. 28 (4), 156–162.

Orsini, C.H., 2005. A nurse transition program for orthopaedics: creating a new culture for nurturing graduate nurses. Orthop. Nurs. 24 (4), 240–249.

Pine, R., Tart, K., 2007. Return on investment: benefits and challenges of a baccalaureate nurse residency program. Nurs. Econ. 25 (1), 13–39.

Porter, C.A., Kolcaba, K., 2010. The effect of a nursing labor management partnership on nurse turnover and satisfaction. JONA 40 (5), 205–210.

Rickard, G., Lenthall, S., Dollard, M., Opie, T., Knight, S., Dunn, S., Wakerman, J., MacLeod, M., Seiler, J., Brewster-Webb, D., 2012. Organisational intervention to reduce occupational stress and turnover in hospital nurses in the Northern Territory, Australia. Collegian 19, 211–221.

Royal College of Nursing, 2018. RCN Bulletin: Is Legislation the Answer? Available online at. . . (Accessed 5 March 18) https://www.rcn.org.uk/magazines/bulletin/ 2018/march/is-legislation-the-answer.

Rush, K.L., Adamack, M., Gordon, J., Lilly, M., Janke, R., 2013. Best practices of formal new graduate nurse transition programs: an integrative review. Int. J. Nurs. Stud. 50, 345–356.

Salt, J., Cummings, G.G., Profetto-McGrath, J., 2008. Increasing retention of new graduate nurses: a systematic review of interventions by healthcare organ- isations. J. Nurs. Adm. 38 (6), 287–296.

Schoessler, M., Waldo, M., 2006. Organizational infrastructure to support development of newly graduated nurses. J. Nurses Staff. Dev. 22 (6), 286–293.

Scott, E.S., Smith, S.D., 2008. Group mentoring: a transition-to-work strategy. J. Nurses Staff. Dev. 24 (5), 232–238.

Shermont, H., Krepcio, D., 2006. The impact of culture change on nurse retention. JONA 36 (9), 407–415.

Silvestre, J.H., Ulric, B.T., Johnson, T., Spector, N., Blegen, M.A., 2017. A multisite study on a new graduate registered nurse transition to practice program: return on investment. Nurs. Econ. 35 (3), 110–118.

Trepanier, S., Early, S., Ulrich, B., Cherry, B., 2012. New Graduate Nurse Residency Program: a cost-benefit analysis based on turnover and contract labor usage. Nursing Economics 30 (4) .

Trice, L.B., Brandvold, C., Bruno, E., 2007. Practice and education: partnering to address the perioperative nursing shortage. AORN J. 85 (2), 259–264.

Tufanaru, C., Munn, Z., Aromataris, E., Campbell, J., Hopp, L., 2017. Chapter 3: systematic reviews of effectivenessIn: Aromataris, E., Munn, Z. (Eds.), Joanna Briggs Institute Reviewer's Manual. The Joanna Briggs Institute. . Available from https://reviewersmanual.joannabriggs.org/.

Twigg, D., McCullough, K., 2014. Nurse retention: a review of strategies to create and enhance positive practice environments in clinical settings. Int. J. Nurs. Stud. 51, 85–92.

Ulrich, B., et al., 2010. Improving retention, confidence, and competence of new graduate nurses: results from a 10-year longitudinal database. Nurs. Econ. 28 (6), 363–375.

Van Camp, J., Chappy, S., 2017. The effectiveness of nurse residence programmes on retention: a systematic review. AORN J. 106 (2), 128–144. doi:http://dx.doi.org/ 10.1016/j.aorn.2017.06.003.

Van der Heed, K., Aiken, L.H., 2013. Nursing workforce a global priority area for health policy and health services research: a special issue. Int. J. Nurs. Stud. 50, 141–142.

Vergara, J.Y., 2017. Implementation of a mentorship program to increase staff satisfaction and retention in critical care. Nurse Lead. 207–217.

Whitehead, J., 2001. Newly qualified staff nurses’ perceptions of the role transition. Br. J. Nurs. 10 (5), 330–339.

Whitehead, B., Owen, P., Holmes, D., Beddingham, E., Simmons, M., Henshaw, L., Barton, M., Walker, C., 2013. Supporting newly qualified nurses in the UK: a systematic literature review. Nurse Educ. Today 33, 370–377.

Williams, T., Sims, J., Burkhead, C., Ward, P.M., 2002. The creation, implemen- tation, and evaluation of a nurse residency program through a shared leadership model in the intensive care setting. Dimens. Crit. Care Nurs. 21 (4), 154–161.

World Health Organisation, 2017. Global Strategy on Human Resources for Health: Workforce 2030 [Online] Available at:. . . (Accessed 13 December 17) http:// www.who.int/hrh/resolutions/en/.

J. Brook et al. / International Journal of Nursing Studies 91 (2019) 47–59 59

World Health Organization, 2010. A Global Survey Monitoring Progress in Nursing and Midwifery Available at:. WHO, Geneva. . (Accessed 3 September 18) http:// www.who.int/hrh/resources/survey/en/.

Wright, C., McCartt, P., Raines, D., Oermann, M.H., 2017. Implementation and evaluation of self-scheduling in a hospital system. J. Nurses Prof. Dev. 33, 19–24.

Zander, B., Aiken, L.H., Busse, R., Rafferty, A.M., Sermeus, W., Bruyneel, L., 2016. The state of nursing in the European Union. Eurohealth 22 (1), 3–6.

Zhang, Y., et al., 2016. The effectiveness and implementation of mentoring program for newly graduated nurses: a systematic review. Nurse Educ. Today 37, 136–144.

Zucker, B., Goss, C., Williams, D., Bloodworth, L., Lynn, M., Denker, A., Givvs, J., 2006. Nursing retention in the era of a nursing shortage. J. Nurses Staff. Dev. 22, 302–306.