readings summary

Staff perceptions of quality of care: an observational study of the NHS Staff Survey in hospitals in England

Richard J Pinder,1 Felix E Greaves,1 Paul P Aylin,2 Brian Jarman,2

Alex Bottle2

▸ Additional material is published online only. To view please visit the journal online (http://dx.doi.org/10.1136/bmjqs- 2012-001540).

1Department of Primary Care and Public Health, Imperial College London, London, UK 2Dr Foster Unit at Imperial College London, Imperial College London, London, UK

Correspondence to Dr Richard J Pinder, Department of Primary Care and Public Health, Imperial College London, Reynolds Building, St Dunstan’s Road, London W6 8RP, UK; [email protected]

Received 24 September 2012 Revised 9 January 2013 Accepted 13 January 2012 Published Online First 21 February 2013

To cite: Pinder RJ, Greaves FE, Aylin PP, et al. BMJ Qual Saf 2013;22: 563–570.

ABSTRACT Background There is some evidence to suggest that higher job satisfaction among healthcare staff in specific settings may be linked to improved patient outcomes. This study aimed to assess the potential of staff satisfaction to be used as an indicator of institutional performance across all acute National Health Service (NHS) hospitals in England. Methods Using staff responses from the NHS Staff Survey 2009, and correlating these with hospital standardised mortality ratios (HSMR), correlation analyses were conducted at institutional level with further analyses of staff subgroups. Results Over 60 000 respondents from 147 NHS trusts were included in the analysis. There was a weak negative correlation with HSMR where staff agreed that patient care was their trust’s top priority (Kendall τ = −0.22, p<0.001), and where they would be happy with the care for a friend or relative (Kendall τ = −0.30, p<0.001). These correlations were identified across clinical and non-clinical groups, with nursing staff demonstrating the most robust correlation. There was no correlation between satisfaction with the quality of care delivered by oneself and institutional HSMR. Conclusions In the context of the continued debate about the relationship of HSMR to hospital performance, these findings of a weak correlation between staff satisfaction and HSMR are intriguing and warrant further investigation. Such measures in the future have the advantage of being intuitive for lay and specialist audiences alike, and may be useful in facilitating patient choice. Whether higher staff satisfaction drives quality or merely reflects it remains unclear.

INTRODUCTION While the pursuit of high-quality healthcare has become the norm,1–3 there remains considerable argument over how to

evaluate performance.4 A broad range of performance indicators have been devel- oped,5 many of which have provoked con- siderable criticism, with particular concern voiced about whether they may mislead, among others, the public. The ideal per- formance indicator has been described as being meaningful, scientifically sound and interpretable.6 While many performance indicators provide useful feedback on spe- cific aspects of complex healthcare systems, providing a summary indicator encompass- ing multiple processes related to a meaning- ful outcome has proven a challenge. The Hospital Standardised Mortality Ratio (HSMR) is one such approach to providing a high-level summary.7 8 Critics suggest that measuring mortality is too blunt an approach, given that adjusting for case mix well is difficult, and that the proportion of deaths that are preventable is relatively small,9 although the performance of hospital-wide mortality measures as a screening tool depends on the definition of ‘preventable’.10

Despite this, in 2011, the English Department of Health (DH) committed to another overall mortality measure, the Summary Hospital-level Mortality Indicator, which summarises mortality during hospital admission and within 30 days of discharge.11 As such, hospital mortality is likely to continue to be consid- ered an important indicator for some time. Over the last decade, and alongside

repeated and substantial structural reorga- nisations of the UK’s National Health Service (NHS), questions have arisen over what improvement such changes may or may not have leveraged. In this time, there has been increasing recognition that quality improvement necessitates changes that go beyond structures, and interest

ORIGINAL RESEARCH

Pinder RJ, et al. BMJ Qual Saf 2013;22:563–570. doi:10.1136/bmjqs-2012-001540 563

L ib

ra ry-S

e ria

ls. P ro

te cte

d b

y co p yrig

h t.

o n N

o ve

m b

e r 2

9 , 2

0 2

1 a

t P o

rtla n

d S

ta te

U n ive

rsity h ttp

://q u a litysa

fe ty.b

m j.co

m /

B M

J Q u

a l S

a f: first p

u b

lish e

d a

s 1 0

.1 1

3 6

/b m

jq s-2

0 1

2 -0

0 1

5 4

0 o

n 2

0 F

e b ru

a ry 2

0 1 3 . D

o w

n lo

a d e d fro

m

has grown in the concept of what has been termed ‘organisational culture’ and how this might relate to healthcare performance.12 Yet there is little robust evidence supporting what might be anticipated to be a straightforward association, due in part to the chal- lenge of quantifying organisational culture as a phe- nomenon.13 However, in their final report, Mannion and colleagues conclude that organisational culture appears to be linked to performance, though they expressed reservations in respect of inferring causality.12

These caveats aside, one approach to evaluating organisational culture is to look at its effects, more specifically on morale, or as a proxy thereof, such as staff satisfaction. However, there has been relatively little published on these more subjective metrics. Although it has been suggested that staff satisfaction may have a role in evaluating overall organisational performance,14 the relationship between staff satisfac- tion and clinical care or outcomes has received less attention. There is, however, evidence that more satis- fied doctors have safer prescribing practices,15 and effect higher levels of concordance among their patients.16 Higher satisfaction among nurses has been linked with better safety,17 shorter length of stay18

and higher patient satisfaction.19–21 Limitations estab- lishing the potential direction of causality are a theme that runs through this literature, reflecting methodo- logical challenges to establishing such a connection. In 2010, a public inquiry was announced in response

to concerns about care provided at a UK hospital trust. Although The Mid Staffordshire NHS Foundation Trust Public Inquiry will report in early 2013, it has already heard from the Medical Director of the NHS in England who has put on record his belief that pro- cesses, including the monitoring of HSMR and the Staff Survey, may have flagged problems at this hospital at an earlier stage than did occur.22 23

Using data taken from the NHS Staff Survey 2009, in this study we aimed to evaluate the potential use of three candidate indicators by comparing them with HSMR for acute NHS trusts in England, including the indicator highlighted by the NHS Medical Director in his evidence. We hypothesised that higher levels of staff satisfaction would be associated with lower HSMR. More specifically, we hypothesised that satis- faction among clinical—especially medical—staff would be more closely correlated with HSMR than non-clinical staff.

METHODS Data sources Trust-level clinical dataset Data on hospital admissions from all NHS hospital trusts in England were extracted from the Hospital Episode Statistics (HES) system for the year 2009–2010. This administrative dataset comprises demographic and clin- ical data. Using the methodology used by Dr Foster

Intelligence,24 HSMRs were calculated, which adjusted mortality rates for available demographic and case-mix information.

Staff dataset Since 2003, the NHS has conducted an annual survey of staff to provide feedback on their experience of working within the service. The results are intended to manage local performance as well as provide information for regulators and the DH. In April 2009, the health service regulator, the Care Quality Commission, took over the running of this annual survey. A quality-assured copy of the dataset was obtained through the UK Data Archive,25 a designated ‘Place of Deposit’ by the UK National Archive. For the NHS Staff Survey 2009, 2 88 000 members of staff were invited by their local trust to take part in the survey,26 representing approximately 24.5% of all staff.27 The sample size from each hospital was deter- mined by the number of employees, ranging from a census for institutions of less than 600 to a sample of 850 for institutions with more than 3000 staff.26

Following two reminders, and between September and December 2009, 1 56 951 paper questionnaires (54.5% of those invited) were returned. From the 2009 survey, we selected three questions a

priori that, we hypothesised, may reflect staff satisfac- tion and organisational culture. Staff were presented with the following statements: (1) ‘Care of patients is my trust’s top priority’; (2) ‘If a friend or relative needed treatment, I would be happy with the standard of care provided by this trust’; (3) ‘I am satisfied with the quality of care I give to patients’. To each of these questions they were asked to select one of the follow- ing five responses: ‘strongly disagree’, ‘disagree’, ‘neither agree nor disagree’, ‘agree’, or ‘strongly agree’. In turn, each of these responses was accorded a number ranging from 1 to 5 with 1 representing that they ‘strongly disagree’ and 5 representing that they ‘strongly agree’. The responses to each question were aggregated at trust level, and a mean response ranging from 1 to 5 was attributed to each trust. A further variable was created for each hospital,

giving the proportion of respondents that agreed (aggregating those who ‘strongly agree’ and ‘agree’). Within the questionnaire, participants were asked to

select their staff group. A further variable was gener- ated that recategorised participants into clinical and non-clinical groups, with the latter including adminis- trative and ancillary staff. Where trusts or providers had changed or been reconfigured, responses were collated at institutional level to bring them in line with current NHS service configuration.

Statistical analysis The statistical analysis was conducted using STATA 12.0 for Mac.28 Due to having non-normal data, rank correlation analysis using Kendall’s τ with 95% CIs

Original research

564 Pinder RJ, et al. BMJ Qual Saf 2013;22:563–570. doi:10.1136/bmjqs-2012-001540

L ib

ra ry-S

e ria

ls. P ro

te cte

d b

y co p yrig

h t.

o n

N o ve

m b

e r 2

9 , 2

0 2

1 a

t P o

rtla n

d S

ta te

U n ive

rsity h ttp

://q u a litysa

fe ty.b

m j.co

m /

B M

J Q u

a l S

a f: first p

u b

lish e

d a

s 1 0

.1 1

3 6

/b m

jq s-2

0 1

2 -0

0 1

5 4

0 o

n 2

0 F

e b ru

a ry 2

0 1 3 . D

o w

n lo

a d e d fro

m

was conducted with pairwise correlation.29 Because the number of respondents per institution was rela- tively similar (mean 409, IQR 360–448), trusts were given equal weights in the correlation analysis. In order to test whether agreement was different

between subgroups, Z-tests were performed on the Kendall τ coefficients, from which p values were also calculated.

RESULTS Respondents numbering 77,730 were included in the analysis from 147 NHS trusts for which HSMRs had been calculated; HSMRs are calculated for acute general hospitals and do not include specialist units or mental health facilities. The HSMR for the 147 acute hospitals ranged from 71.9 to 117.9, with an IQR of 93.0 to 105.8 and a median of 99.5. The mean number of respondents per trust was

409. Within this sample of the wider dataset, it was not possible to determine the response rate for indi- vidual trusts. Respondents numbering 34,817 (58.0%) stated that

they agreed that care was their trust’s top priority compared with 9954 (16.5%) who disagreed (see table 1). Thirty-seven thousand two hundred and eighty-two (62.0%) agreed that they would be happy with the standard of care provided, were a friend or relative be in need of treatment, compared with 7327 (12.2%) who stated they were not. Albeit with fewer respondents, 45 028 (86.4%) agreed with the state- ment that they were satisfied with the quality of care they provided to patients; conversely, 3201 (5.3%) said they were not satisfied with the quality of care that they gave. When these data were analysed at organisational level, they showed marked variation,

with agreement ranging from approximately 70% up to 95% (see table 2). Pairwise correlation analysis (see table 3) revealed a

negative correlation between the ‘care as a priority’ statement and HSMR, suggesting that staff at hospitals with higher than expected death rates were less likely to agree that their Trust considered care of patients as their top priority (Kendall τ=−0.217, p<0.001). The feedback of non-clinical staff correlated as strongly as that of non-clinical staff. Agreement with the statement that staff would be

happy with the standard of care if a friend or relative needed treatment was also negatively correlated (Kendall τ=−0.198, p<0.001), with similar results in the clinical and non-clinical subgroups; there was no difference at conventional statistical significance between these two subgroups (p=0.50). In respect of the statement regarding satisfaction

with patient care provided by themselves, staff agree- ment was very weakly negatively correlated (Kendall τ=−0.062, p=0.27). Further analysis comparing the proportion of staff

responding that they agree or strongly agree against all others showed broadly similar but weaker associa- tions (see table 4). Further analysis shows variation across HSMR hos-

pital quartiles by staff group (see table 5). Again, medical and dental staff demonstrated the lowest agreement with the ‘care as a priority’ statement. These data, overall, showed a correlation between staff feedback for the ‘care as a priority’ and ‘if a friend or relative needed treatment’ questions, but showed no such correlation for the ‘satisfaction with care provided’ statement; these results are reflected by the pairwise correlation analyses (see table 6), which

Table 1 Overall staff responses to questions posed by National Health Service (NHS) Staff Survey 2009 questionnaire at all 147 acute general NHS hospitals in England

Statements Strongly disagree n (%)

Disagree n (%)

Neither agree nor disagree n (%)

Agree n (%)

Strongly agree n (%)

Care of patients is my trust’s top priority (n=60089) 3149 (5.2) 6805 (11.3) 15318 (25.5) 25875 (43.1) 8942 (14.9)

If a friend or relative needed treatment I would be happy to with the standard of care provided by this trust (n=60174)

1944 (3.2) 5383 (9.0) 15565 (38.0) 30796 (51.2) 6486 (10.8)

I am satisfied with the quality of care I give to patients (n=52121)

555 (0.9) 2646 (4.4) 3892 (7.5) 26381 (50.6) 18647 (35.8)

Table 2 Distribution of staff responses by National Health Service trust (n=147)

Hospital percentile

10th 25th 50th 75th 90th

% Agree* that ‘Care of patients is my trust’s top priority’ 67.2 72.6 78.1 83.4 87.1

% Agree* that ‘If a friend or relative needed treatment I would be happy to with the standard of care provided by this trust’

70.0 78.1 84.2 89.0 92.4

% Agree* that ‘I am satisfied with the quality of care I give to patients’ 90.7 92.3 93.5 95.0 95.9

*Includes those participants responding that they ‘Agree’ or ‘Strongly agree’.

Original research

Pinder RJ, et al. BMJ Qual Saf 2013;22:563–570. doi:10.1136/bmjqs-2012-001540 565

L ib

ra ry-S

e ria

ls. P ro

te cte

d b

y co p yrig

h t.

o n

N o ve

m b

e r 2

9 , 2

0 2

1 a

t P o

rtla n

d S

ta te

U n ive

rsity h ttp

://q u a litysa

fe ty.b

m j.co

m /

B M

J Q u

a l S

a f: first p

u b

lish e

d a

s 1 0

.1 1

3 6

/b m

jq s-2

0 1

2 -0

0 1

5 4

0 o

n 2

0 F

e b ru

a ry 2

0 1 3 . D

o w

n lo

a d e d fro

m

demonstrate that nursing staff agreement appeared most strongly correlated with performance as assessed by HSMRs.

DISCUSSION Statement of principal findings The data presented suggest that the majority of staff approved (albeit not strongly) of their institutions across the three domains of inquiry. Agreement with the ‘care as a priority’ and ‘if a friend or relative needed treatment’ statements correlated weakly with HSMR, lending weight to the principal hypothesis. These results suggest that staff feedback may be useful in assessing organisational performance as well as raising interesting questions over the potential value of negative feedback of this nature. That there is a cor- relation between HSMR and staff feedback in the expected direction gives some support to the use of adjusted hospital-wide mortality metrics, despite their relatively modest sensitivity to quality of care. However, these data do not support the hypothesis

that clinical staff satisfaction is more closely correlated with institutional mortality than non-clinical staff. Furthermore, satisfaction among nurses appeared

more closely correlated than did that of medical and dental staff, though this did not achieve statistical significance.

Main analysis The advantage that these indicators have over summary mortality data, length of stay or other metrics is their intuitive nature which may provide more intelligible information to facilitate patient choice. Second, the inter-relationship of organisational culture and performance highlights the importance of involving staff with organisational change, and that staff feedback may be a useful metric in identifying, as well as evaluating improvement of, suboptimal health services. The variation in responses to the three state- ments suggests a degree of specific interpretation and discrimination for each question, in contrast with indi- viduals responding negatively or positively across all the questionnaire domains. The statements regarding whether staff agree that

care is their trust’s top priority, and whether or not they would be happy with the care provided were a friend or relative to need it, are subtly different, though both perhaps assess overall performance.

Table 4 Pairwise Kendall-τ correlation analyses of staff agreement with hospital standardised mortality ratios at all 147 acute general National Health Service hospitals in England

Statement % Agree* Kendall τ (95% CI) p Value

Care of patients is my trust’s top priority Overall 77.6 −0.21 (−0.32 to −0.10) <0.001 Clinical 74.7 −0.20 (−0.30 to −0.09) <0.001

Non-clinical 82.9 −0.21 (−0.32 to −0.10) <0.001

If a friend or relative needed treatment I would be happy to with the standard of care provided by this trust

Overall 83.0 −0.18 (−0.03 to −0.07) 0.001 Clinical 81.8 −0.17 (−0.28 to −0.07) 0.002

Non-clinical 84.9 −0.17 (−0.28 to −0.05) 0.003

I am satisfied with the quality of care I give to patients Overall 93.4 −0.06 (−0.17 to 0.05) 0.27 Clinical 92.2 −0.04 (−0.02 to 0.07) 0.47

Non-clinical 96.2 −0.05 (−0.16 to 0.06) 0.35

*Includes those participants responding that they ‘Agree’ or ‘Strongly Agree’.

Table 3 Pairwise Kendall-τ correlation analyses with 95% CI of staff ratings (scale of 1–5) with hospital standardised mortality ratios at all 147 acute general National Health Service hospitals in England

Statement Responses Kendall τ (95% CI) p Value

Care of patients is my trust’s top priority Overall 60089 −0.22 (−0.33 to −0.11) <0.001 Clinical 38389 −0.20 (−0.30 to −0.09) <0.001

Non-clinical 21700 −0.23 (−0.34 to −0.11) <0.001

If a friend or relative needed treatment I would be happy to with the standard of care provided by this trust

Overall 60174 −0.20 (−0.31 to −0.09) <0.001 Clinical 38324 −0.19 (−0.30 to −0.08) <0.001

Non-clinical 21850 −0.19 (−0.30 to −0.08) <0.001

I am satisfied with the quality of care I give to patients Overall 52121 −0.09 (−0.20 to −0.02) 0.10 Clinical 37369 −0.08 (−0.18 to −0.03) 0.17

Non-clinical 14752 −0.06 (−0.16 to −0.05) 0.33

Original research

566 Pinder RJ, et al. BMJ Qual Saf 2013;22:563–570. doi:10.1136/bmjqs-2012-001540

L ib

ra ry-S

e ria

ls. P ro

te cte

d b

y co p yrig

h t.

o n

N o ve

m b

e r 2

9 , 2

0 2

1 a

t P o

rtla n

d S

ta te

U n ive

rsity h ttp

://q u a litysa

fe ty.b

m j.co

m /

B M

J Q u

a l S

a f: first p

u b

lish e

d a

s 1 0

.1 1

3 6

/b m

jq s-2

0 1

2 -0

0 1

5 4

0 o

n 2

0 F

e b ru

a ry 2

0 1 3 . D

o w

n lo

a d e d fro

m

Trusts may prioritise care; however, in a failing trust that does prioritise care, it remains possible that the quality of care delivered may not be of a satisfactory nature. This difference is reflected in the differing proportions with 14.9% of respondents strongly agreeing that care is their trust’s top priority, but only 10.8% strongly agreeing that they would be happy with care for their friend or relative. That HSMR did not appear to correlate with per-

ceptions of participants’ own satisfaction with the care that they personally delivered is of particular note, and may suggest a degree of cognitive dissonance. While healthcare professionals seek to deliver the highest quality of care, they can at least recognise fail- ures in the broader hospital environment or among other members of staff. Whether these responses high- light individuals perceiving that they deliver care of satisfactory quality despite broader organisational restriction, or whether they are simply unwilling to admit deficiencies in their practice, cannot be deter- mined. It may reflect the phenomenon of illusory superiority (termed by some the Lake Wobegon

effect30). Collectively, however, it highlights a poten- tial weakness inherent in this sort of feedback when used to measure healthcare quality. Yet these Kendall-τ correlations are weak by conven-

tional standards. For this, various explanations may be apparent. The comparison of two measures, both sur- rogates for overall quality, is unlikely to exhibit a perfect correlation given the imprecision associated with either metric, and which assess different aspects of performance. That the correlation is as weak as it is however, suggests that there are further contributing factors and potential confounders of which several are outlined in the limitations that follow. Despite these limitations, this correlation, though weak, remains intriguing.

Staff subgroup analysis That correlation of feedback from non-clinical staff was broadly similar to that of their clinical counter- parts highlights and reinforces the concept of organ- isational culture or morale. It is uncertain, however, whether perceptions among non-clinical staff are a

Table 5 Distribution of staff responses by Clinical Professional Group for HSMR Quartile

HSMR quartile*

First quartile (%)

Second quartile (%)

Third quartile (%)

Fourth quartile (%)

Agree† that ‘Care of patients is my trust’s top priority’ Medical/dental 66.2 69.0 71.2 71.8 Nursing 72.2 71.3 76.3 79.0 Allied health professionals

75.5 75.0 79.6 79.3

Agree† that ‘If a friend or relative needed treatment I would be happy to with the standard of care provided by this trust’

Medical/dental 77.9 81.2 83.6 86.9 Nursing 79.3 80.5 82.0 86.3 Allied health professionals

80.9 82.0 84.7 86.2

Agree† that ‘I am satisfied with the quality of care I give to patients’

Medical/dental 94.4 93.8 94.0 93.7 Nursing 91.0 89.6 90.8 91.1 Allied health professionals

94.5 94.7 95.3 94.4

*First HSMR quartile represents the quartile of hospitals with the highest standardised mortality rate, with the fourth quartile representing the lowest standardised mortality rate. †Includes those participants responding that they ‘Agree’ or ‘Strongly agree’. HSMR, hospital standardised mortality ratios

Table 6 Pairwise Kendall-τ correlation analyses with 95% CI of staff ratings (scale of 1–5) with hospital standardised mortality ratios by staff professional group at all 147 acute general National Health Service hospitals in England

Statement Responses Kendall τ (95% CI) p Value

Care of patients is my trust’s top priority Medical/dental 6422 −0.10 (−0.21 to −0.01) 0.06 Nursing 22291 −0.23 (−0.33 to −0.13) <0.001 Allied health professionals

11476 −0.11 (−0.22 to −0.00) 0.05

If a friend or relative needed treatment I would be happy to with the standard of care provided by this trust

Medical/dental 4614 −0.16 (−0.27 to −0.06) 0.003 Nursing 22232 −0.20 (−0.31 to −0.10) <0.001 Allied health professionals

11478 −0.12 (−0.23 to −0.01) 0.03

I am satisfied with the quality of care I give to patients Medical/dental 4592 0.02 (−0.08 to 0.12) 0.71 Nursing 21966 −0.07 (−0.18 to 0.03) 0.19 Allied health professionals

10811 −0.03 (−0.15 to 0.09) 0.56

Original research

Pinder RJ, et al. BMJ Qual Saf 2013;22:563–570. doi:10.1136/bmjqs-2012-001540 567

L ib

ra ry-S

e ria

ls. P ro

te cte

d b

y co p yrig

h t.

o n

N o ve

m b

e r 2

9 , 2

0 2

1 a

t P o

rtla n

d S

ta te

U n ive

rsity h ttp

://q u a litysa

fe ty.b

m j.co

m /

B M

J Q u

a l S

a f: first p

u b

lish e

d a

s 1 0

.1 1

3 6

/b m

jq s-2

0 1

2 -0

0 1

5 4

0 o

n 2

0 F

e b ru

a ry 2

0 1 3 . D

o w

n lo

a d e d fro

m

response to engaging with patients informally or for administrative reasons, with clinical staff for profes- sional reasons, or reflect a more general organisational culture. In particular, the interaction that this group has with patients may be more ‘honest’, as patients do not have to ‘fear reprisal’ in a way which they might, should they highlight concerns with nursing or medical staff. It is likely that all these factors have some degree of impact. The importance of these non- clinical staff groups should not be forgotten or side- lined, as they are likely to have a substantial impact on organisational effectiveness. More focused analysis of the different professional

groups making up the clinical group demonstrates the importance of nursing staff throughout the healthcare process. Nursing staff feedback was more closely cor- related with HSMR than that of medical or allied health professionals, although the difference in agree- ment between nurses and medical staff did not achieve statistical significance. Although nursing staff numbered more, nurses may be more attuned to the quality of healthcare delivered due to them providing the majority of healthcare interactions; when nursing feedback is compared with medical and dental feed- back, a similar phenomenon may be at work, with patients being more honest with nurses than they are with their doctors.

Strengths and weaknesses of the study The major strength of this study is its coverage of all acute general inpatient NHS services in England, encompassing many thousands of staff respondents. To the authors’ knowledge, this is the first time that the NHS Staff Survey has been correlated with outcomes, although previous work has linked positive staff per- ception of hospital cleanliness (from a previous staff survey) with improved patient experience metrics.31

However, caveats with this process should be noted. Primarily, the use of adjusted overall mortality mea- sures remains contentious and has clear weaknesses as outlined previously in this paper and elsewhere. It is one measure that attempts to capture the performance of the whole hospital without relying on self- reporting, but there are few others. A limitation of the NHS Staff Survey dataset was

that while the response rate overall was 54.5%, we were unable to determine the response rate for acute trusts overall, nor whether response rate varied between trusts. Given the nature of the data collection process (which was delegated to individual trusts), dis- entangling variation in response rate due to logistical reasons or other potential causes would be potentially challenging. Likewise, while the selection of respon- dents was intended to be at random, it is conceivable that trusts may have selected participants resulting in the potential for bias. However, trusts would not have been aware that the methodology employed by this study would use the responses collected, and whether

trusts have the means or intention to systematically attempt to ‘enhance’ staff feedback is doubtful. While the results are from the last few months of

2009, the HSMRs reflect performance from across the 2009–2010 financial year. Given the size of the institutions in the study, it is unlikely that substantial change would have occurred in the quarter following the survey. Furthermore, while the results in question are from 2009 and may not be representative of the organisations in 2012, as a proof of concept linking staff feedback to performance, the results are likely to be as valid in terms of correlation even if not exactly representing the NHS at present.

Mechanisms, unanswered questions and future research There are a number of possible mechanisms that may explain the association between staff morale and HSMR. The first, along the lines of the hypothesis, is that staff can discern quality in healthcare and reflect this by evaluation and survey. Alternatively, more satis- fied staff may provide a higher quality of care and, as such, the HSMR improves; conversely, hospitals with lower HSMR may employ more positive or optimistic staff. A further explanation may involve aspects of a positive feedback loop where staff are buoyed by good results in metrics, such as HSMR, and vice versa. However, the degree to which staff are aware of hospital performance beyond major scandals publi- cised in the media is uncertain. From the results ascer- tained, it is not possible to determine what the underlying mechanism/s is/are, and, like previous research, it is with caution that one should infer caus- ality, given the complexity of healthcare organisations and the staff within them. While the majority of respondents were positive

about their institutions, nearly a fifth of respondents were negative. It is possible that this small group of respondents may provide a sentinel marker of institu- tional poor performance. Further analysis within this group is on-going. Of particular interest are the effects of organisational culture across professional groups, both clinical and non-clinical. Understanding the effect that these different groups play in improv- ing healthcare is much sought after. Should staff satisfaction be considered for future

use, the possibility of gaming must be considered. The effect could be of organisations attempting to improve their staff feedback; providing that this was through genuine means of improving staff morale, and not simply selecting respondents to the question, this may be an asset in the longer term. Establishing causality among these variables is chal-

lenging, and inherently beyond the scope of a cross- sectional analysis. The potential for tracking these data in a longitudinal format presents an opportunity, not least with the NHS Staff Survey as a routine and annual process. Tracking changes in staff satisfaction and their temporal relationship may well be valuable

Original research

568 Pinder RJ, et al. BMJ Qual Saf 2013;22:563–570. doi:10.1136/bmjqs-2012-001540

L ib

ra ry-S

e ria

ls. P ro

te cte

d b

y co p yrig

h t.

o n

N o ve

m b

e r 2

9 , 2

0 2

1 a

t P o

rtla n

d S

ta te

U n ive

rsity h ttp

://q u a litysa

fe ty.b

m j.co

m /

B M

J Q u

a l S

a f: first p

u b

lish e

d a

s 1 0

.1 1

3 6

/b m

jq s-2

0 1

2 -0

0 1

5 4

0 o

n 2

0 F

e b ru

a ry 2

0 1 3 . D

o w

n lo

a d e d fro

m

in contributing to this discourse on causality. The potential for evaluation of these staff-side metrics with the growing literature on patient experience (whether by survey or by unsolicited online review32) also presents opportunities for future research.

CONCLUSIONS These results are intriguing and, while requiring further investigation, support the case for staff feed- back metrics to be considered as indicators for both professional and lay audiences. The primary advan- tage of these metrics over other established indicators is that they are intuitive to all stakeholders in health- care. They may also facilitate better communication between healthcare professionals, decision makers (including politicians) and the public at large. It may hold that in healthcare, as it does in many other areas of life, that it is the insiders that can signpost the path to higher quality care.

Contributors RJP conceived the study, and with AB developed the study design. All authors interpreted the data and critically reviewed drafts of the manuscript. RJP collected and analysed the data and prepared the manuscript. AB is guarantor.

Funding RJP is funded by an Academic Clinical Fellowship from the NIHR Integrated Academic Training Programme. The Department of Primary Care and Public Health at Imperial College is grateful for support from the NIHR Collaboration for Leadership in Applied Health Research and Care (CLAHRC) for North West London (a partnership between Chelsea and Westminster NHS Foundation Trust and Imperial College London), the NIHR Imperial Biomedical Research Centre (a partnership between Imperial College Healthcare NHS Trust and Imperial College London), and the NIHR Imperial Centre for Patient Safety and Service Quality. The authors would like to thank Hilary Watt for her advice on the statistical analysis.

Competing interests All authors have completed the unified competing interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author); BJ, PA and AB are part of the Dr Foster Unit at Imperial, which is principally funded via a research grant by Dr Foster Intelligence, an independent healthcare information company and joint venture with the Information Centre of the NHS. The authors’ work was independent of Dr Foster Intelligence, which had no role in the analysis, interpretation or decision to submit this paper. The Unit is affiliated with the NIHR Imperial Centre for Patient Safety and Service Quality at Imperial College Healthcare NHS Trust.

Ethics approval We have permission from the National Information Governance Board (NIGB) for Health and Social Care under Section 251 of the NHS Act 2006 (formerly Section 60 approval from the Patient Information Advisory Group) to hold confidential HES data. We have ethical approval to use them for research and measuring quality of delivery of healthcare, from the South East Ethics Research Committee.

Provenance and peer review Not commissioned; externally peer reviewed.

REFERENCES 1 Health and Social Care Act, 2012. 2 Darzi A. High quality care for all: NHS Next Stage Review final

report. London, UK: Department of Health, 2008. 3 Department of Health. The NHS Plan: a plan for investment, a

plan for reform. Whitehall, London, UK: Department of Health, 2000.

4 Shahian DM, Wolf RE, Iezzoni LI, et al. Variability in the measurement of hospital-wide mortality rates. N Engl J Med 2010;363:2530–9.

5 Copnell B, Hagger V, Wilson SG, et al. Measuring the quality of hospital care: an inventory of indicators. Intern Med J 2009;39:352–60.

6 McGlynn EA. Choosing and evaluating clinical performance measures. Jt Comm J Qual Improv 1998;24:470–9.

7 Bottle A, Jarman B, Aylin P. Strengths and weaknesses of hospital standardised mortality ratios. Brit Med J 2011;342:c7116.

8 Bottle A, Jarman B, Aylin P. Hospital standardized mortality ratios: sensitivity analyses on the impact of coding. Health Serv Res 2011;46(6pt1):1741–61.

9 Lilford R, Pronovost P. Using hospital mortality rates to judge hospital performance: a bad idea that just won’t go away. Brit Med J 2010;340.

10 Girling AJ, Hofer TP, Wu J, et al. Case-mix adjusted hospital mortality is a poor proxy for preventable mortality: a modelling study. BMJ Qual Saf 2012;21(12):1052–6.

11 Clinical Indicators Team. Indicator Specification: Summary Hospital-level Mortality Indicator: the Information Centre for Health and Social Care, 2011.

12 Mannion R, Davies H, Harrison S, et al. Changing Management Cultures and Organisational Performance in the NHS (OC2): National Institute for Health Research Service Delivery and Organisation programme, 2010.

13 Scott T, Mannion R, Marshall M, et al. Does organisational culture influence health care performance? A review of the evidence. J Health Serv Res Policy 2003;8:105–17.

14 Arnetz B. Subjective indicators as a gauge for improving organizational well-being. An attempt to apply the cognitive activation theory to organizations. Psychoneuroendocrinology 2005;30:1022–26.

15 Williams ES, Skinner AC. Outcomes of physician job satisfaction: a narrative review, implications, and directions for future research. Health Care Manage Rev 2003;28: 119–39.

16 DiMatteo MR, Sherbourne CD, Hays RD, et al. Physicians’ characteristics influence patients’ adherence to medical treatment: results from the Medical Outcomes Study. Health Psychol 1993;12:93–102.

17 Rathert C, May DR. Health care work environments, employee satisfaction, and patient safety: care provider perspectives. Health Care Manage Rev 2007;32:2–11.

18 Harmon J, Scotti DJ, Behson S, et al. Effects of high-involvement work systems on employee satisfaction and service costs in veterans healthcare. J Healthc Manag 2003;48:393–406; discussion 06–7.

19 Atkins PM, Marshall BS, Javalgi RG. Happy employees lead to loyal patients. Survey of nurses and patients shows a strong link between employee satisfaction and patient loyalty. J Health Care Mark 1996;16:14–23.

20 Kammerland P, Dahlgaard JJ, Rutberg H. Climate for improvement and the effects on performance in Swedish healthcare—a survey in the county council of Östergötland. Total Qual Manag Bus Excel 2004;15:909–24.

21 Peltier J, Dahl A, Mulhern F. The relationship between employee satisfaction and hospital patient experiences: Forum for Poeple Performance, Management and Measurement, 2009.

22 Witness Statement of Professor Sir Bruce Keogh. The Mid Staffordshire NHS foundation trust public inquiry. http://www. midstaffspublicinquiry.com/(accessed 20 Sep 2011): p. 68 (Paragraph 219).

Original research

Pinder RJ, et al. BMJ Qual Saf 2013;22:563–570. doi:10.1136/bmjqs-2012-001540 569

L ib

ra ry-S

e ria

ls. P ro

te cte

d b

y co p yrig

h t.

o n

N o ve

m b

e r 2

9 , 2

0 2

1 a

t P o

rtla n

d S

ta te

U n ive

rsity h ttp

://q u a litysa

fe ty.b

m j.co

m /

B M

J Q u

a l S

a f: first p

u b

lish e

d a

s 1 0

.1 1

3 6

/b m

jq s-2

0 1

2 -0

0 1

5 4

0 o

n 2

0 F

e b ru

a ry 2

0 1 3 . D

o w

n lo

a d e d fro

m

23 Inquiry Counsel’s written Closing Submission. Mid Staffordshire NHS foundation trust public inquiry. http://www. midstaffspublicinquiry.com/ (accessed 9 Dec 2011): Chapter 26, p. 1772 (Paragraph 212).

24 Aylin P, Bottle A, Jen MH, et al. HSMR Mortality Indicators: Dr Foster Unit at Imperial, Imperial College London, Dr Foster Intelligence, 2009.

25 Care Quality Commission and Aston University. Aston Business School, National Health Service National Staff Survey 2009 (computer file). Colchester, Essex, United Kingdom: Data Archive (distributor), 2009.

26 Care Quality Commission. National NHS Staff Survey, 2009. 27 The Information Centre for Health and Social Care. NHS staff

1999–2009 overview. National Health Service, Leeds, United Kingdom, 2010.

28 Stata: statistics and data analysis (program). 12.0 SE version. College Station, TX: StataCorp, 2011.

29 Kirkwood BR, Sterne JAC, Kirkwood BRE. Essential medical statistics. 2nd edn. Malden, Mass. Oxford: Blackwell Science, 2003:349–50.

30 Maxwell NL, Lopus JS. The lake-wobegon effect in student self-reported data. Am Econ Rev 1994;84:201–5.

31 Raleigh VS, Hussey D, Seccombe I, et al. Do associations between staff and inpatient feedback have the potential for improving patient experience? An analysis of surveys in NHS acute trusts in England. Qual Saf Health Care 2009;18:347–54.

32 Greaves F, Pape UJ, King D, et al. Associations between internet-based patient ratings and conventional surveys of patient experience in the English NHS: an observational study. BMJ Qual Saf 2012;21:600–5.

Original research

570 Pinder RJ, et al. BMJ Qual Saf 2013;22:563–570. doi:10.1136/bmjqs-2012-001540

L ib

ra ry-S

e ria

ls. P ro

te cte

d b

y co p yrig

h t.

o n

N o ve

m b

e r 2

9 , 2

0 2

1 a

t P o

rtla n

d S

ta te

U n ive

rsity h ttp

://q u a litysa

fe ty.b

m j.co

m /

B M

J Q u

a l S

a f: first p

u b

lish e

d a

s 1 0

.1 1

3 6

/b m

jq s-2

0 1

2 -0

0 1

5 4

0 o

n 2

0 F

e b ru

a ry 2

0 1 3 . D

o w

n lo

a d e d fro

m