RUA

RESEARCH ARTICLE Open Access

How do studies assess the preventability of readmissions? A systematic review with narrative synthesis Eva-Linda Kneepkens1†, Corline Brouwers2†, Richelle Glory Singotani2, Martine C. de Bruijne2 and Fatma Karapinar-Çarkit1*

Abstract

Background: A large number of articles examined the preventability rate of readmissions, but comparison and interpretability of these preventability rates is complicated due to the large heterogeneity of methods that were used. To compare (the implications of) the different methods used to assess the preventability of readmissions by means of medical record review.

Methods: A literature search was conducted in PUBMED and EMBASE using “readmission” and “avoidability” or “preventability” as key terms. A consensus-based narrative data synthesis was performed to compare and discuss the different methods.

Results: Abstracts of 2504 unique citations were screened resulting in 48 full text articles which were included in the final analysis. Synthesis led to the identification of a set of important variables on which the studies differed considerably (type of readmissions, sources of information, definition of preventability, cause classification and reviewer process). In 69% of the studies the cause classification and preventability assessment were integrated; meaning specific causes were predefined as preventable or not preventable. The reviewers were most often medical specialist (67%), and 27% of the studies added interview as a source of information.

Conclusion: A consensus-based standardised approach to assess preventability of readmission is warranted to reduce the unwanted bias in preventability rates. Patient-related and integrated care related factors are potentially underreported in readmission studies.

Keywords: Hospital readmission, Avoidability, Preventability, Assessment, Review, Patient interview

Background The general goal of hospital care is to restore the patient’s health condition to the pre-admission state or to discharge the patient in the best possible health con- dition. Nevertheless, approximately 20% of the hospital admissions in the US result in an unplanned readmission within 30 days after discharge, of which a subset is preventable [1]. These readmissions result in an increase in cost, workload for caregivers and a potential health risk

for patients [2]. Hence, hospital readmission rates are increasingly being used to monitor quality improvement and cost control [3]. Currently, hospitals are being bench- marked in several countries based on their readmissions rate. In some of these countries, high rates can result in financial penalties and they are used as a policy to stimu- late hospitals to implement improvement plans [4]. These improvement plans are generally complex and

costly, therefore, prediction models to identify patients who are at risk for readmissions are being developed [5]. However, these models are often not validated pros- pectively or in other datasets [6]. Furthermore, electronic prediction algorithms tend to overestimate potentially pre- ventable readmissions [7]. It is important to understand

© The Author(s). 2019 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

* Correspondence: [email protected] E.L. Kneepkens and C. Brouwers are shared first authors †E.L. Kneepkens and C. Brouwers contributed equally to the manuscript 1Department of Clinical Pharmacy, OLVG Hospital, Jan Tooropstraat 164, 1061 AE Amsterdam, The Netherlands Full list of author information is available at the end of the article

Kneepkens et al. BMC Medical Research Methodology (2019) 19:128 https://doi.org/10.1186/s12874-019-0766-0

the complex mechanism behind readmissions and to achieve an accurate prediction of preventable read- missions. This can be achieved through medical record review, preferably combined with narratives obtained from patient interviews [7], and other sources, such as a general practitioner (GP). Many studies have examined the preventability rate of

readmissions, but comparison and interpretability of these preventability rates are complicated by the large heterogeneity of methods used to assess the preventa- bility [8]. In addition, (systematic) reviews that studied the preventability of readmissions did not focus on the method of assessment, and whether specific metho- dological options affect the likelihood of finding a high or low preventability rate [7, 9–11]. Understanding the implications of different methodological options could aid in solving a piece of the readmission puzzle. There- fore, the objective of this study is to compare methods and discuss all studies in which preventability of hospital readmissions was assessed by use of medical record review. By these means, we hope to provide the reader guidance in how to conduct and report their study data on readmissions.

Methods Data source and searches A systematic literature search was applied in Pubmed and Embase in December 2016. In the first step of the search strategy(MeSH and tiab)-terms for “readmission” and “rehospitalization” were combined with terms such as “avoidability” or “preventability” (see Additional file 1). In the next step this search was combined with terms such as “quality of health care”, “quality indicators”, and “chart review”. In the last step conference abstracts were ex- cluded from the search. For this search a medical informa- tion specialist was consulted. All citations were imported into Endnote X 7.3.1TM.

Study selection A stepwise study selection (described below) was con- ducted using a consensus-based approach. In case of dis- agreement, an independent senior researcher was consulted (FKC and MdB).

� Step 1: Two researchers (CB, EK) independently screened all abstracts using the major inclusion and exclusion criteria, i.e. English language, manual assessment using, at least, the medical record and a clear method description regarding preventability assessment in the aim, method or result section, see Additional file 2. Cohen’s kappa for interrater agreement (CB and EK) was good (k = 0.70).

� Step 2: References of included articles were assessed and a cited reference search in Web of Science and

Scopus (CB and EK) was performed additionally for all full text articles included in step 1 (n = 77).

� Step 3: Detailed inclusion and exclusion criteria (Additional file 2) were applied to all 77 articles by two researchers independently (equally divided over CB, EK, RS). This additional step was conducted to ensure that the finally selected articles were able to help us reach our study objective; 1. Full text article in English; 2. The article should be based on original patient data; in case of ≥2 or more papers used the same, or partly the same, patient sample only the paper with the most thoroughly described methodology of preventability assessment was included; 3. Studying hospital readmissions should be clearly stated in the aim/ primary objective; 4. Duration between index and readmission should be ≤6 months; 5. Assessment of preventability should be performed via manual medical record review or at least, it should be clear that the preventability assessment was performed on an individual patient level by a care provider and/or trained researcher which cannot be performed without a review; 6. The methodology of the preventability assessment of readmissions should be described clearly in order to perform data-synthesis; this includes a description of criteria of preventability and/or a cause classification (≥3 cause categories) of preventable readmissions and the reviewer process (at least 2 independent reviewers and disagreement should have been solved by reaching consensus and/ or a third independent reviewer OR, in case not performed/ nor reported (NR) > 50 medical files of readmitted patients should have been reviewed).

Critical appraisal of individual sources of evidence A validated critical appraisal was performed to evaluate the reliability, value and relevance of each article. Com- monly used quality appraisal tools were not suitable because of the large heterogeneity in study designs. Hence, a critical appraisal tool was used which is developed by the Cochrane recommendations for narrative data syn- thesis and analysis [12]. This critical appraisal was im- plemented in the data synthesis. The goal of using the narrative synthesis is, similar to other appraisal tools, to avoid bias. The process of narrative data synthesis is rigo- rous and transparent, in which the process is specified in advance. These process steps were followed systematically.

Data synthesis A (textual) narrative synthesis was performed to compare the methods of the included studies and this led to the identification of a set of important variables. The following variables were systematically collected and described in the Result section: study design characteristics, sources of

Kneepkens et al. BMC Medical Research Methodology (2019) 19:128 Page 2 of 12

information to assess preventability, definition of prevent- ability, cause classification (classifying the cause of a re- admission) and reproducibility (i.e. the reviewer process and training) (see Additional file 3). There are several important considerations to take into

account prior to reading the results; (1) the cause classifi- cation and preventability assessment are often integrated; meaning specific causes were predefined as always pre- ventable or not preventable. These studies were called a priori preventability cause classifications; (2) some articles reported the number and percentage of readmissions while others reported the number of readmitted patients, or both. For the purpose of this article, we reported the percentage of preventable readmissions/readmitted patients based on the actual number of reviewed files within one month (if this could be extracted from the provided data); (3) cause classification refers to description of at least three causes; (4) lastly, the index admission is the admission prior to readmission.

Data extraction and analysis Data was collected (CB, EK, RS) using a predefined form which included study characteristics and relevant data with regard to the method of preventability assessment. During the preliminary data synthesis, all data extracted by one researcher was checked by at least one other researcher (CB, EK, RS). During the systematic approach a double check or consensus-discussion was only per- formed in case of doubt because all definitions were thoroughly discussed after the preliminary phase. Lastly, potential associations between preventability rates and study characteristics were explored using the indepen- dent sample t test, Mann-Whitney u test or χ2 test depending on the variable distribution. A value of < 0.05 was considered to be statistically significant. The data were analysed with SPSS version 21.0 software (IBM, New York, USA).

Results Abstracts of 2504 unique citations were screened resulting in 77 full text articles that reported on the assessment of preventability. Step 3 of the stepwise study selection resulted in the final inclusion of 48 (64%) articles. The other studies (n = 29) were ex- cluded because the primary objective of the paper was not focussed on readmissions, the duration (dis- charge index admission to readmission) was longer than 6 months, or because the readmission method of preventability assessment was not explicitly described in the method section. A minimal dataset for the excluded articles, and the reason for exclusion, is shown in Additional file 4. An overview of the selection process is shown in Fig. 1.

Study design and characteristics As shown in Table 1, the studies were published between 1988 and 2017, often as single center studies (n = 37; 77.1%) and often performed in the USA (n = 32; 66.6%). Twelve studies focused on a specific diagnosis (n = 12) or a group (e.g. elderly or children) within a single department (e.g. internal medicine). Furthermore, nine studies examined all-cause readmissions, meaning that patients readmitted at all departments were eligible for inclusion [13–21]. Additional file 5 provides more detailed information on the descriptive characteristics of the studies.

Sources of information Thirteen articles (n = 13) used additional sources of infor- mation, such as interviews, questionnaires or surveys, in addition to the manual medical record review, see Table 1 [14, 21–32]. Additional file 6 provides more information on the interviews with care providers and/or patients. In 7 studies the patient was approached [21–23, 25, 30–32] and in 5 studies the patient or caregiver was approached [14, 26–29]. In 4 studies it was mentioned that the results of the interview were available for the reviewers during their assessment of preventability, however, it was not specified if and how these results influenced the prevent- ability assessment [14, 22, 26, 29]. In the paper of Toomey et al. [27] the preventability was first assessed without the interview results. Subsequently, the interview results were shared with the reviewer and it was documented how this additional information changed the review outcome. This resulted in new information in 31.2% of the cases and a change in the final preventability score in 11.8%. However, no further details were published regarding which in- formation of the interview was crucial for the reviewer to change his or her opinion. The other 5 studies did not specify whether or not the additional patient/caregiver information was used to assess the preventability [25, 28, 30–32]. In the study of Burke et al. [23], only 6 patients were interviewed during a pilot phase. After the pilot, they concluded that the interviews did not provide additional data to the patient’s medical record. Six studies interviewed at least one care provider, of

which mostly the GP, see Additional file 6. Four studies reported that the results of the care provider interview were available for the reviewers [14, 22], or were in- cluded in the preventability judgement [26] and one re- ported that the opinion of the interviewee was included in the final preventability judgement via equal weighing of their opinion with the opinion of the audit team [24].

Preventability A subset of articles used a very broad definition of preventability, such as the study of Ryan et al.;

Kneepkens et al. BMC Medical Research Methodology (2019) 19:128 Page 3 of 12

‘Providers were given no specific guidelines for deciding whether a readmission was preventable. This allowed use of their different backgrounds in choosing which elements of the clinical record to focus on.’ [33]

In addition, the majority of the articles did not explicitly provide the definition of preventability, instead they often directly referred to the cause classification (see Additional file 7), such as Williams et al.; ‘It was noted that readmis- sion could have been avoided if more effective action had been taken in one or more of five areas: preparation for and timing of discharge, attention to the needs of the carer, timely and adequate information to the general practi- tioner and subsequent action by the general practitioner, sufficient and prompt nursing and social services support, and management of medication.’ [28]

Cause classification The cause classification (the description of at least three causes) that was used by the studies varied largely. Several studies used an existing tool, like the STate Action on Avoidable Rehospitalizations (STAAR) initiative [14, 21, 27, 30, 34] or root cause approach [5, 18, 24, 35–37] but all others adapted an existing tool or developed their own

tool based on previous publications. For the purpose of this article we focused only on the distinction between studies using an a priori preventability cause classification [13–16, 19, 21–26, 31, 35, 37–55], or not [5, 17, 18, 20, 27–30, 32, 33, 36, 56–59], see Table 2. As an example of an a priori cause classification, Clarke et al. reported, Unavoidable causes: chronic or relapsing disorder; un- avoidable complication, readmission for social or psy- chological reason, reasons probably beyond control of hospital services, completely different diagnosis from previous admission. Avoidable causes: recurrence or continuation of disorder leading to first admission, recognised avoidable complication, readmission for social or psychological reason, reasons probably within control of hospital services. [39] The majority of the studies did not report whether

they assessed the causal relationship (i.e. whether the readmission is related to the care provided during index admission) explicitly, but ‘causative or causal’ could be extracted from the cause and/or preventability criteria [15, 16, 23, 32, 43, 44, 52, 53]. In addition, a few articles included information on ‘related readmissions’. These readmissions were defined as related based on the same diagnosis (or complication), the same department, or

Fig. 1 PRISMA 2009 Flow Diagram

Kneepkens et al. BMC Medical Research Methodology (2019) 19:128 Page 4 of 12

medical/clinically related [13, 20, 35, 37, 38, 40, 42, 48–51, 56, 57]. Another term used was ‘causation’ [18, 27, 32].

Reproducibility/reviewer process As shown in Table 2, the number of reviewers varied between 1 and 35. Four studies had ≥10 reviewers [17, 32, 36, 43]. The reviewers were most often physicians (spe- cialists) or a combination hereof [5, 13, 15–18, 20–23, 25, 27, 28, 30–32, 35–39, 41, 42, 45, 47, 50, 51, 53–55, 57, 58]. A subset of studies included a multidisciplinary study team consisting of physicians, general practitioners, a medical officer, case managers, (specialized) nurses, me- dical record specialists, social workers and/or administra- tive staff [14, 24, 29, 33, 44, 46, 48]. In three studies senior residents performed the review supervised by a senior physicians [19, 26, 59]. In five studies no information on expertise was reported [40, 49, 52, 56, 60]. As shown in Table 2 roughly three options for review

were possible: a single reviewer without a double check [13, 17, 28, 38, 51, 59], a single reviewer double checked by a second reviewer [15, 18, 32, 36, 45] or a team [24, 40, 43] or a team of 3 to 4 persons which reviewed the readmissions directly [20, 25, 27, 33, 41, 49, 54]. Agree- ment and consensus regarding the preventability was handled differently: a double review of each readmission was performed meaning that both reviewers assessed the preventability of the readmissions and came to a mutual agreement [14, 16, 18, 19, 22, 23, 29–31, 35, 42, 44, 46, 47, 50, 52, 53, 57]. In some cases a team or panel was consulted when mutual agreement on the preventability was not achieved [5, 48, 55, 60]. Two studies could not be allocated to one of these review categories because the review process was not clearly described or because they used a mix of different methods [39, 56]. A subset of the included articles offered some kind of

support to the reviewers to clarify and solidify classifi- cation criteria, to increase the uniformity between the assessments or to refine the study logistics and/or survey instrument or implemented as an educational program [59]. The support was mainly provided by means of a training, instruction session, pilot [17, 22, 27, 32, 36, 42, 52] and/or discussion of preventable causes and readmissions [14, 16, 18, 27, 36, 37, 42, 52, 53, 55]; other options were: a study protocol or review

Table 1 Descriptives of included studies

Study characteristics (n = 48) No. or percentage of studies

Year of publication, range 1988–2016

Country, n (%)

USA 32 (67%)

Other 16 (33%)

Study design, n (%)

Retrospective 30 (63%)

Cross-sectional 10 (21%)

Prospective 8 (16%)

Setting, n (%)

Single center 37 (77%)

Multicenter 11 (23%)

Number of readmissions reviewed, n ± SD 226 ± 208

Planned readmission excluded, n (%)

Yes 30 (63%)

No 11 (23%)

Not reported 7 (14%)

All-cause readmission, n (%)

Yes 9 (19%)

No 39 (81%)

Percentage preventable readmissions, mean, ± SD 27,8 ± 16,7%

Scoring of preventability, n (%)

Binary 22 (46%)

Scale 4 (8%)

Categorical 17 (35%)

Not applicable (a priori studies) 5 (11%)

A priori preventable causes determined, n (%)

Yes 32 (67%)

No 16 (33%)

Training of reviewers, n (%)

Yes 16 (33%)

No 2 (4%)

Not reported 30 (63%)

Number of reviewers, n (%)

Individual 8 (16%)

Duo 23 (48%)

Duo + team 2 (4%)

Individual + team 2 (4%)

Team 5 (11%)

Individual or duo + panel 3 (6%)

Other 5 (11%)

Double check, n (%)

All cases 28 (58%)

Partially 7 (15%)

Table 1 Descriptives of included studies (Continued)

Study characteristics (n = 48) No. or percentage of studies

No 3 (6%)

Not reported 10 (21%)

Additional sources, n (%)

Interview or survey 13 (27%)

None 35 (73%)

Kneepkens et al. BMC Medical Research Methodology (2019) 19:128 Page 5 of 12

T a b le

2 Pr ev en

ta b ili ty

as se ss m en

t o f th e in cl u d ed

st u d ie s (N

= 48 )

A u th o r

Pl an n ed

re ad -m

is si o n s

ex cl u d ed

?a

N o .r ea d -

m is si o n s

re vi ew

ed b

N o .o f p re ve n ta b le

u n p la n n ed

re ad m is si o n s

% p re ve n ta b le

u n p la n n ed

re ad m is si o n sc

Sc o rin

g o f

p re ve n ta b ili ty

A p rio

ri p re ve n ta b le

ca u se s d et er m in ed

Tr ai n in g o f

re vi ew

er s

Re vi ew

er sd

D o u b le ch ec k

o f

p re ve n ta b ili ty

A d d it io n al so u rc es

u se d fo r th e re vi ew

A g ra w al

ye s

30 11

36 ,7

ca te g o ric al

ye s

n o

in d iv id u al

n o

–

A u er b ac h

ye s

10 00

26 9

26 ,9

sc al e

ye s

ye s

d u o

al l ca se s

In te rv ie w f

Ba lla

ye s

27 1

90 33 ,2

b in ar y

n o

N R

d u o

al l ca se s

In te rv ie w e

Bi an co

n o

22 9

10 0

43 ,7

b in ar y

ye s

ye s

d u o

al l ca se s

In te rv ie w e

Bu rk e

ye s

33 5

78 23 ,3

ca te g o ric al

ye s

ye s

d u o

al l ca se s

In te rv ie w e

C ak ir

N R

85 4

4, 7

ca te g o ric al

ye s

N R

in d iv id u al

N R

–

C la rk e

ye s

74 18 ,9

25 ,5

ca te g o ric al

ye s

N R

d u o o r te am

al l ca se s

–

D aw

es ye s

25 8

55 21 ,3

ca te g o ric al

ye s

ye s

d u o + te am

A ll ca se s

–

Ep st ei n

n o

50 1

2, 0

ca te g o ric al

ye s

N R

d u o + te am

A ll ca se s

–

Fe ig en

b au m

n o

53 7

25 0

46 ,6

ca te g o ric al

ye s

ye s

d u o

al l ca se s

In te rv ie w f

Fl u it m an

ye s

50 26

52 ,0

b in ar y

ye s

N R

d u o

al l ca se s

–

Fr an kl

ye s

31 8

28 8, 8

ca te g o ric al

ye s

N R

in d iv id u al + te am

p ar ti al ly

–

G au ta m

ye s

10 9

16 14 ,7

b in ar y

ye s

N R

in d iv id u al + te am

N R

In te rv ie w f

G la ss

N R

96 25

26 ,0

b in ar y

n o

N R

N R

N R

–

G re en

b er g

ye s

97 22

22 ,7

ca te g o ric al

ye s

N R

d u o

N R

–

H ai n

n o

20 0

40 20 ,0

sc al e

ye s

ye s

p an el

al l ca se s

–

H al fo n

ye s

42 9

40 9, 3

N A

ye s

N R

d u o

p ar ti al ly

–

H ar h ay

ye s

20 1

19 9, 5

b in ar y

ye s

ye s

d u o

al l ca se s

Ji m in ez -P u en

te n o

18 5

44 23 ,9

b in ar y

ye s

N R

d u o

al l ca se s

–

Jo n as

n o

24 8

15 6, 0

b in ar y

ye s

N R

in d iv id u al + p an el

p ar ti al ly

–

Ke lly

ye s

32 22

68 ,8

b in ar y

ye s

N R

d u o

al l ca se s

Ko ek ko ek

n o

29 8

45 15 ,1

ca te g o ric al

n o

ye s

in d iv id u al

N R

–

M au re r

ye s

32 3

9, 4

b in ar y

ye s

N R

d u o

p ar ti al ly

–

M ei se n b er g

ye s

72 22

30 ,6

b in ar y

ye s

N R

d u o

al l ca se s

–

M ile s

ye s

43 7

24 5, 5

ca te g o ric al

n o

N R

d u o

p ar ti al ly

–

M it ta l

ye s

35 15

42 ,9

b in ar y

n o

N R

d u o

al l ca se s

N ah ab

n o

17 4

92 52 ,9

N A

ye s

N R

d u o

al l ca se s

–

N ijh aw

an ye s

13 0

62 47 ,7

N A

ye s

N R

d u o + p an el

al l ca se s

–

N je im

N R

16 1

51 31 ,7

b in ar y

n o

ye s

in d iv id u al

n o

–

O d d o n e

N R

51 4

18 3

34 ,2 *

ca te g o ric al

n o

ye s

d u o

p ar ti al ly

–

Pa ce

ye s

14 0

19 13 ,9

ca te g o ric al

ye s

N R

d u o

al l ca se s

–

Kneepkens et al. BMC Medical Research Methodology (2019) 19:128 Page 6 of 12

T a b le

2 Pr ev en

ta b ili ty

as se ss m en

t o f th e in cl u d ed

st u d ie s (N

= 48 ) (C on

tin u ed )

A u th o r

Pl an n ed

re ad -m

is si o n s

ex cl u d ed

?a

N o .r ea d -

m is si o n s

re vi ew

ed b

N o .o f p re ve n ta b le

u n p la n n ed

re ad m is si o n s

% p re ve n ta b le

u n p la n n ed

re ad m is si o n sc

Sc o rin

g o f

p re ve n ta b ili ty

A p rio

ri p re ve n ta b le

ca u se s d et er m in ed

Tr ai n in g o f

re vi ew

er s

Re vi ew

er sd

D o u b le ch ec k

o f

p re ve n ta b ili ty

A d d it io n al so u rc es

u se d fo r th e re vi ew

Ry an

ye s

40 N R

26 ,7

ca te g o ric al

n o

ye s

te am

al l ca se s

–

Sa u n d er s

ye s

28 2

51 18 ,1

b in ar y

ye s

N R

te am

al l ca se s

–

Sh ah

n o

40 7

14 9

36 ,6

N A

ye s

N R

d u o

al l ca se s

–

Sh al ch i

N R

63 45

71 ,4

b in ar y

n o

N R

te am

al l ca se s

–

Sh im

iz u

n o

15 3

50 32 ,7

b in ar y

ye s

N R

p an el

al l ca se s

In te rv ie w e

St ei n

ye s

21 3

64 29 ,5

b in ar y

n o

N R

in d iv id u al

N A

In te rv ie w f

Su th er la n d

ye s

47 11

23 ,4

N A

n o

N R

in d iv id u al

N R

In te rv ie w e

Te je d o r- So jo

n o

14 7

62 42 ,2

ca te g o ric al

ye s

ye s

te am

N R

–

To o m ey

ye s

30 5

90 29 ,5

sc al e

n o

ye s

te am

al l ca se s

In te rv ie w f

Va ch o n

ye s

98 14

14 ,3

b in ar y

ye s

N R

in d iv id u al

N R

–

Va n W al ra ve n

ye s

31 7

70 22 ,1

sc al e

n o

ye s

d u o

p ar ti al ly

In te rv ie w e

Vi n so n

N R

66 10

15 ,2

ca te g o ric al

n o

N R

d u o

N R

In te rv ie w e

W al la ce

ye s

20 4

41 20 ,1

b in ar y

ye s

ye s

d u o

al l ca se s

–

W as fy

N R

89 3

38 0

42 ,6

ca te g o ric al

ye s

ye s

d u o

al l ca se s

–

W ei n b er g

ye s

50 3

6, 0

b in ar y

ye s

N R

p an el

al l ca se s

–

W ill ia m s

ye s

13 3

78 58 ,6

b in ar y

n o

N R

in d iv id u al

n o

In te rv ie w f

Ya m

ye s

60 3

24 6

40 ,8

b in ar y

n o

n o

D u o + p an el

al l ca se s

– a P la n n ed

re ad

m is si o n s w er e co n si d er ed

ex cl u d ed

w h en

th e p la n n ed

re ad

m is si o n s w er e ex cl u d ed

b ef o re

p re ve n ta b ili ty

w as

as se ss ed

. b N u m b er

o f re vi ew

ed ca se s is b as ed

o n th e n u m b er

o f in cl u d ed

p at ie n ts

fo r w h o m

p re ve n ta b ili ty

o f a re ad

m is si o n w as

as se ss ed

, b as ed

o n th e n u m b er

o f in cl u d ed

re ad

m is si o n s fo r w h ic h p re ve n ta b ili ty

w as

as se ss ed

, o r b as ed

o n th e n u m b er

o f p re ve n ta b ili ty

as se ss m en

ts p er fo rm

ed .

c I n ca se

a st u d y ca lc u la te d th e p er ce n ta g e o f p re ve n ta b le

re ad

m is si o n s fo r m u lt ip le

ti m e d u ra ti o n s (t im

e b et w ee n in d ex

an d re ad

m is si o n ) th e ti m e d u ra ti o n o f 3 0 d ay s (o r cl o se st

to 3 0 d ay s) w as

ch o se n to

in cr ea se

th e co m p ar ab

ili ty

o f th e re su lt s w it h th e o th er

st u d ie s. *B as ed

o n p h as e 2 o f th e st u d y

d in d iv id u al = a si n g le

re vi ew

er in d ep

en d en

tl y as se ss ed

th e p re ve n ta b ili ty

o f th e re ad

m is si o n w it h o u t a d o u b le

ch ec k b y o th er

re vi ew

er s o r a co n se n su s m ee ti n g ;i n d iv id u al + te am

/p an

el = a si n g le

re vi ew

er in d ep

en d en

tl y as se ss ed

th e p re ve n ta b ili ty

o f th e re ad

m is si o n , b u t a d o u b le

ch ec k is p er fo rm

ed o n a se le ct io n o f ca se s; d u o = b o th

re vi ew

er s as se ss ed

th e p re ve n ta b ili ty

o f th e re ad

m is si o n s an

d ca m e to

a m u tu al

ag re em

en t; d u o + te am

/p an

el = b o th

re vi ew

er s as se d th e p re ve n ta b ili ty

ad d ed

b y a te am

o r p an

el w h ic h co u ld

ad vi se

th e tw

o re vi ew

er s in

ca se

a m u tu al

ag re em

en t o n th e p re ve n ta b ili ty

w as

n o t ac h ie ve d ; te am

o r

p an

el :c as es

ar e d ir ec tl y re vi ew

ed b y a te am

o f 3 to

4 p er so n s.

e In te rv ie w

(o r q u es ti o n n ai re

o r su rv ey ) w as

co n d u ct ed

w it h th e p at ie n t o n ly ;

f In te rv ie w

(o r q u es ti o n n ai re

o r su rv ey ) w as

co n d u ct ed

w it h th e p at ie n t an

d th e ca re

p ro vi d er

(g en

er al

p ra ct it io n er

o f p h ys ic ia n ).

Kneepkens et al. BMC Medical Research Methodology (2019) 19:128 Page 7 of 12

guide [22, 37, 40], a bimonthly meeting and/or an educational program [59]. Agreement was calculated in different ways: the inter-

rater agreement (i.e. kappa coefficient) [15, 16, 23, 30, 40, 42, 50, 52, 53, 60], intrarater reliability [49] or both [36]; other options were the interclass correlation and a concordance coefficient [39, 41] or the percentage of agreement on preventability [25, 33, 37, 43, 48, 55]. A low level of agreement was associated with the presence of multiple conditions; the more difficult it was to di- sentangle the reason for readmissions, the higher the chance of disagreement between the reviewers [39].

Discussion The aim of this study was to compare the currently avail- able methods to assess the preventability of readmissions, and the implications of these methods in terms of the preventability rates that were found. The focus on the methodology of preventability assessment is unique to this review and the results can be used to contribute to the development of a consensus-based approach to assess the preventability of readmissions. Furthermore, we aimed to provide the reader guidance in how to design, conduct and report their study in a well-considered manner. A large heterogeneity in study designs was identified

which limits the comparability of the preventability rates. In addition, it is currently not possible to distinguish which part of the variation in preventability rate really represents variation in quality of care. Only a consensus- based standardised approach to assess preventability can reduce the unwanted bias caused by methodological dif- ferences and contextual factors. The interpretation of the results was further compli-

cated by inconsistent use of important study definitions (i.e. definition of preventability). Studies were also contra- dictory, for example some studies regarded patient factors such as noncompliance as a potential preventable cause for readmissions as others regarded this non-preventable. Most studies used an a priori preventability cause classi-

fication approach which is less time-consuming to apply. An a priori approach is comparable with an electronic algorithm to predict potentially preventable readmissions. In these cases a prediction is based on a specific connec- tions between variables (i.e. matching or correlated admis- sion diagnosis codes). Such predictive algorithms, based on administrative data, are increasingly used. However, the performance (in terms of the discriminative ability) of risk predictive models has varied significantly [61]. Although, manually applying these algorithm rules may improve the likelihood of identifying true potentially pre- ventable readmissions, it still does not invite the reviewer to look beyond the predefined potential causes of prevent- ability. On the other hand, performing chart review is time-intensive and has a limited reproducibility. Our

results show that researchers try to optimize the reproducibility in different ways, e.g. the training of reviewers, a double check with the use of a second re- viewer and/or a (multidisciplinary) team. Nevertheless, these different variables were not significantly associated with preventability percentages. In the majority of studies the preventability assessment

was performed by a physician or several physicians (often from the same department or specialty). This might increase the risk of reluctancy to consider alternatives to one’s preferred line of thought (i.e. potential causes related to other specialties). In addition, many patients are treated by multiple care providers and this might complicate optimal assessment of the readmissions when a single (medical specialty) perspective is used [62]. It is currently unknown which readmissions should be reviewed by a multidisciplinary team and how that would affect the preventability outcome and the causes found. Most studies only assessed preventability based on chart

review. However, charts usually do not contain all the potential information that can influence the preventability assessment, for example information on the collaboration between care providers or lack of social support. Future research should therefore focus more on examining which information (i.e. on communication, follow-up care or information needs) from which care providers is valuable to optimize the preventability assessment [22]. The studies that did obtain additional information from the patient- and primary care provider perspective often did not describe the added value of this information. This is a missed opportunity because collecting this information is often complex and time consuming. The use of readmission rates to benchmark hospital

performance is controversial [11]. Readmissions often seem to be caused by a multitude of causes, some of which are not modifiable by the hospital (i.e. home environment or social support), meaning hospitals are penalized for causes that are beyond their control. In addition, the use of readmission as a quality indicator may provide a wrong incentive, for example by lengthen- ing hospital stays to decrease the chance of readmissions or hesitation to readmit a patient who might benefit from it. This is contradictory to what the indicator was designed for, namely to provide the incentive to provide higher quality care. Hence, readmissions do not seem to be a useful indicator of quality of care [3]. This was the first review which compared the different

methods used to assess preventability of unplanned hospital readmissions via medical record review, however, some limitations need to be discussed. Unfortunately, the heterogeneity of the studies was large, therefore, the options for a quality appraisal tool were limited and a meta-analysis was not possible. To compensate for this, we performed a (textual) narrative synthesis based on the

Kneepkens et al. BMC Medical Research Methodology (2019) 19:128 Page 8 of 12

T a b le

3 A d va n ta g es ,l im

it at io n s an d co n si d er at io n s o f se ve ra l st u d y d es ig n o p ti o n s

A d va n ta g e

Li m it at io n

Re co m m en

d at io n s

Si n g le ce n te r ve rs u s m u lt ic en

te r

Si n g le ce n te r st u d ie s p ro vi d e in fo rm

at io n

o n o n e’ s o w n p er fo rm

an ce

w h ic h is

n ee d ed

to in d u ce

a q u al it y im

p ro ve m en

t cy cl e

Fo r sc ie n ti fic

p u rp o se s it is ea si er

to id en

ti fy

w h ic h re su lt s ca n b e ex tr ap o la te d to

o th er

in st it u te s w h en

th e re su lt s ar e o b ta in ed

vi a a

m u lt ic en

te r st u d y. Fu rt h er m o re ,i n a

m u lt ic en

te r st u d y b en

ch m ar ki n g b et w ee n

th e ce n te rs is p o ss ib le .

C o m p ar e th e re su lt s w it h th e cu rr en

t lit er at u re

o n th e p re ve n ta b ili ty

o f

re ad m is si o n s, an d b e aw

ar e o f

(in te r) n at io n al an d re g io n al d iff er en

ce s

in o rg an iz at io n o f ca re .

Po p u la ti o n

(F o cu s o n a sp ec ifi c p o p u la ti o n ve rs u s a

b ro ad

p o p u la ti o n )

M an u al re vi ew

is ea si er

to p er fo rm

o n a

sp ec ifi c g ro u p (e .g .d

ia g n o si s h ea rt fa ilu re

o r d ep

ar tm

en t) .

Fo cu s o n si n g le g ro u p ca n ca u se

u n d er es ti m at io n o f th e p re ve n ta b ili ty

re ad m is si o n ra te

an d /o r u n d er re p o rt in g o f

ce rt ai n ca u se s.

C o n si d er

a m u lt id is ci p lin ar y p an el o r

te am

to re vi ew

th e re ad m is si o n s to

re d u ce

b lin d sp o ts .

Re la te d n es s (f o cu s o n re ad m is si o n s th at

ar e re la te d to

th e in d ex

re ad m is si o n ve rs u s

al l-c au se

re ad m is si o n s)

Re ad m is si o n s re la te d to

th e in d ex

h o sp it al iz at io n w ill g en

er al ly id en

ti fy

ca u se s th at

ar e re la te d to

h o sp it al ca re .

A ll- ca u se

re ad m is si o n s ar e ea si er

to id en

ti fy

b as ed

o n ad m in is tr at iv e d at a, p ro vi d e a b ro ad

sc o p e an d w ill id en

ti fy o th er

ca u se s; fo r

ex am

p le ca u se s re la te d to

ca re

in th e p rim

ar y

ca re

se tt in g .

D et er m in e th e sc o p e o f th e q u al it y

im p ro ve m en

t cy cl e; to

id en

ti fy ca u se s

re la te d to

h o sp it al ca re

o r to

ca re

o f

a re g io n

Ty p e o f re ad m is si o n s

(u n p la n n ed

ve rs u s p la n n ed

re ad m is si o n s)

Se le ct in g o n ly u n p la n n ed

re ad m is si o n s

re se m b le s th e re ad m is si o n s th at

ar e u se d

to ca lc u la te

th e re ad m is si o n q u al it y in d ic at o r

Pl an n ed

re ad m is si o n m ig h t al so

h av e

p re ve n ta b le ca u se s w h ic h w ill b e m is se d if

p la n n ed

re ad m is si o n s ar e ex cl u d ed

D et er m in e w h et h er

yo u co n si d er

u n p la n n ed

re ad m is si o n s p re ve n ta b le

p rio

r to

st ar ti n g a re ad m is si o n st u d y

Se tt in g an d so u rc es

(f o cu s o n h o sp it al ve rs u s an

in te g ra te d

ca re

n et w o rk )

A ss es sm

en t b as ed

o n a h o sp it al ’s p er sp ec ti ve

o n ly re q u ire s th e m ed

ic al re co rd

as si n g le

so u rc e.

Fr ag m en

te d an d in co m p le te

d es cr ip ti o n o f

th e p at ie n t’s

jo u rn ey

ca n re su lt in

u n d er re p o rt in g ca u se s re la te d to

in te g ra te d

ca re ,p

at ie n t an d so ci al fa ct o rs .

In te rv ie w ,q

u es tio

n n ai re

o r su rv ey

a (s u b se t) o f p at ie n ts an d o r p rim

ar y

ca re

p ro vi d er s.

In fo rm

at io n an d so u rc es

(w h ic h so u rc es

an d in fo rm

at io n to

in cl u d e; an d in

w h ic h o rd er )

In cl u d in g th e fu ll m ed

ic al re co rd ,o u tp at ie n t

d at a an d ev en

ad d it io n al so u rc es

(e .g .

in te rv ie w s) ca n ch an g e th e p er sp ec ti ve

o n

p re ve n ta b ili ty

an d it s ca u se s.

Re vi ew

er s m ig h t u se

a d iff er en

t ap p ro ac h o f

o b ta in in g /u si n g th e (a d d it io n al ) in fo rm

at io n

w h ic h ca n cr ea te

u n w an te d d iff er en

ce s in

th e

p er sp ec ti ve

o n p re ve n ta b ili ty .

N o te

th at

fo r an

in te rv ie w

o f st ak eh

o ld er s a

cr o ss -s ec ti o n al o r p ro sp ec ti ve

st u d y d es ig n is

n ee d ed

to re d u ce

re ca ll b ia s.

A st ric t p ro to co l an d lo g b o o k as

w el l

as tr ai n in g p rio

r to

st ar t o f th e st u d y.

C o n si d er

to p ro vi d e ad d it io n al

in fo rm

at io n st ep

w is e to

as se ss

it s

ad d ed

va lu e o n th e p re ve n ta b ili ty

as se ss m en

t.

A p rio

ri (p re ve n ta b ili ty ) ca u se

cl as si fic at io n

Ea si er

to p er fo rm

an d p ro b ab ly b et te r

ag re em

en t b et w ee n re vi ew

er s.

D o es

n o t in vi te

re vi ew

er to

lo o k b ey o n d th is

lis t o f p re d ef in ed

(p o te n ti al ly p re ve n ta b le )

ca u se s an d ca n th er ef o re

n ar ro w

th e

re vi ew

er ’s vi ew

.

U sa

a m u lt id is ci p lin ar y ap p ro ac h w it h

m o re

th an

o n e re vi ew

er .T h e u se

o f a

st ric t p ro to co l an d lo g b o o k as

w el l as

tr ai n in g p rio

r to

st ar t o f th e st u d y, an d

ca se

d is cu ss io n d u rin

g th e st u d y, ca n

in cr ea se

u n ifo rm

it y

Re vi ew

er s

(s in g le re vi ew

er ve rs u s d u o /t ea m )

U si n g a si n g le re vi ew

er to

p er fo rm

th e

p re ve n ta b ili ty

as se ss m en

t is le ss

ti m e- co n su m in g .

D u e to

th e p o o r re p ro d u ci b ili ty

so m e ki n d o f

d o u b le ch ec k is n ee d ed

. D o u b le (p ar ti al ) re vi ew

ca n in cr ea se

u n ifo rm

it y. If a d o u b le ch ec k is n o t

p o ss ib le ,c o n si d er

a te am

o r p an el

d is cu ss io n (o f a su b se t) o f ca se s.

M o re o ve r, ca se

d is cu ss io n ad d s to

th e

le ar n in g an d aw

ar en

es s co m p o n en

t o f

th e m ed

ic al re co rd

re vi ew

p ro ce ss .

Ex p er ie n ce

Re si d en

ts as

re vi ew

er ca n co n tr ib u te

to th e le ar n in g en

vi ro n m en

t. So m e st u d ie s su g g es t th at

ye ar s o f ex p er ie n ce

ca n in flu en

ce th e p re ve n ta b ili ty

as se ss m en

t. A p p ro ac h se n io rs to

b e av ai la b le fo r

su p er vi si o n ,d

o u b le ch ec k b y a se n io r

an d /o r tr ai n in g ,s tr ic t p ro to co l o r

d is cu ss io n m ee ti n g s.

Kneepkens et al. BMC Medical Research Methodology (2019) 19:128 Page 9 of 12

T a b le

3 A d va n ta g es ,l im

it at io n s an d co n si d er at io n s o f se ve ra l st u d y d es ig n o p ti o n s (C o n tin u ed )

A d va n ta g e

Li m it at io n

Re co m m en

d at io n s

C o m p le te

o r p ar ti al d o u b le ch ec k

A p ar ti al d o u b le ch ec k is le ss

ti m e

co n su m in g .

Th is ca n in flu en

ce th e ag re em

en t ca lc u la ti o n .

In ca se

o f p ar ti al d o u b le ch ec k u se

th e

ap p ro p ria te

an al ys is .

Fi n al p re ve n ta b ili ty

ju d g m en

t (b in ar y sc o re

ve rs u s sc al e o r ca te g o ry )

U si n g a b in ar y sc o re

fo r p re ve n ta b ili ty

is st ra ig h tf o rw

ar d an d ea sy

to in te rp re t

Si n ce

th e m aj o rit y o f re ad m is si o n s h av e

m u lt ifa ct o ria l ca u se s a b in ar y p re ve n ta b ili ty

sc o re

d o es

n o t re se m b le re al it y; a sc al e o f

ca te g o ry

o ff er s th e o p ti o n o f m ak in g a

th o u g h tf u l d ec is io n

U se

a sc al e o r ca te g o ry

w h ic h in cl u d es

in te rm

ed ia te

sc o re s o n p re ve n ta b ili ty .

Be cl ea r o n w h ic h ca te g o rie s ar e

u se d /c o m b in ed

to ca lc u la te

th e

p re ve n ta b ili ty

p er ce n ta g e.

Kneepkens et al. BMC Medical Research Methodology (2019) 19:128 Page 10 of 12

Cochrane recommendations [12]. In addition, since there was no uniformity amongst studies on the use of (key) words in their title and abstract, it could be that some studies on readmissions were missed during our search because these terms were not included in our search stra- tegy. All phases were either consensus based –driven and/ or performed by at least two independent data extractors. However, this procedure could not prevent that some amount of interpretation bias was present during data collection, synthesis and the interpretation. In conclusion, many articles on preventability of read-

missions are currently available, however, a meaningful comparison is limited due to the large study hetero- geneity (i.e. the included population, definition inconsis- tencies and variation in methods to assess preventability) . Moreover, the majority of assessments was based on a hospital and physician perspective only, resulting in a potentially underestimation of factors related to coordin- ation of care (e.g. integrated care), patient or social sup- port system. Readmissions are most likely multifactorial and readmission rate reduction is a shared responsibility within the network of care providers and the patient or carer himself. Therefore, the scope should switch from the hospital to the organization of care within the region and patient participation. Overall, we recommend that researchers carefully consider the different methodo- logical options (i.e. study population, setting and its modifiable factors, and type of resources) prior to initiat- ing a study to assess the preventability of readmissions. In Table 3 we outlined a few important methodological aspects of readmission studies and provided the ad- vantages, disadvantages and recommendations for each of these aspects. Furthermore, we recommend for future research that the methodological considerations of each readmission study are explicitly reported to increase reproducibility and comparability (e.g. the number of reviewers, review process).

Additional files

Additional file 1: Search strategy. (DOCX 14 kb)

Additional file 2: Inclusion criteria. (DOCX 85 kb)

Additional file 3: Definition of variables. (DOCX 18 kb)

Additional file 4: Characteristics of studies which were excluded based on the inclusion criteria of the flow chart (N=29). (DOCX 24 kb)

Additional file 5: Detailed descriptives of included studies (N=48). (DOCX 33 kb)

Additional file 6: Details regarding patient (and/or caregiver) interview and care provider interview. (DOCX 40 kb)

Additional file 7: Definition of preventability. (DOCX 24 kb)

Abbreviations GP: General practitioner; NR: Not reported; STAAR: The STate Action on Avoidable Rehospitalizations

Acknowledgements The authors are very grateful for assistance by the medical information specialist that assisted with the search.

Authors’ contributions Study conception and design was performed by: FKC, MdB, EK and CB. Two researchers (CB, EK) independently screened all abstracts. Detailed inclusion and exclusion criteria were applied blindly to all eligible articles by CB, EK and RS. Data of the included citations was collected by CB, EK, RS, disagreement was resolved by two independent senior researcher FKC and MdB. Analysis and interpretation of data was performed by all authors. CB, EK en RS drafted the manuscript. FKC and MdB critically revised the manuscript. All authors read and approved the final manuscript.

Funding Not applicable.

Availability of data and materials The datasets supporting the conclusions of this article are included within the article.

Ethics approval and consent to participate Ethics approval is not applicable.

Consent for publication Not applicable.

Competing interests The authors declare that none of them have received honoraria, reimbursement or fees from any pharmaceutical companies, related to this study.

Author details 1Department of Clinical Pharmacy, OLVG Hospital, Jan Tooropstraat 164, 1061 AE Amsterdam, The Netherlands. 2Department of Public and Occupational Health, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam Public Health Research Institute, Van der Boechorststraat 7, NL-1081, BT, Amsterdam, The Netherlands.

Received: 10 August 2018 Accepted: 4 June 2019

References 1. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in

the Medicare fee-for-service program. N Engl J Med. 2009;360(14):1418–28. 2. Felix HC, Seaberg B, Bursac Z, et al. Why do patients keep coming back?

Results of a readmitted patient survey. Soc Work Health Care. 2015;54(1):1–15. 3. Fischer C, Lingsma HF, Marang-van de Mheen PJ, et al. Is the readmission

rate a valid quality indicator? A review of the evidence. PLoS One. 2014; 9(11):e112282.

4. Rau J. Medicare to penalize 2,217 hospitals for excess readmissions 2012. Available from: http://www.kaiserhealthnews.org/stories/2012/august/13/ medicare-hospitals-readmissions-penalties.aspx.

5. Yam CH, Wong EL, Chan FW, et al. Avoidable readmission in Hong Kong-- system, clinician, patient or social factor? BMC Health Serv Res. 2010;10:311.

6. Kansagara D, Englander H, Salanitro A, et al. Risk prediction models for hospital readmission: a systematic review. JAMA. 2011;306(15):1688–98.

7. van Walraven C, Bennett C, Jennings A, et al. Proportion of hospital readmissions deemed avoidable: a systematic review. CMAJ. 2011; 183(7):E391–402.

8. van Walraven C. The utility of unplanned early hospital readmissions as a health care quality Indicator. JAMA Intern Med. 2015;175(11):1812–4.

9. van Walraven C, Jennings A, Forster AJ. A meta-analysis of hospital 30-day avoidable readmission rates. J Eval Clin Pract. 2012;18(6):1211–8.

10. Yam CH, Wong EL, Chan FW, et al. Measuring and preventing potentially avoidable hospital readmissions: a review of the literature. Hong Kong Med J. 2010;16(5):383–9.

11. Benbassat J, Taragin M. Hospital readmissions as a measure of quality of health care: advantages and limitations. Arch Intern Med. 2000;160(8):1074–81.

12. Ryan R. Cochrane consumers and communication review group: data synthesis and analysis; 2013.

Kneepkens et al. BMC Medical Research Methodology (2019) 19:128 Page 11 of 12

13. Cakir B, Gammon G. Evaluating readmission rates: how can we improve? South Med J. 2010;103(11):1079–83.

14. Feigenbaum P, Neuwirth E, Trowbridge L, et al. Factors contributing to all- cause 30-day readmissions: a structured case series across 18 hospitals. Med Care. 2012;50(7):599–605.

15. Halfon P, Eggli Y, van Melle G, et al. Measuring potentially avoidable hospital readmissions. J Clin Epidemiol. 2002;55(6):573–87.

16. Jimenez-Puente A, Garcia-Alegria J, Gomez-Aracena J, et al. Readmission rate as an indicator of hospital performance: the case of Spain. Int J Technol Assess. 2004;20(3):385–91.

17. Koekkoek D, Bayley KB, Brown A, et al. Hospitalists assess the causes of early hospital readmissions. J Hosp Med. 2011;6(7):383–8.

18. Miles TA, Lowe J. Are unplanned readmissions to hospital really preventable? J Qual Clin Pract. 1999;19(4):211–4.

19. Pace R, Spevack R, Menendez C, et al. Ability of nurse clinicians to predict unplanned returns to hospital within thirty days of discharge. Hosp Pract (1995). 2014;42(5):62–8.

20. Shalchi Z, Saso S, Li HK, et al. Factors influencing hospital readmission rates after acute medical treatment. Clin Med (Lond). 2009;9(5):426–30.

21. Stein J, Ossman P, Viera A, et al. Was this readmission preventable? Qualitative study of patient and provider perceptions of readmissions. South Med J. 2016;109(6):383–9.

22. Auerbach AD, Kripalani S, Vasilevskis EE, et al. Preventability and causes of readmissions in a National Cohort of general medicine patients. JAMA Intern Med. 2016;176(4):484–93.

23. Burke D, Link N, Bails D, et al. A taxonomy of seven-day readmissions to an urban teaching hospital. J Hosp Med. 2016;11(1):33–8.

24. Gautam P, Macduff C, Brown I, et al. Unplanned readmissions of elderly patients. Health Bull (Edinb). 1996;54(6):449–57.

25. Shimizu E, Glaspy K, Witt MD, et al. Readmissions at a public safety net hospital. PLoS One. 2014;9:3.

26. Sutherland T, David-Kasdan JA, Beloff J, et al. Patient and provider-identified factors contributing to surgical readmission after colorectal surgery. J Investig Surg. 2016;29(4):195–201.

27. Toomey SL, Peltz A, Loren S, et al. Potentially preventable 30-day hospital readmissions at a Children's hospital. Pediatrics. 2016;138:2.

28. Williams EI, Fitton F. Factors affecting early unplanned readmission of elderly patients to hospital. BMJ. 1988;297(6651):784–7.

29. Vinson JM, Rich MW, Sperry JC, et al. Early readmission of elderly patients with congestive-heart-failure. J Am Geriatr Soc. 1990;38(12):1290–5.

30. Balla U, Malnick S, Schattner A. Early readmissions to the department of medicine as a screening tool for monitoring quality of care problems. Medicine (Baltimore). 2008;87(5):294–300.

31. Bianco A, Mole A, Nobile CG, et al. Hospital readmission prevalence and analysis of those potentially avoidable in southern Italy. PLoS One. 2012;7(11):e48263.

32. van Walraven C, Jennings A, Taljaard M, et al. Incidence of potentially avoidable urgent readmissions and their relation to all-cause urgent readmissions. Can Med Assoc J. 2011;183(14):E1067–E72.

33. Ryan J, Andrews R, Barry MB, et al. Preventability of 30-day readmissions for heart failure patients before and after a quality improvement initiative. Am J Med Qual. 2014;29(3):220–6.

34. Institute for Healtcare Improvement. Available from: http://www.ihi.org/ Engage/Initiatives/Completed/STAAR/Pages/default.aspx.

35. Fluitman KS, van Galen LS, Merten H, et al. Exploring the preventable causes of unplanned readmissions using root cause analysis: coordination of care is the weakest link. Eur J Intern Med. 2016;30:18–24.

36. Oddone EZ, Weinberger M, Horner M, et al. Classifying general medicine readmissions. Are they preventable? Veterans affairs cooperative studies in health services group on primary care and hospital readmissions. J Gen Intern Med. 1996;11(10):597–607.

37. Tejedor-Sojo J, Singleton LM, McCormick K, et al. Preventability of pediatric 30-day readmissions following ventricular shunt surgery. J Pediatr-Us. 2015;167(6):1327.

38. Agrawal K, Kumar P, Markert R, et al. Risk factors for 30-day readmissions of individuals with decompensated cirrhosis. South Med J. 2015;108(11):682–7.

39. Clarke A. Are readmissions avoidable? BMJ. 1990;301(6761):1136–8. 40. Frankl SE, Breeling JL, Goldman L. Preventability of emergent hospital

readmission. Am J Med. 1991;90(6):667–74. 41. Hain PD, Gay JC, Berutti TW, et al. Preventability of early readmissions at a

children's hospital. Pediatrics. 2013;131(1):e171–81.

42. Harhay M, Lin E, Pai A, et al. Early rehospitalization after kidney transplantation: assessing preventability and prognosis. Am J Transplant. 2013;13(12):3164–72.

43. Jonas JA, Devon EP, Ronan JC, et al. Determining preventability of pediatric readmissions using fault tree analysis. J Hosp Med. 2016;11(5):329–35.

44. Kelly CL, Thomson K, Wagner AP, et al. Investigating the widely held belief that men and women with learning disabilities receive poor quality healthcare when admitted to hospital: a single-site study of 30-day readmission rates. J Intellect Disabil Res. 2015;59(9):835–44.

45. Maurer PP, Ballmer PE. Hospital readmissions--are they predictable and avoidable? Swiss Med Wkly. 2004;134(41–42):606–11.

46. Meisenberg BR, Hahn E, Binner M, et al. Insights into the potential preventability of oncology readmissions. J Oncol Pract. 2016;12(2):153 -+.

47. Nahab F, Takesaka J, Mailyan E, et al. Avoidable 30-day readmissions among patients with stroke and other cerebrovascular disease. Neurohospitalist. 2012;2(1):7–11.

48. Nijhawan AE, Kitchell E, Etherton SS, et al. Half of 30-day hospital readmissions among HIV-infected patients are potentially preventable. AIDS Patient Care STDs. 2015;29(9):465–73.

49. Saunders ND, Nichols SD, Antiporda MA, et al. Examination of unplanned 30-day readmissions to a comprehensive cancer hospital. J Oncol Pract. 2015;11(2):e177–81.

50. Shah MN, Stoev IT, Sanford DE, et al. Are readmission rates on a neurosurgical service indicators of quality of care? J Neurosurg. 2013; 119(4):1043–9.

51. Vachon CM, Aaland M, Zhu TH. Readmission of trauma patients in a nonacademic level II trauma center. J Trauma Acute Care. 2012;72(2):531–6.

52. Wallace SS, Keller SL, Falco CN, et al. An examination of physician-, caregiver-, and disease-related factors associated with readmission from a pediatric hospital medicine service. Hosp Pediatr. 2015;5(11):566–73.

53. Wasfy JH, Strom JB, Waldo SW, et al. Clinical preventability of 30-day readmission after percutaneous coronary intervention. J Am Heart Assoc. 2014;3(5):e001290.

54. Weinberg DS, Kraay MJ, Fitzgerald SJ, et al. Are readmissions after THA preventable? Clin Orthop Relat Res. 2017;475(5):1414–23.

55. Dawes AJ, Sacks GD, Russell MM, et al. Preventable readmissions to surgical services: lessons learned and targets for improvement. J Am Coll Surgeons. 2014;219(3):382–9.

56. Glass CC, Gondek SP, Vollmer CM Jr, et al. Readmission following pancreatectomy: what can be improved? HPB (Oxford). 2013;15(9):703–8.

57. Greenberg JK, Washington CW, Guniganti R, et al. Causes of 30-day readmission after aneurysmal subarachnoid hemorrhage. J Neurosurg. 2016;124(3):743–9.

58. Mittal MK, Rabinstein AA, Mandrekar J, et al. A population-based study for 30-d hospital readmissions after acute ischemic stroke. Int J Neurosci. 2017; 127(4):305–13.

59. Njeim M, Chiha M, Whitehouse S, et al. System-based approach to educating internal medicine residents on preventable hospital readmissions. J Grad Med Educ. 2012;4(4):505–9.

60. Epstein AS, Crosbie C, Martin SC, et al. 30-day-or-sooner readmissions of gastrointestinal medical oncology patients following cancer center inpatient service discharge: characteristics and preventability. Hosp Pract (1995). 2014;42(5):34–44.

61. Zhou H, Della PR, Roberts P, Goh L, et al. Utility of models to predict 28-day or 30-day unplanned hospital readmissions: an updated systematic review. BMJ Open. 2016;6(6):e011060.

62. Donze J, Aujesky D, Williams D, et al. Potentially avoidable 30-day hospital readmissions in medical patients: derivation and validation of a prediction model. JAMA Intern Med. 2013;173(8):632–8.

Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Kneepkens et al. BMC Medical Research Methodology (2019) 19:128 Page 12 of 12

BioMed Central publishes under the Creative Commons Attribution License (CCAL). Under the CCAL, authors retain copyright to the article but users are allowed to download, reprint, distribute and /or copy articles in BioMed Central journals, as long as the original work is properly cited.