Assignment: Evidence-Based Project, Part 4: Critical Appraisal of Research

Vaccine 35 (2017) 513–520

Contents lists available at ScienceDirect

Vaccine

journal homepage: www.elsevier .com/locate /vacc ine

Review

Effectiveness of MF59-adjuvanted seasonal influenza vaccine in the elderly: A systematic review and meta-analysis

http://dx.doi.org/10.1016/j.vaccine.2016.12.011 0264-410X/� 2016 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).

⇑ Corresponding author. E-mail address: panatto@unige.it (D. Panatto).

Alexander Domnich a, Lucia Arata a, Daniela Amicizia a, Joan Puig-Barberà b, Roberto Gasparini a, Donatella Panatto a,⇑ aDepartment of Health Sciences, University of Genoa, Genoa, Italy bVaccines Research Unit, FISABIO-Public Health, Valencia, Spain

a r t i c l e i n f o a b s t r a c t

Article history: Received 4 August 2016 Received in revised form 5 December 2016 Accepted 8 December 2016 Available online 23 December 2016

Keywords: Influenza vaccine MF59-adjuvanted vaccine Elderly Influenza vaccine effectiveness Meta-analysis

Background: In the elderly, traditional influenza inactivated vaccines are often only modestly immuno- genic, owing to immunosenescence. Given that adjuvantation is a means of enhancing the immune response, the trivalent inactivated vaccine adjuvanted with MF59 (MF59-TIV) was specifically designed to overcome this problem. Considering that, for ethical reasons, the absolute effectiveness of an influenza vaccine in the elderly cannot be demonstrated in placebo-controlled studies, the present study aimed to assess the effectiveness of MF59-TIV in preventing influenza-related outcomes in the elderly. Methods: We conducted a systematic review of observational studies aimed at evaluating the effective- ness of MF59-TIV against influenza-related outcomes. Results of single studies were pooled whenever possible. Results: Of the 1993 papers screened, 11 (6 case-control, 3 cohort and 2 prospective case-control) studies were identified. Hospitalization due to pneumonia/influenza and laboratory-confirmed influenza were reported in more than one study, while other outcomes (influenza-like illness, cardio- and cerebrovascu- lar accidents) were investigated only by one study each. Pooled analysis of four case-control studies showed an adjusted MF59-TIV effectiveness of 51% (95% CI: 39–61%) against hospitalizations for pneu- monia/influenza among community-dwelling seniors. Pooled results of the adjusted vaccine effectiveness against laboratory-confirmed influenza were also high (60.1%), although the 95% CI passed through zero (�1.3 to 84.3%). Other single community-based studies showed very high effectiveness of MF59-TIV in preventing hospitalizations for acute coronary [87% (95% CI: 35–97%)] and cerebrovascular [93% (95% CI: 52–99%)] events. MF59-TIV proved highly effective [94% (95% CI: 47–100%] in reducing influenza- like illness among institutionalized elderly. Furthermore, MF59-TIV displayed greater efficacy than non-adjuvanted vaccines in preventing hospitalizations due to pneumonia/influenza [adjusted risk ratio 0.75 (95% CI: 0.57–0.98)] and laboratory-confirmed influenza [adjusted odds ratio 0.37 (0.14–0.96)]. Conclusions: Our results suggest that MF59-TIV is effective in reducing several influenza-related out- comes among the elderly, especially hospitalizations due to influenza-related complications. � 2016 The Authors. Published by Elsevier Ltd. This is an openaccess article under the CCBY license (http://

creativecommons.org/licenses/by/4.0/).

Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514 2. Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514

2.1. Eligibility criteria. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514 2.2. Search strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514 2.3. Study selection and data extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514 2.4. Quality assessment of included studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515 2.5. Data analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515

3. Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516

514 A. Domnich et al. / Vaccine 35 (2017) 513–520

3.1. Characteristics of studies included. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516 3.2. MF59-TIV effectiveness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517 3.3. Relative effectiveness of MF59-TIV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 518

4. Discussion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 518 Conflict of interest . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 519 Appendix A. Supplementary material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520

1. Introduction

The elderly are particularly vulnerable to seasonal influenza; indeed, approximately 90% of all influenza-related deaths occur among senior citizens [1]. Vaccination remains the most effective public health measure to reduce the impact of seasonal influenza [2,3]. In this regard, the European Union Council encourages indi- vidual countries to reach a vaccination coverage rate of at least 75% in the elderly [4]. However, traditional trivalent inactivated vaccines (TIVs) often induce only a modest immune response in the elderly, owing to immunosenescence [5].

Adjuvantation is a well-known means of increasing vaccine potency. Moreover, adjuvanted vaccines may induce a more rapid and broader (i.e. towards heterologous serotypes) immune response [6]. The first adjuvant used for seasonal influenza vacci- nes was the squalene-based adjuvant microfluidized emulsion 59 (MF59), an oil-in-water emulsion. Its immuno-stimulatory proper- ties have been extensively demonstrated; it activates local immune cells at the injection site, and promotes transition from monocytes to dendritic cells and recruitment of antigen-presenting cells and antigen uptake [7]. Italy was the first country in which a seasonal TIV vaccine adjuvanted with MF59 (MF59-TIV) was licensed (in 1997) [8].

The immunogenicity and safety of MF59-TIV have been stud- ied in several pooled analyses [9–11]. The early paper by Podda [9] reported that, compared with non-adjuvanted TIVs, MF59- TIV induced significantly higher levels of immunogenicity in terms of geometrical mean ratios, and seroconversion and sero- protection rates for both primary and subsequent immunizations. In that study, MF59-TIV was judged to be well tolerated (most adverse events being mild), although several local and systemic reactions were more common in recipients of the adjuvanted vac- cine [9]. The latter finding was later confirmed by an integrated analysis of safety outcomes [10], in which subjects vaccinated with MF59-TIV showed a higher risk than those vaccinated with non-adjuvanted TIV [weighted risk ratio 1.34 (95% CI: 1.28– 1.40)] in terms of solicited local and systemic reactions within 3 days of immunization. A more recent systematic review by Camilloni et al. [11] suggested that the immunogenicity of MF59-TIV was high in the elderly, and in most instances satisfied the Committee for Medicinal Products for Human Use (CHMP) criteria.

Although randomized controlled trials (RCTs) in which the pri- mary endpoints are immunogenicity and/or efficacy are advanta- geous in terms of rigorous control for biases and prospective monitoring of disease attack rates, observational studies are able to estimate in-field vaccine effectiveness (VE), thereby providing additional useful insights into the ‘‘real world” impact of a vaccine [12]. Furthermore, the absolute VE of a seasonal influenza vaccine in the elderly cannot be demonstrated in placebo-controlled stud- ies for ethical reasons, since immunization of this age-class is highly recommended [8]. The present systematic review aimed to analyze the VE of seasonal MF59-TIV in preventing several influenza-related outcomes in the elderly.

2. Methods

2.1. Eligibility criteria

As a guideline for reporting, the MOOSE (meta-analysis of observational studies in epidemiology) checklist [13] was consulted.

Observational studies investigating the effectiveness of MF59- TIV in elderly populations were potentially eligible. Both cut-offs (60 and 65 years) for defining the elderly were acceptable [14]. Study populations were classified, in accordance with the study settings, as community-dwelling subjects, those institutionalized in long-term care facilities, and mixed.

The primary endpoint was the absolute VE, which was defined as the proportional reduction in the risk of influenza-related out- comes among vaccinees (in comparison with non-vaccinated sub- jects), as recorded in observational studies [12,15]. Influenza- related outcomes included: laboratory-confirmed influenza [through real-time polymerase chain reaction (RT-PCR) or culture], influenza-like illness (ILI), outpatient visits for influenza and pneu- monia, hospitalizations for all respiratory pathologies, for influenza and pneumonia, for cardio- and cerebrovascular accidents, and mortality following hospitalization for influenza and pneumonia [15–17]. Our secondary endpoint was the relative VE, which was assessed by comparing influenza-related outcomes in subjects vac- cinated with MF59-TIV with those of recipients of other influenza vaccines [18].

Owing to the fact that MF59-TIV was first authorized for human use in Italy in 1997 [8] and the first clinical trials were conducted in the early 1990s [19], our search was limited to the period start- ing from 01/01/1990. The last automatic search was performed on 26/04/2016. No language restrictions were applied.

2.2. Search strategy

A comprehensive search was performed in MEDLINE via OVID, Web of Science and Cochrane Library. The search strategy was adapted from [17,20], and is reported in Appendix, Table S1. Refer- ence lists of included studies were manually checked by an expert. The vaccine manufacturer was asked to suggest relevant studies.

2.3. Study selection and data extraction

Once duplicates had been removed, titles and abstracts obtained from the automatic search were independently screened by two investigators (AD and LA). Subsequently, full texts of poten- tially eligible studies were assessed by applying a set of inclusion and exclusion criteria.

The following inclusion criteria were applied: (i) use of com- mercially available formulations of MF59-TIV; (ii) study population aged P60/65 years, regardless of setting; (iii) study period deter- mined by influenza or ILI surveillance systems; (iv) case-control or cohort design; (v) studies reporting at least one of the above- mentioned influenza-related outcomes. Exclusion criteria were

A. Domnich et al. / Vaccine 35 (2017) 513–520 515

formulated as follows: (i) non-original research, case reports and similar; (ii) animal studies; (iii) cross-sectional design; (iv) studies of any design aimed to evaluate immunogenicity, vaccine efficacy, tolerability and safety of MF59-TIV; (v) number of subjects vacci- nated with MF59-TIV < 50; (vi) influenza diagnosed by means of serology; (vii) studies on subjects with particular chronic condi- tions (e.g. immunosuppression); (viii) absence of separate data for subjects of different ages (e.g. elderly and non-elderly adults) or vaccinated with different vaccines. However, if the majority (P90%) of study participants belonged to a group of interest, the study was included.

Data from selected studies were extracted by two reviewers (AD and LA), each working independently, and were inserted into an ad hoc database. The following information was gathered: first author and year of publication, study location, influenza season, study aim, study design, sample size, setting, age of vaccinees, comparator vaccines (if any), influenza-related outcome/s used, unadjusted and adjusted estimates of VE with 95% confidence intervals (CIs), and potential confounders. If a study reported VE estimates in different periods of the influenza season, only the esti- mate calculated close to the peak was recorded.

During the process of study selection and data extraction, any disagreement between the two investigators was settled by con- sensus and/or consultation of the third investigator (RG).

Records identified through automatic search

(N=1,992)

Titles and abstracts screened (N=1,210)

Full texts examined (N=48)

Papers included in qualitative synthesis (N=9)

Studies included in meta-analysis (N=6)

Fig. 1. Flowchart of the stu

2.4. Quality assessment of included studies

The Newcastle-Ottawa scale (NOS) [21] was used to assess the quality of case-control and cohort studies. Evaluation was carried out independently by two authors, and the inter-rater agreement was quantified by means of Cohen’s j. Any between-rater dis- agreement was settled by consensus. Studies were categorized as being at low (61 inadequate item on the NOS), medium (2– 3 inadequate items on the NOS), high (>3 inadequate items on the NOS) or very high (no description of methods) risk of bias [17].

2.5. Data analysis

VE was expressed as [1-odds ratio (OR)]�100% and [1-risk ratio (RR)]�100% for case-control and cohort studies, respectively. VE was deemed statistically significant when its 95% CI did not pass through zero. The relative VE was expressed as an appropriate measure of effect size (OR and RR for case-control and cohort stud- ies, respectively).

The meta-analysis of ORs (vaccinated with MF59-TIV vs non- vaccinated) was carried out in order to obtain pooled estimates of VE. To this end, a random-effects model using the DerSimonian-Laird weighting method was first implemented. If

Duplicates removed (N=782)

Records excluded (N=1,162)

Records excluded, as per exclusion criterion 1 (N=8), 4

(N=30), 5(N=1), 7(N=1)

Papers identified through manual search (N=1)

dy selection process.

516 A. Domnich et al. / Vaccine 35 (2017) 513–520

the observed heterogeneity was low, i.e. with an I2 < 40%, a fixed- effects model using the inverse variance method for weighting was re-applied. Unadjusted and adjusted effect sizes (ESs) were pooled separately. Meta-analysis was not performed when I2 was over 85% [22]. A ‘‘leave-one-out” sensitivity analysis was carried out in order to ascertain that the pooled estimates were not driven by single studies. We planned a priori to conduct a subgroup anal- ysis and/or meta-regression, in order to highlight those study char- acteristics that were significantly associated with heterogeneity among studies. Assessment of publication bias through Doi plot and LFK index was also planned, but could not be carried out. All analyses were performed in MetaXL 5.0 [23].

3. Results

3.1. Characteristics of studies included

Of the 1993 records screened, 9 papers (8 [24–26,28–32] and 1 [27] retrieved through automatic and manual searches, respec- tively), corresponding to 11 studies [24–32] (one paper [26] reported results of three studies), met the inclusion criteria

Table 1 Characteristics of studies included.

Study design Primary study aim Country (influenza season)

Ag sub yea

Case-control To estimate effectiveness of MF59-TIV in preventing hospital admissions for pneumonia

Spain (2002/03)

P6

Prospective To compare adjuvanted and non-adjuvanted vaccines in preventing ILI

Italy (1998/99)

23

Case-control To estimate the effectiveness of MF59-TIV in preventing hospitalizations for acute coronary syndrome

Spain (2004/05)

P6

Case-control To estimate the effectiveness of MF59-TIV in preventing hospitalizations for cerebrovascular accidents

Spain (2004/05)

P6

Case-control To estimate the effectiveness of MF59-TIV in preventing hospitalizations for pneumonia

Spain (2004/05)

P6

Prospective case- control

To estimate seasonal influenza vaccine effectiveness in preventing laboratory-confirmed influenza-related hospitalizations

Spain (2010/11)

P1

Prospective To compare adjuvanted and non-adjuvanted vaccines in preventing hospitalizations for influenza and pneumonia

Italy (2006/07, 2007/08, 2008/09)

P6

Prospective case- control

To evaluate adjuvanted and non-adjuvanted vaccines, compared with no vaccination, in preventing laboratory-confirmed influenza

Canada (2011/12)

P6

Case-control To evaluate the effectiveness of seasonal adjuvanted vaccines available in Italy in the elderly

Italy (2010/11)

P6

Retrospective To compare adjuvanted and virosomal vaccines in preventing influenza-related hospitalizations

Spain (2010/11)

P6

Case-control To evaluate and compare effectiveness of non- adjuvanted (2010/11) and adjuvanted vaccines (2011/12) in reducing hospitalizations for per influenza or pneumonia

Italy (2011/ 12)d

P6

a Subjects aged <65 years accounted for only 3.65%. b Subjects vaccinated with MF59-TIV were all aged P65 years. c Data derived from personal communication of the corresponding author. d Study also considered the 2010/11 season, when, however, a non-adjuvanted vaccin

(Fig. 1). The vaccine manufacturer agreed with the list of studies included. Taken together, the 11 studies enrolled 546,015 person-seasons. Of these, 52.3% were vaccinated with MF59-TIV. Six studies were case-control [24,26,30,32], three were cohort [25,28,31], and two were prospective community-based case- control studies [27,29]. Six studies were conducted in Spain [24,26,27,31], four in Italy [25,28,30,32], and the remaining one in Canada [29]. Most studies (N = 9) were conducted in the com- munity setting [24,26–28,30–32]; the study by Iob et al. [25] was the only one carried out in long-term care facilities. The Canadian study [29] had a mixed population (56.7% of subjects living in long- term care facilities). Across the selected studies, the most frequent influenza-related outcome was hospitalization for pneumonia and influenza [24,26,28,30–32], followed by laboratory-confirmed influenza [27,29], ILI [25], hospitalization for acute coronary syn- drome [26] and cerebrovascular accidents [26]. Five studies [25,28–31] compared the effectiveness of MF59-TIV with that of other adjuvanted and non-adjuvanted vaccines. All included stud- ies were judged (once consensus had been reached) to be at med- ium risk of bias, although inter-rater agreement was low (j = 0.2). The characteristics of the studies included are reported in Table 1.

e of jects, rs

Setting Influenza-related outcome

Sample size (person- seasons)

Ref.

5 Community Emergency hospitalization for pneumonia (ICD-9-MC codes 480–487)

815 (290 cases and 525 controls), 486 of whom vaccinated

[24]

–100a Long-term care facilities

Influenza-like illness 3173, of whom 1487 vaccinated with MF59-TIV

[25]

5 Community Hospitalizations for acute coronary syndrome (ICD-9-MC codes 410–411.89, 413)

402 (144 cases and 258 controls), 295 of whom vaccinated

[26]

5 Community Hospitalizations for cerebrovascular accidents (ICD-9-MC codes 431–436)

380 (134 cases and 246 controls), 275 of whom vaccinated

[26]

5 Community Hospitalizations for pneumonia (ICD-9-MC codes 480–487)

519 (198 cases and 321 controls), 401 of whom vaccinated

[26]

8b,c Community Laboratory-confirmed influenza (RT-PCR)

826, of whom 113c

vaccinated with MF59-TIV

[27]

5 Community Hospitalization for pneumonia and influenza (ICD-9-MC codes 480–487)

164,254, of whom 84,665 vaccinated with MF59-TIV

[28]

5 Mixed Laboratory-confirmed influenza (RT-PCR)

282 (84 cases and 198 controls), 165 of whom vaccinated with MF59-TIV

[29]

5 Community Hospitalization for pneumonia and influenza (ICD-9-MC codes 480–487)

374 (187 cases and 187 controls), 88 of whom vaccinated with MF59-TIV

[30]

5 Community Hospitalization for pneumonia and influenza (ICD-9-MC codes 487–488.89)

373,398, of whom 197,180 vaccinated with MF59-TIV

[31]

5 Community Hospitalization for pneumonia and influenza (ICD-9-MC codes 480–487)

1592 (365 cases and 1227 controls), 519 of whom vaccinated

[32]

e was used.

A. Domnich et al. / Vaccine 35 (2017) 513–520 517

3.2. MF59-TIV effectiveness

In the Canadian study [29], which involved a mixed study pop- ulation of both community and long-term care facility residents, the VE of MF59-TIV in preventing laboratory-confirmed influenza was 35% and 58% on univariable and multivariable analyses, respectively. When a subgroup analysis of community-dwelling subjects was undertaken, VE exceeded 70%: 73% and 72% in unad- justed and adjusted models, respectively [29]. By contrast, in one Spanish study [27], the VE of MF59-TIV in preventing laboratory- confirmed influenza did not reach an a < 0.05 (Table 2). Pooled analyses of these two studies revealed a significant unadjusted VE of 58.8% (95% CI: 10.7–81.0%). When adjusted estimates were pooled, VE slightly increased to 60.1%; however, although the ES

Table 2 MF59-TIV effectiveness in preventing laboratory-confirmed influenza and hospitalization

Outcome Study [Ref.]

Laboratory-confirmed influenza Puig-Barberà [27] Van Buynder [29]

Hospitalization for pneumonia/influenza Puig-Barberà [24] Puig-Barberà [26] Gasparini [30] Spadea [32]

a Adjusted for age, sex, underlying pathologies, smoking, epidemiological week and d b Adjusted for age, gender, presence of chronic conditions, health authority, week of t c Adjusted for presence of heart disease, COPD, asthma, Barthel Index score <60, smo d Adjusted for age, presence of heart disease, COPD, diabetes mellitus, Barthel Ind

vaccination status of the caregive. e Adjusted for age, presence of respiratory disease, smoking and drinking habits. f Adjusted for age, sex, civil status, educational level, citizenship, number of comorbid

A. Laboratory-co Crude odds ratios (fixed-effects model)

B. Hospitalization for Crude odds ratios (random-effects model)

OR 1

Study

Van Buynder, 2013

Overall

Q=0.84 (p=.36)

I2=0%

Puig-Barberà, 2012

OR (95% CI) % Weight

0.27 (0.08–0.86) 42.5

0.412 (0.190–0.893) 100

0.56 (0.20–1.56) 57.5

S

V

O

Q

I2

Pu

OR 10

Study

Gasparini, 2013

Spadea, 2014

Overall

Q=16.03 (p<.001)

I2=81%

Puig-Barberà, 2007

Puig-Barberà, 2004

OR (95% CI) % Weight

0.05 (0.01–0.35) 6.1

0.48 (0.37–0.64) 34.4

0.577 (0.334–0.999) 100

0.73 (0.40–1.35) 25.7

0.91 (0.68–1.23) 33.9

St

G

Pu

O

Q

I2

Sp

Pu

Fig. 2. Pooled estimates of unadjusted a

was large, the 95% CI passed through zero (�1.3 to 84.3%). No heterogeneity was found (both I2 = 0%) (Fig. 2A).

Among four case-control community-based studies [24,26,30,32], unadjusted VE in preventing hospitalization for pneumonia and influenza ranged from 9% [24] to 95.2% [30]; this range narrowed (from 48% [24] to 88% [30]) when considering the adjusted VE (Table 2). Pooling of unadjusted estimates of VE (N = 4 studies) revealed a high level of heterogeneity (I2 = 81%); the random-effects model showed a VE of 42% (95% CI: 0.01– 67%). The sensitivity analysis (Appendix, Table S2) revealed that, after excluding the study by Gasparini et al. [30], the unadjusted VE dropped to 32.4% (95% CI: �7.3 to 57.5%); in any case, heterogeneity was constantly high: at least 70%. By contrast, no heterogeneity was found (I2 = 0%) when adjusted VEs were

for pneumonia/influenza in community-dwelling elderly.

VE (95% CI), % aVE (95% CI), %

43.8 (�56.1 to 79.7) 43.2 (�111.2 to 84.7)a 73 (14–92) 72 (2–93)b

9 (�23 to 32) 48 (20–66)c 27 (�35 to 60) 69 (29–86)d 95.2 (64.6–99.4) 87.8 (�39.4 to 98.9)e 52 (36–63) 49 (30–60)f

ays from onset of symptoms to swab. esting. king status, pneumococcal vaccination, attending outpatient clinics. ex, smoking status, pneumococcal vaccination, number of home visits, influenza

ities.

nfirmed influenza Adjusted odds ratios (fixed-effects model)

pneumonia and influenza Adjusted odds ratios (fixed-effects model)

OR 210

tudy

an Buynder, 2013

verall

=0.55 (p=.46)

=0%

ig-Barberà, 2012

aOR (95% CI) % Weight

0.28 (0.07–0.98) 49.8

0.399 (0.157–1.013) 100

0.57 (0.15–2.11) 50.2

OR 10

udy

asparini, 2013

ig-Barberà, 2007

verall

=2.64 (p=.45)

=0%

adea, 2014

ig-Barberà, 2004

aOR (95% CI) % Weight

0.12 (0.01–1.39) 0.9

0.31 (0.14–0.71) 7.6

0.488 (0.390–0.610) 100

0.51 (0.40–0.70) 64.1

0.52 (0.34–0.80) 27.4

nd adjusted vaccine effectiveness.

M ix ed

C I) ,%

N of

st u di es

V E (9 5%

C I) ,%

aV E (9 5%

C I) ,%

1 35

(� 25

to 81

) 58

(5 –8

2)

0 N /A

N /A

0 N /A

N /A

0 N /A

N /A

0 N /A

N /A

518 A. Domnich et al. / Vaccine 35 (2017) 513–520

meta-analyzed (N = 4 studies), yielding a pooled estimate of 51% (95% CI: 39–61%) (Fig. 2B). The sensitivity analyses (Appendix, Table S2) showed consistently robust results, which signifies that the pooled estimate was not driven by a single study. Owing to the paucity of studies, neither publication bias or subgroup analy- ses could be performed.

Regarding other influenza-related outcomes, in the paper by Puig-Barberà et al. [26], MF59-TIV proved to be highly effective in reducing hospitalizations for acute coronary syndrome in the multivariable analysis [VE: 87% (95% CI: 35–97%)], but not in the unadjusted one [VE: 11% (95% CI: �108 to 63%)]. A similar pattern has been reported with regard to cerebrovascular accidents: VEs of 93% (95% CI: 52–99%) and 44% (95% CI: �40 to 69%) for adjusted and unadjusted estimates, respectively [26].

With regard to long-term care facilities, Iob et al. documented an unadjusted VE of MF59-TIV of 94% (95% CI: 47–100%) [25].

Table 3 summarizes available data on MF59-TIV VE.

n g- te rm

ca re

fa ci li ti es

of st u di es

V E (9 5%

C I) ,%

aV E (9 5%

N /A

N /A

94 (4 7–

10 0)

N /A

N /A

N /A

N /A

N /A

N /A

N /A

3.3. Relative effectiveness of MF59-TIV

Table 4 reports ESs of the comparative VE of MF59-TIV versus other adjuvanted and non-adjuvanted vaccines [25,28–31]. The relative effectiveness of MF59-TIV tended to be significantly higher in comparison with intramuscular non-adjuvanted vaccines, but not virosomal (no longer commercially available) and intradermal TIVs. A meta-analysis was not feasible, owing to the different study designs, settings, comparators and influenza-related outcomes.

Ta bl e 3

M F5

9- TI V

ef fe ct iv en

es s,

by ou

tc om

e, st ud

y de

si gn

an d se tt in g.

O u tc om

e St u dy

de si gn

Se tt in g

C om

m u n it y

Lo

N of

st u di es

V E (9 5%

C I) ,%

aV E (9 5%

C I) ,%

N

La bo

ra to ry -c on

fi rm

ed in fl u en

za Pr os

pe ct iv e ca se -

co n tr ol

2 59

(1 1–

81 )a

60 (�

1 to

84 )a

0

In fl u en

za -l ik e il ln es s

C oh

or t

0 N /A

N /A

1 H os

pi ta li za

ti on

fo r pn

eu m on

ia an

d in fl u en

za C as e- co

n tr ol

4 42

(0 .0 1–

67 )a

51 (3 9–

61 )a

0 H os

pi ta li za

ti on

fo r ac u te

co ro n ar y sy n dr om

e C as e- co

n tr ol

1 11

(� 10

8 to

63 )

87 (3 5–

97 )

0 H os

pi ta li za

ti on

fo r ce re br ov

as cu

la r

ac ci de

n ts

C as e- co

n tr ol

1 44

(� 40

to 69

) 93

(5 2–

99 )

0

St at is ti ca ll y si gn

ifi ca n t es ti m at es

ar e in

bo ld .

a Po

ol ed

es ti m at es .

4. Discussion

Influenza vaccination is highly recommended in the elderly because these subjects are at high risk of developing severe com- plications due to influenza. Therefore, it is important to evaluate the effectiveness of influenza vaccines in this age-class. However, RCTs in this age-class are very uncommon, as ethics committees reject experimental placebo-controlled designs in most jurisdic- tions [33]; consequently, observational studies are a fundamental way of evaluating influenza VE among seniors. Since it was licensed, more than 20 years ago, MF59-TIV has been approved for human use in about 30 countries, and in most of these it is indi- cated only for immunization of the elderly [6]. To our knowledge, the present systematic review is the first that has specifically aimed to evaluate and quantify the effectiveness of MF59-TIV in the elderly.

Our results show that MF59-TIV is effective against a number of influenza-related outcomes in the elderly living in both communi- ties and long-term care facilities. Specifically, the adjusted VE in preventing hospitalizations for pneumonia/influenza and acute coronary and cerebrovascular events exceeded 50%. Our pooled analysis revealed no heterogeneity among the studies included, suggesting that the vaccination effects were consistent in the four studies that evaluated these outcomes. Arguably, our meta- analysis included a limited number of papers; Q and I2 statistics may therefore have had a low power. However, the data support consistency of the estimates, since the 95% CIs of the meta- analyzed studies overlapped.

With regard to laboratory-confirmed influenza, adjusted esti- mates of VE were around 60% in both community and mixed pop- ulations (communities and long-term care facilities). However, in the case of community-dwelling seniors, the lower confidence of the adjusted VE was �1.3% and the 95% CIs were fairly broad, sug- gesting a low precision of estimates. This observation can probably be attributed to the small sample sizes in both of the studies included.

Table 4 Relative effectiveness of MF59-TIV (MF59-TIV versus other vaccines).

Study [Ref.] Design Setting Comparator Outcome ES (95% CI) aES (95% CI)

Iob [25] Prospective Long-term care facility

IM-TIV Influenza-like illness 0.66 (0.53–0.82)a,b N/A

Mannino [28] Prospective Community IM-TIV Hospitalization for pneumonia and influenza

0.97 (0.74–1.25)c 0.75 (0.57–0.98)c

Puig-Barberà [31] Retrospective Community Virosomal TIV Influenza-related hospitalization 0.53 (0.53–1.30)c 0.85 (0.54–1.34)c,d

0.94 (0.37–2.38)c,e

Puig-Barberà [31] Retrospective Community Virosomal TIV Laboratory-confirmed influenza 0.72 (0.44–1.18)c 0.75 (0.46–1.24)c,d

0.84 (0.31–2.26)c,e

Gasparini [30] Case-control Community ID-TIV Hospitalization for pneumonia and influenza

0.43 (0.20–0.91)a,f 0.54 (0.22–1.29)a,f

Van Buynder [29] Prospective case-control

Mixed IM-TIV Laboratory-confirmed influenza 0.58 (0.31–1.09)a 0.37 (0.14–0.96)a

a Odds ratio. b Converted OR (in order to uniform effect sizes, i.e. MF59-TIV vs others); authors’ reported OR was 2.16 (95% CI: 1.56–2.98). c Risk ratio. d Adjusted model. e Multilevel model. f Calculated from raw data provided by the corresponding author; statistically significant estimates are in bold.

A. Domnich et al. / Vaccine 35 (2017) 513–520 519

The only study [25] that assessed MF59-TIV among institution- alized elderly subjects and had the outcome of ILI established a very high VE of 94%. However, in that study, the reported ES had not been adjusted.

It is worth comparing our results with those obtained from pre- viously published systematic reviews and meta-analyses. Our results are in line with the paper by Gross et al. [34], which reported that influenza vaccination was effective in the elderly against some influenza-related outcomes. Specifically, our pooled estimate for hospitalization for pneumonia and influenza was very close to the pooled results of observational studies (53% and 50% in preventing pneumonia and hospitalization, respectively) reported in that meta-analysis [34]. On the other hand, both estimates from the studies included and our pooled results were higher than those of other meta-analyses [14,35]. This finding is, however, plausible, since both earlier analyses [16,35] mostly included studies on non- adjuvanted TIVs.

Concerning the relative VE, it appears that MF59-TIV is more effective that conventional non-adjuvanted TIVs. This finding is consistent with a pooled analysis of 13 immunogenicity RCTs [9], which found significantly higher antibody titers elicited by MF59-TIV than by non-adjuvanted TIVs. It should be acknowl- edged that quadrivalent inactivated vaccines (QIVs) that include both lineages of the type B virus have recently become available [36]; to the best of our knowledge, however, no study comparing MF59-TIV and non-adjuvanted QIVs has been published so far. Considering both a frequent B-type mismatch between trivalent vaccines and circulating B lineages and limited heterotypic activity between the two lineages, QIVs are presumed to provide additional benefits [36]. Indeed, a recent systematic review and meta-analysis by Moa et al. [37] reported that immunogenicity elicited by QIV was similar to that of TIV for the shared three strains, but superior to the non-TIV B lineage. However, it should be noted that, since several factors may impact VE (e.g. age, degree of mismatch, previ- ous vaccination status, priming), the additional benefits of QIV could vary substantially among different populations. Given that children and young adults have relatively higher attack rates of type B virus, these age-classes would probably be major beneficia- ries of QIV [38]. Moreover, Reed et al. [39] estimated that, in the United States population, using QIV instead of TIV would, on aver- age, produce only a modest (about 1.4%) reduction in hospitaliza- tions and mortality, although this estimate was highly dependent on season. Such a modest decrease may be partly attributed to the suboptimal efficacy of non-adjuvanted vaccines, especially in some age-classes [8]. A subsequent evaluation by Beyer et al.

[40], which was based on an umbrella meta-analysis, indicated that the estimates made by Reed et al. [39] might be too optimistic and restricted to very young and unprimed subjects. Future research, including pharmacoeconomic evaluations, should con- sider these issues in comparing aTIV and non-adjuvanted QIVs.

Like all systematic reviews, the present one suffers from some limitations. In keeping with our aims and outcomes, only observa- tional studies, the biases of which are well known, were compre- hensively searched. However, most of the studies included implemented various measures to minimize biases in selection and information and confounders. Moreover, as mentioned above, some of our pooled estimates had large CIs, and therefore a rela- tively low precision of population-wide effect sizes. In the future, large well-designed observational (especially cohort) studies will undoubtedly help to establish more precise point estimates. At the review level, the paucity of studies included did not allow us to carry out the subgroup analysis and/or meta-regression that were planned a priori. For instance, it would have been useful to carry out a subgroup analysis by degree of matching between the strains included in vaccines and those circulating during single seasons. For the same reason, we were not able to check for publi- cation bias. Finally, the inter-rater agreement in evaluating the methodological quality of the studies included may be interpreted only as poor to slight [41]. Although the NOS has been used in Cochrane reviews on influenza vaccines [17], previous research has found that the scale is difficult to use, has low inter-rater agreement [42] and low agreement between reviewers and authors [43].

To conclude, the available evidence suggests that MF59-TIV is effective in ‘‘real world” conditions, especially in preventing hospi- talizations for various influenza complications, and is superior to conventional non-adjuvanted vaccines. In our opinion, well- designed and sufficiently powered future observational studies should investigate MF59-TIV VE against laboratory-confirmed influenza. In particular, considering that we identified only one study in which the population was composed of institutionalized elderly subjects, future research should target this vulnerable pop- ulation group. Indeed, these subjects are particularly at risk of con- tracting influenza and developing severe complications, owing to their multiple underlying chronic conditions [44].

Conflict of interest

The authors have no conflicts of interest to declare.

520 A. Domnich et al. / Vaccine 35 (2017) 513–520

Appendix A. Supplementary material

Supplementary data associated with this article can be found, in the online version, at http://dx.doi.org/10.1016/j.vaccine.2016.12. 011.

References

[1] Simonsen L, Taylor RJ, Viboud C, Miller MA, Jackson LA. Mortality benefits of influenza vaccination in elderly people: an ongoing controversy. Lancet Infect Dis 2007;7:658–66. http://dx.doi.org/10.1016/S1473-3099(07)70236-0.

[2] World Health Organization (WHO). Vaccines against influenza WHO position paper – November 2012. Wkly Epidemiol Rec 2012;87:461–76.

[3] Gasparini R, Lucioni C, Lai P, Maggioni P, Sticchi L, Durando P, et al. Cost-benefit evaluation of influenza vaccination in the elderly in the Italian region of Liguria. Vaccine 2002;20(Suppl 5):B50–4.

[4] Official Journal of the European Union. Council recommendation of 22 December 2009 on seasonal influenza vaccination. <http://eur-lex.europa.eu/ LexUriServ/LexUriServ.do?uri=OJ:L:2009:348:0071:0072:EN:PDF>.

[5] Rümke HC, Richardus JH, Rombo L, Pauksens K, Plaßmann G, Durand C, et al. Selection of an adjuvant for seasonal influenza vaccine in elderly people: modelling immunogenicity from a randomized trial. BMC Infect Dis 2013;13:348. http://dx.doi.org/10.1186/1471-2334-13-348.

[6] O’Hagan DT, Tsai T, Reed S. Emulsion-based adjuvants for improved influenza vaccines. In: Del Giudice G, Rappuoli R, editors. Influenza vaccines for the future. Springer; 2011. p. 327–57.

[7] Banzhoff A, Pellegrini M, Del Giudice G, Fragapane E, Groth N, Podda A. MF59- adjuvanted vaccines for seasonal and pandemic influenza prophylaxis. Influenza Other Respir Viruses 2008;2:243–9. http://dx.doi.org/10.1111/ j.1750-2659.2008.00059.x.

[8] Tsai TF. Fluad�-MF59�-adjuvanted influenza vaccine in older adults. Infect Chemother 2013;45:159–74. http://dx.doi.org/10.3947/ic.2013.45.2.159.

[9] Podda A. The adjuvanted influenza vaccines with novel adjuvants: experience with the MF59-adjuvanted vaccine. Vaccine 2001;19:2673–80.

[10] Pellegrini M, Nicolay U, Lindert K, Groth N, Della Cioppa G. MF59-adjuvanted versus non-adjuvanted influenza vaccines: integrated analysis from a large safety database. Vaccine 2009;27:6959–65. http://dx.doi.org/10.1016/ j.vaccine.2009.08.101.

[11] Camilloni B, Basileo M, Valente S, Nunzi E, Iorio AM. Immunogenicity of intramuscular MF59-adjuvanted and intradermal administered influenza enhanced vaccines in subjects aged over 60: a literature review. Hum Vaccin Immunother 2015;11:553–63. http://dx.doi.org/10.1080/ 21645515.2015.1011562.

[12] Weinberg GA, Szilagyi PG. Vaccine epidemiology: efficacy, effectiveness, and the translational research roadmap. J Infect Dis 2010;201:1607–10. http://dx. doi.org/10.1086/652404.

[13] Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Meta- analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA 2000;283:2008–12. http://dx.doi.org/10.1001/jama.283.15.200.

[14] World health organization (WHO). Definition of an older or elderly person. <http://www.who.int/healthinfo/survey/ageingdefnolder/en/>.

[15] Osterholm MT, Kelley NS, Sommer A, Belongia EA. Efficacy and effectiveness of influenza vaccines: a systematic review and meta-analysis. Lancet Infect Dis 2012;12:36–44. http://dx.doi.org/10.1016/S1473-3099(11)70295-X.

[16] Vu T, Farish S, Jenkins M, Kelly H. A meta-analysis of effectiveness of influenza vaccine in persons aged 65 years and over living in the community. Vaccine 2002;20:1831–6.

[17] Jefferson T, Di Pietrantonj C, Al-Ansary LA, Ferroni E, Thorning S, Thomas RE. Vaccines for preventing influenza in the elderly. Cochrane Database Syst Rev 2010(2):CD004876. http://dx.doi.org/10.1002/14651858.CD004876.pub.

[18] Frey SE, Reyes MR, Reynales H, Bermal NN, Nicolay U, Narasimhan V, et al. Comparison of the safety and immunogenicity of an MF59�-adjuvanted with a non-adjuvanted seasonal influenza vaccine in elderly subjects. Vaccine 2014;32:5027–34. http://dx.doi.org/10.1016/j.vaccine.2014.07.013.

[19] Minutello M, Senatore F, Cecchinelli G, Bianchi M, Andreani T, Podda A, et al. Safety and immunogenicity of an inactivated subunit influenza virus vaccine combined with MF59 adjuvant emulsion in elderly subjects, immunized for three consecutive influenza seasons. Vaccine 1999;17:99–104.

[20] An Advisory Committee Statement (ACS) National Advisory Committee on Immunization (NACI). Literature review on pediatric Fluad� influenza vaccine use in children 6-72 months of age. <http://www.phac-aspc.gc.ca/naci-ccni/ assets/pdf/pediatric-pediatrique-fluad-eng.pdf>.

[21] Wells GA, Shea B, O’Connell D, Peterson JEA, Welch V, Losos M, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. <http://www.ohri.ca/programs/clinical_epidemiology/ oxford.asp>.

[22] Beck CR, McKenzie BC, Hashim AB, Harris RC, Zanuzdana A, Agboado G, et al. Influenza vaccination for immunocompromised patients: systematic review and meta-analysis from a public health policy perspective. PLoS ONE 2011;6: e29249. http://dx.doi.org/10.1371/journal.pone.0029249.

[23] Barendregt JJ, Suhail AD. MetaXL User Guide. <http://www.epigear.com/ index_files/MetaXL%20User%20Guide.pdf>.

[24] Puig-Barberà J, Diez-Domingo J, Pérez Hoyos S, Belenguer Varea A, González Vidal D. Effectiveness of the MF59-adjuvanted influenza vaccine in preventing emergency admissions for pneumonia in the elderly over 64 years of age. Vaccine 2004;23:283–9. http://dx.doi.org/10.1016/j.vaccine.2004.07.01.

[25] Iob A, Brianti G, Zamparo E, Gallo T. Evidence of increased clinical protection of an MF59-adjuvant influenza vaccine compared to a non-adjuvant vaccine among elderly residents of long-term care facilities in Italy. Epidemiol Infect 2005;133:687–93.

[26] Puig-Barberà J, Díez-Domingo J, Varea AB, Chavarri GS, Rodrigo JA, Hoyos SP, et al. Effectiveness of MF59-adjuvanted subunit influenza vaccine in preventing hospitalisations for cardiovascular disease, cerebrovascular disease and pneumonia in the elderly. Vaccine 2007;25:7313–21. http://dx. doi.org/10.1016/j.vaccine.2007.08.039.

[27] Puig-Barberà J, Díez-Domingo J, Arnedo-Pena A, Ruiz-García M, Pérez-Vilar S, Micó-Esparza JL, et al. Effectiveness of the 2010–2011 seasonal influenza vaccine in preventing confirmed influenza hospitalizations in adults: a case- case comparison, case-control study. Vaccine 2012;30:5714–20. http://dx.doi. org/10.1016/j.vaccine.2012.07.006.

[28] Mannino S, Villa M, Apolone G, Weiss NS, Groth N, Aquino I, et al. Effectiveness of adjuvanted influenza vaccination in elderly subjects in northern Italy. Am J Epidemiol 2012;176:527–33. http://dx.doi.org/10.1093/aje/kws313.

[29] Van Buynder PG, Konrad S, Van Buynder JL, Brodkin E, Krajden M, Ramler G, et al. The comparative effectiveness of adjuvanted and unadjuvanted trivalent inactivated influenza vaccine (TIV) in the elderly. Vaccine 2013;31:6122–8. http://dx.doi.org/10.1016/j.vaccine.2013.07.059.

[30] Gasparini R, Amicizia D, Lai PL, Rossi S, Panatto D. Effectiveness of adjuvanted seasonal influenza vaccines (Inflexal V � and Fluad �) in preventing hospitalization for influenza and pneumonia in the elderly: a matched case- control study. Hum Vaccin Immunother 2013;9:144–52. http://dx.doi.org/ 10.4161/hv.22231.

[31] Puig-Barberà J, Natividad-Sancho A, Calabuig-Pérez J, Lluch-Rodrigo JA, Pastor- Villalba E, Martínez-Úbeda S, et al. MF59-adjuvanted and virosomal influenza vaccines for preventing influenza hospitalization in older people: comparative effectiveness using the Valencia health care information system. Vaccine 2013;31:3995–4002. http://dx.doi.org/10.1016/j.vaccine.2013.05.070.

[32] Spadea A, Unim B, Colamesta V, Meneghini A, D’Amici AM, Giudiceandrea B, et al. Is the adjuvanted influenza vaccine more effective than the trivalent inactivated vaccine in the elderly population? Results of a case-control study. Vaccine 2014;32:5290–4. http://dx.doi.org/10.1016/j.vaccine.2014.07.077.

[33] Manzoli L, Ioannidis JP, Flacco ME, De Vito C, Villari P. Effectiveness and harms of seasonal and pandemic influenza vaccines in children, adults and elderly: a critical review and re-analysis of 15 meta-analyses. Hum Vaccin Immunother 2012;8:851–62. http://dx.doi.org/10.4161/hv.19917.

[34] Gross PA, Hermogenes AW, Sacks HS, Lau J, Levandowski RA. The efficacy of influenza vaccine in elderly persons. A meta-analysis and review of the literature. Ann Intern Med 1995;123:518–27.

[35] Jefferson T, Rivetti D, Rivetti A, Rudin M, Di Pietrantonj C, Demicheli V. Efficacy and effectiveness of influenza vaccines in elderly people: a systematic review. Lancet 2005;366:1165–74. http://dx.doi.org/10.1016/S0140-6736(05)67339- 4.

[36] Baxter D. Evaluating the case for trivalent or quadrivalent influenza vaccines. Hum Vaccin Immunother 2016;12:2712–7. http://dx.doi.org/10.1080/ 21645515.2015.1091130.

[37] Moa AM, Chughtai AA, Muscatello DJ, Turner RM, MacIntyre CR. Immunogenicity and safety of inactivated quadrivalent influenza vaccine in adults: a systematic review and meta-analysis of randomised controlled trials. Vaccine 2016;34:4092–102. http://dx.doi.org/10.1016/j.vaccine.2016.06.064.

[38] Ambrose CS, Levin MJ. The rationale for quadrivalent influenza vaccines. Hum Vaccin Immunother 2012;8:81–8. http://dx.doi.org/10.4161/hv.8.1.17623.

[39] Reed C, Meltzer MI, Finelli L, Fiore A. Public health impact of including two lineages of influenza B in a quadrivalent seasonal influenza vaccine. Vaccine 2012;30:1993–8. http://dx.doi.org/10.1016/j.vaccine.2011.12.098.

[40] Beyer WE, Boulfich M, Osterhaus AD. QIV versus TIV: evidence-based impact of lineage mismatch on influenza B vaccine efficacy. An umbrella meta-analysis of published literature. Presented at: The fifth international conference and exhibition on influenza vaccines for the world; 6–9 October 2015, Albufeira, Algrave, Portugal.

[41] Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74. http://dx.doi.org/10.2307/2529310.

[42] Hartling L, Milne A, Hamm MP, Vandermeer B, Ansari M, Tsertsvadze A, et al. Testing the Newcastle Ottawa Scale showed low reliability between individual reviewers. J Clin Epidemiol 2013;66:982–93. http://dx.doi.org/10.1016/j. jclinepi.2013.03.003.

[43] Lo CK, Mertz D, Loeb M. Newcastle-Ottawa Scale: comparing reviewers’ to authors’ assessments. BMC Med Res Methodol 2014;14:45. http://dx.doi.org/ 10.1186/1471-2288-14-45.

[44] Chan TC, Fan-Ngai Hung I, Ka-Hay Luk J, Chu LW, Hon-Wai Chan F. Effectiveness of influenza vaccination in institutionalized older adults: a systematic review. J Am Med Dir Assoc 2014;15. http://dx.doi.org/10.1016/ j.jamda.2013.10.00. 226.e1-6.