Week 3 Critical Appraisal of Practice Guidelines

Domains with Questions:

Score (1-7):

Comments:

Domain 1: SCOPE AND PURPOSE

1. The overall objective(s) of the guideline is (are) specifically described.

7

Kalil et al.'s (2016) overall objectives are found on the first page where an abstract would normally be found.

2. The health question(s) covered by the guideline is (are) specifically described.

6

While not introduced overtly as the health questions covered in this guideline, it is very easy to tell from the introduction summary that Kalil et al. are answering the questions how to diagnose and how to treat HAP and VAP. In the scope and purpose section it is even more clear.

3. The population (patients, public, etc.) to whom the guideline is meant to apply is specifically described.

7

Kalil et al. explain in great detail who the target population of patients is to whom the guideline is meant to apply.

Domain 1 total: 20/21 calculates to 94.4%

Domain 2: STAKEHOLDER INVOLVEMENT

4. The guideline development group includes individuals from all relevant professional groups.

7

Professionals from a multitude of relevant groups (e.g. pulmonary, critical care, infectious disease, pathology, and microbiology medicine groups) who are from at least six countries are part of the Kalil et al. author group. Comment by Lane, Kari: Who is missing from this group/

5. The views and preferences of the target population (patients, public, etc.) have been sought.

2

In their table on interpretation of strong and weak (conditional) recommendations, Kalil et al. take patient preference into consideration, but the patient preference is assumed rather than explicitly sought. This is likely because evidence on patient preferences for how HAP and VAP are diagnosed and treated is scarce and not very applicable as patients are grossly uninformed on the topic.

6. The target users of the guideline are clearly defined.

7

Kalil et al. clearly define the target audience for the guidelines on page nine in the scope and purpose section.

Domain 2 Total: 16/21 calculates to 72.2%

Domain 3: RIGOUR OF DEVELOPMENT

7. Systematic methods were used to search for evidence.

7

Kalil et al. describe their methodology employed for developing the guideline, including that two health science librarians who are experts in searching for evidence developed the literature searches, and more details on the searches can be found in supplementary material.

8. The criteria for selecting the evidence are clearly described.

3

Kalil et al. write that the literature search results were reviewed by panelists who then selected and reviewed the articles. More details are available on how the search was conducted and how the evidence was evaluated after selection than on how the articles were selected. Comment by Lane, Kari: Where are they available from? How did you decide on a score of 3 here, justify your decision.

9. The strengths and limitations of the body of evidence are clearly described.

7

A summary of the evidence with the strengths and limitations is included after every recommendation Kalil et al. give. Comment by Lane, Kari: What limitations do you see? Often the creators do not see limitations in their own work, so really look for downfalls.

10. The methods for formulating the recommendations are clearly described.

7

Kalil et al. include a rationale for the recommendation section after the summary of evidence that accompanies every recommendation.

11. The health benefits, side effects, and risks have been considered in formulating the recommendations.

7

This is thoroughly discussed by Kalil et al. in the rationale for the recommendation section. Comment by Lane, Kari: What is missing?

12. There is an explicit link between the recommendations and the supporting evidence.

DOMAIN 3. RIGOUR OF DEVELOPMENT continued

7

Kalil et al. give their recommendation and then detail the evidence and their rationale, clearly linking every recommendation with the evidence.

13. The guideline has been externally reviewed by experts prior to its publication.

7

Per Kalil et al., external peer reviewers gave feedback, and then the guideline was reviewed and endorsed by the Society of Critical Care Medicine and the Society for Healthcare Epidemiology of America before final review and approval was obtained from the boards of directors of the Infectious Diseases Society of America and the American Thoracic Society.

14. A procedure for updating the guideline is provided.

2

Kalil et al. describe how this guideline is an update from their 2005 guidelines, but they do not address a procedure for future updating.

Domain 3 Total: 47/56 calculates to 81.3%

Domain 4: CLARITY OF PRESENTATION

15. The recommendations are specific and unambiguous.

7

Each recommendation summary is written clearly and precisely by Kalil et al..

16. The different options for management of the condition or health issue are clearly presented.

7

Kalil et al. specify options when appropriate and also include room for provider judgment in individual patients or special situations.

17. Key recommendations are easily identifiable.

DOMAIN 5. APPLICABILITY

7

Kalil et al. do an excellent job in how they present the concise recommendation summaries in the beginning and also how they re-present their recommendations clearly in the more detailed section later.

Domain 4 Total: 21/21 calculates to 100%

Domain 5: APPLICABILITY

18. The guideline describes facilitators and barriers to its application.

4

Barriers to application of the guideline are described by Kalil et al. in a few applicable places. Comment by Lane, Kari: And facilitators? What is missing here, can you see any gaps or loopholes, where this might not work in your setting?

19. The guideline provides advice and/or tools on how the recommendations can be put into practice.

5

Tables are given that can be used as tools to implement the guideline recommendations in specific situations.

20. The potential resource implications of applying the recommendations have been considered.

6

Kalil et al. are aware of resources necessary for diagnosing and treating HAP and VAP, and they address resource considerations. Comment by Lane, Kari: Do they offer resources to assist with implementation?

21. The guideline presents monitoring and/or auditing criteria.

7

Several of Kalil et al.'s recommendations address monitoring patients' response to treatment. Comment by Lane, Kari: Would these be easily applied? How so?

Domain 5 Total: 19/28 calculates to 75%

Domain 6: EDITORIAL INDEPENDENCE

22. The views of the funding body have not influenced the content of the guideline.

7

The content of Kalil et al.'s guideline is very clearly based on researched evidence, and no apparent other influences were noted. Comment by Lane, Kari: Does the guideline specifically declare conflicts of interest or state no conflicts of interest were declared?

23. Competing interests of guideline development group members have been recorded and addressed.

7

Kalil et al. include both a "discloser and management of potential conflicts of interest" section in the body of their guideline and a potential conflicts of interest list at the end.

Domain 6 Total: 14/14 calculates to 100%

Total Domain scores:

1. 20/21 calculates to 94.4%

2. 16/21 calculates to 72.2%

3. 47/56 calculates to 81.3%

4. 21/21 calculates to 100%

5. 19/28 calculates to 75%

6. 14/14 calculates to 100%

Overall Total: 137/161 calculates to 82.6%

Overall Rating of Quality:

6

Using this Agree II Instrument (2009) to structure this evaluation, the guideline by Kalil et al. (2016) performs very well in the areas of scope and purpose, clarity of presentation, and editorial independence, obtaining scores of greater than 90% in all three of those domains. Scores in the other three domains are all over 70%. The lowest score is in the domain of stakeholder involvement, and this area is low partially due to the low applicability of patient preferences in how HAP and VAP is diagnosed and treated.

Recommended for use:

YES

I see no reason why the guideline by Kalil et al. should not be followed in the vast majority of HAP and VAP diagnoses or suspected diagnoses, and the rigor with which the authors evaluated evidence to build the guideline makes it an excellent reference for finding best practices. The clarity with which the recommendations are presented makes it easy to use for all providers.