Research Critique Paper
By S. Labeau, RN, MSc, D.M. Vandijck, RN, MSc, MA, B. Claes, RN, MSc, P. Van Aken, RN, MSc, and S.I. Blot, RN, MSc, PhD, on behalf of the executive board of the Flemish Society for Critical Care Nurses
Background Nurses’ lack of knowledge may be a barrier to adherence to evi- dence-based guidelines for preventing ventilator-associated pneumonia.
Objective To develop a reliable and valid questionnaire for evaluating critical care nurses’ knowledge of evidence-based guidelines for preventing ventilator-
associated pneumonia.
Methods Ten nursing-related interventions were identified from a review of evidence-based guidelines for preventing ventilator-associated pneumonia.
Selected interventions and multiple-choice questions (1 question per interven-
tion) were subjected to face and content validation. Item difficulty, item dis-
crimination, and the quality of the response alternatives or options for
answers (possible responses) were evaluated on the test results of 638 critical
care nurses.
Results Face and content validity were achieved for 9 items. Values for item difficulty ranged from 0.1 to 0.9. Values for item discrimination ranged from
0.10 to 0.65. The quality of the response alternatives led to the detection of
widespread misconceptions among critical care nurses.
Conclusion The questionnaire is reliable and has face and content validity. Results of surveys with this questionnaire can be used to focus educational
programs on preventing ventilator-associated pneumonia. (American Journal
of Critical Care. 2007;16:371-377)
CRITICAL CARE NURSES’ KNOWLEDGE OF EVIDENCE- BASED GUIDELINES FOR PREVENTING VENTILATOR- ASSOCIATED PNEUMONIA: AN EVALUATION QUESTIONNAIRE
Critical Care Evaluation
www.ajcconline.org AAJJCCCC AMERICAN JOURNAL OF CRITICAL CARE, July 2007, Volume 16, No. 4 371
Recently, lack of knowledge was indicated as a barrier for adherence to evidence-based practice.23
Although knowledge does not ensure adherence, misconceptions about effective prevention strategies can be important in decision making. The reduc- tions in the rates of hospital-acquired infection26,27
that occurred after educational programs on strate- gies to prevent infection provide indirect evidence for the value of knowledge.
Our objective was to develop a reliable and valid questionnaire to determine critical care nurses’ knowledge of evidence-based guidelines for prevent- ing VAP.
Methods Selection of Interventions and Design of the Questionnaire
The selection of interventions or strategies to prevent VAP was based on a recently published review14 of evidence-based guidelines. In a search for relevant randomized, controlled trials and system- atic reviews that involved adults who were treated with mechanical ventilation and that were pub-
lished before April 2003, Dodek et al14 looked for physical, body positioning, and pharmacological interventions that might influence the development of VAP. Independently and in duplicate, these authors scored the validity of trials; the effect size and confidence intervals; the homogeneity of results; and safety, feasibility, and economic issues. On the basis of this review,14 a total of 10 interventions or strategies with relevance for nursing practice were selected:
1. Use of oral endotracheal tubes 2. Frequency of ventilator circuit changes 3. Use of a heat and moisture exchanger 4. Frequency of humidifier changes 5. Use of a closed suction system 6. Frequency of change in suction system 7. Drainage of subglottic secretions 8. Use of kinetic beds 9. Use of semirecumbent positioning
10. Chest physiotherapy
A multiple-choice question with 4 response alter- natives or options (the correct answer/response and 3 distractors or alternatives that are not the answer) was developed for each item on the list28 (Table 1). For each test item, the response alternatives included the phrase “I do not know” to avoid gambling by the respondents and 2 interventions with investigated preventive value. In their evidence-based clinical practice guideline for the prevention of VAP, Dodek et al14 advise consideration of 2 interventions, drainage of subglottic secretions and use of kinetic beds, but make no specific recommendations for their use because of cost concerns. Therefore, ques- tions on these 2 interventions were designed to assess knowledge about the impact of the interventions on
V entilator-associated pneumonia (VAP) is defined as pneumonia that develops more than 48 to 72 hours after initiation of mechanical ventilation.1-4 With an incidence of 8% to 68%,5,6 VAP is the most common hospital-acquired infection among patients who require ventilatory support.6-8 Moreover, VAP is associated with high morbidity and mortality rates, increased duration of ventilatory sup-
port and hospitalization, and increased use of healthcare resources.9-13 Prevention of VAP focuses on avoiding microaspiration of subglottic secretions, preventing oropharyngeal colo- nization with exogenous pathogens, and preventing contamination of ventilator equip- ment.5,14,15 Evidence-based guidelines for the prevention of VAP have been developed4,14,16 and have been promoted by programs and campaigns of authoritative organizations.17,18 Neverthe- less, nonadherence to these guidelines has been reported.5,19-21 Also, the results of assessments22-25
of nurses’ knowledge of evidence-based practice in general have been disappointing.
About the Authors S. Labeau is a PhD student in the Faculty of Healthcare, Ghent University College, Ghent, Belgium. D.M. Vandijck is a PhD student in the Intensive Care Department, Ghent University Hospital, and the Faculty of Medicine and Health Sciences, Ghent University. B. Claes is head of the ICU Nursing Department and P. Van Aken is direc- tor of the Nursing Department, University Hospital of Antwerp, Antwerp, Belgium. S.I. Blot is a researcher at Ghent University Hospital and a professor in the Faculty of Medicine and Health Sciences of Ghent University and at Ghent University College, Ghent, Belgium.
Corresponding author: Stijn Blot, PhD, Ghent University Hospital, Intensive Care Dept, De Pintelaan 185, 9000 Ghent, Belgium (e-mail: stijn.blot@UGent.be).
372 AAJJCCCC AMERICAN JOURNAL OF CRITICAL CARE, July 2007, Volume 16, No. 4 www.ajcconline.org
isfying, and values less than 0.15 are bad/mediocre.
Quality of the Response Alterna- tives. The quality of a response alter- native is defined by calculating the proportion of respondents who choose the alternative. Values range from 0.0 to 1.0. Response alterna- tives with a value of 0.0 are not attractive, and those with a value of 1.0 might be too attractive.
Population Surveyed The questionnaire was distrib-
uted and collected during the annual congress of the Flemish Society of Critical Care Nurses (Ghent, Belgium, November 25, 2005). Of the 855 registered participants, 638 com- pleted the questionnaire (response rate 74.6%). The responses were collected anonymously. The ques- tionnaire also included questions on general char- acteristics of the respondents: sex, years of ICU experience, number of ICU beds in the hospital where the respondent worked, and whether the respondent had a special degree in emergency and intensive care.
Results Expert Validation
The experts reported that some items needed to be slightly reworded to be clear. According to the experts, question 10 (on chest physiotherapy) was irrelevant for nurses. Therefore, this question was omitted; the final questionnaire consisted of 9 items on interven- tions to prevent VAP.
Item Analysis Overall values for item difficulty and discrimina-
tion were very good to satisfying (Table 1). For ques- tion 9 (patient positioning), however, the values were borderline, indicating that respondents had a good knowledge of this intervention. Nevertheless, question 9 was kept in the questionnaire because of the enormous impact of patient positioning on the prevention of VAP and the major relevance of this question for ICU nurses. Also, because the question- naire is a criterion-referenced test, an item that is valuable for the content does not necessarily have to be excluded because the item is too easy.32
In the analysis of the quality of the response alternatives, some had values of 0.0, suggesting that
the risk for VAP. For 2 other interventions, closed suc- tion system and frequency of ventilator circuit changes, the recommendations of Dodek et al are based on economic considerations.
Expert Validation The selected preventive interventions and ques-
tionnaire were presented to a panel of 8 experts for face and content validation.29 Each expert had at least 3 years of experience in an intensive care unit (ICU), a master’s degree in nursing sciences (or medicosocial sciences), and a particular interest in ICU-acquired infections.
To achieve face validity, the experts were asked if all questions were clearly worded and would not be misinterpreted. For content validity, the experts eval- uated the nursing relevance of the 10 selected inter- ventions by using a scale of 1 to 3, where 1 = not relevant, 2 = relevant but not necessary, and 3 = absolutely necessary. Additionally, the experts were asked if questions about any other preventive inter- ventions should be added to the questionnaire.
The remarks of the panel were collected and dis- cussed and were used to revise the questionnaire. After the revision, the experts examined the question- naire again; they unanimously declared agreement with its content and clarity.
Assessment of the Questionnaire Revising tests on the basis of test scores is an
essential part of improving instruction.28 Therefore, the items on the questionnaire were analyzed to determine their level of difficulty and discrimination, and the quality of the 4 response alternatives or options for each question was evaluated.28,30,31
Difficulty Level. The difficulty level of an item or question is defined as the proportion of respondents who answer the question correctly.28,30,31 Possible val- ues range from 0.0 to 1.0. Items that are answered correctly by more than 90% of the respondents (value >0.9) are considered too easy; items answered correctly by less than 10% of the respondents (value <0.1) are considered too difficult.
Item Discrimination. A discrimination index indi- cates the extent to which items on the questionnaire discriminate between high scorers and low scorers. The following formula was used to divide respon- dents into high scorers and low scorers, with 27% of respondents in each group: (number of correct answers in the high-scorer group – number of correct answers in the low-scorer group)/total number of correct answers in both groups. Values of 0.35 and higher are (very) good, values from 0.25 to 0.35 are satisfying/good, values 0.15 to 0.25 are mediocre/sat-
www.ajcconline.org AAJJCCCC AMERICAN JOURNAL OF CRITICAL CARE, July 2007, Volume 16, No. 4 373
Nurses’ lack of knowledge may be a barrier to adherence to evidence-based guidelines for preventing ventilator- associated pneumonia.
Nurses may be convinced that an intervention without evidence- based preventive value is preferred over the evidence-based intervention.
Table 1 Questionnairea
Item difficulty
Item discrimination
Quality of the option
1. Oral vs nasal route for endotracheal intubation 0.2 0.60
Ab Oral intubation is recommended 0.2
B Nasal intubation is recommended 0.1 C Both routes of intubation can be recommended 0.6 D I do not know 0.1
2. Frequency of ventilator circuit changes 0.5 0.35 A It is recommended to change circuits every 48 hours (or when clinically indicated) 0.2 B It is recommended to change circuits every week (or when clinically indicated) 0.3
Cb It is recommended to change circuits for every new patient (or when clinically indicated) 0.5
D I do not know 0.0 3. Type of airway humidifier 0.5 0.30
A Heated humidifiers are recommended 0.2
Bb Heat and moisture exchangers are recommended 0.5
C Both types of humidifiers can be recommended 0.1 D I do not know 0.2
4. Frequency of humidifier changes 0.1 0.55 A It is recommended to change humidifiers every 48 hours (or when clinically indicated) 0.6 B It is recommended to change humidifiers every 72 hours (or when clinically indicated) 0.1
Cb It is recommended to change humidifiers every week (or when clinically indicated) 0.1
D I do not know 0.2 5. Open vs closed suction systems 0.2 0.40
A Open suction systems are recommended 0.0
Bb Closed suction systems are recommended 0.2
C Both systems can be recommended 0.7 D I do not know 0.1
6. Frequency of change in suction systems 0.2 0.65 A Daily changes are recommended (or when clinically indicated) 0.5 B Weekly changes are recommended (or when clinically indicated) 0.2
Cb It is recommended to change systems for every new patient (or when clinically indicated) 0.2
D I do not know 0.1 7. Endotracheal tubes with extra lumen for drainage of subglottic secretions 0.6 0.30
Ab These endotracheal tubes reduce the risk for VAP 0.6
B These endotracheal tubes increase the risk for VAP 0.0 C These endotracheal tubes do not influence the risk for VAP 0.1 D I do not know 0.3
8. Kinetic vs standard beds 0.5 0.50 A Kinetic beds increase the risk for VAP 0.0
Bb Kinetic beds reduce the risk for VAP 0.5
C The use of kinetic beds does not influence the risk for VAP 0.2 D I do not know 0.3
9. Patient positioning 0.9 0.10 A Supine positioning is recommended 0.0
Bb Semirecumbent positioning is recommended 0.9
C The position of the patient does not influence the risk for VAP 0.1 D I do not know 0.0
10. Chest physiotherapyc
A Chest physiotherapy reduces the risk for VAP B Chest physiotherapy does not reduce the risk for VAP C The influence of chest physiotherapy on the risk for VAP is unknown D I do not know
Question
Abbreviation: VAP, ventilator-associated pneumonia. a Blank cells = not applicable. b Correct answer. c This question was omitted after experts’ validation of the questionnaire.
reformulation should be considered. Nevertheless, this finding also may indicate that inclusion of the standard response alternative “I do not know” restrained respondents from gambling. In addition, because the response alternatives were restricted to interventions with an investigated preventive value, our formulation possibilities were limited. Of note, the score for the question 9 (patient positioning) option “Supine positioning is recommended” was 0.0, although this intervention is often used in daily practice. This finding illustrates a discrepancy between knowing what is prescribed and what is actually implemented in daily practice. Therefore, despite its low score, this option was not changed for the final version of the questionnaire.
The quality of the response alternatives also indicated the extent of existing misconceptions about the preventive value of certain interventions. The responses to the final questionnaire indicated that nurses thought that both the oral and nasal routes for intubation were recommended (value 0.6); how- ever, the oral route (value 0.2) is recommended in the guidelines. Respondents also thought that a change in humidifiers every 48 hours (or when clini- cally indicated) was recommended (value 0.6), whereas guidelines recommend weekly changes (or when clinically indicated; value 0.1). The respon- dents thought that both open and closed suction sys- tems were recommended (value 0.7), but only closed suction systems (value 0.2) are recommended in the guidelines. For frequency of change of suction sys- tems, nurses thought that daily changes (or when clinically indicated) were recommended (value 0.5), whereas the guidelines recommend changes for every new patient who needs mechanical ventilation (or when clinically indicated; value 0.2).
For all 4 of these items, respondents are con- vinced that an intervention without evidence-based preventive value is preferred over the evidence-based intervention. Mapping out this kind of widely spread misconception is important for better focusing edu- cation of critical care nurses.
Characteristics of the Sample Most of the 638 respondents were women (n =
472, 74%; Table 2). A total of 274 respondents (43%) had more than 10 years of ICU experience, and 274 worked in units with more than 15 beds. Most respondents (n = 437, 68%) had a special degree in intensive care and emergency nursing.
Discussion and Limitations We developed a reliable questionnaire for evalu-
ating critical care nurses’ knowledge of evidence-
based guidelines for preventing VAP. Face and con- tent validity were achieved. As a result of experts’ validation, the original 10-item questionnaire was adapted and reduced to 9 items.
In the United States, some of the interventions mentioned in the ques- tionnaire, such as frequency of venti- lator circuit changes and frequency of humidifier changes, are imple- mented by respiratory care practi- tioners. In Belgium, where this study was conducted, and in the rest of Europe, these 2 interventions are implemented by critical care nurses. We are convinced that all the inter- ventions mentioned in the question- naire are relevant for critical care nurses, because nurses have a major role in monitoring patients’ care to determine if best practices are fol- lowed. Additional interventions or strategies that are directly under the control of nurses in both the United States and Europe, such as chlorhexi- dine mouth rinse, were not included in our questionnaire because the questions address only evidence- based interventions from the review by Dodek et al.14
Item analysis of the questionnaire was based on the responses of 638 nurses who attended the annual congress of the Flemish Society of Critical Care Nurses. This convenience sampling may have led to selection bias and may have created a barrier to extrapolating our results. Nevertheless, our sam- ple represents 21% of all Flemish critical care
Table 2 General characteristics of the population surveyed (n = 638)
166 472
153 111 100 274
104 177 274 83
437 201
26 74
24 17 16 43
16 28 43 13
68 32
Sex Men Women
Years of intensive care unit experience <1 1-5 6-10 >10
No. of beds in intensive care unit <8 8-15 15 Data missing
Degree in intensive care or emergency nursing Yes No
Characteristic No. of
respondents %
www.ajcconline.org AAJJCCCC AMERICAN JOURNAL OF CRITICAL CARE, July 2007, Volume 16, No. 4 375
In the United States, respiratory care practitioners manage ventilator circuit and humidifier changes; in Belgium and elsewhere in Europe, critical care nurses manage these strategies.
376 AAJJCCCC AMERICAN JOURNAL OF CRITICAL CARE, July 2007, Volume 16, No. 4 www.ajcconline.org
nurses. Moreover, this bias should be limited because the federal government in Belgium requires all critical care nurses who have a special degree in intensive care and emergency nursing to attend at least 16 hours a year of continuing education to maintain the degree.
Finally, guidelines can changes over time. Adaptation and reevaluation of the questionnaire will be needed each time new evidence-based inter- ventions for preventing VAP are discovered.
Conclusion A reliable questionnaire was developed to
assess critical care nurses’ knowledge of evidence- based interventions for preventing VAP. Face and content validity were achieved. The results of sur- veys with this questionnaire can be used to focus educational programs on VAP. The questionnaire also can be used before and after educational pro- grams to assess the effect of the programs on nurses’ knowledge of interventions to prevent VAP.
FINANCIAL DISCLOSURES None reported.
REFERENCES 1. Flanders SA, Collard HR, Saint S. Nosocomial pneumonia:
state of the science. Am J Infect Control. 2006;34:84-93. 2. Chastre J. Conference summary: ventilator-associated
pneumonia. Respir Care. 2005;50(7):975-983. 3. Kollef MH. Prevention of hospital-associated pneumonia
and ventilator-associated pneumonia. Crit Care Med. 2004;32(6):1396-1405.
4. American Thoracic Society, Infectious Diseases Society of America. Guidelines for the management of adults with hospital-acquired, ventilator-associated, and healthcare- associated pneumonia. Am J Respir Crit Care Med. 2005;171(4):388-416.
5. Heyland DK, Cook DJ, Dodek PM. Prevention of ventilator- associated pneumonia: current practice in Canadian inten- sive care units. J Crit Care. 2002;17(3):161-167.
6. Salahuddin N, Zafar A, Sukhyani L, et al. Reducing ventila- tor-associated pneumonia rates through a staff education programme. J Hosp Infect. 2004;57(3):223-227.
7. Zack JE, Garrison T, Trovillion E, et al. Effect of an educa- tion program aimed at reducing the occurrence of ventila- tor-associated pneumonia. Crit Care Med. 2002;30(11):2407-2412.
8. Depuydt PO, Myny D, Blot SI. Nosocomial pneumonia: aeti- ology, diagnosis and treatment. Curr Opin Pulm Med. 2006;12:192-197.
9. Heyland DK, Cook DJ, Griffith L, Keenan S, Brun-Buisson C. The attributable morbidity and mortality of ventilator-asso- ciated pneumonia in the critically ill patient. Am J Respir Crit Care Med. 1999;159(4 pt 1):1249-1256.
10. Myny D, Depuydt P, Colardyn F, Blot S. Ventilator-associated pneumonia in a tertiary care ICU: analysis of risk factors for acquisition and mortality. Acta Clin Belg. 2005;60(3):114-121.
11. Rello J, Ollendorf DA, Oster G, et al. Epidemiology and out- comes of ventilator-associated pneumonia in a large US database. Chest. 2002;122(6):2115-2121.
12. Rello J, Rue M, Jubert P, et al. Survival in patients with nosocomial pneumonia: impact of the severity of illness and the etiologic agent. Crit Care Med. 1997;25(11): 1862-1867.
13. Safdar N, Dezfulian C, Collard HR, Saint S. Clinical and eco- nomic consequences of ventilator-associated pneumonia: a systematic review. Crit Care Med. 2005;33:2184-2193.
14. Dodek P, Keenan S, Cook D, et al. Evidence-based clinical practice guideline for the prevention of ventilator-associ- ated pneumonia. Ann Intern Med. 2004;141(4):305-313.
15. Safdar N, Crnich CJ, Maki DG. The pathogenesis of ventila- tor-associated pneumonia: its relevance to developing effective strategies for prevention. Respir Care. 2005;50:725-741.
16. Tablan OC, Anderson LJ, Besser R, Bridges C, Hajjeh R. Guidelines for preventing health-care-associated pneumo- nia, 2003: recommendations of CDC and the Healthcare Infection Control Practices Advisory Committee. MMWR Recomm Rep. 2004;53(RR-3):1-36. http://www.cdc.gov /mmwr/preview/mmwrhtml/rr5303a1.htm. Accessed April 12, 2007.
17. McKay CA, Speers M. Watch for AACN practice alerts: new communication to advise on clinical practice standards. AACN News. 2004;21(2). http://www.aacn.org/AACN/aacnnews.nsf /GetArticle/ArticleThree212. Accessed April 12, 2007.
18. Institute for Healthcare Improvement. 100 000 Lives Campaign: prevention of ventilator-associated pneumonia. http://www.ihi.org/NR/rdonlyres/35B687FE-D4D0-44E5- A671-E76D74BB8D9F/0/cap.pdf. Accessed November 29, 2006.
19. Rello J, Lorente C, Bodi M, Diaz E, Ricart M, Kollef MH. Why do physicians not follow evidence-based guidelines for pre- venting ventilator-associated pneumonia? A survey based on the opinions of an international panel of intensivists. Chest. 2002;122:656-661.
20. Ricart M, Lorente C, Diaz E, Kollef MH, Rello J. Nursing adherence with evidence-based guidelines for preventing ventilator-associated pneumonia. Crit Care Med. 2003; 1:2693-2696.
21. Sierra R, Benitez E, Leon C, Rello J. Prevention and diagno- sis of ventilator-associated pneumonia: a survey on current practices in southern Spanish ICUs. Chest. 2005;128: 1667-1673.
22. Mott B, Nolan J, Zarb N, et al. Clinical nurses’ knowledge of evidence-based practice: constructing a framework to eval- uate a multifaceted intervention for implementing EBP. Con- temp Nurse. 2005;19(1-2):96-104.
23. Pravikoff DS, Tanner AB, Pierce ST. Readiness of US nurses for evidence-based practice: many don’t understand or value research and have had little or no training to help them find evidence on which to base their practice. Am J Nurs. 2005;105(9):40-52.
24. Thompson C, McCaughan D, Cullum N, Sheldon TA, Mulhall A, Thompson DR. The accessibility of research-based knowl- edge for nurses in United Kingdom acute care settings. J Adv Nurs. 2001;36(1):11-22.
25. Biancofiore G, Barsotti E, Catalani V, et al. Nurses’ knowl- edge and application of evidence-based guidelines for pre- venting ventilator-associated pneumonia. Minerva Anestesiol. 2006;72:1-10.
26. Berenholtz SM, Pronovost PJ, Lipsett PA, et al. Eliminating catheter-related bloodstream infections in the intensive care unit. Crit Care Med. 2004;32:2014-2020.
27. Pittet D, Hugonnet S, Harbarth S, et al. Effectiveness of a hospital-wide programme to improve compliance with hand hygiene [published correction appears in Lancet. 2000;356(9248):2196]. Infection Control Programme. Lancet. 2000;356(9238):1307-1312.
28. Haladyna TM. Developing and Validating Multiple-Choice Test Items. Hillsdale, NJ: Lawrence Erlbaum Assoc; 1999.
eLetters Now that you’ve read the article, create or contribute to an online discussion about this topic using eLetters. Just visit www.ajcconline.org and click “Respond to This Article” in either the full-text or .pdf view of the article.
SEE ALSO To learn more about preventing ventilator-associated pneumonia, visit www.aacn.org and read the AACN Practice Alert titled “Ventilator-Associated Pneumonia (VAP)” (issued February 2004).
29. Lynn MR. Determination and quantification of content valid- ity. Nurs Res. 1996;35:382-385.
30. Thorndike RM, Cunningham GK, Thorndike RL, Hagen EP. Measurement and Evaluation in Psychology and Education. 5th ed. New York, NY: Macmillan Publishing Co; 1991.
31. Wiersma W, Jurs SG. Educational Measurement and Test- ing. 5th ed. Boston, MA: Allyn & Bacon; 1990.
32. Wiberg M. Classical test theory vs item response theory: an evaluation of the theory test in the Swedish driving license test. EM No. 50. 2004. Umeå University Department of Edu- cational Measurement Web site. http://www.umu.se /edmeas/publikationer/pdf/EM%20no%2050.pdf. Accessed April 12, 2007.
To purchase electronic or print reprints, contact The InnoVision Group, 101 Columbia, Aliso Viejo, CA 92656. Phone, (800) 809-2273 or (949) 362-2050 (ext 532); fax, (949) 362-2049; e-mail, reprints@aacn.org.
www.ajcconline.org AAJJCCCC AMERICAN JOURNAL OF CRITICAL CARE, July 2007, Volume 16, No. 4 377