ExampleofTestItemWritingandCVIcalculation3.docx

3

FACULTY DEVELOPMENT

Test Item Development, Expert Rating, and CVI Calculation

The following document is an example of the process that you will need to take in order to develop your own test questions.

Step 1: Construct Test

Purpose

The purpose of this education is to improve the medical-surgical nurses’ knowledge in early recognition of patients who develop sepsis. The course will also provide education on documentation responsibilities and notification responsibilities in the care of patients with sepsis. It will also evaluate if the learners’ knowledge led to changes in practice behaviors (documentation of EMR sepsis screening every shift and notification of primary physician within 1 hour) after course completion. The last part will be measured using retrospective chart audits.

Learning Objectives

Upon successful completion of this course, you will be able to:

· Differentiate between SIRs, severe sepsis, and septic shock

· Describe physiologic changes associated with aging and sepsis

· Recognize assessment findings that may be consistent with sepsis

· Define the hospital policy and roles and responsibilities related to care of patients with sepsis

Population

The population is a general group of medical surgical nurses.

Length of the Test

The optimum length of this test is 10 questions.

Difficulty and Discrimination Levels of Test Items

According to Oermann and Gaberson (2014), a criterion-referenced test is frequently used in clinical settings because it is used to measure set standards rather than on the actual score itself. Since this test will be used for continuing education, low level to moderate difficulty questions should be used.

Scoring Procedures to be Used

The goal is to use a separate answer sheet that will then be used to develop a computer-generated item analysis report.

Item Format

The test will be a selected response multiple choice format

Test Blueprint (there were originally 10 questions. In this example, we have listed 1)

Content

Level of Cognitive Skill

K

C

AP

AN

Total

SIRS

1

Continuum of sepsis

1

1

S/S of sepsis and elderly

1

1

Signs and symptoms of early sepsis

1

Hospital policy and management of the sepsis patient

1

1

1

1

Total

2

2

2

4

10

Sepsis Critical Thinking Questions

1) A 74 year old patient in the medical-surgical setting admitted 2 days ago for a hip fracture is confused, and incontinent. Her vitals from 2 hours ago were: BP: 123/78, HR: 89, R: 18, Temp: 97.8, and O2 Sat 95% on room air. The nurse would interpret the following as signs and symptoms of possible sepsis in older adults (Select all that apply)

a) The patient has osteoporosis

b) The patient has a HR of 88

c) The patient is confused

d) The patient has no history of incontinence

Answer: C&D Comprehension-Elderly signs and symptoms

Rationale: Having osteoporosis does not place a patient at risk of sepsis. A HR of 88 would not be indicative of potential sepsis. New onset confusion and incontinence are commonly seen as potential signs of sepsis or infection (Nasa, Juneja, & Singh, 2012).

2) The medical-surgical nurse is receiving change of shift report on a 67 year old patient admitted for cellulitis of the left arm. The CNA reports that the patient’s vital signs are: BP: 70/46 (MAP 55), HR: 150, R: 22, Temp: 102.4, and O2 sat: 96% on room air. The patient appears weak and confused. Which of the following actions would the nurse do first?

a) Call a code sepsis

b) Notify the physician

c) Call a rapid response

d) Infuse 1 liter bolus of normal saline

Answer: A Comprehension-Hospital Policy

Rationale : Per hospital policy, the patient meets criteria for a code sepsis. Calling a code sepsis will not only activate immediate attention from the physician, but will also allow rapid response by laboratory, the ICU team, the shift supervisor, and needed fluid resuscitation (Valley Hospital Medical Center Policy, 2015).

3) The bedside nurse caring for a sepsis patient with a lactate of 2.2 understands that she has achieved the 3 hour sepsis bundle when she has (Select all that apply)

a) Drawn the lactate and blood cultures

b) Infused broad spectrum antibiotics after blood cultures are drawn

c) Infused 2,040 ml of normal saline

d) Inserted a foley catheter

Answer: A, B, C Application-Hospital policy and care of the sepsis patient

Rationale: The Surviving Sepsis Campaign Bundles 3 hour recommendations include: measuring lactate level, obtaining blood cultures prior to administration of antibiotics, administration of broad spectrum antibiotics, and administration of 30 ml/kg crystalloid for hypotension or lactate ≥ 4 mmol/L (Dellinger, et. al., 2012, p. 591). Inserting a foley catheter is not part of the SSC Bundle and invasive procedures should only be performed when absolutely necessary (NC Concept-Based Learning, 2011, pg. 788).

4) The nurse is caring for a patient diagnosed with sepsis who is on a sepsis protocol. A priority for this client is to maintain adequate tissue perfusion. Which of the following findings would indicate to the nurse that the patient’s condition is deteriorating? (Select all that apply)

a) The lactate level changed from 1.7 to 3.1

b) The white blood count is 20,000

c) The patient’s urinary output has changed from 120 ml/hr to 25 ml/hr

d) The patient’s blood pressure is 102/76

Answer: A Analysis-Continuum of sepsis

Rationale: The SSC indicates that sepsis induced tissue hypo perfusion is indicated by a lactate level of ≥ 4mmol/L. An increased lactate level indicates that the patient’s perfusion is deteriorating (Dellinger, 2012, p. 587). The white blood count indicates infection, and urine output decrease indicate hypoperfusion.

5) Which of these patients is at highest risk of developing sepsis?

a) A 35-year-old who has a closed ankle fracture

b) A 10-year-old who has strep throat

c) A 68-year-old with gallstones

d) A 43-year-old who had chemotherapy yesterday

Answer: D Knowledge-Signs and symptoms of sepsis

Rationale: The patient who has undergone recent chemotherapy is immunocompromised and highly susceptible to infection (NC Concept-Based Learning, 2011, p. 90).

6) A 25 year old patient weighing 59 kg who is a direct admit with a diagnosis of pyelonephritis arrives to the medical-surgical unit. The patient’s vital signs are BP: 110/67 (MAP 87), HR: 114, R: 14, Temp: 100.7, and O2 sat 100% room air. Which of the following would indicate to the nurse that the patient’s condition has progressed to severe sepsis?

a) The patient urinated 60 ccs of cloudy urine

b) The patient has a WBC of 12,000

c) The patient has a lactate level of 2.9

d) The BP of 110/67 with a MAP of 87 and a HR of 114

Answer: C Analysis-Continuum of sepsis

Rationale: The patient is urinating > 0.5 mL/kg/hr and therefore not indicative of kidney organ damage. A WBC of 12,000 is indicative of sepsis but does not confirm severe sepsis. A lactate level >2.0 is indicative of anaerobic metabolism and is indicative of hypoperfusion and severe sepsis. The BP of 110/67 is not abnormal and a HR of 114 and these two values alone do not indicate signs of severe sepsis (Kaplan, 2015).

7) The nurse is caring for a 67 year old patient admitted with pancreatitis with complaints of pain, nausea, and vomiting. He has a history of pancreatitis, and depression. The patient’s vital signs are BP: 140/65, HR: 125, R: 18, Temp: 98.4, O2 Sat 98% on RA. Which of the following findings may indicate to the nurse that the patient meet SIRS but does not sepsis criteria?

a) The patient has a HR of 125

b) The patient has a WBC of 15,000

c) The patient has a history of depression

d) The patient has a lactic acid level of 1.7

Answer: A Analysis-SIRS

Rationale: The patient’s lactic acid level indicates that he is not experiencing anaerobic metabolism. A WBC of 15,000 and a HR of 125 could be related to infection, but dehydration could also cause these values. A history of depression has nothing to do with the analysis of SIRS/Sepsis.

8) A nurse is preceptoring a new graduate nurse caring for a patient diagnosed with suspected sepsis. The doctor ordered a sepsis protocol for this patient. What statement would indicate that the new graduate needs clarification on the 3 hour sepsis protocol bundle?

a) “I will monitor the patient’s urine output to make sure she is urinating >30 ml/hr”

b) “The lactate was drawn 1 hour ago. I will redraw a lactate level in an hour”

c) “As soon as I am done infusing this antibiotic, I will draw the blood cultures”

d) “If the patient’s MAP is <65, we will hang a 30 ml/kg normal saline bolus”

Answer: C Analysis-Hospital Policy & Management of Sepsis Patient

Rationale: Blood cultures should be drawn before administration of antibiotics. The other 3 statements meet the goals of the 3 hour bundle.

9) During documentation, a nurse reviewing her orders notices that a patient she is caring for has triggered a sepsis alert on her EMR for a HR: 120, Temp: 101, and Lactate of 3.1. The nurse demonstrates understanding of the sepsis notification when

a) The nurse contacts the patient’s primary care physician within an hour

b) The nurse initiates the sepsis protocol

c) The nurse prepares to transfer the patient to the ICU

d) The nurse completes a sepsis screening

Answer: A Knowledge-Hospital Policy

Rationale: The goal of EGDT is to improve patient outcomes by implementing therapy as soon as sepsis is recognized. (Dellinger, et. al, 2013). The hospital policy states that the nurse needs to notify the physician within 1 hour of sepsis alert.

10. The nurse demonstrates understanding of the sepsis screening tool use when

a) It is documented only on patients who are suspected of sepsis

b) It is documented on all patients at the time of admission

c) It is documented on all patients every 24 hours

d) It is documented on all patients every shift

Answer: D Knowledge-Hospital Policy

Rationale: Per policy, every patient must have a sepsis screening completed every shift. Although the majority of patients who meet sepsis criteria are identified in the ED, approximately 24% of patients develop sepsis during their admission.

Step 2: Submit your test questions to your course instructor for review

Step 3: Develop your Expert Rating Form and Distribute to Each Expert

Experts Rating Form Instructions

Rating instructions: For each item, please indicate the following:

Please rate how relevant each item is to the overall construct of early sepsis identification and care by placing a number in the first box to the right of each item.

1 = Not relevant at all

2 = Slightly relevant

3 = Moderately relevant

4= Highly relevant

Your honest feedback is appreciated and will be used to enhance the quality of this questionnaire.

Expert Rating Form

Item

Relevance Rating

A 74 year old patient in the medical-surgical setting admitted 2 days ago for a hip fracture is confused, and incontinent. Her vitals from 2 hours ago were: BP: 123/78, HR: 89, R: 18, Temp: 97.8, and O2 Sat 95% on room air. The nurse would interpret the following as signs and symptoms of possible sepsis in older adults (Select all that apply)

a) The patient has osteoporosis

b) The patient has a HR of 88

c) The patient is confused

d) The patient has no history of incontinence

The medical-surgical nurse is receiving change of shift report on a 67 year old patient admitted for cellulitis of the left arm. The CNA reports that the patient’s vital signs are: BP: 70/46 (MAP 55), HR: 150, R: 22, Temp: 102.4, and O2 sat: 96% on room air. The patient appears weak and confused. Which of the following actions would the nurse do first?

a) Call a code sepsis

b) Notify the physician

c) Call a rapid response

d) Infuse 1 liter bolus of normal saline

The bedside nurse caring for a sepsis patient with a lactate of 2.2 understands that she has achieved the 3 hour sepsis bundle when she has (Select all that apply)

a) Drawn the lactate and blood cultures

b) Infused broad spectrum antibiotics after blood cultures are drawn

c) Infused 2,040 ml of normal saline

d) Inserted a foley catheter

The nurse is caring for a patient diagnosed with sepsis who is on a sepsis protocol. A priority for this client is to maintain adequate tissue perfusion. Which of the following findings would indicate to the nurse that the patient’s condition is deteriorating? (Select all that apply)

a) The lactate level changed from 1.7 to 3.1

b) The white blood count is 20,000

c) The patient’s urinary output has changed from 120 ml/hr to 25 ml/hr

d) The patient’s blood pressure is 102

Which of these patients is at highest risk of developing sepsis?

a) A 35-year-old who has a closed ankle fracture

b) A 10-year-old who has strep throat

c) A 68-year-old with gallstones

d) A 43-year-old who had chemotherapy yesterday

A 25 year old patient weighing 59 kg who is a direct admit with a diagnosis of pyelonephritis arrives to the medical-surgical unit. The patient’s vital signs are BP: 110/67 (MAP 87), HR: 114, R: 14, Temp: 100.7, and O2 sat 100% room air. Which of the following would indicate to the nurse that the patient’s condition has progressed to severe sepsis?

a) The patient urinated 60 ccs of cloudy urine

b) The patient has a WBC of 12,000

c) The patient has a lactate level of 2.9

d) The BP of 110/67 with a MAP of 87 and a HR of 114

The nurse is caring for a 67 year old patient admitted with pancreatitis with complaints of pain, nausea, and vomiting. He has a history of pancreatitis, and depression. The patient’s vital signs are BP: 140/65, HR: 125, R: 18, Temp: 98.4, O2 Sat 98% on RA. Which of the following findings may indicate to the nurse that the patient meet SIRS but does not sepsis criteria?

a) The patient has a HR of 125

b) The patient has a WBC of 15,000

c) The patient has a history of depression

d) The patient has a lactic acid level of 1.7

A nurse is precepting a new graduate nurse caring for a patient diagnosed with suspected sepsis. The doctor ordered a sepsis protocol for this patient. What statement would indicate that the new graduate needs clarification on the 3 hour sepsis protocol bundle?

a) “I will monitor the patient’s urine output to make sure she is urinating >30 ml/hr”

b) “The lactate was drawn 1 hour ago. I will redraw a lactate level in an hour”

c) “As soon as I am done infusing this antibiotic, I will draw the blood cultures”

d) “If the patient’s MAP is <65, we will hang a 30 ml/kg normal saline bolus”

During documentation, a nurse reviewing her orders notices that a patient she is caring for has triggered a sepsis alert on her EMR for a HR: 120, Temp: 101, and Lactate of 3.1. The nurse demonstrates understanding of the sepsis notification when

a) The nurse contacts the patient’s primary care physician within an hour

b) The nurse initiates the sepsis protocol

c) The nurse prepares to transfer the patient to the ICU

d) The nurse completes a sepsis screening

The nurse demonstrates understanding of the sepsis screening tool use when

a) It is documented only on patients who are suspected of sepsis

b) It is documented on all patients at the time of admission

c) It is documented on all patients every 24 hours

d) It is documented on all patients every shift

Step 3: Calculate your Content Validity Index

Content Validity Index Table

Item

Expert 1

Expert 2

Expert 3

Mean

1

3

4

4

3.67

2

2

1

3

1.8

3

4

4

4

4.0

4

4

3

3

3.67

5

4

4

4

4.0

6

3

3

4

3.33

7

4

3

4

3.67

8

4

4

4

4.0

9

4

3

4

3.67

10

4

4

4

4.0

The procedure consists of having experts rate items on a four-point scale of relevance. Then, for each item, the item (CVI) (I-CVI) is computed as the number of experts giving a rating of 3 or 4, divided by the number of experts-the proportion in agreement about relevance.

The content validity index is calculated using the following formula:

CVR = [(E-(N/2)) / (N/2)] with E representing the number of judges who rated the item as Moderately Relevant or Highly Relevant and N being the total number of judges.

The mean total of all of the means was 3.09 indicating that all of the questions were moderately/highly relevant.

The calculation is as follows:

CVR = [(3-(3/2)) / (3/2)]

CVR = [(3-1.5) /1.5]

CVR = 1.5/1.5