week 4
dianaaranda10
TestBlueprintPresentationforNUR680.pptxTest Blueprinting: getting started!
Purpose of the presentation - This presentation is specifically designed to help faculty enhance their ability to create, edit, and revise test blueprints. The participants will learn evidence-based strategies for this process.
Objectives
Create a test blueprint that is consistent with the integrative processes NCLEX-RN® Detailed Test Plan and that requires critical thinking of students.
Identify common test item errors found in multiple-choice questions.
Write test items that demonstrate use of recommended guidelines for creation of multiple-choice questions.
Analyze test results using statistical data, especially reliability, difficulty and discrimination indexes.
Discuss ways to provide test feedback to students that promote their learning.
2
Purpose of Blueprinting/Planning
Develop purposeful, balanced tests that meet the objectives of the course and program
Provide information for curricular change
Student success!
Avoid writing inappropriate questions and having to manipulate the test after administration
Tests are reflective of the lectures and course objectives
Planning ahead allows time for necessary peer review
Advantages of Blueprinting/Planning
Avoid the common challenges made by students
“That test didn’t cover anything you lectured on”
“You asked 10 questions about something that you didn’t even cover in class”
“That question didn’t make sense”
“There were two right answers”
“ Those questions were so confusing”
“That test was unfair”
Advantages of Blueprinting/Planning
THE BIG PICTURE: the program blueprint/plan
We have to know where we are going in order to figure out how to get there!
An overview of the progression of level of difficulty across the curriculum
Determined by the Program (faculty)
Needs to be progressive and reflective of NCLEX blueprint
Should culminate with most questions being application/analysis level questions
The overall goal: Content
| Major NCLEX Categories | Client Needs | NCLEX Target |
| Safe/Effective Environment | Management of Care | 17-23% |
| Safe/Effective Environment | Safety & Infect Control | 9-15% |
| Health Promotion & Maintenance | Health Promotion & Maintenance | 6-12% |
| Psychosocial Integrity | Psychosocial Integrity | 6-12% |
| Physiological Integrity | Basic Care/Comfort | 6-12% |
| Physiological Integrity | Pharm & Parenteral Therapies | 12-18% |
| Physiological Integrity | Reduce Risk Potential | 9-15% |
| Physiological Integrity | Physiological Adaptation | 11-17% |
How do we know what the goal is for Content?
The NCLEX test plan presents the percentages of each area of content that will appear on the exam.
The overall goal: Difficulty
| Cognitive Level | NUR104 Foundations | NUR201 MS I | NUR205 MSII |
| Knowledge/Remembering | 0-5% | XXX | XXX |
| Comprehension/Understanding | 15-25% | 10-15% | 5-10% |
| Application | 50-60% | 40-50% | 40-50% |
| Analyzing | 5-10% | 30-40% | 30-40% |
How do we know what the goal is for Difficulty?
The majority of NCLEX questions are written at the application/analysis level.
See the below example of progression of difficulty:
Course Blueprint
An overview of the progression of level of difficulty across the course
Reflective of NCLEX blueprint
Progressive from point A to point B
Begin at one level of difficulty and progress to a higher level at the end of the course
Course Blueprint/PLAN
| NUR 104: Foundations of Nursing | ||
| COGNITIVE LEVEL: DIFFICULTY | TARGETS | |
| Knowledge/Remembering | 0-5% | |
| Comprehension/Understanding | 15-25% | |
| Application | 50-60% | |
| Analysis | 5-10% | |
| NCLEX CATEGORY: CONTENT | ||
| Safe , Effective Care Environment | XXX | |
| Management of Care | 17-23% | |
| Safety and Infection Control | 9-15% | |
| Health Promotion & Maintenance | 6-12% | |
| Psychological Integrity | 6-12% | |
| Physiological Integrity | XXX | |
| Basic Care & Comfort | 12-18% | |
| Pharmacological & Parental Therapies | 9-15% | |
| Reduction of Risk Potential | 11-17% | |
| Physiological Adaptation | 17-23% |
Test Blueprint
Reflective of NCLEX blueprint
Balance, balance, balance
Should reflect priorities set in lecture and in material.
If you spent 20% of your lecture talking about one topic it should be given adequate attention on the test blueprint
This is a general rule of thumb!!!
Anatomy of a Test Blueprint
Please refer to the Electronic Blueprint
FOCUS: Reference, Difficulty, and Client needs FIRST
Additional information can be added
| TEST BLUEPRINT: ASN | ||||||||
| Question # | Course Objectives | Reference | Topic | Specialty Areas (if applicable) | Bloom's Taxonomy | Nursing Process | Client Needs Categories | Question Format |
Common Ground
All test Blueprints should be a reflection of the Program and Course objectives
You should be able to draw a line from every test item to the Mission and Philosophy of the Program
If asked “how do you know a student has met the objectives” you should be able to direct that person to test questions that demonstrate that the objective has indeed been met
Test Blueprint
Lecture/
teaching
Test Item
Course Blueprint
Course Objectives
Program Blueprint
Program Objectives
Mission and Philosophy
Test Objectives
The anatomy of the Test Blueprint
Some sections are essential and some are supportive in nature
As we go through the parts, we will discuss the relevance and importance of each section
Question numbering
Two approaches
Randomization of questions
Like NCLEX? Although NCLEX may randomize content, CAT uses increasing levels of difficulty
Sometimes only alternative in computer based testing
Intentionally organized questions escalating in difficulty
References
This is essential for your records!
Book and pages recorded
A well written question should have more than one resource
This information is for YOU more than for the student
Topic
This is supportive documentation
What is the content
Do not just write “dialysis” but rather be more specific “Side effects of hemodialysis”
Pretend you are a third party reading this because that is exactly what it feels like when you look at it later
Avoid using shorthand unless it is widely accepted and understood OR you provide a key.
Specialty Areas
This is supportive documentation
Record these when appropriate mainly because this information helps to frame the test question
Not always required but helpful
| Specialty Areas | ||
| Critical Care | 0 | #DIV/0! |
| Fundamentals | 0 | #DIV/0! |
| Geriatrics | 0 | #DIV/0! |
| Maternity | 0 | #DIV/0! |
| Medical Surgical | 0 | #DIV/0! |
| Pathophysiology | 0 | #DIV/0! |
| Pediatrics | 0 | #DIV/0! |
| Professional Issues | 0 | #DIV/0! |
| Psychiatric/Mental Hlth | 0 | #DIV/0! |
| 0 |
Bloom’s Taxonomy: Difficulty
Bloom’s Taxonomy has been revised recently to demonstrate “how the student thinks” rather than “what they know”
Determining difficulty is ESSENTIAL!
Knowledge/
Remembering
Comprehending/ Understanding
Applying
Analyzing
Creating
Evaluating
What Level is this?
Which of the following blood levels represents a therapeutic range for lithium?
a. 0.1-0.5 mEq/L
b. 0.4-0.8 mEq/L
c. 0.5-1.5 mEq/L
d. 1.0-2.5 mEq/L
This is a basic knowledge/Remembering level question. It requires memorization ONLY! C is the answer
20
Which sign or symptom is the nurse likely to assess if the client’s lithium level is 0.2mEq/L?
a. flight of ideas
b. C/O severe constipation
c. ataxia
d. C/O extreme lethargy
What Level is this?
This requires that the student understand or comprehend and interpret some data. A is the answer as 0.2 is sub therapeutics.
21
The patient is ordered lithium carbonate 450mg twice a day PO. The nurse should hold the lithium carbonate if which of the following levels is present?
a. lithium blood level – 0.3 mEq/L
b. sodium level – 135 mEq/L
c. lithium blood level – 1.8 mEq/L
d. sodium level –145 mEq/L
What Level is this?
This requires that the student apply use or demonstrate information and have rationale’s for their actions. The sodium levels are both within normal range. 0.3 is sub-therapeutic so the only choice is C.
22
You are assigned to care for a group of clients. Which client should you assess first?
a. A 19-year-old with an infection whose first dose of IV antibiotics is infusing and is complaining of throat tightness.
b. A 25-year-old who had an appendectomy 2 hours ago and is complaining of pain.
c. A 66-year-old with anemia who just received a unit of packed red blood cells an hour ago and is complaining of fatigue.
d. A 78-year-old with emphysema who has a pulse ox reading of 92% and respirations are last recorded at 22 per minute and shallow.
What Level is this?
Requires that the student break down information and look for relationships. To analyze well the student must compare and contrast. In this case comparing ages. Medical situation and current complaint (symptom) and objective data (signs). Youngest pt is the highest priority based on the ABCs.
23
A 77-year-old, newly admitted inpatient is prescribed Ativan (lorazepam) 0.5 mg p.o. every night at bedtime. The nurse recognizes that this medication places the patient at risk for
A. injury
B. constipation
C. fluid volume deficient
D. impaired gas exchange
What Level is this?
Requires that the student puts together data in a way that solves a problem or predicts an outcome. This question requires that the student make connections between Lorazepam-CNS-potential loss of coordination. They also should consider age and unfamiliar surroundings as contributing factors.
24
The nurse determines that the client’s teaching about dietary restrictions necessary for Coumadin(Warfarin) therapy has been successful when the client states
a. ”I will take my Coumadin in the morning with water”.
b. “I will limit my intake of red meats and eggs”.
c. “I will try to drink more fresh juices with pulp”.
d. “I will give up my nightly spinach salad”.
What Level is this?
Requires the drawing of conclusions and judgment making. Here the student has to determine what if said would be correct. And it’s D-avoiding foods high in vitamin K
25
Nursing Process
Assessment
Information gathering
Objective
Analysis
Identification of the nursing problem/patient needs
Planning
Goal NOT attained: reassess and reevaluate
Implementation
Evaluation
Goal attained: continue to monitor
Carrying out nursing actions/ plan of care
Developing nursing actions/ plan of care
Subjective
Client Needs: Content
Safe/Effective Environment
Management of Care 17%-23%
Safety & Infect Control 9%-15%
Health Promotion & Maintenance 6%-12%
Psychosocial Integrity 6%-12%
Client Needs: Content cont.
Physiological Integrity
Basic Care/Comfort 6%-12%
Pharm & Parenteral Therapies 12-18%
Reduce Risk Potential 9%-15%
Physiological Adaptation 11%-17%
***Specific information regarding the information held in each heading/subheading can be found at www.ncsbn.org
Question Format
Multiple Choice: majority of test questions will be MC
Alternate Format: Multiple answer and ranking/ordering easiest to use on formal tests. Due to limitations of technology other formats are best placed in learning activities
Innovative idea: use sparingly
Mastery: Items you expect 100% accuracy (controversial)
Post Analysis on Scores: what can it mean?
Overall statistics for reliability of the entire test
Statistics on individual answers and distractors
Possible results
Clear or Unclear stem
Clear of Unclear answer
Clear or unclear distractors
May decide to replace or rewrite any one or combination of the above.
***Poor statistics DO NOT always mean you should throw out a question
Test Analysis: Difficulty
| Blooms Taxonomy | ||
| Knowledge/Remembering | 1 | 13% |
| Comprehension/Understanding | 1 | 13% |
| Application | 2 | 25% |
| Analyzing | 1 | 13% |
| Evaluation | 2 | 25% |
| Creating | 1 | 13% |
| 8 |
Number of items in that category on test
Percentage of items on this test in that category
Test Analysis
BLOOMS
Knowledge/Remembering Comprehension/Understanding Application Analyzing Evaluation Creating 1 1 2 1 2 1Test Analysis: Content
| NCLEX Target | ||||
| Major NCLEX Categories | Client Needs | # | ||
| Safe/Effective Environment | Management of Care | 10 | 20% | 17-23% |
| Safe/Effective Environment | Safety & Infect Control | 5 | 10% | 9-15% |
| Health Promotion & Maintenance | Health Promotion & Maintenance | 3 | 6% | 6-12% |
| Psychosocial Integrity | Psychosocial Integrity | 6 | 12% | 6-12% |
| Physiological Integrity | Basic Care/Comfort | 4 | 8% | 6-12% |
| Physiological Integrity | Pharm & Parenteral Therapies | 8 | 16% | 12-18% |
| Physiological Integrity | Reduce Risk Potential | 6 | 12% | 9-15% |
| Physiological Integrity | Physiological Adaptation | 8 | 16% | 11-17% |
| 50 | 100% |
Number of Items on this test in each category
% on this test
Target %
Curricular Change
Make sure that what you are testing on is meeting the needs of your students in preparing them for NCLEX
Compare student performance on critical areas to determine the need for adjustments to
Content delivery
Testing methods
Cognitive level of questions
References
National Council of State Boards of Nursing (2013). 2013 NCLEX-RN® Detailed Test Plan. Chicago: Author.
National Council of State Boards of Nursing Web site: www.ncsbn.org
Silvestri, L.A (2011). Presentation at NCLEX Regional Updates Conference, hosted by Florida Department of Health-Board of Nursing.
Bristol, T. (2012) Presentation at N-OADN Conference, New Orleans, LA.
Caputi, L. & Engelman, L. (2008). Teaching Nursing: The Art and Science. College of DuPage Press: Illinois.
