CHAPTER I
INTRODUCTION
Registered nurses (RNs) make numerous decisions in every hour of patient care including
weighing physiological responses to treatment, prioritizing care among multiple acute patient
cases, and providing education to families to make informed decisions for their children or other
dependent loved ones. Each of these decisions requires the nurse to consider various contextual
factors to provide effective care. According to Tanner (2006), when a nurse is making multiple,
complex decisions quickly while having to decide between competing priorities, it is a matter of
clinical judgment.
Effective clinical judgment is a vital characteristic of nurses’ thinking and actions while
providing patient care (Betts et al., 2019; Dickison et al., 2019). Clinical judgment is a
combination of critical thinking and decision-making that considers nursing knowledge and the
context of the situation, leading to appropriate action and follow-up evaluation (Betts et al.,
2019; Tanner, 2006). Effective clinical judgment has several components including recognizing
important details, interpreting what is being observed, and responding appropriately (Tanner,
2006). Tanner’s (2006) clinical judgment model has been used to illuminate the cognitions
necessary to display clinical judgment including noticing, interpreting, responding, and reflecting
(Bussard, 2015; Gubrud-Howe, 2008; Hines & Wood, 2016; Rose, 2012).
Lasater (2007) built upon Tanner’s (2006) work by developing the Lasater’s clinical
judgment rubric (LCJR) to measure clinical judgment among prelicensure nursing students
during simulation. The LCJR has been validated numerous times as a measure of clinical
judgment among nursing students and nurses in practice (Adamson et al., 2012; Call, 2017;
Chmil, 2014; McCormick, 2014; Reid, 2016). The LCJR explicated Tanner’s four cognitions by
describing them in 11 dimensions of three to four expected behaviors for each cognition. These
behaviors are described in rubric format and ordered as beginning, developing, accomplished,
and exemplary (Lasater, 2007). The rubric can be used quantitatively by assigning values to each
of the levels from one for beginning to four for exemplary. This results in the ability to provide a
total numerical rating to the students’ simulation performances that ranges from 11 to 44
(Bussard, 2018; Fawaz & Hamdan-Mansour, 2016; Hines & Wood, 2016).
Nurse educators have been conducting research to better understand how to help
prelicensure students develop their clinical judgment including studies about how to measure
clinical judgment and how to teach this skill in various ways. Amid this research, objective
quantitative measures of clinical judgment connected to specific teaching strategies for the
inperson clinical setting were limited. One teaching strategy that might be beneficial is direct
instruction of the dimensions of clinical judgment by highlighting the range of behaviors that
demonstrate clinical judgment development during in-person clinical.
Direct instruction as a pedagogical method has been used for several decades. Seminal
work was done in the 1960s by Engelmann and colleagues (Barbash, 2021). The assumption
underlying direct instruction is all people can learn when instruction is well-designed. This
design is based on specific instruction students have already mastered. New material is taught in
a clear, straightforward way including carefully chosen examples and sequenced to learn in a
stepwise fashion (Stockard et al., 2018). Foo et al. (2017) found a significant increase in clinical
judgment scores among 39 Malaysian registered nurses in two district hospitals after a direct
instruction educational intervention from preintervention to postintervention (M = 24.15 [SD =
6.92], M = 47.38 [SD = 7.20], p < .05, respectively) compared to the control group of 41 nurses
from pretest to posttest (M = 23.80 [SD = 5.77], M = 26.50 [SD = 6.53]).
The Tanner (2006) clinical judgment model, the theoretical framework on which this
research was based, and the LCJR (Lasater, 2007), fit the description of the well-designed
instruction underpinning the direct instruction method. Clinical judgment is best developed based
on nurses’ experiences and knowledge from previous instruction (Tanner, 2006). The LCJR,
therefore, could be used to guide the prelicensure nursing student toward improved clinical
judgment through clear, stepwise descriptions of the dimensions of clinical judgment (Lasater,
2011).
Theoretical Framework
Tanner’s clinical judgment model (2006) was used in this study as the theoretical
framework. Her framework describes four cognitions of clinical judgment including noticing,
interpreting, responding, and reflecting. Noticing starts with what the nurse brings to the situation
in expectations borne of theoretical knowledge and previous experiences such as the nurse’s
background with the current patient and similar patients. Interpreting is the next step and is
approached in different ways depending on the nurse’s noticing. If the nurse has recognized a
pattern, the nurse would likely move directly into responding by taking action the nurse thinks is
best. If the nurse does not recognize a pattern, the nurse would likely form a hypothesis and then
respond with actions that would confirm or disconfirm the hypothesis. According to Tanner, after
responding, the nurse would then move on to the reflecting step. Reflection happens at two
levels. The first level is reflection-in-action, which might be largely subconscious as the nurse
evaluates the patient to see that the action the nurse took was appropriate and effective. The other
level of reflection, reflection-on-action, might also be subconscious if the nurse has taken
appropriate action and this experience has now been folded into the patterns of clinical situations
in the nurse’s mind. This second level of reflection would be more intentional if there has been a
breakdown in clinical judgment and some sort of untoward result has occurred. Reflection-
onaction also occurs as the nurse considers how the nurse’s actions connect to the clinical
outcomes, and whether the result was positive or negative. Clinical judgment is the result of this
decision-making process and experience continues to develop the skill further.
Tanner’s (2006) model has been used to illustrate the development of clinical judgment in
various settings (Bussard, 2015; Gubrud-Howe, 2008; Hines & Wood, 2016; Rose, 2012). It is
often used in tandem with the LCJR to measure the development of clinical judgment among
nursing students (Adamson et al., 2012; Call, 2017; Chmil, 2014; McCormick, 2014; Reid,
2016). The synergy of this pairing is the theoretical framework is clearly explained in Tanner’s
work while the measurement of clinical judgment is clearly described by Lasater (2007).
The LCJR (Lasater, 2007) encompasses measures of effectiveness for each part of
Tanner’s (2006) model. For example, the LCJR begins with “Effective Noticing,” which is
divided into categories of “Focused observation,” “Recognizing deviations from expected
patterns” and “Information seeking” (Lasater, 2007, p. 500). Descriptions of beginner to
exemplary are then listed. The LCJR continues with dimensions for “Interpreting,”
“Responding,” and “Reflecting” containing corresponding leveled descriptions of effectiveness
in each dimension (Lasater, 2007, p. 500).
Statement of the Problem
Nursing students are graduating from nursing school, taking the National Council
Licensure Examination for Registered Nurses, and starting their first jobs without basic
competency in clinical judgment. Kavanagh and Sharpnack (2021) reported that the percentage
of new hires demonstrating entry-level competence in patient safety has been declining. From
2016-2020, only 14% of new hires demonstrated this competency (n = 5,000) and in 2020, the
percentage was only 9% (n = 1,222; sites = 200). Additionally, the National Council of State
Boards of Nursing’s (NCSBN, 2020) most recent practice analysis revealed that (a) 50% of new
graduate nurses were involved in practice errors, (b) only 20% of employers rated their new
graduate nurse hires as satisfactory or better in the provision of safe patient care, and (c) 65% of
the errors were directly related to inadequate clinical judgment. Inadequate clinical judgment
among new nurse graduates places patient safety at great risk.
Statement of the Purpose
The purpose of this study was to test whether direct instruction for prelicensure nursing
students about behaviors that demonstrate clinical judgment and encouragement to intentionally
practice these behaviors during in-person clinical promoted the development of clinical
judgment. The LCJR (Lasater, 2007) scores measured during simulation were compared between
prelicensure students who received direct instruction (n = 12) and the control group of those who
did not (n = 18).
Research Question and Hypotheses
Research Question
Q1 Among prelicensure nursing students, how did direct instruction about how the
Lasater’s clinical judgment rubric (LCJR) could be applied to their in-person
clinical experiences compared with no direct instruction affect their clinical
judgment ratings from the end of first semester to the end of second semester?
Hypotheses
H01 There will be no significant difference in the clinical judgment ratings on the
LCJR from one semester to the next for prelicensure students who have received
direct instruction as compared to students who have not received direct
instruction.
HA There will be a significantly greater increase in clinical judgment ratings on the
LCJR from one semester to the next for prelicensure students who have received
direct instruction about how the LCJR can be applied to their in-person clinical
experiences as compared to the ratings for those students who have not received
direct instruction.
Significance of the Study
Clinical judgment is the first of the top 10 high priority skills required of a nurse by the
NCSBN (2018). The NCSBN performs a practice analysis of registered nurses every three years
to identify the knowledge, skills, and abilities needed for newly graduated registered nurses. In
the most recent analysis, the NCSBN (2020) found that patients had more complex needs and
tended to be of higher acuity than in the last analysis. They also found that nurses were being
relied upon with greater frequency for more complex decision-making than before (NCSBN,
2020). Research indicated that students continue to emerge from nursing school without the level
of clinical judgment necessary to begin safe practice as new graduate nurses (Billings, 2019;
Dickison et al., 2019; Fitzpatrick, 2017).
Definition of Terms
Clinical Judgment. “An interpretation or conclusion about a patient’s needs, concerns, or health
problems, and/or the decision to take action (or not), use or modify standard approaches,
or improvise new ones as deemed appropriate by the patient’s response” (Tanner, 2006, p.
204).
Direct Instruction. Instruction that is designed to be based on previously mastered material, laid
out in a straightforward manner using carefully selected examples, and approached in a
stepwise fashion (Stockard et al., 2018).
Summary
Clinical judgment development among prelicensure nursing students needs to be
addressed intentionally because patient safety is at risk. Pedagogical methods that promote
clinical judgment development were limited in the nursing literature. Tanner’s (2006) clinical
judgment model provided the theoretical framework and the LCJR (Lasater, 2007) provided
straightforward, stepwise descriptions needed to guide direct instruction to prelicensure nursing
students for this study. The research was hypothesized to result in evidence for the use of direct
instruction using the LCJR for prelicensure nursing students’ development of clinical judgment.
This gap in the literature is made clear in the following literature review.
CHAPTER II
REVIEW OF THE LITERATURE
Chapter I presented an introduction and the background for this study. An overview of the
theoretical perspective that served as the basis for the study was introduced, the research question
and hypotheses were presented, and key terms were defined. The purpose of this study was to
test whether direct instruction for prelicensure nursing students about behaviors that demonstrate
clinical judgment and encouragement to intentionally practice these behaviors during in-person
clinical promoted the development of clinical judgment. In this chapter, a detailed literature
review is presented that supports the need for this study.
Literature Review
Registered nurses have the responsibility to strive for patient safety as they provide
competent, compassionate care. Competency in patient safety is predicated upon nurses
possessing clinical judgment. However, Kavanagh and Sharpnack (2021) reported that clinical
judgment has been declining among new graduate nurses with the most recent report showing
that in 2020, the percentage of hew hires demonstrating entry-level competence was only 9% (n
= 1,222; sites = 200).
Nurse educators have been working to confront this problem with studies about how to
measure clinical judgment and how to teach this skill in various ways. Amid this research,
objective quantitative measures of clinical judgment connected to specific teaching strategies for
the in-person clinical setting were limited. This research project sought to fill this gap in nursing
knowledge by investigating the following research question:
Q1 Among prelicensure nursing students, how does direct instruction about how the
Lasater’s clinical judgment rubric (LCJR) can be applied to their in-person
clinical experiences compared with no direct instruction affect their clinical
judgment ratings from the end of first semester to the end of second semester?
This literature review was undertaken using the following MeSH subject headings used
as search terms in PubMed: “students, nursing,” “clinical,” and “decision-making” since “clinical
judgment” had no comparable subject headings in PubMed. After filtering for the last 10 years
and English language, 728 articles were listed. Using CINAHL Plus with Full Text and ProQuest
Dissertations & Theses Global, “nurs*,” clinical judgment,” and “student” were searched in
combination. This search originally produced 268 articles when limited to English and the last 10
years. Articles were then vetted by title and abstract to include those addressing educational
methods for clinical judgment development or clinical judgment measurement tools used among
pre-licensure nursing students. From this process, 114 articles were accessed and reviewed. By
excluding those not focused on clinical judgment or its development, a final sample of 67 articles
was a basis for this literature review.
The final sample of articles included four seminal articles that described important
definitions and model development from 2000 (Scheffer & Rubenfeld), 2006 (Tanner), 2007
(Lasater), and 2011 (Lasater). All remaining articles (n = 63) were published in the last decade.
The sample included three systematic or integrative literature reviews (Cappelletti et al., 2014;
Carvalhoa et al., 2017; Fisher & King, 2013), seven teaching strategy development and
descriptions (Billings, 2019; Cason & Reibel, 2021; Gonzalez, 2018; Gonzalez et al., 2021;
Jessee, 2018; Timbrell, 2017; Want, 2022), one analysis of multiple case studies (Nielsen, 2016),
and five qualitative studies (Barner et al., 2023; Glynn, 2012; Hunter & Arthur, 2016; Pardue et
al., 2023; Smith, 2021). The 46 remaining articles reported quantitative studies using descriptive
and experimental designs. Most of the research was of high quality as evidenced by moderate to
large sample sizes, use of established models and instruments, and measurements by raters as
more common than by self-assessment or reports.
Clinical Judgment Defined
Several related terms were often used interchangeably including critical thinking, clinical
reasoning, and clinical judgment. When Tanner (2006) wrote her seminal synthesis from over
200 studies about clinical judgment that served as the foundation for her clinical judgment
model, she differentiated these terms, making clear that critical thinking and clinical reasoning
are the basis for clinical judgment. Critical thinking refers to the use of cognitive skills such as
“analyzing, applying standards, discriminating, information seeking, logical reasoning,
predicting, and transforming knowledge” (Scheffer & Rubenfeld, 2000, p. 352). Clinical
reasoning is when a nurse applies critical thinking to clinical situations. When the nurse decides
about the action to take or not take, the nurse has exercised clinical judgment (Tanner, 2006).
Tanner defined clinical judgment as “an interpretation or conclusion about a patient’s needs,
concerns, or health problems, and/or the decision to take action (or not), use or modify standard
approaches, or improvise new ones as deemed appropriate by the patient’s response” (p. 204).
The field of clinical judgment has continued to develop since Tanner’s (2006) work. For
example, recently, the NCSBN (2019) weighed in concerning clinical judgment development and
offered a definition:
The observed outcome of critical thinking and decision making. It is an iterative process
using nursing knowledge to observe and assess presenting situations, identify a
prioritized client concern and generate the best possible evidence-based solutions in order
to deliver safe client care. (p. 1)
The definitions for clinical judgment are similar in that clinical judgment is based on
certain thinking processes that culminate in a decision and action followed by evaluation of
outcomes and re-thinking if necessary. The current study was based on Tanner’s (2006) clinical
judgment model. Also, the instrument for measuring clinical judgment used in this study was
based on the LCJR (Lasater, 2007). Therefore, for clarity and ease of use, Tanner’s definition of
clinical judgment was used.
Educational Methods to Help Students Develop Clinical Judgment
Tanner (2006) based her clinical judgment model on five conclusions she made from her
review of the literature. First, clinical judgment is based more on the nurses’ background and
experience than on the objective data of a particular situation. Second, clinical judgments are
partially based on the nurses’ knowledge of their patients’ trends of responses. Third, clinical
judgments are influenced by the context of the situation including the nursing culture of the
healthcare institution. Fourth, individual nurses use various patterns of reasoning to arrive at
clinical judgments. And fifth, improvement in clinical judgment is affected by reflection, often
resulting from incidences of ineffective clinical judgment.
In reviewing the updated literature about clinical judgment from 2006 to 2014,
Cappelletti et al. (2014) concluded there was a sixth influence on clinical judgment: educational
strategies to which the nurse was exposed. Cappelletti and colleagues included 23 research
studies about educational methods to increase clinical judgment development and found no
conclusive evidence for a particular pedagogical strategy. The current literature review here gives
evidence that Cappelletti et al.’s conclusion continues to be true.
It has become increasingly clear in the last several years, as evidenced by Kavanagh and
Sharpnack (2021), that clinical judgment for patient safety competence is declining. Therefore,
the NCSBN (2019) developed a model in which new item types could be written designed to
assess clinical judgment for inclusion in the National Council Licensure Examination, which
began in 2023. In the arena of educational methods, Tanner’s (2006) model and its elements of
noticing, interpreting, responding, and reflecting have provided a framework for many studies of
clinical judgment development among nursing students. These methods have guided classroom,
clinical, and simulation experiences. Each of these types of educational experiences is reviewed
in the following sections.
Classroom Experiences
Classroom-based methods for clinical judgment development provide opportunities for
students to practice thinking habits before applying them to in-person clinical situations.
Findings from a systematic review of the literature (n = 6) found that problem-based learning
increased critical thinking scores (Carvalhoa et al., 2017). Although critical thinking is not
synonymous with clinical judgment, it is foundational to effective clinical judgment (Tanner,
2006).
Further support for the notion that problem-based learning would strengthen clinical
judgment was found in a study by Alfayoumi (2019) who implemented a concept-based
curriculum and concept mapping activities for second year Iranian nursing students (n = 40).
Clinical judgment significantly increased during in-person clinical as measured by self-ratings (p
= .005) and by clinical instructors’ observations (p = .032). Concept mapping activities were also
added for two other groups of students, yielding positive results in clinical judgment (Kaddoura
et al., 2016; Kinyon et al., 2021). In the research by Kaddoura et al. (2016), 106 junior
prelicensure students prepared 12 concept maps during their medical-surgical course, receiving
feedback from clinical instructors. At the end of the course, they self-evaluated using the clinical
judgement self-evaluation tool based on the LCJR (Lasater, 2007) and over 80% of students
rated themselves at the excellent level in each of the four cognitions of clinical judgment. After
Kinyon et al. (2021) re-focused the curriculum in a health assessment course and included
regular concept mapping, the number of students to meet or exceed standards related to skills
performance and application of clinical judgment after assessments increased from 67% to
100%.
Exploration for classroom-based clinical judgment development led one nursing faculty
member to bring a recording of a high-fidelity simulation into her course (Klenke-Borgmann,
2020). Student volunteers were recorded performing the simulation and then the recording was
viewed during the next class session. Students debriefed in small groups followed by a facilitated
debrief with the faculty member as a whole group following the LCJR (Klenke-Borgmann,
2020).
Other methods being used in a classroom setting have included web-based situational
learning, structured reflective writing, and instructional storytelling. The web-based situational
learning was contrasted with traditional teaching strategies among nursing students (n = 101) and
the traditionally taught had significantly higher internship scores and clinical evaluations based
on the LCJR (Lasater, 2007) than their web-based counterparts (p < .05; H.-M. Chen et al.,
2021). The authors suggested that although participants in the traditional teaching strategies
group had greater increases in measures of clinical judgment, the teaching strategy only
accounted for 28% of the variance. Therefore, the authors concluded that clinical judgment
development was based on more factors than teaching strategy alone. In another study of 15
traditional Bachelor of Science in Nursing (BSN) students, structured reflective writing based on
Tanner’s clinical judgment model revealed students perceived their clinical judgment and clinical
confidence improved through the reflection activities (Glynn, 2012). Lastly, Timbrell (2017)
developed a method of telling a short clinical story and then asking questions to students using the
concepts from Tanner’s (2006) clinical judgment model and found it yielded greater depth in
clinical judgment discussions.
One important conclusion made by a few authors was that instruction in clinical
judgment in nursing programs yielded the greatest improvement when a clinical judgment
model was selected and followed throughout the program of study. This was important for
consistency of language and instruction from course to course. When all faculty and clinical
instructors were trained in use of the model, this consistency could be achieved (Jessee et al.,
2023; Nielsen et al.,
2023; Sommer et al., 2021).
Clinical Experiences
Nursing programs have a variety of options available for providing undergraduate
students with face-to-face clinical experiences and simulated clinical experiences. Each of these
clinical experiences requires a different skill set of teaching strategies; however, all are aimed at
increasing clinical competence and facilitating the development of clinical judgment so graduates
of nursing programs are ready to transition to practice. Evidence supporting the use of strategies
to increase clinical judgement is detailed in the sections that follow as organized by the type of
clinical experience.
Virtual Simulations
Screen-based individual virtual simulations (Vsims) have been used for teaching clinical
judgment. Fogg et al. (2020) found clinical judgment increased as measured by the number of
attempts that it took students to meet competency levels in Vsims (p = .000). Further,
selfassessments with the LCJR showed significant improvement in scores (p = .000) among BSN
senior level students (n = 234). In another study based on Vsims, the senior level BSN students
(n = 42) self-assessed their clinical judgment skills using the Skalsky Clinical Judgment Scale.
Findings indicated a significant increase in clinical judgment from pre-intervention to
postintervention (M = 32.17 [SD = 4.178], M = 34.12 [SD = 4.992], p < .007, respectively)
(Kool, 2022).
Pardue et al. (2023) performed a qualitative research study with 19 nursing students after
a Vsim experience. Four open-ended questions were asked to the focus groups to illicit
discussion about their thinking processes during the simulation. Researchers analyzed the
responses and found evidence of the four cognitions of clinical judgment described within the
LCJR for Vsim. They concluded that more research along these lines could provide further
support for the use of Vsim for the development of clinical judgment in prelicensure nursing
students.
A systematic review of 14 research studies comparing Vsim with high fidelity simulation
(HFS) on clinical judgment yielded mixed results. This gives evidence that more research could
be done to determine which simulation method is more efficient for clinical judgment
development (Martin & Tyndall, 2022). As an aside, these simulation methods vary widely in
their cost with Vsim generally being much lower. Therefore, it would be important to perform
further research in this area from a cost-effectiveness standpoint also.
Gaming
Gaming is being explored as a possible medium for increasing clinical judgment among
prelicensure nursing students. Five students were in an exploratory study in which they
participated in an escape room program designed to guide them to problem solve their way from
room to room to care for a case-based patient. Each was evaluated for clinical judgment by
faculty observers using the LCJR. Their findings showed clinical judgment ratings from 18.33 to
32.33 out of 44 showing that clinical judgment behaviors could be assessed during this activity.
The authors suggested that more quantitative research studies could provide additional evidence
for clinical judgment development through this medium (Barner et al., 2023).
High Fidelity Simulation
High fidelity simulation (HFS) has been studied as a way to provide students with
opportunities to practice clinical judgment skills in a safe environment. An integrative review by
Fisher and King (2013) found 8 of 18 studies correlated HFS with increased critical thinking,
clinical reasoning, and/or clinical judgment skills. Fawaz and Hamdan-Mansour (2016)
corroborated this finding with their study of first-year nursing students (n = 56) in Lebanon. In
this study, the intervention group (IG) experienced HFS of heart failure content while the control
group (CG) experienced classroom content for heart failure. All the participants then attended an
in-person clinical followed by an end-of-semester HFS during which they were evaluated with
the LCJR. A significant increase in clinical judgment was found in the HFS group compared to
the CG (t = 5.23, p < .001, M = 29.5 [SD = 5.4], M = 22.1 [SD = 5.7], respectively).
Klenke-Borgmann et al. (2021) studied the effect of multiple exposures to HFS for
clinical judgment development. They found increased clinical judgment measured by the LCJR
among third-year nursing students (n = 45) in their pathophysiology course after three
simulations (p < .001). Coram (2016) reported on another opportunity to increase clinical
judgment during HFS by introducing expert role modeling to the IG before they participated in
the HFS. Among the junior level BSN students (n = 43), evaluations on the LCJR by faculty
showed significantly greater scores in clinical judgment for the IG over their CG counterparts (t
= -5.69, p = .000, M = 29.32 [SD = 3.65], M = 21.45 [SD = 5.31], respectively).
Ayed et al. (2022) found increased clinical judgment among baccalaureate nursing
students (n = 150) in a pediatric nursing course at an Arab American University. Participants
were randomly assigned to HFS followed by two days of pediatric inpatient care (n = 75) or
traditional clinical groups for lecture and two days in pediatric inpatient care (n = 75). Both
groups were rated using the LCJR and the HFS group score significantly higher (t = 7.20, p
< .001, M = 31.37, [SD = 11.18], M = 18.03 [SD = 11.51], respectively).
The importance of structured pre-briefing was emphasized by two groups (Kim et al.,
2019; Page-Cutrara & Turk, 2017). In each study, 76 senior nursing students participated in HFS
learning experiences. The IGs (n = 40, n = 42, respectively) included standard pre-briefing as
well as a specified structured pre-briefing inserted before they experienced the HFS scenario.
The CGs (n = 36, n = 34, respectively) were guided through standard pre-briefing followed by
the HFS scenario and debriefing facilitated by trained nursing faculty. Kim’s study’s structured
pre-briefing included concept mapping and problem-solving exercises and Page-Cutrara and
Turk’s structured pre-briefing included a worksheet and facilitated reflection based on Tanner’s
clinical judgment model. The IGs in both studies demonstrated significantly greater increase in
their clinical judgment scores. Kim measured via the LCJR Korean version
(F = 123.781, p < 0.001, M = 39.5, M = 29.9). Page-Cutrara and Turk measured via the CCEI
clinical judgment subscale (U = 128.5, Z = −6.2, p < 0.001).
Specific methods in debriefing following HFS have also been associated with increased
clinical judgment (Høegh-Larsen et al., 2023; Want, 2022). Høegh-Larsen et al. (2023) found that
using the Promoting Excellence and Reflective Learning in Simulation (PEARLS) model for
debriefing yielded statistically significant increases in clinical judgment using the LCJR for
selfevaluation compared to standard debriefing. And Want (2022) described the use of scripted
debriefing based on the NCSBN clinical judgment model as a method for increasing clinical
judgment among prelicensure nursing students.
Standardized Patients
Standardized patient actors (SPs) have been used effectively for simulation with the aim
of clinical judgment development. Hambach et al. (2023) demonstrated this among sophomore
nursing students (n = 47) using a single group repeated measures study in which participants
experienced three simulations with SPs and showed significantly increased clinical judgment
scores (F [2, 92] = 11.41, p < .001, n2p = 0.20) using the LCJR. Andrea and Kotowski (2017)
used SPs for learning health history communication skills among first semester BSN health
assessment students (n = 80). Self-evaluation by these students with the LCJR showed significant
increases in scores ( F [2, 76] = 19.15, p < 0.01, M = 35.01, M = 33.16) from baseline through
two time points.
In-Person (Traditional) Clinical
Hines and Wood (2016) were interested in exploring whether debriefing could be used
during clinical experiences to increase students’ clinical judgment skills. They developed a
scripted debrief based on Tanner’s clinical judgment model and provided this after each of six
inperson clinical days and two simulation experiences. There were three different evaluations
performed. In the first, clinical instructors evaluated students (n = 53) in the LCJR’s reflection
section after each debrief and found a significant increase in clinical judgment (p = .002). For the
second evaluation, students in this study used the LCJR after the second and fifth of the six
clinical experiences and after each of the two simulations. Findings showed clinical judgment
increased for three of the four categories of the LCJR (p = .000) and for reflecting (p = .003).
Finally for the third evaluation, independent raters watched recordings of the two simulation
experiences. Dependent t-test analysis indicated that there were significant increases in three of
the four categories of the LCJR: noticing (t = 5.109, p = .000), interpreting (t = 5.463, p = .000),
and reflecting (t = 6.058, p = .000). In the category of responding there was a significant
decrease (t = 15.044, p = .000). The authors suggested that this decrease could have been
expected because the two scenarios for which the students were evaluated included elements for
which the students did not have sufficient experience to be proficient. The authors further
pointed out that the student learning occurred in the reflecting that happened during the
debriefing. Therefore, three cognitions of clinical judgment were improved in these students and
further research is needed to study whether increased experience in the content areas of these
scenarios would increase the cognition of responding as well.
In related research, the time spent in a single clinical site was studied for nursing students
(n = 150) at a university in China. One group attended clinical for three months in consistent
specialized units and the other group attended only one month in a standard hospital unit. The
consistent specialized unit group significantly improved in clinical decision making (t = 7.677, p
< .05) and on 3 subscales of social problem solving: positive problem orientation (t = - 0.709, p
< .05), negative problem orientation (t = 0.439, p < .05), and rational problem solving (t = 0.299,
p < .05). These authors concluded that these areas of clinical judgment were more improved
among students who attended a single clinical setting at higher specialization than those who
attended their traditional clinical setting (L. Chen et al., 2021).
Authors have also described methods by which nursing students can be guided in their
thinking during their in-person clinical experiences. Eisenmann (2021) reported about an
innovative concept mapping technique students can be taught to use to organize and prioritize
clinical data. Her map leads the student to write their notes in blocks that graphically illustrate
key connections between data with the goal that increased clinical judgment develops.
Measuring Clinical Judgment
An important consideration for studies of clinical judgment is the method used to
measure the concept. Some authors have studied students’ self-assessed or self-reported
perceptions of clinical judgment (Byrne et al., 2023; Glynn, 2012; Lavoie et al., 2017). Other
authors have included the clinical instructors’ perceptions of the degree to which the students
have demonstrated clinical judgment (Hunter & Arthur, 2016; Smith, 2021). Some researchers
have used observations of clinical instructors, peers, or researchers with or without rating
instruments (Cason & Reibel, 2021; Hines & Wood, 2016; Strickland et al., 2017).
For example, in the skills lab study (Andrea & Kotowski, 2017) and in the Vsim studies
(Fogg et al., 2020; Kool, 2022), clinical judgment was measured by self-evaluation. This can be
useful, but it is important to compare students’ self-assessment with objective observations.
Selfreport bias is very common in research. Bauhoff (2011) explained that many times
researchers will collect self-report data and objectively measured data to develop a model that
describes the extent of the self-report bias in a particular factor. This model can then be used to
adjust selfreport data in subsequent collection to more closely match the objective data without
the cost of only using objective measures. Strickland et al. (2017) performed this research when
the principal investigator (PI) evaluated BSN students (n = 94) using the LCJR and the students
also self-evaluated using the LCJR in an adult health course. The self-reported scores were
higher than PI scores (M = 33.48 [SD = 3.719], M = 31.19 [SD = 3.220], p = .030, respectively).
The authors concluded this likely correlated with increased clinical confidence among students.
This is a reassuring finding but needs to be tempered with clinical faculty wisdom to continue to
protect patient safety (Kool, 2022; Strickland et al., 2017).
Further, the review of the literature showed that there are two primary rating scales used
for this area of study. These are the Creighton competency evaluation instrument and the
Lasater’s clinical judgment rubric. Each of these measurement choices will be reviewed in detail.
Student Perceptions
Student perceptions of their own clinical judgment have been used as a measure of
clinical judgment. Glynn (2012) developed a structured reflective writing assignment for the
classroom setting for BSN students based on Tanner’s clinical judgment model. Focused
interviews conducted after the reflective writing indicated that students (n = 15) perceived their
clinical judgment and clinical confidence as improved. Lavoie et al. (2017) developed a
structured simulation debriefing model called the REsPoND tool for 19 BSN students in a
critical care course. Focused interviews based on Tanner’s clinical judgment model showed that
students perceived an increase in clinical judgment.
Clinical Instructor Perceptions
The perceptions of clinical instructors have been used as an indicator for clinical
judgment among nursing students, but results have been contradictory. In a study by Smith
(2021), clinical instructors (n = 4) reported that having a framework was helpful when they
needed to provide guidance to students about clinical judgment. In contrast, Hunter and Arthur
(2016) reported clinical instructors’ feedback of frustration with the clinical evaluation tool they
were provided. Specifically, they communicated that there was not a clear way to measure or
communicate students’ clinical reasoning development. The difference in these outcomes had to
do with having a clear clinical judgment framework from which to evaluate students.
Qualitative Measurement
Qualitative methods have provided guidance for researchers and educators who are
interested in measuring clinical judgment among nursing students. In one study, researchers
designed a guided reflective writing activity and rubric for junior and senior BSN students. The
questions to guide the writing were designed to encourage thinking within the categories of
Tanner’s clinical judgment model and the rubric for grading was based on the LCJR. After
completing the assignment, student participants (n = 47) completed a survey about their
perceptions of the effect of the reflective writing on their clinical judgment. Focused interviews
with the four clinical faculty who graded the writing assignments also yielded data. The clinical
faculty answered questions about the value of the assignment for evaluating student clinical
judgment development. The data from the student surveys and faculty interviews were analyzed
using the Graneheim and Lundman content analysis method (Smith, 2021). The themes resulting
from the student survey analysis included a subtheme for juniors that the reflective writing
helped them organize their thinking to plan care and for the seniors that the reflective writing
helped them gain a sense of wholeness in their care. The faculty interviews yielded themes about
the value of the assignment for encouragement of deep thinking among students and as a tool to
provide students with feedback to improve their clinical judgment (Smith, 2021).
Quantitative Measurement
Evaluation of clinical judgment by observation was performed in several studies. In a
study by Bussard (2018), nursing clinical faculty evaluated diploma nursing students (n = 70)
enrolled in a med-surg course after each of four simulations using the LCJR. Findings indicated a
significantly increased mean from the first to the fourth simulation (M = 24.10 [SD = 2.59], M
=40.17 [SD = 2.99], p < .001 respectively). In a study by Hines and Wood (2016), clinical
instructors providing scripted debriefing in clinical post-conferences, used the LCJR’s reflection
section to evaluate clinical judgment among senior level students (n = 53) following simulations
and in-person clinicals. These ratings demonstrated improvement in clinical judgment (p = .002).
Independent raters also evaluated these students using the LCJR during simulations and found
significant increases in three of the four categories of clinical judgment in the LCJR (p = .000).
Thus, there is strong evidence that clinical judgment can be effectively assessed by independent
raters or clinical instructors when using the LCJR. Interestingly, there was a significant decrease
in the fourth category, responding, (p = .000). These authors concluded, in consultation with
other literature and in considering the simulation scenarios used in this study, that the decrease in
responding would have been expected because the simulation required advanced communication
skills and the students had limited experience in this area.
Quantitative measurement was also used by clinical instructors in a study by Manetti
(2018) using the LCJR with students in the in-person clinical setting. This study compared
clinical judgment ratings for junior (n = 75) and senior level students (n = 61). The seniors
scored significantly higher (p < .001). The juniors scored at the accomplishing level (M = 29.77
[SD = 4.7]) and the seniors at the exemplary level (M = 36.10 [SD = 5.4]). The authors of this
study established evidence of the construct validity of the LCJR as senior students are expected
to have higher clinical judgment ratings than that of their junior peers.
Lastly, Call (2017) was interested in evaluating whether the type of learning experience
provided to students would result in different LCJR scores. To study this, Call measured the
opportunities provided to perform clinical judgment behaviors comparing two HFS experiences
and 12 objective structured clinical exam (OSCE) stations. Findings indicated that there were
significantly more LCJR indicators in the OSCE stations (p < .05) as compared to the HFS
experiences. Call concluded that students might need to be evaluated in more than two
simulations using the LCJR before arriving at a final clinical judgment score.
Measurement Instruments
Creighton Competency Evaluation Instrument
The Creighton Competency Evaluation Instrument (CCEI) is a 23-item tool used for
measuring clinical competency including subscales for assessment, communication, clinical
judgment, and patient safety (Hayden et al., 2014). This tool has been used by researchers to
report both the total clinical competency score and the subscale scores. Hansen and Bratt (2017)
used the CCEI and found no significant difference in clinical competence by comparing the
sequencing of simulation and in-person clinical among nursing students (n = 48) in their first
clinical course. Raman et al. (2019) evaluated fourth semester BSN students (n = 74) using the
CCEI at the end of their maternity nursing course during an objective structured clinical exam
(OSCE) in the skills lab at their university in Oman. These authors found no significant
differences in clinical competency among students who experienced 100% traditional clinical
experiences and those who experienced 75% traditional experiences plus 25% high fidelity
simulation experiences. Lastly, Kidd (2017) used the CCEI to measure clinical competency
during synthesis simulation observations among fourth semester BSN students (n = 108) and
reported the clinical judgment subscale demonstrated significantly higher scores among male
participants.
Lasater’s Clinical Judgment Rubric
The Lasater’s clinical judgment rubric (LCJR) is the instrument selected for this proposed
research because of its sole focus on clinical judgment. It offers detailed dimensions on which to
base a clinical judgment score (Lasater, 2007). The LCJR was first developed for use in HFS.
Since its development, the LCJR has also been applied to virtual simulations (Vsims), reflective
writing, and in-person clinical.
The studies by Bussard (2018) and Hines and Wood (2016) focused on different
educational methods for clinical judgment development using the LCJR as their evaluative
measurement tool. An advantage of selecting the LCJR is that individuals in different roles,
including clinical instructors, peers, and researchers, have used the LCJR suggesting it is a
feasible tool to implement (Cason & Reibel, 2021; Strickland et al., 2017). In a correlational
study by Stuedemann Fedko and Thomas Dreifuerst (2017), the LCJR was used during HFS and
compared to observation for 11 specific nursing actions. Among senior traditional BSN students
(n = 22) they found a significant relationship (p = .04) between clinical judgment in simulation
and practical action.
The LCJR has been used as the basis for measuring clinical judgment aside from the
original use in simulation. For example, Georg et al. (2019) based their ratings of clinical
judgment from student free text responses in Vsim on the dimensions of the LCJR. Bussard
(2015) inspected reflective writing after simulation for the dimensions of clinical judgment from
the descriptions in the LCJR. And Kubin and Wilson (2017) evaluated students using the LCJR
while observing inpatient pediatric assessments. These examples show the versatility of the
LCJR.
Evidence for Direct Instruction for Clinical Judgment Development
Direct instruction as a method for clinical judgment development among nursing students
is supported by this literature review. Jessee (2018) developed the integrated clinical education
theory by synthesizing principles from four educational frameworks in the literature including
situational learning theory, deliberate practice, expert practice, and Tanner’s clinical judgment
model. This author found clinical reasoning is developed from several angles during clinical.
These included a supportive, authentic clinical context; multiple experiences over time;
integration of cognitive, psychosocial, and affective situational factors; and one-to-one clinical
instruction. This one-to-one instruction needs to include discussion and meaningful feedback that
is specific, close in time to the reference clinical situation, with suggestions for how to improve,
and encouragement for reflection.
Although clinical reasoning is not synonymous with clinical judgment, it is a building
block of clinical judgment (Tanner, 2006). In a teaching tip, Billings (2019) corroborated that
clinical judgment can be taught by coaching students with specific prompts to identify and
practice clinical judgment behaviors. One study investigated methods clinical instructors
intentionally use to help students develop clinical reasoning (Hunter & Arthur, 2016). Their study
revealed that clinical instructors asked probing questions, seized opportune moments during the
clinical day to discuss decision-making, modeled clinical judgment components such as how to
collect pertinent data and prioritize it, and led discussions in post-conference about the meaning
of clinical reasoning and how it applied to making clinical judgments in patient situations.
Gonzalez (2018) also encouraged taking a direct instruction approach in that she developed
lessons and activities based on the LCJR for use with her students in the clinical setting. Nielsen
(2016) advocated for concept-based coaching during the clinical day and investigated its effects
on LCJR indicators for prelicensure ADN and BSN students (n = 39). Gonzalez et al. (2021)
emphasized that it is important to be intentional about teaching clinical judgment and suggested
an appropriate method is using open-ended Socratic questioning at higher levels of Bloom’s
taxonomy based on the LCJR. Similarly, Fontenot (2021) developed a framework of specific
Socratic question topics to ask students during clinical “touchpoints.” The questions guide the
student through their clinical reasoning leading to improvement in clinical judgment. Each of the
methods these authors used provided direct instruction about some aspect of clinical judgment.
And in each case, there was improvement in the aspect studied and/or in clinical judgment
overall.
Gap in the Literature
There is a gap in the literature around quantitative clinical judgment measurement after
application of an educational intervention related to in-person clinical. The closest match found
is research by Hines and Wood (2016). These authors applied scripted debriefing for both
inperson and simulation experiences. They then had independent raters evaluate students during
the last two simulation experiences of the course. The students were encouraged to practice
clinical reasoning by the scripted reflection questions in the debriefing sessions. In contrast, the
education intervention applied in the current study was direct instruction about clinical judgment
behaviors as described in the LCJR. Direct instruction was provided via webinar. Participants
were expected to apply the information from the webinar to their in-person clinical experiences
to help them develop their clinical judgment skills.
Summary
In this literature review, the definition of clinical judgment was discussed and clarified.
This review also demonstrated that several educational methods have been investigated to help
students to develop clinical judgment. Additionally, two main instruments have been used to
measure clinical judgment among nursing students, One, the Creighton competency evaluation
instrument, is focused on overall clinical competency with a subscale dedicated to clinical
judgment. The other, the Lasater’s clinical judgment rubric, is entirely dedicated to clinical
judgment. Further, direct instruction as a clinical judgment development method is supported in
the literature though it has not been consistently applied. The current study addressed the gap in
the literature to understand whether direct instruction about the dimensions of clinical judgment
provided to prelicensure nursing students resulted in an increase in their clinical judgment
ratings. In the following chapter, the methods used for this research are described.
CHAPTER III
METHODS
Prior chapters provided a review of the literature and identified gaps that this study
addressed in its use of the Lasater’s clinical judgment rubric. This chapter describes the methods
used including the research design that guided the study. The study setting, sample, and sample
size are described. The measurement instrument for the study variables is reviewed including the
psychometrics supporting its use. Analyses that were used to answer the research questions are
identified.
Research Design
This study used a two-group, quasi-experimental design to measure the clinical judgment
of nursing students using Lasater’s (2007) clinical judgment rubric (LCJR). The intervention
group (IG) received direct instruction about how to demonstrate clinical judgment based on the
LCJR and the control group (CG) did not. Participants’ LCJR ratings were assessed during
simulation labs that were required as a part of their program of study. The LCJR ratings of the
groups were compared using the Wilcoxon signed rank test and independent samples
MannWhitney U Test to answer this research question:
Q1 Among prelicensure nursing students, how does direct instruction about how the
Lasater’s clinical judgment rubric (LCJR) can be applied to their in-person clinical
experiences compared with no direct instruction affect their clinical judgment
ratings from the end of first semester to the end of second semester?
The independent variable was direct instruction provided to the IG participants. This
instruction provided teaching about the meaning of each of the rubric dimensions in the LCJR as
well as to how to practice each behavior during traditional face-to-face clinical experiences (see
Appendix A for the direct instruction scripts). The dependent variables were the LCJR ratings of
the participants during their final simulations at two time points: at the end of their first semester
and at the end of their second semester of the nursing program.
This research design was appropriate for this study because it provided the opportunity to
observe changes in clinical judgment over time both with direct instruction for the IG and
without direct instruction for the CG. A strength of this design was in its longitudinal aspect. It
was expected that prelicensure students would increase in their clinical judgment from one
semester to the next (Manetti, 2018). By having provided direct instruction to the IG, it was
expected that the increase in clinical judgment would be even greater over the same time period
(Foo et al., 2017). This strategy addressed the internal validity threat of maturation. Another
strength of this design was the comparison between similar groups in similar settings.
There were limitations to the quasi-experimental design. An internal validity threat was
that the study used a convenience sample of those already enrolled in a particular course.
Randomization of participants did not occur in this study design for feasibility reasons. The
sampling strategy was also a threat to external validity as all the participants were enrolled in the
same nursing program at the same university. As a result, generalization to other nursing
programs would not be expected to be as robust as might occur with other designs such as a
multi-site study.
Setting
Concordia University St. Paul (CSP, 2021) was the setting of this study. It is a midsize
university with approximately 5,000 students in undergraduate, graduate, and online programs.
Approximately 500 students are enrolled each year in CSP’s bachelor’s level nursing programs
that include a campus-based option in St. Paul, Minnesota and accelerated (ABSN) hybrid
options based in St. Paul, Minnesota and Portland, Oregon. All the students invited to participate
were enrolled in the ABSN option based in Portland, Oregon. The Portland-based ABSN
program enrolls nursing students three times a year. Each cohort is comprised of 70-100 students
for up to 300 students admitted annually.
The ABSN program is completed in four semesters lasting 16 months. This program
covers 61 credits including 720 hours of clinical education which is a combination of skills labs,
simulation labs with manikins, simulation labs with volunteer patients, virtual clinicals, and
inperson clinicals spread throughout the four semesters (CSP, 2021). However, for the purpose of
this study, the focus was solely on the first two semesters of the four-semester program. In
semester one for both the CG and IG, students were scheduled for 30 hours of skills lab, 36 hours
of high-fidelity simulation, and 54 hours of traditional face-to-face clinical. In semester two due
to shortage of in-person clinical sites during the CG session, students were scheduled for 54
hours of virtual simulation, 30 hours of skills lab, and 36 hours of high-fidelity simulation.
During the IG semester two session, the students were scheduled for 30 hours of skills lab, 36
hours of high-fidelity simulation, and 54 hours of traditional face-to-face clinical.
The university has two skills and simulation laboratories in the Portland area for nursing
students. The simulation laboratories are set up to replicate private hospital rooms with one
highfidelity human simulator, the Laerdal Sim Man Essential, as the patient in each. Students
participating in a simulation also have access to the simulated patient medical record, a phone to
simulate calls to other healthcare staff members, and practice doses of medications. The adjacent
control room houses the computer equipment for the operation of the Sim Man, a one-way mirror
for observation of the simulation room, and the connecting phone to allow for responding to
students’ phone calls. A camera and microphones are set up in the simulation room to provide
live feed of video and audio to the observation conference room.
The CSP program of study requires approximately three simulation lab experiences in the
first semester and another three in the second semester. These might be scheduled at any time in
the semester but usually one occurred within the first five weeks, one within the last three weeks,
and one sometime in between these two. Each simulation experience requires the students to
complete preparation work by studying the possible medical conditions, medications, and
nursing implications of the upcoming scenario. During a simulation lab, two student nurses
participate in the simulation room while the other six of their clinical group are in the observation
room. Simulation faculty lead the students in interactive pre-brief and debrief discussions
preparing for, and then reflecting on, the simulation scenario.
Sample
The target sample for this study was CSP ABSN students in the first semester of their
nursing program of study. Inclusion criteria included enrollment in the semester one cohort in the
fundamentals nursing course, willingness to participate through the first two semesters of the
nursing program, and signing of the informed consent form. All students were over the age of 18
upon admission to the program. There were no exclusion criteria.
Sample Size
An a priori power analysis using G*Power 3.1.9.4 revealed that with a significance level
set at p < 0.05, an effect size of 0.3, and a power of 0.80, 90 students needed to participate in
each study group to capture the within-group effect using two-tailed tests (Apponic, 2021). A
two-tailed test is recommended for health-related research even in conjunction with a directional
alternative hypothesis (Keller & Kelvin, 2013). An a priori power analysis revealed that with a
significance level set at p < .05, an effect size of 0.5, and a power of 0.80, 64 students were
needed in each study group to capture the between-group effect using a two-tailed test (see
Appendix B for G*Power analyses). According to these analyses, the target sample size was 180
with 90 in the IG and 90 in the CG.
Recruitment
As the primary investigator (PI), I attended the orientation meeting via Zoom for the
semester one cohorts in the fall semester of 2022 and the spring semester of 2023. The research
was explained, informed consent forms were provided, and questions were answered (see
Appendix C for the orientation agenda). Participants signed the informed consent forms at the
meeting and returned them to my research assistant who attended the orientation on site. The
informed consent form for the CG was provided to the fall 2022 cohort and the informed consent
form for the IG was provided to the spring 2023 cohort (see Appendix D for the informed
consent forms).
Procedures
Each cohort participated through the first two semesters of the nursing program following
the timeline shown in Figure 1. The CG was recruited first to lessen the likelihood of crossover
effect between CG and IG participants in the same semester. The ABSN program enrolled 74
students in the fall 2022 and 72 in the spring 2023 semesters. Since the enrollment was less than
90 each semester, the desired sample size was not reached. Measurements using the LCJR were
obtained during the final set of simulations each semester for the CG and the IG (T1, T2, and/or
T3).
Figure 1
Recruitment and Participation of Cohorts
Fall Semester 2022 (T1) Spring Semester 2023 (T2) Summer Semester 2023 (T3)
Participation
Note. Clinical judgment was measured for each participant at the end of T1, T2, and T3.
All eligible students who returned the informed consent form were emailed the
demographic questionnaire with confirmation of their participation along with a copy of the
LCJR. A minority of participants completed the demographic questionnaire and returned it via
email. The demographic questionnaire was resent via email after two weeks’ time. A few more
were received as a result. The rest of the demographic forms were distributed during the
simulation observations and collected by my research assistant or me (see Appendix E for the
demographic questionnaire).
Cohort 1
Control Group
(n = 41)
Cohort 1
Control Group
(n = 25)
Cohort 2
Intervention
Group
(
n
= 23)
Participation
Cohort 2
Intervention
Group
(
n
= 27)
Instrumentation
Kathie Lasater (2007) developed her Lasater’s clinical judgment rubric (LCJR) based on
Tanner’s clinical judgment model. The LCJR was developed to be used by clinical instructors to
evaluate students’ clinical judgment during simulation. The LCJR provides clear language to
describe behaviors in line with the cognitions of clinical judgment described by Tanner’s (2006)
model, namely noticing, interpreting, responding, and reflecting. Each of the four cognitions is
broken down further in the LCJR into two to four specific dimensions and then rubric
descriptions are given to level each dimension from beginning, to developing, to accomplished,
to exemplary (Lasater, 2007). The LCJR was used quantitatively by assigning values to each of
the four levels from “1” for beginning to “4” for exemplary. This provides a total score ranging
from 11 to 44 with higher scores indicating a greater level of clinical judgment (see Appendix F
for the LCJR and Appendix G for permission to use the LCJR).
Adamson et al. (2012) performed reliability and validity studies of the LCJR for use in
simulations with nursing students. The intraclass correlation coefficient was found to be 0.899.
Interrater reliability of two raters using the LCJR simultaneously was found to be 96%
consistently after 72 simulation observations. These authors also confirmed the construct validity
of the LCJR by blinding raters to the level of nursing education of students observed. Their
ratings in all four areas of cognition separated the junior level from the senior level students with
higher ratings for clinical judgment obtained for the senior level students, as would be expected.
Manetti’s (2018) findings replicated this finding.
Reliability and validity estimates obtained from studies of the LCJR have been strong.
Authors of four studies reported Cronbach alpha ratings of .886 to .910 (Chmil, 2014;
GubrudHowe, 2008; Manetti, 2015; Shin et al., 2014). An alpha estimate of .70 is generally
considered minimally acceptable for a new scale while .80 is the baseline acceptability level for
an established scale (Remler & Van Ryzin, 2015). These findings suggested the LCJR had
sufficient internal consistency reliability. Confirmatory factor analysis was also performed for the
dimensions of each of the four cognitions. In this statistical method, the researcher could
determine if each part of the model fits together with the whole as written (Remler & Van Ryzin,
2015). The LCJR was deemed valid by this method (Shin et al., 2014; Yang et al., 2019).
In this study, my research assistant and I evaluated all participants by using the LCJR and
took steps to estimate its reliability. First, IRR was obtained as described in the following
section. Next, Cronbach’s alpha was analyzed for the ratings for the 11 items of the LCJR using
IBM SPSS 29.0.1.0 and found to be 0.93. This gave evidence of internal consistency for the use
of this instrument in this study.
Data Collection
Data collected for this research included several items. First, the CG and IG participants
provided demographic data via the questionnaire. Then, my research assistant and I evaluated the
clinical judgment demonstrated by each participant using the LCJR. I made note of attendance
records for the required webinars for the IG. Finally, I collected the clinical attendance records
for the CG and IG from CSP.
Demographic information collected included age, gender, number of years of college
education, and number of years of healthcare experience through employment or volunteer
activities (see Appendix E). The participants returned the demographic questionnaire to me via
email or in hard copy to my research assistant or me onsite.
The LCJR was used to evaluate each participant twice during the final simulations of
their first and second semesters in the ABSN program. The LCJR scores were recorded on the
data spreadsheet according to participant identification number.
Clinical attendance records were requested from the CSP clinical coordinators. This
information was used to confirm the exact clinical education schedule followed for the cohorts of
students from which participants were recruited and to identify if any of the participants had
deviations from the nursing program’s clinical education schedule. Any deviations were listed by
participant identification number in the data spreadsheet.
In addition, the IG participants were required to attend a webinar for direct instruction on
the LCJR. This was scheduled to coincide with the first week of in-person clinical of each
semester of participation. If they could not attend the webinar directly, their review of the
recording of the webinar was counted for attendance. This was recorded in the data spreadsheet
according to their identification number. At this webinar, I provided the meaning of each of the
LCJR’s dimensions applied to how to practice each behavior during traditional face-to-face
clinical experiences. See Appendix A for the direct instruction scripts.
Interrater Reliability
Reliability refers to how consistently a measure represents what is said to represent.
Foundational to reliability is interrater reliability (IRR) that refers to how congruent the scores
are between two or more raters who are using the same instrument. If the raters are not in
agreement on how they rate the subjects or if they are inconsistent with how they rate
individually, then the precision of the measures will be in question and the findings of a study
may be erroneous. Percent agreement is a method by which quantitative data assessed by two or
more raters can be compared to evaluate consistency. The higher the percentage of like scores,
the more consistent, and reliable, the ratings (Alavi et al., 2022; Burns, 2014).
For the current study, IRR was calculated by taking the number of the ratings for the
dimensions of clinical judgment on the LCJR for which the two raters (research assistant and PI)
agreed, divided by the total number of dimensions rated for a particular participant. The goal was
that we would agree on at least 90% of the dimensions. The LCJR lists 11 dimensions. If the
participant was rated on all 11 dimensions, the goal was that we would agree on at least 10
dimensions, yielding a 91% agreement.
We, my research assistant and myself as PI, prepared for agreement between our ratings
with practice ratings (Kardong-Edgren et al., 2017). We met via Zoom several times in advance
of the first round of data collection. During the first meeting, we discussed the LCJR at length,
drawing from our experiences as clinical instructors and as supervisors of clinical instructors, to
come to a common understanding of the descriptions of clinical judgment in each of the rubric
cells.
The next step in our assessment of clinical judgment was to find online recordings of
student nurses in clinical simulations that we could use for practice ratings. We independently
rated two recordings. We found this exercise useful for discussion but realized the purpose of
most recordings of simulations online was for training. Because of this, the online recordings
illustrated expert modeling rather than variable examples of student nurses’ experiences.
To better assess the IRR prior to actual data collection, I arranged with the undergraduate
nursing program at University of Northern Colorado (UNC) to solicit student volunteers to allow
viewings of their recorded clinical simulation experiences. Institutional Review Board approval
was granted by UNC and informed consents were signed by 93% (N = 64) of the students in two
courses. A UNC nursing faculty member recorded the simulations and made these available
through a secure link. As the simulation videos often involved six to eight students, I identified
which student in each recording we would rate using the LCJR.
My research assistant and I then viewed eight of these recordings, rating the identified
students independently using the LCJR. (The rest of the recordings were reserved for later use.)
We recorded our ratings independently on a shared Google Sheet. I calculated our percent
agreement by comparing whether we marked the same rating for each of the 11 dimensions of
clinical judgment in the LCJR for each participating student. Before discussing our results, I
found 18% agreement in our ratings. We then met via Zoom to discuss each of the recordings and
shared the rationale for our ratings.
There were a few key differences in our rationales. The first difference was in timing of
the rating for a few of the dimensions of clinical judgment. For example, regarding focused
observations, one of us was only rating this dimension based on the student’s initial assessment
during the simulation, whereas the other rater was continuing to include how the student
performed re-assessment later in the simulation within this dimension. When we realized this, we
made notes to only score the initial assessment within this dimension. The second difference was
in discriminating between the Beginning and Developing levels of a few of the dimensions. For
example, within the dimensions of the rubric that measure reflection, we needed to clarify the
meaning of how students self-evaluated during debriefing to differentiate these levels. After
discussion, we independently re-rated two of the recordings and found 82% agreement and
decided to proceed in the following way.
Data Collection Began
The first round of data collection, time T1, occurred one week after we completed our
practice ratings in November of 2022. There were two simulation laboratory sites where
simulations were occurring simultaneously according to the regular nursing program schedule.
As a result, my research assistant made observations at one site and I made observations at the
other, both rating the participants using the LCJR as practiced. At both sites, we observed the
participants from the adjacent control room through the one-way mirror, listening to the
streaming audio from the simulation lab. After approximately half of the participants had been
rated, I commuted to her site and my research assistant and I rated one participant simultaneously
and independently. Our ratings without discussion had 82% agreement. After discussion, we had
100% agreement for a total IRR = 91%. This was above the intended goal of 90% IRR so we
proceeded to complete the first round of data collection with each of us having our distinct set of
students to rate. The ratings from these LCJR evaluations were the first set of clinical judgment
data for the CG.
At the beginning of T2, I sent an email message to the CG participants as they were
starting semester two of the ABSN program. This message reminded the participants of the
research study and that they would be evaluated one more time during their final simulation that
semester. Participants were requested to ask questions in reply to this email as needed. No
participants responded to this email.
Also, at the beginning of T2, I recruited participants for the IG. I attended the orientation
meeting for this new cohort via Zoom and my research assistant attended onsite. The students
were introduced to the study and their participation was requested. Informed consent forms were
available in hard copy for students in attendance (see Appendix D for the informed consent
form). Students were instructed to sign the informed consent form and hand it to my research
assistant. All eligible students who returned the informed consent form were emailed the LCJR
and demographic questionnaire with confirmation of their participation. A few participants
returned the demographic questionnaire via email. For those who did not return the demographic
questionnaire via email, hard copies were distributed at the time of the simulations and all
missing questionnaires were collected.
During T2, I scheduled the educational intervention, the direct instruction synchronous
webinar, to coincide with the first week of in-person clinical for the participants. To determine
the right timing for the webinar, I first requested a schedule of the in-personal clinical start dates
from the clinical coordinator for the CSP ABSN program. The participants had two different start
times scheduled to begin in-person clinical. Half of the cohort started in week two of the
semester while the other half began in week nine. I then sent out an email Doodle Poll to all
participants to solicit input about when students would be available to attend the webinar. The
response rate was about 50%. I used this information to schedule six webinars from which the
participants could choose to attend. During this webinar, I instructed the participants about the
LCJR and how they could use it to intentionally practice behaviors that demonstrate clinical
judgment. Eight participants attended the webinar in person and three additional participants
watched the recording of the webinar after it occurred. Their attendance was recorded on the data
spreadsheet by identification number of each participant. See Appendix A for the direct
instruction script that was presented during the webinars.
In preparation for the second round of data collection scheduled for March 2023, at the
end of T2, my research assistant and I met via Zoom in February to practice using the LCJR as a
refresher. We viewed one recording from UNC that we had not yet seen and independently rated
it. Our percent agreement was 80% before discussion and 90% after discussion. The result was
below the intended average goal of 90% for interrater reliability. We then independently rated
two more students from previously unviewed recordings. Our percent agreement before
discussion was 80% and after discussion was 100% which met the expected average IRR goal of
90%.
At the end of T2, data collection for the second round proceeded two and a half weeks
after these refresher practice sessions in March 2023. There were no opportunities for
simultaneous rating during the second round of data collection. My research assistant and I
attended the final simulation labs of the semester for the IG at the two simulation lab sites as
before. We evaluated the participants with the LCJR at our particular sites, from the adjacent
control room through the one-way mirror, listening to the streaming audio from the simulation
lab. The ratings from these LCJR evaluations were the first set of clinical judgment data for the
IG.
Lastly, at the end of T2, my research assistant and I attended the final simulation labs of
the semester for the CG, now in their second semester. We evaluated the participants using the
LCJR in the same manner as before. This was the second set of clinical judgment data for the
CG.
At the beginning of T3, I sent an email to the IG to remind them of the research, that
there would be a webinar to coincide with the first week of their in-person clinical, and that my
research assistant or I would be evaluating them with the LCJR during the final simulation of the
semester. During the webinar for the IG participants in the second semester of their participation,
I instructed them about the LCJR again, this time encouraging personal reflection about how
they could practice the behaviors that demonstrate clinical judgment during in-person clinical
experiences this semester. Eight participants attended the webinar and this was recorded on the
data spreadsheet by the identification number of each participant. See Appendix A for the direct
instruction scripts presented during the webinars.
In preparation for the third round of data collection scheduled for June and July 2023, my
research assistant and I met again for LCJR practice rating sessions in June 2023. We rated one
simulation recording independently and our percent agreement was 73% before discussion and
100% after discussion. We then scheduled another meeting and rated an additional recording
independently in which we reached 90% agreement before discussion and 100% after discussion.
As a result of meeting the 90% threshold, we proceeded to perform the final data
collection observations at the end of T3, beginning two weeks after this practice. We attended the
final simulation labs of the semester for the IG, now in their second semester. We evaluated the
participants using the LCJR in the same manner as before. This was the second set of clinical
judgment data for the IG.
Data Collection Alternative
The simulation sessions occurring at any one time did sometimes include more than one
participant simultaneously. If the evaluation of both participants simultaneously was not possible,
there needed to be an alternative. Therefore, I arranged in advance to have each simulation
session recorded so that the recording could be observed later in the day to complete the
evaluation of the second participant. Participants were informed about being recorded both in the
informed consents and then were reminded again in person before each simulation scenario.
Relying on the recordings to evaluate participants occurred more frequently in the CG than the
IG due to greater numbers of participants in the CG.
Data Analysis
First, the data were examined for completeness. The data for any CG participant were
considered complete if there was a signed informed consent form, a finished demographic
questionnaire, two LCJR evaluation ratings, and full attendance in all clinical education
experiences according to the nursing program schedule or a make-up session per ABSN program
guidelines if there was an absence. The data for any IG participant were considered complete
with the same criteria as the CG with one addition: attendance at the direct instruction webinars
in both semesters. Participants were included in the analysis according to the completeness of
their data sets.
Second, descriptive statistical analysis was applied to the demographic data using SPSS
version 27 to identify any significant differences between the CG and IG or within either group.
This included statistical differences in percentage of male to female participants, age ranges,
number of years of education, and number of years in healthcare employment or volunteering.
Each of these factors of the participants was listed in tables showing measures of central
tendency.
Third, there was statistical analysis of the LCJR ratings for the CG using SPSS version
28.0.1.1(15) and the IG using SPSS version 29.0.1.0. The means, standard deviations, and ranges
for the LCJR ratings for each set of data were calculated. From this information, it was clear the
data were not normally distributed. Because the data were not found to be normally distributed,
the Wilcoxon paired signed rank test was performed to determine statistical significance as
applied to the CG and the IG data separately. This test was appropriate related to the dependent
pairs of data, namely the repeated measures of clinical judgment from semester one to semester
two for each group (Remler & Van Ryzin, 2015).
Fourth, there was statistical analysis of the LCJR for between groups using SPSS version
28.0.1.1(15). Since the data were not normally distributed the independent samples Mann-
Whitney U-Test was applied to the data to compare the differences between the IG and CG.
Data Security
I oversaw data security and made every effort to keep participant identity and collected
data confidential. Participant identity was listed on a code spreadsheet with the participants’
names, email addresses, and identification numbers. This code spreadsheet was saved in its own
folder in a password protected computer accessible only to me. A data spreadsheet listed data by
participant identification number only. This data included demographics, LCJR ratings,
aberrations from clinical education attendance records, and attendance at the webinars for IG
participants. The data spreadsheet was saved in its own folder in the password protected
computer. Any informed consent forms or demographic forms received by electronic means were
saved in separate folders in a password protected computer.
All hard copy informed consent forms or demographic questionnaires received by the
research assistant were kept locked securely and then locked securely by me for the duration of
the study. These hard copies, along with hard copies of LCJR evaluation forms filled in by me
and my research assistant, were kept under lock in my private office file cabinet. Signed
informed consent forms were transferred to the research advisor to be secured by her per
University of Northern Colorado protocol. All records from this research will be secured for
three years.
Ethical Considerations
This study protocol was submitted to the Institutional Review Board of the University of
Northern Colorado and received exempt status (see Appendix H). Exempt status was sought
because there were no foreseeable risks to participants beyond what would normally be
experienced in their educational program. Attendance records are not considered protected
information requiring special disclosure according to Family Educational Rights and Privacy Act
guidelines (U.S. Department of Education, 2021). Therefore, these were requested from the
clinical coordinator and kept on the data spreadsheet in a password protected computer along
with other study-acquired data. As with all data and participant identifiers, every effort was made
to keep information confidential.
Summary
This quasi-experimental study investigated whether students’ clinical judgment ratings on
the LCJR yielded statistically significant differences between participants who experienced
direct instruction about clinical judgment behaviors and those who did not receive direct
instruction. The CSP ABSN students were recruited in their first and second semesters of their
four-semester program of study. The CG was evaluated according to the LCJR by me or my
research assistant two times: at the final simulation lab of each semester of participation. The IG
followed the same schedule for evaluation as the CG with one addition. The IG was encouraged
to participate in a direct instruction webinar teaching the meanings of how clinical judgment is
demonstrated according to the LCJR. The LCJR ratings of all participants were used for data
analysis. Statistical analysis of results was performed using Wilcoxon paired signed rank tests
and independent samples Mann-Whitney U Tests and are described in the following chapter.
CHAPTER IV
RESULTS
The purpose of this study was to test whether direct instruction for prelicensure nursing
students about behaviors that demonstrate clinical judgment and encouragement to intentionally
practice these behaviors during in-person clinical promoted the development of clinical
judgment. The LCJR ratings were measured from semester one to semester two for all
participants. In this chapter, the participants’ demographics and the clinical judgment ratings are
described for within and between groups as calculated using IBM SPSS versions 28.0.1.1 and
29.0.1.0.
Participants
Demographic data were analyzed for the control group (CG; see Table 1) and the
intervention group (IG; see Table 2). Participant groups in this study were compared
demographically for age, gender, years of college completed, years of employment in health
care, and years of volunteering in healthcare. The typical participant was 30 years old, female,
had completed more than four years of college, and had two to three years of experience in a
healthcare environment. All were enrolled in the Concordia University St. Paul ABSN nursing
program performing their in-person clinical experiences in Portland, Oregon.
Table 1
Demographic Descriptors for the Control Group
M
Range
Age (years)
29
20 - 42
Gender
Male
18
3
Female
82
14
Years of College Completed
3.9
0 - >7*
Years of Work in Healthcare
2.2
0 - >4+
Years of Volunteering in Healthcare
1.7
0 - >4**
Note. N = 17. *Assumption for mean that >7 years of college = 10yrs; +Assumption for mean
that >4years of work in healthcare=5years; **Assumption for mean that >4years of volunteering
in healthcare=5years.
Table 2
Demographic Descriptors for the Intervention Group
M
Age (years) 31.2
Gender
Male 0 0
Female 100 10
Years of College Completed 4.1 0 - >7*
Years of Work in Healthcare 1.45 0 - >4
Variable
%
n
Variable
%
n
Range
21
-
51
Years of Volunteering in Healthcare 1.25 0 - >4**
Note. N = 10. *Assumption for mean that >7years of college=10yrs; +Assumption for mean that
>4years of work in healthcare=5yrs; **Assumption for mean that >4years of volunteering in
healthcare=5yrs.
The results of the comparative analysis of the demographic data are shown in Table 3.
The CG and IG were found to be statistically similar using the independent samples
MannWhitney U Test at p = .050 as calculated except for gender. For gender, the groups
significantly differed. The control group included 18% male and 82% female compared to the
intervention group that was 100% female. The nonparametric test of independent samples Mann-
Whitney U was used because the sample was not of sufficient size to yield normal distributions.
The null hypothesis that the variables were the same across the CG and IG was retained at the
significance level p = .050 for all variables except gender.
Table 3
Comparisons of Demographics Across Control and Intervention Groups
Comparisons using Independent
Samples Mann-Whitney U Test
Asymptotic Significance
(2-sided test)
Conclusion
Age (years)
.513
Retain null hypothesis.
Years of College Completed
.749
Retain null hypothesis.
Years of Work in Healthcare
.481
Retain null hypothesis.
Years of Volunteering in Healthcare
.471
Retain null hypothesis.
Attrition
Attrition occurred at a high rate during this study. Among control group participants, the
attrition rate was 39%. The intervention group experienced an attrition rate of 73%. No
participants directly withdrew from the study. Attrition took several forms; the major causes of
attrition are noted in Table 4.
Table 4
Causes of Attrition
Cause Study Sample Size Remaining Sample
Attrition (n) Before Loss After Loss
Not present at simulation
3
68
65
Role of family member
9
65
56
Withdrew from CSP
5
56
51
Failed a CSP course
11
51
40
Incomplete rating form
1
40
39
Missing direct instruction (IG)
12
39
27
Demographic data from those who contributed to attrition by withdrawal for any reason
can be seen in Table 5. The typical participant who withdrew from the study was similar to the
participants in the total sample. The final sample with complete data was n = 27.
Table 5
Demographic Descriptors of Participants of Attrition
Variables
%
n
Mean
Range
Age (years)
29
21-46
Gender
Male
Female
22
78
8
28
College Completed (years)
4.9
0->7*
Healthcare Work (years)
2.4
0->4+
Healthcare Volunteer (years)
1.0
0->4**
Missing Demographic Form
5
Note. N = 41. *Assumption for mean that >7years of college=10yrs; +Assumption for mean that
>4years of work in healthcare=5yrs; **Assumption for mean that >4years of volunteering in
healthcare=5yrs.
Lasater’s Clinical Judgment Rubric Ratings
Control Group
The final CG included 17 participants. The LCJR ratings showed a statistically
significant increase from first semester (T1; M = 20.58 [SD = 4.25]) to second semester (T2; M
= 26.37 [SD = 5.56]) according to Wilcoxon paired signed rank test (z = 3.392, p < .001), with
large effect size (r = .86) according to Cohen 1988 criteria. This showed that the second
measures were an average of 79% greater than the first measures (Sullivan & Feinn, 2012; see
Table 6).
Table 6
Clinical Judgment Ratings for the Control Group
4
19
27
5
26
25
8
21
34
9
22
23
Participant
T1
T2
11
23
37
14
17
33
15
16
25
16
27
30
17
25
33
18
25
33
19
18
21
20
15
20
24
16
22
29
20
21
30
17
24
31
23
22
41
28
30
Note. n = 17.
Intervention Group
The IG included 10 participants. The LCJR ratings did not show a statistically significant
increase from semester one, T2, (M = 29.40 [SD = 4.17]) to semester two, T3, (M = 28.80 [SD =
6.61]) according to Wilcoxon paired signed rank test (z = -.357, p = .721). Table 7 provides the
clinical judgment ratings for the intervention group.
Table 7
Clinical Judgment Ratings for the Intervention Group
Participant
T2
T3
43
30
22
46
34
27
47
22
29
48
33
30
51
33
37
53
29
37
59
25
23
65
34
33
67
26
17
68
28
33
Note. n = 10.
Comparison Data
The independent samples Mann-Whitney U test was conducted to compare the CG and
IG ratings. The result showed that the mean rankings of the ratings of the CG (M = 16.82) were
significantly higher than those of the IG (M = 9.20; U = 37.000; p = .015).
Summary
The results of the analysis of the demographics of the CG and IG showed they did not
significantly differ statistically. The CG showed a statistically significant increase in clinical
judgment as demonstrated by the Wilcoxon paired signed rank test while the IG did not show a
statistically significant increase. Comparing the two groups, the CG demonstrated a greater
increase in clinical judgment than the IG according to the independent samples Mann-Whitney U
Test. In the next chapter, these results are discussed.
CHAPTER V
DISCUSSION
In previous chapters, the background, literature review, methods, and results were
described. This chapter discusses this study’s findings in view of the literature and prior
evidence. Limitations of the study and their implications for future research and nursing
education are also explained.
Discussion of the Study’s Findings
The purpose of this study was to test whether direct instruction for prelicensure nursing
students about behaviors that demonstrate clinical judgment and encouragement to intentionally
practice these behaviors during in-person clinical promoted the development of clinical
judgment. The following research question guided this study:
Q1 Among prelicensure nursing students, how did direct instruction about how the
Lasater’s clinical judgment rubric (LCJR) could be applied to their in-person
clinical experiences compared with no direct instruction affect their clinical
judgment ratings from the end of first semester to the end of second semester?
Answering this question came in three steps. First, the clinical judgment ratings were
compared within the control group (CG) from semester one to semester two. Next, the same
comparison was made for the intervention group (IG). Lastly, between groups comparisons were
analyzed to describe whether the IG mean clinical judgment rating had a greater increase from
semester one to semester two than that of the CG.
The expectation was that each group would show increased clinical judgment ratings
from semester one to semester two. As prelicensure nursing students progress through each
course in the program, they have the opportunity to practice and apply the behaviors and thinking
processes of clinical judgment and typically show increases in their clinical judgment scores. The
findings from this study were mixed compared to those of Manetti (2018). In her study, raters
were blinded to the academic level of prelicensure nursing students (n = 136) and the raters
assigned higher clinical judgment ratings on the LCJR for senior students than junior students as
would be expected. In the current study, the CG clinical judgment ratings increased while the IG
clinical judgment ratings did not. The CG’s clinical judgment ratings increased an average of
79% from the first to the second measure (Sullivan & Feinn, 2012). Unfortunately, the IG clinical
judgment ratings decreased slightly, though not statistically, leading to the conclusion that this
study did not provide evidence that direct instruction about the dimensions of clinical judgment
would have a positive effect on clinical judgment development among prelicensure nursing
students.
The unexpected result from the IG led to considerations about influencing factors. The
very prominent attrition rates and the subsequent small sample size certainly had an impact.
Methodology and procedures might also have contributed.
Small differences among participants in a small sample size can be magnified in the
associated results. For example, in this study, differences might have existed among the
participants in ways that affected their abilities to perform during the simulation. The first three
dimensions of clinical judgment on the LCJR had to do with effective noticing. According to
Tanner (2006), effective noticing has to do with the nurse more than the patient. For example, the
nurse’s previous experience with the patient and others like the patient would influence what the
nurse noticed. In the case of a simulation, knowing the patient would have to do with the effort
and thinking the student had invested in preparation for the simulation scenario. This could
include preparatory assignments and studying the patient chart of the simulated patient prior to
entering the scenario. If a few students were unable to put in significant time and energy into
completing their preparation assignment, their diminished ability to effectively notice in the
simulation could have influenced their clinical judgment ratings enough to skew the IG’s overall
rankings in the statistical analysis. Other internal validity threats such as tiredness or external
stressors could have affected IG participants’ abilities to effectively notice. However, these same
factors likely could have been experienced by the CG. Without data about these factors among
the participants, it was difficult to understand whether these played a role in the results.
Direct Instruction as an Intervention
Because the IG did not increase their clinical judgment scores, there was no support for
direct instruction as a method to increase clinical judgment in prelicensure students in this study.
This finding conflicted with the literature of direct instruction that supported the notion of
providing teaching in a clear and stepwise fashion to garner greater increases in learning
(Barbash, 2021). In examining the failure of any intervention to perform as expected, the dose,
duration, and strength of the intervention should be considered. In this study, the dose of the
intervention differed with some participants attending one direct instruction offering while others
attended two. This change was initiated to help maximize the sample size, but it may have
contributed to the lack of significant results.
In reviewing the literature for other ways to handle direct instruction and a booster
session during an intervention, two studies provided strategies that might have been more
successful. According to Cesta et al. (2016), continued engagement with the participants is
important to prevent attrition. In the current study, participants provided input about the webinar
dates and times and received a copy of the webinar schedule. Reminder emails were sent to
participants prior to the webinars. Only one reply was received from the reminder emails which
said the participant was planning to attend. Subsequently, that participant did not attend the
scheduled webinar. Thus, engagement strategies were employed; however, it is not known if the
chosen strategies were meaningful to participants or if there was something else that could have
been planned.
Abshire et al. (2017) studied retention methods for longitudinal studies (n = 19) in which
attrition rates were less than 20%. According to focused interviews by Abshire et al. with the
researchers who conducted the longitudinal studies, several themes emerged. First, community
involvement in study design, recruitment, and retention was important for retention. This
criterion was met in the current study by soliciting study design ideas from others who had done
research with nursing students. The initial idea for direct instruction as an intervention came
from these discussions, particularly highlighting the specific behaviors that demonstrated clinical
judgment from the LCJR. Another theme was to be careful to explain all the details of the study
to those being recruited for the study; this criterion was met during an orientation session at the
beginning of the semester. Students were also given the opportunity to ask questions onsite with
the research assistant. A third theme was to have a clear method for communication and to
provide reminders. Criteria for this theme were met by using the same email address for all
communications and reminders. The fourth applicable theme was to make the benefits of the
study clear to participants. The participants were informed that their participation in the direct
instruction webinar could help them more effectively develop their clinical judgment. The last
theme that emerged was to offer incentives. While incentives might encourage some participants
to remain in a study, these could be seen as coercive. In the current study, the primary
investigator was an adjunct nursing faculty member in the course in which the participants were
enrolled. To avoid the potential for coercion, incentives were not offered.
While the state of the science for direct instruction did not offer evidence for the best
timing or length of the provided instruction, these recommendations from intervention research
might help to move the science forward. Further, exploring the limitations of the study might
provide other guidance.
Limitations
There were several methods challenges during the study. Despite efforts to promote
recruitment and retention, recruitment lagged behind predicted rates and high attrition among
participants greatly impacted the final sample size. Approximately 38% of the participants who
enrolled in the study completed the study activities. While most longitudinal studies have some
attrition, one of the challenges to complete data sets might have been mitigated by changing
some of the study procedures.
Data Collection Procedures
Full data sets for all participants (n = 68) included signed informed consent, completed
demographic questionnaire, LCJR ratings from semester one and semester two, and completion
of the standard in-person and simulation clinical schedule for CSP. In addition, the intervention
group participants needed to attend the direct instruction webinar in semester one and/or
semester two.
Part of the plan for collecting full data sets was successful. There was no attrition related
to signing the informed consent or filling in the demographic form. All participants signed the
informed consent (n = 68) and the completed demographic form was collected from all
participants of the final sample (n = 27).
Other study procedures such as the process for collecting LCJR ratings for participants
met with a few challenges. Students who did not attend their scheduled simulation session (n = 3)
could not be included because it was not feasible for the researchers to attend make-up sessions.
A second challenge was some students were assigned the role of the family member during the
scenario (n = 9). In this case, they were not demonstrating the nursing role and, as a result, their
simulation performance could not receive an LCJR rating. In another case, measurement error
occurred when one dimension of clinical judgment on the LCJR form for one participant (n = 1)
was missed, yielding an incomplete rating.
Another factor affecting the collection of full datasets was beyond the control of the
researchers. Some students withdrew from the CSP nursing program for reasons such as an
intended withdrawal (n = 5) or failure of a nursing course (n = 11). More participants withdrew
by these means from the CG (intended withdrawal n = 3; failure of a course n = 8) than from the
IG (intended withdrawal n = 2; failure of a course n = 3). In either case, the participant did not
attend their simulation sessions, which led to missing data.
The final factor affecting the acquisition of full data sets and attrition was when
participants in the IG did not attend the direct instruction webinar in at least one semester (n =
12). While the plan to have two direct instruction webinars was based on educational evidence
and scientific rationale, in retrospect, this might have led to an increase in participant burden
during a challenging semester.
Additional Ways to Increase Retention and Minimize Attrition
Other strategies might have increased recruitment, retention, and, ultimately, the final
sample size. For example, communication with the participants could have been done differently.
It is common nowadays to receive appointment reminders via text. This could have been more
effective than email. Participants could have been asked during the orientation what method of
communication they would have preferred. This might have increased the likelihood of
attendance at the direct instruction webinars.
Group Differences
Although the groups did not differ demographically other than gender, as a result of
limited clinical sites, the groups were treated differently within the nursing program. When the
CG participants were in their second semester, no in-person clinical sites were available for their
cohort. As a result, the ABSN nursing faculty provided virtual simulation as a substitution for the
usual 54 hours of traditional in-person clinical. In contrast, when the IG participants were in their
second semester, in-person clinical sites had been obtained and these students attended in-person
clinical for the 54 hours and did not participate in any virtual simulation.
The nursing literature about virtual simulation has been increasing in recent years. Two
studies stood out that might give insight into the effect this substitution might have had on the
CG. Fogg et al. (2020) studied clinical judgment using virtual simulation with senior students (n
= 234). Their findings included increased student perception of clinical judgment after virtual
simulation (p = .000) and fewer attempts to achieve minimal scores of clinical judgment as their
experience with the medium increased (p = .000). Rim and Shin (2022) developed a multi-user
virtual simulation for pediatric scenarios for prelicensure nursing students (n = 45). Six scenarios
were used and, in all cases, the students’ clinical judgment scores on the LCJR increased from
pre-test to post-test (p = .000).
Even though these studies did not directly compare students experiencing virtual
simulation versus traditional in-person clinical, it is possible the clinical judgment of the CG was
increased by participating in the virtual simulation. Since the IG did not experience the virtual
simulation, they would not have had this extra increase if it was actually from the virtual
simulation itself. More research ought to be done to directly compare groups in this way. It
would shed light on potential options to specifically increase clinical judgment among
prelicensure nursing students.
Site-Related Factors
Finally, this study was subject to site-related limitations that affected the sample size and
generalizability of the findings. First, the participants were all from one nursing program at one
university. Even though they were scheduled to perform their simulation experiences at two
different physical locations, they were all students in the same curriculum. It is not known if
direct instruction improvements would be obtained in other semesters or in other nursing
programs. Second, the sample sizes were small. Concordia University Saint Paul (2021) enrolls
70-100 students per semester in the Portland ABSN cohort but during the recruitment period, the
actual enrollment was less than 75 in each cohort. Therefore, the statistical goal of 90 participants
per group was ambitious.
Implications for Future Research
Implications for further research are many. First, a multi-site study recruiting participants
from several universities and from various geographic locations would provide more robust and
generalizable results. This would provide the opportunity for enrolling a much larger sample
size. Additionally, attrition would be easier to absorb.
Second, designing the procedures to include the direct instruction webinar in conjunction
with a previously scheduled participant commitment would have greatly reduced attrition for
attendance. The design for future studies of this kind might include more active networking with
nursing faculty at the study site to encourage greater collaboration. For example, the clinical
instructors might offer the direct instruction in person with their clinical groups during
postconference. This would allow the instructor to provide examples from the clinical
experiences of that day, which might heighten participants’ interest and knowledge. Active
debriefing following simulation has been shown to increase clinical knowledge and judgment
(Lee et al., 2020). A similar finding might be likely when applied to post-conferences for in-
person clinical experiences.
Implications for Nursing Education
This study provided impetus for further research regarding how to help prelicensure
nursing students develop their clinical judgment skills. The direct instruction provided for this
research included several components. Some components worked well while others had limited
benefit. Components deemed positive were planning the webinar for small groups, direct use of
the LCJR, and application as demonstrated through a relevant clinical example. The group
setting was a benefit and the poll questions embedded in the presentation added interest as
students participated more readily. Providing the LCJR for participants to read during the direct
instruction helped make the examples clear. Lastly, the provision of a fictional nursing student
vignette and poll questions helped student participants apply clinical judgment examples to a
representative person and situation.
While there were positive outcomes from the direct instruction design, there were also
strategies that could be improved. The direct instruction might have been more engaging if the
fictional nursing student vignette was provided as a video rather than just audio. Computer
captioning for the audio vignette was used but this was not always completely accurate.
Therefore, a video might have made the clinical situation even more clear and engaging.
In the second semester, the direct instruction webinar focused more on how the students
could apply the sections of the LCJR to their own in-person clinical experiences rather than
focusing on the fictional nursing student. The participants’ responses included that they thought
the ideas presented in the webinar were practical and helped them understand more clearly how
to practice clinical judgment. These comments suggested that aligning direct instruction with
individual coaching of nursing students might be helpful. Individual booster sessions provided as
a follow-up in clinical one-on-one by clinical instructors might help connect clinical judgment
more firmly to actual clinical situations. Further research of how to mentor clinical faculty to
provide direct feedback from the LCJR might result in even greater increases in clinical
judgment ratings.
Another important design choice would be to provide the direct instruction in a way or at
a time that allowed more participants to fulfill the attendance requirement. For example, offering
direct instruction during the scheduled clinical post conferences might be beneficial. Clinical
instructors might need help in revising agendas that are already fairly packed for clinical
conferences. Support from administration to allow time for clinical instructors to develop and
employ direct instruction might be needed. Direct instruction might be more acceptable if
presented from a champion in each semester who already had personal relationships with the
clinical instructors.
Along this same vein, the amount of direct instruction needed to make a significant
difference would be important to explore. When simulation was starting to become more readily
used in nursing programs, the amount of simulation that was best needed to be understood.
Hayden et al. (2014) conducted a landmark study replacing <10%, 25%, or 50% of traditional
inperson clinical with high-quality high-fidelity simulation. They found that nursing knowledge,
evaluation by clinical instructors, NCLEX pass rates, nor evaluations within the first six months
of new graduate nursing practice differed statistically significantly among the three groups. This
work has led many administrators of nursing programs to begin to use this option for clinical
experiences, especially when clinical sites were sparse. This kind of study could also be
conducted to compare the use of varying amounts of virtual simulation for the outcome of
clinical judgment ratings.
Summary
While the study results partially supported the notion that clinical judgment increased
from one semester to the next, there was no support for direct instruction about the LCJR to
enhance students’ clinical judgment ratings. High attrition and a small sample size limited the
validity of the findings related to direct instruction. Repetition of this study using larger sample
sizes from multiple universities and various locations might provide more robust evidence.
Minimizing attrition would also have increased the generalizability of the study results. The
potential to increase clinical judgment via virtual simulation ought to be further explored to
compare traditional in-person clinical experiences without virtual simulation. Nurse educators
could learn from this study’s limitations as they continue to test methods to increase clinical
judgment among prelicensure students. Continued growth in this area would result in nursing
students developing their clinical judgment more effectively. In this way, new graduate nurses
will be better prepared to demonstrate entry-level competence in patient safety.
