decision making- week 4
Meekoangelmia
Monteiro2019Criticalthinkingbiasesanddualprocessing.pdf
66 | wileyonlinelibrary.com/journal/medu Medical Education. 2020;54:66–73.© 2019 John Wiley & Sons Ltd and The Association for the Study of Medical Education
1 | INTRODUC TION
Virtually everyone would agree that a primary, yet insufficiently met, goal of schooling is to enable stu- dents to think critically. (Willingham, 20071)
Not everyone. We do not agree. In this paper, we defend the po- sition that the above assertion (ie that the central focus of education should be to inculcate general skills like critical thinking, problem solv- ing, clinical reasoning and reflection) is indeed a myth. Although the idea of general thinking skills has a long history, it first emerged as a
major focus of curriculum reform and research effort in the 1960s, was discounted by evidence in the 1970s and 1980s, but has re- emerged under different banners in the new millennium.
Our central claim is that the preponderance of evidence, in medical education and cognitive psychology, does not support this assertion. Instead, the evidence demonstrates again and again that the essence of expertise is the possession of a large, organised and retrievable body of both formal and experiential knowledge, not any kind of gen- eral thinking skills. In this paper, we annotate a brief history of the rise and fall, and rise again, of this assertion, providing the perspective from both cognitive psychology and medical education research.
Received: 30 May 2018 | Revised: 10 October 2018 | Accepted: 13 February 2019
DOI: 10.1111/medu.13872
P R A C T I C E
Critical thinking, biases and dual processing: The enduring myth of generalisable skills
Sandra Monteiro1,2 | Jonathan Sherbino2,3 | Matthew Sibbald2,3 | Geoff Norman1,2
1Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada 2McMaster Education Research, Innovation and Theory Programme, McMaster University, Hamilton, Ontario, Canada 3Department of Medicine, McMaster University, Hamilton, Ontario, Canada
Correspondence Sandra Monteiro, David Braley Health Sciences Centre, McMaster University, 100 Main Street, 5th Floor, Hamilton, ON L8P 1H6, Canada. Email: [email protected]
Abstract Context: The myth of generalisable thinking skills in medical education is gaining popularity once again. The implications are significant as medical educators decide on how best to use limited resources to prepare trainees for safe medical practice. This myth- busting critical review cautions against the proliferation of curricular inter- ventions based on the acquisition of generalisable skills. Structure: This paper begins by examining the recent history of general thinking skills, as defined by research in cognitive psychology and medical education. We describe three distinct epochs: (a) the Renaissance, which marked the beginning of cognitive psychology as a discipline in the 1960s and 1970s and was paralleled by educational reforms in medical education focused on problem solving and problem- based learn- ing; (b) the Enlightenment, when an accumulation of evidence in psychology and in medical education cast doubt on the assumption of general reasoning or problem- solving skill and shifted the focus to consideration of the role of knowledge in expert clinical performance; and (c) the Counter- Enlightenment, in the current time, when the notion of general thinking skills has reappeared under different guises, but the fundamental problems related to lack of generality of skills and centrality of knowl- edge remain. Conclusions: The myth of general thinking skills persists, despite the lack of evi- dence. Progress in medical education is more likely to arise from devising strategies to improve the breadth and depth of experiential knowledge.
| 67MONTEIRO ET al.
Not surprisingly, the noble idea that teaching and learning should be about thinking, not knowledge, is not unique to medicine. One area in which it featured prominently was American education around the turn of the last century, when, as in classical European education, the emphasis was on the development of “mental fac- ulties” exemplified by the study of Latin, Greek and logic. Evidence to the contrary was presented by Thorndike in 1906, who showed that typical transfer effects across dissimilar tasks were very low (described in Lehman et al2): general skills (“mental faculties”) did not exist. However, as many health professionals can attest, the course in Latin remained in the public school curriculum long after the ratio- nale for its existence had disappeared.
In psychology, “mental faculties” were replaced by behaviourism, which dominated scientific psychology, particularly in the USA, to mid- century. However, in the latter decades of the 20th century, this mechanistic and reductionist theoretical perspective was even- tually superseded by cognitive psychology. Concurrently, the focus of health professions education moved away from behavioural ob- jectives towards the development of underlying thinking processes.
2 | THE RENAISSANCE: 1960 - 1980
2.1 | The emergence of information- processing models
2.1.1 | Cognitive psychology
A driving force behind the cognitive revolution was the develop- ment of computers as a metaphor for human “information pro- cessing.” Early forays into the machine as a metaphor of the mind led to research in artificial intelligence such as Newell and Simon's “general problem solver” (the name says it all), based on the premise that human (and machine) problem solving was a matter of adopting general strategies that could then be mobilised with specific knowl- edge bases to solve problems.3,4 The lay literature was permeated by numerous content- free strategies—brainstorming, lateral think- ing, synectics—that purported to lead to large increases in problem solving and creativity, but did not. We now know that the metaphor was taken too literally; human minds do not work in the same way as the computers that were designed in the 1970s and 1980s.5
2.1.2 | Medical education
The origins of research on expert diagnostic reasoning are credited to seminal work conducted in the 1970s based on the assumption that careful observation of expert clinicians would help identify a set of expert problem- solving skills that could be taught directly to trainees.6-8
A general process did emerge from these studies. It consisted of two stages: an initial “hypothesis generation” stage occurring in the first few seconds or minutes of the encounter, followed by a long, sequential and systematic search for additional confirmatory data. However, three additional findings arose which presented a serious challenge to the underlying assumption. Firstly, the process
was too general. Everyone, from first- year medical student to expert clinician, was essentially using the same process of generating and testing hypotheses; it was simply noted that experts were doing it better. Secondly, when the outcome of the process—diagnostic ac- curacy—was examined, it was found to relate primarily to only one variable: the content of the hypothesis. If participants thought of the diagnosis early on, they got the correct outcome; if they didn't, well, they didn't. Finally, and critically, success on one problem was no guarantee of success on another. The typical correlation across problems was 0.1- 0.3. Thus, the notion of general problem- solving strategies failed an empirical test.
Elstein et al, in a later paper, summarised these findings ele- gantly: “A purely formal syntax of clinical reasoning stripped of con- text and content could account neither for difference in the quality of hypotheses generated nor for clinicians’ variability across cases. It seemed, rather, that differences in domain- specific knowledge must lie behind the ability to generate better hypotheses in specific cases.”9
3 | THE ENLIGHTENMENT: 1980 - 20 0 0
3.1 | The role of knowledge and specificity of skills
3.1.1 | Cognitive psychology
It soon emerged that human problem solving was not a matter of ac- quiring elaborate skills; rather, it amounted to fairly simple strategies operating on extensive and rich knowledge networks.4
This central role of knowledge in reasoning then led to a new field of research based on understanding expertise in different domains. Phenomena such as “deliberate practice” emerged di- rectly from these findings.10 In this view, expertise has nothing to do with general strategies and everything to do with experiential knowledge acquired from practice with many, indeed, thousands of problems.
The limited generalisability of cognitive skills emerged in an- other research domain: studies of transfer. An extensive research programme examining transfer—using knowledge acquired in one context to solve problems with the same conceptual structure in a different context—revealed consistently that far transfer was exqui- sitely difficult.11
Finally, the continued use of computers to simulate human problem solving revealed that programs using general methods like “means–end analysis,” while useful for simple problems, were inef- fective in knowledge- rich domains and were referred to as “weak methods.”4 Alternatives based on expert knowledge, called “expert systems,” were far superior in specific domains.12
3.1.2 | Medical education
The early findings pointing to the centrality of knowledge led to a change of emphasis in the study of clinical reasoning. Instead of fur- ther pursuing some generalisable reasoning skill, researchers began
13652923, 2020, 1, D ow
nloaded from https://onlinelibrary.w
iley.com /doi/10.1111/m
edu.13872, W iley O
nline L ibrary on [03/07/2023]. See the T
erm s and C
onditions (https://onlinelibrary.w iley.com
/term s-and-conditions) on W
iley O nline L
ibrary for rules of use; O A
articles are governed by the applicable C reative C
om m
ons L icense
68 | MONTEIRO ET al.
to explore the kinds of knowledge structures that experts use. There emerged a plethora of possibilities: exemplars, prototypes, verbal propositions and semantic axes.13-15 Regrettably, the very number of possibilities led the field into disarray as the available research methods were not sufficiently powerful to distinguish one represen- tation from another. Indeed, it is likely that there is no one central form of knowledge; rather, one hallmark of expertise is the mastery of vast domains of knowledge ranging from analytical—base rates and physiological mechanisms—to experiential.
Further developments in artificial intelligence led to computer “decision support systems” based on expert knowledge and applica- ble to limited domains.16 Although computer power was minuscule in comparison with that of today's hardware, a number of computer applications were shown to provide a small but consistent benefit to diagnosticians.17
4 | THE COUNTER- ENLIGHTENMENT: 20 0 0 TO THE PRESENT
4.1 | The re- emergence of thinking skills
The new millennium found a resurgence of interest in general think- ing strategies, both in general education and psychology, and in medical education. However, the names changed. Terms like “criti- cal thinking,” “metacognition” and “reflection” entered the lexicon. Moreover, “dual process” theories of decision making came to domi- nate discourse on problem solving and diagnosis.
4.2 | Cognitive psychology
4.2.1 | Critical thinking
Critical thinking is often described as a skill that students can de- velop in parallel with (but not necessarily connected to) knowledge acquisition.18 Abrami et al wrote: “According to psychological views, critical thinking requires gaining mastery of a series of discrete skills or mental operations and dispositions that can be generalised across a variety of contexts.”18
By far the majority of the literature assumes that critical thinking is a general, context- independent set of skills: “judgement, analysis, evaluation, inference” and attitudes “inquisitive, well- informed… open- minded, flexible… prudent…” and so on.19 The effect of general critical thinking instruction is commonly assessed by measures such as the Watson–Glaser Critical Thinking Appraisal (WGCTA), de- signed to test general critical thinking processes.20 Studies use it ei- ther as an independent variable to show that scores on the WGCTA are correlated with in- course grades or other outcomes (suggesting critical thinking is a trait or stable skill), or as a dependent variable to show that some instructional intervention or just years of education improve test scores (suggesting it is educable).18 However, there is no unanimity on the subject; the review cited above18 specifically discusses the issue of general versus content- specific thinking, as well as multiple problems in interpretation.
4.2.2 | Metacognition
Metacognition refers to awareness of one's thinking. Most research on metacognition assumes that it consists of general self- regulatory activities that can be learned distinct from knowledge and that “help[ing] students monitor, reflect upon, and improve their strat- egies for learning and problem solving.”21 However, Bransford and Schwartz point out: “Research also suggests that metacognitive activities have strong knowledge requirements; they are not gen- eral skills that people learn “once and for all.” For example, without well- differentiated knowledge of the performance requirements of a particular task, people cannot accurately assess whether they are prepared to perform that task.”11
Thus, although descriptions of metacognition and critical think- ing emphasise generality, and instruction and assessment tend to look at general processes, there are occasional acknowledgements of the role of knowledge.
4.2.3 | Dual process theories and cognitive biases
Dual process theories posit two underlying thinking processes: a fast, unconscious, contextually bound process (System 1 or Type 1) and a slow, conscious, effortful, decontextualised process (System 2 or Type 2).22 These theories are descriptive, not predictive; they de- scribe how ambiguous problems are solved rather than prescribing how they should be solved or predicting how they will be solved. It is quite germane to understanding the source of the myth that many authors have interpreted this theory to offer a predictive model of cognitive error.
The dominant dual process theory, called “default- interventionist,” posits System 1 as the default strategy and System 2 as a backup with which to intervene as appropriate to correct the inevitable errors of System 1. System 1 reasoning makes heavy use of cognitive shortcuts or “heuristics” as a consequence of the limita- tions of human information processing. These in turn are expected to be error- prone and to lead to bias.23 Most errors are presumed to arise in System 1 as a consequence of biases in System 1 and can only be resolved using System 2. Dual process models of diagnosis amount to a direct test of the effectiveness of general, analytical methods (System 2) against strategies that depend on local content and contextual knowledge (System 1).
The assumption that errors originate in System 1 and are cor- rected by System 2 runs directly counter to the findings discussed earlier in the Enlightenment section, in which simple strategies based on elaborate specific knowledge consistently outperform more general, knowledge- lean strategies. Although it is obviously extreme to suggest that System 2 strategies are characterised by rationality, not specific knowledge, domain knowledge does not fea- ture prominently in this discourse.
The claim that errors arise uniquely in System 1 has been chal- lenged by some “dual processing” theorists. Evans and Stanovich state unequivocally: “Perhaps the most persistent fallacy… is the idea that Type 1 processes (intuitive, heuristic) are responsible for all
13652923, 2020, 1, D ow
nloaded from https://onlinelibrary.w
iley.com /doi/10.1111/m
edu.13872, W iley O
nline L ibrary on [03/07/2023]. See the T
erm s and C
onditions (https://onlinelibrary.w iley.com
/term s-and-conditions) on W
iley O nline L
ibrary for rules of use; O A
articles are governed by the applicable C reative C
om m
ons L icense
| 69MONTEIRO ET al.
bad thinking and that Type 2 processes (reflective, analytic) neces- sarily lead to correct responses. Thus, various forms of dual process theory have blamed Type 1 processing for cognitive biases in reason- ing and judgement research… Correspondingly, logical reasoning, ra- tional decision making, and nonstereotypical judgements have been attributed to Type 2 processing.”22
Although the notion that errors originate in System 1 and are ameliorated in System 2 is pervasive, recent theory and evidence suggest otherwise.
4.3 | Medical education
4.3.1 | Critical thinking
Similar to the psychology literature, some studies examine the rela- tionship between scores on a critical thinking test and outcomes such as clinical decision making or clerkship performance,24-26 implicitly assuming that critical thinking is trait- like. Alternatively, studies look for improvement in critical thinking with years of education,27 which presumes it is skill- like, gradually increasing over time. In general, as- sociations are modest, leading authors of review articles to conclude “limited concurrent validity,”28 “evidence… is still unsubstantiated,”24 and “results… are mixed and contradictory.”28
4.3.2 | Metacognition
Within medicine, awareness of one's own thinking has been trans- lated into the act of reflection. Many education programmes incor- porate formal and informal reflective activities to help develop skills related to professionalism. Formally, being a reflective practitioner or having a reflective professional practice is associated with the classic work of Donald Schön, who emphasises activity that allows examination of one's actions and thoughts from the near “reflection in action” or distant “reflection on action” past.29 Typically, this activ- ity occurs organically, in response to complexity, ambiguity or uncer- tainty.30 This has been interpreted by some authors to suggest that a general strategy of meta- awareness and reflection used routinely will improve performance.31,32
We are aware of very few attempts to operationalise reflection and conduct systematic study of its effectiveness. One exception is the programme of research initiated by Mamede et al.33-37 However, in their studies, “reflection” is designed as a strategy to mobilise rele- vant knowledge using questions like “What features go against your diagnosis?” and “What other diagnoses might be relevant?”
Several studies have incorporated these specific “reflection” instructions in either cross- sectional designs (in which one group uses some variant on a reflective protocol and another group does not) or longitudinal designs (in which participants initially go through cases quickly and then are encouraged to revisit the cases). The structured strategy33-37 shows relatively consistent results—a benefit for simple cases with students and complex cases with residents. Other studies evaluating a less structured approach (ie instructions to simply take another look or to be
systematic, consistent with realistic constraints of practice) have shown no effect for simple or complex cases.38,39
Moreover, physicians and students have difficulty recognis- ing when they have made an error.38 For example, in a study by Monteiro et al,38 physicians were unable to determine which cases required further reflection. Similarly, Friedman et al found that stu- dents, residents and physicians were consistently overconfident in their diagnoses.17,40
Metacognitive strategies to improve reasoning have received some attention in the literature, often with suggestions of im- proved outcomes resulting from training in self- monitoring.41,42 However, upon closer inspection these improved outcomes do not transfer to diagnostic accuracy. For example, targeted instruction in solving challenging syllogisms reduced overconfidence, but had no impact on accuracy.41 Specific instructions in metacognition focusing on the process used in diagnosis were shown to lead to improved “metacognitive accuracy” but no improvement in diag- nostic accuracy.42
4.3.3 | Dual process theories and cognitive biases
As indicated, a dual process model of thinking has become the ac- cepted theory of clinical reasoning. Central to the theory is the no- tion that successful reasoning reflects effective thinking processes and, conversely, that errors are a consequence of flawed reasoning, originating in cognitive heuristics. This involves two strong assump- tions about the nature of clinical reasoning, both of which are sub- ject to empirical testing.
Assumption 1 Almost all errors of diagnosis are a consequence of cognitive biases originating in System 1 thinking.43
The notion that System 1 thinking and cognitive biases are primary causes of diagnostic error has been described by many authors43-46 and features in the Institute of Medicine report.47 However, the supporting evidence is weak. Although the numbers of biases described in the literature range from 30 to 130, three sys- tematic reviews of cognitive biases in medicine identify a total of 24 biases for which there is evidence.48-50 However, these encom- pass not just diagnostic errors, but also errors of management and prognosis, as well as patient- related biases; only seven are related to diagnostic error. Moreover, only three biases—availability, confir- mation and hindsight—are cited in all three systematic reviews.48-50 Each has issues related to interpretation with respect to the asser- tion that cognitive bias is intimately linked to System 1 thinking.
Here, we examine the established definitions of three common biases.
Availability bias: the disposition to judge things as being more likely, or frequently occurring, if they readily come to mind. Thus, recent experience with a disease may inflate the likelihood of its being diagnosed.43
13652923, 2020, 1, D ow
nloaded from https://onlinelibrary.w
iley.com /doi/10.1111/m
edu.13872, W iley O
nline L ibrary on [03/07/2023]. See the T
erm s and C
onditions (https://onlinelibrary.w iley.com
/term s-and-conditions) on W
iley O nline L
ibrary for rules of use; O A
articles are governed by the applicable C reative C
om m
ons L icense
70 | MONTEIRO ET al.
Availability is one of the commonest cognitive biases identified by reviewers of diagnostic errors occurring in episodes of care.51,52 However, this is logically impossible. An observer, or an auditor review- ing a written case, has no way of knowing what “readily came to mind” to the clinician. For that matter, as System 1 thinking is characterised as an unconscious, intuitive process, neither does the clinician who man- aged the patient.
Confirmation bias: the tendency to look for confirm- ing evidence to support a diagnosis rather than look for disconfirming evidence to refute it, despite the latter often being more persuasive and definitive.43
Confirmation bias was first studied by Wason53 using sets of num- ber sequences generated by an analytical rule. In a clinical context, as the definition suggests, this arises in the systematic gathering and weighting of evidence to preferentially support hypotheses. This is not a consequence of System 1 reasoning.
Hindsight bias: knowing the outcome may profoundly influence the perception of past events and prevent a realistic appraisal of what actually occurred.43
Hindsight arises when the outcome is known. Thus, the determina- tion of underlying causes of error is a central issue for those reviewing clinical cases, but is not a problem for a clinician seeking a diagnosis. Hindsight bias is not related to System 1 thinking.
As one example of hindsight, Zwaan et al54 experimentally ma- nipulated scenarios with two equi- probable diagnoses and gave them to expert reviewers who were asked to identify biases. When the conclusion of the scenario did not align with the initial diagnostic approach, raters identified twice as many biases as when it was con- sistent with the diagnosis, an illustration of hindsight bias.
On close inspection, many of the definitions appear to overlap. An empirical question concerns the extent to which observers can consistently and reliably identify cognitive biases. The study by Zwaan et al54 systematically explored inter- rater agreement on the presence or absence of cognitive biases and demonstrated that reli- ability for six common biases was effectively zero.
There are also problems with the methods used to investigate biases. Two broad strategies have been used: retrospective observa- tion and experimental study. We have already discussed the problem of hindsight in retrospective review.
Experimental studies typically use written scenarios that are ex- plicitly designed to illustrate particular cognitive biases. The strategy is to create situations in which the most likely response based on experience is at variance with the normative (correct) response.55 The question then concerns the extent to which participants choose the intuitive or normative response.56
What makes this approach problematic is that the scenarios are atypical in three fundamental ways. Firstly, they are designed to contain one or more cognitive biases and hence the prevalence of bias in this population is 100%. Secondly, they are designed so that
the intuitive and normative responses are in opposition in order to detect the presence of bias, but this automatically makes them atyp- ical. Finally, they typically do not examine a relationship to expertise. However, expertise does affect cognitive bias; three studies have shown that bias disappears with expertise.57-59
Assumption 2 Because errors are a consequence of cognitive biases, error re- duction strategies should focus on approaches that help clinicians identify cognitive biases and effectively use System 2 reasoning to correct the errors inherent in System 1 thinking (eg cognitive forcing strategies).43,51,60
Cognitive forcing strategies are general strategies for im- proving metacognition and reasoning “in the moment.”61 The basic concept is to increase self- awareness of one's own think- ing and identify potential errors by avoiding common cognitive biases.43,60,61
Perhaps the simplest “cognitive forcing strategy” is some form of instruction to caution the clinician to be systematic, to slow down or to consider alternatives. In a number of studies, rapid diagnosis has been compared with slower systematic reflec- tion.38,39,62-64 A uniform finding is that instructions to be system- atic and thorough result in longer processing time, but have no impact on diagnostic errors.
Several reviews have examined the effectiveness of strat- egies designed to teach students to recognise biases.32,49,50,65 Training increases awareness of cognitive biases.66-68 However, studies of the effect of debiasing on diagnostic errors have been negative.69-71
4.4 | Summary
The evidence shows that generalised, content- independent strate- gies—debiasing, reflection or whatever—to reduce errors have no or minimal effectiveness for the simple reason that errors derive not from inadequate thinking skills but from inadequate knowledge. Reflection strategies may have a small benefit, although effects are uneven.65 Debiasing strategies have shown uniformly null effects.65 Thus, attempts to encourage clinicians to reflect on the process or reasoning, or to apply general analytical approaches, may not be ef- fective for the simple reason that they focus on analytical skills, or on weak methods, not knowledge.
As Dhaliwal said: “If you have not heard about myasthenia gravis, you cannot cognitively debias your way into that diagnosis. […] In the realm of expert performance, knowledge is king.”72
5 | DISCUSSION
In light of the accumulated evidence, we must address why the myth of general skills has persisted in medical education for half a cen- tury. One reason may be that it offers a shortcut to mastery of the many areas of knowledge required in the practice of medicine. In
13652923, 2020, 1, D ow
nloaded from https://onlinelibrary.w
iley.com /doi/10.1111/m
edu.13872, W iley O
nline L ibrary on [03/07/2023]. See the T
erm s and C
onditions (https://onlinelibrary.w iley.com
/term s-and-conditions) on W
iley O nline L
ibrary for rules of use; O A
articles are governed by the applicable C reative C
om m
ons L icense
| 71MONTEIRO ET al.
2002, Croskerry wrote: “…we need to know whether we can make clinicians better decision makers without simply waiting for the improved judgement that comes with the benefit of accumulated experience.”60
Medical education curricula suffer from restricted time and re- sources. Perhaps medical educators and curriculum designers are inevitably drawn towards generalisable skills that have the illusion of transfer, thereby avoiding the necessity of learning and practice in myriad different knowledge domains. The myth persists because everyone is too busy.
The movement towards competency- based education illus- trates the pervasiveness of the myth. Although some competen- cies are based on specific knowledge, such as a lumbar puncture or intubation, many are framed around content and the context- free competencies to be acquired, such as “performing a complete and systematic history” or “demonstrating interprofessional collabora- tion skills.” Underlying the “good history” is an implicit assumption of generalisability. It should not matter how or even if the approach is paired with specific clinical content; the approach should transfer easily. However, the evidence we have presented, from both psy- chology and medical education, casts doubt on the generalisability of any cognitive skill.
Finally, the assumption of debiasing strategies is that the root cause of error is cognitive bias. This has some appeal: it absolves the clinician from blame; cognitive bias is simply part of the human condition. However, if cognitive bias is so central to diagnostic error, then training in critical thinking or cognitive debiasing should result in improved reasoning skill and reduced error. Evidence for such an effect is conspicuously absent.
The consequence of the persistence of the myth is that valuable instructional and learning time may be devoted to mastering inter- ventions that, in the end, are not effective. Although it may be use- ful to draw attention to the prevalence of diagnostic error, the long litany of cognitive biases has no added value. Ample evidence has demonstrated repeatedly that identifying biases is not equivalent to reducing error.
Reducing error rates is a laudable goal. However, strategies based on the broad assumption that this can be achieved by tun- ing up a general problem- solving process in order to compensate for the perceived defects of intuitive reasoning have been shown to be ineffective. Progress in this area is more likely to arise from accept- ing the power of System 1 reasoning and devising strategies such as interleaved practice to improve the breadth and depth of the experi- ential knowledge used by System 1.
6 | CONCLUSIONS
We have shown that the current models of reasoning and thinking popularised in medical education perpetrate a theoretical position that is inconsistent with the evidence accrued for over a century in psychology and half a century in medicine. There is no justification for the position that knowledge- rich, efficient strategies, as used by
experts, can be viewed as inferior to general analytical and “rational” approaches.
We are not the first to challenge the assertion that rational gen- eralisable rules are more effective than heuristics derived from case experience.73,74 Dreyfus, acknowledged as the guru of artificial in- telligence, wrote: “We must be prepared to abandon the traditional view that runs from Plato to… Piaget … Chomsky… that a beginner starts with specific cases and… as he or she becomes more profi- cient, abstracts and interiorises more and more sophisticated rules. It might turn out that skill acquisition moves in just the opposite di- rection: from abstract rules to particular cases.”75
The deification of rational, decontextualised strategies is consis- tent with some claims that they represent an evolutionary adapta- tion from contextualised heuristic approaches. It would be a marvel indeed if human cognition were able to evolve purely through men- tal effort. In the current era of specialised expertise, there is every reason to challenge the assertion we introduced at the beginning of this paper: that the central focus of education should be to incul- cate general skills like critical thinking, problem solving, clinical rea- soning and reflection. Perhaps Homo sapiens—“wise man”—should be replaced by “Homo sciens”—“knowing man”—at the apex of evolution.
ACKNOWLEDG EMENTS
None.
CONFLIC T OF INTERE S T
None.
AUTHOR CONTRIBUTIONS
SM contributed to the conceptual design and layout of the manu- script. SM wrote the first draft and contributed to significant revi- sions of subsequent drafts. JS contributed to the conceptual design and layout of the manuscript. He contributed edits to the first draft of the manuscript. MS contributed to the conceptual design of the manuscript. MS contributed to revisions of the first draft of the man- uscript. GN contributed to the conceptual design and layout of the manuscript. GN contributed to significant revisions of subsequent drafts.
E THIC AL APPROVAL
Not applicable.
ORCID
Sandra Monteiro https://orcid.org/0000-0001-8723-5942
Matthew Sibbald https://orcid.org/0000-0002-0022-2370
Geoff Norman https://orcid.org/0000-0003-1053-7998
13652923, 2020, 1, D ow
nloaded from https://onlinelibrary.w
iley.com /doi/10.1111/m
edu.13872, W iley O
nline L ibrary on [03/07/2023]. See the T
erm s and C
onditions (https://onlinelibrary.w iley.com
/term s-and-conditions) on W
iley O nline L
ibrary for rules of use; O A
articles are governed by the applicable C reative C
om m
ons L icense
72 | MONTEIRO ET al.
R E FE R E N C E S
1. Willingham DT. Critical thinking. Am Ed. 2007;31(3):8-19. 2. Lehman DR, Lempert RO, Nisbett RE. The effects of graduate train-
ing on reasoning: formal discipline and thinking about everyday- life events. Am Psychol. 1988;43(6):431-442.
3. Newell A, Simon HA. Human Problem Solving. Englewood Cliffs, NJ: Prentice-Hall; 1972.
4. Perkins DN, Salomon G. Are cognitive skills context- bound? Ed Res. 1989;18(1):16-25.
5. Searle JR. Minds, brains, and programs. Behav Brain Sci. 1980;3(3):417-424.
6. Elstein AS, Shulman LS, Sprafka SA. Medical Problem Solving an Analysis of Clinical Reasoning. Cambridge, MA: Harvard University Press; 1978.
7. Barrows HS, Norman GR, Neufeld VR, Feightner JW. The clinical reasoning of randomly selected physicians in general medfvical practice. Clin Invest Med. 1982;5(1):49-55.
8. Neufeld VR, Norman GR, Feightner JW, Barrows HS. Clinical problem- solving by medical students: a cross- sectional and longi- tudinal analysis. Med Educ. 1981;15(5):315-322.
9. Elstein AS, Shulman LS, Sprafka SA. Medical problem solving: a ten- year retrospective. Eval Health Prof. 1990;13(1):5-36.
10. Ericsson KA, Krampe RT, Tesch-Römer C. The role of deliberate practice in the acquisition of expert performance. Psychol Rev. 1993;100(3):363.
11. Bransford JD, Schwartz DL. Chapter 3: Rethinking trans- fer: a simple proposal with multiple implications. Rev Res Educ. 1999;24(1):61-100.
12. Schoenfeld AH. Mathematical Problem Solving. New York, NY: Academic Press; 1985.
13. Norman G, Young M, Brooks L. Non- analytical models of clinical rea- soning: the role of experience. Med Educ. 2007;41(12):1140-1145.
14. Bordage G, Zacks R. The structure of medical knowledge in the memories of medical students and general practitioners: categories and prototypes. Med Educ. 1984;18(6):406-416.
15. Patel VL, Groen GJ. Knowledge based solution strategies in medical reasoning. Cog Sci. 1986;10(1):91-116.
16. Miller RA, Pople HE Jr, Myers JD. Internist- I, an experimental computer- based diagnostic consultant for general internal medi- cine. New Engl J Med. 1982;307(8):468-476.
17. Friedman CP, Elstein AS, Wolf FM, et al. Enhancement of clinicians’ diagnostic reasoning by computer- based consultation: a multisite study of 2 systems. JAMA. 1999;282(19):1851-1856.
18. Abrami PC, Bernard RM, Borokhovski E, et al. Instructional inter- ventions affecting critical thinking skills and dispositions: a stage 1 meta- analysis. Rev Educ Res. 2008;78(4):1102-1134.
19. Facione PA. Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction. Millbrae, CA: The California Academic Press; 1990.
20. Watson G, Glaser EM. Watson-Glaser Critical Thinking Appraisal. San Antonio: PsychCorp; 1980.
21. Flavell JH. Metacognitive aspects of problem solving. In: Resnick LB, ed. The Nature of Intelligence. Hillsdale, NJ: Lawrence Erlbaum Associates; 1976:231-235.
22. Evans JS, Stanovich KE. Dual- process theories of higher cognition: advancing the debate. Persp Psychol Sci. 2013;8(3):223-241.
23. Kahneman D. Thinking, Fast and Slow. New York, NY: Farrar, Straus and Giroux; 2011.
24. Lee DS, Abdullah KL, Subramanian P, Bachmann RT, Ong SL. An integrated review of the correlation between critical thinking ability and clinical decision- making in nursing. J Clin Nurs. 2017 Dec;26(23–24):4065-4079.
25. Scott JN, Markert RJ. Relationship between critical thinking skills and success in preclinical courses. Acad Med. 1994;69(11):920-924.
26. Ross D, Loeffler K, Schipper S, Vandermeer B, Allan GM. Do scores on three commonly used measures of critical think- ing correlate with academic success of health professions trainees? A systematic review and meta- analysis. Acad Med. 2013;88(5):724-734.
27. Scott JN, Markert RJ, Dunn MM. Critical thinking: change during medical school and relationship to performance in clinical clerk- ships. Med Educ. 1998;32(1):14-18.
28. Adams BL. Nursing education for critical thinking: an integrative review. J Nurs Educ. 1999;38(3):111-119.
29. Mann K, Gordon J, MacLeod A. Reflection and reflective practice in health professions education: a systematic review. Adv Health Sci Ed. 2009;14(4):595-621.
30. Mamede S, Schmidt HG. The structure of reflective in medicine. Med Educ. 2004;38(12):1302-1308.
31. Chisholm CD, Dornfeld AM, Nelson DR, Cordell WH. Work in- terrupted: a comparison of workplace interruptions in emer- gency departments and primary care offices. Ann Emerg Med. 2001;38(2):146-151.
32. Graber ML, Kissam S, Payne VL, et al. Cognitive interventions to reduce diagnostic error: a narrative review. BMJ Qual Saf. 2012;21(7):535-557.
33. Mamede S, Schmidt HG, Penaforte JC. Effects of reflec- tive practice on the accuracy of medical diagnoses. Med Educ. 2008;42(5):468-475.
34. Mamede S, van Gog T, van den Berge K, et al. Effect of availability bias and reflective reasoning on diagnostic accuracy among internal medicine residents. JAMA. 2010 Sep 15;304(11):1198-1203.
35. Mamede S, Splinter TA, van Gog T, Rikers RM, Schmidt HG. Exploring the role of salient distracting clinical features in the emer- gence of diagnostic errors and the mechanisms through which re- flection counteracts mistakes. BMJ Qual Saf. 2012;21:295-300.
36. Mamede S, Schmidt HG. Reflection in diagnostic reasoning: what really matters? Acad Med. 2014;89:959-960.
37. Mamede S, van Gog T, van den Berge K, van Saase JL, Schmidt HG. Why do doctors make mistakes? A study of the role of salient dis- tracting clinical features. Acad Med. 2014;89:114-120.
38. Monteiro SD, Sherbino J, Patel A, Mazzetti I, Norman GR, Howey E. Reflecting on diagnostic errors: taking a second look is not enough. J Gen Intern Med. 2015;30(9):1270-1274.
39. Monteiro SD, Sherbino JD, Ilgen JS, et al. Disrupting diagnostic reasoning: do interruptions, instructions, and experience affect the diagnostic accuracy and response time of residents and emergency physicians? Acad Med. 2015;90(4):511-517.
40. Friedman CP, Gatti GG, Franz TM, et al. Do physicians know when their diagnoses are correct? J Gen Intern Med. 2005;20(4):334-339.
41. Ackerman R, Thompson VA. Meta- reasoning: monitor- ing and control of thinking and reasoning. Trends Cogn Sci. 2017;21(8):607-617.
42. Feyzi-Behnagh R, Azevedo R, Legowski E, Reitmeyer K, Tseytlin E, Crowley RS. Metacognitive scaffolds improve self- judgments of accuracy in a medical intelligent tutoring system. Instr Sci. 2014;42(2):159-181.
43. Croskerry P. The importance of cognitive errors in diagnosis and strategies to minimize them. Acad Med. 2003;7(8):775-780.
44. Royce CS, Hayes MM, Schwartzstein RM. Teaching critical thinking: a case for instruction in cognitive biases to reduce diagnostic errors and improve patient safety. Acad Med. 2018;94(2):187-194.
45. Redelmeier DA. The cognitive psychology of missed diagnoses. Ann Intern Med. 2005;142(2):115-120.
46. Chapman GB, Elstein AS. Cognitive processes and biases in med- ical decision making. Decision making in health care: theory, psy- chology, and applications. In: Chapman GB, Sonnenberg FA, eds. Decision Making in Health Care: Theory, Psychology, and Applications. New York, NY: Cambridge University Press; 2000:183-210.
13652923, 2020, 1, D ow
nloaded from https://onlinelibrary.w
iley.com /doi/10.1111/m
edu.13872, W iley O
nline L ibrary on [03/07/2023]. See the T
erm s and C
onditions (https://onlinelibrary.w iley.com
/term s-and-conditions) on W
iley O nline L
ibrary for rules of use; O A
articles are governed by the applicable C reative C
om m
ons L icense
| 73MONTEIRO ET al.
47. Corrigan JM. Crossing the quality chasm. In: Corrigan JM, ed. Building a Better Delivery System: A new Engineering/Health Care Partnership. Washington, DC: National Academies Press; 2005:95-97.
48. Blumenthal-Barby JS, Krieger H. Cognitive biases and heuristics in medical decision making: a critical review using a systematic search strategy. Med Decis Making. 2015;35(4):539-557.
49. Saposnik G, Redelmeier D, Ruff CC, Tobler PN. Cognitive biases associated with medical decisions: a systematic review. BMC Med Inform Decis. 2016;16(1):138.
50. Lambe KA, O'Reilly G, Kelly BD, Curristan S. Dual- process cognitive interventions to enhance diagnostic reasoning: a systematic review. BMJ Qual Saf. 2016;25:808-820.
51. Graber ML, Franklin N, Gordon R. Diagnostic error in internal med- icine. Arch Intern Med. 2005;165(13):1493-1499.
52. Zwaan L, Thijs A, Wagner C, van der Wal G, Timmermans DRM. Relating faults in diagnostic reasoning with diagnostic errors and patient harm. Acad Med. 2012;87(2):149-156.
53. Wason PC. On the failure to eliminate hypotheses in a conceptual task. Q J Exp Psychol. 1960;12(3):129-140.
54. Zwaan L, Monteiro S, Sherbino J, Ilgen J, Howey B, Norman G. Is bias in the eye of the beholder? A vignette study to assess recog- nition of cognitive biases in clinical case workups. BMJ Qual Saf. 2017;26(2):104-110.
55. Lopes LL. The rhetoric of irrationality. Theor Psychol. 1991;1(1):65-82. 56. Lopes LL. Three misleading assumptions in the customary rhetoric
of the bias literature. Theor Psychol. 1992;2(2):231-236. 57. Weber EU, Böckenholt U, Hilton DJ, Wallace B. Determinants of di-
agnostic hypothesis generation: effects of information, base rates, and experience. J Exp Psych: Learn. 1993;19(5):1151-1164.
58. Christensen C, Heckerling PS, Mackesy ME, Bernstein LM, Elstein AS. Framing bias among expert and novice physicians. Acad Med. 1991;66(9):76-78.
59. Hatala RA, Norman GR, Brooks LR. The effect of clinical history on physician's ECG interpretation skills. In: Scherpbier AJJA, van der Vleuten CPM, Rethans JJ, van der Steeg AFW. Advances in Medical Education. Maastricht: Springer, Dordrecht; 1997: 608-610.
60. Croskerry P. Achieving quality in clinical decision making: cognitive strategies and detection of bias. Acad Emerg Med. 2002;9(11):1184-1204.
61. Croskerry P. Cognitive forcing strategies in clinical decisionmaking. Ann Emerg Med. 2003;41(1):110-120.
62. Norman G, Sherbino J, Dore K, et al. The etiology of diagnostic er- rors: a controlled trial of system 1 versus system 2 reasoning. Acad Med. 2014;89(2):277-284.
63. Sherbino J, Dore KL, Wood TJ, et al. The relationship between re- sponse time and diagnostic accuracy. Acad Med. 2012;87(6):785-791.
64. Ilgen JS, Bowen JL, Yarris LM, Fu R, Lowe RA, Eva K. Adjusting our lens: can developmental differences in diagnostic reasoning be harnessed to improve health professional and trainee assessment? Acad Emerg Med. 2011;18(suppl 2):S79-S86.
65. Norman GR, Monteiro SD, Sherbino J, Ilgen JS, Schmidt HG, Mamede S. The causes of errors in clinical reasoning: cognitive biases, knowledge deficits, and dual process thinking. Acad Med. 2017;92(1):23-30.
66. Reilly JB, Ogdie AR, Von Feldt JM, Myers JS. Teaching about how doctors think: a longitudinal curriculum in cognitive bias and diag- nostic error for residents. BMJ Qual Saf. 2013;22(12):1044-1050.
67. Bond WF, Deitrick LM, Arnold DC, et al. Using simulation to instruct emergency medicine residents in cognitive forcing strategies. Acad Med. 2004;79(5):438-446.
68. Ogdie AR, Reilly JB, Pang MW, et al. Seen through their eyes: res- idents’ reflections on the cognitive and contextual components of diagnostic errors in medicine. Acad Med. 2012;87(10):1361-1367.
69. Sherbino J, Dore KL, Siu E, Norman GR. The effectiveness of cogni- tive forcing strategies to decrease diagnostic error: an exploratory study. Teach Learn Med. 2011;23(1):78-84.
70. Sherbino J, Kulasegaram K, Howey E, Norman G. Ineffectiveness of cognitive forcing strategies to reduce biases in diagnostic reason- ing: a controlled trial. Can J Emerg Med. 2014;16(1):34-40.
71. Shimizu T, Matsumoto K, Tokuda Y. Effects of the use of differen- tial diagnosis checklist and general de- biasing checklist on diagnos- tic performance in comparison to intuitive diagnosis. Med Teach. 2013;35:e1218-e1229.
72. Dhaliwal G. Premature closure? Not so fast. BMJ Qual Saf. 2017;26:87-89.
73. Marewski JN, Gaissmaier W, Gigerenzer G. We favor formal models of heuristics rather than lists of loose dichotomies: a reply to Evans and Over. Cogn Process. 2010;11:177-179.
74. Boreham NC. The dangerous practice of thinking. Med Educ. 1994;28(3):172-179.
75. Dreyfus HL. Skilled Coping as Higher Intelligibility in Heidegger's’ Being and Time’. London: Uitgeverij Van Gorcum; 2008.
How to cite this article: Monteiro S, Sherbino J, Sibbald M, Norman G. Critical thinking, biases and dual processing: The enduring myth of generalisable skills. Med Educ. 2020;54:66– 73. https ://doi.org/10.1111/medu.13872
13652923, 2020, 1, D ow
nloaded from https://onlinelibrary.w
iley.com /doi/10.1111/m
edu.13872, W iley O
nline L ibrary on [03/07/2023]. See the T
erm s and C
onditions (https://onlinelibrary.w iley.com
/term s-and-conditions) on W
iley O nline L
ibrary for rules of use; O A
articles are governed by the applicable C reative C
om m
ons L icense
