RESEARCH TO PRACTICE
Increasing theUseofEvidence-Based Teaching inSTEMHigherEducation:
AComparisonofEightChangeStrategies
MauraBorregoa and CharlesHendersonb
aVirginia Tech,bWesternMichiganUniversity
Abstract Background Prior efforts have built a knowledge base of effective undergraduate STEM pedagogies, yet rates of implementation remain low. Theories from higher education, man- agement, communication, and other fields can inform change efforts but remain largely inaccessible to STEM education leaders, who are just beginning to view change as a scholarly endeavor informed by the research literature.
Purpose This article describes the goals, assumptions, and underlying logic of selected change strategies with potential relevance to STEM higher education settings for a target audience of change agents, leaders, and researchers.
Scope/Method This review is organized according to the Four Categories of Change Strat- egies model developed by Henderson, Beach, and Finkelstein (2011). We describe eight strat- egies of potential practical relevance to STEM education change efforts (two from each category). For each change strategy, we present a summary with key references, discuss their applicability to STEM higher education, provide a STEM education example, and discuss implications for change efforts and research.
Conclusions Change agents are guided, often implicitly, by a single change strategy. These eight strategies will expand the repertoire of change agents by helping them consider change from a greater diversity of perspectives. Change agents can use these descriptions to design more robust change efforts. Improvements in the knowledge and theory base underlying change strategies will occur when change agents situate their writing about change initiatives using shared models, such as the one presented in this article, to make their underlying assumptions about change more explicit.
Keywords curriculum change; instructional change; theories of change
Introduction Increasingly, high-profile organizations are calling for widespread improvement in undergradu- ate science, technology, engineering, and math (STEM) education. These calls are frequently framed in terms of increasing the number, diversity, and quality of STEM graduates (American Society for Engineering Education [ASEE], 2009, 2012; Hawwash, 2007; King, 2008; National Academy of Engineering [NAE], 2004; President’s Council of Advisors on Science and Tech- nology [PCAST], 2012). While these broad goals are not new, growing attention is being paid
Journal of Engineering Education VC 2014 ASEE. http://wileyonlinelibrary.com/journal/jee April 2014, Vol. 103, No. 2, pp. 220–252 DOI 10.1002/jee.20040
to the instructional practices of STEM faculty, specifically to encourage more widespread use of instructional strategies grounded in the research on how students learn (NRC, 2012).
Tremendous investment and related efforts over the past few decades have built up a sub- stantial knowledge base about STEM learning and many effective pedagogies and interven- tions (Borrego, Froyd, & Hall, 2010; NRC, 2012; Prince & Felder, 2006). Yet these prestigious organizations are increasingly expressing dissatisfaction with the rate of imple- mentation, adoption, and scale-up of research-based instructional strategies (ASEE, 2009; 2012; NRC, 2012; PCAST, 2012). It has become painfully clear that higher education change processes are at least as complex as the pedagogies and learning processes they seek to promote. STEM education change agents, leaders, and researchers are just beginning to view change as a scholarly endeavor that can and should be informed by the research litera- ture. While fields such as management, higher education, and communication have devel- oped a wealth of literature to inform such change efforts, this knowledge remains largely inaccessible to STEM education leaders and researchers.
This research review describes the Four Categories of Change Strategies model previously developed by Henderson, Beach, and Finkelstein that allows for categorization of change strategies (Henderson, Beach, & Finkelstein, 2011; Henderson, Finkelstein, & Beach, 2010) and uses this model to describe and compare eight change strategies that are relevant to STEM higher education settings. The target audience is STEM higher education change agents, leaders, and researchers. This article, organized according to the Four Categories of Change Strategies model, is not meant to be an exhaustive literature review of possible change strategies, but rather is meant to highlight and provide an overview of what we see as some important perspectives on change. Two strategies were selected to illustrate each cate- gory of change strategy, and we present a STEM education example for each. These selec- tions were based on our perception of the current or potential use of each strategy as well as our desire to have the two strategies represent significantly different ways of operating within each category. The Discussion and Conclusion sections focus on implications for change agents and future directions for research.
This review focuses on higher education. We acknowledge that a significant body of literature describes extensive change efforts related to precollege education (e.g., Hargreaves, Lieberman, Fullan, & Hopkins, 2009; Sykes, Schneider, & Plank, 2012). Our goal, however, is not to sum- marize this work and translate it to higher education settings, because they feature greater instructor autonomy, far less government control, and limited reliance on standardized tests for accountability than in precollege settings. We follow a United States, higher education- based definition of STEM, to encompass primarily biology, chemistry, engineering, geosciences, mathematics, and physics. When our examples or claims apply primarily to engineering edu- cation, we use “engineering education” instead of “STEM education.”
TheoryinSTEMEducationResearch There are two primary reasons a review of change strategies in STEM higher education is needed. First, change is not traditionally a domain that STEM leaders have thought of as informed by theory or literature. The relevant literature on change in higher education is not necessarily accessible to those who need to apply it. This literature is scattered in disciplines and journals outside STEM, and many ideas, although promising, are understudied in the higher education context. Additionally, work being done in instructional change in one STEM discipline is not necessarily connected to similar work in other STEM disciplines.
Increasing Evidence-Based Teaching in STEM Education 221
Chapter 8 of the Discipline-Based Educational Research (DBER) report calls for more research into the extent to which educational research has influenced undergraduate instructional practices within and across STEM disciplines (NRC, 2012).
Second, engineering educators and engineering education researchers have limited experi- ence with education and social science theories. Descriptions of theory use in the DBER report imply that engineering lags far behind physics and chemistry education in its engagement with learning theory (NRC, 2012). Engineering education scholars (Beddoes & Borrego, 2011; Borrego, 2007; Koro-Ljungberg & Douglas, 2008) have called for more explicit use of theory in educational research, yet there are few detailed discussions in the engineering education lit- erature about what theory means and how it is best applied in engineering education research and practice. Flyvbjerg (2001) argues that knowledge accumulation is fundamentally different in the natural and social sciences in ways that link theory and methodology much more closely in the social sciences. This difference means that administrators and instructors trained in technical engineering may be unfamiliar with the ways in which theory informs decisions about educational methods.
In the education literature, Creswell (2009) defines theory in quantitative educational research as “an interrelated set of constructs (or variables) formed into propositions, or hypothe- ses, that specify the relationship among variables (typically in terms of magnitude or direction) . . . it helps to explain (or predict) phenomena that occur in the world” (p. 51). Using theory to inform interventions and investigations helps us focus on the most important factors to effect the desired changes, whether they are related to student learning or instructional change. In qualitative research, theory is closely linked to choices guiding methodology (Crotty, 1998). Theory helps link the results of an otherwise isolated study to a broader body of research: “Theoretically grounded work,” according to Beddoes and Borrego (2011), “connects researchers, allows generalizations across studies, and advances the field of engi- neering education by avoiding re-inventing the wheel” (p. 283). Thus, linking change efforts to existing theory ensures that new initiatives are informed by and build upon prior efforts. The pressing economic and environmental challenges facing the world and the need to prepare engineers to meet these challenges mean we simply cannot afford to rediscover key aspects of change with each new initiative.
ChangeTheories Higher education leaders have been considering questions of how to change faculty instruc- tional practices for decades, and researchers have attempted to make sense of the literature on instructional change for nearly as long (e.g., Emerson & Mosteller, 2000; Levinson-Rose & Menges, 1981; Weimer & Lenze, 1997). Contributing to the complexity is the variety of levels of focus, including individual instructors, departments, institutions, and broader edu- cation systems. More recent summaries and reviews attempt to capture the complex higher education change processes that bridge individual and organizational scales (Amundsen & Wilson, 2012; Henderson et al., 2011; Kezar, 2001; Seymour, 2002; Stes, Min-Leliveld, Gijbels, & Van Petegem, 2010). While these reviews have helped to situate different per- spectives on change with respect to each other and to identify the blind spots of a particular approach, the field has not yet developed a coherent understanding of what perspectives are most effective in a given set of circumstances. These reviews tell us that there are many per- spectives and approaches to change that focus on certain aspects of complex higher education systems. We know that certain approaches are a better fit for certain situations, but we do
222 Borrego & Henderson
not have a systematic way of thinking about which change perspectives are most appropriate in a given situation. Our review does not attempt to solve this problem; rather, we are describing (by comparing and contrasting) different approaches as they apply specifically to engineering and STEM higher education. We argue that change in higher education has not been conceptualized well enough to have its own specific theories, and that articulating the underlying logic of specific change strategies will help develop theory. To frame this discus- sion, we employ the Four Categories of Change Strategies model developed by Henderson and colleagues, an interdisciplinary team that included physics education researchers who explicitly focused their analysis on change in STEM higher education (Henderson et al., 2010, 2011). This model provides a way to categorize change strategies; it is necessarily less complex than reality in order to make sense of reality. It describes but does not explain nor predict the effectiveness of various change strategies (as a theory would be expected to do). In order to be useful, the model must be applied. We explore the change-strategies model by describing eight specific strategies, two in each of four change categories and highlight the contribution of this model to relating various strategies to one another. Situating the strate- gies will help change agents articulate the underlying logic and assumptions of their efforts; doing so will support the eventual development of theory.
FourCategoriesof ChangeStrategies
On the basis of a literature review of 191 journal articles published between 1995 and 2008, Henderson et al. (2010, 2011) developed the Four Categories of Change Strategies model to categorize strategies that have been used to conceptualize or to create change in undergradu- ate STEM instruction. The similarity of these categories to those developed through an independent review of an overlapping literature base (Amundsen & Wilson, 2012) suggests that the four categories are robust and meaningful. The four categories, shown in Figure 1, are based on two categorization criteria.
The first criterion focuses on the aspect of the system that is to be changed; these aspects range from individual instructors to environments and structures. Our use of the terms instructor and faculty throughout this article is meant to include instructors at all levels, including temporary and part-time instructors and tenure-track and tenured professors. Some of the organizations that have applied these strategies have, however, only focused on the pressures and reward systems for tenured and tenure-track faculty members.
The second criterion focuses on whether the intended outcome of the change strategy is known in advance, that is, whether the result of the change process is prescribed or emer- gent. For example, using a specific set of curricular materials, textbook, technology (clickers), or assessment tool is a prescribed outcome.
In our experience, it is common for reviewers to argue that there is significant overlap among the four categories created by these criteria. For example, enacting a policy has impacts on individuals and may include strategies to encourage individuals to support the strategy. Similarly, creating a teaching and learning center to develop reflective teachers requires organizational and administrative action. It is important to remember, however, that while change efforts can and perhaps should involve multiple strategies, the articles analyzed by Henderson et al. (2010, 2011) and later by Amundsen and Wilson (2012) tended to focus their discussion on one primary strategy. These criteria were the salient ones distinguishing the set of articles analyzed. We are not claiming that these categories and their strategies have
Increasing Evidence-Based Teaching in STEM Education 223
no interconnections and overlaps; rather, we are trying to make distinctions that assist in relating the various common change strategies to one another. Again, this model is, like any model, less complex than the reality it attempts to describe.
Figure 1 clarifies the role of the change agent in each of the four categories. Henderson et al. (2011) and Henderson, Beach, Finkelstein, and Larson (2008) found that each cate- gory was closely associated with a different community of professionals and their publishing venues. The Curriculum and Pedagogy category was dominated by STEM instructors including DBER scholars. Most of the Reflective Teachers category publications were writ- ten by faculty developers, for example, teaching and learning center staff. Most Policy publi- cations reflected the interests of higher education researchers, and the few Shared Vision publications were authored by administrators describing their practices.
In the following sections, we discuss two change strategies in each of the four change categories. Information for each strategy is summarized in Table 1. For each change strategy, we present a sum- mary with key references, discuss their potential applicability to STEM higher education, and
Figure 1 Change theories mapped to the four categories of change strategies. Figure adapted from Henderson et al. (2011). The italicized text in each box lists the eight change strategies discussed in further detail in the text.
224 Borrego & Henderson
discuss implications for change efforts and engineering education research on change. Then we provide an example of how the strategy has been applied in STEM higher education.
UnderlyingLogicof ChangeStrategies
A useful concept for our discussion of change strategies is from the field of evaluation: logic models. In evaluation, a logic model is a detailed map developed to clarify and communicate goals, intermediate outcomes, and measures for a specific project (W. K. Kellogg Foundation, 2004). Logic models make explicit which actions are intended to cause desired changes (McLaughlin & Jordan, 1999). They are closely related to theories of action and theories of change (Center for Civic Partnerships, 2007; Milstein & Chapel, n.d.), all of which empha- size their focus on communicating the logic and assumptions underlying a change effort. Logic models and theories of change can be quite extensive, including full-page or larger maps connecting boxes of activities, outcomes, stakeholders, and indicators (Keystone Accountability, 2009; W. K. Kellogg Foundation, 2004). In arguing for using theories of change to inform evaluation, Carol Weiss (1995) uses the term program theory to describe a less detailed version that is more in line with our approach but still articulates some of the underlying assumptions of how change happens. Since we are describing general strategies and not specific programs or projects, we cannot present full logic models. Rather, inspired by this concept, we articulate the underlying logic for each of the change strategies described in this article, summarize it in a sentence or two, and supplement it by the more detailed description of the approach and its limitations and assumptions. We hope our open discus- sion of underlying logic encourages others to be more explicit in their change logic as well.
ChoosingtheRightTheory Readers might wonder how to use this information that compares a range of change strate- gies. Our goal in this article is modest: to clarify the goals, assumptions, and underlying logic of each strategy in order to encourage STEM higher education change agents to situate their own efforts within this model of strategies and make their underlying assumptions about change more explicit.
A good starting point, particularly for those without social science backgrounds, is to focus on one strategy that fits their situation best (in terms of resources, goals, locus of change, and implicit assumptions about change already being followed). Readers should trust their wis- dom and consider selection of a theory as a design problem: within the constraints, some options fit better than others, but there is no clear right or wrong answer. In publications, part of the peer-review process is evaluating the appropriateness of the theory or perspective taken. For example, when studying effectiveness of dissemination efforts, a dissemination perspective is most likely to identify productive variables and processes on which to focus.
At the end of this article, we suggest that multiple change strategies will increase the like- lihood of success. However, STEM education change agents are unaccustomed to discussing their work in terms of the broader change literature or the categories of strategies presented here. The past efforts reviewed in Henderson et al.’s synthesis (2010, 2011) had a strong tendency to focus on just one of the four categories, without attempting to combine strate- gies. This model of four categories of change strategies is based on how change efforts have been described in the prior literature; it provides little critique or guidance as to how these efforts should be described in the future. This article attempts to provide a foundation and
Increasing Evidence-Based Teaching in STEM Education 225
T ab
le 1
S u
m m
ar y
o f
C h
an g e
C at
eg o ri
es an
d S
tr at
eg ie
s (a
cc o rd
in g
to ca
te g o ri
es in
F ig
u re
1 )
C h
an g e
ca te
g o ry
an d
st ra
te g y
S u m
m ar
y K
ey m
et ap
h o r
K ey
ch an
g e
ag en
t ro
le K
ey ch
an g e
m ec
h an
is m
T yp
ic al
m et
ri cs
o f
su cc
es s
I. C
u rr
ic u
lu m
& P
ed ag
o g y
D if
fu si
o n
In n
o va
ti o n
s ar
e cr
ea te
d in
o n
e lo
ca ti
o n
, th
en ad
o p
te d
o r
ad ap
te d
b y
o th
er s.
M u lt
i- st
ag e
ad o p ti
o n
p ro
ce ss
.
S ca
tt er
in g
D ev
el o p
a q
u al
it y
in n
o va
- ti
o n
an d
sp re
ad th
e w
o rd
. A
d o p ti
o n
d ec
is io
n s
b y
p o te
n ti
al u
se rs
. N
u m
b er
o f
u se
rs o r
am o u n
t o f
in fl
u en
ce o f
th e
in n
o va
ti o n
Im p
le m
en ta
ti o n
A se
t o f
p u
rp o se
fu l
ac ti
vi ti
es ar
e d
es ig
n ed
to p
u t
p ro
ve n
in n
o va
ti o n
s in
to p
ra ct
ic e
in a
n ew
se tt
in g .
T ra
in in
g D
ev el
o p
a tr
ai n
in g
p ro
- g ra
m th
at in
vo lv
es p
er -
fo rm
an ce
ev al
u at
io n
an d
fe ed
b ac
k .
T ra
in in
g o f
p o te
n ti
al u
se rs
. F
id el
it y
o f
u se
o f
in n
o va
ti o n
II .
R efl
ec ti
ve T
ea ch
er s
S ch
o la
rl y
te ac
h in
g In
d iv
id u
al fa
cu lt
y re
fl ec
t cr
it i-
ca ll
y o n
th ei
r te
ac h
in g
in an
ef fo
rt to
im p
ro ve
.
S el
f- re
fl ec
ti o n
E n
co u ra
g e
fa cu
lt y
to re
fl ec
t o n
an d
co ll
ec t
d at
a re
la te
d to
th ei
r te
ac h
in g .
E vi
d en
ce -b
as ed
re fl
ec ti
o n
o n
p ra
ct ic
e. S
el f-
re p
o rt
ed ch
an g es
in b el
ie fs
, te
ac h
in g
p ra
ct ic
es ,
o r
sa ti
sf ac
ti o n
w it
h st
u -
d en
t le
ar n
in g
F ac
u lt
y le
ar n
in g
co m
m u
n it
ie s
A g ro
u p
o f
fa cu
lt y
su p
p o rt
s ea
ch o th
er in
im p
ro vi
n g
te ac
h in
g .
C o m
m u
n it
y d
ev el
o p
m en
t B
ri n
g fa
cu lt
y to
g et
h er
an d
sc af
fo ld
co m
m u
n it
y d
ev el
o p m
en t.
P ee
r su
p p
o rt
/a cc
o u n
t- ab
il it
y; ex
p o su
re to
n ew
vi ew
s ab
o u
t te
ac h
in g
an d
le ar
n in
g .
S el
f- re
p o rt
ed ch
an g es
in b el
ie fs
, te
ac h
in g
p ra
ct ic
es ,
o r
sa ti
sf ac
ti o n
w it
h st
u -
d en
t le
ar n
in g ;
m o ti
va ti
o n
to w
ar d
s te
ac h
in g
II I.
P o li
cy Q
u al
it y
as su
ra n
ce M
ea su
ra b le
ta rg
et o u tc
o m
es ar
e id
en ti
fi ed
an d
p ro
g re
ss to
w ar
d s
th em
is as
se ss
ed an
d tr
ac k ed
.
A cc
re d
it at
io n
D ev
el o p
m ea
su ra
b le
o u t-
co m
es ,
d efi
n e
su cc
es s,
co ll
ec t
ev id
en ce
.
P re
ss u re
to m
ee t
o u tc
o m
es .
D eg
re e
to w
h ic
h o u tc
o m
e m
ea su
re s
ar e
m et
226 Borrego & Henderson
T ab
le 1
(c o n
ti n
u ed
)
C h
an g e
ca te
g o ry
an d
st ra
te g y
S u m
m ar
y K
ey m
et ap
h o r
K ey
ch an
g e
ag en
t ro
le K
ey ch
an g e
m ec
h an
is m
T yp
ic al
m et
ri cs
o f
su cc
es s
O rg
an iz
at io
n al
d ev
el o p m
en t
L ea
d er
d ev
el o p
s n
ew vi
si o n
an d
p la
n s
a st
ra te
g y
fo r
al ig
n -
in g
em p
lo ye
e at
ti tu
d es
an d
b eh
av io
rs w
it h
th is
vi si
o n
.
L ea
d er
sh ip
D ev
el o p
n ew
vi si
o n
. A
n a-
ly ze
al ig
n m
en t
o f
p ar
ts o f
th e
o rg
an iz
at io
n w
it h
th e
n ew
vi si
o n
an d
id en
ti fy
st ra
te g y
fo r
cr ea
ti n
g al
ig n
m en
t.
S tr
at eg
ic w
o rk
b y
th e
le ad
er to
co m
m u
n i-
ca te
vi si
o n
an d
n ee
d fo
r ch
an g e
an d
to d
ev el
o p
st ru
ct u
re s
to m
o ti
va te
em p
lo ye
es to
w o
rk to
w a rd
s it
.
P ro
d u
ct iv
it y-
re la
te d
m et
- ri
cs (e
.g .,
cr ed
it h
o u
r p
ro d
u ct
io n
, g ra
d u
at io
n ra
te s,
et c.
)
IV .
S h
ar ed
V is
io n
L ea
rn in
g o rg
an iz
at io
n s
L ea
d er
w o rk
s to
d ev
el o p
an o rg
an iz
at io
n al
cu lt
u re
th at
su p
p o rt
s k n
o w
le d
g e
cr ea
ti o n
.
T ea
m le
ar n
in g
M o ve
d ec
is io
n -m
ak in
g fu
r- th
er fr
o m
th e
to p .
In ve
st in
d ev
el o p
in g
em p
lo ye
es ’
p er
so n
al m
as te
ry ,
m en
ta l
m o d el
s, sh
ar ed
vi si
o n
, te
am le
ar n
in g .
T ea
m -l
ev el
q u
es ti
o n
- in
g an
d re
vi si
o n
o f
m en
ta l
m o d
el s
(i .e
., d
o u
b le
lo o p
le ar
n in
g ;
A rg
yr is
& S
ch €o n
, 1 9 7 4 )
fa ci
li ta
te d
b y
m id
d le
m an
ag er
s.
V ag
u e
an d
si tu
at io
n d
ep en
d en
t
C o m
p le
xi ty
le ad
er sh
ip In
a co
m p
le x
sy st
em ,
re su
lt s
o f
ac ti
o n
s ar
e n
o t
ea si
ly p
re -
d ic
te d
. C
h an
g e
ag en
ts ca
n cr
ea te
o rg
an iz
at io
n al
co n
- d
it io
n s
th at
in cr
ea se
th e
li k e-
li h
o o d
o f
p ro
d u
ct iv
e ch
an g e.
E m
er g en
ce D
is ru
p t
ex is
ti n
g p
at te
rn s,
en co
u ra
g e
n o ve
lt y,
an d
ac t
as se
n se
m ak
er s.
N ew
id ea
s em
er g e
th ro
u g h
in te
ra ct
io n
s o f
in d
iv id
u al
s. F
o rm
al le
ad er
s en
co u ra
g e
th is
p ro
ce ss
b y
cr ea
ti n
g d
is eq
u il
ib ri
u m
an d
am p
li fy
in g
p ro
d u
ct iv
e in
n o va
ti o n
s.
V ag
u e
an d
si tu
at io
n d
ep en
d en
t
Increasing Evidence-Based Teaching in STEM Education 227
common language for productive future developments. We encourage readers to apply this work, critique it, and build on it.
CategoryI CurriculumandPedagogy
Change strategies in this category focus on changing individuals (typically faculty members) in a prescribed way. Henderson et al. (2011) found that among STEM undergraduate edu- cation researchers, strategies in this category are the most commonly used and discussed. In fact, discussion about how to improve undergraduate STEM instruction is typically concep- tualized solely within this category. Working from the assumption that faculty have limited time and expertise to develop improved teaching methods, STEM change agents develop and perfect highly structured and specific interventions meant to be easily implemented by others. Developing highly specified interventions is the basic change model behind many change initiatives in undergraduate STEM (Seymour, 2001) and also the change model implicit in influential funding programs for undergraduate STEM instructional improve- ment, such as the NSF (2010) Transforming Undergraduate Education in STEM (TUES) and its predecessor, Course, Curriculum and Laboratory Improvement (CCLI). These ideas are elaborated below in specific descriptions and examples of diffusion and implementation.
Diffusion Underlying logic STEM undergraduate instruction will be changed by altering the behavior
of a large number of individual instructors. The greatest influences for changing instructor behavior lie in optimizing characteristics of the innovation and exploiting the characteristics of individuals and their networks.
Description The term diffusion of innovations was popularized by a book of that title first published by Everett Rogers in 1962 and now in its fifth edition (Rogers, 2003). The theory has been used to describe adoption of a wide range of innovations, including agricul- tural equipment, public health interventions, and cellular telephones, and has demonstrated relevance to diffusion of instructional strategies. Three features of Rogers’s theory are often implicit in discussions about STEM education change initiatives.
First, much of diffusion of innovation theory focuses on the characteristics of the innovation (such as an instructional strategy or curricular approach); that is, the discussion may focus, for example, on how much better the innovation is than current practice (relative advantage) or how hard it is to understand and use the innovation (complexity). The second feature is that adoption is conceptualized as an individual choice. Potential adopters (faculty members) are often categorized in terms of their “innovativeness” (e.g., whether they are innovators, early adopters, early majority, late majority, or laggards), and this information may be used to target influential leaders or individualize dissemination strategies by adopter type. Finally, once enough people adopt an innovation (Rogers suggests between 10% and 25%), it will reach a criti- cal mass (or what (Gladwell [2000] calls a “tipping point”), after which the innovation will con- tinue to spread on its own until it saturates the system. We note that although this theory has been applied in settings around the world, the conceptualization of curriculum change as an indi- vidualized, course-based act is more characteristic of the United States. For example, European views of knowledge and curriculum may be more interconnected and holistic than those in the United States (Borrego & Bernhard, 2011; de Graaff & Kolmos, 2007).
Another important aspect of Rogers’s view of diffusion of innovations is the representa- tion of adoption decisions as a series of stages. Adopters do not move from knowing nothing
228 Borrego & Henderson
about an innovation to adopting it in one step. While there are many descriptions of the stages through which an adapter reaches the point of using an innovation, the five-stage description offered by Rogers (2003) provides a useful framework:
1. Awareness – Awareness of the innovation, but lacking complete information about it
2. Interest – Growing interest and information seeking
3. Evaluation – Decision whether or not to try innovation based on present and future situation
4. Trial – Making use of the innovation
5. Adoption – Continued use of the innovation
Research in multiple STEM disciplines (Borrego et al., 2010; Froyd, Borrego, Cutler, Henderson, & Prince, 2013; Henderson, Dancy, & Niewiadomska-Bugaj, 2012; Prince, Borrego, Henderson, Cutler, & Froyd, 2013) suggests that strategies currently used by change agents have been relatively successful at creating awareness and interest, but have not been as successful at supporting faculty during the trial stage; lack of success at this stage may lead many faculty to discontinue use or to modify the innovation in ways that likely diminish its effectiveness. Diffusion of innovations is a robust theory that allows for some influence from the system, much adaptation, phased adoption, and many other aspects of change. Many of the limitations arise because STEM education change agents have not paid attention to all of the important aspects of a diffusion perspective in their change efforts or they have attempted to apply a diffusion perspective in situations where it is not appropriate. For example, instructional strategies that rely on computer programs have not always been designed for others to easily adapt them to local conditions.
Building on the work of Rogers and many others, Wejnert (2002) provided a very useful framework of three categories of variables associated with diffusion of innovations (Table 2). Many of the variables identified by Wejnert have not yet been studied in a higher education context; however, they should provide guidance to researchers and change agents in develop- ing a change initiative. One important aspect of change that is beginning to be articulated by STEM education researchers is related to the variable of position in social networks. While many STEM-based change agents have tended to think of diffusion as a process that occurs by a developer using mass-marketed distribution methods (e.g., publishing articles or giving talks), mounting evidence suggests that the most successful diffusion occurs through personal interactions between individuals or in small groups (e.g., Dancy, Turpen, & Henderson, 2010; Prince et al., 2013). These personal interactions occur, for example, during informal conversations with colleagues or when instructors were trained as graduate teaching assistants early in their careers. Although STEM education dissemination efforts have in recent years been focused at a national or international level, diffusion theory helps explain the importance of local communities and networks that STEM education leaders are increasingly noticing.
Diffusion example Montfort, Brown, and Pegg (2012) used diffusion of innovations theory to study adoption of an assessment instrument designed for use in capstone design courses. The team conducted interviews with developers of the instrument, current users, and educators who attended workshops to learn about the instrument. The authors used stages of adoption (knowledge, persuasion, passive rejection, decision, implementation, active rejection, and confirmation) to describe the participants and contextualize their responses. They compared participants’ perceptions
Increasing Evidence-Based Teaching in STEM Education 229
of the instrument in terms of compatibility with their teaching approaches and environment, relative advantage over current approaches, and complexity. Montfort and colleagues found that compatibility was the only useful construct that consistently explained adoption decisions. For example, if study participants were not using the instrument, it was because they did not feel it was com- patible with the capstone design course in their depart- ment (and alternatively, those who were using it thought it was compatible with how the course was run). While the compatibility explanation was consistent, different participants described the same features of the instru- ment as positive or negative, depending on their perspec- tive and local environment. The authors also explored the role of communication channels and found that, con- sistent with predictions from diffusion of innovation theory, interpersonal channels (word of mouth) were more effective than mass media (workshops). They also suggested that future work needs to more carefully con- sider how the wide variety of communication modes identified by the theory can be applied to engineering education. Finally, Montfort and colleagues noted limitations of the individualized focus of diffusion of innovations because the capstone design setting is often coordinated by a team of instructors and assessment specialists who must agree on adoption decisions.
Implementation Underlying logic STEM undergraduate instruction will be changed by developing research-
based instructional “ best practices” and training instructors to use them. Instructors must use these practices with fidelity to the established standard.
Description An important distinction between diffusion and implementation is the level of deliberateness that lies behind an implementation strategy. While diffusion-based strate- gies are characterized by developing good products and spreading the word about them, implementation-based strategies are characterized by the focus on carefully developing a set of activities designed to put the innovation into successful practice in a new setting. Although implementation strategies are not yet widely used for higher education curricular innovations, these strategies have been used extensively in K–12 settings (Cooper, 2008). Fixsen, Naoom, Friedman, and Wallace (2005) described implementation in terms of five components (see Figure 2). The process is driven by a proven instructional product or practice (the source) that the change agent seeks to have adopted by instructors (the target). The change agent identi- fies current practices of the target instructors and sphere of influence surrounding the target instructor (that is, the cultures and structures in target departments) and provides appropriate training and coaching (the communication link) to the target instructor. Progress is measured by documenting knowledge and use of the new practice by the target instructor. Implementa- tion has been advocated by the National Science Foundation’s TUES program, for example, through encouraging projects that “promote widespread implementation of educational innovations” (Feser, Borrego, Pimmel, & Della-Piana, 2012; NSF, 2010, p. 4).
On the basis of a review of implementation literature in a variety of fields, Fixsen et al. (2005) found that implementation efforts are most successful when the core components of
Table 2 Conceptual Framework
of Diffusion Variables
Characteristics of innovations Public versus private consequences Benefits versus costs
Characteristics of innovators Societal entity Familiarity with the innovation Status characteristics Socioeconomic characteristics Position in social networks Personal characteristics
Environmental context Geographical settings Societal culture Political conditions Global uniformity
Note. Adapted from Wejnert (2002).
230 Borrego & Henderson
the source program or practice are known and clearly defined. One study in engineering edu- cation focused on defining these core components and advocating for more attention to implementation (Borrego, Cutler, Prince, Henderson, & Froyd, 2013). Performance evalua- tion and feedback are crucial in supporting successful use of the core components (Fixsen et al., 2005; Henderson et al., 2011). In implementation, there is a strong emphasis on com- munication and feedback that connects the source practice (new instructional method) to the target (implementing faculty). As Fixsen et al. (2005) summarized as a primary finding of their literature review, “information dissemination alone is an ineffective implementation method” (p. 70). This conclusion suggests that STEM higher education change agents should pay more attention to feedback from users during dissemination.
Implementation example Gallos, Van den Berg, and Treagust (2005) described the use of an implementation approach to the improvement of instruction in a general chemistry course. Once a new instructional style was developed and pilot tested, 13 instructors who taught the course were enrolled in a training program, which began the semester prior to implementation. The instructors observed the pilot version of the revised course and attended weekly training ses- sions. During the training sessions, the instructors learned about the philosophy behind the new course and were engaged in refining curricular materials to fit with the new philosophy. During the implementation semester, the 13 instructors met with the developer in small weekly group ses- sions to discuss implementation issues. The developer and another pedagogical expert regularly observed lessons and provided feedback and assistance to the instructors. Metrics related to imple- mentation of the new instructional style were developed and used to assess the fidelity of the implementation. These metrics included the number of minutes in each class session that were used for instructor lecture, student work, and closure or summary, as well as a checklist of variables related to the student work portion of the class, for example, whether the instructor assigns student work to individuals or pairs, or whether the instructor moves around the classroom. On the basis of these metrics, Gallos et al. concluded that nine of the 13 instructors were successful in changing their instruction by implementing the changes with fidelity to the standard curriculum.
Summary Change strategies in the Curriculum and Pedagogy category all rely on having a product (e.g., an instructional strategy or teaching materials) to disseminate. Defining this product and the range of acceptable variation is an important step in any change strategy in this category.
Figure 2 A conceptual view of implementation. Adapted from Fixsen et al. (2005).
Increasing Evidence-Based Teaching in STEM Education 231
In this category, diffusion and implementation represent two different emphases that could be successfully combined. Diffusion emphasizes developing a good product and letting potential users know about it, perhaps by targeting influential opinion leaders. STEM educa- tion research indicates that diffusion strategies have been successful at the early awareness stages, but cater less to instructors who need to understand the details of implementation. An implementation approach is much more focused on supporting the potential user through the entire process. Like diffusion, an implementation perspective requires communicating to potential users the compatibility and relative advantage of the innovation. Unlike diffusion, implementation emphasizes providing the user with appropriate monitoring, feedback, and support during the implementation process. Combining these two perspectives would be pos- sible and productive for change agents to emphasize diffusion at early stages to raise aware- ness and convince instructors to try the strategies, and at later stages to use implementation to support users in ongoing use.
CategoryII ReflectiveTeachers
Strategies in the Reflective Teachers category develop instructors as reflective practitioners (Brookfield, 1995; Sch€on, 1983, 1987; Zeichner & Liston, 1987; Zeichner & Noffke, 2001). The focus is on instructors either as individuals or as part of a community, and the outcome is emergent. The emphasis is on engaging and empowering faculty to reflect on their teaching practice, frequently through consideration of assessment evidence, in order to make instruc- tional changes on the basis of on their best judgment. The process is often informed by the education research literature (Mettetal, 2001), and this evidence often also directs faculty to use (or independently develop) research-based instructional strategies. Developing reflective teachers is the approach most frequently taken in teaching and learning centers, which provide consultation services to motivated faculty and may also sponsor faculty learning communities. While not specifically limited to STEM education, these strategies have been applied in pro- grams that target engineering and STEM instructors.
ScholarlyTeachingandScholarship ofTeachingandLearning
Underlying logic STEM undergraduate instruction will be changed when more individual faculty members treat their teaching as a scholarly activity.
Description Scholarly teaching has been defined as “a method of finding out what works best in your own classroom so that you can improve student learning” which “fits in the center of a continuum ranging from personal reflection at one end to formal educational research at the other” (Mettetal, 2001, p. 1). Scholarly teaching is similar to but often con- sidered less formal than the scholarship of teaching and learning (SoTL; Borrego, 2007; Hutchings & Shulman, 1999; Streveler, Borrego, & Smith, 2007). We acknowledge that the two concepts are usually described in different literatures, but we combine them none- theless for the purposes of this brief review because both represent the broad category of developing reflective teachers. Since SoTL resists precise definition (McKinney, 2007), in Table 3 we offer the set of characteristics developed by Connolly, Bouwma-Gearhart, and Clifford (2007) to describe their adaptation of SoTL to STEM education.
Instructors may be initially motivated to improve their teaching or to create required materials for their dossiers. These projects can create a greater sense of excitement about
232 Borrego & Henderson
teaching thus prompting teachers to change their teaching (Mettetal, 2001). Many publica- tions provide advice on developing scholarly teaching and SoTL among engineering instruc- tors specifically (e.g., Baillie, 2007; Felder, Brent, & Prince, 2011; Kolmos, Vinther, Andersson, Malmi, & Fuglem, 2004). Perhaps the most salient characteristic of this advice is that scholarly teaching and SoTL emphasize using data in decision making (Connolly et al., 2007; McKinney, 2007; Mettetal, 2001), which may appeal to engineering faculty but not necessarily be their initial strategy. The individual nature of scholarly teaching and SoTL is highlighted by practical advice from Mettetal (2001):
Since the goal [of scholarly teaching and SoTL] . . . is to inform decision-making, the question or problem should look at something under teacher control, such as teaching strategies, student assignments, and classroom activities. The problem should also be an area in which you are willing to change. There is no point in conducting a . . . proj- ect if you have no intention of acting on your findings. Larger institutional questions might be tackled, if the institution is committed to change. (p. 3)
Mettetal also advises instructors to select a feasible project in a course that is progressing well and gradually work up to tackling more challenging problems.
Scholarly teaching example Raubenheimer and Myka (2005) describe using scholarly teaching methods in the improvement of a first-year zoology laboratory course. The lab instructor (one of the authors) was dissatisfied with students’ learning and, in collaboration with a faculty member in education (the other author), sought to make improvements. Their article describes three cycles of applying a research approach to improving the lab. In each cycle, the authors posed questions, collected and analyzed data, and made changes to instruc- tion. The authors describe very nicely the cyclical inquiry process that is a core feature of scholarly teaching. As exemplified in the article, one of the key reasons that scholarly teach- ing is so powerful is that each cycle leads the instructor to ask more sophisticated questions and collect different types of data to answer the questions.
FacultyLearningCommunities Underlying logic STEM undergraduate instruction will be changed by groups of instructors
who support and sustain each other’s interest, learning, and reflection on their teaching. Description While scholarly teaching is primarily an individual endeavor, faculty learn-
ing communities seek to create a support network around similar ideas of using evidence to
Table 3 Main Characteristics
of Teaching-as-Research
Drawing upon the work of others, including disciplinary colleagues, education researchers, and students
Posing an explicit question about the effectiveness of one’s practice Creating and following an explicit design or plan Collecting credible evidence to answer the question Analyzing and interpreting evidence Reflecting on one’s findings Acting on one’s findings Engaging in ongoing and cyclical inquiry Documenting and disseminating processes and outcomes of inquiry Being principally responsible for conducting the inquiry on one’s own practice.
Increasing Evidence-Based Teaching in STEM Education 233
improve teaching. Cox (2004) defines faculty learning communities in higher education as a type of community of practice:
a cross-disciplinary faculty and staff group of six to fifteen members . . . who engage in an active, collaborative, yearlong program with a curriculum about enhancing teaching and learning and with frequent seminars and activities that provide learning, develop- ment, the scholarship of teaching, and community building. (p. 8)
Faculty learning community participants are expected to select a target course in which to try out new and innovative teaching approaches and then assess resulting student learning (e.g., as described in Calkins & Light, 2007; Lynd-Balta, Erklenz-Watts, Freeman, & Westbay, 2006). Evidence shows that faculty learning communities increase instructor interest in teaching and learning and provide safety and support for instructors to change longstanding instructional practices (Beach & Cox, 2005). The ultimate goal of instructor participation in faculty learning communities is improved student learning; however, it is common for participants to engage in action research in their courses and present project results at local, regional, and national confer- ences and in disciplinary journals.
Faculty learning communities address the weaknesses of traditional professional develop- ment approaches by providing a long-term collaborative structure of safety and support for instructors to investigate, attempt, assess, and adopt teaching methods that are new to them (Beach & Cox, 2005; Cox, 2004). Learning communities allow for individual instructors to choose the approaches most interesting to them, help instructors through the implementa- tion dip, and assume that instructors want and need to adapt teaching approaches to their unique courses. Some specific implementations of faculty learning communities are described in Erklenz-Watts, Westbay, and Lynd-Balta (2006); Fleming, Shire, Jones, Pill, and McNamee (2004); Scott and Weeks (1996); and Wildman, Hable, Preston, and Magliaro (2000).
Participation in a faculty learning community increases awareness of different teaching and learning styles and broadens awareness of different cultures and disciplines. It engages and empowers instructors to make changes, potentially at the curriculum level, through collaboration. In fact, it may have some overlap with learning organizations (below), because all members of the group are learners, and the organization is structured to learn as a whole system (Cox, 2004). Faculty learning communities also have positive outcomes for faculty retention (Cox, 2004).
We note that there is extensive literature on the central role of teacher learning communi- ties to school change efforts in K–12. Reviewing and translating these to higher education settings is beyond the scope of this review, but readers are directed to sources such as Cochran-Smith and Lytle (1999) for more detail.
Faculty learning communities example Lynd-Balta et al. (2006) describe a year-long fac- ulty learning community in which an interdisciplinary group of eight faculty met monthly for three-hour evening sessions. The participants were interested in working to improve their teach- ing to better develop students’ critical thinking skills. The group was facilitated by the first two authors, one from biology and one from education. The learning community was developed to introduce participants to some important results from education research and to allow them to reflect and discuss how the ideas apply to their courses. Participants were expected to focus on a particular instructional unit that they wished to improve. These units were implemented near the end of the faculty learning community year, and results were reported to the group. Participants identified the structured reflection and peer support of the learning community as important con- tributors to their ability to make significant changes to their instruction.
234 Borrego & Henderson
Summary Strategies within the Reflective Teachers category have the major advantage of supporting instructors in changing aspects of instruction they are interested in altering. The support and encouragement often offered to instructors by these strategies helps to promote productive out- comes and, in particular, helps to sustain faculty commitment to change during the risky trial period when new instructional approaches are first implemented. Being strongly instructor- driven is both the major strength and the major weakness of these strategies. This approach does little to involve resistant faculty members. It is focused and dependent on faculty motivation to a greater extent than the other approaches, and it is clearly a bottom-up approach to change.
CategoryIII Policy
We now shift from strategies directly targeting individual instructors to those that target the environments in which they work. This category emphasizes guiding organizations (and the people within them) towards a pre-identified goal, primarily through policy changes that influence behavior. In this setting, the goal would be improved teaching or increased use of research-based instructional strategies, although much of the literature is written independ- ent of the specific goal because it assumes change agents have one in mind. The first strategy, quality assurance, focuses on policies of evidence-based decision making and improvement, which leads to accreditation in higher education settings. The second strategy, organizational development, focuses on policies developed by leaders who have a clear vision for influencing individual behaviors that result in changes to the overall organization. With the exception of the prevalence of accreditation systems governed by organizations such as ABET (a U.S.- based accreditation organization) and Engineers Australia, these strategies have generally not been applied as change strategies in undergraduate STEM instruction.
QualityAssurance Underlying logic STEM undergraduate instruction will be changed by requiring institutions
(colleges, schools, departments, and degree programs) to collect evidence demonstrating their success in undergraduate instruction. What gets measured is what gets improved (cf. Steering Committee for Evaluating Instructional Scholarship for Engineering, 2009).
Description Quality assurance is a process by which organizations collect and analyze their own evidence to evaluate and improve their ability to meet stated goals. In higher edu- cation, the process has evolved to include these major steps: setting goals or targets, prepar- ing a self-study report with evaluation evidence, hosting an external visit by peers, and responding to the external review report (Ewell, 1997; Rhoades & Sporn, 2002). Quality assurance is becoming increasingly popular in institutions and engineering programs around the world. The dominant example of quality assurance in the United States (and increasingly in other countries) is ABET accreditation of degree programs. In Australia, the Institute of Engineers coordinates a nearly identical process. Quality assurance in European engineering education is somewhat different and is currently focused on the Bologna process for promot- ing “convergence and transparency in qualification structures in Europe” (De Wit, 2000, p. 9) and aligning engineering programs to encourage mobility of graduates. Regional or national accreditation organizations also focus at the institution level, across all disciplines. Engineering accreditation systems, however, have generally been ahead of regional systems; for example, in the United States, they have required outcomes-based accreditation a few
Increasing Evidence-Based Teaching in STEM Education 235
years before regional accreditation agencies. These engineering accreditation standards are one important reason why in their discussions of quality assurance, engineering education leaders tend to focus on ABET and similar systems rather than regional ones.
Quality assurance is linked with many management trends from industry, including stra- tegic planning (Rhoades & Sporn, 2002). That these trends jump sectors from industry to higher education through people who bridge them, such as industrial advisory board mem- bers (Birnbaum, 2000; Rhoades & Sporn, 2002), may explain why in the United States engi- neering education quality assurance is more developed through ABET accreditation than through regional accreditation activities.
In general, as Rhoades and Sporn (2002) explain, “Quality assurance in the U.S. has never been taken to mean a high standard of comparable quality across institutions” because accredi- tation associations “have focused on ensuring a minimum level of competence” (p. 376). Thus, quality assurance operates more at the trailing edge of change efforts rather than at the leading edge. The rate of changes effected by quality assurance can seem painfully slow, but they ensure that all participating institutions meet a minimum level of compliance. Therefore, quality assurance is less useful as a leading-edge change strategy than it is helpful in bringing a large number of programs up to a new standard.
Finally, we note the issues related to the types of evidence used in quality assurance and accreditation. An increasing link between quality assurance and internal and external resource allocation is beginning to lead to emphasis on productivity and focus on outcomes that are easy to measure (Rhoades & Sporn, 2002). However, aspects of learning that are easy to measure or have been traditionally measured are not necessarily the best evidence of quality educational experiences. Student evaluations of teaching are one prevalent source of evidence in the United States, where they have been used since the 1970s, and are now adopted by up to 95% of institu- tions (Cashin, 1999; Rhoades & Sporn, 2002). Yet many argue that these are indirect and invalid measures of teaching effectiveness (Greenwald, 1997; Kulik, 2001). Nonetheless, the consensus of several hundred studies indicates that student evaluations are sufficiently valid and reliable for administrative purposes (Marsh & Roche, 1997), although they should be used in conjunction with other sources of information to evaluate teaching (Braskamp & Ory, 1994). Teaching eval- uations are also typically considered in faculty promotion and tenure decisions. As stakes are raised, it is increasingly important to ensure that assessment measures align to desired outcomes.
Quality assurance example Over the course of a multiyear transition beginning in 1996, ABET implemented new EC2000 outcomes-based assessment criteria, a significant depar- ture from the previous system focusing on hours of instruction in various subjects (Prados, Peterson, & Lattuca, 2005). The change to outcomes-based criteria spurred countless administrators and faculty who had survived an accreditation visit to share their experience, advice, and assessment systems through publications, most commonly at the Frontiers in Education conference. In this Journal, then Georgia Tech Engineering Associate Dean for Academic Affairs Jack Lohmann (1999) described the process of engaging engineering fac- ulty in development of mission or vision statements, outcomes and objectives, and systems for collecting and using assessment data. He offers the following advice for others preparing for their first accreditation visit under the new criteria:
Focus on what is important to your College first; focus on what is important for accreditation second.
Improve existing assessment processes and measures first.
236 Borrego & Henderson
Share information and collaborate as much as possible.
Clarify terminology and establish the key elements of the assessment plans early in the development process.
Identify benchmark institutions and key constituents.
Gather data, and lots of it.
Develop a system to document the use of results. (pp. 308–309)
Other early publications also focus on developing learning objectives and outcomes (Felder & Brent, 2003; Scales, Owen, Shiohare, & Leonard, 1998).
ABET has been a prominent force in impacting engineering higher education. On the one hand, it has positively impacted evidence-based assessment and teaching in engineering; on the other hand, ABET has been criticized for stifling instructional innovation and improvement in engineering education (ASEE, 2009; Hickman & Sussman, 2013). The major change to outcomes-based accreditation in 2000 has been cited as having a significant positive impact on engineering education. Volkwein, Lattuca, Harper, and Domingo (2007) found substantial increases in emphasis on professional skills, active learning, and assessment- informed curricular improvements between 1994 and 2004 as a result of ABET’s implemen- tation of the EC2000 accreditation criteria. Similarly, Froyd, Wankat, and Smith (2012) identified outcomes-based accreditation through ABET as one of “five major shifts in 100 years of engineering education.” Yet in the decade since this change, most engineering accreditation activity has become routinized, reducing its ability to encourage further changes to engineering education. National surveys of engineering faculty, department chairs, and deans found that up to 67% of chairs and 45% of associate deans believed ABET criteria lim- ited their ability to improve undergraduate curricula at least slightly (D. B. Knight, personal communication, October 31, 2013), while faculty described ABET accreditation as barrier to innovation (unpublished survey data from Besterfield-Sacre, Cox, Borrego, Beddoes, & Zhu, 2014). The perception of some engineering faculty and administrators that ABET is a barrier to change may occur because the path of least resistance for most faculty is to keep teaching and documenting student learning the same way from year to year. Changing the way an engineering course is taught, for example to implement more projects or active learning, would require not only changing the course itself, but also rethinking learning outcomes and documentation. If previous practice resulted in a program being accredited, there is little incentive to change courses or engage in faculty professional development. Through this dis- incentive to change previously successful approaches, program accreditation has become a perceived disincentive to innovation in undergraduate instruction (Harvey, 2004; Hickman & Sussman, 2013).
OrganizationalDevelopment Underlying logic STEM undergraduate instruction will be changed by administrators with
strong vision who can develop structures and motivate faculty to adopt improved instructional practices. Description Another management trend that has moved into higher education is organi-
zational development and organizational transformation, which are “aimed at the planned change of organizational vision [or] work settings” (Porras & Silvers, 1991, p. 54). (Although researchers in this area distinguish between organizational transformation and organizational development, we will combine them for the purposes of this overview.) As shown in Figure 3,
Increasing Evidence-Based Teaching in STEM Education 237
planned changes (interventions) in work settings are intended to create cognitive changes in employees, which lead to behavioral changes and improved organizations. Improvement of the organization can mean a better fit between its capabilities and current environment or a predicted future environment (Porras & Silvers, 1991).
Although some of the approaches are aimed at individuals and may result in bottom-up change, the overall approach is top-down in the sense that management identifies a mis- match and initiates a planned change effort. The interventions comprise both organization- level and individual perspectives (Porras & Silvers, 1991).
Kotter (1995) and Kotter and Schlesinger (1979) presented accessible versions of these approaches in terms of strategies for influencing individuals and organizations at various stages of the change process. Graham (2012) argued that Kotter’s (1996) stage model of top- down change has been the most influential one in engineering education, perhaps because Froyd, Penberthy, and Watson (2000) provided an adaptation of the steps to engineering undergraduate instructional change initiatives, which is listed in Table 4. Yet it is difficult to point to publications describing specific engineering or STEM instructional change efforts that have been guided by this perspective.
In general, many of the details of organizational development, such as measuring change success in terms of profit or productivity, require adaptation from an industry setting to higher education. Nonetheless, research on organizational development does provide some guidance on evaluating the readiness to undertake change efforts on the part of individuals (Fisher, Merron, & Torbert, 1987; Gardner, Dunham, Cummings, & Pierce, 1987; Piderit, 2000) and organizations (Beer, 1987; Nadler & Tushman, 1989). Porras and Silvers (1991) also present a promising array of quantitative and qualitative methods for evaluating the impact of these types of change efforts.
Organizational development example Ponitz (1997) describes the use of an organiza- tional development strategy in the implementation of learning outcomes at Sinclair Commu- nity College. According to Ponitz, in the 1980s Sinclair’s president felt external pressures to emphasize student performance as a measure of institutional effectiveness. These pressures led the president to develop the vision of a guarantee for the competencies of Sinclair gradu- ates that would ensure that each graduate was ready for the workplace or for transfer to a baccalaureate degree program. According to this policy (approved by the board of trustees),
Figure 3 Logic model for organization transformation and organization devel- opment. Adapted from Porras and Silvers (1991).
238 Borrego & Henderson
Sinclair graduates could receive additional free education if they were judged by their em- ployer to not possess a job skill identified as a learning outcome of their degree program.
In support of this vision, the president called for the development of learning outcomes for each program and procedures for documenting that students have met these outcomes. This effort was coordinated by an assessment steering committee, which was charged with reviewing current assessment practices at Sinclair and making recommendations for improve- ment. The committee adopted a set of 12 principles of assessment to guide the assessment initiatives. It then developed several assessment-related policies that were adopted by the board of trustees. For example, one policy required all degree programs to have clearly defined learning outcomes, to publish these outcomes, and to identify specific courses that prepare students to achieve each outcome. The steering committee annually reviewed the results of the program and made improvements as needed. One of the key mechanisms used to com- municate the president’s assessment vision was the job competency guarantee enacted throughout the college.
Summary The strategies in the Policy category are strongly influenced by management practices in industry. While quality assurance policies have been widely adopted in higher education and particularly in engineering program accreditation, they are not often cited as an effective change strategy. Quality assurance is better thought of as a long-term, often evolutionary, change process that may have more impact on bringing all programs up to a minimum level of quality. As discussed, policy changes based on quality assurance can result in significant changes over time (beyond a minimum level of quality), as has been seen with the ABET cri- teria. There is also often a synergy between quality assurance and organizational development in that organizational development-driven policies are often initiated as a response to changes in external circumstances, such as new accreditation criteria. Understanding the goals and assumptions of these strategies (for example, maintain a minimum level of quality) helps advocates and change agents understand their limitations and select an appropriate strategy.
Table 4 Froyd et al.’s (2000) Adaptation of Kotter’s (1996)
Stage Model to Engineering Education
1. Establish need and energy for a curricular change (Establish a sense of urgency)
2. Gather a leadership team to design and promote the curricular change (Create a guiding coalition)
3. Define and agree upon new learning objectives and a new learning environment (Develop a vision and strategy)
4. Discuss the new objectives and environment with the college and revise based on feedback (Communicate the change vision)
5. Implement new curriculum using a pilot, if necessary (Empower broad-based action)
6. Conduct a formative evaluation of the program, investigating strengths and weaknesses of the current implementation, and indicators of short-term gains (Adjust for growing pains and generate short-term wins)
7. Decide how the new approach may be used for the entire college and prepare an implementation plan (Consolidate gains and produce more change)
8. Prepare faculty and staff for the new implementation, implement, and follow up with improvements (Anchor new approaches in the culture)
Note. Kotter’s original recommendations are in parentheses
Increasing Evidence-Based Teaching in STEM Education 239
It should be emphasized that these strategies are a better fit for a specific change, such as a new curriculum or system for advising or evaluating faculty members. More general efforts directed at changing the culture for supporting teaching improvement might best be handled through a shared-vision approach.
CategoryIV SharedVision
Change strategies in the final Shared Vision category focus on changing environments and struc- tures to support the development of emergent teaching innovations. This category seeks innova- tions from the front lines of the organization and engaging individuals with diverse perspectives in further developing these innovations. Henderson et al. (2011) found that strategies of this type are not widely used or discussed by STEM education researchers or even higher education researchers. Most ideas in this category have been developed within an industry context. Thus, while these ideas seem promising, their applicability is largely untested in higher education.
LearningOrganizations Underlying logic Innovation in higher education STEM instruction will occur through
informal communities of practice within formal organizations in which individuals develop new organizational knowledge through sharing implicit knowledge about their teaching. Leaders culti- vate conditions for both formal and informal communities to form and thrive.
Description The idea of learning organizations is popular in the field of management. The core idea is that businesses need to continually learn and improve in order to be successful in the rapidly changing marketplace. There are two essential conditions for a learning organization: (1) new ideas must be developed within the organization and (2) these ideas must lead to changes in the way the organization operates (Dill, 1999). Unlike prior views of an organization, however, in which the management is the brains and the line workers simply implement the ideas of man- agement, in a learning organization, the organizational knowledge required for success comes from throughout the organization. The role of management is to foster conditions that support the development of this knowledge. Two seminal books related to this idea are The Fifth Disci- pline (Senge, 1997) and The Knowledge Creating Company (Nonaka & Takeuchi, 1995).
A central theme of both books is that individuals’ models about the way the world works (which are often implicit) shape how they perceive and interact with the world. Organiza- tional knowledge is created when individuals attempt to make their mental models explicit and share these models with others in the organization. Middle managers or line managers are typically seen as the interface between the front-line workers and the upper levels of the organization that facilitate this knowledge-creating process. Nonaka and Takeuchi (1995) call this learning organization structure middle-up-down management in order to contrast it with more familiar top-down or bottom-up styles:
Knowledge is created by middle managers, who are often leaders of a team or task force, through a spiral conversion process often involving both the top and the front- line employees (i.e., bottom). The process puts middle managers at the very center of knowledge management, positioning them at the intersection of the vertical and hori- zontal flows of information within a company. (p. 127)
Senge (2000) suggests that department chairs are the middle managers of higher educa- tion and that their important role in a learning organization is “to facilitate ongoing
240 Borrego & Henderson
reflection and conversation to identify clear goals and establish agreed-upon strategies to move towards those goals. These strategic conversations link them vertically to those above them in the hierarchy, as well as to those below them” (p. 285).
The use of department-level collaborative management has been linked with faculty use of more student-centered instruction (Martin, Trigwell, Prosser, & Ramsden, 2003; Ramsden, Prosser, Trigwell, & Martin, 2007). Collaborative management for teaching involves depart- ment heads continually engaging faculty in systematic discussions of the student experience in department courses and working collaboratively with faculty to improve these experiences. This type of department-level leadership is uncommon (Martin et al., 2003). Department- based team learning focused on teaching is also sometimes described as analogous to the devel- opment of new knowledge by a research team. For example, Marbach-Ad et al. (2007) described a “research group approach” that was used to develop a sequence of microbiology courses. A small group of faculty members identified instructional goals, searched the literature for relevant instructional methods, developed new instructional methods, and tested and refined these meth- ods. Kerr and Runquist (2005) described a similar approach using diverse teams of faculty and students in their chemistry curriculum development project. As they noted, working as a team, as opposed to a committee, for example, is not a common work mode in higher education, and team training is important.
Learning organizations example Although the learning organization framework has not been applied much in the higher education literature, there is some evidence that higher educa- tion institutions can shift their management processes to operate more like learning organiza- tions (Dirckinck-Holmfeld & Lorentsen, 2003; Kezar, 2006). For example, Aalborg University (Denmark) used a learning organization perspective to increase the productive incorporation of information technology throughout institutional operations (Dirckinck-Holmfeld & Lorentsen, 2003). On the basis of this perspective, they organized their initiative around development proj- ects in which small groups would develop new ideas for their local environments. Through increased sharing of ideas between these groups, good ideas would gradually become apparent and then these could be institutionalized. A number of specific mechanisms were used to facili- tate the sharing of ideas and resulting organizational learning. For example, to facilitate organi- zational learning from the development projects, there was a core project group consisting of instructors from across the campus. One of the activities of this group was to regularly visit and learn about specific projects and then work to incorporate productive new ideas into formal organizational policy through participation in various institutional boards.
ComplexityLeadershipTheory Underlying logic STEM undergraduate instruction is governed by a complex system. Inno-
vation will occur through the collective action of self-organizing groups within the system. This col- lective action can be stimulated, but not controlled.
Description Complexity has been an increasingly productive area of study in both natural and social sciences (e.g., Goldstein, Hazy, & Lichtenstein, 2010). Complex systems cannot be completely described and, thus, the dynamics of a complex system cannot be completely con- trolled or predicted. An important phenomenon in complex systems is emergence – the devel- opment of new system properties or structures from relatively small or routine interactions of system elements. Complexity leadership theory combines ideas from complexity science and social network analysis to help organizations better understand how to create organizational conditions that are likely to lead to productive emergence. Cycles of emergence can occur at a variety of scales in an organization, from the emergence of new work patterns in a small work
Increasing Evidence-Based Teaching in STEM Education 241
group to restructuring of the entire organization. Within each cycle, the role of the leader is to create the conditions for productive emergence to occur and then to take the productive results from this process (not all results will be productive) and ensure that these are integrated into the organization (Schreiber & Carley, 2008).
Plowman et al. (2007) identified three key mechanisms that leaders can and should use to encourage and support emergence: disrupting existing patterns, encouraging novelty, and acting as sense makers. They based these mechanisms on a literature review of traditional and complexity views of leadership along with a rich case study of the transformation of an urban church. They found that disruption of existing patterns is necessary for new ideas to emerge. Leaders can amplify or reinforce existing disruption from the external environment (e.g., changing market conditions; Goldstein, Hazy, & Lichtenstein, 2010) or intentionally disrupt existing patterns by, for example, developing heterogeneous working groups (Uhl- Bien & Marion, 2009). The tension created by disruption of existing patterns creates the conditions for new ideas to emerge. Yet emergence of new ideas does not result just from disruption. Members of the organization need to feel some degree of interdependence and be working towards a simple and understandable goal; such conditions can and should be fostered by leaders (Uhl-Bien & Marion, 2009; Uhl-Bien, Marion, & McKelvey, 2007). Finally, leaders need to act as sense makers, through the recognition and amplification of good ideas developed in one pocket of the organization to the broader organization (Goldstein et al., 2010) and by using consistent language to connect good ideas to core organizational goals. These aspects of complexity leadership theory are summarized in Table 5.
Ideas from complexity leadership theory are promising because they have been shown to be productive in understanding leadership in business settings (e.g., Goldstein et al., 2010; Uhl-Bien & Marion, 2009; Uhl-Bien et al., 2007). We recommend that future research efforts explore this promise.
Complexity leadership theory example We are not aware of the application of complex- ity leadership theory in higher education settings. Thus, we draw this example from the non- profit sector. Plowman et al. (2007) conducted a case study of the transformation of “Mission Church.” Within a short period of time, this church in an urban center reversed a fifty-year decline to become a vibrant community focused on advocacy and support for the city’s poor. The authors interpreted the case study through the three complexity leadership
Table 5 Summary of Leader Actions from a
Complexity Leadership Theory Perspective
Leadership mechanism Complex leadership compared with traditional leadership
Disrupt existing patterns Complex leaders enable emergent futures by disrupting existing patterns through the use of conflict and uncertainty; whereas traditional leaders create knowable futures by minimizing conflict and eliminating uncertainty.
Encourage novelty Complex leaders enable emergent self-organization by encouraging innovation through the articulation of simple goals and the promotion of autonomous interdependence; whereas traditional leaders operate as controllers by leading through command and control.
Act as sense makers Complex leaders enable emergent self-organization by interpreting emerging events and amplifying good ideas; whereas traditional leaders operate as controllers by directing order.
Note. Adapted from Plowman et al. (2007).
242 Borrego & Henderson
mechanisms described in Table 5. The church leadership disrupted existing patterns by highlighting conflict (internal conflict between members of the church) and uncertainty about the future and also encouraging new voices to enter the conversation. They promoted novelty by applying heuristics (“What would Jesus do?”) and fostering interactions among community members. Finally, the church leadership facilitated sense making by labeling behaviors and helping the church develop a common language to describe its emerging focus on homeless ministry. Plowman et al. concluded by summarizing their complexity leadership perspective in which the role of leaders is to “help organizations take the moment and make the best of it, without knowing what is going to happen next” (p. 355).
Summary The Shared Vision category of change strategies emphasizes the importance of innovations coming from the front lines of the organization and the importance of these innovations being developed by groups of individuals with diverse perspectives. Two ways for this to hap- pen have been identified. The learning organization perspective emphasizes the importance of middle managers in the knowledge-creation process by mediating the interactions among knowledge producers on the front lines and incorporating this knowledge into organizational operations. The complexity leadership perspective emphasizes disruption and the role of organizational leaders at all levels to recognize and incorporate new ideas and practices into organizational operations, often while managing complex and unpredictable consequences. The theory’s concepts of emergence and organizational learning are important new focus areas for research on change in higher education.
Discussion For several decades, researchers have attempted to make sense of the complex relationships within and across scales of STEM higher education change, for example, organizational and individual change. The cutting edge of this research maps the variety of perspectives on change with respect to each other. We based this article on the model of change strategies developed by Henderson et al. (2010, 2011). Now that we have identified different change strategies and articulated some of the relationships between them, an important next step for change researchers is to understand under what circumstances which strategies work best, and what are particularly powerful combinations of different strategies. It is sensible to assume that employing multiple perspectives on change will lead to better results, but the fact remains that there is little empirical evidence or theory-based rationale to support or refine this assertion (which itself is an important direction for future research).
STEM higher education change agents are ready for a fresh perspective on change, even though there are many aspects of diffusion and implementation (Category I) yet to be explored. In STEM undergraduate instructional change, many of us believe too much effort has gone into creating plug-and-play curriculum modules that are not flexible enough for instructors to comfortably adapt to their teaching situations and goals (Feser et al., 2012). Current accreditation practices such as quality assurance (Category III) have a bad reputation in the United States as change strategies because the ABET engineering reforms adopted in 2000 appear to have stagnated and the more general regional accreditation is often seen as only minimally effective. The literature helps us understand that quality assurance in higher education should not be considered as a cutting-edge change strategy; rather, the approach is suited to bringing a large number of programs up to a minimum standard. Faculty learning
Increasing Evidence-Based Teaching in STEM Education 243
communities and their variations (Category II) are increasing in use; these programs include projects funded by NSF’s Widening Implementation and Demonstration of Evidence-based Reforms (WIDER) program (National Science Foundation, n.d.). The greatest opportunity for innovation is the Shared Vision category. There are few published examples of how STEM educators have employed strategies such as learning organizations and complexity leadership theory for the purposes of instructional change. Yet there is enthusiasm for explor- ing this category (as evidenced by the emphasis on complexity, systems, and transformation in the call for papers for this special issue of this Journal ) and for creatively combining strate- gies from multiple categories. Exploring unfamiliar categories of change strategies requires change agents to let go of some control, just as many teaching innovations require instructors to do. Change agents must take on the less familiar role of a facilitator who adjusts to new information and uncertain situations.
More appropriate than focusing exclusively on Category IV would be to develop systems- level thinking about how various change initiatives focus on different aspects of STEM higher education and therefore complement each other (Groff, 2013). Over time and across initiatives, it is wise to employ a range of perspectives. Focusing too narrowly on one per- spective increases the chances of overlooking influential factors and processes. Collectively, our change efforts and investigations in engineering education should consider all four quad- rants of Figure 1 to encompass a variety of approaches and frames for addressing the prob- lem of improving undergraduate STEM instruction. Considering a wide range of change strategies is similar to the complementary insights gained by considering the results of both quantitative and qualitative studies: single studies use a particular approach, but the whole picture is understood much better in the context of a variety of approaches, as in, for exam- ple, STEM student retention. An important key to these advances is connecting studies that follow a different change strategy or theory by contextualizing them in the bigger picture of alternative foci and the approaches of the four change categories presented here.
Considering diverse change strategies across the STEM educational system will require a diverse set of goals focusing at different levels. In STEM instructional change, prescriptive outcomes have traditionally focused on very specific content or learning outcomes and have diffused through disciplinary or subdisciplinary networks (e.g., mechanics, thermal sciences). Targeting subdisciplinary networks stands in stark contrast to organization-focused change strategies across a department or college of engineering. We are not necessarily saying that the categories presented here are incompatible, but they have traditionally been undertaken by different groups working toward the same broad agenda of STEM higher education instructional change. A systems-level perspective could help better coordinate these efforts so that important connections could be made to synergistically reap the benefits of multiple strategies. Recommending such a perspective is fundamentally different from suggesting everyone should be working in Category IV on complexity leadership approaches.
Finally, this article has focused on efforts to change faculty, but it is interesting to con- sider the implications of various strategies on students. These instructional changes aim to produce students who are more innovative, flexible, and teamwork oriented and able to navi- gate complexity and ambiguity. Engineering instructors and educational environments should be modeling these skills and values for students, and in some cases actively engaging students in change initiatives. Getting out of the familiar Curriculum and Pedagogy category (I) opens up creativity (emergent categories, II and IV) and collaboration (environments and structures categories, III and IV).
244 Borrego & Henderson
Conclusion Strong evidence suggests that most people only consider a single perspective or a limited set of perspectives when they undertake change (Henderson et al., 2011), at least in part because underlying assumptions about change often remain implicit. In this review, we have tried to make these assumptions explicit.
Choice of a change strategy for any specific situation should be based on many factors, such as the type and specificity of change desired, the resources available, and the power and position of the change agent. We cannot emphasize enough the importance of change agents and researchers communicating their assumptions, emphases, and interpretations. We hope that the descriptions of change strategies presented in this review will help to facilitate this communication. Being explicit about the underlying logic guiding the change initiative or evaluation of the initiative will help to advance our collective thinking and make sense of the myriad approaches and perspectives on change.
There is particular interest in complexity theories and systems-level perspectives, which attempt to integrate a diverse set of influences. Although significantly absent from current practice within STEM instructional change, documents such as the call for papers for this special issue of this Jour- nal are increasingly emphasizing complexity and systems approaches to change. The Shared Vision category is potentially the most transformational of the strategies. Transformation is currently in vogue, likely due to dissatisfaction with the rate of undergraduate STEM instructional change. But not every change effort should aspire to be transformational. Small-scale changes and incremental improvements are important steps toward long-term goals of changing undergraduate STEM instruction for the better. It is through coordinated work at multiple levels that true transformational change is likely to occur. It may be that the most important step we can take toward transforming STEM higher education lies in informing and evaluating our change efforts by linking them to liter- ature, models, and theory, and in understanding the limitations of any one particular strategy. In order for change efforts to be truly scholarly, change agents should articulate their change strategy and collect and report evidence to evaluate the effectiveness of the strategy. In reports and articles, strategies should be situated in the broader context of the possible strategies that could have been selected. Articulating and situating change strategies will ensure that STEM education efforts con- tinue to build upon each other and have the desired – and cumulatively transformational – impact.
Acknowledgments This work was supported by the U.S. National Science Foundation through grant 1037671, and while M. Borrego was working at the Foundation. The opinions expressed are those of the authors and do not necessarily represent those of the NSF. The authors wish to thank editors and anonymous reviewers whose constructive comments improved the quality of the final product.
References American Society for Engineering Education (ASEE). (2009). Creating a culture for schol-
arly and systematic innovation in engineering education: Ensuring U.S. engineering has the right people with the right talent for a global society. Washington, DC: Author.
American Society for Engineering Education (ASEE). (2012). Innovation with impact: Creating a culture for scholarly and systematic innovation in engineering education. Washington, DC: Author.
Increasing Evidence-Based Teaching in STEM Education 245
Amundsen, C., & Wilson, M. (2012). Are we asking the right questions?: A conceptual review of the educational development literature in higher education. Review of Educa- tional Research, 82(1), 90–126. doi: 10.3102/0034654312438409
Argyris, C., & Sch€on, D. A. (1974). Theory in practice: Increasing professional effectiveness. San Francisco, CA: Jossey-Bass.
Baillie, C. (2007). Education development within engineering. European Journal of Engineer- ing Education, 32(4), 421–428.
Beach, A. L., & Cox, M. D. (2005). The impact of faculty learning communities on teaching and learning: Results of a national survey. Paper presented at the 30th Association for the Study of Higher Education (ASHE) Annual Conference, Philadelphia, PA.
Beddoes, K., & Borrego, M. (2011). Feminist theory in three engineering education research journals: 1995–2008. Journal of Engineering Education, 100(2), 281–303.
Beer, M. (1987). Revitalizing organizations: Change process and emergent model. Academy of Management Executive, 1, 51–55.
Besterfield-Sacre, M., Cox, M. F., Borrego, M., Beddoes, K., & Zhu, J. (2014). Changing engineering education: Views of U.S. faculty, chairs, and deans. Journal of Engineering Education, 103(2) [this issue].
Birnbaum, R. (2000). The life cycle of academic management fads. Journal of Higher Educa- tion, 71(1), 1–16.
Borrego, M. (2007). Conceptual difficulties experienced by engineering faculty becoming engineering education researchers. Journal of Engineering Education, 96(2), 91–102.
Borrego, M., & Bernhard, J. (2011). The emergence of engineering education research as an internationally connected field of inquiry. Journal of Engineering Education, 100(1), 14–47.
Borrego, M., Cutler, S., Prince, M., Henderson, C., & Froyd, J. E. (2013). Fidelity of implementation of research–based instructional strategies (RBIS) in engineering science courses. Journal of Engineering Education, 102(3), 394–425. doi:10.1002/jee.20020
Borrego, M., Froyd, J. E., & Hall, T. S. (2010). Diffusion of engineering education innova- tions: A survey of awareness and adoption rates in U.S engineering departments. Journal of Engineering Education, 99(3), 185–207.
Braskamp, L. A., & Ory, J. C. (1994). Assessing faculty work: Enhancing individual and insti- tutional performance. San Francisco, CA: Jossey-Bass.
Brookfield, S. D. (1995). Becoming a critically reflective teacher. San Francisco, CA: Jossey- Bass.
Calkins, S., & Light, G. (2007). Promoting learning focused teaching through a project based faculty development program. To Improve the Academy, 26, 217–229.
Cashin, W. E. (1999). Student ratings of teaching: Uses and misuses. In P. Seldin (Ed.), Changing practices in evaluating teaching (pp. 25–44). Bolton, MA: Anker Publishing.
Center for Civic Partnerships. (2007). Theory of change. Retrieved from http://www.civic- partnerships.org/docs/tools_resources/Theory%20of%20Change%209.07.htm
Cochran-Smith, M., & Lytle, S. L. (1999). Relationships of knowledge and practice: Teacher learning in communities. Review of Research in Education, 24, 249–305. doi:10.3102/ 0091732x024001249
Connolly, M. R., Bouwma-Gearhart, J. L., & Clifford, M. A. (2007). The birth of a notion: The windfalls and pitfalls of tailoring an SoTL-like concept to scientists, mathematicians, and engineers. Innovative Higher Education, 32(1), 19–34.
Cooper, J. D. (2008). Professional development: An effective research-based model. Boston, MA: Houghton Mifflin Harcourt.
246 Borrego & Henderson
Cox, M. D. (2004). Introduction to faculty learning communities. In M. D. Cox & L. Richlin (Eds.), New directions for teaching and learning (pp. 5–23). San Francisco, CA: Jossey-Bass.
Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: Sage.
Crotty, M. (1998). The foundations of social research: Meaning and perspective in the research process. Thousand Oaks, CA: Sage.
Dancy, M. H., Turpen, C., & Henderson, C. (2010). Why do faculty try research based instructional strategies? Proceedings of the AIP Conference, Portland, OR.
de Graaff, E., & Kolmos, A. (2007). Process of changing to PBL. In E. de Graaff & A. Kolmos (Eds.), Management of change: Implementation of problem-based and project-based learning in engineering (pp. 31–44). Rotterdam, the Netherlands: Sense.
De Wit, H. (2000). The Sorbonne and Bologna declarations on European higher education. International Higher Education, 18, 8–9.
Dill, D. D. (1999). Academic accountability and university adaptation: The architecture of an academic learning organization. Higher Education, 38(2), 127–154.
Dirckinck-Holmfeld, L., & Lorentsen, A. (2003). Transforming university practice through ICT-integrated perspectives on organizational, technological, and pedagogical change. Interactive Learning Environments, 11(2), 91–110.
Emerson, J. D., & Mosteller, F. (2000). Development programs for college faculty: Prepar- ing for the twenty-first century. Educational Media and Technology Yearbook, 25, 26–42.
Erklenz-Watts, M., Westbay, T., & Lynd-Balta, E. (2006). An alternative professional development program: Lessons learned. College Teaching, 54(3), 275–279.
Ewell, P. T. (1997). Strengthening assessment for academic quality improvement. In M. W. Peterson, D. D. Dill, L. A. Mets, & associates (Eds.), Planning and management for a changing environment: A handbook on redesigning postsecondary institutions. San Francisco, CA: Jossey-Bass.
Felder, R. M., & Brent, R. (2003). Designing and teaching courses to satisfy the ABET engineering criteria. Journal of Engineering Education, 92(1), 7–25.
Felder, R. M., Brent, R., & Prince, M. (2011). Engineering faculty development. Journal of Engineering Education, 100(1), 89–122.
Feser, J., Borrego, M., Pimmel, R., & Della-Piana, C. (2012). Results from a survey of National Science Foundation Transforming Undergraduate Education in STEM (TUES) program reviewers. Paper presented at the ASEE Annual Conference & Exposition, San Antonio, TX.
Fisher, D., Merron, K., & Torbert, W. R. (1987). Human development and managerial effectiveness. Group and Organization Studies, 12, 257–273.
Fixsen, D. L., Naoom, S. F., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, National Implementation Research Network.
Fleming, S., Shire, J., Jones, D., Pill, A., & McNamee, M. (2004). Continuing professional development: Suggestions for effective practice. Journal of Further and Higher Education, 28(2), 165–177.
Flyvbjerg, B. (2001). Making social science matter: Why social inquiry fails and how it can suc- ceed again. Oxford, UK: Cambridge University Press.
Froyd, J. E., Borrego, M., Cutler, S., Henderson, C., & Prince, M. (2013). Estimates of use of research-based instructional strategies in core electrical or computer engineering courses. IEEE Transactions on Education, 56(4), 393–399.
Increasing Evidence-Based Teaching in STEM Education 247
Froyd, J. E., Penberthy, D., & Watson, K. (2000). Good educational experiments are not neces- sarily good change processes. Paper presented at the ASEE/IEEE Frontiers in Education Conference, Kansas City, MO.
Froyd, J. E., Wankat, P. C., & Smith, K. A. (2012). Five major shifts in 100 years of engi- neering education. Proceedings of the IEEE, 100, 1344–1360.
Gardner, D. G., Dunham, R. B., Cummings, L. L., & Pierce, J. L. (1987). Employee focus of attention and reactions to organizational change. Journal of Applied Behavioral Science, 23, 351–370.
Gladwell, M. (2000). The tipping point: How little things can make a big difference. Boston, MA: Little, Brown.
Goldstein, J., Hazy, J. K., & Lichtenstein, B. B. (2010). Complexity and the nexus of leader- ship: Leveraging nonlinear science to create ecologies of innovation. New York, NY: Palgrave Macmillan.
Graham, R. (2012). Achieving excellence in engineering education: The ingredients of suc- cessful change. London: Royal Academy of Engineering.
Greenwald, A. (1997). Validity concerns and usefulness of student ratings of instruction. American Psychologist, 52(11), 1182–1186.
Groff, J. S. (2013). Dynamic systems modeling in educational system design and policy. Journal of New Approaches in Educational Research, 2(2), 72–81. doi: 10.7821/naer.2.2.72-81
Hargreaves, A., Lieberman, A., Fullan, M., & Hopkins, D. (Eds.). (2009). Second interna- tional handbook of educational change. Dordrecht, the Netherlands: Springer.
Harvey, L. (2004). The power of accreditation: Views of academics. Journal of Higher Educa- tion Policy and Management, 26(2), 207–223. doi: 10.1080/1360080042000218267
Hawwash, K. (2007). Attractiveness of education. In C. Borri & F. Maffioli (Eds.), TREE: Teaching and research in engineering in Europe: Re-engineering engineering education in Europe. Florence, Italy: Firenze University Press.
Henderson, C., Beach, A., & Finkelstein, N. (2011). Facilitating change in undergraduate STEM instructional practices: An analytic review of the literature. Journal of Research in Science Teaching, 48(8), 952–984. doi: 10.1002/tea.20439
Henderson, C., Beach, A., Finkelstein, N., & Larson, R. S. (2008). Facilitating change in undergraduate STEM: Initial results from an interdisciplinary literature review. Paper pre- sented at the Physics Education Research Conference, Edmonton, Canada.
Henderson, C., Dancy, M. H., & Niewiadomska-Bugaj, M. (2012). The use of research- based instructional strategies in introductory physics: Where do faculty leave the innovation-decision process? Physical Review Special Topics – Physics Education Research, 8(2), 020104.
Henderson, C., Finkelstein, N. D., & Beach, A. (2010). Beyond dissemination in college science teaching: An introduction to four core change strategies. Journal of College Science Teaching, 39(5), 18–25.
Hickman, C. W., & Sussman, J. L. (2013). Dispelling myths: COMMON misconceptions about ABET and accreditation. Retrieved from http://www.abet.org/uploadedFiles/ Events/Presentation_from_Past_Events/ASEE_2011/asee-dispelling-myths-about-abet- 2013.pdf
Hutchings, P., & Shulman, L. S. (1999). The scholarship of teaching: New elaborations, new developments. Change, 31(5), 10–15.
Kerr, S., & Runquist, O. (2005). Are we serious about preparing chemists for the 21st century workplace or are we just teaching chemistry? Journal of Chemical Education, 82(2), 231–233.
248 Borrego & Henderson
Keystone Accountability. (2009). Developing a theory of change. Retrieved from http:// www.keystoneaccountability.org/sites/default/files/2%20Developing%20a%20theory%20of% 20change.pdf
Kezar, A. (2001). Understanding and facilitating organizational change in the 21st century (ASHE-ERIC Higher Education Report, Vol. 28, No. 4). Washington, DC: U.S. Department of Education.
Kezar, A. (2006). Redesigning for collaboration initiatives: An examination of four highly collaborative campuses. Journal of Higher Education, 77(5), 804–838.
King, R. (2008). Addressing the supply and quality of engineering graduates for the new century: Surry Hills, NSW, Australia: Carrick Institute.
Kolmos, A., Vinther, O., Andersson, P., Malmi, L., & Fuglem, M. (Eds.). (2004). Faculty development in Nordic engineering education. Aalborg, Denmark: Aalborg University Press.
Koro-Ljungberg, M., & Douglas, E. P. (2008). State of qualitative research in engineering education: Meta-analysis of JEE articles, 2005–2006. Journal of Engineering Education, 97(2), 163–176.
Kotter, J. P. (1995). Leading change: Why transformation efforts fail. Harvard Business Review, 73(2), 59–67.
Kotter, J. P. (1996). Leading change. Boston, MA: Harvard Business School Press. Kotter, J. P., & Schlesinger, L. A. (1979). Choosing strategies for change. Harvard Business
Review, 57(2), 106–114. Kulik, J. A. (2001). Student ratings: Validity, utility, and controversy. In M. Theall, P. C.
Abrami, & L. A. Mets (Eds.), The student ratings debate: Are they valid? How can we best use them? (pp. 9–25). San Francisco, CA: Jossey Bass.
Levinson-Rose, J., & Menges, R. J. (1981). Improving college teaching: A critical review of research. Review of Educational Research, 51(3), 403–434.
Lohmann, J. R. (1999). EC 2000: The Georgia Tech experience. Journal of Engineering Education, 88(3), 305–310.
Lynd-Balta, E., Erklenz-Watts, M., Freeman, C., & Westbay, T. D. (2006). Professional development using an interdisciplinary learning circle: Linking pedagogical theory to practice. Journal of College Science Teaching, 35(4), 18–24.
Marbach-Ad, G., Briken, V., Frauwirth, K., Gao, L.-Y., Hutcheson, S. W., Joseph, S. W., & Mosser, D. (2007). A faculty team works to create content linkages among various courses to increase meaningful learning of targeted concepts of microbiology. CBE Life Sciences Education, 6(2), 155–162. doi: 10.1187/cbe.06-12-0212
Marsh, H., & Roche, L. (1997). Making students’ evaluations of teaching effectiveness effective. American Psychologist, 52(11), 1187–1197.
Martin, E., Trigwell, K., Prosser, M., & Ramsden, P. (2003). Variation in the experience of leadership of teaching in higher education. Studies in Higher Education, 28(3), 247–260.
McKinney, K. (2007). Enhancing learning through the scholarship of teaching and learning. Bolton, MA: Anker.
McLaughlin, J. A., & Jordan, G.B. (1999). Logic models: A tool for telling your program’s performance story. Evaluation and Program Planning, 22(1), 65–72.
Mettetal, G. (2001). The what, why and how of classroom action research. Journal of the Scholarship of Teaching and Learning, 2(1), 6–13.
Milstein, B., & Chapel, T. (n.d.). Developing a logic model or theory of change. University of Kansas, Lawrence, KS, The Community Toolbox. Retrieved from http://ctb.ku.edu/en/ tablecontents/sub_section_main_1877.aspx
Increasing Evidence-Based Teaching in STEM Education 249
Montfort, D., Brown, S., & Pegg, J. (2012). The adoption of a capstone assessment instru- ment. Journal of Engineering Education, 101(4), 657–678.
Nadler, D. A., & Tushman, M. L. (1989). Organizational frame bending: Principles for managing reorientation. Academy of Management Executive, 3, 194–204.
National Academy of Engineering (NAE). (2004). The engineer of 2020. Washington, DC: National Academies Press.
National Research Council (NRC). (2012). Discipline-based educational research: Understand- ing and improving learning in undergraduate science and engineering. Washington, DC: National Academies Press.
National Science Foundation (NSF). (2010). Transforming Undergraduate Education in Sci- ence, Technology, Engineering and Mathematics (TUES). Retrieved from http://www.nsf. gov/pubs/2010/nsf10544/nsf10544.pdf
National Science Foundation (NSF). (n.d.). Retrieved from http://www.nsf.gov/award search/advancedSearchResult?ProgEleCode51133& BooleanElement5ANY&Boolean Ref5ANY& ActiveAwards5true& #results
Nonaka, I., & Takeuchi, H. (1995). The knowledge-creating company. New York, NY: Oxford University Press.
Piderit, S. K. (2000). Rethinking resistance and recognizing ambivalence: A multidimen- sional view of attitudes toward organizational change. Academy of Management Review, 25(4), 783–794.
Plowman, D., Solansky, S., Beck, T., Baker, L., Kulkarni, M., & Travis, D. (2007). The role of leadership in emergent, self-organization. Leadership Quarterly, 18(4), 341–356. doi: 10.1016/j.leaqua.2007.04.004
Ponitz, D. H. (1997). The journey of transformation for Sinclair Community College. In T. O’Banion (Ed.), A learning college for the 21st century (pp. 104–126). Westport, CT: Oryx Press.
Porras, J. I., & Silvers, R. C. (1991). Organization development and transformation. Annual Review of Psychology, 42, 51–78.
Prados, J. W., Peterson, G. D., & Lattuca, L. R. (2005). Quality assurance of engineering education through accreditation: The impact of Engineering Criteria 2000 and its global influence. Journal of Engineering Education 94(1), 165–184.
President’s Council of Advisors on Science and Technology. (2012). Engage to excel: Produc- ing one million additional college graduates with degrees in science, technology, engineering, and mathematics. Washington, DC: Author.
Prince, M., Borrego, M., Henderson, C., Cutler, S., & Froyd, J. (2013). Use of research- based instructional strategies in core chemical engineering courses. Chemical Engineering Education, 47(1), 27–37.
Prince, M., & Felder, R. M. (2006). Inductive teaching and learning methods: Definitions, comparisons, and research bases. Journal of Engineering Education, 95(2), 123–138.
Ramsden, P., Prosser, M., Trigwell, K., & Martin, E. (2007). University teachers’ experien- ces of academic leadership and their approaches to teaching. Learning and Instruction, 17(2), 140–155.
Raubenheimer, C. D., & Myka, J. L. (2005). Using action research to improve teaching and student learning in college. Journal of College Science Teaching, 34(6), 12–16.
Rhoades, G., & Sporn, B. (2002). Quality assurance in Europe and the U.S.: Professional and political economic framing of higher education policy. Higher Education Quarterly, 43, 355–390. doi:10.1023/A:1014659908601
Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York, NY: Free Press.
250 Borrego & Henderson
Scales, K., Owen, C., Shiohare, S., & Leonard, M. (1998). Preparing for program accredita- tion review under ABET Engineering Criteria 2000: Choosing outcome indicators. Jour- nal of Engineering Education, 87(3), 207–210.
Sch€on, D. (1983). The reflective practioner. New York, NY: Basic Books. Sch€on, D. (1987). Educating the reflective practioner: Toward a new design for teaching and
learning in the professions. San Francisco, CA: Jossey-Bass. Schreiber, C., & Carley, K. M. (2008). Network leadership: Leading for learning and adapt-
ability. In M. Uhl-Bien & R. Marion (Eds.), Complexity leadership: Part 1. Conceptual foundations (pp. 291–331). Charlotte, NC: Information Age Publishing.
Scott, D. C., & Weeks, P. A. (1996). Collaborative staff development. Innovative Higher Education, 21(2), 101–111.
Senge, P. M. (1997). The fifth discipline. New York, NY: Doubleday. Senge, P. M. (2000). The academy as learning community: Contradiction in terms or realiz-
able future? In A. F. Lucas (Ed.), Leading academic change: Essential roles for department chairs (pp. 275–300). San Francisco, CA: Jossey Bass.
Seymour, E. (2002). Tracking the processes of change in US undergraduate education in sci- ence, mathematics, engineering, and technology. Science Education, 86(1), 79–105.
Steering Committee for Evaluating Instructional Scholarship in Engineering. (2009). Devel- oping metrics for assessing engineering instruction: What gets measured is what gets improved. Washington, DC: National Academy of Engineering.
Stes, A., Min-Leliveld, M., Gijbels, D., & Van Petegem, P. (2010). The impact of instruc- tional development in higher education: The state-of-the-art of the research. Educational Research Review, 5(1), 25–49.
Streveler, R. A., Borrego, M., & Smith, K. A. (2007). Moving from the scholarship of teaching and learning to educational research: An example from engineering. To Improve the Academy, 25, 139–149.
Sykes, G., Schneider, B., & Plank, D. N. (Eds.). (2012). Handbook of educational policy research. New York, NY: Routledge.
Uhl-Bien, M., & Marion, R. (2009). Complexity leadership in bureaucratic forms of organizing: A meso model. Leadership Quarterly, 20(4), 631–650. doi: 10.1016/j.leaqua.2009.04.007
Uhl-Bien, M., Marion, R., & McKelvey, B. (2007). Complexity leadership theory: Shifting leadership from the industrial age to the knowledge era. Leadership Quarterly, 18(4), 289– 318. doi: 10.1016/j.leaqua.2007.04.002
Volkwein, J. F., Lattuca, L. R., Harper, B. J., & Domingo, R. J. (2007). Measuring the impact of professional accreditation on student experiences and learning outcomes. Research in Higher Education, 48(2), 251–282.
W. K. Kellogg Foundation. (2004). Logic model development guide. Battle Creek, MI: Author.
Weimer, M., & Lenze, L. F. (1997). Instructional interventions: A review of the literature on efforts to improve instruction. In R. P. Perry & J. C. Smart, (Eds.), Effective teaching in higher education: Research and practice (pp. 205–240). New York, NY: Agathon Press.
Weiss, C. H. (1995). Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives for children and families. In J. P. Connell (Ed.), New approaches to evaluating community initiatives: Concepts, methods, and contexts (pp. 65–92). Queenstown, MD: Aspen Institute.
Wejnert, B. (2002). Integrating models of diffusion of innovations: A conceptual framework. Annual Review of Sociology, 28(1), 297–326. doi: 10.1146/annurev.soc.28.110601.141051
Increasing Evidence-Based Teaching in STEM Education 251
Wildman, T. M., Hable, M. P., Preston, M. M., & Magliaro, S. G. (2000). Faculty study groups: Solving “good problems” through study, reflection, and collaboration. Innovative Higher Education, 24(4), 247–263.
Zeichner, K. M., & Liston, D. P. (1987). Teaching student teachers to reflect. Harvard Educational Review, 57, 23–48.
Zeichner, K. M., & Noffke, S. E. (2001). Practitioner research. In V. Richardson (Ed.), Handbook of research on teaching (4th ed., pp. 298–330). Washington, DC: American Educational Research Association.
Authors Maura Borrego is Associate Dean and Director of Interdisciplinary Graduate Education
in the Graduate School and associate professor of engineering education at Virginia Tech, 660 McBryde Hall (0218), Blacksburg, VA 24061; mborrego@vt.edu.
Charles Henderson is an associate professor with appointment between the Department of Physics and the Mallinson Institute for Science Education at Western Michigan University, 1903 W. Michigan Avenue, Kalamazoo, Michigan, 49008-5252; charles.henderson@ wmich.edu.
252 Borrego & Henderson