Introduction Organizational Readiness for an Innovation

The science of effective practice continues to grow. We are getting better and

better at understanding causal models of health disorders and the interventions that can

alleviate symptoms and promote health and wellness. There are a variety of different

types of innovations that can help us to reach outcomes. An innovation can be any

policy, program, process, or technology that is new to a setting (e.g. Hall & Hord, 2011;

Rogers, 2003). However, the innovations that result from science are not always

effectively implemented among organizations (Durlak & Dupre, 2008; Fixsen, Naoom,

Blasé, Friedman, & Wallace, 2005). When we implement innovations with quality, we

put them into practice so that they reach their intended outcomes (Meyers, Katz, Chien,

Wandersman, Scaccia, et al., 2012).

To create quality in health services, we need to deliberately and comprehensively

approach the process of implementation (Wandersman, Duffy, Flaspohler, Noonan,

Lubell, et al., 2008). Implementing innovations with quality is especially difficult and

complex (Fixsen et al., 2008; Meyers, Durlak, & Wandersman, 2012). An increased

emphasis on the factors that enhance implementation can better help organizations put

innovations into place. To that end, we often need to consider the conditions that

contribute to whether or not an innovation will have its intended impact. These

conditions can predict how ready an organization is to implement an innovation.

In the organizational literature there is general agreement that readiness is an

essential part of successfully implementing an innovation (e.g. Drzensky, Egold, & Van

Dick; Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004; Hall & Hord, 2011;

Simpson, 2002; Weiner, 2009). Readiness is a considered a necessary precursor to

successful organization change (Weiner, Amick, & Lee, 2008) and is often embedded

within larger implementation frameworks (Aarons, Hurlburt, & Horwitz, 2011;

Damschroder et al., 2009; Greenhalgh et al., 2004; Powell et al., 2012). Beyond the

consensus that readiness is an important factor in successful change implementation,

however, there has been little agreement about what constitutes readiness as a construct

or how to best measure an organization’s readiness for a given innovation (Aarons et al.,

2011; Damschroder et al., 2009; Greenhalgh et al., 2004; Rafferty, Jimmieson, &

Armenakis, 2013; Simpson, 2002; Weiner et al., 2008). In a comprehensive literature

review on readiness, Weiner et al. (2008) found that (55%) of the articles had no

conceptual definition of readiness, instead deferring to the collective “common sense” of

the readers.

One of these common phrases to describe the change process is that someone or

something must be ready, willing, and able to change. These are important terms but also

indistinct, unclear, and ultimately redundant. The lack of a precise understanding of

readiness above colloquial catchphrases is somewhat troubling, as organizational

readiness is often discussed in the context of determining whether or not a particular

organization will receive a given innovation or support for that innovation (Flaspohler,

Meehan, Maras, & Keller, 2012). When readiness is high, it is presumed that there will

be greater effort dedicated to the change process and more successful implementation

(Weiner et al., 2008). When it is low, not only will implementation not be successful,

organizations may be non-receptive to supportive interventions.

The literature on readiness is vast, complex, and covers multiple organizational

and psychological fields and content areas. This dissertation is not a synthesis of those

frameworks. Rather, I attempt to frame readiness in terms of implementation and the

Support System; that is, how we can address and build the organizational conditions that

foster better implementation. This dissertation attempts to refine readiness as it relates to

implementation of an innovation. I will present a flexible model for readiness that can be

used in assessment, planning, implementation, and evaluation. I will overview some of

the preliminary evidence that suggest how we can go about building readiness.

Readiness can be better understood as a continuous and dimensional construct that

includes multiple components. This model can be applied to multiple settings, multiple

levels, and for multiple innovations. I will overview each of the components of readiness

(motivation, innovation-specific capacity, and general capacity) and discuss the

implications for enhancing the ability of organizations to put innovations into place. This

dissertation approaches readiness not in terms of a summative evaluation framework.

Measuring readiness as an outcome would tell us how certain components have changed,

perhaps as a result of specific support strategies. However, readiness may be more

beneficial if used to inform planning and mid-course changes in implementation

strategies.

This dissertation attempts to fill gaps that were identified by Greenhalgh et al.

(2004) in their highly influential review paper on how innovations are diffused in

organizations. They specify several key questions that are lacking in implementation

literature. These include 1) what steps must be taken to move toward system readiness?

and, 2) how can this process be supported and enhanced? I hypothesize that critically

examining the components of readiness, monitoring these over time, and taking

deliberate steps to build and sustain them may lead to enhanced implementation quality,

and ultimately better outcomes.

The Three Components of Readiness.

Organizations are stable systems of people who work together to achieve common

goals through a division of labor and hierarchy of ranks and responsibilities (Rogers,

2003). Broadly, Organizational Readiness is the extent to which an organization is both

willing and able to implement a particular innovation (Drzensky et al., 2012; Rafferty et

al., 2013; Weiner et al., 2008; Weiner, 2009). This includes the organization’s motivation

to implement and the organizational capacities to implement and intentional change

(Flaspohler, Duffy, Wandersman, Stillman, & Maras, 2008; Weiner et al., 2008). This

definition reflects the colloquial understanding of readiness, as well as Weiner et al.’s

(2008) review of the literature indicating that authors generally approach readiness either

in terms of psychological beliefs, attitudes, and intentions, or alternatively in terms of

structural capabilities.

To unite and extend the concepts of willingness and ability, we operationalize

three specific and dynamic components within this construct. Organizational readiness

consists of an organization’s motivation to implement a specific innovation, the general

organizational context and capacities, and their specific capacities for a specific

innovation (Scaccia, Cook, Lamont, Wandersman, Castellow et al., in press). Simply

focusing on the capabilities to put an innovation into place neglects important cognitive

and affective variables. In order to facilitate successful implementation, we must also

examine the motivations and perceptions about an innovation.

Organizational readiness for change involves all three of these dimensions.

Organizational readiness for a specific innovation (notated as i) can thus be framed in the

following manner:

Readinessit = (Motivationi x Innovation-Specific Capacityi x General Capacity)t

Or, as a heuristic,

R = MC2

Each of the interactive constructs can be measured independently and thus offer a

nuanced and actionable understanding of readiness. This heuristic, which is abbreviated

as the non-mathematical R= MC2, suggests that an organization can be high in some

facets of readiness (e.g., motivation) while low in other domains (e.g., innovationspecific

capacity). Readiness can be cross-sectionally assessed at any time during an innovation’s

lifespan (this is the time t). Organizations can be described as more or less ready at any

given time during the lifespan of implementation. The components of readiness can also

change in a positive or negative direction over time depending on a variety of internal

and external influences. This relationship has both qualitative and quantitative utility

depending on the precision of the measurement model that is used.

For a simple example of the components of readiness on an individual-level, see

Appendix A.

Grounding discussions of Readiness: Accountability and Dissemination

Frameworks

When innovations are actively introduced into organizations (as opposed to

passively diffused), there is almost always a formal adoption process, following by a

planning, evaluation, and sustainability phase (Aarons et al., 2011; Chinman, Imm, &

Wandersman, 2004; Greenhalgh et al., 2004). Readiness is often discussed in the context

of determining whether or not an organization is capable of putting a particular

innovation into practice (Flaspohler et al., 2012). Therefore, readiness is an important

construct in dissemination and implementation processes.

Readiness is sometimes contained as minor part within a larger implementation

framework (e.g. Aarons et al., 2011; Damschroder et al., 2009; Greenhalgh et al., 2004).

I view readiness as part of a comprehensive planning framework that includes needs

assessment, goal setting, identification of best or promising practices, planning, and

evaluation (Chinman et al., 2004; Powell et al., 2012). This dissertation proposes the

readiness is not just a precursor to implementation, but a construct that encompasses the

conditions that are necessary to ensure quality implementation through the entirety of the

innovation’s lifespan (adoption, planning, implementing, and institutionalizing).

We further ground readiness within a conceptual model that articulates how innovations

can be supported and implemented. The Interactive Systems Framework for

Dissemination and Implementation (ISF, Wandersman et al., 2008) proposes that within a

larger system perspective there are bidirectional relationships between providers and

support staff that influence how innovations are disseminated and implemented. There

are three different systems in the ISF (Figure 1). The Delivery System is the organization

or community setting that puts an innovation into practice. These are the front-line

practitioners or providers. The Support System uses various strategies to strengthen the

Delivery System’s ability to implement with quality (Wandersman, Chien, & Katz, 2012).

Figure 1.1: Readiness in the Interaction Systems Framework for Dissemination and

Implementation

The Synthesis and Translation System critically evaluates and condenses the

products of research, science, and continuous quality improvement into user-friendly

formats that can be easily accessed and understood by practitioners in the Support and

Delivery Systems (Rapkin, Weiss, Lounsbury, Thompson, Goodman et al., 2012). This

process allows for appropriate innovations to be brought to the attention of organizations

and made more accessible for dissemination (Simpson, 2002). Synthesis and Translation

processes intentionally and deliberately expose the organization to the innovation

(Simpson, 2002), although this knowledge can be obtained through more passive and

informal means (e.g. diffusion; Rogers, 2003)

The ISF has an explicit focus on identifying and building capacity; i.e.,

knowledge, skills, and abilities that are needed in order to implement innovations.

Delivery System

General Capacity Innovation-Specific

Capacity

Motivation

Support System

Readiness Building Strategies

General Capacity Innovation-Specific

Capacity

TranslationSynthesis

Synthesis and Translation System

Existing Research and

Theory

Implementation

Funding

Socio-

political

Climate

Macro-

policy

Outcomes

Motivation

Capacity is the ability of the Delivery System to enact what is required to reach an

intended outcome. In the ISF, the Support System helps to increase the ability of the

Delivery System to implement innovations by building their capacity. Much work in

evaluation and community psychology, especially in Empowerment Evaluation

(Fetterman & Wandersman, 2005) focuses on capacity building as a strategy to increases

the likelihood that innovations will be put into place. Organizational capacities for

behavioral health are informed both internally by the needs and resources of the

organization, as well as externally by the demands of both the service recipient and

community. Innovations should address and fill the service gaps specified by the needs

and resources of the organizations (Flaspohler et al., 2008). Increasing Delivery System

capacity may enhance how well an organization implements an innovation (Chinman et

al., 2004; Elliott, 2003; Flaspohler et al., 2008).

The concept of Readiness was not initially addressed in the ISF (Wandersman et

al., 2008). Some models of readiness propose that building organizational capacity

(either general or innovation- specific) will build readiness to implement an innovation

(Flaspohler et al, 2012; Glisson, 2007). While building capacity is a necessary method

for getting an organization ready to implement, it is also insufficient (Wandersman et al.,

2008; Weiner, 2009; Weiner, Amick, & Lee, 2008). Thus, a distinction must be made

between organizational capacity, organizational resources (e.g. Simpson, 2002), and

organizational readiness (Weiner, 2009). Capacity and Readiness as functional terms are

not interchangeable. An organization may have the capacity to implement a specific

innovation, but not the motivations to put it into practice. Readiness is reflected in the

organizational members’ beliefs, attitudes, and intentions regarding the extent to which

changes are needed (Armenakis, Harris, & Mossholder, 1993) and the organizational

capacity to successfully make those changes. An organization needs to have the

“will/desire/drive” to put an innovation into place, the necessary know-how, and the

organizational conditions to support it. When the Support System works to build the

ability of the Delivery System to implement innovations, they need more holistic

readiness-building strategy that includes motivation and capacities (Figure 1).

Additionally, in order to build readiness in the Delivery System, the Support System must

have its own readiness to enact readiness-building strategies.

Readiness and the Appropriate Innovation.

A person or organization cannot be ready for an innovation that is not specified.

Innovations can often exist in clusters which include multiple, distinguishable parts that

are closely related to one another (Rogers, 2003) and come with a host of conversations,

discourses, and texts (Ford, Ford, & D’Amelio, 2008). Readiness for an innovation can

be successfully determined only after the innovation that meets the underlying

organizational needs is selected. Implementation is also distinct from the initial selection

of an innovation (i.e., adoption) (Klein & Knight, 2005; Klein, Conn, & Sorra, 2001;

Weiner et al., 2008). An organization may have the willingness to try a new innovation,

but it may not have the capacity to implement. Readiness represents an intermediate step

between selecting an innovation and putting it into practice.

An innovation should increase organizational control over the intended outcomes

(Rogers, 2003). If it is not clear how an innovation will reach the intended outcomes,

then this may not be a useful innovation to introduce. Consequently, this dissertation

notes that the definition of readiness rests on the critical assumption that an innovation is

appropriate for an organization and is grounded in the evidence-base for effective practice

(Chinman et al., 2004). Many implementation frameworks that discuss readiness

specifically consider the underlying needs and resources of the organization (e.g.

Damschroder et al., 2009; Simpson, 2002). This should be determined prior to building

readiness (Damschroder & Hagedorn, 2011), not addressed as part of the

readinessbuilding process.

The readiness formula does not imply that an organization should or should not

adopt a specific innovation. Rather, it is a way to describe the current conditions with

respect to the innovation. We do not make any evaluative statements about which type of

innovation is “best” for an organization. If the innovation or change process was chosen

without regard to underlying needs or the evidence-base, then is no rational reason that

the innovation capacity building process will lead to the intended outcomes (Klein, Conn,

& Sorra, 2001). Simply adopting an evidence-based practice does not guarantee

effectiveness in an organizational setting (Durlak & Dupre, 2008; Glisson, 2007). The

organization may have increased readiness to implement, but the innovation may not

have the intended effect, and consequently may not lead to outcomes.

A Need to build Readiness

Readiness for an innovation is often described as a categorical (and sometimes

dichotomous) construct with evaluation implications (Flaspohler et al., 2012; Oetting et

al., 1995; SAMHSA, 2010). Pre-defined cut-points or thresholds are used in assessment

to determine a “stage” of readiness (e.g. Addiction Technology Transfer Center (ATTC),

2010; Hawkins & Catalano, 2002; Oetting et al., 1995; SAMHSA, 2010). This

assessment may be incorporated into a decision-making process, such as whether an

organization receives grant funding or particular support services. When readiness is

categorical (i.e. ready/not ready), there is an underlying premise that organizations that

are not ready will not be able to effectively implement an innovation. Though this

categorization may be necessary in certain contexts (e.g., the allocation of limited fiscal

resources), there are some functional limitations. Organizations with the largest need for

implementation support are often labeled as not being ready for the innovation. The basic

assumption is that these “not ready” organizations will be non-responsive to support

strategies such as technical assistance (TA), which may result in a waste of resources.

However, we view differences in readiness in organizations as a matter of degree

(level of readiness). It is likely that some highly capable organizations are overlooked

because of a low level of initial readiness when, in reality, this level of readiness can be

augmented with support over time. A dimensional model for readiness that is more

multi-faceted is a more actionable construct for measurement and for guiding

interventions strategies because it recognizes these differences. As seen in Appendix A, it

is not sufficient to assign a global construct of readiness. Rather, we need to pinpoint

specific areas within readiness that can be enhanced in order to improve implementation.

This moves away from the idea of “resistance” to an innovation (Ford et al., 2008) by

helping to identify specific components of readiness that can be enhanced.

Support for Readiness. Significant resources are being devoted to innovation

support like training and technical assistance (Wandersman et al., 2012). Readiness as

defined here can be used to proactively identify potential barriers to change and help to

develop strategies to facilitate implementation of a desired innovation (Damschroder &

Hagedorn, 2011). This can help to reduce the likelihood of poor implementation (i.e.

Type III error; Weiner et al., 2008), and ensure that intended outcomes are reached

(Durlak & Dupre, 2008; Meyers et al., 2012b). Proactive support is particularly

important for organizations that are mandated to adopt specific innovations (e.g.

provisions within the Affordable Care Act like the Community Health Needs Assessment

for non-profit hospitals). In many cases, organizations may be unprepared about how to

enact mandated changes. Although mandates from regulatory agencies or funders can

increase an organization’s motivation to adopting an innovation (Beidas et al., 2013; Hall

& Hord, 2011; Flaspohler et al., 2008), mandates do not help to build the capacity of an

organization (Greenhalgh et al., 2004.) Addressing organizational readiness allows the

Support System proactive toward providing support, rather than reactive when addressing

resistance.

Therefore, there is a growing need for tailored, proactive, and effective Support

System activities that can build and sustain innovation readiness in organizations (Baker

et al., 2012; Wensig et al., 2011). Since readiness is a complex, multifaceted construct,

readiness building strategies will need to be matched to the conditions of the host

organization. By assessing each construct separately using R= MC2, the nuances of

readiness in a particular organization for a specific innovation can be better understood.

Organizations that are higher on readiness will not necessarily respond “better” to

Support System strategies. Consistent with empowerment evaluation principles

(Fetterman & Wandersman, 2005) all organizations likely have the potential to get “more

ready” for a particular innovation. Use of R= MC2 helps to identify the level of readiness

among the three components and pinpoint specific areas in need of improvement and

highlights areas of relative strength that can be used as leverage for improving readiness

over time. There are a variety of different strategies available to build readiness, and

provided that they are correctly matched to the readiness of the organization, different

types of readiness building strategies may lead to the same results (Weiner, 2009).

However, this requires that the support strategies be tailored to the components of

readiness and target whatever deficits are assessed (Armenakis et al., 1993; Wensig et al.,

2012). At this time, there is no synthesis of the research that identifies which support

strategies are more effective at enhancing the specific components of readiness.

While R= MC2 is currently non-mathematical (i.e. we have not yet developed

scales or a relative scoring system) there are important logical implications. The only

way that an organization could be completely lacking readiness is for any one of these

factors in R= MC2to be zero (i.e., any number multiplied by zero equals zero). If this

zeroing-out occurs, this is may be when organization may be deemed “not ready.” In

these cases, there is a critical accountability decision about whether Support System

activities should take place under these circumstances since labeling an organization as

“not ready” may rule out the organizations that are most in need of help (Rogers, 2003).

However, when is organizational readiness too low to be responsive to Support System

activities? This is an unresolved question in the literature on Support System activities

and it may be the case that there is no “minimum” amount of readiness. Synthesizing

and Interpreting the Collective Readiness of Individuals in an

Organization:

According to Hall and Hord (2010), “to change an organization we must change

the individuals within it.” Rafferty, Jimmieson, and Armenakis (2013) similarly argue it

is impossible to separate organizational readiness from a perspective that incorporates

multiple levels. Consequently, R=MC2 may be different at individual, group, and

organizational levels. There can be subtle differences between levels of analysis in

organizations, with variations seen in the readiness of individuals (Miller & Rollnick,

2013), groups of individuals (Hall & Hord, 2011; Rogers, 2003), and the organization as

a whole (Rafferty et al., 2013; Simpson, 2002; Weiner, 2009).

Individuals within an organization are hypothetically subject to the same

readiness formula and considering the individual level may be an important part of an

organizational analysis. On the individual level, the concept of readiness is

welldeveloped (DiClemente & Velazquez, 2002; DiClemente, Schlundt, & Gemmel,

2004; Greenhalgh et al., 2004; Miller & Rollnick, 2013). An organization can have

people who are high on certain skills, but low on things specific to an innovation, similar

to how Rogers (2003) described the characteristics of adopters at different time points

(i.e. innovators, early, middle, late, and laggards). While this dissertation addresses the

overall readiness of an organization, it is necessarily made up of individuals within the

organization. This becomes a specific challenge when considering how to assess, manage,

and build the components of organizational readiness. An organization can influence the

individuals within the organization to use an innovation in a number of ways. It can

occur as a result of collaboration, unitary decisions (i.e. the group decides on what

everyone will do), it can be mandated in a top-down manner, or individuals within the

organization can make independent choices whether to use the innovation. The

specific level of analysis (Damschroder & Hagedorn, 2011) must be determined prior to

assessment and provision of support system activities. Differences between levels can be

handled in two ways. First, either individual or organizationreferenced items can be

aggregated. Weiner et al (2008) suggest that the appropriate focus of aggregated items

should be contingent of the degree of task interdependence. An individual-level

aggregate approach is appropriate when the sum of individual capabilities is related to

organization performance then. This is particularly relevant for innovation-specific

capacities (e.g. the number of behavioral health providers with expertise in Motivational

Interviewing.) Motivation is often framed in terms of individual-level shared perceptions

of the innovation that influence adoption and implementation processes (Damschroder &

Hagedorn, 2011; Weiner et al., 2008). Secondly, difference between levels can be

handled through consensus when the organization-level is targeted. “In such

circumstances, what is important is not what I think I can do, or even what I think you

can do, but rather what we think we can do together,” (Weiner et al., 2008). In these

situations, the individual is asked to provide ratings on the organization. This means that

people are no longer thinking about the change in terms of themselves, but rather the

group/organization’s readiness. For example, measures of organizational culture are

consensus-based; individuals share their beliefs about how work is collectively done in

the organization (Glisson & James, 2002).

Work groups commonly arrive at shared beliefs, meaning, and narrative about change

(Rafferty et al., 2013; Rogers, 2003).

Although there is likely to be individual-level variation within groups, it is often

necessary to measure some readiness constructs through within-group agreement

(Glisson & James, 2002; Rafferty et al., 2013). If within-group agreement on constructs

is too low, then the attribute cannot reasonable be said to apply to an organizational level

(Klein, Conn, Smith, & Sorra, 2001; Weiner, 2009). In these cases, the constructs of

organization readiness do not exist as an emergent, “shared team property,” (Weiner,

2009). While it would possible to subdivide people in an organization into categories

based on how they vary on the three components of readiness and provide readiness

building support strategies tailored to the subgroups, this might be a labor intensive

process (and not within the readiness of the Support System).

This dissertation focuses on the evidence for building and sustaining

organizational readiness intentionally. In cases where readiness-building strategies are

not feasible, then other research traditions, specifically passive diffusion studies

(Gladwell, 2001; Greenhalgh et al., 2004; Rogers, 2003) may provide insights on how

ideas naturally spread through an environment through social means.

The Systemic Context of Readiness

Readiness is part of a larger implementation framework that exists in a broader

systemic context composed of economic, political, and social considerations (Aarons et

al., 2011; Damschroder et al., 2009; Wandersman et al., 2008). These factors include

regulatory policies, sociopolitical context climate, client/consumer advocacy, the existing

research literature, and available funding (Aarons et al., 2011; Wandersman et al., 2008).

While these influence the context in which an innovation will be implemented, they are

not directly controlled by organizations. Consequently, they are less easily changed

through deliberate actions. Disseminating information through the macro system may

require the managing of mass media channels (Armenakis et al., 1993; Powell et al.,

2012). Other strategies include working to change accreditation or membership

requirements, liabilities laws, and licensure standards (Powell et al., 2012).

Summary

Readiness as a construct needs to be flexible enough to lead to allow for multiple

measurement strategies, qualitative and quantitative, and help facilitate supportive

interventions from those wishing to implement an innovation. R = MC2 allows a program

developer or Support System provider to better delineate the specific factors that makes

an organization more or less ready for an innovation or innovation support. I now turn to

discussion of the three components of R= MC2 and their various factors or

subcomponents. This will provide a fuller description of how organizations can vary on

the component of readiness. I will present some preliminary strategies that the Support

System can use to help increase and strengthen each factor or subcomponent.

Chapter 2: Motivation for an Innovation

The first part of R= MC2 is the motivation to use a particular innovation

(Motivationi). Motivations are beliefs about the innovation and the innovation supports

that contribute to innovation use. Motivation is the cognitive and affective perceptions of

an innovation that attracts or pushes an organization toward use of an innovation. Many

authors refer to motivations as the characteristics of the innovations (Damschroder et al.,

2009; Flaspohler et al., 2008; Greenhalgh et al., 2004; Rogers, 2003). Simpson (2002)

alternately defines motivations as perceived needs and pressure for change. Hall and

Hord (2010) describe the “feelings, preoccupations, thoughts, and considerations give to a

particular issue or task.” The factors that influence motivation address how organizations

feel about for an innovation and how this influences the decision to use and continue

using an innovation (Rafferty et al., 2013). In this sense, the traditional concept of “buy

in” (e.g. Flaspohler et al., 2008) can be further subdivided into specific, measurable, and

ultimately actionable factors.

Factors that influence motivation involve not just the collective perceptions an

innovation. Rather, it accounts for whether and how these perceptions contribute to the

desire to use the innovation. They contribute to how a person or organization

conceptualizes the functional consequences of an innovation. These are collective beliefs

that contribute to an implementation effort (i.e. a shared resolve; Weiner, 2009).

Consequently, building motivation involves creating foster conditions that increase the

intent to change (Aarons & Sommerfeld, 2012; Miller & Rollnick, 2013; Rogers, 2003).

When the Support System builds Delivery System motivation, they build awareness that

the new innovation can enhance the organization (Greenhalgh et al., 2004), rather than

transfer tangible skills.

“Negative” motivations have been commonly framed as resistance (Hall & Hord,

2011; Ford et al., 2008; Weiner et al., 2008). When there are negative perceptions of the

innovation (i.e. when motivation is low) this can hinder support for a change (Rafferty et

al., 2013). However, resistance to change should not be considered the opposite of

readiness. Rather, it is a state of lower readiness, rather than a condition of nonreadiness.

Identifying areas of resistance provides an opportunity for positive organizational

development (Ford et al., 2008). This is consistent with applications of R= MC2 that

argue that any level of the components of readiness, even what these components are low,

provides information about how to support an implementation process.

There is a substantial research tradition in diffusion studies that look at how an

individual perceives and thinks about an innovation. In a review of diffusion studies,

Rogers (2003) identified that 49-87 percent of the variance in the adoption rate of an

innovation can be explained by five, innovation-specific variables; relative advantage,

compatibility, complexity, trialability, and observability. However, Rogers’s model has

not traditionally been used to facilitate implementation in a prospective way

(Damschroder & Hagedorn, 2011). In addition, the perceptions of support for an

innovation, the prioritization of the innovation, may be important component of

motivation.

The following factors that influence motivation (Table 2.1) are not

stable/permanent features of an innovation and do predict implementation in and of

themselves (Greenhalgh et al., 2004). Rather, these are beliefs about the innovation and

innovation support. These are beliefs that may be changed through deliberate Support

System activities.

Table 2.1: Ways to address factors that influence Motivation

Aspects of

Motivations

Possible ways to

Address

Authors

Relative

Advantage

Persuasion,

incentive

management

Armenakis, Harris, and Mossholder, 1993;

Rafferty, Jimmieson & Armenakis, 2013;

Gladwell, 2001; Weiner, 2009

Compatibility Translation System Chinman et al., 2004; Durlak & Dupre, 2008;

Fetterman and Wandersman, 2005; Rogers, 2003

Complexity Core components

vs. Adaptation

Fixsen et al., 2005; Meyers, Durlak &

Wandersman, 2012; Wandersman et al., 2008

Trialability Piloting Rapkin et al., 2012; Rogers, 2003

Observability Evaluation Beutler, 2001; Chinman et al., 2004; Rossi, Lipsey,

& Freeman, 2004

Priority Social influences Armenakis & Harris, 2009; Greenhalgh et al.,

2004; Ford, Ford & D’Amelio, 2008

Relative advantage is the degree to which a particular innovation is perceived as

being better than the innovations that it is being compared against. This is whether or not

the innovation is valued by the organization (Weiner, 2009). When relative advantage is

high, then the innovation is more likely to be adopted (Rogers, 2003).

There are many different ways in which the relative advantage of an innovation

can be construed. This includes economic profitability, initial and ongoing cost of the

innovation, decrease in subjective discomfort, social prestige, efficiency, and immediacy

of reward (Damschroder et al., 2009; Rogers, 2003). Relative advantage is not

necessarily an objective measure and whether or not an innovation has value over another

is determined through subjective means by the person or organization who wishes to

implement it. An innovation may be perceived as better, or alternately the current

practice can be perceived as intolerable. Other authors phrase this construct in terms of

valance; i.e., the change has value on a cost/benefit ratio for their job and role (Rafferty et

al., 2013) and that the overall outcomes will be beneficial (SAMHSA, 2010; Schoenwald

& Hoagwood, 2001). The relative advantage of an innovation can also be influenced by

the end-consumer demand for the innovation (Powell et al., 2012).

Articulating the relative advantage may be a key component of building tension

for a change (Greenhalgh et al., 2004). This involves fostering the idea that change is

needed and has benefits over the current conditions (Armenakis et al., 1993; Armenakis

& Harris, 2003). Persuasive communication that is rich, i.e., tailored to the organization,

increases the impact of these messages and can enhance understanding of the

innovation’s advantages (Armenakis et al., 1993). Additionally, preemptively developing

effective responses to common objections can help to address negative motivations (Ford

et al., 2008). Management of incentives and disincentives can greatly impact the

perceived, positive attributes of an innovation (Hall & Hord, 2011; Powell et al., 2012;

Rogers, 2003; Simpson, 2002), though do not necessarily lead to quality implementation

(Greenhalgh et al., 2004; Rogers, 2003).

Compatibility is the degree to which an innovation is subjectively perceived at

being consistent with the existing values, cultural norms, past experiences with similar

innovations, and needs of potential adopters (Rogers, 2003). This is also referred to as

the “innovation-system fit,” (Greenhalgh et al., 2004). If an innovation is perceived as

more compatible to an organization, it is more likely to be adopted (Greenhalgh et al.,

2004). This is because it represents less of a drastic change in behaviors (Rogers, 2003).

Within a strategic planning process like Getting to Outcome ® (GTO, Chinman et al.,

2004), compatibility is addressed as part of the innovation-selection process. This

narrows down the innovation that is likely to be “best” given the needs, goals, and fit

with the organization. Using indigenous knowledge systems to participate in the program

planning process can help to build compatibility with an innovation (Durlak & DuPre,

2008; Fetterman & Wandersman, 2005; Greenhalgh et al., 2004; Powell et al., 2012;

Rogers, 2003). Furthermore, if an organization can develop ownership over an

innovation, this increases the likelihood that it will be seen as relevant (Armenakis et al.,

1993; Simpson, 2002). The innovation can also be deliberately packaged and named in a

way that increased perceived compatibility (Damschroder et al., 2009; Rogers, 2003). If

these steps have not been addressed, then the innovation is not likely to be perceived as

compatibility.

Complexity is the degree to which an innovation is perceived as relatively

difficult to understand and use (Rogers, 2003). If something is complicated and hard to

use, then this can preemptively affect how willing an organization is to adopt it.

Implementation of innovations can be an extremely laborious process (Fixsen et al., 2005;

Meyers et al., 2012b). If a new innovation is easier to use, then people will be more

likely to adopt it. Consequently, as the scale of a change increases (i.e., the complexity),

responses to the innovation become more negative (Rafferty et al., 2013).

Misperception of complexity can be a significant barrier that prevents initial adoption.

Something may look easy at first glance, but the logistics of learning and routinizing how

to use it may be daunting (Hall & Hord, 2011). By not fully appreciating the depth and

requirements of quality implementation, this can contribute to increases in perceptions of

complexity, which decrease the likelihood of adoption (Rogers, 2003). This can decrease

motivation in the early phases of use (Klein & Knight, 2005).

Managing complexity requires effective processes to frame the innovation into

user friendly and easily understood components (Wandersman et al., 2008). The more

clearly the core components of the innovation are specified, the more readily that a

program can be implemented (Fixsen et al., 2005; Greenhalgh et al., 2004). Developing a

glossary of implementation terms can help to promote a common understanding of the

innovation (Hall & Hord, 2011; Powell et al., 2012). For example, the Affordable Care

Act (2010) is 974 pages long, far too in-depth to reasonably expect organizations and

providers to parse apart and change policies accordingly. In response, the Department of

Health and Human Services has set up a user-friend website explains the changes

(http://www.healthcare.gov/law/) and how it affects individuals, families, and employers.

This allows the complexity of the law to be distilled in a way that allows the end-user to

gain a smoother understanding of the nuances of the innovation.

Additionally, there is a significant need to analyze the tradeoff between adapting

an innovation to a setting (i.e. addressing both compatibility and complexity) and

maintaining fidelity to the original innovation (Damschroder et al., 2009; Durlak &

Dupre, 2008). Enhancing the compatibility and reducing complexity of an innovation

requires that the distinction between core and adaptable components be clearly separated

(Damschroder & Hagedorn, 2011). Choices that are made in the readiness building

process (i.e. adapting an innovation to reduce complexity) may adversely affect whether

or not the innovation does what it is supposed to do.

Trialability is the degree to which an innovation can be tested and experimented

with by the organization (Rogers, 2003). When the outcomes of the innovation are

uncertain, this allows people to experiment and see the results on a limited basis

(Greenhalgh et al., 2004). By facilitating active participation in the implementation of

the change, this increases opportunities to form more sophisticated perceptions of the

innovation (Armenakis et al., 1993; Fetterman & Wandersman, 2005). If people have the

opportunity to try to the innovation prior to formal implementation, this increases the

likelihood of use (Rogers, 2003). In this instance, the use of piloting individuals with an

organization may be beneficial in building initial motivation (Hall & Hord, 2011).

Observability is the degree to which the outcomes that results from the innovation are

visible to others (Rogers, 2003). If people can see what happens when the innovation is

used, this can increase the rate of adoption. This source of evaluation data provides

tangible feedback about the benefits of a particular innovation (Beutler, 2001;

Damschroder et al., 2009). Having sufficient evaluation capacity can increase how

observable the innovation is (Labin, Duffy, Meyers, Wandersman, & Lesesne, 2012).

Change facilitators in the Support System need to be realistic about the intended

outcomes on an innovation in order to cultivate accurate expectations (Ford et al., 2008).

Observability is particularly an issue in the case of preventative interventions (Rogers,

2003). This is because there is greater uncertainty about the relationship of the

innovation to the outcome (e.g. someone who doesn’t get cancer). In these cases, there is

less tangible incentive to adopt the innovation and consequently lower motivation to use.

Priority. In addition to the perceived attributes of an innovation, there can be a

perceived implementation climate that is specific to an innovation (Beidas et al., 2013;

Damschroder et al., 2009). These beliefs are the shared perceptions of the importance of

the innovation in the organization. This includes the degree to which an innovation is

expected, rewarded, and supported (Klein et al., 2001). Urgency, the amount of time that

is available before a change must take place, can also influence the prioritization of an

innovation (Armenakis et al., 1993; Damschroder et al., 2009). The urgency of an

innovation can be influenced by whether or not there is significant pressure to change

(Flaspohler et al., 2008; Lehman et al., 2002). While pressure can be driven internally

through social factors (especially when the innovation is home-grown; Damschroder et

al., 2009; Rogers, 2003), mandates are often used to influence/direct whether or not there

is an expectation that specific innovation should be implemented. As stated before,

mandates have a positive influence on increasing motivations, but do not have an impact

of the overall capacities of an organization (Beidas et al., 2013; Hall & Hord, 2011).

The motivational climate for a particular innovation can be affected by the influence of

key individuals, such as leadership, program champions, or administrative bodies

(Aarons & Sommerfeld, 2012; Atkins et al., 2008). This can be fostered by

“articulating a compelling and inspiring reason for innovation use, expressing their own

fallibility and need for team, members’ assistance and input, and communicating to team

members that they are essential, valued, and knowledgeable,” (Klein & Knight, 2005).

There can be specific social pressures to adopt a particular innovation (Armenakis et al.,

1993; Gladwell, 2001; Rogers, 2003). Organizational members look to each other for

cues regarding ongoing expectations about the innovation (Armenakis et al., 1993). This

is especially true for the “late-adopters,” who are not the initial people to begin use of an

innovation (Damschroder et al., 2009; Rogers, 2003). As the number of individuals in an

organization develop motivation increases, this can accelerate within-organization

motivation (Gladwell, 2002; Rogers, 2003).

Chapter 3: Innovation-Specific Capacities

The second component of R= MC2 is innovation-specific capacity.

Innovationspecific capacities are the human, technical, and fiscal conditions that are

necessary to successfully implement a particular innovation (Flaspohler et al., 2008). At

the organizational level, innovation-specific capacities refer to the operational realities

that allow or prevent innovation development and implementation. This is the technical

domain of the service system (Glisson, 2007). These are the knowledge, skills, abilities,

and technological equipment that are needed to put a specific innovation into place (Table

3.1). These are also referred to as process-specific capacities, as they relate directly to

innovation use (Livet, Courser, & Wandersman, 2008).

Table 3.1: Ways to address subcomponents of Innovation-Specific Capacity

Aspects of

InnovationSpecific

Capacity

Innovation

Specific KSA

Ways to Address Authors

Vary according to

complexity of

innovation -

identify, provide

EBSIS

EBSIS; Wandersman, Chien, & Katz, 2012

Program

Champion

Identifying and

utilizing connectors,

mavens (innovators),

and salesmen

Gladwell, 2002; Grant, 2013; Livet,

Courser, & Wandersman, 2008; Rogers,

2003

Implementation

climate (Supports)

Social influences,

leadership

Armenakis et al., 1993; Beidas et al., 2013;

Fetterman and Wandersman, 2005; Hall &

Hord, 2011; Rogers, 2003; Schoenwald and

Hoagwood, 2001

Interorganizational Formalized Powell et al., 2012

Relationships agreements;

coalitions building

Innovation-Specific Knowledge, Skills and Abilities. Each new policy, program,

or process has its own set of knowledge, skills, and abilities that are needed in order to

implement with quality and reach intended outcomes. The readiness building process for

every innovation will be somewhat different. Some innovations may be exceptionally

simple (and having few capacities to acquire), while others may be system-wide

transformations of complex care arrangements (e.g. Philadelphia Department of

Behavioral Health and Intellectual disAbility Services [sic], DBHIDS, 2011). There are

several steps involved in the process of building innovation-specific capacity. All

components of the innovation need to be thorough specified and standardized. The core

components are those that cannot be altered without substantial impact on the integrity of

the innovation. These are essential for achieving the intended outcomes of the innovation

(Fixsen et al., 2005; Meyers et al., 2012a). However, if organizations are able to adapt

certain elements of a program, it will be adopted more easily (Greenhalgh et al., 2004).

One possible strategy to manage this balance is to use an innovation configuration map

(IC Map; Hall & Hord, 2011). This can help to standardize the measurement of an

innovation across settings and track any adaptations.

Innovation champion. A champion is a charismatic individual who put his or her

organizational weight behind an innovation (Rogers, 2003). They tend to occupy a key

linking position in the organization (i.e. not so senior they are inaccessible, but not so

minor they cannot influence change), possess skills in understanding other’s motives and

aspirations, and have good interpersonal negotiating skills (Rogers, 2003). By modeling

positive emotional responses to a change, champions can influence how people feel about

the innovation process (Rafferty et al., 2013). More so than general leadership,

champions influence those around them through their expertise, experience,

representativeness, and credibility (Armenakis et al., 1993; Dougherty, 2009;

Damschroder et al., 2009; Greenhalgh et al., 2004). Champions are actively associated

with implementation (Atkins et al., 2008; Damschroder et al., 2009) and their presence is

related to both higher levels of use and higher quality of use (Livet et al., 2008).

Gladwell (2001) discussed several types of sub-types of individuals that influence how an

innovation can be adopted. Connectors are people who have many different types of

contacts and therefore can bridge many types of relationships, i.e. having many

“weak ties” (Grant, 2013; Greenhalgh et al., 2004). Mavens are those who have

considerable expertise and interest in a particular innovation. Their characteristics would

also be consistent the early adopters of innovators (Rogers, 2003). Implementation

leaders, those who are more ready to put an innovation into place, need to have a deeper

level of innovation expertise (Meyers et al., 2012b). Finally, salesmen possess a special

skill at persuading those about the relative advantages of innovation. Ideally,

champion(s) involve all three of these types of people in order to facilitate the use of the

innovation within the organization.

Implementation Climate Supports. Because there can be separate

implementation climate supports for separate innovations, it is included as an

innovationspecific capacity (Beidas et al., 2013; Damschroder et al., 2009). This is the

extent that the innovation is tangibly supported. Without strong, convincing, informed,

and demonstrable management support for implementation, employees are likely to

conclude that the innovation is a passing fad (Klein & Knight, 2005). These supports

include whether resources are available for a specific innovation, the number and

strategic placement of supporters in the organization (Greenhalgh et al., 2004), and

whether there is consistent leadership support for the innovation (Aarons et al., 2011;

Aarons & Sommerfeld, 2012; Klein et al., 2001; Weiner et al., 2009). A supportive

implementation climate can be a significant predictor of whether the innovation is

actually used (Klein &

Knight, 2005; Meyers et al., 2012).

Interorganizational Relationships. Flaspohler et al. (2008) define external

relationships as a general organizational capacity. I have included this with

innovationspecific capacities because the extent to which relationships between

organizations are needed will depend on the type and specific components of an

innovation. These can refer to relationships between the Support and Delivery System

and between different Delivery System organizations.

Certain types of innovations (e.g. Systems of Care) require more extensive

collaboration and cooperation (Powell et al., 2012), while others (e.g. adopting a new

paper stock) require only placing orders with a vendor. Innovations can also spread

between organizations (Aarons et al., 2011; Rogers, 2003). Communities of practice that

include all relevant stakeholders can help to share and dissemination information about

implementation challenges (Aarons et al., 2011; Fetterman & Wandersman, 2005) and

threats to ongoing readiness. Interorganizational relationships can be cultivated through

coalition building, developing resource sharing agreement, obtaining formal

commitments (such as memorandums of understanding; MOUs), and developing

partnerships with academic units (Powell et al., 2012).

Chapter 4: General Capacity

The third component of R= MC2 is general capacity. General capacities are the

skills, characteristics, and the overall functioning that are associated with the ability to

implement or improve any innovation (Flaspohler et al., 2008). General capacities

include the infrastructure, skills, abilities, context, environment, and processes in which

the innovation will be introduced (Greenhalgh et al., 2004). These capacities can be

applicable to many different types of innovations and across multiple situations on an

organizational level. In many cases, general capacities must be in place if the

innovationspecific capacities are to be implemented and sustained over the long term

(Fixsen et al., 2005; Greenhalgh et al., 2004; Livet et al., 2008). General capacities are

likely to be normally distributed across organizations. An organization that is low in

general capacity is likely to be distressed in some manner with dysfunctional elements

preventing the organization from operating in a positive and productive manner.

Building general capacities is a system-level intervention, meaning it can apply to

many different types of organizational tasks (Fixsen et al., 2005; Glisson, 2007). It can

be a lengthy and involved process, especially when addressing more stable features like

organizational culture. Some specific strategies linked to this and other capacities are

described below (Table 4.1).

Table 4.1: Ways to address subcomponents of General Capacity

Aspects of

General

Capacity

Ways to

Address

Authors

Culture

Availability,

Responsiveness,

Control (ARC)

Glisson, 2007; Glisson & Schoenwald, 2005;

Hemmelgarn et al., 2006

Climate

Reshape Vision

(Hall & Hord,

2011)

Lehman et al., 2002; Hall & Hord, 2011. Drzensky,

Egold, & Van Dick, 2012

Innovativeness Indirect

leadership

Support

Aarons & Sommerfeld, 2012; Atkins et al., 2008;

Fetterman & Wandersman, 2005; Greenhalgh et al.,

2004; Rafferty et al., 2013; Rogers, 2003

Resource

Utilization

Expanding

incoming

resources

Armstrong et al., 2006; Powell et al., 2012; Rogers,

2003; Simpson, 2002

Leadership Development McShane & Glinow, 2009; Becan, Knight, & Flynn,

2012; Beidas et al., 2013; Fixsen et al., 2005;

Rafferty et al., 2013.

Structure Revising

policies and

procedures,

developing new

teams

Glisson & Schoenwald, 2005; Lehman et al., 2002;

Rogers, 2003

Staff Capacity Attraction,

Screening,

hiring, attrition.

Flaspohler et al., 2008; McShane & Glinow, 2009;

Rafferty et al., 2013

Organizational Culture is the set of expectations about how things are done in an

organization (Glisson & James, 2002; Hemmelgarn, Glisson, & James, 2006). This is

how an organization or a system functions (Glisson, 2007). Very rigid cultures can be

inflexible when efforts made to alter their processes (Glisson, 2007). This can include an

organization’s identity, or is the extent to which central and enduring characteristics

distinguish it from other organizations (Drzensky et al., 2012). A vision statement

articulates the underlining philosophy that guides the type and quality of services. A

clear organizational vision provides a benchmark for all organizational operations to be

directed and ultimately compared toward (Hall & Hord, 2011). However, having a vision

does not ensure that organization change as terms and language may be devoid of any

operational meaning (Fixsen et al., 2005).

Part of being culturally competent is thoroughly assessing organizational culture

and climate (Fit, Gregory, Orden, Joran, Portnoy, Welsh, et al., 2012). Cultural

competency refers to the set of academic and interpersonal skills that allow for increased

understanding and appreciation of cultural differences within, among, and between

groups (Chinman et al., 2004). This is distinct from compatibility. Cultural Competency

as a general capacity is the set of skills and expectations that are applied toward any

innovation rather than the fit of a particular innovation as judged by the organization that

implements it.

Organizational Climate. Organizational climate refers to how employees

collectively perceive, appraise and feel about their current working environment (Glisson

& James, 2002; Lehman et al., 2002; Hall & Hord, 2006). Climate is an aggregate

construct that represents within-group agreement (or disagreement) about the work

environment (Glisson & James, 2002). This can include how individuals identify with an

organization, measures of job satisfaction, how engaged people are in their work, how

functional their interactions with coworkers are, and how stressful they perceive their

day-to-day tasks (Damschroder et al., 2009; Glisson, 2007). Climate is fostered when

collective perceptions about work environment emerge (Glisson, 2007). Climate is more

temporary and transient than culture, responding to various internal and external

influences over time (Gregory et al., 2012).

Climate is somewhat an analogue to factors identified in to Motivation. As

opposed to a specific innovation, these are the shared perceptions that people have toward

to the organization a whole. As such, climate as described by Glisson and James (2004)

and Glisson (2007) is a general capacity. A positive climate has been positively

associated with implementation (Beidas et al., 2013; Glisson, 2007) and service outcomes

(Aarons et al., 2011; Glisson, 2007). However, evidence conflicts about whether or not

identification with an organization facilitates implementation. When there is a strong

organizational identity, individuals may be less likely to adopt and innovation because the

strong emotional stake in the well-being of the organization may discourage risk taking

(Ford et al., 2008). Organizational identification is positively related to motivation when

the perceived benefits (i.e. relative advantage) are in favor to the organization (Drzensky

et al., 2012). Some strategies to foster increased identification with an organization and

ownership over an organizational change include a developing clear, consistently-

articulated narrative of the organization’s history, successes, and capabilities (Armenakis

et al., 1993).

Perceived Stress is also is a crucial factor for organizations wishing to implement

change (Lehman et al., 2002) with large amounts of negative stress linked to poorer job

satisfaction (Glisson, 2007). Typically stressors such as role conflict, role overload,

ambiguity over tasks and responsibilities, and emotional exhaustion can interfere with

daily program operations (Glisson, 2007; Hall & Hord, 2011; White, 2008). When a

certain practice or organizational condition is intolerable, a tension for change can

emerge (Damschroder et al., 2009; Greenhalgh et al., 2009). When there is this tension,

the discrepancy between current conditions and the possible benefits of change can

enhance the relative (dis)advantage of practice as usual (Armenakis et al., 1993).

However, as stress is not necessarily linked to readiness for a specific innovation (Hall &

Hord, 2011), and it is included as a general capacity.

Organizational Innovativeness. This is how generally receptive an organization

is toward change, i.e., whether the organization tries new things and fosters a learning

environment (Fetterman & Wandersman, 2005; Hall & Hord, 2006; Rogers, 2003).

Innovativeness is separate from motivation for a specific innovation, as it can be applied

to many different types of innovations. Some organizations may foster an environment

that is open to new innovations while others may be more inflexible and immobile

(Damschroder et al., 2009; Greenhalgh et al., 2004; Glisson, 2007; Hall & Hord, 2011;

Klein et al., 2001; Rafferty et al., 2013; Rogers, 2003; Rogers, 2003). Innovativeness sets

a tone in how an organization reacts to a change by promoting a strong futureorientated

perspective (Rafferty et al., 2013). When an organization’s culture is receptive to change,

this is positively related to the perceived benefits that can results from a change process

(Drzensky et al., 2012). This may be influenced by past experiences with the change

process, which in some instances may have been negative (Weiner, 2009).

Organizations that have more connections with external organizations are more likely to

implement new innovations quickly (Damschroder et al., 2008; Gladwell, 2001; Rogers,

2003). The extent that organizations are externally connected to other organization (i.e.

openness) is positively linked to how innovative they are (Rogers, 2003). Glisson (2007)

calls these proficient organizations. Openness is particularly relevant when facilitating

the initial dissemination of innovations.

There is little research on how to directly increase overall organizational

innovativeness. Indirectly, establishing innovation workgroups within an organization is

associated with positive implementation (Greenhalgh et al., 2004). Innovativeness can be

influenced by key individuals or administrative bodies facilitating a learning environment

that encourages experimentation and risk taking that is unconstrained by a fear of failure,

tolerates mistakes, and fosters teamwork (Aarons & Sommerfeld, 2012; Aarons et al.,

2011; Atkins et al., 2008; Klein & Knight, 2005). For example, Many public health

systems are very defensive and passive toward innovation due to the need to insulate

themselves from criticism, administrative sanctions, and litigation (Glisson & James,

2002).

Resource Utilization. Resources are existing structures, funding, programs‚ and

other activities that are potentially available for programming (Chinman et al., 2004).

Resources are not the general capacities. As a general capacity, resource utilization is

how resources are acquired and used. Large organizations tend to have more “slack,” or

discretionary/uncommitted resources that can be devoted to innovations (Klein et al.,

2001; Rogers, 2003; Lehman et al., 2002). Therefore, how this slack is dedicated is a

general capacity. There can also be physical resources such as adequate office space,

equipment, and technological capacity (e.g. computer access and integrated clinical data

collection systems) that can be dedicated toward different types of innovations (Simpson,

2002; White, 2008.) This can also include the concept of time, i.e. the amount of work

hours available or allotted for an organizational change process (Greenhalgh et al., 2004).

Because larger organizations tend to have greater slack resources, they tend to be more

innovative (Rogers, 2003). Furthermore, the experience and skills that an organization

has at seeking alternative and additional streams of funding is a general capacity (Powell

et al., 2012). Examples of this strategy would be applying for grants and/or expanding

the number of insurance providers that an organization works (Powell et al., 2012) or

developing a strategic financing plan (Armstrong et al., 2006).

Leadership. Quality leadership is motivational, considerate, engaging to staff,

and promotes a climate for change (Aarons & Sommerfeld, 2012). Leadership can apply

to more than one particular innovation (Fixsen et al., 2005; Flaspohler et al. 2008.) As it

relates to an innovation, leaders need to be able to develop, communicate, model, and

build commitment toward a strategic vision (McShane & Glinow, 2009). High quality

leadership is associated with better staff attitudes toward adopting an innovation (Beidas

et al., 2013; Rafferty et al., 2013) increased risk tolerance and positive self-concept

(Simpson, 2002) and increased likelihood of implementation (Becan, Knight, & Flynn,

2012). However, tenure increases (specifically among chief executive officers (CEOs)),

leaders tend become less likely to introduce fundamental changes into organizations

(Rafferty et al., 2013).

Organizational Structure include such factors as organizational architecture, size,

specialization, power structures, staff autonomy, staff cohesiveness, communication

pathways, and internal decision-making processes that can impact how well an

organization functions on a day-to-day basis (Damschroder et al., 2009; Flaspohler et al.,

2008; Greenhalgh et al., 2004; Lehman et al., 2002; McShane & Von Glinow, 2009).

Typical structural stressors for organizations may include work overload, incivility, low

task control, role conflict, ambiguity over tasks and responsibilities, and negative

attitudes to work (Glisson & James, 2002; McShane & Von Glinow, 2009; White, 2009).

Methods to address organizational structure issues include revising professional roles and

job characteristics (Glisson & Schoenwald, 2005) creating new teams and services sites

(Powell et al., 2012), and developing new administrative policies and procedures

(Donahue, Allen, Romero, Hill, Vasaeli, et al., 2009).

A structural balance between openness and control must be navigated during

implementation processes. Different structural elements may be more important at

different points in implementation. Structural flexibility and decentralization are

positively associated with the positive motivation toward an innovation (Rafferty et al.,

2013) but not necessarily the successful adoption of an innovation (Rogers, 2003).

Organizations that are more centralized and have control consolidated in a few

individuals tend to show less innovativeness (Rogers, 2003). Rigidity or formalization,

the degree to which an organization emphasizes following rules and procedures, is also

negatively linked to innovativeness (Rogers, 2003). However, formalization can

facilitate implementation of an innovation (Rogers, 2003). Therefore, there is a tradeoff

between developing the necessary, formalized organizational structure to implement an

innovation and having the general capacity that is receptive to change.

Staff capacities are the general skills, education, and expertise that the staff

possesses (Flaspohler et al., 2008; Rogers, 2003). Certain general staff attributes include

perceived opportunities for growth and professional development, feelings of efficacy in

ability to carry out job duties, the mutual influence that staff have over each other, and

staff adaptability to changing work demands (Simpson, 2002). General staff capacity can

be built through attracting quality candidates, screening and hiring appropriate

candidates, and retention quality employees. The individuals who fit best within the

organization are more likely to be retained and contribute to the organizational climate

and culture (Rafferty et al., 2013). Retention is specifically linked positively to the

organizational climate and job satisfaction (McShane & Van Glinow, 2009).

Chapter 5: The Dynamics of Readiness

Readiness is typically assessed in the pre-adoption phase of implementation in

order to identity barriers to putting the innovation into place (e.g. Chinman et al., 2004;

Damschroder et al., 2009; SAMHSA 2010 SAMHSA, 2011; Hawkins & Catalano, 2002).

When an organization meets certain criteria, an innovation may be introduced or receive

support services (Hawkins & Catalano, 2002; Oetting et al., 1995; SAMHSA, 2010).

Once an organization is “ready”, then the implementation process can begin and is likely

to be successful. For organizations that fall below a readiness threshold, it is assumed

that there are likely to be many barriers that will interfere with successful change effort.

As a consequence, organizations with the largest need for supportive processes are often

labeled as not being ready for the innovation, or alternately, will be non-responsive to

supporting strategies like technical assistance (TA). Furthermore, there is an implicit

assumption that readiness will be a static condition over the lifespan of the innovation

(e.g. SAMHSA, 2010; Simpson, 2009) and will not need to be addressed after

implementation.

However, change is not an event; it is process (Hall & Hord, 2011). It is not

sufficient just to consider readiness as a precursor to change and then fail to monitor its

properties over the course of implementation. For example, key staff may have leave

through turnover, a better, more advanced innovation is introduced, or other

responsibilities may compete with implementation of the innovation. Neglecting these

variables can have negative consequences on quality of the implementation (Meyers et

al., 2012).

Readiness is ongoing, dynamic, and flexible construct. All factors and

subcomponents may change over time. Being able to adopt and implement an innovation

is an interaction between perceptions of the innovation, the organizations, and the context

(Greenhalgh et al., 2004). Any study or assessment must recognize that all three

variables are subject to fluctuations over the lifespan of the innovation. Readiness can be

assessed prior to implementation, monitored during implemented, measured as an

outcome and condition for the sustainability of an innovation, and targeted through

tactical CQI changes during the course of implementation (Damschroder & Hagedorn,

2011). Capacity and motivation must be monitored for intended or unintended changes

during implementation as these may either positively or negatively influence the impact

of the innovation (Damschroder et al., 2009; Meyers et al., 2012b; Rapkin et al., 2012;

Stirman et al., 2012). Using one time, cross-sectional methods to study components like

motivations (Rogers, 2003) only captures a partial picture.

In the early phases of implementation, putting an innovation into place often

results in poorer team performance and/or organizational performance (Klein & Knight,

2005). This “hassle” factor has implications for the readiness of organizations during the

early lifespan of an innovation. Given the stressors of implementation, the factors that

influence motivation may actually decline during the beginning stages of use. At these

beginning stages, addressing concerns that people have can be a way to help disarm early

negative motivation toward an innovation (Ford et al., 2008; Hall & Hord, 2011; Rogers,

2003).

The processes that underlie when an innovation is sustained versus discontinued

are particularly under-researched (Aarons et al., 2011; Greenhalgh et al., 2004). There

can be a variety of reasons by an innovation may be discontinued, including finding a

better innovation (Hall & Hord, 2011) or changing needs of the population (Scaccia,

Castellow, & Wandersman, in press). While addressing sustainability is often seen in

terms of capacity, the ongoing perceptions of the innovation’s usefulness also need to be

monitored (Chinman et al., 2004; Hall & Hord, 2011). For example, an organization may

have high general and innovation-specific capacity, but the front-line staff and

administrators responsible for implementation may lack sufficient motivation to continue

to implement the innovation because there are other organizational priorities.

Maintaining motivation is something that must be continually cultivated over the

implementation process if an innovation is likely to have any sustainability (Hall & Hord,

2011; Meyers et al., 2012; Stirman et al., 2012).

At this time our knowledge about what constitutes each of these constructs,

including their relative weights and how they are linked to a specific innovation, is still in

its infancy. While I have attempted to maintain boundaries between each of these

components, these constructs in implementation frameworks can often be indistinct and

overlap (Damschroder et al., 2009; Flaspohler et al., 2008). Additionally, the literature on

the interactions between components of readiness is sparse. The precise interconstruct

dynamics may be dependent, for example, on the expansiveness/scope (Flaspohler et al.,

2008) and complexity (Rogers, 2003) of the innovation, so the interrelationships are

difficult to predict and generalize.

Currently, the instruments that assess readiness are similarly nebulous, with many

researchers developing measures without actually defining readiness as a construct

(Drzensky et al., 2012). Weiner, Amick, and Lee (2008) extensively reviewed the change

measurement literature and identified 43 instruments for assessing readiness. In these,

they noted substantial deficits in terms of both validity and reliability. Only seven

instruments had undergone any systematic psychometric testing and they recommended

that caution be used when applying these instruments to other settings and innovations.

Specifically, they propose the effective instruments would, a) focus on a specific

innovation, b) use group referenced items, c) capture change commitment, d) be flexible

enough to specify other innovations. For example, the widely used Organizational

Readiness for Change (ORC, Lehman et al., 2002) scale does not identify a specific

innovation. While it can help to identify general capacities that are present in the

organizational environment, it neglects both innovation-specific capacities and

motivation related to an innovation. Within the ORC, motivations are defined in terms of

needs, training need, and pressures for change (Lehman et al., 2002), which would be

considered part of a larger program planning framework, not as a specific

implementation-facilitating framework. Because of these measurement deficits, it is

difficult to make generalized dimensional statements about how much “more” readiness

an organization might have and how much is necessary for implementation quality.

Future, more nuanced measurement models will be able to help better distinguish

between the levels of readiness profiles and can be used to inform evidence-based

support strategies.

Despite the deficiencies in measurement, there is still a need to enhance the

components of readiness. Having a specific, deliberate program to increase the

innovation use is associated with better implementation (Greenhalgh et al., 2004; Klein &

Knight, 2005). Targeted, specific strategies can be used to build readiness through the

Evidence-Based System for Innovation Support (EBSIS; figure 5.1). EBSIS strategies

include developing tools, delivering training, providing technical assistance (TA), and

developing quality assurance/quality improvement (QA/QI) systems (Wandersman et al.,

2012). Each of these components has its own literature and evidence-base (Wandersman

et al., 2012). Tools are resources that are designed to organize, summarize, or

communicate knowledge. Training is a planned, instructional activity intended to

facilitate acquisition of knowledge, skills, and attitudes in order to enhance learner

performance (Furjanic & Trotman, 2000; Wandersman et al., 2012).

Figure 5.1: Building Readiness through EBSIS

Motivation

General

Capacity

Innovation-

Specific

Capacity

Delivery

System

Outcomes

forms of EBSIS

include:

Tools

Training

Technical Assistance

Quality Assurance/

Quality Improvement

Motivation

General

Capacity

Innovation-

Specific

Capacity

Changes in

Delivery

System

EBSIS

Support

System

However, training in and of itself is generally insufficient to produce intentional

change within an organization (Wandersman et al., 2012). Technical Assistance is an

individualized support system activity and hands-on approach to capacity-building in

organizations and communities, often conducted after training (Chinman et al., 2004;

Durlak & Dupre, 2008; Wandersman et al., 2012).

The goals of technical assistance are to maintain providers’ motivation and

commitment, improve their skill levels where needed, and support local problem

solving efforts. Depending on the situation, technical assistance may include

some combination of re-training of initial providers, training of new staff, and

providing emotional support. (Durlak & Dupre, 2008)

Quality Assurance/Quality Improvement strategies involve the use of tools and

logic to assess (QA) or enhance (QI) quality performance. The capacity to evaluate an

innovation is positively linked to implementation quality (Flaspohler et al., 2008;

Greenhalgh et al., 2004; Labin et al., 2012). There are a variety of strategies to build

evaluation capacity, including developing QA systems and tools, having auditing policies,

using reminders, and providing supervision (Powell et al., 2012).

Generally, having positive relationships between the Delivery and Support System

will be linked to more positive implementation (Dougherty, 2009; Greenhalgh et al.,

2004; Wandersman et al., 2012.) Change agents, the person(s) facilitating the

implementation of an innovation, can be a powerful conduit in purposeful dissemination

(Armenakis et al., 1993; Glisson & Schoenwald, 2005; Rogers, 2003). Change agents

(i.e., the Support System providers of EBSIS) can influence perceptions, attitudes, and

decisions at multiple levels by providing technical information, describing characteristics

of the innovation, and facilitating linkages between multiple groups (Gladwell, 2002;

Glisson & Schoenwald, 2005; Rogers, 2003). As described by Rogers (2003), this is

done by 1) developing the need for a change, 2) establishing an information-exchange

relationship, 3) diagnosing the problem (Simpson, 2002), 4) creating an intent to change

(i.e. building motivation for the innovation), 5) translating this intent into action, 6)

stabilization implementation and prevent discontinuation (Hall & Hord, 2011), and 7)

termination (Chinman et al., 2005; Dougherty, 2009). Change agents have better

effectiveness when they are homophilous (i.e. they share attributes with and are similar to

the potential innovation users), can develop good relationships, can assess community

needs, and allow potential users to make independent decisions about using of the

innovation (Greenhalgh et al., 2004; Rogers, 2003). Provided that support strategies are

delivered with quality, this will lead to enhanced levels of the targeted components, and

consequently improved implementation (Klein, Conn, & Sorra, 2001; Powell et al., 2011;

Rafferty el al., 2013; Rogers, 2003; Wandersman et al., 2012).

While EBSIS was initially developed for building the capacity for a specific

innovation (Wandersman et al., 2012), it can be extended to each component of readiness:

motivation, innovation-specific capacities, and general capacities. The EBSIS process

can pinpoint specific areas in need of improvement and highlights areas of relative

strength that can be used as leverage for improving readiness over time. It can help guide

thinking on which dimensions are particularly strong and where the Support

System needs to intervene.

While it is likely necessary that certain component of readiness must be in place

in order to ensure the implementation happens with quality (Greenhalgh et al., 2004;

Glisson, 2007; Meyers et al., 2012), there is no consensus about what these necessary

components might be. There may be variation in the R=MC2 conditions that are needed

for different types of innovations. Determining the relationship of these components to

implementation outcomes would allow for more specialized Support System strategies as

it would provide information about the components that are most influential. Further

research and synthesis is also needed to determine what types of tailored strategies are

best practice for the specific readiness constructs (Armenakis et al., 1993; Glisson &

Schoenwald, 2005; Glisson, 2007; Powell et al., 2012; Wandersman et al., 2012).

Chapter 6: Research Questions, Methods and Data Analysis Plan

In order to determine how to best build the components of R=MC2 using targeted

Support System strategies, the following questions will be addressed in this dissertation:

1. How are the subcomponents of readiness defined across content areas?

a. What, if any, evidence supports the existence of subcomponents of

readiness that were not mentioned/addressed in the introduction?

2. What are the best methods to build the factors that influence motivation?

3. What are the best methods to build the innovation-specific subcomponents of

readiness?

4. What are the best methods to build the general capacity subcomponents of

readiness?

5. Do tailored support system activities (i.e., those that address specific components

of readiness) lead to better innovation outcomes than those that do not? I

hypothesize that tailored Support System activities that target specific

components of readiness of the Delivery System will show better innovation

outcomes. 6. When is organizational readiness too low to be responsive to tailored

Support

System activities?

I hypothesize that there is no evidence that organizations will be non-responsive

to Support Systems activities that are tailored to readiness. Figure 6.1 illustrate the

general causal chain that this dissertation plans to investigate.

Figure 6.1 Supporting Readiness for an Innovation through EBSIS

Methods

A broad-based research synthesis will be employed to assess the evidence that

supports addressing and building readiness as a means to enhance the quality of

innovation outcomes. Research synthesis involves techniques that include meta-analysis,

but have a broader scope of inclusion criteria (Labin et al., 2012) that includes qualitative

findings. Research synthesis is governed by six steps and subsequent decision rules

including, 1) defining the research questions, 2) collecting information sources, 3)

selecting information sources bases on inclusion/exclusion criteria, 4) extracting and

coding data (which includes assessing potential risks of bias), 5) analyzing the data, and

6) presenting the findings. (Labin et al., 2012; Noyles et al., 2011))

Two broad contents areas will be incorporated into this synthesis: behavioral

health (e.g. Lehman et al., 2002) and health care/medicine (e.g. Weiner, 2009; Gawande,

Motivation

General

Capacity

Innovation-

Specific

Capacity

Delivery

System

Innovation

Outcomes

forms of EBSIS

include:

Tools

Training

Technical Assistance

Quality Assurance/

Quality Improvement

Motivation

General

Capacity

Innovation-

Specific

Capacity

Changes in

Delivery

System

EBSIS

Support

System

2013). These literatures have a tradition of studying how new innovations are

implemented. The following search engines will be used: PsycInfo and PsychArticles

(Behavioral Health), Medline and CINAHL (Health Care). There are no time limits

placed upon the search parameters as there is no clear rationale for excluded literature

based on publication date.

For the first set of research questions, the search terms are each factor or

subcomponent of readiness AND implementation AND each EBSIS strategy (tools OR

training OR technical assistance OR quality assurance OR quality improvement). A

secondary search that uses similar search terms and synonyms may be necessary because

different research traditions may use different language to define similar constructs

(Greenhalgh et al., 2004). A literature search will not be conducted for the subcomponent

of innovation-specific knowledge, skills, and abilities. There is likely too high a degree

of specialization for specific innovations to conduct any meaningful or realistic synthesis.

However, innovation-specific capacities will be coded and analyzed if they are part of a

study and implementation effort.

One major threat in conducting a synthesis is publication bias, or the tendency for

positive results to be submitted and accepted for publication over null results (Rosenthal

& DiMatteo, 2001; Shadish, Cook, & Campbell, 2002). Furthermore, many technical or

evaluation reports may not be submitted to journals, though self-published in other

formats. In order to address this “grey literature” (Hammerstørm et al, 2010) a search of

the Science.gov and PAIS International database will be used to collect published

material in the social sciences that is not otherwise indexed. As a synthesis of all

available sources is not feasible, a maximum of three articles per subcomponent, selected

by relevance, will be included. A diagram of the selection process can be seen in Figure

6.2.

Figure 6.2. Diagram of Study Selection and Exclusion Process

Out of these total numbers of articles, the titles and abstracts will be reviewed for

inclusion and exclusion criteria which include relevance to the initial research questions.

Only articles that appear to report empirical results will be included in the synthesis;

theory or conceptualizations-style papers that do not include any reported evidence will

not be utilized. Duplicate results will also be removed.

Coding

After inclusion and exclusion decision rules have been applied, the remaining

articles will be reviewed and coded. Descriptive qualitative and quantitative information

will be collected via this dissertation’s coding form (see Appendix B). Quantitative data

will be collected based on indicators in the coding forms that correspond to current

knowledge about concepts discussed in the introduction. A guidebook for the coding

Initial Search

Medical/Behavioral Health

(each subcomponent of Readiness)

Title/Abstract

Review

Articles selected

for coding

Excluded

•Theory-based

•Concept-papers

•No apparent

information about

innovation outcomes

form will also be developed (see Appendix C). An initial review of three articles per

subcomponent was conducted to assess initial uniformity around their definitions

(Appendix D).

Background information will be collected for all articles. This includes authors,

the content area, the innovation that is specified, sample size of the study, and the project

timeline. The unit of analysis for coding will be an information source; in cases where

two or more information sources refer to the same project, the two sources will be

counted separately to the extent that each information source provides new and unique

information.

Coding form pilot. A pilot of the coding forms will be conducted with 17 cases by

a coding team consisting of this author and at least one additional research associate

(Gwet, 2010). This will take place after all cases have been initially identified, and will

be drawn from that sample. Input from the coding team will be used to resolve

inconsistencies or other limitations associated with the use of the coding forms and to

inform additions to the coding guidebook. A reliability assessment will be conducted

using Cohen’s Kappa statistic and percent agreement analyses (Hallgren, 2012). It is

expected that there will be iterative refinements made to the coding tools (the form and

the guidebook) until reliability reaches an acceptable level (Kappa > .70). The final

version of the coding form will be put into an electronic format to ease data collection

and analysis.

Data Analysis Plan

To answer question 1) How are the subcomponents of readiness defined across

content areas?, a uniform definition will be refined for each subcomponent of readiness

and used in the coding guidebook to determine what subcomponents are being examined

in each study. Frequency and percentages will be computed for the occurrence with

which subcomponents appear in the literature as whole and for each content area. This

answer to this question will help to ascertain consensus and recent scholarship around the

components of readiness.

To answer questions 2-4), information will be gathered about the types of

techniques used to address and build specific subcomponents of readiness. Frequency

counts and percentages will be computed to determine which support strategies occur

most often when addressing the sub-components of readiness.

Evidence supporting the use of support strategies for each subcomponent of

readiness will be qualitatively synthesized and assigned into a category using a version

of Greenhalgh et al.’s (2004) modified World Health Organization Health Evidence

Network criteria (WHO-HEN) (Øvertveit, 2003) (see table 6.1). This has been adapted to

more directly include information about the evaluation design of the study. Coding

information for the evaluation design can be found in Appendix C.

Table 6.1 Levels of evidence

Level of

evidence

Description Evaluation

Requirement

Strong direct

evidence:

Consistent findings in two or more empirical studies

of appropriate design and high scientific quality

undertaken in health service organizations (include

both behavioral health and health care)

Requires

participant

randomization

Moderate direct

evidence

Consistent findings in two or more empirical studies

of less appropriate design and/or of acceptable

scientific quality undertaken in health service

organizations

Requires

comparison

group

Limited Only one study of appropriate design and acceptable Requires

evidence quality available, or inconsistent findings in several

studies.

quantitative

measurement

Minimal

evidence

At least one study of minimal quality available. This

includes practitioner self-reports, single organization

case-studies, and other qualitative reports.

No evidence: No relevant study found

The categories in this table will provided a measure of the qualitative strength of

the findings. Additionally, when effect sizes are reported or can be computed based on

information reported in the article, and if the evaluation model is of high quality (i.e.,

strong evidence), meta-analysis will be used to synthesize effectiveness findings about

the subcomponents of readiness. Meta-analysis allows for combining of descriptive

statistics from several studies and the quantitative examination of inconsistencies in the

field (Rosenthal & DeMatteo, 2001). Cohen’s d will be computed, by which changes in

means are divided by the pooled standard deviation of the outcome measure (Rosenthal

& DeMatteo, 2001). Prior to coding, it is uncertain whether any studies will meet the

reporting threshold needed for meta-analysis.

Within-study bias will be assessed by examining evaluation, theory, and

implementation failure (Wandersman, 2009). First, threats to internal validity will be

assessed using a modified version of the Cochrane Collaboration Risk of Bias scale

(Higgins et al., 2011). A second reason why a particular innovation does not have the

intended outcomes is that it may not have been appropriate innovation for the underlying

need. This was discussed in chapter one and is also known as theory failure

(Wandersman, 2009). The coding form will capture information related to the rationale

for the innovation. This is particularly important for innovations that are mandated to be

implemented. The quality with which the support strategy is implemented may also

affect how the subcomponents of readiness may change. Implementation quality will be

measured quantatively by component six of the Quality Implementation Tool (QIT);

Evaluating the Effectiveness of Implementation (Meyers et al., 2012). This includes an

evaluation of fidelity, dosage, quality of delivery, differentiation, reach, and adaption (see

Appendix C for definitions).

To answer question 5), Do tailored support system activities (i.e., those that

address specific components of readiness) lead to better innovation outcomes than those

that do not?, evidence supporting the use of tailored support strategies over non-tailored

strategies will be quantitatively analyzed using logistic regression.

To answer question 6), When is organizational readiness too low to be responsive

to tailored Support System activities?, the percentage of articles that explicitly report no

change in readiness will be computed. Within these articles, the reasons for support

failure will be recorded in terms of the specific readiness subcomponents that were

nonresponsive to support strategies. This will be qualitatively compared against other

possible reasons for failure, including innovation, implementation, and evaluation failure

(Wandersman, 2009).

Across all articles, qualitative and illustrative passages that are especially

evocative will be gathered.

Reporting the Evidence:

The data that are collected for this dissertation will be presented in a format that is

organized according to the plan for data analysis (described above). Quantitative findings

about each subcomponents of readiness will be further illuminated using qualitative

descriptions that are derived from information sources. Should articles be of acceptable

empirical quality (i.e., strong evidence) and if they report statistics sufficient for meta-

analyses, Cohen’s d will be reported.

There will be likely great variation in the quality of the reported evidence about

how the subcomponents of readiness can be built using support system strategies. By

identifying the state of the evidence for these subcomponents, this dissertation hopes to

synthesize the state of the evidence for the best strategies to address organizational

deficits in the components of readiness. Knowing the effectiveness of various support

strategies, with provide increased guidance for the Support System. This also can help to

address an additional reason why innovations may fail to meet their outcomes, support

failure, i.e. inappropriate or inadequate innovation assistance (Wandersman, 2009). This

information will allow a more refined, empirically-based model of innovation readiness

that can be used by program planners and evaluators to better bring innovations into

organizations.

Chapter 7: Results

A total of 4378 potentially relevant articles were identified during the initial

screening. From these, 297 were selected for full text review. These 297 were selected

because the article abstracts appeared to indicate that a support strategy was provided

with the intent of addressing one of the factors or subcomponents of readiness. All

statistics were computed in the R statistical package (2014).

Reliability Process.

Seventeen articles were picked to be initially coded. The number of articles

corresponds to the N needed in order to have adequate confidence in the kappa coefficient

between two coders (κ , Gwet, 2010). These initial articles met two criteria. First, they

were part of the initial title/abstract screening. Second, they were available through the

University of South Carolina journal accounts. These articles were picked to represent a

diverse range of readiness constructs. There were no other preconditions attached to the

initial review.

Coding was completed by the author and one additional coder. The additional

coder was conceptually familiar with the items being coded, having worked with the

author on several projects related to organizational readiness since 2012. The additional

coder did not know the categorization of the articles prior to coding. However, there was

still a one-hour training session to review the coding form and glossary. Coders generally

could code the presence of an item even if the authors did not use the same precise

terminology as the coding manual.

After the first wave of coding, only 11 articles were retained. The κ coefficient on

the Is this article codable? item was 0.86. Therefore, an additional nine articles (for a

total of 26) were selected to be coded in order to reach an N of 17 (with the assumption

that articles would be retained at the same rate as in the initial coding wave; ~70%).

Reasons why articles were not coded included having a fictitious example, did not have

any data, and describe a process of support (not provision of support).

Kappa coefficients were computed for each individual item, following procedures

detailed by Viera and Garrett (2005). This process helped to identified particular items

that were problematic for further review. Items with a κ of ≤0.40 (moderate agreement,

Landis & Koch, 1977; Viera & Garrett, 2005) were examined more closely. All

discrepancies were reviewed by both coders through a discussion that involved consulting

the original articles. A consensus process was used to resolve discrepancies. Where

discrepancies were conceptual, particular changes were made to coding form and glossary

(Appendices A and B.) These changes and rationale are outlined below:

Background Information. One coder made the decision to code articles that

dealt with public health as other. An alternative option would have been to code as

health care. To make a distinction, the category of public health was added. No

additional other items were coded in the remainder of the process. The technology item

was removed due to lack of distinction and possible redundancy with intervention and

process. A qualitative item, reason why article is not codable, was added.

Subcomponents of Readiness. Coding for compatibility was refined, as one

coder conflated “legitimacy” and “commitment” with compatibility. These two items

were coded as priority.

Readiness outcomes. Two items changed format. The computation cell and type

in qualitative outcomes collected non-categorical data. The other outcomes category was

deemed redundant, and was removed. These items were not included in the overall κ

calculation.

Innovation outcomes. Like above, two items changed format. The computation

cell and type in qualitative outcomes collected non-categorical data. They were removed

from the overall κ calculation.

Possible Sources of Within-Study Bias. We discussed several items under

implementation quality. We clarified the distinction between fidelity of the innovation

(the innovation is put into place as specified) versus fidelity of the support strategy (the

support strategy is provided as specified). The fidelity item refers to fidelity of the

support strategy. The item quality implementation was dropped due to its qualitative

nature and conceptual overlap with participant responsiveness.

Following this process, κ was computed at 0.76. As κ may understate interrater

agreement and result in a misclassification to a lower rating when there is little or no

variability in ratings (Labin et al., 2012), percent agreement was calculated as an

additional measure of interrater agreement. Percent agreement was calculated at 89.64.

The final codebook and guide can be found in Appendices B and C.

One hundred seventy-three articles met inclusion criteria and were used in the full

analysis. The reference list for the articles in the synthesis can be found in Appendix E.

Figure 7.1 is a modified PRISMA diagram (Moher, Liberati, Tetzlaff, Altman, The

PRISMA Group, 2009) detailing the selection process, including reasons why particular

article were excluded from the primary analyses. There were a number of reasons why

articles were excluded from the analysis: 1) the article described how a support system

strategy should be provided, but did not actually provided the support in a setting (N =

43), 2) the article described how a factor or subcomponent was linked to innovation

outcomes, but not how these factors or subcomponent were changed by the provision of

support system activities (N = 42), 3) the article described a model of the factor or

subcomponent, but not a real-life application of the model (N = 24), 4) the article was a

review paper that discussed how a factor or subcomponent applied to several settings, but

not how the factor or subcomponent was changed, and, 5) the example provided in the

article was fictitious or hypothetical (N = 2).

Figure 7.1. Modified PRISMA Diagram of Study Selection and Exclusion Process

Descriptive Characteristics of Included Studies.

Content Area. Health care constituted the largest number of articles (N = 112;

65%), followed by public health (N = 29; 18%), behavioral health (N = 14; 8%), and

education (N = 11; 6%). Four articles were from the business literature (2%), and three

concerned the U.S. Federal Government (1.7%). Articles were published between 1972

and 2013, with 94% being published after the year 2000, although no date restrictions

were established on the search parameters a priori.

Types of Innovations. Twenty-six (15%) of the articles dealt with the

introduction of a policy, 12 (7%) with a promotion intervention (e.g. health promotion),

31 (18%) with a preventative intervention (18%), 21 (12%) with a treatment intervention,

22 (13%) with a non-specified intervention, and 105 (61%) with a process. Some articles

involved more than one category of innovation (N = 44; 25%)

Components of Readiness. Eighty-six (50%) articles dealt with factors that

influence motivation, 103 (60%) dealt with innovation-specific capacities, and 113 (65%)

dealt with general capacities. A majority of articles included more than one component of

readiness. Seventy (40%) articles dealt with only one component, 77 (45%) dealt with

two of the components, and 26 (15%) dealt with all three. A correlation matrix of how

the three components co-occurred can be found in Table 7.1. As the variables are

dichotomous, these are phi coeffecients, or rφ.

Table 7.1. Co-occurrence of the three Components of Readiness.

Motivation Innovation-Specific

Capacities

Innovation-Specific Capacities 0.14

General Capacities -0.37*** -0.25***

*p<0.05

**P<0.01

***p<0.001

Motivation. Of the 86 articles that addressed factors that influence motivation, 31

(36%) dealt with relative advantage, 41 (48%) with compatibility, 24 (28%) with

complexity, 17 (20%) with trialability, 20 (23%) with observability, and 38 (44%) with

priority. Forty-nine (57%) articles dealt with one factor, 19 (22%) with two factors, 14

(16%) with three factors, three (4%) with four factors, and six (7%) with five factors. No

article dealt with all six factors identified in the preliminary literature search. A

correlation matrix (rφ) for how the factors that influence motivation co-occurred can be

found in Table 7.2.

Table 7.2. Co-occurrence of the Factors that Influence Motivation

Relative

Advantage

Compatibility Complexity Trialability Observability

Compatibility 0.27***

Complexity 0.34*** 0.29***

Trialability 0.40*** 0.36*** 0.26***

Observability 0.30*** 0.10 0.27*** 0.25**

Priority 0.19* 0.03 -0.01 0.01 0.11

*p<0.05

**P<0.01

***p<0.001

Innovation-specific capacities. Of the 103 articles that addressed

innovationspecific capacities, 91 (88%) articles dealt with innovation-specific knowledge,

skills, and abilities (though this was not a specified search term, as discussed in the

methods section), 30 (29%) with program champions, 39 (38%) with implementation

climate supports, and 27 (26%) with interorganizational relationships. Of the articles that

dealt with interorganizational relationships, 13 (48%) with relationships between delivery

systems, seven (26%) with relationships between delivery and support systems, and seven

(26%) with both types of relationships. Sixty-seven (65%) articles dealt with one

innovation-specific capacity, 22 (21%) with two capacities, 16 (16%) with three

capacities, and 3 (3%) with all four capacities. A correlation matrix (rφ) for how

innovation-specific capacities co-occurred can be found in Table 7.3.

Table 7.3. Co-occurrence of Innovation-Specific Capacities

KSA Champion Imp. Climate

Program Champion 0.22**

Implementation Climate Supports 0.21** 0.45***

Interorganizational Relationships 0.06 0.10 0.07

*p<0.05

**P<0.01

***p<0.001

General Capacity. Of the 113 articles that addressed general capacity, 45 (39%)

articles dealt with organizational culture, 22 (19%) with organizational climate, 8 (6%)

with organizational innovativeness, 12 (11%) with resource utilization, 49 (43%) with

leadership, 49 (43%) with organizational structure, and 28 (25%) with staff capacity.

Sixty-six (58%) articles dealt with one general capacity, 28 (25%) with two, 21 (19%)

with three, three article (3%) with four, and three articles (3%) with five. No article dealt

with either six or seven general capacities. A correlation matrix (rφ) for how general

capacities co-occurred can be found in Table 7.4.

Table 7.4. Co-occurrence of General Capacities

Culture Climate Innovativeness Resource Leadership Structure

Climate 0.29***

Innovativeness 0.18* 0.16*

Resource

Utilization

-.11 -0.10 -0.06

Leadership -0.02 0.07 0.11 -0.12

Structure 0.20** 0.10 0.10 0.13 -0.06

Staff Capacity 0.06 0.21** 0.13 0.13 0.04 0.03

*p<0.05

**P<0.01

***p<0.001

Use of EBSIS techniques. A correlation matrix for how the EBSIS techniques

co-occurred can be found in Table 7.5. As these variables were also dichotomous, these

are phi coefficients (rφ).

Table 7.5. Co-occurrence of EBSIS Techniques to Address Readiness

Tools Training TA QA

Training 0.02

Technical Assistance -0.06 0.17*

Quality Assurance 0.04 -0.02 <0.01

Quality Improvement <0.01 -0.38*** -0.17* 0.07

*p<0.05

**P<0.01

***p<0.001

Type of Outcomes Data Reported. Readiness outcomes are reported changes in

the proposed factor or subcomponents of readiness. Twenty-four (14%) reported both

quantitative and qualitative data, while 29 (18%) articles reported solely quantitative data,

and 120 (69%) reported solely qualitative data. Innovation Outcomes are whether the

innovation (policy, program, or process) that was being introduced to the setting had its

intended outcomes or not. These were less frequently reported in these studies (N = 76;

43%), with 97 (56%) articles reporting no innovation outcome data.

Methodological Characteristics of Included Studies. One hundred fourteen

(66%) articles used versions of case study analyses. Of the 49 articles that used

comparison groups (28% of total), only eight (16%) had random assignment procedures.

Eighty-one (47%) articles had pre-post measurement designs, 57 (33%) solely posttests, 5

(3%) solely pretest, and 30 (17%) had no explicit measurement model. In this

dissertation, an indeterminate/non-explicit measurement design generally meant that the

changes were reported as part of an ongoing process narrative, and not as part of a

traditional results section. Only seven (4%) articles contained no innovation rationale.

Table 7.6 shows how these characteristics were distributed in each factor or

subcomponent.

Table 7.6. Percent of each Factor that Influences Motivation and the Subcomponents of Innovation-Specific and General Capacities

that have the certain Methodological Characteristics

Factor or Subcomponent Evaluation Design Innovation Rationale

Measurement Model

Case

Study

Compari

son

Groups

PretesPost- Pre-Post Indeter.

t test

Only Only

None Includ

Manda ed

ted

Both

Motivati

Relative Advantage (N

=31)

(65%)

8 (26%) 1

(3%)

(42%)

(13%)

(3%)

(68%)

2 (6%) 6

(19

)

Compatibility (N =41) 26

(63%)

(39%)

(7%)

(41%)

(32%)

(20%)

(5%)

(80%)

2 (5%) 4

(10

)

Complexity (N=24)

(58%)

6 (25%) 2

(8%)

(38%)

9 (38%) 4

(17%)

(0%)

(88%)

1 (4%) 1

(4%

)

Trialability (N=17) 8

(47%)

7 (41%) 1

(6%)

(13%)

8 (47%) 2

(12%)

(0%)

(82%)

1 (6%) 2

(12

)

Observability (N=20) 11

(55%)

7 (35%) 0

(0%)

(35%)

(55%)

(10%)

(0%)

(85%)

1 (5%) 2

(10

)

Priority (N =38) 26

(68%)

9 (24%) 2

(5%)

(34%)

(55%)

2 (5%) 0

(0%)

(74%)

(11%)

(13

)

Innovati

on-

specific

capacity

Knowledge, Skills, and

Abilities (N =91)

Champion (N = 30)

(60%)

(63%)

(30%)

(33%)

(3%)

(35%)

(43%)

(48%)

(40%)

(13%)

(5%)

(0%)

(79%)

(73%)

4 (4%)

(13%)

(11%

)

(10

Factor or Subcomponent Evaluation Design Innovation Rationale

Measurement Model

Case

Study

Compari

son

Groups

PretesPost- Pre-Post Indeter.

t test

Only Only

None Includ

Manda ed

ted

Both

)

Implementation Climate

Supports

(N = 39)

(62%)

(38%)

(0%)

(33%)

(56%)

(10%)

(5%)

(67%)

(10%)

(18%

)

Inter- Support and

organizatio Delivery

nal System

Relationshi (n =14)

ps (N=27)

(79%)

2 (14%) 0

(0%)

(43%)

3 (21%) 5

(12%)

(7%)

(86%)

0 (0%) 1

(7%)

Between

Delivery

Systems (n

=20)

(75%)

5 (25%) 0

(0%)

(45%)

4 (20%) 7

(35%)

(10%

)

(90%)

0 (0%) 0

(0%)

General

Capacity

Organizational Culture

(N=45)

(82%)

(27%)

(2%)

(22%)

(56%)

(20%)

(4%)

(80%)

2 (4%) 5

(11%

)

Organizational Climate

(N=22)

(64%)

9 (41%) 1

(5%)

(23%)

(55%)

(18%)

(0%)

(86%)

2 (9%) 1

(5%)

Organizational

Innovativeness (N=8)

(50%)

6 (75%) 0

(0%)

(38%)

4 (50%) 1

(13%)

(0%)

(75%)

(13%)

(13%

)

Resource Utilization (N 9 2 (17%) 0 4 5 (4%) 3 0 10 0 (0%) 2

Factor or Subcomponent Evaluation Design Innovation Rationale

Measurement Model

Case

Study

Compari

son

Groups

PretesPost- Pre-Post Indeter.

t test

Only Only

None Includ

Manda ed

ted

Both

=12) (75%) (0%) (33%) (25%) (0%) (83%) (17

)

Leadership (N =49) 37

(76%)

(31%)

(0%)

(41%)

(43%)

(16%)

(0%)

(88%)

1 (2%) 5

(10

)

Organizational Structure

(N =49)

(78%)

(22%)

(4%)

(29%)

(51%)

(29%)

(2%)

(82%)

2 (4%) 6

(12

)

Staff Capacity (N =28) 19

(68%)

(36%)

(0%)

(25%)

(34%)

(11%)

(7%)

(75%)

2 (7%) 3

(11%

)

Total (N

=173)

114

(66%)

(28%)

(3 )%

(33%)

(47%)

(17%)

(4% )

141

(83%)

) (3%6 19

%(11

)

Table 7.7 reports how elements of implementation quality (Meyers et al., 2012)

were distributed in each factor or subcomponent. One hundred twenty (69%) articles

tracked at least one indicator of implementation quality of the support technique that was

used. An index score was created for Implementation Quality (IQ) by summing each

element of implementation quality (i.e., fidelity, dosage, participant responsiveness,

differentiation, reach, adaptation) that was present in a study into a single number. The

possible scores on this index ranged from 0 to 6. However, the maximum score obtained

was five, obtained by three studies (Ganz et al., 2009; Bonell et al., 2010; Leon et al.,

2013), meaning that five of the six elements of implementation quality were present. For

all studies included in the analyses, the average implementation quality was 1.28 (1.22),

meaning that the average study only reported slightly more than one component of

implementation quality. This statistic will also be reported for each subsection.

Table 7.7. Frequency that Elements of Implementation Quality appeared for each Factor that Influences Motivation and

Subcomponent of either Innovation-Specific or General Capacity

Fidelity Dosage Participant

Responsiveness

Differentiation Reach Adaptation Average

IQ (SD)

Motivation Relative Advantage (N =31) 3 (9.6%) 14

(45%)

10 (32%) 3 (10%) 6

(19%)

11 (35%) 1.52

(1.52)

Compatibility (N =41) 7 (18%) 17

(41%)

13 (32%) 2 (4.9%) 6

(15%)

13 (32%) 1.41

(1.50)

Complexity (N=24) 3 (13%) 9

(38%)

14 (58%) 2 (8.3%) 2

(8.3%)

10 (42%) 1.67

(1.43)

Trialability (N=17) 1 (11%) 10

(59%)

6 (35%) 0 (0%) 3

(18%)

3 (18%) 1.35

(0.93)

Observability (N=20) 0 (0%) 5

(25%)

9 (45%) 2 (10%) 2

(10%)

2 (10%) 1.00

(0.79)

Priority (N =38)

6 (18%) 16

(42%)

15 (39%) 3 (8.0%) 7

(18%)

11 (29%) 1.53

(1.45)

Innovationspecific

capacity

Knowledge, Skills, and

Abilities (N =91)

Champion (N = 30)

9 (10%)

7 (23%)

(46%)

(43%)

30 (33%)

12 (40%)

3 (3.3%) 3

(10%)

(19%)

(27%)

21 (23%)

11 (37%)

1.34

(1.22)

1.80

(1.61)

Implementation

Supports

(N = 39)

Climate 7 (18%) 16

(41%)

17 (46%) 5 (19%) 7

(18%)

15 (39%) 1.64

(1.51)

Interorganizational

Relationships

(N=27)

Support and

Delivery

System

(n =14)

0 (0%) 2

(14%)

5 (36%) 2 (14%) 1

(7.0%)

4 (29%) 1.00

(0.96)

Between

Delivery

Systems(n

=20)

1 (5.0%) 9 (45%) 8 (40%) 1 (5.0%) 2

(10%)

9 (45%) 1.50 (1.40)

General

Capacity

Organizational Culture (N=45) 4 (8.9%) 10 (22%) 18 (40%) 3 (7%) 7

(16%)

19 (42%) 1.36 (1.35)

Organizational Climate (N=22) 1 (4.5%) 8 (36%) 16 (73%) 2 (9.1%) 4

(18%)

6 (27%) 1.68 (1.29)

Organizational Innovativeness

(N=8)

1 (13%) 1 (13%) 4 (50%) 1 (13%) 1

(13%)

2 (25%) 1.25 (1.83)

Resource Utilization (N =12) 1 (8.3%) 3 (25%) 2 (17%) 0 (0%) 3

(25%)

5 (42%) 1.17 (1.40)

Leadership (N =49) 2 (4.1%) 25 (51%) 22 (45%) 3 (6%) 9

(18%)

15 (31%) 1.55 (1.31)

Organizational Structure (N =49) 2 (4.1%) 17 (35%) 15 (31%) 1 (2%) 11

(24%)

18 (37%) 1.33 (1.39)

Staff Capacity (N =28)

3 (11%) 10 (46%) 15 (54%) 0 (0%) 8

(29%)

9 (61%) 1.61 (1.45)

Total (N

=173)

14 (8.1%) 64 (37%) 64 (37%) 6 (3.5%) 27

(16%)

47 (27%) 1.28 (1.22)

Note: N =7 articles dealt with both types of interorganizational relationships

Research questions.

1. How are the subcomponents of readiness defined across content areas? What, if

any, evidence supports the existence of subcomponents of readiness that were not

mentioned/addressed in the introduction?

2. What are the best methods to build the factors that influence the motivational

component of readiness?

3. What are the best methods to build the innovation-specific subcomponents of

readiness?

4. What are the best methods to build the general capacity subcomponents of

readiness?

This section is structured in the following way. First, the different ways that the

factors or subcomponents were discussed will be identified (question 1). For questions

24, the evidence for changing the specific factor or subcomponent using certain

techniques will be reviewed both quantitatively and qualitatively for each of the proposed

factors that influence motivation and subcomponents of innovation-specific capacities

and general capacities. Methodological characteristics of the studies are reported for each

factor or subcomponent.

At the conclusion of each section, the evidence for changing either a factor or

subcomponent is qualitatively synthesized and assigned into a category using a version of

Greenhalgh et al.’s (2004) modified World Health Organization Health Evidence Network

criteria (WHO-HEN) (Øvertveit, 2003) (see Table 7.8), which was adapted to include

information about the support system evaluation design.

Table 7.8: Levels of evidence

Level of

evidence

Description Evaluation

requirement

Strong direct

evidence:

Consistent findings in two or more empirical studies

of appropriate design and high scientific quality

undertaken in health service organizations (include

both behavioral health and health care)

Requires

participant

randomization

Moderate direct

evidence Consistent findings in two or more empirical studies

of less appropriate design and/or of acceptable

scientific quality undertaken in health service

organizations

Requires

comparison

group

Limited

evidence

Only one study of appropriate design and acceptable

quality available, or inconsistent findings in several

studies.

Requires

quantitative

measurement

Minimal

evidence

At least one study of minimal quality available. This

includes practitioner self-reports, single organization

case-studies, and other qualitative reports

No evidence: No relevant study found

Following this lengthy section, I will then turn to questions five and six:

5. Do tailored support system activities (i.e., those that address specific components

of readiness) lead to better innovation outcomes than those that do not?

6. When is organizational readiness too low to be responsive to tailored Support

System activities?

Motivation.

This section reviews the evidence for support system strategies for each factor that

influences motivation.

Relative Advantage. Definitions of relative advantage were reasonably

consistent across articles. These definitions included the value of specified innovation

over an alternative innovation (Carlfjord, Lindberg, Bendtsen, Nilsen, & Andersson,

2010; Diker, Cunningham-Sabo, Bachman, Stacey, Waters, et al., 2013; Erwin,

Hamilton, Welch, & Hinds, 2006; Foley, Pockey, Helme, Song, Steward, et al., 2012;

Gordon, Jones, Goshman, Foley, & Bland, 2000; Leon, Lewin, & Matthews, 2013; Rikli,

Huizinga, Schafer, Atwater, Coker et al., 2009; Smith, Murphy, Phillips, Paulsen,

Vislosky et al., 2009; Stenger, Montgomery, Briesemeister, 2007), a explication about

how the innovation will benefit the employee (Grass & Worsley, 2001; Ramos &

Ferreira-Pinto, 2002), and an a priori belief in the efficacy of the innovation (Meredith,

Yano, Hickey, & Sherman, 2005).

No studies used random assignment procedures to control for influences on

relative advantage. No studies attempted to quantitatively measure relative advantage as

an independent variable. While some studies included item-level attempts to measure

relative advantage (e.g. Hammond, Gresch, & Vitale, 2011; Meredith et al., 2005;

Varnell, Haas, Duke, & Hudson, 2007), this information was not disaggregated from

global measures of “provider attitudes,” and “buy-in,” when it was reported. For

instance, in a nutritional education study by Diker et al (2013), training and TA led to

gains in “motivation to deliver” a cooking intervention at nine months post-training, but

they did not report relative advantage as a specific variable.

Twenty studies reported changes in relative advantage qualitatively. Multiple

support system activities were used to address relative advantage. Carlfjord et al (2010)

used a random assignment design to providing training and TA that specifically targeted

relative advantage versus a “common sense” support method. However, the way the

authors reported readiness outcomes (i.e. qualitative impressions of relative advantage,

both positive and negative, for both groups) did not allow for distinctions about how the

targeted group was distinct from the control group.

Other groups offered incentives when an innovation was used by either

individuals (Bassett, Vollman, Brandwene, & Murray, 2013; Mayer et al., 2011; Paarlberg

& Perry, 2007) or organizations (Bonnell, Sorhaido, Strange, Wiggins, Allen, et al.,

2010). For example, in Paarlberg and Perry (2007) employment interviews indicated that

incentives contributed to increased use of the innovation when these incentives were

consistent with preexisting employee values. However, in Mohammadi, Mohammadi,

Hedge, Zohrabi, and Ameli, (2007) incentives that were given to teams (i.e.

the organizational level) for participating in QI projects did not impact changes in

qualitative assessment of individual-level relative advantage of a medical process.

In two studies, relative advantage was addressed by removing perceptions of

punishments resulting from participating in QI activities in a hospital setting (Bagian,

Lee, Gosbee, DeRosier, Stalhandske, et al., 2001; Brush, Balakrishnan, Brough, Hartman,

Hines et al., 2005). In both studies, this strategy led to an increase in how often safety

violations were reported. This increase was accomplished by also targeting the hospitals’

organizational culture, and will be discussed later under the special case of “safety

culture.” Lekan-Rutledge (2000) provided a list of suggested TA strategies to build

relative advantage for an innovation to assist urine voiding in a geriatric population, but

did not report whether these strategies were actually used.

Methodological Characteristics of Included Studies. Of the 31 studies that

addressed relative advantage, sixty-five percent (N = 20) of the studies were single

organization case studies. Thirteen studies had pre-post measurement designs, 13 had

posttest only designs, one study has a pretest only design, and four had indeterminate

measurement designs. Only eight studies used comparison groups. Twenty-one studies

reported some type of innovation rationale, three studies mandated innovation use, and

five studies reported both criteria. The average implementation quality of the support

system strategies on relative advantage in these studies was 1.52 (SD =1.52).

The evidence reviewed above from the qualitative case studies indicates that

relative advantage can be impacted by support system activities. Due to the current lack

of quantitative models to measure relative advantage for an innovation, there is currently

minimal evidence that support techniques can change perceived relative advantage for an

innovation.

Compatibility. There was variability in how broadly compatibility was defined.

This included adaptability of the innovation (Bonvin, Barral, Kakebeeke, Kriemler,

Longchamp, et al., 2013; Carlfjord et al., 2010; Edwards, Moloney, Jacko, & Sainfort,

2008; Foley et al., 2012; Ganz, Yano, Saliba, & Shekelle, 2009; Harshbarger, Simmons,

Coelho, Sloop, & Collins, 2006; Leon et al., 2013; Sipilä, Ketola, Tala, & Kumpusalo,

2008; Stevens, Lancer, Smith, Allen, McGhee, 2009), conceptual overlap between the

innovation and the organizational mission (Keats, 2009; Vatieri, Gopaul, Brown, &

Hostetler, 1994), the time commitment required (Erwin et al., 2006), the relevance,

efficiency, and acceptability of the innovation (Lyon, Charlesworth-Attie, Vander Stoep,

& McCauley, 2011; Meredith et al., 2005), feasibility of the innovation (Bonnel et al.,

2010; Maffli, Schaaf, Jordan, & Güttinger, 2008), the cultural salience of the innovation

(Yancey, Lewis, Guinyard, Sloane, Nascimento, et al., 2006), and the overlap with the

organization’s current capacities (Shah, Noble, Umulisa, Dushimiyimana, Bukhman, et

al., 2008; Welton & Jarr, 1997).

There was no quantitative assessment of the perceived compatibility of an

innovation. One study used random assignment procedures to control for influences on

compatibility (Bonvin et al., 2013). Thirty-eight studies assessed changes in

compatibility qualitatively. Multiple studies used training (N = 27) and technical

assistance (N = 16) to address compatibility of an innovation by targeting the fit of the

innovation with those who would implement it. This included addressing individuallevel

values (Diker et al., 2013; Kirsh, Schaub, & Aron, 2009) and organizational culture

(Diker et al., 2013; Edmundson, 2012; Kuper, Gold, Callow, Quraishi, King, et al., 2011;

Lyon et al., 2011). Diker et al. (2013) illustrated how these individual and organizational

levels were addressed by specifically providing training on how a school-based

nutritional program complemented the school curriculum and the context of the local

community. Post-training, one participant reflected on the innovation, saying, “This is

origin, history, geography, the universal connectiveness [sic] of how we get food,

historically and currently. I was so happy to see that,” (as quoted in Diker et al., 2013).

Harshbarger et al (2005) tracked innovation changes that made an already

culturallytailored HIV prevention program more appealing to target populations during

implementation. They stressed the importance of TA in maintaining the innovation’s

implementation quality; "Without TA and training, prevention providers may

unintentionally risk altering intervention effectiveness by modifying interventions in

ways that either eliminate or change core elements,” (Harshbarger et al., 2005).

Two studies used tools (i.e., reference guides) to address compatibility by

standardizing and clarifying the innovation (Brown, 2009; Hall & Eccles, 2000). In some

studies, compatibility was addressed as part of the innovation selection process before the

provision of support systems activities (e.g. Edmundson, 2012; Lyon el al., 2011).

Bonnel et al. (2012) and Edwards et al. (2008) gathered formative feedback from

innovation users during a pilot period to make incremental changes to increase the

innovation’s compatibility.

In some studies, addressing compatibility of an innovation negatively impacted

implementation outcomes, particularly fidelity (Bonvin et al., 2013; Campanaro, 2008;

Flaschberger, Nitsch, & Waldherr, 2012; Hall & Eccles, 2000). A randomized control

trial by Bonvin et al. (2013) allowed organizations to determine implementation policies

for an innovation in an effort to increase compatibility. However, 1) subsequent poor

implementation contributed to null differences between the intervention and control

groups, and, 2) compatibility was not quantitatively measured. Similarly, Flaschberger et

al (2012) addressed compatibility of a health promotion intervention by allowing a school

to determine how participation in the implementation would be structured. This

contributed to a low priority to support implementation of the intervention.

Methodological Characteristics of Included Studies. In the literature (N = 41) on

support system influences on compatibility, sixty-three percent (N = 26) of the articles

were case studies. Thirteen studies had pre-post measurement designs, 17 had posttest

only designs, six had a pretest only design, and eight had indeterminate measurement

designs. Sixteen studies used comparison groups, with one using random assignment

(Bonvin et al., 2013). Thirty-three studies reported some type of innovation rationale and

six studies mandated innovation use. The average implementation quality of the support

system strategy in the studies on compatibility was 1.41 (SD =1.50).

While there is some minimal evidence that support techniques can change

compatibility of an innovation, there is conflicting evidence about how appropriate

support activities to increase compatibility given the variation in implementation

outcomes. Quantitative measurement of compatibility is currently underdeveloped.

Complexity. Complexity was defined as difficulty of use (Carlfjord et al., 2010;

Diker et al., 2013), degree of simplicity (Ganz et al., 2009; Lyon et al., 2011; Siddiqi,

Young, Cheater, & Harding, 2008), the number of components in an intervention (and the

interaction of components) and the actions required from users (Brady, Stott, Norrie,

Chalmers, St. George et al. 2011; Herring, Caldwell, & Jackson, 2010), and potential

sources of unnecessary variation (Hunter & Segrott, 2010).

No studies directly measured changes in perceived complexity quantitatively.

Brady, Stott, Norrie, Chalmers, St. George, et al. (2011) attempted to quantitatively

measure complexity by breaking an oral health intervention down into 25 components

and measuring changes in awareness of these components following a staff training.

However, this type of measurement model only measures changes in knowledge (as an

innovation-specific capacity), not perceived complexity of the intervention. Although

Carlfjord et al (2010) used a random assignment procedure, as noted earlier they did not

provide sufficient readiness outcome information about how complexity changed

following targeted training and TA between conditions.

Twenty-one studies attempted to measure changes in complexity qualitatively. QI

was used in six studies to identify the necessary innovation components versus those that

contributed to unwanted variation in outcomes (Hardy, Wertheim, Bohan, Quezada, &

Henley, 2013; Vioral & Kennihan, 2012; Young & Wachter, 2009). Vos , Dücker,

Wagner, & van Merode (2010) reported that QI did not impact perceived complexity and

speculated that not adapting QI strategies for different organizations led to null results.

Maffli et al (2008) addressed the perceived complexity of a tool by soliciting user

feedback to optimize the tool’s organization. Training has a positive impact in reducing

complexity when used prior to implementation (Diker et al., 2013; Ganz et al., 2009;

Kirsh et al., 2009; Lyon et al., 2011). Lekan-Rutledge (2000) suggested using training and

TA to address complexity by simulating use of an innovation but did not provide evidence

of the effectiveness of this technique.

Using tools to standardize sources of perceived complexity did not consistently

lead to readiness outcomes. Herring et al. (2010) used an adapted checklist to structure

and standardize medical rounds, reporting that it useful to manage demands of patients

and professionals. Hunter and Segrott (2010) implemented a standardized child birth

procedure through the use of a clinical pathway. Doctors qualitatively reported that using

the pathway did not enhance patient safety because the pathway did not accurately

account for the perceived complexity of childbirth (Hunter & Segrott, 2010).

Methodological Characteristics of Included Studies. Of the 24 articles that

addressed complexity 58% (N = 14) were case studies. Nine studies had pre-post

measurement designs, nine had posttest only designs, two had a pretest only design, and

four had indeterminate measurement designs. Six studies used comparison groups.

Twenty-two studies reported some type of innovation rationale, two studies mandated

use, and one study had both criteria. The average implementation quality of the support

system strategy in the studies on complexity was 1.67 (SD =1.43).

Qualitative studies indicate the complexity can be impacted by support system

activities. However, the lack of stronger evaluation designs and quantitative

measurement indicate that there is currently only anecdotal and minimal evidence at this

time.

Trialability. Trialability was very consistently defined across all studies: the

opportunity to test use of the innovation (Carlfjord et al., 2010; Diker et al., 2013;

Donald, Dower, & Bush, 2013; Foley et al,.2012; Peltzer, Mataseke, Azwihangwisi,

Babor, 2008; Rikli et al., 2009; Schleyer, Teasley, & Bhatnagar, 2005). There was no

quantitative measurement of trialability. No studies directly addressed how to make an

innovation “more trialable.” The techniques that were used to promote trialability

provided different opportunities to test the innovation. Thirteen studies addressed

trialability within a training setting (e.g. Diker et al., 2013). Of these, 61%, (N= 8)

paired training with follow up and ongoing TA (Carlfjord et al., 2010;

Donald et al., 2013; Lekan-Rutledge, 2000; Richardson, Bromirski, & Hayden, 2012;

Rikli et al., 2009; Shah et al., 2008; Stenger et al., 2007). Three studies addressed

trialability through QI by iteratively providing opportunities to test potential

improvements to the either the innovation (Edwards, et al., 2008; Welton & Jarr, 2005) or

the implementation plan (Rikli et al., 2009)

Methodological Characteristics of Included Studies. In the literature on support

systems influences on trialability (N = 17), forty-seven percent (N = 8) of the articles

were case studies. Eight studies had pre-post measurement designs, six had posttest only

designs, one had a pretest only design, and two had indeterminate measurement designs.

Seven studies used comparison groups. Fourteen studies reported some type of

innovation rationale, one study mandated use, and two studies had both criteria. The

average implementation quality of the support system strategy in the studies on

trialability was 1.35 (SD =0.93).

There is no current evidence that trialability can be changed as a result of support

system activities. Trialability appears to be a dichotomous variable that is either present

or not present. The implications of how this factor is constructed will be addressed in the

discussion section.

Observability. Observability was consistently defined across studies. This

included ongoing data reporting from the innovation (Bassett, Vollman, Brandwene, &

Murray, 2012; Cinquini & Vainieri, 2008; Cox, Wilcock, & Young, 1999; Schwoebel &

Creely, 2010; Varughese, Hagerman, & Townsend, 2013; Yi, Wray, Jones, Bass,

Nishioka et al., 2013), specifically as the result of a feedback system (Bagian et al., 2001;

Carlfjord et al.,2010; Diker et al., 2013; LeKan-Rutledge, 2000; Peltzer et al., 2008;

Petruzzi, 2010; Ring, 2010; Yates, Hochman, Sayles, Stockmeier, 2004), perceived

effectiveness (Cramm, Strating, Bal, Nieboer, 2013), ongoing organizational visibility

(Gordon et al., 2000; Shaha, Brodsky, Leonard, Cimino, McDougal et al., 2005), and the

ability to use outcome measurement techniques tied to the innovation (Vos et al., 2010).

No studies used comparison groups to control for influences on observability

independently. Cramm et al (2013) measured observability quantitatively using a

fouritem scale comparing different QI teams across multiple organizations longitudinally.

Participating in a QI collaborative lead to improved observability scores on the specific

QI projects even after controlling for the influence of team in a multilevel analysis (N =

208; β = 0.07; SE = 0.02; p < 0.01).

Fifteen studies measured changes in observability qualitatively following support

system activities. TA was used following training in six studies, specifically by providing

observability information to participants and leadership on innovation outcomes (Bassett

et al., 2012; Carlfjord et al., 2010; Cinquini & Vainieri, 2008; LekanRutledge, 2000;

Peltzer et al., 2008; Varughese et al., 2013). Cinquini & Vainieri (2008) reported that in

one site implementing a medical measurement system, “It is highly motivating to be able

to measure the output of your own work (especially of teamwork),”

Two studies reported no changes in observability following training alone (Diker et al.,

2013; Kirsh et al., 2009).

Ten studies addressed observability during QI processes. In five of these case

studies, QI results were passively disseminated (e.g. posted on walls so all team members

could see) so that participants could note changes that resulted from the QI project

(Bagian et al., 2001; Cox et al., 1999; Schwoebel & Creely, 2010; Gordon et al., 2000;

Yates et al., 2004). However, when data was presented to users in a confusing manner, it

did not impact preserved observability (Ring, 2010). Similarly, a QA-only support

strategy was ineffective because end users reported that the data was too complex to

interpret and use (Yi et al., 2013).

Methodological Characteristics of Included Studies. Fifty-five percent (N = 11)

of the articles were case studies. Eleven studies had pre-post measurement designs, seven

had posttest only designs, and two had indeterminate measurement designs. There were

no studies with comparison groups. Seventeen studies reported some type of innovation

rationale, one study mandated use, and two studies had both criteria. The average

implementation quality of the support system strategy in the studies on observability was

1.00 (SD = 0.79).

Due to the presence of quantitative models and multiple case studies, there

appears to be limited evidence the observability can be impacted by support systems

activities. However, these quantitative measurement models are not in wide use at this

time.

Priority. Priority was defined as importance of the innovation (Alhatmi, 2011;

Bagian et al., 2001; Cox et al., 1999; Ganz et al., 2009; Hall & Eccles, 2000; Leon et al.,

2013; Mohammadi et al. 2007; Ring, 2001; Ross & Crumpler, 2007; Yates et al., 2004),

in one instance following a seminal event (i.e. an organ transplant with mismatched blood

types; Alton, Frush, Brandon, & Mericle, 2006), commitment to implement the

innovation (Bassett et al., 2012; Eliopoulus, 2013), perceived organizational status of the

innovation (Bohanon, Fenning, Carney, Minnis-Kim, Anderson-Harriss, et al., 2006;

Richardson et al., 2012; Stenger et al., 2007; Thomas & Galla, 2012), professed

leadership support for the innovation (Donald et al., 2013; Ganz et al., 2009; Hammond

et al., 2011; Kennerly, Richter, Good, Compton, & Ballard, 2011; Meredith et al., 2005;

Nehlin, Fredricksson, Grӧnbladh, & Jannson, 2012; Rask, Parmalee, Taylor, Green,

Brown et al., 2007; Rikli et al., 2009; Rohrbach, Graham, & Hansen, 1993; Shurman &

Lynch, 1994; Shaha et al., 2005), and perceived organizational support (McCormick,

Mâsse, Cummings, & Burke, 1999).

A distinction arose between support system activities to raise the priority of an

innovation versus activities to set priorities. Setting priorities generally occurred as part

as a needs assessment process (e.g. Alexander, Memiah, Henley, Kaiza-Kangalawe,

Shumbusho, et al., 2012; Robinson, Williams, Dickinson, Freeman, & Rumbold, 2012).

This dissertation focuses on the activities to enhance the innovation’s priority. Articles on

priority-setting were not retained for the analyses unless another component of readiness

was addressed.

The quantitative measurement of priority was more developed than other factors

that influence motivation. In a comparison group study on an educational intervention

(Rohrbach et al., 1993), principals were trained on the importance of an intervention. The

staff who ultimately implemented the intervention reported greater perceived support and

encouragement for the intervention than the comparison group (t(58) = 2.98; p < 0.01) at

the end of the school year. Bohanon et al. (2008) used items within a survey to assess the

priority of effective positive behavior supports (innovation). After initial training and TA,

priority for the innovation changed positively from baseline (U = 892.5, T = 3238.5, p <

0.001).

Following training in patient safety culture, Thomas & Galla (2013) reported a

positive change in mean of 10.9 points on a scale measuring perceived expectations

regarding safety behaviors though did not report statistical significance level of this

change. McCormick et al. (1999) found no changes in perceived priority of a skin cancer

prevention intervention following training for nurses (Friedman two-way ANOVA (Fr) =

2.80, p = 0.25) and doctors (Fr = 2.00; p = 0.37). Although a comparison group was

used, there were no differences in changes between either group: nurses (WilcoxonMann-

Whitney (Wx) = 4232.5; z = -0.90; p = 0.37; r = -0.09) or doctors (Wx = 3480.0, z

=-1.52; p = 0.13; r = -0.15). This null result was attributed to “too many competing

priorities in their practices,” (McCormick et al., 1999). Hammond et al. (2011) reported

posttest scores only in “intention to support,” and “perceived buy-in,” following an

extended period of QI, and a noted that a longer term follow-up would be needed to

determine whether or not these changes in priority would be sustained.

Thirty-four articles reported qualitative changes in priority. Training led to

changes in perceived importance in priority in multiple studies (e.g. Alhatmi, 2011; Yates

et al., 2011,) especially when leadership was involved in communicating the importance

in the innovation (Bagian et al., 2001; Eliopoulus, 2013; Ganz et al., 2009; Gifford,

Davies, Tourangeau, & Lefebre, 2011; Mohammadi et al., 2007). In two QI projects,

priority was addressed by collectively agreeing on the goals and importance of a project

(Cox et al., 1999; Kuper et al., 2011). Conversely, lack of agreement between users

negatively impacted priority in one study (Vos et al., 2010).

Mandating participation in support system activities and innovation use was

effective in increasing perceived priority of an innovation (Leon et al., 2013; Nehlin et

al., 2012; Rask et al., 2007; Vioral & Kennihan, 2012). Priority was significantly

correlated with relative advantage (rφ = 0.19; p < 0.05) and often involved framing the

innovation “not only as a requirement, but as a feasible and desirable way of improving

[services]” (Leon et al., 2013). Several studies noted that priority was important to an

implementation process, but not how it was priority was changed (Ross & Crumpler,

2007; Ring, 2001).

Methodological Characteristics of Included Studies. In the literature of support

systems influences on priority (N = 38), sixty-eight percent (N = 26) of the articles were

case studies. Twenty-one studies had pre-post measurement designs, 13 had posttest only

designs, two had a pretest only design, and two had indeterminate measurement designs.

Nine studies used comparison groups. Twenty-eight studies reported some type of

innovation rationale, four studies mandated use, and five studies had both criteria. The

average implementation quality of the support system strategy for influencing priority

was 1.53 (SD = 1.45).

Although there are quantitative models assessing changes in priority following

support system activities, the findings are inconsistent. As such, there is currently limited

evidence that priority can be change.

Motivation Section Summary. Table 7.9 summarizes the evidence for using

support system activities to promote change in the factors that influence motivation.

Table 7.9. Summary Evidence Table for Factors that Influence Motivation

Evidence

Minimal

Evidence

Limited

Evidence

Moderate

Direct

Evidence

Strong

Direct

Evidence

Relative

Advantage

Compatibility X

Complexity X

Trialability X

Observability X

Priority X

Innovation-Specific Capacity.

This section reviews the evidence for support system strategies changing each

subcomponent of innovation-specific capacities.

Knowledge, Skills, and Abilities. Knowledge, skills, and abilities (KSA) was not

used as a search term due to the potential for wide variability between studies. This is

because each innovation has its own set of KSA. Therefore, the majority of the KSA

findings reported in this section are from studies that identified KSA in conjunction with

other readiness sub-components (N = 88), with five additional studies only focusing on

innovation KSA. In all cases, KSA were framed in terms of specific innovation

requirements (e.g. Anogianakis & Maglaverra, 2001; Gordon et al., 2010; Harding,

Taylor, Leggat, & Wise, 2011; Leitz, 2008; Mayer, 2011; Mohammadi et al., 2007;

Varnell et al., 2007; Yates et al., 2004).

KSA were measured quantitatively in only 11 studies (Auon, Shahid, Le, &

Packer, 2012; Diker et al., 2013; Donald et al., 2013; Gordon et al., 2010; Green, Malsch,

Kothari, Busse, & Brennan, 2012; Harding et al., 2011; Joly, Booth, Shaler, & Mittal,

2012; Kirsh et al., 2009; McCormick et al., 1999; Olson, Muchmore, & Lawrence, 2006;

Ten Have, Nap, & Tulleken, 2013). In 91% of these studies (N = 10), positive changes in

KSA were reported following training. Only three studies reported follow-up TA (Donald

et al., 2013; Green et al., 2012; Joly et al., 2012). The study that did not report changes in

KSA (Harding et al., 2011) noted low training dosage and low priority as possible

explanation for null effects.

Two studies used randomized control designs (Cleland, Hall, Price, & Lee, 2007;

McCormick et al., 1999). McCormick et al. (1999) reported between-group changes in

knowledge as the results of training in how to screen for skin cancer (general skin

knowledge; F(1, 78) = 3.96; p = 0.051); skin cancer prevention knowledge (F(1, 78) =

5.97; p = 0.02)), but not in abilities to screen for skin cancer (F(1, 78) = 0.75; p = 0.39).

Cleland et al. (2007) reported improvement in innovation outcomes, but not how KSA

changed between groups.

Eighty-two articles reported qualitative changes in KSA. However, these reported

changes were often vague and implicit, referring to how staff were trained in some

particular intervention (e.g. Christianson, Pietz, Taylor, Woolley, & Knutson, 1997; Lietz,

2008; Ross & Crumpler, 2007; Sheth, Operario, Latham, & Sheoran, 2007;

Sowden, Hill, Konstantinou, Khanna, Main, et al., 2012; Tachibana & Nelson-Peterson,

2007; Varnell et al., 2007). Training was the primary support strategy, used in 80% of

these studies (N = 66). In the remaining studies, QI (Tachibana & Nelson-Peterson,

2007) and TA (Pascaris, Shields, & Wolf, 2008) were used to positively enhance KSA.

Methodological Characteristics of Included Studies. Sixty percent (N = 55) of the

articles were case studies. Forty-four studies had pre-post measurement designs, 32 had

posttest only designs, three had a pretest only design, and twelve had indeterminate

measurement designs. Twenty-six studies used comparison groups, with two

(McCormick et al., 1999; Cleland et al., 2007) using a randomized designs. Seventy-two

studies reported some type of innovation rationale, four studies mandated use, and ten

studies had both criteria. The average implementation quality of the support system

strategy in the studies on KSA was 1.53 (SD =1.45).

Within the studies reviewed for this dissertation, there is limited evidence that

KSA can be influenced by support system activities. However, this review only looked at

KSA when it occurred in conjunction with other readiness components, and therefore

they may not been the direct focus on the studies that were included.

Champions. Champions were defined as key opinion leaders (Alton et al., 2006;

Bassett et al., 2012; Hall & Eccles, 2000; Hammond et al., 2011; Swain, Schubot,

Thomas, Baker, Foldy, et al., 2004), a person(s) who models an innovation (Beeri,

Dayan, Vigoda-Gadot, & Werner, 2013; Bonuel, Byers, & Gray-Becknell, 2009; Donald

et al., 2013; Ellman, Rosenbaum, & Bia, 2007; Sipilä et al., 2008; Yates et al., 2004),

innovation-specific project leaders (Ross, O’Tuathail, & Stubberfield, 2005; Yancey et

al., 2006), and innovation advisors (Auon et al., 2012),

Only one study (Beeri et al., 2013) measured champions quantitatively by

assessing perceptions of leadership specific to the innovation being studied. One year

post-training and TA, they found no statistically significant differences in how champions

were recognized by fellow staff (Beeri et al., 2013). While some studies controlled for

the presence of a champion (e.g. Donald et al., 2013), no studies controlled for influences

on champion development. A case study by Foley et al. (2012) qualitatively compared an

appointed versus a volunteer champion for a tobacco cessation program. The appointed

champion was better able to integrate organizational support (i.e. implementation

climate), but the volunteer champion was better able to develop innovation-specific

capacity for the innovation. No other studies measured changes in champion behaviors.

This dissertation made no assumptions or hypotheses about the sequence in which

the components of readiness had to be in place and introduced. However, the literature

on champions often cited them as a necessary precursor for other support activities. The

recruitment process for champions varied. Some were selected for training at the

beginning of the innovation (Mayer et al., 2011; Rutland et al., 2009) or appointed to

oversee implementation (Donald et al., 2013; Kuper et al., 2011; Ouslander, Perloe,

Givens, Kluge, Rask et al., 2007). In two cases, the champions volunteered before

implementation began (Auon et al., 2012; Bonuel et al., 2009). In four studies, champions

emerged organically, either following a training (Radke et al., 2011; Sables-Baus, & Zuk,

2000; Yates et al., 2006) or during a QI process (Siddiqi et al., 2008; Swain et al., 2004).

Beeri et al (2013) found that having a champion in an ethics program significantly and

positively predicted implementation climate supports and general organization climate.

Finally, Leon et al. (2013) reported that champions were used to address factors that

influence motivation among the staff.

Methodological Characteristics of Included Studies. In the literature of support

systems influences on champions (N = 30), 63% (N = 19) of the articles were case

studies. Twelve studies had pre-post measurement designs, 13 had posttest only designs,

one had a pretest-only design, and four had indeterminate measurement designs. Ten

studies used comparison groups. Twenty-two studies reported some type of innovation

rationale, four studies mandated use, and three studies had both criteria. The average

implementation quality of the support system strategy in the studies on champions was

1.80 (SD = 1.61).

At this time, there is no evidence that champion development can be impacted or

changed by support system activities. However, the presence of a champion is similar to

trialability in that it is treated as a binary condition within the implementation literature.

Implications for this finding will be addressed in the discussion section.

Implementation Climate Supports. Implementation Climate Supports were not

as precisely defined as other innovation-specific subcomponents. Descriptions included

tangible organizational supports (Cramm et al., 2013; Green et al., 2012; Mayer et al.,

2011; Naylor, Macdonald, Zebedee, Reed & McKay, 2006; Rozenbaum, Brezis, & Porat,

2013; Sables-Baus & Zuk, 2012; Stenger et al., 2007; Talaat, Kandeel, Rasslan, Hajjeh,

Hallaj et al., 2006; Thomas & Galla, 2012; Tyler, Taylor-Seehafer & Murphy-Smith,

2004), leadership support dedicated to using the innovation (Donald et al., 2013; Ellman

et al., 2007; Gifford et al., 2011; Leon et al., 2013; Lynch & Schurman, 1994; Ouslander

et al., 2009; Philliber & Nolte, 2008; Richardson et al., 2012; Wick, Hobson, Bennett,

Demski, Maragakis et al., 2012; Yancey et al., 2006), organizational processes that were

changed to support the innovation (Bassett et al., 2012; Bonuel et al., 2009; Brady et al.,

2011; Douglass & Klerman, 2012), and specific staff dedicated to innovation (Foley et

al., 2012; Rask et al., 2007; Rikli et al., 2009).

No studies used randomization to control for influence on implementation climate

supports, though some studies randomized implementation climate supports as part of the

treatment condition (Carlfjord et al., 2010; Kolko, Baumann, Herschell, Hart, Holden et

al., 2012; Naylor et al., 2006; Rask et al., 2007). Implementation climate supports were

assessed quantitatively in four studies. Two studies used the Agency for Healthcare

Research and Quality’s (AHRQ) Hospital Survey on Patient Safety Culture (HSOPSC)

tool, but reported their results in dissimilar ways (Mayer et al., 2011; Thomas & Galla,

2012). Training led to improved team supports for a safety intervention at one month

follow-up (t(1,97) = -6.20; p <0.001) and 12-month follow-up (t(1,85) = -6.2; p <0.001)

(Mayer et al., 2011). Training also contributed to a positive change in how supervisors

promoted a safety intervention in a hospital (+10.9 change in mean on the scale of

Supervision expectation and actions promoting patient safety). Although the authors

qualitatively labeled this change as an “area of strength,” statistical significance was not

reported (Thomas & Galla, 2012).

Joly et al. (2012) evaluated a multi-state learning collaborative (MLC) to improve

the ability of local public health department to conduct QI projects. The techniques the

MLC used to improve QI included training and TA (P. Russo, personal communication,

3/28/14). Public health departments that participated in the MLC did not show a change

in implementation climate (F(1, 404) = 1.71, p = 0.192). However, within the MLC,

agencies in the lowest quartile of QI capabilities (scoring the lowest on the assessment

tool), showed the greatest change in implementation climate when compared to the other

quartiles (F(14,404)=122.23; p <.0001). In Cramm et al. (2013), QI had a positive

impact on organizational supports (N = 208; β = 0.04; SE = 0.02; p < 0.05) and specific

managerial support (N = 208, β = 0.13; SE = 0.02; p < 0.001) for a QI project.

Twenty-three studies measured changes in implementation climate qualitatively.

TA was used to facilitate additional innovation supports (Bassett et al., 2012; Bonuel et

al., 2009; Carlfjord et al., 2010; Green et al., 2012; Philliber & Nolte, 2008). Cosmetic

additions to organizations (e.g. the hanging of posters communicating characteristic on

the innovation) were used to build implementation climate support in four studies

(Alhatmi, 2011; Bassett et al., 2012; Foley et al., 2012; Yancey et al., 2006). QI was also

used to identify specific staff that could be dedicated to support a safety intervention

(Alhatmi, 2011).

In several studies, the support systems strategy addressed implementation climate

before training (Naylor et al., 2006), TA (Rikli et al., 2009) or QI (Sables-Baus & Zuk,

2012) was provided. In ten studies, leadership was specified as a necessary precondition

for implementation climate supports. Leadership dedicated resources specifically for the

innovation in multiple studies (Alton et al., 2006; Bonuel et al., 2009; Douglass &

Klerman, 2012; Rikli et al., 2009; Wick et al., 2012). In Gifford et al. (2011) training

helped leaders identify specific supports for a diabetes intervention and develop a plan to

implement them. Leadership was also used to mandate participation in training and TA

(Leon et al., 2013; Rask et al., 2007). In Verschoor et al. (2007), senior leadership used

regular walkthroughs (i.e. QA) on the patient floor to non-punitively identify processes

inconsistent with a safety initiative.

In Sables-Baus & Zuk (2012) changes in implementation climate were not

sufficient to keep the intervention sustained. In Yancey et al., (2006), failure to have

adequate implementation climate support for a 13-week health promotion training

intervention was cited a reason for implementation failure (i.e., the innovation was not

implemented with quality).

Methodological Characteristics of Included Studies. Sixty-two percent (N = 24) of

the articles were case studies. Twenty-two studies had pre-post measurement designs, 13

had posttest only designs, and four had indeterminate measurement designs. Fifteen

studies used comparison groups. Twenty-three studies reported some type of innovation

rationale, three studies mandated use, and seven studies had both criteria. The average

implementation quality of the support system strategy for implementation climate

supports was 1.64 (SD = 1.51).

Because several studies measured implementation climate supports quantitatively,

there is there is currently limited evidence that implementation climate can be changed as

the result of support system activities.

Interorganizational Relationships. There are two types of interorganizational

relationships that were examined in this dissertation, those between the Support and

Delivery Systems and those between different organizations within the Delivery System.

No studies used random assignment procedures to control for relationship-building

supports strategies.

Support/Delivery System Relationships. Relationships between the Support and

Delivery System were defined specifically and tangibly (e.g. organizational partnerships

built around resources (Olson et al., 2006; Smith et al., 2009; Stevens et al., 2009). No

studies measured Support/Delivery System relationships quantitatively. Although 14

articles addressed Support/Delivery system relationships, only five studies reported

qualitatively changes in these relationships. In these studies, ongoing TA helped to

enhance relationships between the Support and Delivery system (Bassett et al., 2012;

Carlfjord et al., 2010; Ross et al., 2005; Pascaris, Shields, & Wolf, 2008; Philliber &

Nolte, 2008). In the other nine studies, the relationship was identified as part of the

support process, but changes in the relationship were not reported.

Delivery System Relationships. Delivery System relationships were either defined

as partnerships maintained between similar organizations working toward similar goals

(Donald et al., 2013; Maynard et al., 2012; Watson-Thompson, Woods, Schober, &

Schultz, 2013) or in terms of quality of the relationships (e.g. openness and collaboration

between agencies; Cambridge & Parkes, 2006).

Only one study measured relationships between Delivery Systems quantitatively

(Donald et al., 2013). When comparing targeted TA to a training-only group in order to

implement a suicide prevention program, the TA group saw improvements in number of

networks for liaison and support (F(1, 249) = 5.90; p = 0.016), for information exchange

and training (F(1, 249) =12.89; p < 0.001), and local planning (F1, 249) = 10.64; p =

0.001). No change was seen in number of organizational links for client referral (F(1,

249) = 1.36; p < 0.245) or case conferencing (F(1, 249) = 0.022; p < 0.883).

Qualitative outcomes for strategies to build relationships between delivery

systems were addressed in 12 studies. Following a two-day training that targeted

Delivery System relationships, Cambridge and Parkes (2006) reported key learning

outcomes of:

increased awareness of shared and linked policies and procedures, an appreciation

of respective roles and responsibilities of the workers and different agencies

involved, knowledge of the actions required to help achieve more effective

interagency working and case co-ordination, [and] increased confidence in

information sharing between interests and in managing confidentiality.

Learning collaboratives (also called communities of practice) were specifically

used to build relationships between Delivery Systems in six studies (Cohen, Shore, &

Mazade, 1991; Donald et al., 2013; Erwin et al. 2006; Hayes, Yousefi, Wallington, &

Ginzburg, 2010; Linehan, 2010; Maynard et al., 2012). In Maynard et al., (2012) and

Smith et al. (2009) “mentor” organizations paired with less developed organizations

helped to facilitate enhanced use of QI. Following a two-year period of TA, Pascaris et

al. (2008) found that staff reported increased quality of interorganizational relationships

in the mental health system.

Methodological Characteristics of Included Studies. Eighty-one percent (N=22)

of the articles were case studies. This was the second highest percentage of case studies

among all factors and subcomponents in the analysis. Six studies had pre-post

measurement designs, 13 had posttest only designs, and eight had indeterminate

measurement designs. Five studies used comparison groups. Twenty-three studies

reported some type of innovation rationale, and one study additionally mandated use.

The average implementation quality of the support system strategy inthe studies on

interorganizational relationships was 1.14 (SD = 1.28). The average implementation

quality of the support system strategy in the studies on support/delivery system

relationships was 1.00 (SD = 0.96). The average implementation quality of the support

system strategy in the studies on delivery system relationships was 1.50 (SD = 1.40).

Given the presence of quantitative comparison designs, there is limited evidence

that relationships within Delivery System can be enhanced using support system

activities. However, there is minimal evidence that relationship between the Support and

Delivery System can be enhanced through Support System activities.

Innovation-Specific Capacity Section Summary. Table 7.8 summarizes the

evidence for using support system activities to promote change in innovation specific

capacities

Table 7.10. Summary Evidence Table for Innovation-Specific Capacities

Evidence

Minimal

Evidence

Limited

Evidence

Moderate

Direct

Evidence

Strong

Direct

Evidence

100

Knowledge, Skills, and Abilities X

Champion X

Implementation Climate

Supports

Interorgan- Support X

izational and Relationships Delivery System

Between X Delivery Systems

General Capacity

This section reviews the evidence for support system strategies for each

subcomponent of general capacities.

Organizational Culture. Organizational Culture was defined as a group of

people that express and interact through values, beliefs, goals, policies, operations, and

uniform expectations on how things are done that are passed down to new members

(Alhatmi, 2011; Bonell et al., 2010; Christianson, Pietz, Taylor, Woolley, & Knutson,

1997; Moore & Putnam, 2008; Pronovost, Weast, Rosenstein, Sexton, Holzmueller, et al.,

2005; Schwoebal & Creely, 2010; Wallis & Kennedy, 2013). Other related definitions

included how things are done in the organization (Beeri et al., 2013; Eliopoulos, 2013;

Edmundson, 2012; Fox et al., 2012; Rikli et al., 2009; Swain et al., 200;), the mission and

goals of the organization (Mayer et al., 2011; Varkey, Karlapudi, & Hensrud, 2008),

organizational values (Chung & Nguyen, 2005; Sables-Baus & Zuk, 2012; Paarlberg &

Perry, 2007; Tumerman & Carlson, 2012), a healthy work environment (Herbst,

101

Swengros, & Kinney, 2010), a shared vision (Green et al., 2012) and the “community” of

the organization (Naylor et al., 2006).

Safety culture was mentioned in 20 studies. It was specifically defined as all the

activities and behaviors in a hospital that are relevant to patient safety (Alton, Mericle, &

Brandon, 2006; Alton et al., 2006a; Edwards, Scott, Richardson, Espinoza, Sainfort et al.,

2008; Fudickar, Hörle, Wiltfang, & Bein, 2012; Leonard, Graham, & Bonacum, 2004;

Thomas & Galla, 2013; Timmel, Kent, Holzmuetller, Paine, Schulick et al., 2010; Shaha

et al., 2005; Verschoor et al., 2007). Safety-culture was split into two constructs for

purposes of this dissertation: 1) Organizational Culture, or the expectations of how things

were done in an organization, and 2) Organizational Structure, the organizational

processes that allow for activities to take place. Splitting safety culture apart in this way

allowed for it to be analyzed as two general capacities. Further, it was treated as general

capacity because each of these capacities was fundamental to the operations of a hospital

and translated across different innovations and conceptually consistent with quality aims

for health care as articulated by the Institute of Medicine (2000). This section will focus

solely on the first part of this construct and the support system strategies that were shown

to influence organizational culture. The structural component of safety culture will be

addressed in the section on organizational structure.

No studies used random assignment to control for influences on organizational

culture. Though several studies assessed different groups, only one varied the support

methods between groups. Using a delayed implementation design for a Comprehensive

102

Unit-Based Safety Protocol (CUSP) that included training and QI in two hospital ICUs,

Pronovost et al. (2005) found improvements in staff ratings on organizational

expectations for safety. However, statistical significance of the changes was not reported.

Eight additional studies measured changes in organizational culture quantitatively,

though using varying degrees of evaluation rigor. In response to a fifteen month QI

intervention, Edwards et al. (2008) reported positive changes in safety expectations using

the AHRQ HSOPSC measure from baseline assessment (F = 7.40; p =0.008), degrees of

freedom not reported). Green et al. (2012) found changes in one of two Head Start sites

on a measure of shared vision (t(30) = -2.44, p <0.05) at the end of the second year of

implementation.

Echoing findings discussed earlier, Joly et al., (2012) did not find differences in

organizational culture for those who participated in TA and a learning collaborative (F(1,

404) =0.04; p = 0.84), but did find improvement among organizations that were initially

rated as low in “QI maturity” (F(1, 404) = 120.07; p <0.001). Following training and

TA, Beeri et al. (2013) found changes in expectations about how things “should” be done

in an organization (t(108) =7.71; p <0.01), but not in how people behaved in the

organization (t(108) =1.19; p =0.22). A TA and QI safety program led to a 6% increase in

perceptions of hospital teamwork and 8% increase in perceived hospital safety (N = 28)

from pretest measures, both changes reported as significant at p < 0.001 (Timmel et al.,

2010). A QI program in an academic hospital (Varkey et al., 2008) led to statistically

significant improvements in item-level measures of culture.

Chin, Pun, Ho, & Lau (2002) used QI to influence changes in a comprehensive

model of organizational culture. They reported positive changes pre-post changes on a

103

number of dimensions: teamwork (F(1, 198) = 11.45; p < 0.001), participation (F(1, 198)

= 373.65; p < 0.001), corporate vision (F(1, 198) = 8.19; p <0.01), communications (F(1,

198) = 141.85; p <0.001), feedback (F(1, 198) = 8.63; p <0.01), and recognition (F(1,

198) = 36.49; p <0.001). Four additional dimensions (continuous improvement,

measurement, empowerment, and training and career development) were not statistically

significant. Using only posttest frequency count data following training and QI in safety

culture (Alhatmi, 2011), 16% of staff agreed that they had seen an unsafe practice, 38%

believed that errors went unreported, and 71% agreed that the unit took time to identify

errors. This was interpreted as an increase in safety culture. Mohammadi et al. (2007)

reported that training and TA in patient safety led to 70% of respondents saying that the

support strategies had a positive impact on culture.

Qualitative changes in organization culture were reported in 17 studies. A tool

without any other support strategy was used to reframe an organization’s values and

mission in Clossey, Mehnert, & Silva (2011). Paarlberg and Perry (2007) used training

and tools (in the form of organizational posters) to communicate organizational values

throughout the U.S. Department of Defense. Positive changes in organizational

philosophy toward service delivery were reported following TA (Pascaris et al., 2008) and

following training (Herbst et al., 2010). Eliopoulos (2013) used both training and TA to

target organizational culture but did not report any changes. In Hardy et al. (2013), QI

was used to shape group efforts to form mission and vision statements for an organization

at the beginning of a childhood obesity prevention program.

Alton et al. (2006a) and Shaha et al. (2005) reported that following QI for safety

culture, there were qualitative increases in staff ownership over safe patient practices that

104

because routinized. Training in safety culture had similar results (Alhatmi, 2011; Moore

& Putman, 2008). QI led to improve expectations for error reporting in three studies

(Alhatmi, 2011; Alton et al., 2006b; Verschoor et al., 2007). While QI had a similar focus

on culture in Brush et al. (2005), culture was reported to be as a qualitative barrier to

error reporting. Anonymous error reporting and blind chart review was used to sidestep,

rather than change, perceived organizational mistrust.

Sables-Baus and Zuk (2012) interpreted the mainstreaming of a medical process

that was being monitored by QA but without TA support to be evidence of a culture

change in a neonatal intensive care unit (NICU). Moore and Putman (2008) similarly

inferred a deeper cultural shift when an innovation was unofficially disseminated to staff

that did not take part in training. The mainstreaming of an innovation was also reported

as culture change in Chung & Nguyen (2005) and Fox et al. (2012).

Methodological Characteristics of Included Studies. Eighty-two percent (N = 37)

of the studies on organizational culture were case studies. This was the highest

percentage of case studies across all subcomponents and factors. Twenty-five studies had

pre-post measurement designs, 10 had posttest only designs, one had a pre-test only

design, and nine had indeterminate measurement designs. Twelve studies used

comparison groups. Thirty-six studies reported some type of innovation rationale, two

studies mandated use, and five studies had both criteria. The average implementation

quality of the support system strategy in the studies on organizational culture was 1.36

(SD = 1.35).

Although the measurement of organizational culture is well developed,

particularly in the medical field around the issue of safety, only one study of adequate

105

comparative design could be found for this analysis (Pronovost et al., 2005). Because of

the lack of strong evaluation models, there is currently limited evidence that

organizational culture can be changed through support system activities.

Organizational Climate. Organizational Climate was defined as satisfaction

with the work environment (Cox et al., 1999; Krugman & Smith, 2003; Leonard et al.,

2004; Mohammadi et al., 2007; Varkey et al., 2008; Wallis & Kennedy, 2013), the

psychological impact of the work environment on well-being (Kolko et al., 2012), an

organizational atmosphere of trust (Mayer et al., 2011; Williamson & Taylor, 2001), staff

stress (Green et al., 2012), staff morale (Anogianakis & Maglaverra, 2000; Williams,

Sims, Burkhead, & Ward, 2002), identification with an organization (Beeri et al., 2013),

and comprehensively as “distinct areas of work life, consisting of perceived workload,

control, reward, community, fairness, values, exhaustion, efficiency, and cynicism,”

(Cummings, Spiers, Sharlow, Germann, Yurtseven, et al., 2013).

Two studies used comparison groups to study influences on organization climate.

Kolko et al. (2008) randomized practitioners into two different support strategies for a

CBT intervention: 1) training plus TA or, 2) a training-as-usual condition. Climate was

measured with the Organizational Social Climate Questionnaire (Glisson & Schoenwald,

2010). There were no significant differences between groups. At 6-month follow-up,

there was a statistically significant decline in climate (β = -8.44, p <0.001) that did not

vary between conditions. Williamson & Taylor (2001) measured the impact of a

leadership training program on enhancing perceived trust within a nursing unit, finding “a

statistically significant difference of 54% in the training group versus 21% in the

comparison group.” No other statistics were reported.

106

Seven additional studies measured climate quantitatively. Krugman & Smith

(2003) found improvement in staff satisfaction (F(4, 1400) =4.81; p <0.001) over a five

year period of implementing a charge nurse training program. Following training and QI

over the course of two years, Timmel et al., (2010) reported statistically significant (p <

0.01) percent increases in job satisfaction (65% to 71%) and in perceptions of working

conditions (48% to 55%).

Five quantitative studies did not report change in climate. In Mohammadi et al.,

(2007), only 35% of staff members who responded to a survey rated a hospital QI

initiative as having a positive impact on satisfaction. Varkey et al. (2008) reported a

nonsignificant change (p = 0.60) in an item level measure of climate two years following

a QI process that targeted climate. Cummings et al. (2013) trained leadership on climate

and found no significant change in how participants viewed the working environment,

with the effects of the training wearing off over time cited as the reason for this null

outcome. Wallis and Kennedy (2012) reporting no significant change (p > 0.09) in

satisfaction with the team environment following training. Training and TA in

organizational ethics had no impact on perceived commitment to the organization (t(108)

= -0.15; non-significant) or perceived quality of work life (t(108) =1.10, non-significant)

(Beeri et al., 2012).

Seven studies reported qualitative outcomes. Green et al. (2012) reported

improvements in climate following training and TA in both sites in a comparison study on

child behavioral intervention. Leonard et al. (2004) reported improved satisfaction with

the work environment following use of a tool to structure nursing communication

strategies. Improved satisfaction was also reported following QI in two studies (Cox et

107

al., 1999; Rikli et al. 2009). Training in an intervention to increase empathy led to

reported improvements in nursing morale and, “a calmer work environment,” (Herbst et

al., 2010). Training in an onboard medical software led to reported improvements in

maritime ship morale (Anogianakis & Maglaverra, 2000). However, a training and TA

program was not successful in addressing climate in one site and was attributed to the

innovation being mandated (no factors that influence motivation were addressed)

(Douglass & Klerman, 2012).

Other subcomponents of readiness had a positive impact on climate in five

studies. Climate was reported qualitatively improved following efforts of a champion

(Rikli et al., 2009), improved organization culture (Herbst et al., 2010) staff capacity

(Green et al., 2012; Herbst et al., 2010), and leadership (Krugman & Smith, 2003;

Williamson & Taylor, 2001).

Methodological Characteristics of Included Studies. Sixty percent (N = 14) of the

articles were case studies. Twelve studies had pre-post measurement designs, five had

posttest only designs, one had a pre-test only design, and four had indeterminate

measurement designs. Nine studies used comparison groups. Nineteen studies reported

some type of innovation rationale, two studies mandated use, and one study had both

criteria. The average implementation quality of the support system strategy in the studies

on organizational climate was 1.68 (SD = 1.29).

The literature on support system interventions to address organizational climate

was more inconsistent than other subcomponents and factors. Because of these

inconsistent findings across studies, there is currently limited evidence that support

systems strategies can promote changes in organizational climate.

108

Organizational Innovativeness. Innovativeness was defined as general norms

about change (Birdi, 2007; Carlfjord et al., 2010; Ganz et al., 2009; Wallis & Kennedy,

2012), organizational risk taking and tolerance of mistakes (Cramm et al., 2013),

continuous organizational learning (Edwards, Scott, et al., 2008; Lynch & Schurman,

1994), and being able to apply QI techniques to other projects (Rikli et al., 2009).

Cramm et al. (2008) used a delayed implementation design to measure the impact

of QI on innovativeness in several organizations. They found a small, but statistically

significant decrease in innovativeness (t(286) =-2.99; p <0.001). However, in Edwards,

Scott et al. (2008) QI contributed to a significant increase in organizational learning

(mean change 3.7 to 3.9; p < 0.01) in a community hospital, but not in an academic

hospital (p = 0.34). Birdi (2007) compared three named workplace creativity training

programs. Two of these, Business Beyond The Box (BTBB); (r =0.26, p < 0.001) and

Divergent Thinking (Lateral thinking; r = 0.18, p < 0.001) had significant associations

with the ability to generate work ideas. BTBB was also positively associated with

implementation of ideas at work ( r= 0.24, p < 0.01). A training curriculum in critical

thinking, Six Thinking Hats had no significant impact on creatively. Cramm et al. (2008)

addressed the antecedents of innovativeness, finding that observability (β = 0.07, SE =

0.02; p < 0.001), implementation climate (β = 0.04, SE = 0.02; p < 0.05), and leadership

(β =0.13; SE = 0.02; p < 0.001) predicted innovativeness after controlling for the

influence of organization.

Two studies reported changes qualitatively. Following training and TA, staff

reported seeing the benefits of continued change in Carlfjord et al. (2010). In Rikli et al.

109

(2009), the use of QI led staff to report that the QI process that was used could be

generalized to other projects in a hospital.

Methodological Characteristics of Included Studies. Fifty percent (N = 4) of the

articles were case studies. Three studies had pre-post measurement designs, three had

posttest only designs, and one had an indeterminate measurement design. Six studies

were in comparison groups designs. Six studies reported some type of innovation

rationale, one studies mandated use, and one study had both criteria. The average

implementation quality of the support system strategy in the studies on organizational

innovativeness was 1.25 (SD = 1.83).

The literature on organization innovativeness was smaller than any other factor or

subcomponent. However, given the presence of quantitative group designs, there appears

to be moderate evidence that innovativeness can be influenced by support system

activities.

Resource Utilization. Resource Utilization was defined as cost-effectiveness in

operations (Friedman, Rathod, Farias, Graham, Powell et al., 2010), skills at acquiring

reimbursement (Phillips-Angeles, Song, Hannon, Celedonia, Stearns, et al., 2013), fund

raising and proposal writing skill (Ramos & Ferreira-Pinto, 2002), the ability to conduct

gap analyses to free up resources (Richardson et al., 2012), the ability to dedicate time for

the innovation (Varughese et al., 2013), and ability to estimate cost and productivity

(Leshikar, Pierce, Salcedo, Bola, & Galante, 2013).

No studies used random assignment to control for influences on the development

of resource utilization capacities. Resource utilization was measured quantitatively in

three studies. By using QI to review redundancy in a central venous line placement

110

training program, Leshikar et al. (2012) were able to decrease supply costs by 90% and

faculty costs by >$12,000. The rate of infections from central venous lines also

subsequently dropped. In a similar program, Varughese et al. (2013) used QI plan-

dostudy-act cycles to help free up ten hours per week of nurse practitioner time to

dedicate to a specific screening intervention. Training and TA helped staff develop fund-

raising and proposal writing skills for AIDS prevention in 285 organizations (Ramos and

Ferreira-Pinto, 2002). However, while the number of organizations that were able to

incorporate as non-profits institutions approached significance, (p = 0.062), a

nonsignificant number of organizations were able to acquire additional funding sources (p

=0.6502).

Five studies measured changes in resource utilization qualitatively. QI was used

to examine areas of process redundancy, and subsequently non-committed resources were

reinvested in the building infrastructure (Phillips, 2005; Richardson et al., 2012;

Robinson et al., 2012). A training program for a preventative colon health intervention

specifically addressed how to receive reimbursement for services (which were

implemented in the first six months post-training) (Phillips-Angeles et al., 2013). TA was

used to help develop support systems relationships to secure additional grant funding

(Stevens et al., 2009).

Methodological Characteristics of Included Studies. Seventy-five percent (N = 9)

of the articles were case studies. Five studies had pre-post measurement designs, four

had posttest only designs, and three had indeterminate measurement designs. Two studies

were found within comparison groups designs. Ten studies reported some type of

innovation rationale with two studies mandating innovation use. The average

111

implementation quality of the support system strategy in the studies on resource

allocation was 1.17 (SD = 1.40).

Given that there is some quantitative evaluation on how to enhance resource

allocation capacities, there is currently limited evidence that resource allocation can be

influence by support system activities.

Leadership. Leadership was defined in terms of attributes that ideal leadership

should possess and behaviors they should demonstrate. This included management skills

(Austin, Regan, Samples, Schwartz, & Carnochan, 2011; Cohen et al., 1991; Gagliano,

Ferris, Colton, Dubitzky, Hefferman et al., 2010; Kuo, Thyne, Chen, West, & Kamei,

2010; Matovu, Wanyenze, Mawemuko, Wamuyu-Maina, Bazeyo et al., 2011; Omoike,

Stratton, Brooks, Ohlson, & Storfjell, 2011; Ten Have, Nap, & Tulleken, 2013),

relationship skills (Cummings et al., 2013; Grass & Worsley, 2001; Lew, Martinez, Soto,

& Baezconde-Garbanati, 2011; Tumerman & Hedberg Carlson, 2012; Wallis & Kennedy,

2012) , communication skills (Ten Have et al., 2013) and community-building skills

(Austin et al., 2011; Gagliano et al., 2010; Kuo et al. 2010; Lew, Martinez, Soto, &

Baezconde-Garbanati, 2011; Matovu et al., 2011).

Additionally, six studies defined leadership as the capacity to engage others

within an organization to use an innovation (Alleyne & Jumaa, 2007; Bonuel et al., 2009;

Cramm et al., 2013; Mohammadi et al., 2007; Shekleton, Preston, & Good, 2010; Wick et

al., 2012). Three studies used a specific leadership development framework, the Kouzes

and Posner model, which includes five key domains: ability to challenge the process,

inspire a shared vision, enable others to act, model the way, and encourage the heart

112

(Crofts, 2006; Cummings et al., 2013; Krugman & Smith, 2003). However, these three

studies did not report changes in leadership in similar ways.

There were no studies that used random assignment to control for influences on

leadership development. Leadership was measured quantitatively in six studies. Charge

nurse training (Krugman & Smith, 2003), led to posttest improvements in three of the

Kouzes and Posner dimensions; challenging the process (t(56) = -3.18, p =0.002),

inspiring a shared vision (t(56) = -2.26, p =0.02) and modeling (t(56) = -3.18, p =0.002).

They reported declines in two dimensions: enabling others to act (t(56) = 2.25, p =0.01)

and encouraging the heart (t(56) = 2.35, p = 0.01). However, this study was weakened

by a poor respondent rate and turnover, with only 28% of the total number of nurses

returning measures. Cummings et al. (2013) used training to develop leadership using

the same Kouzes and Posner framework, but only found positive changes in two scales

(inspiring a shared vision and challenging the process) a specific type of leader (those in

operational roles). Furthermore, these changes were not sustained at follow-up.

Although Cummings et al. (2012) used quantitative measurements, they did not provide

sufficient information (i.e. no information about variances) to perform a meta-analysis of

these two studies.

Fielden et al. (2009) compared two types of individual-level TA for nursing

leadership development: mentoring (non-directive, sharing wisdom and encourage

professional development) versus coaching (directive problem solving). Mentees

outperformed coaching recipients on a measure of leadership development that included

statistically significant gains in: effectiveness of management style (t(12)=-3.83, p =

113

0.002), negotiation skills (t(13)=-2.51, p = 0.026), networking with professional contacts

(t(13)=-2.83, p = 0.014), self-confidence (t(13)=-2.28, p = 0.04), leadership skills and

capabilities (t(12)=-3.41, p = 0.005), ability to be open and direct others (t(11)=-2.83, p =

0.014), ability to problem solve (t(12)=-2.74, p = 0.018), perceived leadership ability

(t(11)=-2.80, p = 0.017), and ability to negotiate (t(11)=-5.75, p = 0.0004). Coaching

recipients saw statistically significant gains only in effectiveness of management style

(t(13)=-6.50, p < 0.001) and ability to negotiate (t(11)=-3.89, p = 0.002). Cleary et al.

(2005) and Grass and Worsley (2001) reported quantitative gains in leadership skills, but

provided insufficient information to determine statistical significance. Using a

pretest/posttest design, a training and TA program to build organizational capacity

(including leadership) for an early childhood promotion intervention did not lead to

statistically significant changes on perceived effectiveness of leadership at the end of the

second year of implementation (Green et al., 2012).

Six additional studies measured leadership develop qualitatively. TA helped

leadership improve capacity to deliver quality services (Alleyne & Jumaa, 2007) and

selfassessed personal leadership competencies (Law & Aquilina, 2013). Training and TA

helped to build capacity to take on more leadership responsibilities in Austin et al. (2011)

and Matovu et al. (2011). Training led to improved individual ability to foster

collaborations in Cohen et al. (1991). Using a thematic content analysis, Carr, Lhussar,

Reynolds, Hunter, & Hannaway, (2009) reported that participants in public health

leadership development training that was followed by TA gained:

An increased capacity for self-reflection, an energizing effect, an increased

political astuteness and confidence as leaders, enhanced strategic thinking

114

abilities, greater awareness of health improvement tools and an enhanced

evidence base for practice.

TA helped prioritize leadership-development, but not actually change leadership

behaviors in substance abuse prevention coalitions (t(6)=0.55, p < 0.05)

(WatsonThompson et al., 2013). In eleven studies, leadership was mentioned as a

necessary precondition for further readiness development, particularly in terms of priority

and implementation climate supports (e.g. Cramm et al., 2013; Ellman et al., 2007; Ganz

et al., 2009; Gifford et al., 2011) and organizational climate (Cummings et al., 2012;

Tumerman & Hedberg Carlson, 2012). This will be expounded upon in the discussion.

Methodological Characteristics of Included Studies. Seventy-six percent (N = 37)

of the articles were case studies. Twenty-one studies had pre-post measurement designs,

twenty had posttest only designs, and eight had indeterminate measurement designs.

Fifteen studies were found within comparison groups designs. Forty-three studies

reported some type of innovation rationale, one study mandated innovation use, with five

studies mandating innovation use. The average implementation quality of the support

system strategy in the studies on leadership was 1.55 (SD = 1.31)

Although there is some evidence from quantitative comparison studies,

inconsistent findings indicated that there is currently limited evidence that leadership can

be enhanced through the use of support systems strategies.

Organizational Structure. Organizational structure was defined as the

communication and workflow process (Cox et al., 1999; Cramm et al., Edwards et al.,

2008; Hall & Eccles, 2000; Hunter et al., 2010; Krugman & Smith, 2003; Leonard et al.,

2004; Mayer et al., 2011; Moore & Putman, 2008; Rasmussen, Kondrup, Staun,

115

Ladefoged, Lindorff, et al., 2006; Richardson et al., 2012; Ross et al., 2005; Siddiqi et al.,

2008; Sipilä et al., 2008; Varkey et al., 2008), the ease in which processes take place

(Alhatmi, 2011), how the organizational system is organized (Brown et al., 2003; Grass

& Worsley, 2001), the processes and people involved in care (Herring et al., 2011;

Lehman, Hudson, Appley, Sheehan, & Slevin, 2011; Leon et al., 2013; Pronovost et al,

2005; Vioral & Kennihan, 2012), degree of teamwork (Lamb, Green, Benn, Brown,

Vincent, et al., 2013; Lekan-Rutledge, 2000), job requirements (Nelson, Batalden, Plume,

& Mohr, 1996), degree of autonomy in practice (Christianson et al., 1997), shared

leadership (Perry, 2000), and implementation teams (Talaat et al., 2006).

This section will also continue the discussion of safety culture, focusing on the

structural components that promote patient safety in health care settings. Again, I have

included this within general capacities because these processes are fundamental to the

operation of these facilities. Twenty articles addressed the structural component of safety

culture (e.g. Alton et al., 2006a, 2006b; Verschoor et al., 2007; Wick et al., 2012).

No studies used random assignment to control for influences on structure.

Twenty studies measured organizational structure quantitatively. Five studies used the

AHRQ HSOPSC (discussed earlier) to measure changes following a support system

intervention using a pre-post design (Edwards et al., 2008; Kennerly et al., 2009; Mayer

et al., 2011; Schwoebel & Creely, 2010; Thomas & Galla, 2013). Relevant to

organizational structure, the HSOPSC measures perceptions of teamwork,

communication openness, feedback, and hospital transitions. Mayer et al. (2011)

reported statistically significant changes in two hospital intensive care units (ICUs,

pediatric (PICU) and surgical (SICU). Changes were reported following QI in the

116

median values in two dimensions: communication openness (PICU; F(2,95) = 22.99, p <

0.01); (SICU; F(2,88) = 16.28, p < 0.01), and improvements in the median values for

teamwork in the SICU (F(2,89) = 0.41, p = 0.04) but not the PICU. In Edwards et al.

(2008), mean scores on the HSOPSC were reported following QI. Positive changes were

only seen in feedback (F(2,428) = 9.688, p < 0.001), whereas negative findings were seen

in hospital transitions, (F(2,428) = 13.25, p < 0.001). Thomas and Galla (2013) only

reported posttest percent change in HSOPSC scales following training, with “significant

changes” in communication openness, feedback, teamwork (qualitatively labeled as an

area of strength), and hospital transitions. The number of respondents was not was

reported. Kennerly et al (2009) reported posttest-only data following QI on the number

of staff who strongly agree there was increased communication and cooperation around

patient safety. Finally, Schwoebal and Creely (2010) reported a targeted safety QI

initially helped to improve scores in teamwork and communication, but only reported raw

percentages, did not provide information about how this percentage should be interpreted,

and did not report statistical significance of the changes between hospitals studied.

Lamb et al. (2013) reported the cumulative impact that training (F(1,430) = 5.051,

p < 0.05), top down QI (F(1,477) = 48.756, p < 0.05); staff-driven QI (F(1,592) = 20.679,

p < 0.05), then tools (F(1,619) = 69.174; p <0.05) had on improving communication

(change in mean 29.6 to 38.3) and quality of teamwork (37.8 to 43.0) in making

carebased decisions. Training and QI increased awareness of necessary communication

strategies for safety by 48% (p < 0.05) in one ICU (Pronovost et al., 2005). Similar

findings were found following training in Moore & Putnam (2008), who found that staff

members were two to four times more likely to report medical errors. At follow-up, an

117

organizational-level survey showed that staff that had not been trained were also

increasing their error reporting behaviors. Varughese et al. (2013) reported that QI helped

reduce redundancy in a screening process which ultimately allowed nurse practitioners to

more efficiently complete work tasks. Lehman et al. (2010) reported that QI led to the

elimination of 57 mid-level positions and consequently improved overall organizational

operations. TA in Ramos & Ferreira-Pinto (2002) led to a decreased average number of

people on leadership boards (13.8-10.5, p = 0.69), which they interpreted as evidence of

structural streamlining. TA was unsuccessful in helping coalitions who prioritized

developed operating structures make changes to their daily operations (t(6)=1.37, p =

0.22) (Watson-Thompson et al., 2013)

Changes in organizational structure were addressed qualitatively in twenty-five

studies. Inter and intra-departmental communications in hospitals were improved

through the use of QI (Fudickar et al., 2012; Maynard et al., 2012; Rikli et al., 2009;

Varkey et al., 2008), TA (Williams et al., 2002), and tools (Herring et al., 2011). Tools

were used in two studies to reframe the decision-making structures in organizations

(Clossey et al., 2011; Leonard et al., 2004). QI also led to new patient safety reporting

policies in Alton et al. (2006b). Jensen, Johannsson, and Löfström (2013) reported that a

failure to attend to structure as part of QI contributed to null outcomes when using QI to

implement a public health policy. Lekan-Rutledge (2001) also reported minimal

structural changes following TA for a urine-voiding intervention.

Staff roles were reorganized and restructured in several studies using TA (Brown

et al., 2003; Winslow, Fickley, Knight, Richards, Rossen et al., 2011), training

(Christianson et al., 1997; Sipilä et al., 2008) and QI (Richardson et al., 2012). Training

118

helped to centralize operations of medical partners who used a nuclear medicine

department (Grass & Worsely, 2001), though the reported techniques that facilitated this

centralization were poorly specified. A steering committee was formed in Perry (2000) to

structurally support the implementation of a health process improvement innovation.

Similar governance structures were formed in Talaat et al. (2006) for a patient safety

initiative.

Methodological Characteristics of Included Studies. Seventy-eight percent (N =

38) of the articles were case studies. Twenty-five studies had pre-post measurement

designs, fourteen had posttest only designs, two had pretest only designs and eight had

indeterminate measurement designs. Eleven studies used comparison group designs.

Forty studies reported some type of innovation rationale, two studies mandated

innovation use, with six studies met both criteria, and one study had no innovation

rationale. The average implementation quality of the support system strategy in the

studies on organizational structure was 1.33 (SD = 1.39).

There are substantial measurement models supporting the use of support system

methods to positively influence changes in organization structure, and multiple studies

that used comparison groups to track changes in organizational structure (e.g. Mayer et

al., 2011; Pronovost et al., 2005). Currently, there is moderate evidence that support

system strategies can be used to influence changes in organizational structure.

Staff Capacity. There were many different job types within the articles included in these

analyses, and therefore staff capacity was defined diversely. This included the ability to

conduct QA processes (Bouchet, Francisco, & Øvretveit, 2002), teacher’s abilities to

respond opportunistically and flexibly to student needs (Campanaro, 2007), adequate

119

career development (Fielden et al., 2009), ability to use best practices to support socio-

emotional learning (Green et al., 2012), engagement in QI projects (Hayes et al., 2010;

Schwoebel & Creely, 2009), skills at connecting with patients (Herbst et al., 2010), ability

to retain qualified staff (Hillman & Foster, 2011; Williams et al., 2002), ability to do

facilitation, advocacy, collaboration, and culture/community competency (Lew et al.,

2011), skills to respond to client needs (Lietz, 2008), technical and project management

skills (needs assessment, evaluation design) (Philliber & Nolte, 2008; Ramos & Ferreira-

Pinto, 2002; Watson-Thompson et al., 2013), and skills in medication procedures

(Rozenbaum et al. 2012). Each of these tasks was considered an essential and

fundamental part of the profession within the article, and are therefore included in general

capacities instead of innovation-specific capacities.

No study used random assignment to control for influences on the development of

general staff capacity. Fielden et al. (2009) compared mentee versus coaching TA

relationships for career development skills. Both groups showed significant increases:

mentee (t(11)=-3.64; p = 0.004), and coaching (t(12) = -3.40; p =0.005)).

Three studies measured changes in staff capacity through retention rates in

nursing departments. Williams et al (2002) compared a nursing development program

versus nursing units that did not implement this strategy. This led to significant reduction

in turnover (t(2)=-3.707; p =0.002). Hillman and Foster (2010) reported on an extensive

screening and nurse development process that consisted of matching and ongoing

professional development TA to reduce nursing turning. However, they only reported

post-training retention rates. Winslow et al. (2011) set up a tiered TA nursing

development program that had nurse move up through higher “levels” indicative of

120

greater clinical skills. They reported that <1% of staff in the top levels left the

organization, indicating considerable stability among nurses who were assessed as having

the greatest capacity.

Lietz (2008) used training to develop group supervision skills for critical thinking

in case evaluation. Those who participated in training reported an increase in ability to

use critical thinking (t(236)=-5.05, p < 0.01). Following training and TA, Ramos &

Ferreira-Pinto (2009) reported an increase in organizations (N = 235) that were able to

conduct needs assessments (p = 0.065) saying, “these changes were assumed to indicate

an increase in capacity.” Following a TA prioritization process, Watson-Thompson et al.

(2013) found increases in staff capacity in the ability to: analyze information (t(6)=2.44; p

<0.05), implement effective interventions (t(6)=2.44; p <0.05), and use evaluation skills

(t(6)=2.42; p <0.05).

Six studies reported changes in staff capacity qualitatively. Olson et al. (2006)

reported ongoing QI contributed to the development of a community of practice for

diabetes care. Schwoebal & Creely (2010) promoted further use of QI for a safety culture

by actively recruiting and selecting team members with interest in "reporting, analysis,

[and] feedback.” Improvements in a child care staff’s ability to communicate and interact

with families were qualitatively assessed via staff interviews following training

(Douglass & Klerman, 2012). Extensive training helped to reduce turnover for providers

delivering teen pregnancy prevention interventions (Philliber & Nolte, 2008). In Sipilä et

al. (2008), an ongoing TA and QI program for inter-professional care coordination led to,

“common treatment practices—‘house rules’….increased evidence-based knowledge of

important volume diseases, new skills and tools for patient education and

121

selfmeasurement.” A QI program for preventative services was unable to forestall

attrition of key members and consequently program sustainability was in jeopardy in

Tyler et al.

(2004).

Methodological Characteristics of Included Studies. Sixty-eight percent (N = 19)

of the articles were case studies. Eighteen studies had pre-post measurement designs,

seven had posttest only designs, and three had indeterminate measurement designs. Ten

studies used comparison group designs. Twenty-one studies reported some type of

innovation rationale, two studies mandated innovation use, with three studies having both

criteria. Two studies presented no innovation-rationale. The average implementation

quality of the support system strategy in the studies on staff capacity was 1.61 (SD =

1.45).

Due to the presence of quantitative models and use of comparison groups, there is

currently moderate direct evidence that general staff capacity can be built with support

system activities.

Section Summary. Table 7.11 summarizes the evidence for using support system

activities to promote change in general capacities.

Table 7.11. Summary Evidence Table for General Capacities

Evidence

Minimal

Evidence

Limited

Evidence

Moderate

Direct

Evidence

Strong

Direct

Evidence

Organizational Culture X

Organizational Climate X

Organizational

Innovativeness

122

Resource Utilization X

Leadership X

Organizational Structure X

Staff Capacity X

Question 5: Do tailored support system activities (i.e., those that address specific

components of readiness) lead to better innovation outcomes than those that do not?

Among the 85 articles that reported innovation outcomes, 71 (83%) specifically

targeted one of the components of readiness. Tailored support systems activities were

coded categorically as “targeted” or “not targeted.” Innovation outcomes refers to

whether or not the innovation that was implemented in the article has its intended

outcome (i.e. the innovation was effective). Innovation outcomes were coded as “met

intended outcomes,” (i.e. the results of the innovation were as predicted) and “did not

meet intended outcomes.” Logistic regression was used to answer this question since the

outcome variable, innovation outcomes, was categorical.

Support system activities that specifically targeted a component of readiness, had

a log odds of 1.92 (SE = 0.84; p = 0.0234). This is equal to an odds ratio of 6.8 with a

95% confidence interval [1.18,38.83], meaning that support systems that target a

subcomponent of readiness are almost seven times likelier to have innovation outcomes

than those that do not specifically target a subcomponent of readiness. The Wald test (a

two-degree of freedom chi-square in which the second degree of freedom is the

covariance) indicated that the effect of targeted support system activities was significant

(X2 (2, 85) = 6.2, p = 0.044). Table 7.12 contains the OR for each specific support system

technique. While only QI was statistically significant, tools approached significance.

123

95% CI

Tools 0.23 [0.04-1.01] 0.06

Training 1.65 [0.18-12.44] 0.63

Technical Assistance 1.81 [0.36-12.34] 0.58

Quality Assurance >100 [<.001- NA] 0.99

Quality Improvement

*No OR significant at p <0.05

10.94 [1.47-243.76] 0.05

Table 7.12. Summary Evidence Table for EBSIS technique leading to innovation to

outcomes

Question 6: When is organizational readiness too low to be responsive to tailored

Support System activities?

For this question, I examined whether there were any systematic difference

between articles that reported changes in readiness outcomes verses those that did not

report changes. Readiness outcomes refer to whether or not a factor or subcomponent of

readiness changes in response to support systems activities. The purpose was to see

whether there were certain conditions in which readiness outcomes were less likely.

Thirty-one articles (18%) did not report changes in the components of readiness.

Readiness outcomes were coded as “met intended outcomes,” (i.e. the subcomponent or

factor changed in a positive direction) and “did not meet intended outcomes. Table 7.13

reports the frequencies for each factor or subcomponent.

Table 7.13. Factors and Subcomponents of Readiness in Studies that Reported Changes

in Readiness Outcome versus Those that Did Not Report Changes.

Readiness Component Reported No Reported

Change Change

Motivation Relative Advantage (N =31) 26 (84%) 5 (16%)

124

OR P value

Compatibility (N =41) 31 (76%) 10 (24%)

Complexity (N=24) 18 (75%) 6 (25%)

Trialability (N=17) 12 (71%) 5 (29%)

Observability (N=20) 14 (70%) 6 (30%)

Priority (N =38) 31 (82%) 7 (18%)

Innovation-

Specific

Capacities

Knowledge, Skills, and Abilities (N

=91)

Champion (N = 30)

74 (81%)

30 (100%)

17 (19%)

0 (0%)

Implementation Climate Supports (N

= 39)

35 (90%) 4 (10%)

Inter- Support/Delivery organizational

(N=8)

Relationships Delivery System

(N=27) (N =12)

6 (75%)

12 (100%)

2 (25%)

0 (0%)

Both (N=7) 6 (86%) 1 (14%)

General

Capacities

Organizational Culture (N=45)

Organizational Climate (N = 24)

42 (98%)

22 (96%)

3 (2%)

2 (4%)

Organizational Innovativeness (N =8) 6 (75%) 2 (25%)

Resource Utilization (N =12) 11 (92%) 2 (8%)

Leadership (N = 49) 38 (64%) 11 (36%)

Organizational Structure (N = 49) 44 (90%) 5 (10%)

Staff Capacity (N = 28) 21 (75%) 7 (25%)

Total (N 142 (82%) 31 (18%)

Support system activities that specifically targeted a component of readiness had a

log odds of 1.13 (SE = 0.46; p = 0.0137). This is equal to an odds ratio of 3.1 with a 95%

confidence interval [1.23,7.48], meaning that support systems that targeted a factor or

subcomponent of readiness are about three times likelier to have readiness outcomes than

those that do not specifically target a factor or subcomponent of readiness. The Wald test

125

=173)

indicated that the effect of targeted support system activities on changes in readiness was

significant (X2 (2, 173) = 58.7, p < 0.001).

Statistical differences between the frequencies in the article characteristics

between articles that reported versus did not report changes in readiness were computed

using the chi-square statistic (Table 7.14). There were three significant differences

between types of articles. Articles that used a case study format were more likely to

report changes in readiness (Χ2(1, N = 173) = 15.54, p <0.001), articles that used random

assignment procedures were less likely to report changes in readiness (Χ2(1, N = 173) =

5.87, p = 0.02), and articles that did not explicitly state a rationale for the innovation

being implemented were less likely to report changes in readiness (Χ2(1, N = 173) =

7.63, p < 0.01).

Table 7.14. Difference in Article Characteristics between Studies that Reported Changes

in Readiness vs. Those that Did Not Report Changes in Readiness.

Source of

Bias

Reported

Change

(N =

142)

Reported

Change

(N = 31)

X2 P value

Evaluation

Characteristics

Case Study

103 11 15.54 <0.001*

Assessment

Model

None

Pretest only

0.72

1.12

0.40

0.30

Posttest only 45 12 0.57 0.45

Both 69 12 1.00 0.32

Groups No Group 105 19 2.01 0.16

Comparison 27 7 0.21 0.65

Groups present

Random 4 4 5.87 0.02*

126

Assignment

Test of Between- 6 1 0.07 0.80

group

equivalence

Innovation- Rationale None 3 4 7.63 0.01*

Specification Included 117 24 0.42 0.52

Mandated 5 1 0.01 0.94

Both 17 2 0.79 0.37

*X2 significant at p <0.05

Logistic regression was used to examine whether implementation quality was a

significant predictor of readiness outcomes. The index score on Implementation Quality

(IQ) was not a significant predictor of readiness outcomes (log odds = -0.08, SE = 0.16; p

= 0.6; OR = 0.92, 95% CI [0.69-1.27]). None of the individual components of

implementation quality (e.g. fidelity, dosage, etc.) were significantly linked to readiness

outcomes at p < 0.05. None of the individual components of EBSIS (i.e. tools, training,

TA, QA, and QI) were significantly linked to readiness outcomes at p < 0.05.

There are some differences between studies that are linked to changes in readiness

outcomes. However, due to difficulties in standardizing measurement of the factors and

subcomponents or readiness, it was not possible to statistically determine whether there

are particular conditions or “amount” of readiness under which the parts of factors or

subcomponents will be non-responsive to support system activities. This will be

addressed in the limitations section of the discussion.

127

Chapter 8: Discussion

Overview.

In the implementation literature, there are a number of factors and subcomponents

that are associated with an increased likelihood of achieving outcomes. Organizations

who wish to get results from their innovations have an interest in making sure that these

factors and subcomponents are in place. However, having awareness that certain

capacities and factors that influence motivation are linked to improved innovation

outcomes does not necessarily help organizations to get “more ready.” There is a need for

organizations to know if and how they can effectively put these factors and

subcomponents into place. Broadly, can the Support System help to build the readiness

of organizations (readiness outcomes) in order to help them achieve better innovation

outcomes?

This dissertation set out to synthesize the strength of the evidence on how the

Support System can use various techniques and interventions to build organizational

readiness for implementing innovations. In the introduction, I first brought together two

themes in the readiness literature to enhance current models of organizational readiness:

organizational capacity (e.g. Flaspohler et al., 2008) and organization motivation (Weiner,

2009). I then demonstrated how certain factors that influence motivation and

subcomponents of the capacities identified in the literature are linked to improved

innovation outcomes (e.g. Greenhalgh et al., 2004; Powell et al., 2012; Rogers, 2003).

This dissertation had a number of goals. First, I investigated the strength of the

evidence for changing/enhancing different factors and subcomponents of readiness using

128

the strategies of EBSIS (tools, training, technical assistance, and quality assurance/quality

improvement). Second, I examined whether support system activities that specifically

targeted readiness factors and sub-components as part of an innovation implementation

process demonstrated better innovation outcomes than non-targeted support system

activities. Finally, I examined whether there was any evidence that particular factors or

subcomponents were less responsive to Support System activities, and therefore should

get less weight as part of a readiness-building process. Figure 8.1 illustrates the causal

chain of this dissertation.

EBSIS

Use of:

•Tools

•Training

•Technical

Assistance

•QA/QI

Readiness

Outcomes

Changes in:

Motivation

Innovation-

Specific

Capacity

General

Capacity

Innovation

Outcomes

Did the innovation

have its intended

outcomes?

Figure 8.1. Use of EBSIS to influence Readiness Outcomes and Innovation Outcomes

To answer these questions, I screened 4397 articles in the behavioral health and

medical literature that potentially dealt with the factors and subcomponents during the

process of implementing an innovation. From this larger set, the full text of 297 articles

were reviewed and coded. Ultimately, 173 articles were retained and included in the

syntheses (Figure 7.1). The results were: 1) there is evidence that support system

activities can enhance certain factors and subcomponents of organizational readiness; 2)

support system activities that target readiness are more likely to achieve innovation

outcomes than those that do not, and, 3) there are some statistical differences in articles

129

Implementation

Quality (IQ) of Support

System Strategy

that report changes in readiness versus those that do not. In the following sections, I will

discuss some specific findings, study limitations, and possible future directions.

Discussion.

There was a statistically significant negative correlation between motivation and

general capacity (rφ = -0.37; p < 0.001), meaning that the more an article addressed the

factors that influence motivation, the less likely this article was to address general

capacities. This provides some evidence that the concepts of general capacity and

motivation have not fully been addressed together within the organizational support

literature, which is consistent with the theses in this dissertation’s introduction and

Scaccia et al. (in press). Although there was a small, statistically significant negative

correlation between general capacity and innovation-specific capacity (rφ = -0.25; p <

0.001), this finding was more unexpected since both these concepts are discussed

together in the literature (e.g. Wandersman et al., 2008). What this correlation indicates is

that when articles focused on at least one innovation-specific specific capacity, it was less

likely to address at least one general capacity, and vice versa. While this finding is

inconsistent with work by Flaspohler et al. (2008) that articulates the importance of both

these components when building capacity for an innovation, a cursory literature search

found only one instance of these terms “general capacity,” and “innovation-specific

capacity” co-occurring together prior to 2008 (Wandersman, Stillman, Horwitz, Duffy,

Blachman, et al., 2005), when a special issue of the American Journal of Community

Psychology introduced the ISF. Additionally, this finding may represent a deficit in

reporting, wherein factors that were not the primary focus of the article were not

discussed. Since the factors and subcomponents of organizational readiness have been

130

shown to be linked to innovation outcomes independently, a broader approach to building

readiness that addresses both motivation and capacity represents an improvement in how

Support Systems activities can be provided.

Table 8.1 summarizes the strength of the evidence for each factor that influences

motivation and subcomponent of innovation-specific capacity and general capacity.

While there is variation in the strength of the evidence between the factors and

subcomponents, there generally appears support for the hypothesis that the Support

System can target and build readiness in organizations to implement innovations. The

evidence suggests that Support system activities can be used to enhance nearly all of the

components of readiness (except trialability and champion) in order to improve both

readiness outcomes and innovation outcomes.

Table 8.1. Summary Evidence Table for Factors that Influence Motivation and the

Subcomponents of Innovation-Specific and General Capacities

Evidenc

Minima

Evidenc

Limited

Evidenc

Moderat

e Direct

Evidenc

Strong

Direct

Evidenc

Motivatio

Relative Advantage

Compatibility

Complexity X

Trialability X

Observability X

Priority X

Innovatio

n-specific

capacity

Knowledge, Skills,

and Abilities

Champion

Implementation

Climate Supports

131

Inter- Support

organization and

al Deliver

Relationship y

s System

Betwee

Deliver

System

General

Capacity

Organizational Culture

Organizational

Climate

Organizational

Innovativeness

Resource Utilization X

Leadership X

Organizational Structure X

Staff Capacity X

The strongest evidence was found in general capacities (which also had the largest

number of overall articles) where all subcomponents met the criteria for limited evidence.

In only one factor that influenced motivation (trialability) and one subcomponent

(program champion) was there no evidence that they can be changed by Support System

activities. One possible reason for this can be found in how these constructs are

represented. It is difficult to produce variations and changes within a binary construct;

the construct is either present or not present. Because there is no variation within

trialability, it is not something that can be enhanced, per se, it can only be introduced.

While it may be possible to track dosage of trialability (i.e. more opportunity to practice

with an innovation), no studies measured the dosage of trialability in this way. Further

132

research questions might look at how much practice/exposure is necessary before overall

motivation to use an innovation is impacted.

The presence of a champion was similar to trialability in that it is treated as a

binary condition within the implementation literature. Further studies into the

characteristics of champions may provide information about how to effective select

and/or cultivate champions. For example, Damschroder, Banaszak-Holl, Kowalski,

Forman, Saint et al., (2009) noted,

Active champions directly shape organizational change through four critical

functions: 1) protecting those involved in implementation from organizational

rules and systems that may be barriers, 2) building organizational support for new

practices, 3) facilitating the use of organizational resources for implementation,

and 4) facilitating growth of organizational coalitions in support of

implementation. A champion's effectiveness depends on the strategies used to

engage individuals across professions, and engagement strategies should be

tailored to the organizational setting.

In addition to identifying activities that a champion performs, this observation

also highlights an important point about the relationships between readiness

subcomponents. There are complex relationships between the components of readiness.

There relationships have important implications about the sequence in which these

components are addressed.

For example, one phenomenon that emerged from the synthesis was the

relationship between implementation climate and organizational culture. In Sables-Baus

and Zuk (2012), the mainstreaming of a medical process that was monitored by QA but

133

without ongoing TA was considered evidence of a culture change in a neonatal intensive

care unit (NICU). The mainstreaming (when the innovation becomes part of the shared

expectation of the organization) is considered to be indicative of the innovation being

incorporated into the organizational culture. Therefore, the process of innovation

sustainability (as discussed extensively in Stirman et al., (2012)) may be an integral part

of a larger culture change. This was particular true of patient safety initiatives (e.g. Alton

et al., 2006), with “culture change is at the heart of this quest [for patient safety]”

(Leonard et al., 2004).

However, there was less consensus in the literature around the sequencing of other

factors and subcomponents. For example, in some studies, organizational culture

preceded staff capacity (Timmel et al., 2010) and organizational climate (Varkey et al.,

2008), whereas in Paarlberg & Perry (2007) staff capacity preceded organizational

culture. Key individuals (e.g., specific champions and leaders) were seen as important

preconditions that preceded other parts of readiness in several studies. Champions were

cited as a primary step in several articles in building implementation climate (Rikli et al.,

2009; Sipilä et al., 2008; Spence & Henderson-Smart, 2011; Wallis & Kennedy, 2013).

Champions were also seen as important in addressing factors that influence motivation.

In Leon et al., (2013),

The champion framed the new intervention not only as a requirement but also as a

feasible and desirable [innovation]….the framing of [the innovation] as both an

opportunity to address a service gap may have strengthened the willingness of

nurses to consider its implementation.

134

Leadership was seen as a primary step for building implementation climate and

priorities (Bonuel et al., 2009; Cullen, Greiner, Greiner, Bombei, & Comried, 2005;

Ellman et al., 2007; Ganz et al., 2009; Gifford et al., 2011), organizational climate

(Cummings et al., 2012), organizational culture (Kennerly et al., 2011), organizational

structure (Gaucher & Kratochwill, 1993; Perry, 2000; Lekan et al., 2010), and factors that

influence motivation (Schleyer et al., 2005). It is likely that there are multiple feedback

loops that influence the development (or deterioration) of various components. As

illustrated in Mohammadi et al., ( 2007),

CQI is an organizational culture and largely the product of an organization's

leadership and motivational system. Building a culture takes time. Although CQI

is a long term effort, we should not wait until the ideal culture has evolved.

Results themselves build culture.

As there is no consensus about the sequence with which the components and

factors should be addressed, a more comprehensive Support System approach that

address many (if not all) parts may be the most impactful strategy until there is greater

evidence about the relative strengths and interrelationships among the subcomponents

and factors.

One counter-intuitive finding of note was the minimal evidence that addressing

compatibility of an innovation did not necessarily lead to better quality of

implementation. Although compatibility was a major theme in the implementation

literature (Greenhalgh et al., 2004; Powell et al., 2012; Rogers, 2003), several studies

qualitatively reported that attempts to increase compatibility negatively impact the quality

of implementation and subsequently innovation outcomes (e.g. Bonvin et al., 2013;

135

Flaschberger et al., 2012). So while compatibility (and other factors and subcomponents)

are important predictors of innovation use, organizational readiness by itself will not

guarantee innovation outcomes. The innovation still must be implemented and evaluated

with quality.

Following the results from dissertation research questions one through four, if the

support system can properly and accurately determine specific areas of low readiness in

an organization, it is possible to build readiness with targeted interventions. Support

system strategies that specifically target readiness are about three times more likely to see

changes in the factors and subcomponents of readiness, and nearly seven times more

likely to see positive innovation outcomes. This highlights the utility of using this model

of organizational readiness as a means of getting organizational prepared for putting

innovations into practice.

Only two EBSIS techniques approached significance for innovation outcomes.

QI had a positive impact on innovation use (log odds = -2.39; SE = 1.21; p < 0.05; OR =

10.90; 95% CI [1.47-243.76]). QI tends to be more involved than other EBSIS activities,

and therefore there may be a dosage or participatory component of QI that makes this a

more useful strategy for achieving innovation outcomes. The use of tools (log odds = -

1.49; SE = 0.79; p = 0.06; OR =0.23; 95% CI [0.04-1.01]) had a negative impact; i.e.

using tools means that innovation outcomes were less likely to be achieved. This is

consistent with discussion by Wandersman et al., (2012) that using tools will not be

sufficient to have innovation outcomes. Although the statistical significance of this

finding was just outside of p = 0.05, what this suggests is that tools may be harmful if not

combined with other EBSIS strategies. As tools are commonly distributed through

136

websites without any additional oversight or guidance, future research should examine

the soundness of this strategy.

While it was not statistically possible to isolate the influence of each individual

support method (i.e. TA, QI, etc.) on a particular specific factor or subcomponent due to

low power and subsequently biased parameter estimates, there is some preliminary

support that using techniques in combination with each other leads to incrementally better

results. In a noteworthy study, Lamb et al (2013) showed that cumulative introduction of

training, QI, then tools led to statistically significant improvements in measures of

organizational structure. Although QA and QI are presented together within the EBSIS

framework, there was a low, and non-significant correlation between them (rφ = 0.07; p =

0.38). However, QA appeared in only 11% (N = 20) of the included articles, and this may

have biased parameter estimates.

There were several quantitative differences between studies that showed readiness

outcomes and those that did not show readiness outcomes. First, case studies were more

likely to have readiness outcomes. This may be an artifact of publication bias, whereby

articles that may be more subjected to confounds (like some case studies and process

narratives) might more likely to be submitted for publications because they are showing

results. Second, articles that showed no change in the components of readiness were

more likely to have used random assignment procedures. While this finding is tempered

by a very small sample size (N = 15), it indicates the need for more controlled models to

test the influence on support systems strategies on readiness. Small sample size may also

influence the interpretation of the finding that articles that did not state a rationale for an

innovation were less like to report readiness outcomes. Although this is consistent with

137

the concept of theory failure (Rossi et al., 2004; Wandersman, 2009), this finding should

be interpreted with caution because so few articles did not state an innovation rationale (N

= 7).

Overall, these results synthesize the literature on targeted support. While it has

been previously noted that it is important to account for characteristics of the host

organization (Damschroder et al., 2009; Greenhalgh et al., 2004), this dissertation

presents evidence that it is possible to enhance specific parts of the organization that are

important to reach innovation outcomes. Consistent with the principles of empowerment

evaluation, specifically the principles of improvement and capacity-building (Fetterman

& Wandersman, 2005) it appears from the evidence that organizations have the potential

to get more ready by specifically addressing distinct parts of readiness. It is a somewhat

common phenomenon that the most capable organizations (i.e. the ones who are most

ready) are the ones that will qualify for externally funding. However, many organizations

may be strategically positioned (e.g., in underserved areas) but have deficits in their

readiness to implement high-priority programming like the Community Health Needs

Assessments specified by the Affordable Care Act. What the evidence in this synthesis

shows is that organizations can generally be assisted to get more ready for an innovation.

There is potential to enhance the capabilities of organizations and therefore improve their

ability to get positive innovation outcomes. This is an important finding because it

allows funders to maximize their investments in organizations by more effectively

providing supports to build the capabilities to put innovations into place.

Limitations.

138

There were a number of related limitations that qualify how the results from this

dissertation can be interpreted.

Publication Bias. As with all research syntheses, there is a strong potential that

not all relevant articles were included. In the file drawer problem, null results are

underrepresented because they are not submitted for publication (Shadish et al., 2002).

This is a particular problem in the research on readiness because so many of the studies

are single organizations case studies and process narratives of a particular innovation. It

is more than likely that many organizations do not submit their internal innovation

change processes to the academic literature. While there was an attempt to gather

unpublished reports through various grey literature databases, the number of change

efforts that had null results is likely to be underreported, and subsequently may influence

the conclusions in this study. However, it is also plausible that a number of positive

findings are not reported and remain internal to an organization. So while there are likely

to be underreported null finding, it is also possible that many positive findings are not

reported.

Methodological Characteristics of the Studies. As noted earlier, a substantial

number of studies (66%) were organizational case studies. As such, these are subject to

many sources of bias. This is not to say that the data that is presented in these studies is

flawed, but rather there is greater potential for other sources of variation that could have

confounded the results. Since so many of studies organizational readiness literature come

from a case study approach, it may be difficult to generalize findings from a single

organization or a single innovation across multiple settings. Therefore, this dissertation

139

should be thought of as an incremental step toward assembling evidence that can improve

the focus of support system activities.

Reported Support Strategies Quality. Related to the above challenge, the

specific techniques of the support systems (e.g., the training or TA model) was described

in adequate detail in very few instances. As noted earlier, the average implementation

quality of support system strategy was 1.28 across all studies (on a scale of 0 – 6),

meaning that most studies did not report important support strategy implementation

characteristics like dosage and adaptation. Because of this, it is difficult to determine

whether or not a particular support strategy in a particular study was better than another

since there was no uniform way in which the strategy was described. The lack of

reporting on quality negatively impacts specific utility of many articles. Even with a

promising finding, it is difficult for another organization to replicate the strategy. Some

possible solutions to address this are discussed in the future directions section.

Lack of measurement standardization: Only 30% of the articles had any sort

of quantitative measurement model. Due to variation of measurement rigor and the

number of constructs involved, it was not possible to synthesize findings in a way that

would lead to estimates of effect size. In a particularly strange example, even though five

studies used the same reporting tool (the AHRQ HSOPSC; Edwards et al., 2008;

Kennerly et al., 2009; Mayer et al., 2011; Schwoebel & Creely, 2010; Thomas & Galla,

2013), they all reported different subscales as they pertained to specific projects. In the

case of Kennerly et al. (2011), readiness outcomes were framed as a snapshot of “key

results.” Although there was the potential to gather effects sizes on organizational culture

and organizational structure, the lack of standardized reporting prevented stronger

140

inferences about the strength of the support system strategy for these subcomponents.

The lack of measurement standardization was more pronounced across other factors and

subcomponents. While there were differences in the characteristics of studies that

reported readiness outcomes, it was not possible to attribute any difference in outcomes

to a small “amount” of a factor or subcomponent.

In addition to the major limitations above, other sources of bias may have been

introduced through reactivity (both coders knew of this dissertation’s hypotheses).

Although the readiness model introduced in this dissertation attempted to be

comprehensive, it is possible that other constructs may influence organizational readiness

are needed. Possible candidates include organization momentum (D. Osher, personal

communication, 7.29.14) and organizational affect (Markle, in prep)

Future Directions:

Although this dissertation provides preliminary evidence that organizational

readiness can be enhanced by Support System activities, there are a number of future

avenues that could continue to enhance Support System effectiveness. First, there is a

need to more fully examine the interrelationships between the factors that influence

motivation and subcomponents of innovation-specific and general capacity in order to

gain a better understanding of how they may interact synergistically or antagonistically.

One possible method is to utilize qualitative comparison analysis (QCA; Kane, Lewis,

Williams, Kahwati, 2014; Ragin, 1999). In QCA, set theory is used to examine necessary

and sufficient components that are related to outcomes of interest. Once items are coded,

the proportion of cases that show outcomes that exhibit a particular component are

reported (Kane et al., 2014; Ragin, 1999). Using this method, certain factors and

141

subcomponents may emerge as being more critical to achieving innovation outcomes than

others. Furthermore, the factors and subcomponents themselves may be coded as

outcomes to determine which factors and subcomponents are necessary and sufficient to

produce changes in them. For example, QCA could be used to examine how

organizational climate is influenced by leadership (Cummings et al., 2012),

organizational culture (Varkey et al., 2008) and champions (Rikli et al., 2009; Sipilä et

al., 2008; Spence & Henderson-Smart, 2011; Wallis & Kennedy, 2013). This may allow

the Support System to focus even further to target specific components first as part of a

comprehensive readiness-building strategy. The data gathered in this dissertation may

allow for such a follow-up analyses.

Second, there is a need to develop standard readiness evaluation models that

incorporate all components of R=MC2. Several new measures like (Shea et al., 2014) that

are adapted from the Weiner model (Weiner, 2009) get at some, but not all of these

constructs. Within the studies that were reviewed for this synthesis, there was negative

correlation between motivation and general capacity and a negative correlation between

innovation-specific and general capacity. Therefore, a standardized framework that

addresses all three of the components is needed. The relatively broad range of definitions

for several subcomponents (e.g. implementation climate supports) highlights the need for

better qualitative and quantitative standardization. Already some evaluation frameworks

are looking to assess organizational readiness for external funding by examining their

responses to Funding Opportunity Announcements (Dymnicki, Wandersman, Osher,

Grigorescu, & Huang, 2014). Similar work is being prepared to address how public

health departments can enhance their readiness to implement quality improvement

142

practices. This work will help to more precisely measure readiness when making

decisions about the how viable and sustainable an organizational change processes might

be. This has the potential to benefit funders by providing more information about where

their investments might be most effective. This will also allow for more detailed analyses

of when a factor or subcomponent might be too low for support system

activities.

When more advanced measurement models are developed, structural equation

modeling can also be used to examine the possible sequencing of the components. For

example, in the relative advantage literature, the use of incentives led to increased use of

the innovation (e.g. Bassett et al., 2013). In this case, it is plausible that trialability can

be a mediating step toward increasing relative advantage. This type of information would

further allow the Support System to be even more targeted in building organizational

readiness.

Thirdly, the quality with which support system strategies are reported can be

improved. Currently, there is an extensive syntheses being prepared that examines the

quality with which technical assistance is being provided (Katz, in prep). There is no

similar reporting framework for training and tools, which is especially troubling since

117 articles included training as some part of a support strategy. While the quality of

QA/QI reporting is improving, there is not a consensus around a reporting framework.

Some frameworks have been proposed like the Standards for QUality Improvement

Reporting Excellence process (SQUIRE; Davidoff, Batalden, Stevens, Ogrinc, &

Mooney, 2008). Similar communities of practice are being built around the use of QI in

public health. When describing lessons learned from how support was provided during

143

the Multi-State Learning Collaborative (seen in Joly et al. (2012), which was included in

the synthesis), Robert Wood Johnson Foundation senior program officer Pamela Russo

said,

One element you didn’t mention was learning by peer exchange. This was a huge

part of building the momentum and showing people models done by their peers,

on public health issues rather than Toyota or health care examples….We have

tried a number of different methods of learning from peers – plenary sessions,

concurrent seminars, small roundtables where people can get into the weeds and

ask questions they’d be shy to ask in a larger group. And that’s why we built

PHQIX.org [an online QI repository]– so that people could continue to learn from

their peers, see the tools they used, the materials they created (templates and

others that they upload) at any time, as most of the field doesn’t manage to go to

Open Forum or…trainings. (Russo, P. 3.28.14, personal communication)

This type of setting could provide a collective template to specify the precise

elements of a QI process. These are critical steps if organizational use of innovations is to

incrementally improve through the use of smaller, iterative interventions like QI.

Improving quality of support system reporting is critical because even when

innovation results are reported, it is currently difficult for others to learn from the support

system on how to improve specific parts of readiness. Application of the EBSIS

framework, which is grounded in the Getting to Outcomes ® process (Wandersman,

Chien, & Katz, 2012; Chinman et al., 2004) provides an additional method of evaluating

support system strategies. A more thorough and standardized methodological framework

will allow the lessons learned from these case studies to be 1) more thoroughly evaluated

144

so their conclusions can be vetted, and 2) more precisely disseminated throughout the

Support System. When this happens, the overall quality of support systems activities will

be improved and consequently the services that the support system provides will have a

higher likelihood of reaching intended outcomes.

Finally, there is opportunity to better explore the use of communities of practice

for building readiness (Donald et al., 2013; Williams et al., 2002). A community of

practice is akin to developing a Support System within the Delivery System to help foster

sustainability (Stirman et al., 2012) and institutionalization (Hall & Hord, 2010).

Communities of practice (or collaborative learning communities, which are similar, or

professional learning communities) can play supportive function that is, in a way,

embedded in the Delivery System. The Support System can help to facilitate the

communities of practice, which can then serve as sustainable sources of support for the

communities. (Katz, J. personal communication, 4.24.14)

This helps to reduce the likelihood that new knowledge will be lost as it is

invested across the network rather than one individual (Donald et al., 2013). However,

there is a danger if communities of practices are used as a complete substitute for

externally delivered support systems strategies, the communities of practices can actually

lead to sharing of poor practices if not done in an evidence-based way. Applying the

EBSIS framework to communities of practice can help to develop and evaluate how these

collectives can be used to continue to foster ongoing organizational readiness for

innovations.

Conclusions:

145

This dissertation provides evidence that readiness (the factors that influence

motivation, innovation-specific capacities, and general capacities) can be enhanced in

organizations. Because of these findings, the Support System can provide targeted

support in order to facilitate positive changes in the readiness of organization. This

dissertation has provided a step in improving how the Support System conceptualizes

possible targets for support. This targeted readiness-building process can help to improve

the ability of all organizations to reach outcomes in the population that they serve. It is

possible for organizations to improve readiness.

146

Students also viewed