Introduction In recent years we have witnsed

Introduction In recent years we have witnessed

In recent years we have witnessed a shift in the innovation landscape of

organizations from closed, to more open models which embrace solutions from the outside.

Widespread use of the internet and web 2.0 technologies have made it easier for organizations to

connect with their clients, service providers, and the public at large, for more collaborative

problem solving and innovation. This open approach to innovation is operationalized through a

number of emergent practices which include open innovation, crowdsourcing, and prize contests.

Though originating as a private sector practice, this open approach to problem solving

and innovation has recently spread to public sector organizations. For example the US federal

government has moved from small scale experimentation with prize contests (referred to as

challenges) in a handful of agencies, to a government-wide policy pronouncement. Challenges

are viewed as a desirable alternative to the commonly used grants and contracts, as they open up

the problem solving space to a wider audience with more diverse capabilities. The US Open

Government initiative introduced in 2009, and the America Competes Reauthorization Act of

2010 (U.S. Congress, 2011), provided legislation and guidelines to institutionalize the use of

challenges in all federal agencies. The General Service Administration (GSA) was tasked with

providing the technology infrastructure (challenge.gov web platform) and institutional support to

agencies.

1.2 Problem Statement

Introduction of this government-wide policy on challenges signalled an unprecedented

commitment by the US federal government to stimulating new types of innovation and creativity

in problem solving. The policy empowered agencies to open up their problem solving space

beyond their regular pool of contractors, and to engage members of the public in finding

solutions to the nation’s most complex problems. Requirements for traditional grant and research

proposals are normally very specific and may limit creativity in both the solicitation and

submissions. With challenges it is easy to specify the desired result without identifying the path

to achieving it, thus enabling the use of novel and nontraditional approach to solutions. The wide

reach of challenges also makes it possible for a problem from one domain to be solved by

someone from a different domain. Therefore talented people who would not normally qualify for

grants and contracts would encounter fewer barriers to participation.

According to (Rogers, 2003) an authority innovation-decision (such as the open

government initiative) should trigger compliance by all agencies, though in reality some may

circumvent the requisite action. Regarding this particular decision made by the White House

Office of Science and Technology Policy (OSTP), there has not been full compliance by all

agencies. Out of the hundreds of executive departments and independent agencies (as listed on

usa.gov), preliminary analysis of listings on the challenge.gov platform revealed that challenges

had not been implemented by the majority of agencies as expected.

After the first year of operation only thirty six (36) departments and agencies had

implemented challenges, with an additional eleven (11) coming on board by the second year.

Further, there were noted disparities in the frequency and manner in which challenges were being

used across agencies. For example, five (5) agencies were responsible for close to half (41%) of

all challenges posted. These were NASA with 17, Environmental Protection Agency with 12,

Health and Human Services 11, Department of Defense 10, and Air Force 10. On the other hand

the majority of agencies (70%) had each posted 3 or less challenges. Thus the rate of

implementation observed in the first two years represents a divergence from the widespread

implementation mandated by the Open Government directive and issued by the Office of

Management and Budget (OMB) in December 2009.

The America Competes Reauthorization Act of 2010 gave broad authority to agencies,

allowing them to run prize competitions to help spur innovation, solve tough problems, and

advance their core missions (White House, 2012). Preliminary empirical analysis of listings on

the challenge.gov platform revealed that challenges were not always being employed for the

intended purpose. Some agencies consistently used challenges to solve complex mission-related

problems requiring highly specialized skills such as designing combat vehicles, energy efficient

light bulbs, and energy efficient homes. On the other hand some agencies focused on creativity

contests (e.g video, poster, photo, and logo contests) pertaining mainly to public service

announcements and producing informational material not requiring highly specialized skills. The

concern was that some uses did not promote innovation as intended by the Open

Government initiative and articulated in the America Competes Reauthorization Act. It could be

argued that some challenge implementations did not innovate, but rather reinforced the status

quo. One visible sign of disparity in the level of complexity and importance to the agency was

revealed in the amount of prize money offered and awarded. Some agencies paid out large sums

of money such as the Department of Energy which offered $27 million for 7 challenges, and the

Department of the Treasury which offered over half a billion dollars for 1 challenge. Others paid

out moderate amounts such as the Navy which offered $2 million for 2 challenges, the

Department of Defense hosted 10 challenges totaling $251,000, and the Corporation for National

and Community Service offered $500,000 for 2 challenges. At the lower end of the scale many

executed smaller amounts under $5000, and the US EPA executed 12 challenges with no

monetary compensation offered.

A further intention of the America Competes Act of 2010 was the institutionalization of

challenges as an added option to more common methods such as grants and contracts. Efforts to

facilitate an enabling environment for more widespread and long term use among agencies can

be enhanced by understanding how challenges are currently being used and by whom. The

development and deployment of policies to support innovation should be informed by an

understanding of critical aspects of the innovation process, best captured using efficient data

collection and analysis of innovation activities (OECD and Eurostat, 2005). Since challenges

represent a relatively new practice, any further development or refinement of policies, legislation

and guidelines should be informed by empirical data collected from adopting agencies. This

study provides an opportunity to get a first hand understanding of the motivations, organizational

practices and conditions which favor challenge adoption.

While questions regarding disparities in adoption rates and patterns have continually been

investigated for various types of organizational innovations, research reviews suggest that they

have not been answered definitely, and further research is recommended (Damanpour &

Schneider, 2006; Tidd, 2001; Wolfe, 1994). Further, these questions need to be answered in the

context of this specific type of organizational innovation (challenges) which is still an emerging

practice.

1.3 Objective and Research Questions

The main objective of this study is to understand and explain how, and under what

conditions challenges are being used by US federal agencies and departments as a tool to

promote innovation. In response to the issues identified in the problem statement, the specific

research questions are:

1. How are federal agencies using challenges, and how do these challenges differ in terms

of the level of innovation involved?

2. How do agencies adopt and implement challenges?

3. How do the various adoption and implementation factors influence the type of enactment

of the adopting agency?

1.4 Methodological Approach

This was an exploratory study aimed at understanding adoption of an emerging

phenomenon in the US Federal Government. Research was conducted using a single case study

of challenge.gov, using data from multiple agencies which were early adopters of the platform.

Prominent concepts from the various streams of literature were used as sensitizing concepts to

guide initial data collection and analysis. Data was collected from two primary sources- archival

data from the challenge.gov web platform, and interviews with challenge managers and

administrators. Related documentation including policy documents, reports, and websites were

used to supplement and corroborate the main data. Data analysis incorporated a combination of

deductive and inductive approach.

1.5 Theoretical Approach

Challenges were introduced as a mechanism to help agencies solve complex problems by

reaching out to a wider audience. In a broad sense, challenges can be viewed as an organizational

innovation because they were introduced for their potential to contribute to the performance or

effectiveness of the organization (Damanpour, 1991; West & Anderson, 1996). The study was

organized around the three research questions.

Research question 1 “How are federal agencies using challenges, and how do these

challenges differ in terms of the level of innovation involved” explored the different ways in

which challenges were being used by adopting agencies. Response to this question involved

qualitative inductive analysis of archival listings on the challenge.gov web platform. The

objective of the analysis was to uncover any discernible patterns in the types of challenges

implemented, the topics they addressed, and the level of innovation invoked. A typology of

challenges emerged following a descriptive and analytical classification process.

Research question 2 “How do agencies adopt and implement challenges” investigated the

process by which agencies adopt and implement challenges. This question drew on multiple

research streams related to the adoption process, organizational innovativeness, and the

distinctiveness of public sector agencies. Research on the adoption process focuses on the

sequence of events related to adoption (Rogers, 2003) and identifies three major stages: i)

preadoption or initiation ii) the adoption decision, and iii) post-adoption or implementation

(Damanpour & Schneider, 2006; Rogers, 2003; Zaltman, Duncan, & Holbek, 1973, Zmud,

1982;). While there is general consensus on the three main stages there is no consensus on the

activities and events which make up each of these stages. Therefore several variations have been

suggested such as: Hage & Aiken, 1970; Klein & Sorra, 1996; Van de Ven, Angle, & Poole,

2000; Zaltman et al., 1973. Also influencing the adoption process, organizational innovativeness

research examines the various determinants which can enhance or hinder an organization’s

innovation efforts (Damanpour, 1991). This research stream is broad and fragmented and lacks

consensus on which factors are the most influential (Crossan & Apaydin, 2010; Read, 2000;

Wolfe, 1994). This study departs from the norm of separating process and innovativeness

research, and combines them in one model to form a more holistic picture of the challenge

adoption process.

While the organizational innovation literature provides the main theoretical foundation

for this study, it did not offer a specific theory or model which directly addressed adoption of

challenges in government agencies. Thus it became necessary to expand the literature search to

address the contextual aspects of the type of innovation and the type of organization. Many of

the prominent models and concepts have been derived around research on technological

innovations (Damanpour & Aravind, 2012; Rogers, 2003; Tornatzky, Fleischer, & Chakrabarti,

1990). Challenges on the other hand represent a mechanism to facilitate problem solving and

innovation. Though administered via a technology platform, challenges cannot strictly be

classified as a technological innovation as defined in the organization innovation literature.

Adoption took place in public sector agencies triggered by a government wide mandate,

prompting the review of two additional literature streams pertaining to open innovation and

public sector organizations. Open innovation covers new mechanisms and approach to problem

solving and includes derivatives such as crowdsourcing, challenges and prize contests. Various

studies highlight factors which influence the adoption process for open innovation mechanisms,

the more prominent of which are: organization commitment and not-invented-here syndrome

(Chesbrough & Crowther, 2006); and the use of intermediaries, and the types of problems

addressed (Brabham, 2008; Brabham, 2009). The public sector innovation literature is

introduced mainly to highlight the different circumstances and conditions that may influence

innovation in public sector organizations. The primary contribution from this literature stream is

organizational bureaucracy and red tape, which if present can hinder the adoption process

(Bozeman & Feeney, 2011; Vigoda-Gadot, Shoham, Schwabsky, & Ruvio, 2005). The more

prominent factors of the various streams of literature are then used to guide data collection and

analysis.

Question 3 “How do the various adoption and implementation factors influence the type

of enactment of the adopting agency” investigates how variations in components of the adoption

process influence the type of innovation enactment. It builds on the model derived from question

2 as well as Orlikowski’s (2000) types of enactment to build a typology of adoption displayed by

different groups of agencies.

1.6 Remainder of Document

This chapter gives an introductory look at the use of challenges in US Federal

government agencies. It outlines the background and motivation for the study, specific research

questions, and the theoretical and methodological approach.

Chapter 2 reviews the extant literature relevant to this study. It starts with organizational

innovation literature followed by contextual literature on newer forms of open innovation, and

touches on public sector organizations. The chapter concludes by highlighting sensitizing

concepts used to guide the study. Chapter 3 traces the background and history of the Open

Government policy and legislation. It also outlines the use of challenges and the challenge.gov

web platform. Chapter 4 outlines the research philosophy and research design including data

collection and analysis strategies. Chapter 5 presents results of the analysis organized according

to the three research questions. Results are discussed in chapter 6 and connections are made with

the research literature. Chapter 7 is the concluding chapter and it contains a summary of

contributions, recommendations, limitations, and directions for future study.

2 Literature Review

This is an exploratory study of the adoption of challenges as a problem solving

mechanism in public sector organizations. Challenges fit within the broad definition of

organization innovations, and relevant portions of the extensive and well documented

organization innovation literature were reviewed. This literature provided theoretical direction

for the study by giving insight into stages of the adoption process and various determinants of

organization innovations. However it did not offer one specific theory or model which could be

used to address adoption of this problem-solving innovation instituted in response to a

government wide mandate.

Challenges fall under the open innovation paradigm where organizations use solutions

from outside the organization to address internal problems. This approach to problem solving

started out as a private sector practice and was later instituted as a policy instrument of the US

federal government. In light of this, the contextual literature on open innovation is introduced,

followed by specific considerations for public sector organizations. These literature streams all

contributed to the early conceptual development of the study.

The chapter starts with the definitions and dimensions of organizational innovation,

followed by the broad and fragmented literature on innovation processes and determinants. This

is followed by the literature on open forms of innovation, followed by specific considerations for

public sector organizations.

2.1 Innovation Research

Over the past few decades innovation research has become increasingly fragmented, as

researchers with varied academic backgrounds employ a wide range of theoretical lenses and

methodologies (Crossan & Apaydin, 2010; Damanpour, 1991; Gopalakrishnan & Damanpour,

1997). All these differences have led to complexity in the conceptualization and measurement of

the phenomenon ( Damanpour & Aravind, 2012). Given the fundamental differences in the

various types of innovation, no single universal innovation theory has emerged (Read, 2000;

Crossan and Apaydin, 2010) and it is inappropriate to assume that a universal theory will fit

across all types (Dewar & Dutton, 1986; Downs & Mohr, 1976; Wolfe, 1994) . As a result,

researchers have found it more productive to focus on specific aspects of innovation such as

processes, dimensions, determinants (Damanpour & Aravind, 2012). This section presents

definitions, dimensions and processes of organizational innovation.

2.1.1 Definitions

Common definitions for innovation describe it as an idea, practice, or object which is

perceived to be new by a particular unit (Dewar & Dutton, 1986; Rogers, 2003; Zaltman et al.,

1973). The innovating unit can be an individual, team, organization (or organizational unit),

industry, or economy (Damanpour and Avarind, 2012). Even if the idea, practice or object may

already be in existence, it is considered an innovation if it is perceived as new by that particular

unit (Gopalakrishnan & Damanpour, 1997).

Innovation can be contrasted with the related term invention, which refers to the first time

the idea, process or product is discovered (Rogers, 2003). While an invention may not

necessarily have any practical application, an innovation is expected to be useful (Read, 2000).

An invention is therefore considered an innovation only after it becomes commercially available

(Rogers, 2003). Invention and innovation of a product or process can occur in quick succession

or with a lengthy time lag (Rogers, 2003).

An organizational innovation can be defined as any new product, process, system (Daft,

1978; Damanpour & Evan, 1984), structure, plan or program (Damanpour, 1991) introduced into

an organization. Organizational innovations are commonly introduced for their potential to

contribute to the performance or effectiveness of the organization (Damanpour, 1991; West &

Anderson, 1996), and may represent tactical or preemptive moves in response to changing

internal and external environmental conditions (Damanpour, 1991).

Therefore challenges fit into the definition of organization innovation, introduced for their

potential to contribute to the problem solving capabilities of the organization.

2.1.2 Dimensions of organizational innovation

The literature makes distinctions between specific dimensions (or types) of organizational

innovation. These dimensions are usually represented as dichotomies such as: technological vs

administrative/managerial, product versus process; radical versus incremental (Dewar & Dutton,

1986; Gopalakrishnan & Damanpour, 1997; Peled, 2001; Read, 2000). The first major

dichotomy distinguishes administrative from technological innovations. The other major

dichotomies fall under the banner of technological innovations: product versus process and

radical versus incremental. These are introduced below.

Administrative innovations are related to the social structure of the organization and

cover organization policies such as recruitment, task structure, resource allocation and rewards

(Daft, 1978). Thus administrative innovations focus on managerial aspects and have an indirect

relationship with the basic work activity of the organization (Crossan & Apaydin, 2010;

Gopalakrishnan & Damanpour, 1997). Other descriptions of non-technical forms of innovation

refer to it as management innovation because they reflect changes in the way the work of

management is carried out. This includes a departure from traditional processes, practices and

techniques (Volberda, Van Den Bosch, & Heij, 2013) and contribute to improving organization

performance (Mol & Birkinshaw, 2009). Technological innovations on the other hand include

products, processes and technologies which contribute directly to the basic work activity of the

organization (Gopalakrishnan and Damanpour, 1997; Crossan and Apaydin, 2010). They include

technologies such as knowledge-derived tools, artifacts, and devices by which people extend and

interact with their environment (Tornatzky, Fleischer, & Chakrabarti, 1990, p11). Technological

innovations play an important role in organizational change by helping organizations change

what they do and how they do it (Tornansky et al, 1990).

Technological innovations can be classified further based on whether they are a terminal

product or part of a process to achieve some further outcome. Product innovations are outputs of

the organization and are an end product for their creators and users. Process innovations are used

to help the organization achieve a further outcome such as improved production or an enhanced

product (Read, 2000; Tornansky and Fleischer,1990). The radical vs incremental dimension

represents the level of new knowledge embedded in a technological process innovation (Dewar

and Dutton, 1986; Read, 2000). It is also an indication of the extent to which the internal

activities and outputs of the organization are changed (Damanpour & Aravind, 2012).

Incremental innovations result in minor improvements while radical innovations involve

fundamental or revolutionary changes in technology (Dewar and Dutton, 1986; Read, 2000).

The two dimensions actually represent a theoretical continuum, but are frequently represented as

a dichotomy due to difficulty in interpretation and measurement of middle values (Dewar &

Dutton, 1986; Hage, 1980).

While challenges may not be an exact match with any of the innovation dimensions

described above, their characteristics most closely match an administrative innovation.

Challenges are primarily a problem-solving mechanism employed to solve a range of problems

which may or may not be directly related to the basic work activity of the organization. Thus

similar to administrative innovations, challenges can be viewed as a means to achieving the basic

work activity, thus the indirect relationship. In line with the description of administrative

innovations above, challenges represent a departure from the traditional practices and techniques

of problem solving, and contribute to improving organizational performance. By contrast,

technological innovations were identified earlier as the products, processes and technologies

which contribute directly to the basic work activity of the organization (Gopalakrishnan and

Damanpour, 1997; Crossan and Apaydin, 2010).

2.1.3 Innovation Processes: Generation, Adoption, Diffusion

This section examines the processes through which innovations enter the organization.

Organizational innovations can be developed internally or acquired from an external

source (Damanpour and Avarind, 2012; Damanpour and Evan, 1984; Lippert & Govindarajulu,

2006). The process of internal development is referred to as generation and involves activities

related to the creation and development of new ideas. These activities include recognizing the

opportunity, research, design, commercial development, marketing, and distribution (Damanpour

and Avarind, 2012; Tornatzky et al., 1990).

The process by which an innovation is introduced into the organization (after it is

generated) is referred to as adoption (Damanpour & Schneider, 2006; Zaltman et al., 1973). This

process starts with knowledge of the innovation and forming an attitude towards it, to a decision

to adopt or reject, and to implementation (Rogers, 2003). Adoption may be directed by a

deliberate choice by management or may be imposed by external conditions (Damanpour and

Schneider, 2006). The innovation can be adopted by the generating organization, or made

available on the market for purchase and adoption by other organizations (Damanpour &

Aravind, 2012; Klein & Sorra, 1996). Adoption results in implementation of a new product,

service, technology or practice (Daft, 1978; Klein & Sorra, 1996) which address specific needs

or problems within the organization (Damanpour & Wischnevsky, 2006). Generation on the other

hand is a creative process which does not necessarily address a specific problem.

Diffusion is another related term which is widely used and researched. Diffusion is

defined by Rogers (2003) as “the process in which an innovation is communicated through

certain channels over time among the members of a social system” (p5). Thus diffusion can only

spread through the organization after adoption has taken place (Crossan & Apaydin, 2010).

While there is a vast body of literature examining the diffusion of innovations by individuals

acting on their own and within organizations, this is outside the scope of this study. The main

focus of this study is on organizational adoption of challenges.

2.1.4 Adoption of Organizational Innovations

Research on the adoption of organizational innovations is wide-ranging and spans many

fields. Two major streams of this literature cover separate aspects of the adoption process-

process theory and variance research (Wolfe, 1994; Hameed, Counsell, & Swift, 2012).

Process Theory focuses on the process by which an innovation is introduced and

eventually used within the organization. It views the innovation process as a series of sequential

steps and investigations try to identify the various stages and their sequence (Rogers, 2003).

Wolfe (1994) identifies two generations of process theory: i) stage model research views

adoption as a series of stages which fall in a particular time and order ii) process research focuses

on theory building and uses longitudinal in-depth qualitative inquiry to get full description of

sequences and conditions of the innovation process. Variance research examines the

determinants which make an organization more or less likely to innovate, also referred to as

organizational innovativeness (OI). This approach identifies the various factors and innovation

attributes which influence patterns of adoption over time (Benbasat,1984; Rogers, 2003; Wolfe,

1994 ).

The stage model approach and variance approach are considered separate research

streams because most studies generally focus on one single approach (Wolfe, 1994). The

intention of this study is not to stay limited to one specific stream, but rather to get a more

complete picture of adoption and implementation by combining approaches. It is expected that

the stage model and process research literature will inform research on the important stages,

while the variance literature will provide direction on factors and conditions which influence the

stages.

2.1.4.1 Stages in the Adoption process

The adoption process is described as the main sequence of events leading up to the actual

use of an innovation within an organization (Rogers, 2003,Frambach & Schillewaert, 2002). The

process begins with activities leading towards the adoption decision followed by the action

needed to put it into continual use (Damanpour, 1991). These activities have been classified in a

variety of stages by different researchers for example: awareness, selection, adoption decision,

implementation, routinization (Klein & Sorra, 1996); knowledge/awareness, evaluation/choice,

adoption, implementation, expansion (Meyer & Goes, 1988); initiation, development,

implementation, termination (Van de Ven et al., 2000); knowledge awareness, attitude formation,

adoption decision , initial implementation, sustained implementation (Zaltman et al., 1973);

evaluation, initiation, implementation, routinization (Hage & Aiken, 1970). Despite the

variations in terminology the significant amount of overlap between models cannot be ignored

(Wolfe, 1994).

Wolfe (1994) summarizes the common pattern among models as follows: the organization

first becomes aware of the innovation, it is matched with a problem or opportunity, followed by

evaluation of costs and benefits, during which forces in favor of or against may try to influence

the decision. The decision is then made to adopt (or reject) followed by implementation.

Following implementation the decision is reviewed and the innovation may either be continued

or discontinued. If continued it becomes routine and is eventually infused into organizational

operations. A more common simplification of the various models group all stages into three

major sub-processes: i) pre-adoption or initiation ii) the adoption decision and iii) post-adoption

or implementation (Rogers, 2003, Damanpour and Schneider, 2006; Zaltman et al., 1973, Zmud,

1982). I will now look at each of these stages below.

Initiation

This first stage in the adoption process consists of activities leading to the adoption

decision. Before a decision is made to adopt a particular innovation, a need is recognized

followed by the search for a solution. At this point organization personnel become aware of the

innovation, assess its suitability, and propose its adoption to others (Meyer & Goes, 1988);

Damanpour and Schneider, 2006). Initiation activities have been identified as problem

recognition, information gathering, formation of attitude, evaluation, and identification of

resources (Damanpour, 1991), or grouped into smaller sub-phases agenda setting, and matching

(Rogers, 2003).

Rogers (2003) describes the agenda-setting and matching stages as follows. During

agenda-setting organizational problems are identified, followed by a search for the appropriate

innovation to address the problem. This is ongoing and may last several months to several years.

While some organizations start by seeking solutions to a problem, the majority opportunistically

scan the environment for new ideas and innovations without having a specific problem in mind.

Thus in many cases organizational adoption of innovations are driven by solutions rather than

problems. During the matching stage the problem is conceptually matched with the innovation to

see whether there is a fit. This stage involves planning, design, and feasibility testing where the

benefits and potential problems are identified. If a mismatch is found between the problem and

solution, the decision makers may choose to abort before actual implementation. If it is identified

as a good fit then the process is given the go-ahead. This stage is critical as there must be a good

fit if the new idea can be sustained over time (Rogers, 2003).

Adoption Decision

Once the innovation is evaluated a decision can be made to adopt or reject. If found to be

strategically, technically, and financially acceptable it is chosen as the desired solution (Meyer

and Goes, 1988). The decision may be influenced by internal factors such as a performance gap,

or may be imposed by external conditions like an environmental change (Damanpour and

Schneider, 2006). The adoption decision is usually (though not always) followed by some form

of organization commitment (Damanpour and Schneider, 2006) accompanied by allocation of

resources for acquisition, alteration and assimilation (Meyer and Goes, 1988).

Rogers (2003:p403) identifies three types of innovation decisions which precede

organizational adoption. Optional decisions are taken when an individual makes a decision to

adopt or reject an innovation independent of decisions by other individuals in the organization.

Collective decisions are made when the adoption decision is reached by consensus among a

group of individuals in the organization. Authority decisions to adopt are made when one

individual or small group of individuals with some form of power, social status or expertise

makes a decision to adopt or reject. In some situations two of more types of adoption decisions

are made in sequence, for example an optional decision is taken by an individual after a

collective or authority decision has been made. These are referred to as contingent decisions.

The open government initiative represented an authority decision by the White House

which instructed executive departments and agencies to immediately adopt challenges as an

innovation tool. This study aims to understand and explain how and why various agencies

responded differently to that authority adoption decision, particularly the subsequent contingent

decision to adopt. Non-adopters are outside the scope of this study.

Implementation

An innovation is not truly adopted until it has been implemented, allowing the intended

objectives to be met (Damanpour & Schneider, 2009). Once the adoption decision is made the

next stages are focused on putting it into use (Zaltman et al, 1973) by organizational personnel,

clients, or customers (Damanpour and Schneider, 2006). Implementation consists of activities

relating to modifications in the innovation itself, or preparations within the organization

including trial use and acceptance (Damapour, 1991, Meyer and Goes, 1988).

Rogers (2003) identified three sequential sub-processes of implementation:

redefining/restructuring, clarifying, and routinizing. During initial implementation there is some

redefining/restructuring of both the innovation and the organization to accommodate each other.

In some cases the innovation is re-invented or modified to suit the needs of the organization. In

other situations changes have to be made to the organization structure to facilitate its use. These

changes may include the establishment of a new organizational unit, new job roles and

procedures. Internally developed innovations are usually familiar to employees and perceived as

compatible with the organization. Externally developed innovations are perceived as more

compatible with the organization if they are flexible and allow significant amounts of reinvention

by employees (Rogers, 2003). As an externally developed innovation, challenges may be more

compatible with some agencies than others based on how much re-invention is undertaken,

signified by the different types of challenges implemented.

During the next stage – clarifying - the meaning of the innovation becomes clearer as it is

used more widely. This allows for the correction of any misunderstandings or unwarranted side

effects. As new arrangements are being made, use of the innovation is becoming stabilized and

more embedded in the organizational structure. Initial implementation is surrounded by

uncertainty around questions such as how it works, who is responsible, who it affects. The

clarifying process provides an opportunity to clear up these uncertainties by allowing employees

to gradually gain a common understanding of the innovation (Rogers, 2003).

Routinizing is the final stage of implementation where the innovation eventually loses its

separate identity and becomes incorporated into regular organizational activities. At this stage a

decision can be made to discontinue or sustain the innovation over a longer term. Some factors

such as a high level of participation by employees or re-invention of the innovation increase the

likelihood of the innovation being sustained by the organization (Rogers, 2003).

Though this outcome is desirable it is not automatic, as the organization has to first

undergo a set of structural changes to accommodate the innovation. According to DeSanctis &

Poole (1994) and Orlikowski (2000) changes may be reflexive in that the new structures give rise

to changes in the innovation as users change habits during recurrent use. If use is ongoing the

process is stabilized and eventually becomes integrated into the operations of the organization.

This reflects one of the original objectives of the Open Government Initiative that challenges

become integrated into regular organizational activities.

Though there is often a major gap between the adoption decision and actual

implementation (Crossan & Apaydin, 2010), an innovation is not truly adopted until it has been

put in use by the adopting organization (Damanpour and Schneider, 2009). Hence use of both

terms adoption and implementation.

2.1.4.2 Patterns of Adoption and Implementation (Enactment)

While the major phases of the adoption process may be similar for the various agencies,

widely differing conditions and contexts within these agencies may lead to different patterns of

adoption and the way the innovation is invoked (or enacted). Orlikowski (2000) investigated

adoption of the same technological innovation in different organizational contexts and found that

varying circumstances and contexts cause users and decision makers to enact it in different ways

leading to different organizational outcomes. The type of enactment was classified according to

the degree of change that was effected in the organization: inertia produced little or no change,

application produced some enhancements but not full change, and change produced a complete

transformation.

In a study of the adoption and use of technological innovations by members of a

university community, (Lin, Singer, & Ha, 2010) found support for the types of enactment

represented in Orlikowski’s (2000) typology. Their findings indicate that different users enacted

the technology differently in different contexts. Inertia was represented by members who made

limited use of the technology, application represented by those who used the technology to

enhance teaching and augment service, while some users made maximum use of the technology

representing the change enactment.

While adopting agencies are all US federal government agencies, they vary widely in

terms of mission, operations and conditions. It is likely that any innovation adopted and

implemented under such varying conditions will be enacted in different ways, leading to varying

levels of change.

2.1.5 Organizational Innovativeness

The focus of the previous section has been the process by which an organization adopts

and implements an innovation. In this section the focus shifts to organizational innovativeness

which examines an organization’s propensity to innovate (Wolfe, 1994) with emphasis on the

various determinants which can enhance or hinder those efforts (Damanpour, 1991). These

determinants may be internal to the organization or may be a part of the larger external system

(Van de Ven, 1993).

Research on the determinants of innovation is extensive and fragmented and lacks

consensus on a theory or on which specific factors are most influential (Wolfe, 1994; Crossman

and Apaydin, 2010). The broad range of commonly studied determinants varies in influence

depending on the context and requirements of the organization (Read, 2000). Tornatzky et al.

(1990) group relevant elements into three main contexts: technology, organization and

environment. They describe the technological context as the internal technology practices and

equipment, as well as new technologies available on the outside. Some important considerations

include the amount of divergence between the new technology, the ability of the new technology

to fit into the existing practices, and the willingness and ability of the organization to learn about

the new technology. The organization context refers to elements which are internal to the

organization and include characteristics such as organization size, structure, human and financial

resources. Other factors related to operations include decision making, communication and

interaction between employees, and mechanisms for monitoring changes in the external

environment. Environmental context refers to the arena outside of the organization which has the

potential to influence the innovation process. Elements of the environment include the industry,

collaborators and competitors, government regulations, and information flow from the outside.

The organization’s capacity to make effective adoption and implementation decisions is affected

by the manner in which it is linked to these environmental factors.

The technological context is not as relevant in this situation as the focus is on the problem

solving and innovation capabilities of challenges rather than the technological aspect. The

environmental context is relevant because of the wide range of external forces, events and trends

in the external environment that can trigger organizational innovation ( Damanpour, 1991; Tang,

1998). More innovative organizations are likely to emerge from an external environment where

the social and cultural norms favor innovation (Tang, 1998). The industry or sector of an

organization can influence its innovativeness (Damanpour, 1991; Van de Ven, 1986) as

innovation is likely to occur where there are necessary and sufficient conditions in the external

environment to support it (Tang, 1998). For example in private enterprise, some firms seek to

innovate as a reaction to pressure of maintaining competitive advantage or recognition within an

industry (Lippert & Govindarajulu, 2006).

Among the three contexts described above, organizational variables are the most widely

studied and are regarded as the primary determinants of organizational innovation (Damanpour,

1987; 1991; Kim, 1980). This includes factors such as the nature of an organization’s task,

vision, mission, and strategy. For example an organization whose primary task is stable and

undemanding is less likely innovate (Tang, 1998). Other prominent organizational factors which

influence innovation include support from management and the availability of organizational

resources (Read, 2000; Spivey, Munson, & Wolcott, 1997). Management plays a crucial role in

setting innovation goals, encouraging initiatives from employees, and making the decision to

approve or reject innovation proposals (Daft, 1978). Other factors such as organization size and

structure have been researched repeatedly and found to influence the level of organizational

innovativeness (Zaltman et al, 1973; Frambach and Schillewaert, 2002). However there are

contradictory indications for organization size as larger organizations are seen as having a greater

need to innovate, while smaller organizations are seen as more flexible and hospitable to

innovation (Frambach and Schillewaert, 2002). Organization structure has also been identified

for both enabling and inhibiting adoption in different studies (Damanpour, 1991). Agencies in

the study vary on several dimensions such as size, structure and mission. With such a wide range

of possible organizational innovativeness factors, it is expected that factors will influence

agencies differently, and that some will be more influential than others.

2.2 Context of Challenge Adoption

Wolfe (1994) describes innovation research as complex and context-sensitive, and urges

researchers to minimize ambiguity by clearly addressing certain aspects of their study. These

include areas such as: the stream of innovation research relevant to the research questions, the

stage of the innovation process that is the focus of the study, the type of organization, and the

attributes of the innovation itself. So far the literature review has outlined the various streams of

the organization innovation literature relevant to this study such as the adoption process and

determinants of innovativeness. However the extant literature, and by extension the literature

review, has not specifically addressed the context of adopting a new problem solving mechanism

as an organizational innovation.

The literature on adoption processes and determinants of innovativeness has been

dominated by studies conducted around technological innovations (e.g. Damanpour and Avarind;

2011; Rogers, 2003; Tornatzky et al., 1990). This includes research on how firms stimulate and

develop new technology, products and services (Crossan and Apaydin, 2010; Volberda et al,

2013). However despite the focus on technological innovation and the product development

process, the literature is applied in all contexts (Damanpour and Avarind, 2012). In a systematic

review of organizational innovation literature spanning a twenty seven (27) year period, Crossan

and Apaydin (2010) found that not enough attention had been paid to the type of innovation

studied. They found that the majority of articles published within that time period (about fifty

percent) spoke of some general or unspecified type of innovation. This means that any theoretical

or empirical differences that may be attributed to the type of innovation remain understudied or

unknown. Of the remaining articles, they found that eighteen percent (18%) referred to

technology (product or process) innovations, and another twenty percent (20%) to product or

service innovation and six percent (6%) to knowledge innovations. There was no evidence that

significant consideration was given to problem solving type of innovations in this literature.

While it is expected that challenge adoption will conform to major theoretical aspects of

the organization innovation literature discussed above, some form of divergence is expected to

account for the unique characteristics of challenges. For example, challenges do not exactly

match any of the dimensions of organization innovation described earlier. It is not an

administrative/management innovation because the focus is not on how the organization is run.

Though administered via a technology platform, challenges differ from the typical products,

processes or technologies normally classified as technological challenges. Rather, they represent

a problem solving mechanism employed to spur further innovation. Technological innovations

are employed for their contribution to the basic work activity of the organization, which is

usually the production of goods and services. Challenges, on the other hand are deployed in

many different ways that do not always relate directly to the organization itself. The introduction

of challenges into government organizations was primarily aimed at promoting research and

development in science and technology areas related to national importance. This research and

development would essentially create an environment conducive to private sector investment in

areas where market forces do not adequately encourage investment (National Academy of

Engineering, 1999). Challenges are sometimes used to motivate scientists and engineers to focus

their efforts on societal goals (Stine, 2009) and to encourage entrepreneurship around new

technologies. Thus in many cases the agency hosting the challenge is not the primary beneficiary.

A distinct feature of challenges is the practice of embracing outside contributions in

solving problems, which overlap with extant practices like prize contests and newer practices like

crowdsourcing and open innovation. According to Wolfe (1994) different types of innovation

employ distinct innovation processes, and the determinants of adoption differ as the

characteristics of the innovations differ. This new trend of opening up the problem solving space

is an emergent practice in organizations, in contrast to the types of organization innovations

typically studied. The differences in approach are distinct enough to warrant a closer look at the

literature on open innovation and the relevant related concepts introduced in the sections below.

The following section helps understand the context of the more open forms of innovation.

The section starts by defining prizes and contests, as well as details of their background and

benefits. This is followed by the definition and explanation of the related crowdsourcing

concept, followed by definition and review of the more general open innovation literature,

focusing on adoption determinants.

2.2.1 Prizes and Contests

The definition of prizes and contests below were taken from the National Academy of

Engineering report (1999) commissioned from the White House, and which represents the

official start of a national dialog on the use of prizes in the federal government. Prizes are

rewards in the form of cash or non-monetary compensation given to individuals or groups for a

particular behaviour or achievement. Recognition prizes acknowledge some accomplishment that

has already occurred such as the highly regarded Nobel Prizes (National Academy of

Engineering, 1999). Inducement or incentive prizes offer cash or other type of reward as an

incentive to get people or groups to meet a specified goal. Contests refer to the competition

through which prizes are awarded. Both recognition and inducement prizes are executed through

contests where a number of submissions are judged according to specific criteria, with the prize

being awarded to the winner (National Academy of Engineering, 1999).

The term challenge is used to represent prize contests being adopted by federal agencies,

where members of the public are challenged to help solve a problem in exchange for some

reward. Challenges are hosted on the public web-based platform challenge.gov.

2.2.1.1 Background of Prize Contests

Prize contests have been used throughout history to push the boundaries of technological

discovery and innovation. For example the modern day canning industry can be traced to

enhancements in food preservation techniques dating back to the 1700s (Stine, 2009). At that

time the French government offered a prize of 12,000 francs to anyone who could develop

technologies to advance food preservation for military troops. Modern chemistry as we know it

today is the result of a prize contest launched by the French Academy over 200 years ago when a

prize of 100,000 francs was offered for the production of soda alkali from sea salt (Kalil, 2006).

The often-cited Longitude Prize dates back to 1714 and was launched by the British Parliament.

It led to the invention of the marine chronometer which solved the problem of measuring

longitude at sea (McKinsey & Company, 2009).

In more recent times the Ansari X-Prize identified as the largest prize in history led to the

building and launching of a spacecraft by a private team based on certain predefined

specifications. The prize launched in 1996 was awarded in October 2004 and led to the

formation of a new industry by opening up the spaceflight industry to a non-government entity

for the first time in history (X PRIZE Foundation, 2011). While many of the earlier prizes focus

on advancing technology for the benefit of society, some of the more recent ones launched by

private companies aim at advancing internal operations to boost profits. For example the widely

discussed Netflix Prize launched in 2006 offered a prize purse of $1 million to any individual or

team who could design an algorithm to increase the accuracy of the company’s personalized

movie recommendation (NetFlix.com, n.d). The movie recommendation system relied on movie

ratings from Netflix viewers to predict and recommend future movie choices. After receiving

entries from more than 40,000 teams in 186 countries, the prize was awarded in 2009 to a seven

member team BellKor's Pragmatic Chaos representing four countries. The use of the innovative

algorithm has allowed Netflix to stay competitive in a highly volatile and fast changing market

segment by reaching out to innovators such as mathematicians and statisticians who were

previously not part of the Netflix’s own R&D department (Netflix.com, n.d.).

2.2.1.2 Prize Types

Different prize types present opportunities for different approaches to stimulating

innovation. Using research in the philanthropic sector, McKinsey and Company (2009) identified

seven different change levers representing ways in which prizes could produce innovation and

change. The change levers identified are: (i) identify excellence (ii) influence public perception

(iii) focus communities on specific problems (iv) mobilize new talent (v) strengthen

problemsolving communities (vi) educating individuals and (vii) mobilizing capital. Varying

combinations of these change levers result in six (6) different prize archetypes based on how they

are combined. The six prize archetypes and associated change levers are:

•Exemplar prizes define excellence and focus attention in a particular area by giving

recognition and showing appreciation to accomplishments. The publicity associated with

these prizes also serves to influence public opinion and set an agenda within the

discipline. The change levers are identifying excellence and influencing perception.

•Exposition prizes draw attention to a broad list of potential ideas and select the best

among them. They influence change by identifying excellence and mobilizing capital.

•Network prizes aim to create and strengthen networks among communities around a

particular theme by fostering interaction among stakeholders. Network prizes influence

change by identifying excellence, strengthening community, and mobilizing capital.

•Participation prizes aim to get broad participation with the goal of influencing

participants to make behavior and lifestyle changes. These prizes attempt to influence

change by strengthening community and educating and improving skills.

•Market stimulation prizes aim to counter market forces that prevent the achievement of a

desirable social outcome. They work to mobilize previously unidentified talent, drive

costs down by stimulating competition and exposing latent demand. Market stimulation

prizes seek to effect change through four change levers: identifying excellence,

mobilizing talent and capital, focusing a community, and influencing perception.

•Point solution prizes focus on finding a solution to a well-defined problem which

requires some form of innovation. Companies such as InnoCentive serve as

intermediaries for client companies seeking solutions to point solution prizes.

Intermediaries use open innovation platforms to pose the problems to their solver

community. Point solution prizes seek to influence change by focusing a community and

mobilizing talent.

These six (6) prize types were also highlighted in the 2010 memo entitled “Guidance on

the Use of Challenges and Prizes to Promote Open Government” from the Office of Management

and Budget (OMB) to heads of executive departments and agencies. Agencies considering the

use of challenges were reminded of the broader aim of spurring change, and encouraged to

consider which of the six types would best accomplish that goal (White House, 2010).

2.2.1.3 Prize Contests to stimulate innovation

In the private sector, recognition and inducement prizes are widely used to promote

breakthrough in a given field and to reward advances or revolutionary development in traditional

and nontraditional thinking (Schroeder, 2004). While many of the earlier prize contests in

history were privately sponsored, recent trends show an increase in the use of prizes in

government. This trend is triggered by a need for creative thinking among government suppliers

(Stallbaumer, 2006) and indicates an increasing belief in government agencies of the

effectiveness of prizes in driving innovation (McKinsey & Company, 2009).

According to the NAE (1999) report, the introduction of prizes in US federal agencies

was primarily aimed at stimulating private sector investment in R&D and innovation in areas of

national importance. These areas include safety, energy efficiency, and public health, where

market forces do not adequately encourage investment. A related objective was to motivate

scientists and engineers to focus their efforts on societal goals not yet reached (Stine, 2009) and

inspire participation from those who may not otherwise be inclined to participate (Stallbaumer,

2006).

2.2.1.4 Benefits of Prizes and Contests

Traditionally the US federal government showed a preference for using grants,

competitively-bid contracts, and the patent system to pay for innovation (Mckinsey & Company,

2009). Prizes offer an alternative to these traditional methods and other investments which may

otherwise be viewed as too risky or costly (Stine, 2009).

Prizes stimulate creativity and unconventional approaches to innovation by allowing the

government to set a goal for technological advancement without dictating the method (National

Academy of Engineering, 1999). They also play a very important role in public outreach and

education by drawing attention to the societal challenge being tackled, and the potential role of

science and technology in addressing the challenge (National Academy of Engineering, 1999).

An additional benefit of the public outreach is that it may generate enthusiasm around science

and technology related skills and entrepreneurship opportunities and encourage more young

people to pursue careers in that field (Kalil, 2006). While highlighting the importance of prizes in

motivating science and technology innovation, Stallbaumer (2006) cautioned that their use

should not conflict with private sector efforts and that advancement of some new technologies

are best left to private enterprise. By the same token government intervention is welcomed if the

private sector has fallen short.

2.2.2 Crowdsourcing

Crowdsourcing is a term used to refer to the practice of outsourcing to crowds. Jeff

Howe (2006) is widely credited with coining the term and the following widely used definition:

Crowdsourcing is the act of taking a job traditionally performed by a designated

agent (usually an employee) and outsourcing it to an undefined, generally large

group of people in the form of an open call (Howe, 2006).

Crowdsourcing involves a diverse set of practices from contests where individual winning

entries are selected and rewarded, to pooling of individual contributions to contribute to a single

solution (Schenk & Guittard, 2009). It is used in various types of situations such as: i)

inexpensive completion of routine tasks by members of the crowd acting either independently or

collectively ii) to provide a number of possible solutions to complex problems iii) for creative

tasks modelled after offline poster and logo design contests Schenk and Guittard (2009).

Crowdsourcing can also be used for idea generation where a search is broadcast to the crowd for

new ideas (Hennala, Parjanen, & Uotila, 2011) or to gather user feedback for development or

improvement of products and services (Poetz & Schreier, 2011).

Crowdsourcing is more than just the latest web 2.0 fad, but rather can be viewed as a

strategic model capable of producing superior solutions by reaching out to motivated members of

the public (Brabham, 2008; Terwiesch & Xu, 2008). While the practice may not be new, the

interactive nature of the internet and web 2.0 tools provides the organization with free and easy

access to the crowd (Schenk & Guittard, 2011). Organizations make the details of their problems

publicly available and invite potential solvers to submit solutions (Jeppesen & Lakhani, 2010).

Thus rather than contacting known experts, the call goes out to anonymous individuals who are

usually amateurs (Schenk & Guittard, 2009).

One of the major benefits of this practice is the opportunity to get ideas and solutions

from individuals with diverse views and experiences, thus presenting a different perspective from

employees (Brabham, 2008). Crowdsourcing allows firms with limited budgets to get access a

wide range of skills at reasonable cost (Whitla, 2009) as the crowd labour is often worth more

than payments for winning solutions (Brabham, 2008).

One commonly cited example of the early successes of open innovation and

crowdsourcing involves the company InnoCentive which played the role of knowledge broker to

firms in science and high-technology industries. Lakhani & Panetta (2007) provide a

description of the business model. InnoCentive provides client firms with a forum for posting

difficult science-related problems for which an in-house search for a solution may be too

expensive or time consuming. Cash prizes ranging from $5,000 to $100,000 were usually offered

for an acceptable solution. Potential solutions were submitted by members of InnoCentive’s

solver network which consisted of over 120,000 scientists from countries around the world. In

return for the prize money the solver with the winning solution gave up all intellectual property

rights, and the solution became the property of the seeker firm.

Based on the definition and uses stated above, prize contests (Challenges) can be viewed

as one form of crowdsourcing where members of the crowd submit individual entries and

compete to win a prize. There are other forms of crowdsourcing where members of the public

jointly collaborate or contribute to the end product, and there is no search for a ‘winning

solution’. This type of crowdsourcing is not considered challenges or prize contests.

Crowdsourcing is part of a growing trend in which organizations are opening up their innovation

processes to embrace ideas and solutions from external sources broadly referred to as open

innovation. This topic is introduced next.

2.2.3 Open Innovation

The open innovation concept was first proposed by Henry Chesbrough in his 2003 book

entitled “Open Innovation: The New Imperative for Creating and Profiting from Technology”.

Since then the topic has generated much interest, discussion and research. The most cited

definition given by Chesbrough (2006, p1) describes it as:

the use of purposive inflows and outflows of knowledge to accelerate internal innovation

and expand the markets for external use of innovation

Sometimes an organization encounters a problem which cannot be solved internally for

various reasons related to knowledge or capacity and needs to look outside for possible solutions

(Jeppesen & Lakhani, 2010). Some organizations lack internal capacity to develop and

commercialize all internal discoveries and are forced to shelve spillover ideas from their

Research and Development (R&D) efforts (Chesbrough & Crowther, 2006). Thus openness in

innovation refers to the practice of embracing outside participation as well as sharing with others

on the outside (Chesbrough, 2011).

The three distinct approaches to open innovation based on the core processes are: i)

outside-in where companies utilize external ideas and technologies from suppliers, customers and

other partners to enrich their own operations and production (Gassmann & Enkel, 2004) ii)

inside-out where companies market their internally developed ideas and technologies to the

outside, for example amazon.com offering its online retailing technology to other large vendors

for a fee. iii) co-creation which combines the outside-in with inside-out processes, and use joint

operations to develop and commercialize innovation (Enkel, Gassmann, & Chesbrough, 2009;

Gassmann & Enkel, 2004). The internet allows the easy flow of information and the connection

of widely dispersed experts and enthusiasts (Chesbrough, 2003).

The most common definitions of open innovation reveal an emphasis on outside-in

process which involves the opening up of organizational boundaries to knowledge in-flows from

the outside (Enkel et al., 2009; West & Gallagher, 2006). Prize contests and crowdsourcing

practices as described above fall under the outside-in category where solutions are solicited from

outside the organization to solve internal problems.

2.2.3.1 Benefits of Open Innovation

Both research and practice have labelled this new approach of opening up to outsiders as

beneficial to solving problems. Empirical data suggests that the primary driver of open

innovation in private firms goes beyond the mere desire to adopt a new paradigm, and is targeted

at growth in new products and revenue (Chesbrough & Crowther, 2006). When used in the new

product development process the innovating company can receive input directly from its

intended audience in the early stages of idea generation and design, translating into innovation at

a faster rate and with lower risk (Chakravorti, 2010).

In an empirical study in the scientific community Lakhani, Jeppesen, Lohse, & Panetta,

(2007) found that open innovation led to the successful resolution of one-third of problems which

were previously unsolved by experts within Research and Development departments. When

comparing ideas for new products generated by professional employees of a firm versus those

submitted by users in an idea generation contest, Poetz & Schreier (2011) found that the user

submitted ideas had significantly higher scores on novelty and customer benefit. While this

approach has proven effective in improving innovation outcomes in a number of cases, Lakhani

and Panetta (2007) caution that traditional organizations should not view it as a ‘silver bullet’

solution to all innovation problems but rather as a supplement to their on-going innovation

strategies.

2.2.3.2 Factors influencing adoption of Open Innovation Practices

Companies which decide to invest in open innovation are sometimes hindered from

profiting from these activities due to certain risks and barriers which they face (Enkel et al,

2009). Thus despite the stories of success and benefits being achieved by some implementing

firms, the open innovation process still proves to be extremely challenging for many others. Also

there is still uncertainty surrounding the long term sustainability of these new types of initiatives

(Chesbrough & Appleyard, 2007).

In a study of Small and Medium Enterprises (SMEs), Van de Vrande, De Jong,

Vanhaverbeke, & De Rochemont (2009) found a diverse set of organizational, managerial and

cultural issues brought on by interaction with external partners. An empirical investigation

conducted in a range of industries outside the high technology sector (Chesbrough and Crowther,

2006) sought to document practices which provided favourable conditions for adopting open

innovation practices within an organization. They found a critical requirement was the

introduction of highly focused practices which were in alignment with overall objectives of the

business, and identified two critical adoption challenges: i) the not invented here (NIH)

syndrome, and ii) sustaining internal commitment long enough to realize the benefits of open

innovation.

In a detailed assessment of a ‘successful’ implementation of open innovation at Proctor

and Gamble, Huston & Sakkab (2006) provided a first-hand account of some of the changes

experienced. By their account, the company had to cultivate an internal culture of change

different from the existing centralized and internally focused culture. This involved promoting

the exchange of ideas emanating both within and outside the organization. Another practice that

was adopted involved a reward structure to recognize employees involved in the development of

a successful product, whether the ideas came from inside or outside. One of the reasons given

for providing rewards was to force change in the culture of resistance and “not invented here”

syndrome. They report that initial resistance was centred on the belief that the importation of

external ideas would somehow reduce the need for internal ideas and that some people would

lose their jobs. On the contrary, there was a need for new skills, and the practice has now been

generally embraced despite a few pockets of resistance. The report underscores the importance

of top management support, including explicit declaration as a company strategy from the CEO.

They claim that any initiative is destined to fail if it is seen merely as an isolated trial handled by

a selected minority.

2.2.4 Gaps in Open innovation Research

The emerging research on open innovation and crowdsourcing focuses mainly on the use

of these concepts in private business firms, particularly in the manufacturing and production

sectors. Other studies found that open innovation concepts could be suitably applied in a broader

context outside of the high tech manufacturing industry (e.g Chesbrough and Crowther, 2006).

Open innovation concepts can be readily applied to the provision of services as it allows

companies the opportunity to work closely with customers in the development of new solutions

(Chesbrough, 2011). There is also a strong and increasing incidence of open innovation in Small

and Medium Enterprises (SMEs) and service firms (Van de Vrande et al., 2009). While there is

increasing utilization of open innovation in many firms (Enkel et al., 2009; Huizingh, 2010) there

are shortfalls in the knowledge and understanding of organization-related mechanisms to

implement open innovation, as well as a lack of understanding of when to adopt and how to

receive maximum value from it (Enkel et al., 2009). As far as the various forms of open

innovation are concerned, it is important to understand the processes and conditions which best

facilitate their adoption into various organization types, including the public sector.

Many of the prominent prize contests in history outlined in the literature (e.g. Stine, 2009,

Kalil, 2006, McKinsey & Company, 2009; X PRIZE Foundation, 2011) as well as the more

recent examples like Netflix (recounted above) were privately sponsored events aimed at solving

one specific problem, and not as a regular organizational occurrence. Other prominent work on

crowdsourcing (e.g. Lakhani and Panetta, 2007), examine the work of the popular innovation

broker Innocentive in facilitating crowdsourcing for different companies in the high technology

industry. The focus of this work is on ideal features of prizes rather than as an organizational

innovation. With the exception of a handful of studies (e.g. Chesbrough & Crowther, 2006;

Huston & Sakkab, 2006) the research literature has not adequately addressed topics such as the

organizational conditions and processes necessary to facilitate and institutionalize the adoption of

these emergent forms of innovation. Therefore this study can bridge the gap between research on

organization innovation and newer forms of open innovation.

2.3 Organizational Context

This study examines the adoption of a practice which started in the private sector and

later spread to public sector institutions. Historically the greater emphasis of innovation research

has been in the private sector (Damanpour & Schneider, 2009). A significant proportion of this

innovation literature emanates from theories of new product development and technological

innovation (Hartley, 2005). Similar to the private sector, the public sector innovation literature is

fragmented and lacking consensus on general theories and concepts (de Vries, Bekkers, &

Tummers, 2014). Vigoda-Gadot et al (2005) express support the application of existing

knowledge from the private business sector to inform public sector innovation research, but

should also be informed by the public management doctrine.

The intention for this study is to use concepts from the more established and better

developed organization innovation literature to inform the study on adoption in public sector,

while also acknowledging and making allowances for differences which may influence variation

in innovation activities. While the type of organization is not the focus of the study, it is

important to examine prominent differences which may have an impact on the findings.

Comparing the two sectors, the most commonly mentioned difference is the underlying

goal of market viability and profit in private institutions versus the multiple, often intangible and

sometimes conflicting goals of the public sector (Caudle, Gorr, & Newcomer, 1991). In the

private sector successful innovation is highly desirable as a means of ensuring competitiveness

(Hartley, 2005; Vigoda-Gadot et al, 2005). In the public sector innovation is viewed as

justifiable only when it is seen to increase public value in the efficiency and quality of

governance and services (Hartley, 2005, De Vries, H.A., Bekkers, V.J.J.M., Tummers, L.G.,

2014).

According to Potts & Kastelle (2010) there is a structural disconnect between the political

mandate to introduce new initiatives and the subsequent development and implementation by

employees whose general approach may be less than enthusiastic. They contrast this with private

sector innovation where incentives are aligned so that those backing a new idea receive

compensation if it succeeds but bear the risk if it is a failure. Additionally, public sector

leadership may display some aversion to getting involved in complex innovation projects

(Vigoda-Gadot et al, 2005).

Public sector agencies are often described as bureaucratic and conservative with the

tendency to follow strict rules and methods with which they are familiar (Vigoda-Gadot et al,

2005). However in recent decades, some movement away from classic bureaucratic structures

has made them somewhat more receptive to the introduction of innovative ideas (Vigoda-Gadot

et al., 2005). There are increasingly more incentives to innovate at all levels of the public sector

driven largely by public and media expectations for improved quality of services, combined with

the pressure of tightening public finances (Clark, Good, & Simmonds, 2008).

In a survey of public sector innovators the most commonly reported situations that led to

their innovations were identified as: the political system, new leadership, crisis situation, internal

problems, and new opportunities (Borins, 2001). Of these the most frequently identified

innovation trigger was internal problems which included conditions such as resource constraints,

and inability to coordinate some policy or meet demand for a program (Borins, 2001). Other

reported barriers to innovation include the lack of incentives to innovate, lack of funding, and the

need for public support (Damanpour & Schneider, 2009). While opening up to the outside is an

exciting prospect, its use may be limited to particular circumstances. For example there is a need

for secrecy in certain sectors like the intelligence community where public articulation of a need

may be a sign or admission of vulnerability (Doney, 2009).

In a recent review of public sector innovation research, De Vries et al (2014)

recommended further research in some specific areas which they found inadequate. Two of these

areas are directly relevant to this study and have potential to influence the findings. These are i)

recognizing importance of the environmental context which includes effects of an external push

such as policy and political pressures and ii) identifying linkages and interrelationships between

the various influential factors.

The factors mentioned in this section highlight pertinent considerations for adoption by

public sector organizations, though it also does not specifically address newer forms of

innovation. For this study three major streams of literature were reviewed: organizational

innovation, open innovation, and public sector innovation. Since no one theory or framework

provided the ideal fit, prominent concepts from the various streams were combined to provide

conceptual direction for this study. These concepts are discussed in the next section along with

reasons for their selection.

2.4 Conceptual Framework

The focus of this study is to understand and explain the adoption of a newer form of

organizational innovation in public sector agencies. From the literature review it was still not

clear which of the wide range of innovation concepts and models would offer the most accurate

explanation for adoption and implementation in the particular context being studied. Further,

because of the emerging nature of the phenomenon my objective was not to test existing

concepts, hypothesis, and theories derived from other contexts, but rather to inductively build a

framework which was truly representative (Merriam, 2009). Therefore I used an exploratory

approach which acknowledged the potential contributions from the literature while leaving room

for emergence of more relevant explanations from the data.

Sensitizing concepts are often used in qualitative research to orient data collection and

analysis (Patton, 2002). While the inductive nature of qualitative research encourages us to be

open to learning as much as we can about the phenomenon, the use of sensitizing concepts helps

ensure that we do not enter the field with a completely blank slate (Patton, 2002). This is also in

line with advice from Eisenhardt’s (1989) and Yin (2009) concerning case research where they

advocate using existing literature to help frame the research problem and questions, and identify

some initial concepts and variables to guide data collection.

Following this advice I identified six (6) sensitizing concepts based on their prominence

in the literature, and from observations and preliminary analysis of listings on the Challenge.gov

web platform. I used these concepts at the beginning of the study to build a conceptual

framework and to guide data collection and analysis. The original objectives of the study were to

identify the enabling and hindering conditions for challenge implementation as well as factors

influencing the adoption decision. At that point I focused around the contextual literature on open

innovation and crowdsourcing, reflected in the limited focus of the sensitizing concepts. Due to

the iterative nature of qualitative research, additional components of the implementation process

and different patterns of adoption started to emerge as the study progressed. This presented an

opportunity to adjust the research questions and expand the literature reviewed. Despite the

expanded objectives, the original propositions are still critical as they help trace the conceptual

development of the study and origin of the interview instrument. I present the initial sensitizing

concepts below along with reasons for their selection.

2.4.1 Management support

As stated earlier, the organizational innovation literature is extensive and fragmented with

inconsistent findings. Notwithstanding these inconsistencies, the importance of management in

leading and influencing innovation activities remains the most commonly identified factor. For

example Read (2000) identified management support for innovation and an innovative culture as

the most important determinant. Damanpour (1991) identified attitudes of top managers and

their willingness to take risks and contribute to the successful implementation as critical.

Mumford & Licuanan (2004) highlighted the importance of leadership guidance and support in

creating conditions conducive to innovation, and Vigoda-Gadot et al. (2005) found leadership

and vision to be an important determinant for public sector organizations in Europe. While there

seems to be consensus on the role of management, the level and type of support may not always

be equal. Dewar and Dutton (1986) highlight the important role of senior executives though

indicating that they may or may not embrace change. Crossan and Apaydin (2010) underscore

the importance of leadership from all levels of the organization to lead as well as maintain

momentum. The type and level of support from management is expected to have an important

impact on the adoption process.

2.4.2 Employee attitudes (Not-Invented-Here Syndrome)

This concept was selected for its prominence in the open innovation literature. Regarding

attitudes to inputs from open innovation, the Not-Invented-Here (NIH) syndrome is identified as

a major influential factor (Chesbrough & Crowther, 2006; Chesbrough, 2011; Huston & Sakkab,

2006). This concept imported from the research and development literature refers to internal

resistance to outside knowledge. In describing the build-up to Proctor and Gamble’s successful

open innovation practices, Huston and Sakkab (2006) refer to the realization that “massive

operational changes” were needed to shift the company’s attitude to innovations from resistance

to enthusiasm, from “not-invented-here” to “proudly-found-elsewhere”. Companies in the study

reported they were able to overcome NIH and obtain buy-in from employees through articulating

reasons why internal efforts were not adequately meeting objectives. Through the process they

were also able to create a greater level of organizational commitment and alignment to the open

innovation approach. Like other open innovation practices, challenges solicit contributions from

outside the organization and efforts may be hindered if the NIH syndrome is present.

2.4.3 Organization commitment

Organization commitment is a broad term used by Chesbrough and Crowther (2006) in

the open innovation literature to refer to the level of commitment displayed by the organization.

The level of commitment was found to be a significant determinant and was evidenced by the

presence of enabling factors such as resources and modified processes to promote innovation. In

their study of open innovation in European Small and Medium Enterprises, Enkel et al., (2009)

found that one of the critical challenges encountered was the sustenance of internal commitment

long enough until the benefits were realized. Some indicators of this commitment were the

allocation of dedicated human and financial resources, and fostering a balance between open

innovation activities and regular business. Chesbrough and Crowther (2006) identified internal

commitment as one of the critical factors in how organizations ‘adopted, deployed and

encouraged’ open innovation. Their indicators of commitment include: senior management

support, funding, innovation champions, revised internal processes, metrics and incentives.

Therefore it is likely that the presence of organization commitment will have a positive influence

on the adoption and implementation process of challenges.

2.4.4 Use of intermediaries

From the open innovation and crowdsourcing literature Terwiesch & Xu (2008) used the

example of pioneer crowdsourcing intermediary InnoCentive to describe the role of an

intermediary in executing contests for its clients. Potential solvers register with the intermediary

indicating their area of specialization (or interest). The intermediary works with the seeker

organization to help formulate a description of the problem and rules of the contest, including

considerations for ownership or transfer of intellectual property rights. The problem description

is then made available to the pool of solvers, some of whom then attempt the problem with a

(usually) smaller number submitting solutions. These solutions are then forwarded to the seeker

who makes a judgment on suitability, and makes arrangements for advertised rewards where

appropriate. Huizingh (2010) suggests that both smaller firms and larger organizations can

benefit from using intermediaries, especially when engaging in the essential but time-consuming

tasks related to locating and establishing partnerships for outbound open innovation. Chang &

Kannan (2008) highlight the rising engagement of third-party firms to act as intermediaries

between government departments trying to reach out to citizens using various forms of new

technologies.

Preliminary analysis of the challenge.gov website revealed that among agencies

producing the largest number of challenges, over sixty percent (60%) had their challenges hosted

on third party sites such as Innocentive, TopCoder, Health 2.0. This is an indication that the

challenges were managed by intermediary organizations hired or working in partnership with

adopting agencies. Due to the extra duties involved in the adoption and implementation of

challenges, it is projected that the use of intermediary organizations will have a positive influence

on the adoption process.

2.4.5 Nature of problem faced

While the promises of open innovation and crowdsourcing practices seem appealing, the

process is not a good fit with all types of problems (Terwiesch and Xu, 2008). Problems may

vary in complexity (e.g simple, complex, creative) (Schenk and Guittard, 2009) and further

research can help determine which types of ventures succeed and which ones fail (Brabham,

2008). For example, a problem which can be clearly defined and all the data provided can be

easily outsourced (Brabham, 2009). One of the earliest and most prominent observations made

from preliminary analysis of the challenge.gov platform was the variation in challenge types

from basic public service announcements to highly complex scientific and technological

problems. Therefore it is important to examine how those variations influence the likelihood of

adoption as well as the nature of the adoption process.

2.4.6 Organizational red-tape

Organizational red-tape refers to rules that create unnecessary compliance burdens for the

organization without contributing to functional objectives (Bozeman & Feeney, 2011). This is

especially relevant for public organizations as they tend to be more bureaucratic, in direct

contrast to the more flexible and dynamic nature of private organizations which can more easily

facilitate innovation (Vigoda-Gadot et al., 2005). The authority decision to adopt challenges as a

government-wide practice had the underlying assumption that this practice could easily be

transferred from private to public sector agencies. It is a common perception that bureaucracy

and red-tape in public sector agencies act as a hindrance to innovation efforts.

In contrast to common perceptions, Pandey and Bretschneider (1997) found that

organizational red-tape had differentiated effects on different organizations, producing different

dynamics and outcomes. Thus while high levels of red tape generally acted as a hindrance to

innovation adoption, Pandey and Bretschneider (1997) found that high levels of red-tape could

actually motivate organizations to adopt innovations as a way of overcoming the red-tape. While

it is expected that red-tape will have some effect on challenge adoption, the extent and level of

this effect are not known at this point.

Having examined the various literature streams and highlighting the conceptual direction

for this study, the focus now shifts to the research setting.

3 The Research Setting

This chapter traces the background of the Open Government Initiative and the

introduction of challenges in federal agencies.

3.1 Overview

Challenge.gov is a web-based platform that enables U.S. federal agencies to post

challenges and invite members of the public to submit solutions. The platform is administered

by the U.S. General Services Administration (GSA) though their Office of Citizen Services and

Innovative Technologies, in partnership with the developer ChallengePost (GSA, 2011a). While

the platform is administered by the GSA, implementing departments and agencies bear full

responsibility for all activities related to the planning, formulation, execution, and posting to the

web platform. GSA provides guidance and policy support while the provider ChallengePost

provides technical support.

3.2 Background

As far back as 1999 the National Academy of Engineering (NAE) acting on a request

from the President’s National Economic Council began a national dialogue to explore the

possible use of prizes as a way to spur technological innovation to benefit society as a whole.

This dialogue was held in the form of a one and a half day workshop hosted by the NAE and

sponsored by the National Science Foundation (NSF). The event brought together experts from

academia, industry and government with the objective of building a knowledgebase to guide

policy, and increase public understanding on the use of prizes to promote innovation.

The NAE (1999) report identified four different ways in which federally funded

inducement prize contests can be used to promote technological advancement: 1) new or best

inventions awarded for the invention of a new technology or technique based on a specified

technical objective 2) new applications awards offered to anyone who can extend an existing

technology to meet a new objective 3) performance improvement awards offered for improving

the performance of an existing product 4) technology diffusion offered to those who help a new

technology spread in the marketplace (e.g. through sales or production activities).

One of the recommendations coming out of the NAE led workshop was that limited

experiments should be encouraged among federal departments that sponsor research and

technology development in science and engineering (NAE, 1999). Based on this

recommendation, Congress authorized limited use of Prizes in 2003 (Stine, 2009), and pilot

programs were set up in a small number of science and technology oriented departments and

agencies.

The first prize contest referred to as the Defense Advanced Research Projects Agency

(DARPA) Grand Challenge was launched in 2004 under authorization of the Bob Stump

National Defense Authorization Act for Fiscal Year 2003 (H.R. 4546, Sec. 2374b) (Stine, 2009).

The focus of this Grand Challenge was to advance research and development in unmanned

combat vehicles that could navigate intelligently and save American lives on the battlefield. A $1

million dollar prize was offered to the individual or team who could conduct a successful field

test under prescribed conditions. The price went unclaimed as none of the 15 finalists were able

to get their vehicles over the difficult desert route. In 2005 the second DARPA Grand Challenge

was hosted with the same objective of designing and testing an unmanned combat vehicle, this

time increasing the prize amount to $2 million and changing the route for the field test. The

prize was won by the Stanford racing team with the fastest time out of the five teams that

completed the prescribed route (DARPA, 2007). The third prize contest titled the DARPA

Urban Challenge was launched in 2007, and involved the safe manoeuvre of unmanned vehicles

in high traffic areas. Prizes were awarded to three winning teams out of eleven qualifiers (Stine,

2009).

The next agency to launch prize contests was NASA under the Centennial Challenges

program launched in 2005 (NASA, n.d.). NASA sought to drive progress and solve technical

problems in aerospace technology by inviting participation from independent inventors, student

groups and companies. The Power Beaming Challenge was launched in 2005 and continued in

subsequent years until a winner was announced in 2009 (NASA, n.d). The focus of this

challenge was to wirelessly power mechanical devices so they could propel themselves up a

vertical cable. The technology could allow NASA to remotely propel rovers and other

instruments on the moon. A prize was awarded in 2009 to LaserMotive LLC of Seattle,

Washington (NASA, 2009). The Strong Tether Challenge was launched in 2006 and offered a

prize of $2 million for the design of very long, strong cables with higher than normal strength to

weight ratio. Contests were held every year (except 2008) with no team claiming the prize to

date. Other past prizes offered under the NASA Centennial Challenge program include the

Astronaut Glove Challenge launched in 2009 and awarded in 2010, and the Green Flight

Challenge sponsored by Google in 2011. Competitions were managed on behalf of NASA by

third party non-profit organizations.

The Department of Defense launched the $1million Wearable Power Prize Competition in

2007 under authorization from the John Warner National Defense Authorization Act of 2007

(P.L. 110-36) (Stine, 2009). The objective was to develop a lightweight, long-endurance power

pack to be used by war fighters in the field. Six finalist teams were selected from 169 registered

entries, and the prize was shared between first, second and third place winners (US Department

of Defense, 2008). Other agencies which launched pilot prizes during that period include the

Department of Energy (DOE), and the Biomedical Advanced Research and Development

Authority (BARDA) which falls under the jurisdiction of the Department of Health and Human

Services (HHS).

3.3 US Open Government Initiative – White House

More widespread use of prizes in the US federal government was introduced as part of

the Open Government initiative of the Obama administration. On his first day of office in

January 2009, President Obama signalled his administration’s commitment to creating

unprecedented level of openness and transparency in government. In a memorandum issued to

all heads of departments and executive agencies he identified transparency, participation, and

collaboration as the three cornerstones of his open government initiative. He also highlighted the

critical role of openness in strengthening democracy, efficiency, and effectiveness in Government

(White House, 2009a). This was followed by an Open Government Directive issued by the

Office of Management and Budget (OMB) in December 2009, which mandated executive

departments and agencies to advance their open government initiatives and to set goals and

deadlines for action. The OMB reiterated its commitment to helping agencies find costeffective

and innovative solutions through the use of contests and challenges, thereby promoting the open

government initiative (White House, 2009b).

3.4 America Competes Reauthorization Act 2010

In January 2010 the US Congress passed the “America Creating Opportunities to

Meaningfully Promote Excellence in Technology, Education, and Science Reauthorization Act”

commonly referred to as the “America COMPETES Reauthorization Act of 2010”. The stated

objective of the Act was “to invest in innovation through research and development, to improve

the competitiveness of the United States, and for other purposes” (America Competes

Reauthorization Act, 2010). The Act gave broad authority to agencies allowing them to run prize

competitions, with the objective of it being used as a standard tool to help agencies “spur

innovation, solve tough problems, and advance their core missions” (White House, 2012). The

Act outlined the legal and operational framework under which contests could be implemented by

federal agencies. It addressed issues such as types of challenges, participant eligibility,

government liability, intellectual property and post implementation reporting guidelines, among

other things. It also specifically identified the role of the General Services Administration (GSA)

in providing administrative assistance to help agencies in their preparations to implement prize

competitions. Within 180 days of the enactment, GSA was expected to help further agency

efforts by facilitating the government wide sharing of best practices, and provide access to

relevant products and services through the development of a contract vehicle, and provide

technical assistance in planning and implementation of challenges (America Competes

Reauthorization Act, 2010).

3.5 Challenge.gov Platform

In March 2010 the OMB provided specific guidance and instructions to agencies on the

projected use of challenges and prizes (simply referred to as challenges). Agencies were

encouraged to make use of this tool as a means to further their mission and the principles of open

governance (White House, 2010). They were instructed to increase their capacity to support,

design and manage prizes, and to proactively identify and address any legal, regulatory or

technical barriers hindering the implementation of these challenges. In that memo, the GSA was

tasked with making available a web-based platform within 120 days which would allow agencies

to post challenges and invite submissions from potential solvers. Reiterating the role laid out in

the America COMPETES Reauthorization Act, the GSA was tasked with facilitating the sharing

of best practices across government in the longer term. They were also tasked with expeditiously

providing a contract vehicle where agencies could access relevant products and services to aid

their challenge initiatives (White House, 2010). In response to the OMB directive, the GSA

selected ChallengePost as the provider for the no cost government challenge platform

Challenge.gov, after evaluating offers from eight organizations (GSA, 2011a).

The web-based platform Challenge.gov was launched in the public domain in September

2010 and served as a central site for all prize contests hosted by federal agencies (GSA, 2011a).

The platform is administered by GSA in partnership with provider ChallengePost, and is made

available at no cost to all federal agencies. The platform facilitates the public broadcast of

problems by agencies in the form of contests (also called challenges). For each challenge,

members of the public or specific groups are invited to submit solutions, which are evaluated by

a panel of judges or put up for public vote. Generally the submitted solutions remain in the

public domain to allow other users to comment or vote on the various entries. Various

combinations of monetary and non-monetary incentives are awarded to the winner(s) of each

specific challenge (GSA, 2011a).

After the first year of operation the platform had hosted 115 Challenges from 36 agencies,

received 1,515 user submissions, and paid out $38 million in prizes (GSA, 2011b). By the

second year the number of contests had increased to over 200 submitted by 47 agencies.

According to a report from the White House OSTP (2012), significant new activity had been

observed in areas of national priority such as health, veterans’ services, and employment.

3.6 Adopting Units

Due to the emergent nature of challenge adoption across the federal government, there are

wide variations in the structure of adopting units across agencies and departments. Adopting

units vary in features such as composition, place in organizational structure, level of formality

and autonomy within the agency. For example in some agencies adoptions follow the formal

structure of the agency and is contained within functional divisions, while in other cases adoption

is informal and based on collaboration across functional divisions. A small number of agencies

have a centralized approach where one person or unit is formally charged with the responsibility

of planning and managing all challenges in partnership with the relevant functional units.

However, the majority of agencies have a decentralized approach where functional units and

individuals take full responsibility. Some challenge managers were (formally or informally)

appointed by administration, while some voluntarily took up the responsibility of leading the

initiative due to professional interests or functional overlap. Duties were sometimes carried out

by designated personnel acting alone or with the support of external contractors, and in other

cases they were added on to regular professional duties. The number of projects executed per

adopting unit ranged from a onetime occurrence to a portfolio exceeding 10 projects per year.

The next chapter presents the research methodology.

4 Methodology

This chapter starts with the philosophical perspective that guided the research and

explains the various choices for the research design and strategy.

4.1 Research Philosophy

The way a researcher conducts an inquiry is influenced by the way he/she views the

world (Schutt, 2009). This study is guided by the interpretive perspective which postulates that

reality is socially constructed, and the main aim of social science research is to understand the

meanings that people give to that reality (Schutt, 2009). Interpretivist researchers believe there is

no one single reality, but rather multiple realities which represent different interpretations of a

single event (Merriam, 2009). In direct contrast positivist researchers view reality as objective

and concrete, which can be observed and measured with scientific methods (Schutt, 2009).

Therefore the underlying objective of interpretive research is not to ‘find’ knowledge, but to

construct it (Merriam, 2009).

I selected the interpretivist perspective primarily because of the exploratory nature of this

research. The main objective of the study was to get an understanding of the reasons and

conditions that influenced an agency’s adoption of challenges as a tool to spur innovation. As an

emerging phenomenon I thought it was important to gain an understanding of the situation from

the perspective of the human actors involved in the process, rather than imposing explanations

developed in some other external contexts (Orlikowski & Baroudi, 1991). This perspective

placed value on the experiences of the public employees and how they interpreted what was

happening within their respective agencies. This I believed would present a more realistic picture

of what was actually happening.

Interpretive research also recognizes the researcher as an actor in the process which is

being studied and capable of interpreting what they see and hear in the field (Miles & Huberman,

1994). I therefore acted as an instrument for data collection and analysis by collecting and

interpreting data directly from participants (Merriam, 2009). This had an advantage because at

any time during the interview I could respond or adapt immediately to the situation at hand. For

example I was able to ask clarification questions, summarize and interpret responses, and get

verification and feedback from participants (Merriam, 2009). I felt this perspective was ideal as

it recognized that as a researcher I was not coming into this study as a blank slate. My several

years of work experience in implementing and managing public sector innovations could

enhance my understanding and interpretation of the data collected. However I also had to be

mindful that I did not bring any preconceived biases to the aanalysis and interpretation of results

(Patton, 2002). In this case bias was not an issue since my experiences had been in another

country and I had no personal interest or benefit in the outcome of this study. Further, the use of

informants from many different agencies helped ensure that multiple views were captured,

reducing effects of any potential personal bias.

4.2 Qualitative Research as a tradition of Interpretivist perspective

Interpretivist researchers employ qualitative research methods which capture people in

their natural settings (Lincoln & Guba, 1985). The term ‘qualitative’ suggests an emphasis on

the qualities of the entities and processes studied, as opposed to quantities and intensity

measured by quantitative methods ( Denzin & Lincoln, 2000). Qualitative research focuses on

understanding how social experiences are created and how they are given meaning by the

participants (Denzin and Lincoln, 2000, Merriam, 2009). This makes it an ideal approach for

exploring disparities in the number and types of challenges being adopted and implemented by

the various agencies. Therefore I used qualitative inquiry to understand why and how challenges

were being implemented in each agency, and more importantly to understand the differences.

4.3 Research Design

The inquiry took the form of a single-case study of the challenge.gov web platform. Data

was collected from multiple early adopting agencies that hosted challenges on challenge.gov

during the first two (2) years - September 2010 to August 2012. The two primary sources of data

were: i) interviews with challenge managers, planners and administrators, ii) archival listings of

challenges from the challenge.gov web platform. Related documentation from reports, policy

documents and websites were used to supplement and corroborate the main data. Below I

provide details and justification for various components of the research design.

4.4 Case Study

Benbasat, Goldstein, & Mead (1987) advocated the case research strategy for studying

organizational innovation. This he felt provided the opportunity to capture knowledge directly

from practitioners which could then be used to develop theory. Eisenhardt (1989) also

highlighted the importance of using case study evidence to generate theory. The case study

strategy is suitable for this study because the objective was to understand and explain major

events and conditions related to the adoption and implementation of challenges as an

organizational innovation. An expected outcome was to offer a tentative theory based on the data

collected.

Other justification for using the case study approach is provided in Yin’s (2009)

guidelines for when a case study is the preferred method. The first condition identified by Yin

(2009) relates to the type of research question that is asked. Research questions that ask “how”

or “why” suggest an objective which is explanatory, and deals with tracing operational links over

time. He contrasts these to “who” and “where” and certain “what” questions which suggests

descriptive or predictive goals, and are more likely favor survey methods. This guideline speaks

directly to this study’s intention to understand and offer an explanation for how challenges are

being adopted and implemented. The three research questions ask “how”, for example: 1) How

are federal departments and agencies using challenges, and how do challenges differ in terms of

the level of innovation involved? 2) How do agencies adopt and implement Challenges? 3) How

do the various adoption and implementation factors influence the type of innovation enactment

of the implementing agency?

The second and third conditions identified by Yin (2009) where case study is a favored

method refers to a need to manipulate events, and the level of control the researcher has over

research conditions. For example a case study is appropriate where conditions do not need to be

manipulated as in experiments. For this study I was interested in understanding how the different

events and processes occurred in their natural setting, and was not interested in manipulating the

conditions or events. For the third condition Yin (2009) identified case studies as the appropriate

choice for studying contemporary events and practical problems. At the time, the challenge.gov

platform was less than two (2) years old, and the practices in the various agencies were still

emerging.

4.4.1 Case Selection

A case can be described as an integrated system (Stake, 1995) which is bounded by some

common obvious boundary (Merriam, 2009). An important first step in any case study is to

identify the case boundaries (Merriam, 2009). All federal agencies adopting challenges under

the Open Government Initiative are required to list their challenges on the centrally administered

Challenge.gov web platform. Challenge.gov therefore provides a visible boundary for the case.

Selection of this particular case for study was in line with the guidelines articulated by

Stake (1995). For example, he advocates that the first criterion for selecting a particular case is

that it enables us to maximize what can be learnt within the limited time available. Further,

selection may also be based on the need to get understanding and insight into a particular

situation or question. Accordingly, this case provided an ideal opportunity to get direct exposure

and insight into how federal agencies were adopting this new approach to problem solving and

innovation. Stake (1995) also advised that we should select cases which are “hospitable to our

inquiry” and provide easy access to willing participants (p4). This was one of my primary

motivators for selecting this particular case, as the administering agency the General Services

Administration (GSA) was very interested in the results of the study. They agreed to give full

access to case related data such as contact information for potential participants, documentation,

and invitation to activities related to the case in exchange for a report on the findings.

Case design may be either single or multiple depending on the situation at hand.

According to Yin (2009) single-case designs are sometimes considered to be vulnerable because

everything depends on this one case, in comparison to multiple case designs which are

considered to be more robust. However there are situations where single-case designs are

appropriate and justifiable such as when the case is unique (Yin, 2009). This case is unique

because of the unprecedented and unique nature of the Open Government Initiative. To date this

represents the biggest policy push to adopting challenges as an innovation tool in public sector

agencies. The policy is being promoted from the highest levels of the US Federal Government

and has been accompanied by legislation and allocation of specific resources.

Another justification for the appropriateness of single case designs can be found in

Stake’s (1995) categorization of intrinsic case studies. He describes an intrinsic case study as

one where the case itself is of primary interest, and is selected because the researcher wants to

know more about the specific case, and not for its ability to represent other cases. In this

situation my interest is in understanding and explaining the adoption and implementation of

challenges among US Federal agencies. I consider this an intrinsic case study because the

phenomenon is tied to the context. The intended outcome is to offer a conceptual framework to

represent this specific case, not to build general theory.

4.4.2 Unit of analysis

This is a single case study of challenge.gov with multiple adopting units. The presence of

sub-units provides an opportunity for rich analysis and better understanding of the case, as data

can be analyzed separately within each sub-unit as well as across all the sub-units (Baxter &

Jack, 2008). Interview data was collected and analyzed at the level of the sub-unit (adopting

unit.

4.5 Data collection

Case studies usually have multiple sources of evidence such as interviews, documents and

archives, and observations (Denzin & Lincoln, 2000; Eisenhardt, 1989). The main data sources

were interviews with challenge managers, planners and implementers; and archival listings on

challenge.gov web platform. Documentation related to the open government initiative and the

challenge initiatives were used to corroborate and supplement the main data sources.

4.5.1 Archival data from Challenge.gov

I collected data from the challenge.gov web platform in two phases. The first phase was

conducted in January 2012 and involved one hundred and forty three (143) challenge listings

posted on the platform from its launch in September 2010 until December 31 2011. The second

phase was conducted in September 2012 and involved an additional sixty (60) challenges which

had been added to the site from January to mid-September 2012. This brought the total number

of challenges hosted on the platform from inception to the second anniversary to just over two

hundred (200).

Challenges were listed on the site in reverse chronological order. The listing for each

challenge was represented by an image and basic identifying information such as name, agency,

prize value, and submission deadline. Each listing had a link to more detailed information such

as objectives, background, narrative descriptions, and submission and judging details. Though

the descriptions had a standard layout, there were variations in the level of details provided by

different host agencies. For each phase of data entry I manually entered basic identifying

information for each challenge into an electronic spreadsheet. Each row represented a separate

challenge listing and each column represented an attribute. Some attributes represented

qualitative data such as description, objectives, and submission type, while others represented

quantitative data such as prize value, contest dates and number of submissions.

Some challenges were hosted entirely on the challenge.gov platform, so that all related

data from announcement, submissions, judging, and prize awards could be found in one central

location. For others the platform was used solely to announce the contest, and third party sites

were used for managing submissions and judging.

4.5.2 Interviews

In a case study interviews are essential to get descriptions and interpretations of what we

cannot observe for ourselves from others (Stake, 1995) and to find out information from

someone else’s mind and understand their perspective (Patton, 2002). Interviews allowed me to

discover the multiple realities and viewpoints from the perspective of the challenge managers

and administrators (Stake, 1995). It was important to get their perspectives because I wanted to

understand the intentions and motivations driving adoption as well as their various experiences

during implementation.

4.5.2.1 Preliminary Interviews and data gathering

I made initial contact with the relevant GSA officers in August 2011 through a member of

my dissertation committee. Over the next few months we conducted a series of preliminary

discussions and informal interviews via telephone and email, to help get a better understanding

of the initiative and how it had progressed to date. I then formulated and shared a high level

outline of the objectives and expected outcomes of the proposed research to ensure there was an

overlap between my objectives and theirs. Over the next few months we held additional informal

interviews and discussions as I refined the research proposal. The GSA officers were very keen

on the proposed research and agreed to provide the necessary endorsement and access to research

data and participants.

Additional preliminary data gathering took place in March 2012 at a challenge Meet and

Greet event in Washington DC organized by the GSA challenge team. During the proceedings I

conducted brief preliminary interviews with twelve (12) participants representing challenge

programs from different agencies. I enquired about the background and current status of

initiatives at their agencies, and some of the major hindrances they faced. I took handwritten

notes which I used later to modify the research questions and provide direction for the literature

review.

4.5.2.2 Sampling and Recruitment for main interviews

Purposive sampling refers to the practice of selecting a sample from which the most can

be learned (Merriam, 2009). I thought that the most could be learned by capturing data from as

many agencies as possible, and employed a comprehensive sampling strategy where every unit of

interest is selected for data collection (Miles & Huberman, 1994). The challenge.gov program

managers provided contact names and information for the challenges listed on the challenge.gov

platform. I sent out individually addressed email to potential participants from fifty one (51)

adopting units, introducing the study and soliciting their participation (see appendix for

recruitment email). For those who responded and were willing I set up appointments for an

interview at their convenience. For those who were not immediately responsive I followed up

with a second email, followed by a telephone call if still not responsive after two weeks.

As recruitment activities and interviews progressed I noted that it would not be possible

to interview all players within the projected time frame since some remained non-responsive

even after follow-up emails and phone calls. As a result I had to adjust the sampling strategy to

make it more purposive and ensure that I had representatives from the now emerging theoretical

groupings based on frequency of executions. The adjustment of sampling strategy is not unusual

in qualitative studies and Strauss & Corbin (1998) caution that sticking too rigidly to the set

sampling procedure can have negative effects on creativity and the analytic process. At this point

I targeted certain programs on both ends of the spectrum: large numbers of challenges (e.g

greater than 10) as well as infrequent implementers (3 or less).

The final sample consisted of 36 interviewees representing 31 adopting units. Table 1

lists the number of challenges and interviewees by agency. These interviewees included mainly

challenge managers, contractors, and overall administrators. Some of the units (3) were

represented by two respondents who agreed to do joint interviews.

4.5.2.3 Interview Procedure

I conducted interviews during the period May to October 2012. Each interview lasted

approximately 45 minutes to one hour, and was conducted face-to-face or via telephone as

logistics permitted. This was a study involving human subjects and I sought and obtained

Institutional Review Board (IRB) approval (see Appendix) before commencement of interviews.

All participants received information on the objectives of the study and were given an

opportunity to ask questions or seek clarification prior to the interview. I also sought permission

to record interviews while ensuring participants of anonymity and confidentiality when reporting

results. I recorded interviews using a hand held digital recorder, and in two cases where

permission was not given to record I used handwritten notes. All participants received an

informed consent form and were given the opportunity to opt out of the study with no

consequences.

Executive Agencies and Departments Implementing

Challenges

No of listings on

challenge.gov

No. of

Interviewees

US Department of Agriculture 4

Center for Nutrition Policy and Promotion 1

Forest Service 1

National Institute of Food and Agriculture 1 1

US Department of Commerce 1

US Patent and Trademark Office 2

US Department of Defense 11

Air Force 12 2

Army 1

Navy 2 1

National Security Agency 1

US Department of Education 6 2

US Department of Energy 8 1

US Department of Health and Human Services 14 2

Administration on Aging 1

Centers for Disease Control and Prevention 6

Centers for Medicare & Medicaid Services (CMS) 1

Let's Move Faith and Communities 1

National Institute of Health 6 2

Office of the National Coordinator for Health Information Technology 21 2

Substance Abuse & Mental Health Services Administration 3 1

US Department of Homeland Security Transportation

Security Agency (TSA) 1 1

Federal Emergency Management Agency(FEMA) 1

US Department of Housing and Urban Development 1

US Department of Justice National

Institute of Justice 1 1

US Department of Labor 5 1

Occupational Safety and Health Review Commission 2

US Department of State 2 1

US Department of the Interior 1

National Park Service 1

US Department of the Treasury 3

US Department of Transportation 8 2

Distraction.gov 1

US Department of Veterans Affairs 4 2

Executive Office of the President The

White House 3

Council on Environmental Quality 1

Independent Agencies

Consumer Product Safety Commission (CPSC) 1

Corporation for National and Community Service 5 1

Election Assistance Commission 1

Environmental Protection Agency (EPA) 15 3

Federal Communications Commission (FCC) 3

General Services Administration (GSA) 4 3

National Aeronautics and Space Administration (NASA) 20 3

National Archives and Records Administration (NARA) 2 1

National Science Foundation (NSF) 4 1

Small Business Administration (SBA) 7 1

Social Security Administration (SSA) 1

US Agency for International Development (USAID) 3 1

TOTALS 205 36

Table 1: Number of Interviewees by Agency

4.5.2.4 Interview Protocol

The interview protocol was developed using the research questions and sensitizing

concepts selected from the literature as a guide. Development of the protocol was an iterative

process involving several rounds of revisions after consultation with committee members, GSA

program officers, and pretesting with two challenge managers. Questions were open-ended and

followed a semi-structured format which allowed for exploration of main themes from the

literature, while allowing flexibility to explore emerging themes and new ideas. Questions

focused on the processes, enabling and hindering conditions, outcomes, lessons learned, and

recommendations. Plain language was used to make questions easy to understand, and care was

taken to avoid technical jargon and concepts directly from the literature (Merriam, 2009).

Questions were organized into six (6) major groupings described below (See appendix for

entire protocol).

a) Background Information

The first set of questions served as a warm-up for the interview and helped establish basic

facts about the respondent and the implementing unit. Questions enquired about

respondents’ educational and professional background, and their role within the

organization. It also enquired about the mission and operations of their agency and

implementing unit.

b) Agency Innovations

In this section the questions focused on general innovation practices including common

sources of motivation and inspiration to innovate.

c) Decisions/Processes

This section captured critical data on reasons for the decision to adopt, enablers, and

hurdles. It asked participants to recall specific challenges and recount their motivation for

adopting, and the nature and level of decision making required. It also probed into

specific enablers and hurdles identified in Chapter 2 as sensitizing concepts. These

include: special initiatives and organization commitment, management support, rules and

red tape, attitude of colleagues and NIH syndrome. Questions also probed into

alternatives routes that were considered, challenges abandoned along the way, other

hurdles, and surprising incidences encountered.

d) Lessons Learned and recommendations for public managers

The last section of the instrument asked participants to reflect on their experiences and to

offer recommendations based on their assessments. It solicited personal opinions on

effectiveness of the method, and the types of problems that were suitable. Some questions

used repetition to ensure that the most salient points were captured, and participants were

asked again to reflect on the most important enablers and hurdles.

e) Opportunity for additional information

At the end of the interview there was an opportunity for the respondent to add anything

pertinent that did not come up in the interview. There was also an opportunity to ask

questions or seek clarification on anything concerning the research project.

4.5.3 Documentation and web resources

In case study research documents play an important role in corroborating and

supplementing data from other sources as well as providing leads to other sources of data (Yin

2009). I consulted various forms of documentation such as reports, press releases, agency and

provider websites, blogs, YouTube videos and social media pages to help corroborate and

supplement data from challenge.gov listings and interview data.

Concerning archival data collection on challenges, the non-standard layout of external

sites led to many instances of missing or difficult to locate data. I relied extensively on related

documentation such as agency websites, media announcements, articles and blogs to locate any

missing data. For the analysis and categorization of challenges there were many cases where I

had to rely on reports and media releases to supplement data and provide insight into initial

goals, plans and outcomes.

I used agency websites and the social media platform LinkedIn to collect background

information on interviewees prior to the interview. I also used documentation from agency

websites and media releases to supplement and corroborate interview data concerning planning

and execution of prominent challenges. I also used repositories of reports and policy documents

found on websites of agencies such as Department of Health and Human Services (HHS), NASA

Center of Excellence, GSA and the challenge.gov Listserv to help provide a more complete

picture of processes, guidelines and trends for challenge execution.

4.6 Data Analysis

Qualitative analysis is a highly intuitive process which is creative, dynamic and iterative

(Taylor & Bogdan, 1998) Throughout the analysis I was constantly trying to refine my

interpretations (Taylor and Bogdan, 1998) by going back and forth between the data, emerging

themes and the literature. First I analyzed the archival listings of challenge.gov and then moved

to coding and analysis of interview data.

4.6.1 Analysis of archival data from web platform

I collected and analysed data from the platform in two phases which were eight months

apart. Initial steps involved browsing through the list of challenges looking for repetitions and

common themes to help identify common attributes which could inform the classification

process. I closely examined each column of attributes such as description, objectives,

submission requirements, judging criteria and prize value. First I compared values of each

attribute looking for patterns and variations. As I began to notice similarities I formed loose

groupings of similar types and iteratively tried to combine smaller groupings into larger ones.

For example in examining the attribute submission types, I identified various formats such as

video contests, photography contests, logos, mobile applications, data management software for

institutions, blueprints, white papers, etc. During the next iteration I combined these smaller

groupings into four larger groups: creativity contests (posters, video, photos, slogan, logo, etc.);

software products (mobile and institutional apps); research papers and ideas; and product

(design, prototype, actual product) or working model of a product.

I used the similar method to look for groupings among other attributes. The most

prominent groupings were found among the objectives attribute, which I was able to link to the

submissions groupings. For example I was able to link challenge objectives related to public

education and engagement with submission type creativity contest, and established similar links

among other groups. Then I went a step further in the analysis and placed categories along an

innovation continuum by making comparisons among the objectives, descriptions and outcomes

of the different groups. The emergent categories and their properties are detailed in the Findings

chapter which follows.

Validation and support for the classifications came from three sources: 1) coding was

conducted in two distinct phases nine months apart and categories remained stable between

phases 2) consultation with one of the GSA program managers for challenge.gov agreed that the

representation was accurate and 3) interview data from challenge managers revealed support for

the view that not all challenges were equal, and that there were differences in level of

sophistication and innovation.

4.6.2 Analysis of interview data

Analysis of the interview data was an iterative process which began during data collection

and continued long after data collection was complete. Following Miles and Huberman (1994),

after each interview I wrote a memo to document any major thoughts, hunches, ideas which

could inform the ongoing data collection and analysis. Recorded interviews were professionally

transcribed into word processing documents. For each interview I read through the transcript

while listening to the recording. This helped ensure accuracy of the transcript and helped regain

some of the context that may have been lost in transcription. I read through transcripts a second

and third time and made notes on emerging themes and patterns.

There were two major activities in the analysis i) coding to identify the main activities

and decisions of the challenge adoption process and ii) identification of agency groupings. I will

first describe the coding and development of the adoption model.

I loaded electronic transcripts into the qualitative data analysis computer software

package NVivo 10 along with corresponding memos. I also imported my provisional list of start

codes into the software. This list contained eighty six (86) codes derived from sensitizing

concepts identified earlier, combined with concepts from the research questions (Miles and

Huberman, 1994) and the first set of variables emerging from the data (Strauss & Corbin, 1998).

The provisional start list captured presence or absence of concepts from the data such as:

employee attitude (NIH syndrome), management support, special funding, dedicated employee,

use of intermediaries, organizational red-tape. As I progressed through coding more interviews I

was continuously revising the codes to ensure that it reflected the data. For example some codes

did not have sufficient segments to fit them and eventually had to be discontinued or combined

with other codes, while others with extensive number of segments had to be broken down into

sub codes (Miles and Huberman, 1994).

The first level of coding was descriptive and the focus was on naming, labeling and

identifying categories and variations within these categories. For example one category was

agency mission, with variations within that category identified like public engagement, research

and development, service provision. Another emerging category was external trigger with

dimensions policy, exposure, and mandate. I continued identifying categories through several

rounds of coding until I felt there were no new categories emerging which were relevant to the

research question (Lincoln & Guba, 1985). The final list of codes grew to over two hundred

(200) codes (see appendix).

The next level of coding was more analytic and focused on identifying and explaining

patterns and recurrences (Miles and Huberman, 1994). At this point I sought to find relationships

between the various codes, as well as reasons for these relationships. This involved looking at

the various contexts, circumstances, and intervening conditions under which adoption was likely

to take place. Utilizing data displays as suggested by Miles and Huberman (1994), I used a

diagram to represent the categories and their relationships. I represented the various categories as

boxes and used lines to represent the relationship between them. The diagram went through

several iterations and revisions during which time I got feedback and input from colleagues and

members of my committee. The final diagram which emerged (chapter 5, Figure 3) represents the

various components and relationships involved in challenge adoption and implementation.

After writing memos and reflecting on the first set of interviews I noticed a pattern

emerging with two different groups of agencies. In the first group, agencies were adopting

awareness type challenges which focused largely on public service announcements and education

campaigns. The other group adopted research and solution type challenges related to science and

technology related topics. As I continued reading through interviews and memoing, I noticed a

third group that seemed to fall somewhere in the middle. That third group of agencies were

implementing software application type challenges (service) for use by their constituents.

Recalling the typology of challenges and continuum of increasing innovation levels which

emerged from analysing the archival list from challenge.gov, I concluded that the three groups of

adopting agencies were implementing challenges with varying levels of innovation. With this

realization I went back to the literature in search of some similar phenomenon. During this

literature search I encountered Orlikowsk’s typology where technology adoption in organizations

followed three enactment types: inertia, application and change. I was able to identify agencies

that fit each type of enactment. From that I was able to build a profile for each enactment type

where I highlighted factors from the adoption model such as external trigger, internal motivation,

preparation type. Once I built this profile for each enactment type, I used pattern matching (Yin,

2009) which allowed me to compare new cases. Therefore I was able to group the remaining

agencies according to the three enactment types.

4.7 Accuracy and trustworthiness of results

Overall, reliability and generalizability do not feature very prominently in qualitative

research. What is more important is ensuring the accuracy (Creswell, 2006) and trustworthiness

of the findings (Yin, 2009). Creswell (2006) recommends that qualitative researchers use at least

two of the common verification techniques to help ensure accuracy of findings. Triangulation is

the most well known strategy of increasing the credibility of research results. I employed data

triangulation which involves the use of multiple data sources to help corroborate the findings

(Yin, 2009). I conducted multiple interviews to get a wide range of perspectives using people

from different agencies and implementing units, and representing different roles such as agency

challenge managers and coordinators, program managers, and private contractors executing

challenges. There was consistency in the findings regarding the adoption process and influencing

factors resulting in the representation of the process and different agency groupings. This

consistency provided an indication of validity of findings (Patton, 1999).

Another method to ensuring validity of results was maintaining a chain of evidence by

carefully documenting all steps increases the reliability of information gathered for a case study.

This makes it possible for an external observer to follow (and replicate if desired) and trace the

steps from research questions to conclusion (Yin 2009, p122). The steps for this study have been

carefully documented to ensure that proper procedures were followed, and that it can be

replicated by another researcher if necessary.

Results are presented in the next chapter.

5 Findings

This chapter presents findings based on the three research questions which guided the

study. The first question examined the types of challenges posted by agencies on the

challenge.gov web platform. Analysis involved classification of over two hundred (200)

challenges into four (4) major categories falling along a continuum of increasing innovation.

Research question 2 sought to reconstruct the sequence of events, processes and conditions

leading to challenge adoption and implementation. The various components are represented in a

diagram followed by the narrative description. Research question 3 examined how variations in

the components of the adoption and implementation process result in different enactments of

challenges. The three types of enactment used by Orlikowski (2000) - inertia, application and

change – are used as a typology to classify the different patterns observed.

Detailed presentation of findings follow.

5.1 Research question 1

How are federal departments and agencies using challenges, and how do challenges differ

in terms of the level of innovation involved?

One of the primary objectives of the challenge program was to provide a tool to help

agencies innovate. This research question sought to gain deeper insight into how challenges

were deployed at the various agencies; and to what extent they were put to innovative uses. For

example, one popular way that challenges were used to spur innovation was to create a climate

and conditions under which others could innovate:

A very central part of what we are trying to do…is to create a climate where a lot of these

tools can be developed. But we are a very small team, and we are not so quick to develop

the tools ourselves, but we can create a climate and conditions under which tools can be

developed then (Informant 8)

From my preliminary observations of the challenge.gov platform, it was evident that not all

challenges were being used to spur innovation. Some were being used to replicate existing

practices and maintain the status quo rather than transforming the way of doing things. Some of

the more savvy informants articulated observations that some challenges were not meeting the

intended innovation goals. They spoke of the need for more carefully selected objectives to allow

maximum creativity and target knowledge gaps. The following quotes illustrate views of two

participants who stressed that challenges only make sense when there is room for creativity and

innovation.

I think that if there’s a piece of technology that you need and you have very, very

clear specifications, and there’s not much room for creativity or building in

other functionality, if what you need is a key to fit into a lock, then challenges

probably aren’t the way to go. (Informant 9)

I feel like a challenge really makes more sense when you don’t know what the solution is.

Like if we want to do some amazing thing and we don’t know how to figure it out.

(Informant 22)

Concerning why challenges were not always deployed for innovative uses, a key informant

suggested that some agencies were implementing challenges because it was the popular thing to

do and they did not want to be left behind.

…it’s a new thing that a lot of people are trying, and people don’t want to be left out or

left behind, so they feel like well, I better try this too. And, I’d better show my

management that I’m thinking about this before they come to me. So, government

personnel just want to show their leadership hey, we’re thinking about this and we’re

trying to get this going….(Informant 24)

Some also felt that there was tremendous pressure on agencies to implement regardless of the

nature of, or appropriateness of the problem:

I think there can be value in giving outside perspectives to solve problems. And that

challenges can be a valuable tool for solving those problems. I think for some time now,

since challenges have become a bigger thing in government, there’s pressure to do

challenges, whether or not you really have a problem that a challenge makes sense as

your way of finding the solution [..]. . I guess that they have a place, but there’s been a

push to do them, just to do them. (Informant 22)

The differing perspectives on why challenges are not always used for innovative means are

summed up in the following quotes. The first quote identifies a fundamental lack of

understanding for challenges as the main reason.

There is a lack of fundamental understanding of how challenges work and where, and

how to create them in a way that might actually meet your goals, and what kind of issues

lend themselves to being solved through challenges. (Informant 35)

The second quote has a more opportunistic outlook that agencies are ‘playing it safe’ because the

program is still new and not fully accepted. However, once fully accepted it would potentially

draw riskier but more innovative uses.

Given that there were a lot of eyes on this program, a natural inclination is to play

it safe. And I think that once challenges are more accepted, and more part of the

norm across government, there will be more space to take on those larger

challenges. Challenges that are higher risk, but also potentially higher reward

(Informant 9)

The remainder of the question examines the actual manner in which challenges have been used

among the various agencies.

Challenge Categories

Following qualitative analysis of over two hundred (200) challenges listed on the

challenge.gov platform, I identified four (4) distinct categories. The categories represented

varying patterns in objectives and complexity of problems addressed. The majority embraced an

element of public education and outreach. These required minimum skills to participate,

produced low innovation and made minimum contribution to agency operations. At the other

end of the spectrum there were some complex problems which required specialized training or

skills to solve, produced higher levels of innovation, and made more significant contributions to

agency operations.

Following this I arranged the categories along a continuum of increasing levels of

innovation based on my perceptions of the level of novelty required for the solution, as well as

level of specialized skills and knowledge required to solve. The largest category (awareness)

focused on public service announcements and education campaigns, which generally represented

lower levels of innovation. The operational category focused on functional outcomes resulting

from new or extended knowledge and represented the highest innovation level. Service and

research categories fell at different points in between, with service representing a slightly higher

level.

Below I present a profile of each category accompanied by examples and quotes. The

examples have been extracted from archival listings on challenge.gov and identify challenge

name and agency name. The quotes were extracted from interview data and were used to support

placement of the various categories along the innovation continuum. In order to maintain

anonymity of interviewees, the highlighted challenge examples did not coincide with quotes

from interviews.

5.1.1 Awareness

Challenges in this category had a mandate to inform, sensitize or educate on topics with

wide public appeal such as the community, health, and the environment. Participation usually

involved submission of creative products such as videos, posters, photos, slogans, logos, and to a

lesser extent nominations for recognition awards. Minimum skills were required for

participation, translating into low barriers to entry and low innovative outcome. The general

public was usually the main beneficiary, though in some cases specific groups were targeted.

Awareness challenges represented forty percent (40%) of total implementations, making it the

largest category.

Name of

Challenge

Agency Contest Objective

(extracted directly from

challenge.gov)

Description

(extracted directly from

challenge.gov)

National Radon

Poster Contest

Environmental

Protection Agency

(US EPA)

Raise radon awareness in

schools using posters

Engage student leaders in raising

radon awareness through their

posters among their family, school

and community

Youth

Sustainability

Challenge:

Creating An

America Built to

Last

Council on

Environmental

Quality

Raise awareness of young

people on practices to

foster sustainability in the

US and the world at large

Create a video to tell the world

what you are doing in your

community to foster sustainability

and create an America and a world

built to last

Picture It! Safe

Workplaces for

Everyone

Occupational Safety

and Health

Administration

(OSHA)

Kick off a national

collaboration that relies on

talent, imagination and

creativity to raise awareness

of workplace safety and

health

Capture an image of workplace

safety and health and share it with

the OSHA

Stop Bullying Video

Challenge

Department of

Health and Human

Services

To help prevent and end

bullying in schools and

communities across the

nation.

Create a video to tell us how you

can be more than a bystander and

help kids who are involved in

bullying

Absentee Voting

Slogan Contest

Department of

Defense,

Federal Voting

Assistance program

Promote absentee voting

by the Uniformed Services,

their families, and U.S.

citizens residing outside the

United States.

Create a simple slogan (or slogans)

that express the importance of

voting and will inspire others to

vote.

Two-Second

Turnoff Distracted

Driving Video

Challenge

Department of

Transportation,

National Highway

Safety

Administration

Bring awareness to

distracted driving

Create a video with a Public

Service Announcement against

distracted driving and post it on

YouTube

HBCU Mental

Health Promotion

Campaign 2011

Substance Abuse &

Mental Health

Services

Administration

To utilize the creative

talents of Historically Black

Colleges and Universities

(HBCU) students to produce

a communication campaign

to raise awareness and

education about mental

health, its impact on the

overall wellbeing of

students and associated

impacts on academic

performance.

Produce a communication

campaign to raise awareness and

education about mental health, its

impact on the overall well-being of

students attending Historically

Black Colleges and Universities

(HBCUs) and associated impacts on

academic performance.

Table 2 Examples of Awareness Type Challenges One informant who previously ran a video

contest expressed the view that many of these contests did not solve any specific problem, an

indication of its lower ranking on the innovation scale:

Video challenges to me seem like a contest. There’s not really a problem to solve, it’s just

we’re saying we want you to make a video on a topic, and give some money to the best

one. It’s like do you really need the videos? If you really wanted the videos, why not have

that specially created. I guess I can see the value of having the public submit videos,

but what is that really solving? (Informant 22)

Another informant who ran several large scale challenges saw awareness challenges as an

important starting point for agencies to learn from, though advocating progression to more

sophisticated types.

I think it’s worthwhile for everybody to get their feet in the water, to try these things out,

but to try to do things a little more sophisticated than poster competitions and video

competitions, things like that. I think, that’s a great place to start, but they would be

wasting an opportunity if they got stuck there. (Informant 26)

Another expressed excitement over the prospect of moving from awareness challenges to larger

more innovative projects, which was the ultimate goal:

I think it’s exciting as we move from small scale videos and apps to larger sort of point

solution challenges…. (Informant 8)

Examples of Awareness type challenges extracted from challenge.gov are listed in Table 2.

5.1.2 Research

These challenges sought ideas to expand industry knowledge on particular topics, and in

some cases foster development of new technologies or products. Topics were usually science and

technology related. The targeted participants had to have knowledge and understanding of the

field, but did not necessarily need practical experience. Submissions were sought in the form of

white papers, conference papers, idea submissions, and usually required some form of research

or new knowledge, but did not require operationalization. This was the smallest category and

contained fourteen (14%) of the total number of challenges.

The challenges in this category can be considered more innovative than awareness

challenges because they involve the discovery of new knowledge or ideas. However the level of

innovation is limited because the solicitations focus on the discovery and presentation of the new

knowledge. Neither the hosting agency, nor participants seemed particularly interested in

immediate application of that knowledge to solve any identified problem. The following

illustrates:

The way the challenge was set up was to invite ideas for other ways to use [new

technology name]. Assuming that it’s out in the world, which we’re not certain of, but if

it were, what should we do with it? So there’s what we call an ideation challenge, which

is to say, we just want to know your thoughts, just a white paper. (Informant 16)

The next illustrates one common example of research challenges which involved the search for

ideas on how to jumpstart a particular type of software development.

It wasn’t a problem that we were facing; I wouldn’t describe it that way. We are trying to

jumpstart data application development across the country. And so we put out a (notice)

that encourages the researchers in the universities and colleges across the country to

submit a white paper to us, and then we would look at it and see if the ideas are good or

not, and if they were, we would ask them to submit a bigger proposal to us. But that

doesn’t really reach into the open source developer community. So in order to reach out to

the hundreds of thousands of open source developers, to hold a competition would be a

better approach. So, we’re doing both simultaneously. We still want to reach out to the

universities and the researchers that are there, but we want to reach out to anyone who

has a great idea for [project name], and we don’t care where the persons are located. We

really want to get the best ideas. (Informant 32)

Generally the objective of research challenges was not for the internal use of hosting agencies,

but rather to expand knowledge and jumpstart use of new technologies in industry. The

following quote describes the agency’s intention of encouraging research on a particular type of

material, which can then be used by practitioners to manufacture commercial products.

What we’re trying to do is spur innovation. We are not trying to create something that

we can hold and use and be the proprietors of. We want this to be something that can be

used by the public. We will maintain rights to use it within the federal system, but the

developer has the right to continue developing it, or to sell it to do whatever they want.

Because we’re obviously not in the business of making money. We just want to get the

feel, pushing in this direction, because there’s a need among the practitioners, and we’d

like to have people focus on that. (Informant 30)

Examples of Research type challenges are listed in Table 3.

Name of

Challenge

Agency Contest Objective

(extracted directly from

challenge.gov)

Description

(extracted directly from

challenge.gov)

DTIC Student

Paper Competition

Department of

Defense; The

Defense Technical

Information Center

(DTIC)

-To promote the generation

of new ideas and the

conduct of new research in

areas of strategic interest to

DTIC

-To promote understanding

of DTIC's mission and

program among

information professionals

Defense Technical Information

Center (DTIC) solicits student

papers for consideration on the

agenda of its annual conference

NIH Lessons About

Bioscience (LAB)

Challenge

National Institutes

of Health

Contribute to collection of

engaging, inexpensive

experiments for students

from kindergarten through

high school

Help us bring cool experiments

into the classroom so everyone can

enjoy doing science! Submit

written procedures of

experiments. The best experiments

will become part of an official

collection that NIH will distribute

for free in print and electronically.

StartUp Amarica

Policy Challenge

The White House Help knock down barriers to

innovation and

entrepreneurship in health

care IT, clean energy, and

learning technologies

Potential entrepreneurs post

barriers to growth of their

business, and members of the

public submit ideas for potential

solutions.

Connected Vehicle

Technology

Challenge

Department of

Transportation,

Research and

Innovative

Technology

Administration

(RITA)

Solicit ideas that will help

build the transportation

system of the 21st century

With new wireless technology,

vehicles can “talk” to each other.

The DOT challenged the public to

submit white papers to share ideas

for what vehicles should say to

each other.

Sponsoring

Scholars in Science

Navy,

Office of Naval

Research (ONR)

-Stimulate K-12, higher Ed

and educational research

initiatives designed to

heighten student interest in

the STEM disciplines. -

Growing the pipeline of

future scientists and

engineers for the naval

workforce.

Present white papers aimed at

generating innovative projects that

cultivate student interest and

participation in science,

technology, engineering and

mathematics (STEM). Each winner

will receive a research grant worth

up to $100,000 to further develop

their proposal.

Table 3 Examples of Research Type Challenges

5.1.3 Service

The objective of these challenges was to initiate or improve delivery of government

services by facilitating production of software tools and other methods. Software tools included

applications for institutions providing public services and mobile applications to facilitate public

access to government services. Other objectives included promoting the use of publicly

available datasets and spurring entrepreneurship and industry around provision of a service. The

target audience generally consisted of software developers and enthusiasts. This represented the

second most common type of implementation and accounted for twenty six percent (26%) of the

total.

The following quote illustrate the use of a challenge for the development of software

applications to help consumers. The reason I classify these service challenges as slightly more

innovative than research challenges, is that they go beyond solicitation of ideas, and result in an

end product that can be used for public benefit.

… this is where the prizes and challenges come in. Because of many of the new and

sophisticated tools, like social media platforms. We the government are not so advanced

in developing these. So what we have is a lot of raw data. And much of why we are

interested in prizes and challenges, is because we are trying to incite, promote, sort of

leverage the talent, skills, and knowledge of those outside of the department. Those who

can help build tools using those platforms that help consumers and physicians and

researchers, and so on, access [agency name] data for the purposes of improving their

health. (Informant 8)

So we try to get challenges that have high operational impact, and strategic value. And

what I mean by that is we don’t want ideas for changing the color on our logo. We want

how do we enhance the customer experience at […] which is a really important issue. So

we don’t want fluffy challenges we want real impact. (Informant 31)

Name of

Challenge

Agency Contest Objective

(extracted directly from

challenge.gov)

Description

(extracted directly from

challenge.gov)

US DOT

Motorcoach Safety

Data Student

Challenge

Department of

Transportation;

Federal Motor

Carrier Safety

Administration

(FMCSA)

Seeking fresh ideas and

innovative solutions to

make vast amounts of data

easier to access and

understand for motorcoach

travelers

Create an online or mobile

application product to translate

safety data into a web-based or

mobile application that helps bus

travelers quickly choose the safest

bus available.

Apps for

Entrepreneurs

Small Business

Administration

Give small businesses and

entrepreneurs better tools

to navigate the Federal

Government more

effectively

Create apps to help small

businesses and entrepreneurs

identify useful programs and

services on federal government

websites

Electronic Health

Record (EHR)

Accessibility

Module Challenge

Office of the

National

Coordinator for

Health Information

Technology

To promote accessibility

and usability in health IT for

the disabled community,

and significantly improve

the health of disabled

individuals.

Create and test an application that

makes it easy for disabled

consumers to access and interact

with the health data stored in their

EHRs.

Apps for Energy

Department of

Energy

Help homeowners and

businesses take action,

understand their usage and

make better-informed

decisions.

Create new apps that help utility

customers make the most of their

Green Button electricity usage

data.

CDC Flu App

Challenge

Centers for Disease

Control and

Prevention

To promote healthy

behavior for flu prevention

by introducing the software

development community to

those resources, and

generating useful and

innovative solutions with

those data

Use CDC flu data to develop an

innovative mobile or web app, data

visualization, system, tool, or

game.

Department of

Commerce

Business App

Challenge

Department of

Commerce

To help businesses identify

opportunities, grow,

enhance productivity and

create jobs.

Design an app for innovative ways

to utilize Department of Commerce

and other publicly available data

Blue Button for All

Americans

Department of

Veteran’s Affairs

(VA Innovation

Initiative)

To encourage widespread

use of Blue Button℠

personal health records

(PHRs) outside Federal

health care programs to

benefit all Americans --

including Veterans who

receive care from non-VA

providers

Looking for a simple, secure and

convenient app which uses the

Blue Button(sm) technology and

which is installed on the websites

of 25,000 physicians across

America

Occupational

Employment

Statistics Challenge

Department of

Labor

Finding new ways to use

Occupational Employment

Statistics

Create data visualizations that

assist in planning education or

training, making career choices,

moving to a new area, or

negotiating pay.

Table 4 Examples of Service type Challenges

While these challenges ranked fairly high on the innovation continuum, those who were running

them recognized that there was a higher level of innovation to aspire to:

I don’t know that we’ve developed something that it is incredibly innovative. As

innovative as some of the apps [software applications] have been, they’re still mobile

apps. There hasn’t been any sort of industry disrupting technologies. (Informant 9)

And though they had not quite gotten to the type of challenges they considered the most

innovative, there was definite desire to move to in that direction:

But so far, the contests that we have held are primarily more educational and like

working with our records to produce mash-ups and things like that. Maybe in the future

we’re kind of always thinking about how we could run a contest or a challenge on some

kind of tech heavy questions or things like that, but we just haven’t gotten there yet.

(Informant 35)

I would say to date, most of our challenges have been around apps, or… promotional

videos. That is changing. I mean we are moving toward more complex solutions.

(Informant 8)

Examples of Service type challenges are listed in Table 4.

5.1.4 Operational

These challenges sought to provide a functional solution to a specific problem and were

generally science and technology related. Solutions in this category were highly technical and

targeted participants with specialized knowledge and skills. The submission format included

finished products or plans, designs, models, prototypes of actual products (e.g. energy saving

light bulb, combat vehicle). Unlike other categories, solutions were sometimes intended for use

in operations of the hosting agency. This category represented twenty (20%) of challenges.

Challenges within this category were viewed by informants as closer to the ideal of what

challenges should be designed to achieve. Informant 22 who worked on a video challenge

referred to a challenge ran by another agency as epitomizing the ideal:

the Department of Energy did some challenge about creating a better light bulb or

something. That was really looking for innovative outside ideas that came from smart

people that maybe didn’t work at their department, or their agency. To me that’s a

challenge. We’re asking you to solve it however you want and we want a specific end

result. (Informant 22)

Informant 27 who had ran numerous operational challenges articulated the need for more

technical challenges to stretch the knowledge environment level and promote innovation:

I think to engage the scientific and engineering workforce out there; they need to be

technical challenges, not just ideation challenges or apps challenges which folks are

doing, which I don’t think stretch the knowledge environment level. (Informant 27)

Informant 9 who ran one of the larger and more prominent challenge programs provided a vision

for the ideal transformational challenge which goes beyond the technological and scientific to

transform entire processes or workflows. This type of challenge may be complex and risky, but

potentially represent innovation at the highest levels:

Those with higher risk and potentially higher reward might be more on the lines of how

do we change an entire process. And that is something where a solution would go beyond

just a technological solution. Maybe it requires an entire change in workflows or a

change in mentality, or building an entirely different group of people into a process. That

may be more difficult to implement and would not necessarily guarantee results...but have

potentially higher reward. (Informant 9)

Examples of Operational type challenges are listed in Table 5.

Name of

Challenge

Agency Contest Objective

(extracted directly from

challenge.gov)

Description

(extracted directly from

challenge.gov)

Medical

Transportation

Device for Combat

Rescue

Air Force Deliver a concept/design for

a next-generation medical

transportation device

Produce innovative designs for a

transportation device that can be

carried into a combat site by a single

rescuer and safely evacuate the

injured

USPTO Algorithm

Challenge

Patent and

Trademark Office

Deliver an algorithm that

can automatically identify

and locate specific elements

within patent documents

Develop new algorithms to aid in

patent examination

Design and

simulation of an

accurate

shooterlocator

Air Force Looking for a design of a

shooter locator that can

detect any small arms fire

within a fraction of a

second and pinpoint its

source accurately (within a

few meters) within a few

seconds

Requires a written proposal and

proof-of-concept data including a

computer simulation of the radio

frequency and electrical performance

of the system. Convincing evidence of

feasibility based on component

availability and physics or empirical

simulation is required for the award.

Experimental evidence desired but

not required.

Bright Tomorrow

Lighting Prize

(L-Prize)

Department of

Energy

-Spur development of ultra-

efficient solid-state lighting

products to replace the

Design and produce lamps that are

suitable for replacing 90-watt 120V

standard halogen or 70-watt 120V

common light bulb -

Substantially accelerate

America's shift from

inefficient, dated lighting

products to innovative,

high-performance products

halogen infrared (HIR) PAR 38 lamps.

These products will allow direct

replacement of halogen PAR 38 lamps

in lighting fixtures with medium

screw-in (Edison) sockets. As such,

winning products must be similar to

the products targeted for

replacement, in terms of size, shape,

operating environment, and light

output, distribution, and quality.

Sample Return

Robot Challenge

NASA Encourage innovations in

automatic navigation and

robotic manipulator

technologies

Demonstrate a robot that can locate

and retrieve geologic samples from a

wide and varied terrain without

human control.

2011-2012 FAA

Design

Competition for

Universities

Federal Aviation

Administration,

Department of

Transportation

National Design

Competition for Universities

addressing airport design

challenges

Students can address technical

challenges regarding the safety,

capacity and efficiency of the nation’s

airports, offer innovative solutions,

and win cash for outstanding

proposals

Table 5 Examples of Operational type Challenges (extracted from Challenge.gov)

Type of

challenge

Submission

format

Objective Target

Audience

(Participants)

Beneficiary % of

total

Awareness

- Creativity contests

(poster, photo,

video, art, music,

slogan, logo)

- Nominations for

public recognition

- Ideas

Inform, sensitize, educate

on certain issues or

services, gather ideas

General public General public 40%

Research

- Research paper

- proposal

-increase knowledge

about new technology

-spur entrepreneurship

around new technology

-engage broader group

of potential solvers

-inform and influence

policy

Specialists in

particular field

(e.g.

researchers,

scientists,

engineers,

universities)

Specific industry

or sector

14%

Service

- software products

- mobile applications

-encourage use of public

data

-enhance and market

government services

-encourage

entrepreneurship

around provision of

service

- Software

developers

- enthusiasts

- Constituents

- entrepreneurs

26%

Operational

(functional)

- physical product

- software system

- Representation of

solution

(e.g Blueprint,

Prototype,

proposal, plan)

Improve or solve

functional problem

- Scientists

- Engineers

- software

developers

- other

professionals

- Host agency

- Industry

20%

Table 6: Summary Characteristics of the various Challenge Types

20%

Figure 1: Challenge Categories

awareness

40%

operational

services

26%

research

14%

Figure 2: Continuum of innovation levels

Question 1 investigated the types of challenges hosted on the challenge.gov platform.

Analysis and classification of 203 challenges hosted by 47 agencies revealed four distinct

categories, which ranked from low to high in the level of innovation attempted. The next

question investigates the stages and influencing conditions involved in the adoption and

implementation of challenges.

5.2 Research Question 2

How do agencies adopt and implement Challenges?

This question sought to reconstruct the set of events, factors, and conditions involved in

the adoption and implementation of challenges. Responses were extracted from interview data

and were based on informants’ description of their experiences before, during, and after

adoption.

The resulting model (Figure 3) emerged as a representation of the challenge adoption

process. The model outlines the various stages of the process, the factors that influence the

stages, and the relationships between them. The main stages of the model are i) external trigger,

ii) internal motivation, iii) selection of challenge characteristics iv) implementation preparation,

and v) execution and feedback. The factors which influence those stages are a) agency mission

and experiences, and b) enablers/hurdles.

Figure 3: Challenge adoption model

The adoption process illustrated in the challenge adoption model (Figure 3) can be

described as follows:

An external stimulus triggers the process. Depending on the nature of the stimulus, the

agency develops an internal motivation to act. The motivation influences decisions about

challenge characteristics such as type and objective. Agency characteristics such as the mission

and experiences influence the type of motivation as well as challenge characteristics. The agency

makes preparations to implement using either an ad-hoc or the more strategic learn first

approach. Implementation is influenced by the degree to which certain critical factors are

present. When present, these factors act as enablers, while their absence serves as a hindrance.

Challenge execution involves activities such as collection of submissions, judging, and awarding

prizes. The execution experience during one challenge provides feedback for, and influences the

External

Trigger

•

Mandate

•

Exposure

•

Policy

Internal

ation Motiv

•

edience Exp

•

eriment Exp

•

blem- Pro

ng Solvi

lenge Chal

Characteristics

•

Objective

•

Type

Preparation

•

Ad-hoc

•

Learn-first

Execution

Enablers/Hurdles

•

Internal processes

•

Management Support

•

Resources

•

Feedback

Agency

Characteristics

•

Mission

•

Experiences

motivation for succeeding challenges. Further description for each component is detailed

below.

5.2.1 External Trigger

A trigger from the external environment sensitized the adopting agencies to the

availability of challenges as a problem solving tool. While the policy directive was sufficient to

act as a stimulus in some agencies, additional triggers were necessary in others. For example one

informant representing a large challenge program explained how the passage of the America

Competes Act legislation was the primary driver of what they were doing:

A big driver of all the things that we’re doing now, around challenges stems from a

particular piece of legislation that was called the America Competes Act. […].The

America Competes Act gave government agencies the ability to use challenges as a

mechanism to spur innovation. Prior to America Competes, essentially the only way that

a government agency could pay you, in sort of, the broader sense, was either through a

grant, or through a contract. Now with the Competes Act, government agencies are able

to use competitions and prize challenges as a way to encourage people to innovate, to

build applications and thus be rewarded for their innovations. So, a big part of why

we’re running the challenges through [agency name], is that it is specifically focused on

the Competes Act and using the challenge mechanism (Informant 33)

From another perspective an informant recalled wanting to run a particular competition before

hearing about challenge.gov:

We had actually tried to do this competition even before Challenge.gov, we were kind of

sensitized to it (Informant 10)

However in the absence of the special legislation they were unable to run a competition and

award a prize. This changed when America Competes was passed.

…But then we realized that competitions actually fall in a different category, and we

didn’t have authorization to actually have competitions and make prize awards. So then

we had to kind of put that in the back burner. And when the America Competes Act came

along, then we were able to go ahead and really fully realize our vision for the whole

program. (Informant 10)

These are just examples of external triggers. Further analysis revealed three main categories for

external triggers: the legislation and policy, specific directives from outside the agency, and

exposure to the practice in other agencies. Brief explanations follow.

i. Mandate: Following passage of the government wide policy, some agencies received

specific directives to implement. These directives were communicated orally or via

informal communication channels, and in some cases were accompanied by a timeline

within which to act.

ii. Exposure: Some agencies were exposed to the practice in scientific and research

organizations in the business sector, while others witnessed the adoption by other

government agencies. In some cases the exposure happened before the policy and

legislation were implemented, but had to wait for the legislation. In other cases some

were aware of the legislation, but were only prompted to act after seeing it implemented

in other agencies within or outside of government.

iii. Introduction of Policy and Legislation: Before running challenges, agencies must be

legally authorized to do so by an Act of Congress. Prior to 2010 a limited number of

agencies were able to run challenges under legal authority specific to their functions.

Others had done background research and were contemplating ways they could employ

challenges. In 2010 Congress passed the America Competes Reauthorization Act which

granted legal authority to all agencies to run challenges and award prizes, in addition to

guidelines and stipulations for use. When the legislation was enacted it served as an

external stimulus to those agencies that had already been considering it, thus triggering

the formal adoption.

5.2.2 Organizational Mission and Experiences

Organizational characteristics such as mission and experiences were important

determinants in the challenge adoption process. The primary mission types emerging from the

data were (1) service provision, (2) public education and information, (3) regulatory, and (4)

research and development related to science and technology. Some challenges reflected the core

mission of the agency, and were seen as more critical than those addressing topics outside the

core mission. In the following quote, the informant indicated that his challenge effort was

enabled because he was able to link it to work his organization had been doing for years.

I think what helped me a lot was that I was able to link the challenge effort to work we’d

already been doing for years. I think these are really crucial to be rooted in strategic

planning. The organization in my opinion needs to fully understand why they’re moving

in a particular direction, and if they choose new problem solving tools like this, why they

are choosing them. And then to understand where are they applying them for the most

effect. (Informant 12)

In a contrasting situation, an agency whose primary mission was regulatory encountered some

resistance, because their mission did not readily accommodate the level of innovation and change

which challenges were meant to promote.

One of the important pieces that frames all of this discussion about us being a regulatory

agency, there’s an added layer of resistance to coming up with innovative approaches to

our regulations, because it takes a lot to change. So we don’t have some of the same

flexibility as the other agencies do. (Informant 20)

Outside of challenge adoption, different agencies had varying experiences with producing

innovations. The agencies which stood out as having more innovative challenges, also had in

place very clear and established processes for regular innovation. The two quotes below

represent examples of agencies with clearly articulated innovation initiatives outside of

challenges.

We absolutely do have a process. In fact we’ve got a few different ones depending on the

nature of the innovation or the issues that we’re dealing with. Or the constituency from

which we’d like to garner the ideas. For example, if you’re to take the [contest name]

that we run once a year, that’s the process that begins in late fiscal year, let’s say 2011.

We identified challenge areas we want to focus on at a strategic level for the [agency

name] and by February, we’ve effectively settled on a group of 4 to 5 different issues, and

we write them into problem statements that are no more than a few pages long each.

[…] And we use that in order to solicit ideas from industry. That’s not into the prize

challenge, but it’s one of the ways in which we go about soliciting ideas and investing in

innovation. What follows after that is a white paper phase. We take those white papers

and we determine which ones we want to seek full proposals from. (Informant 17)

Internally, one of the programs that I helped develop is a contest called [contest name].

And this is a contest we run twice a year where we invite employees to submit write-ups

about innovations that they have developed, or they have developed in partnership with

contractors. […].We invite our employees to submit innovations, and then we have a

social networking platform where we display the top 20 or so, and we ask our entire

community of employees to vote, comment on which innovations they like best, which

ones they think would be most applicable in their home agency, and then we select

winners and honorable mentions, and we celebrate them at a ceremony that we telecast

across the department, and the winners actually receive cash awards.[…] We’re trying to

show our employee base that these are things that they too could develop, and, try and

generate good ideas, and or showcase things that could be applied elsewhere (Informant

The agency mission and experiences influence the agency’s internal motivation to adopt, as well

as the challenge characteristics.

5.2.3 Motivation

Depending on the nature of the external stimulus, the agency develops an internal

motivation to act. That motivation is critical because it set the tone for subsequent events and the

overall adoption pattern. In some cases there was no real reason for adopting, except that they

were following instructions. In other cases the agency simply sought to take advantage of an

opportunity created by the America Competes Legislation. For example:

So it wasn’t a specific problem that we had, rather it was an opportunity that we had a

lot of data, and a lot of software developers who might want to use it. And, he said let’s

take advantage of that opportunity. (Informant 21)

In other cases there was a deliberate strategic move to address specific areas that needed

innovating:

Coming out of our strategic plan and looking at now having learned about open

innovation tools, we did an analysis of our portfolio of work to see what problems we

might want to address. […]. With any portfolio of work like that, you have gaps,

insufficient resources or knowledge or technologies. So we took those gaps and we

worked with [name] to use this 2 by 2 quadrant and criteria to map our gaps to various

types of collaborative opportunities, and we did a whole workshop with him to do that.

So we selected 12 challenges to potentially run a prize, and then we worked through

certain criteria to see if they would map to the open innovation quadrant. And the ones

that did, we then felt had a high likelihood of success. (Informant 27)

On other cases it was a combination of reasons, for example:

We did a challenge really for two things. One was to understand how to work with

challenges period, because this agency had no experience of working with challenges.

And the other one was to be able to attract interest and attention from communities that

we don’t know of, to become involved in [technology name]. (Informant 15)

The different forms of internal motivation can be classified as follows:

i. Expedience: These agencies were faced with situations where compliance with the

policy or some directive was the ultimate objective. Therefore they chose the easiest and

quickest way to get the Challenge over and done with.

ii. Experiment: Some informants spoke of being exposed to the use of challenges in

practice and were motivated to try it out. At this point they were not sure how or under

what circumstances it would be best utilized, but were excited at what they saw and

wanted to experiment. They usually took a wait and see approach before attempting to

implement any long term program.

iii. Problem solving: Some were able to match this new tool with existing problem areas

which were either previously tackled or were being contemplated. For others it was an

opportunity to address gaps in their strategic plans or to attempt something which they

were previously unable to do. This usually included a long term objective of integrating

challenges into the regular operations of the agency for purposes such as an alternative to

grants and contracts.

5.2.4 Challenge Characteristics

Selection of challenge characteristics refers to the selections made for challenge type and

objective. While this represents two activities, the separation is often indistinguishable in

practice. As mentioned earlier, the agency mission and experiences influence the challenge topic

and type of challenge selected.

Challenge Objective

The challenge objective refers to the problem or topic which the challenge addresses.

Objectives that closely match the agency mission or operations give an indication of high

importance to the agency. Objectives that are loosely related to the agency mission are usually

seen as less important. The challenge objective also gives an indication of the level of innovation

employed. For example, did they just use members of the public to do the same tasks that they

would have done anyway (little or no innovation)? Or did they use it as an opportunity to

produce something which they were not capable of doing before (true innovation)? The ideal

objective as described by one informant was for innovative uses, and not for traditional uses that

have been repeated year after year.

If you want to do something that you typically do, but you just want to use

Challenge.gov, and you already have a process in place, and, you think well

maybe I can speed it up using Challenge.gov, that’s really not going to work.

[…]. But if it’s something related to innovation and we’re seeking to solve a

problem that we have been unable to solve over the past couple years. So

there’s a difference between an expected deliverable that you’re doing year after

year, or an existing issue that there is no deliverable for, because you haven’t

been able to solve. If that’s the case, then use Challenge.gov. If it’s to

reinvent a deliverable of sorts, that might not be the best. Then you really need

to look, inside and think otherwise (Informant 13)

Stressing the ideals of innovation, challenges are most effective when they have an objective of

helping the agency accomplish something they were not able to do before using traditional

methods.

Well, it was easy to make that decision. So, these were gaps where we had tried

other tools. We had tried research grants, or we had tried small business

proposals, or both. Possibly even contracts. These were areas where people felt

they were somewhat stuck. That makes it a little bit easier to be at the end of the

line where people pulled out gaps that they thought were important problems, but

they hadn’t been able to solve those yet. (Informant 27)

The three categories of topic motives emerging from the data are listed below, ordered by

increasing level of innovativeness:

i. No specific problem; introduced mainly for the sake of using the tool ii.

Try to find an easier way of doing some existing task

iii. Do something they were not able to do before

Challenge Type

The type is characterized by factors such as objective, task, submission format, and gives

an indication of the level of innovation involved. Four main challenge types emerged during

analysis of challenge.gov listings: Awareness, Research, Service, and Operational (detailed in

research question 1 above). Challenges with awareness as the main objective represent lower

innovation levels, while those with operational objectives represent the highest level. Awareness

and research challenges are generally considered easier to execute and often associated with

adhoc implementation. Service and operational challenges often require more organizational

effort and can be associated with long term planning and more systematic organizational effort

and commitment.

5.2.5 Preparation

Different agencies had different approaches to challenge implementation, and this was

reflected in their preparations for the process. Preparation involved planned deliberate attempts

to provide structure and guidance at the organization level, and included organization initiatives

such as internal contests, designated personnel, training, and the creation of a framework of

guiding policies. In the final analysis avoiding procedural difficulties all came down to the

organization’s prior preparation and ability to learn from past experiences.

Those with a long term view of integrating challenges into regular operations had a

strategic learn-first approach. They spent time upfront conducting preparation activities to ensure

a smooth implementation. These activities included internal initiatives such as pilots, education

and training, and setting up and documenting internal policies and procedures. Additionally,

some contracted out to private or non-profit entities with experience in that area, freeing them of

time-consuming procedures. The following illustrates:

And so much of the work that I’ve done on prizes and challenges is background policy

work. A lot of what I’ve done is setting up the framework. So, you know, the America

Competes Act was passed a year ago, and agencies were sort of handed this new

authority, and yet we didn’t have any systems. I mean we had like no mechanism for

actually paying our winners, and we had no policies, and we had no framework, so for

example talk about one policy in particular is our delegation of authority. So the

Secretary was given authority to run prizes and challenges. But an organization as large

as ours, it doesn’t make sense to channel everything through the Secretary. So we

developed a delegation authority whereby the secretary delegated that authority to our

agency heads, and then we created another policy where prizes and challenges that are

above a certain monetary threshold go through secretarial review. But anything that’s

smaller would just go through an agency review. (Informant 8)

This type of preparation and arrangement was even more critical for large departments that have

several agencies under their jurisdiction. This next example outlines how one large department

centralized the process to make it easier for their agencies.

So when you have [number] different agencies, if you democratize this process, what you

have is chaos. So, no one knows what the other person is doing, how the process really

works, best practices, minimum requirements, legal jargon, etcetera. So what we’ve

done is we’ve centralized it. We have educated each of our agencies not only on the

potential, but on the requirements, on how they should think about their challenges.

And, they are responsible on most cases for coming up with the idea. But we are

responsible for coordinating and running and leading the management of

implementation, execution, evaluation, and then conclusion, of the challenges. And it’s a

group of individuals that work together to ensure that all are aware. Otherwise, it’s over-

democratized, and you lose control, and then something might get lost. (Informant 13)

The end result of this centralization was described as a fast track process to help agencies under

that department execute a challenge.

What we have provided them is with a fast track process of being able to formulate,

launch, and, and execute the challenges, then conclude them, evaluate, and then award.

It is a formalized process, easy to follow, and you don’t need to be a subject matter

expert. (Informant 13)

Others with a short term focus used an ad-hoc or trial and error type of approach where they

learned as they went along. With that approach there were no internal processes in place and

implementation was not usually a smooth process. For example in the arrangement below,

everyone is left on their own to figure out how a challenge should be run.

There is no one person responsible for all the challenges coming out of the agency. I

think (there are) pockets of one or two people around the agency that want to do a

challenge and then figure out how to get it done (Informant 29)

5.2.6 Enablers/Hurdles

The path taken by agencies during preparation influenced internal readiness to deal with

the rest of the implementation process. Those agencies which took the learn-first approach took

measures in advance, to ensure the presence of enabling conditions, and had a smoother

implementation experience. The quote below identifies some of what the informant considered

important enablers which should be put in place.

If you’re going to run a challenge, make sure to allocate the right amount of resources.

Be ready to invest the time. Be ready to learn how other successful challenges have

been run. Be ready to create compelling price structures, and a list of judges, not just

within the federal scope. And reach out to departments that have done it before and

have done it well. (Informant 13)

And, speaking of specific factors that enabled to successful execution of multiple challenges:

I think just departmental leadership, openness to doing things in a different way.

Looking outside of your sandbox, and saying so this is how things are being done outside

of our department, learning from them. Having an open mind at the, at the highest level.

And we’ve gotten a lot of support from our leadership - the Deputy Secretary is a

champion in this area. (Informant 13)

Agencies which took the ad-hoc approach did not do as much advance preparation, resulting in

less availability or absence of enabling conditions. The absence of an enabler (or its availability

in lesser quantities) is viewed as a hindrance.

The list below identifies the major enabling conditions emerging from the data:

i. Internal processes: This refers to the existence of an internal mechanism with

clearly identified and documented procedures and guiding policies to run

challenges. It is a reflection of the extent of process knowledge within the

agency and covers activities such as: a) weeding out non-viable challenge topics,

b) allocation and payment of prizes, and c) obtaining legal compliance.

ii. Management support: This is a reflection of the level of support displayed by

leadership of the agency for the challenge initiative. The level of support was not

equal across agencies. Some agencies received visible support from the highest

levels such as the Secretary or administrator, signaling the agency’s commitment

to supporting all aspects of the program. Other agencies received passive

endorsement with no visible support, or agency-wide commitment for the

initiative.

iii. Financial resources: The larger challenge programs had dedicated financial

resources which could be used to offer larger prize purses, thus attracting higher

levels of innovation. Financial resources also facilitated the hiring of external

contractors and partners with expertise, removing the burden of challenge

management and implementation from employees. In some agencies funds were

allocated to run internal pilot programs, which enabled the learning process and

promoted the use of challenges. Financial resources were also more likely to be

present among agencies with ongoing research and development or innovation

portfolios.

iv. Human resources: some agencies had a designated challenge manager for whom

implementation related activities were a priority. In other cases, duties had to be

absorbed by Program or Communication Officers, who performed them in

conjunction with their regular duties. The adoption and implementation

processes were described as time-consuming and burdensome, and influenced

future decisions to execute challenges.

5.2.7 Execution and Feedback

Execution follows the preparation stage, and the execution experience is influenced by

the presence (or absence) of process enablers. Execution covers all necessary activities needed to

complete the Challenge. Activities include publicity, hosting on platform, judging, and awarding

the prize. Publicity refers to activities related to the promotion and marketing to the public and

target audiences using traditional and new media channels. It may also involve promotion to

agency employees with the objective of obtaining organization buy-in. Next, the challenge is

hosted on the challenge.gov platform, and in some cases agency websites of third party providers

or implementing agency. This activity involves technical actions associated with uploading and

managing on the platform, as well as responding to queries from the public. Judging involves the

selection of judges, and ensuring that the judging criteria are adhered to. Following judging, an

award is made to the winning submission, accompanied by a certain level of publicity.

Given that the program was new, the execution experience was not always

straightforward, leading to sometimes difficult execution experience as recounted in the quotes

below:

None of the early challenges have been easy, simply because they’re new. One of the

things that happens when you have a new tool is people are scared. People think about

everything that could go wrong, they think about are we complying with every regulation

and rule we’re supposed to. And so I mean the approval process to do this was

complicated, not because the challenge was inherently complicated, but because it was

one of the first. (Informant 8)

I think you need to have somebody who is devoted to being in charge of that. Like you

need to have a clear project manager to deal with the legal, to deal with judging, to deal

with the submissions- just the administrative tasks that have to be done in a challenge.

Challenge.gov is nice because you can collect all of your submissions and things in one

place. But you still have to have judges go in and judge them. And you have to find

judges and coordinate when they’re judging. So, there’s a lot of administrative work that

somebody needs to be devoted to doing. (Informant 22)

Following completion of one challenge, feedback from the experience will influence the

agency’s motivation for subsequent challenges. This is represented in the model by a feedback

loop. For agencies with an initial goal of running multiple challenges, the early experiences were

used to fine-tune internal processes and conditions, contributing to smoother implementation for

subsequent challenges. For others, feedback from the first execution experience helped

determine future direction of the challenge program. These agencies were able to learn from

early experiences and made appropriate internal adjustments, enabling them to replicate the

process later on. The following quote illustrates the numerous difficulties that were associated

with selection of judges. However once the appropriate steps were covered, this experience was

used as feedback to inform a department policy to guide future challenges.

One of the issues that came up in this challenge was the issue of judging, and who was

allowed to judge a challenge. And we have fixed rules that state that inherently

governmental functions must be done by the government, and if they’re not done by the

government, then they need to be done by special government employees […]. And one

of the difficulties with this first challenge was with selecting who be the appropriate

judges. And so that just took a while. But I mean the benefit is once we went through

this, then we were able to come up with a policy for the department around how we select

judges. (Informant 8)

Some agencies lost interest in continuing the program:

...there was a long gap, and there was some interest in starting up some challenges again

here. I think they still maybe on the table, although I don’t know where they are in the

process. (Informant 35)

Others lacked the desire or ability to make the necessary adjustments, and abandoned the idea of

future challenges.

5.3 Research Question 3

How do the various adoption and implementation factors influence the type of innovation

enactment of the implementing agency?

5.3.1 Types of Enactment

The challenge adoption model presented and described in Figure 3, represents the various

stages, factors and conditions that agencies navigate during adoption and implementation.

Following Orlikowski’s (2000) typology of enactment introduced in Chapter 2, I categorized

agencies based on the level of change stimulated by challenge adoption. This resulted in three

agency groupings characterized by the following descriptions:

•Inertia retains the existing way of doing things. It represents the lowest level of

innovation because it signifies no change in organizational practices.

•Application augments or refines the existing ways of doing things, and demonstrates

moderate innovation levels.

•Change substantially alters (or transforms) the existing way of doing things. This

represents the highest level of innovation due to the substantial changes in existing

practices.

For each group I demonstrate how the enactment type is related to specific conditions at each

stage. For example inertia enactment is associated with a specific type of external trigger

(directive), a specific type of motivation (expedience), specific type of challenge objective (no

specific problem), and so on. The conditions for each enactment type are illustrated and

supported with examples from interview transcripts.

5.3.2 Inertia Enactment

This form of enactment evoked little or no change in the agency’s approach to problem

solving and innovation. Agencies within this group were responding to a specific directive to

adopt, or external pressure to conform. They primary motivation was expedience, leading to the

selection of challenge objectives and types they felt were the easiest and quickest to execute.

There was minimal upfront preparation, leading to an absence of critical enablers such as internal

processes and funding. This led to numerous difficulties during execution, making repetition

unlikely.

5.3.2.1 External Trigger

During the early days of the Open Government initiative, there were formal

communications to agencies strongly urging them to employ prizes and challenges as a problem

solving mechanism. There was also informal marketing and promotion from the White House

Office of Science and Technology Policy (OSTP), using word of mouth and personal

communication channels. This informal evangelizing sometimes resulted in specific directives to

individual agencies to undertake a challenge project, usually within a relatively short time frame.

This directive was sometimes relayed directly to program staff, and in other instances relayed

through the agency head. The following quotes illustrate examples of both.

we were told that we needed to do some type of event, and that we had a few months to

pull it together, and that we had to use challenge.gov to do it. (Informant 34)

it came about in a very straightforward manner. …the then Technology Officer told us that

he wanted us to do a challenge. And he wanted it out in two weeks (Informant 30)

We were told to use a challenge… it came down that Challenge.gov is launching; we need

to have challenges (Informant 22)

These mandates coming from outside the agency triggered a specific type of motivation, which

influenced subsequent implementation events.

5.3.2.2 Organization Mission and Experiences

The agencies with missions related to public outreach often selected awareness

challenges, while those with research missions deployed ideation type research challenges. Both

of these types fall on the lower end of the innovation continuum for challenge types. The

challenges executed in this group were not seen as central or critical to the agency mission, and

in one case the challenge was contradictory to the mission.

We provide leadership, we don’t generally provide implementation. Implementation is

supposed to be a job for the states, not of the national office. So this was something kind

of outside of our agreed responsibilities in order to do this. (Informant 34)

In terms of experiences, these agencies did not indicate any elaborate ongoing innovation

processes.

5.3.2.3 Internal Motivation (Expedience)

Directives given with a tight timeframe led to hasty selections. At this point the focus was

on complying with the directive, rather than making strategic choices about problem solving or

innovation. For example, informant 22 who was responsible for executing a video contest

admitted that this was the only idea they could come up with at the time:

And so it was told us that we needed to do a challenge, and to think of a challenge, and

that was what we came up with (Informant 22)

Similarly, informant 34 who executed a photography contest admitted that they were not dealing

with any specific issue, but rather complying with the directive that had been given:

Ours really wasn’t that issue based. Challenge.gov is something that we were told from

higher ups within the government that they wanted used (Informant 34)

The internal motivation was very closely tied to challenge objective, and by extension the

challenge type.

5.3.2.4 Challenge Characteristics (objective and type)

Owing to the random manner in which these challenge projects were selected, there was

not sufficient time to be strategic about selecting problems or issues to advance the mission or

work plan of the agency. As a result the objectives were neither central nor critical to the work

plan of the agencies, and did not invoke high levels of innovation. For example, Informant 34

who admitted that their photo contest was not trying to solve any specific problem:

…So we weren’t looking at an issue. We came up with the only thing we could think of,

and that was to do a photography contest. (Informant 34)

Another admitted that they had no idea what to with a challenge and struggled to find an

appropriate topic:

… so we said so what should we be doing with a challenge. We think it’s a useful tool, but

we’re not exactly sure what problem fits it (Informant 16)

There is a very close relationship between the challenge objective and challenge type. The two

are interrelated and often the choices are made in conjunction with each other. The types of

challenges implemented were geared towards awareness (see description in research question 1)

which include creativity contests such as video, poster, photo, slogans, etc. These invoke the

lowest levels of innovation. For example one informant indicated that their poster contest did not

involve any form of innovation since it could have easily been done internally:

I just would have had our graphic designer do it. I mean we have those resources

available (Informant 22)

There were also two research type challenges in this group however these were on the lower end

of the scale because they were mainly looking for ideas on how to deploy different technologies.

5.3.2.5 Preparation

The approach to implementation is an indicator of whether the agency intends to build a

sustainable challenge program, or whether it is intended as a one-time event. Within this group,

agencies proceeded in an ad-hoc manner, and went straight to implementation once challenge

objective and type were identified. They did not make provisions for advance preparation and

experimentation, and actions did not demonstrate intentions of developing a sustainable program.

As a result, informants in this group reported an absence of the critical enablers, discussed in the

section below.

5.3.2.6 Enablers/Hurdles

One of the biggest and most common hurdles reported was the absence of internal

processes to guide challenge implementation. This was very closely related to a lack of

knowledge both around the challenge process, and to a lesser extent the requirements of the

platform itself. Challenges in this group were implemented in the early days of the Open

Government initiative, and there was little institutional knowledge across the agencies. In the

absence of any guiding mechanism or prior training, those charged with the task struggled

considerably with the process and were forced to learn while doing.

One informant explained that it had taken over a year to figure out the mechanisms for

internal procedures such as disbursing money and awarding prizes.

We had to figure out an internal mechanism for making it happen. And that’s taken over

a year. So to get that really down right, and then to get procedures for disbursing the

money, for granting the prizes, for reviewing the different proposals, to do it right takes

some time. (Informant 30)

Another explained that the process was not systemized within their agency:

…when I heard of it (challenges) it wasn’t systematized at all (Informant 35)

Another common hurdle reported from this group was that running the challenge came with

additional responsibilities which had to be absorbed by current staff. Because it was mandated,

some felt that responsibilities were just thrust on them. They described challenge responsibilities

as time consuming and burdensome. Informant 34 talked about the personal experience of having

to work overtime to get it done.

We weren’t given any funding. It was something we had to add to our current job

responsibility. I had to come in on the weekends in order to work on this project, because

I just couldn’t fit it into my regular assignment. (Informant 34)

Another key informant put it in general perspective:

The lack of dedicated resources is a big one. This is pretty much an unfunded mandate or

unfunded option for people to use. And so it ends up being an additional task that

somebody takes on in addition to their other responsibilities. I can easily see it being a

full time job for one person, at least for an agency. (Informant 35)

The lack of certain key enablers made for a difficult implementation experience and reduced the

likelihood of agencies running repeat challenges.

5.3.2.7 Execution and Feedback

For those agencies with awareness challenges, they had run one or two challenges, with

no immediate plans to repeat. In response to the question whether there were plans for more

challenges, informants indicated that repeats were unlikely for various reasons stemming from

lack of fit with agency mission, changing priorities with departure of an administrator, and

seemingly a loss of interest. Some of the reasons are illustrated below:

Question: Do you plan on running any more challenges?

Response: Not unless we’re given another mandate to do one. (Informant 34)

In the case referred to by informant 22, there were plans on the way which included funding to

do additional challenges. However with the departure of the administrator, the plans as well as

the funding were put on hold and so the agency did not run any more challenges. The following

quote illustrates:

So there was money put aside. An idea was that that money would be used across the

agency to support challenges. And that the agency had a goal of doing, think it was 20 or

25 challenges in the next year. But I don’t think that that’s a goal any longer, since that

administrator is no longer here […]. So if that administrator is no longer here, and that

money is no longer being set aside for challenges, I think that was part of the concern.

[…]. So no, we haven’t done any other challenges (Informant 22)

For the research oriented agencies, there were no initial plans to institute more challenges.

However, there was now the potential to work on setting up internal processes to help enable

future challenges.

5.3.2.8 Inertia Enactment Summary

The following table provides a summary for characteristics of inertia enactment.

Inertia Enactment

External trigger -Directive

Internal motivation -Expedience (choose the easiest and quickest way to get this done)

Challenge objective

and type

-Problems were not very critical, not taken directly from work plan

-Types: Awareness and research

Agency Mission and

Experiences

-Mainly public outreach, some research oriented

- No elaborate innovation processes

Approach to

Implementation

-Did not spend time to build internal processes

-Encountered difficulties during implementation

Enablers/Hurdles -Lack of dedicated financial and human resources

-Lack of internal processes

Execution and

Feedback

-Encountered difficulties during execution

-Abandoned idea after one or 2 challenges

Table 7: Inertia Enactment Summary

5.3.3 Application Enactment

Application enactment refers to a situation when an agency uses challenges to improve

(but not fully transform) existing processes. For example, the development of a software

application to make the agency’s data more accessible to the public can be viewed as a form of

application. However if this approach is used inconsistently, it has not caused a fundamental

change in the way the agency solves problems or innovates, and does not represent change or

transformation. Adoption in these agencies started out as capitalizing on an opportunity to try

this new tool and matching it with a suitable problem/issue. Another form of application

enactment involved the use of challenges to improve a task that was previously performed using

another means.

We had tried to operate something similar to these before Challenges. So we had some

structure and rules in place which the General Counsel helped us develop. So we just

adapted them for the challenge, that we retain ownership of the submissions and stuff like

that. (Informant 4)

Adoption did not usually start out with a plan for long term integration. However a positive

learning experience could lead to the continuation of the program while the opposite led to its

termination.

5.3.3.1 External Trigger (Exposure)

A common thread echoed by informants in this group was that outside of the policy and

legislation, they were moved to act by some form of encounter or exposure to challenges in

practice. Some of these interactions occurred while attending formal challenge events and

demonstrations. For example informant (4) recalled attendance at one such event:

Well quite a few of us have attended a session sponsored by the White House talking

about challenges. So we brought that message back (Informant 4)

In addition to attendance, the conversations and personal interactions that followed were

instrumental in making an impression. Informant (32) recalled one such interaction:

Robynn Sturm at OSTP came and talked to [my group] maybe a year ago, and I’ve had

many telephone conversations with her, to kind of clarify issues. And so she was very

helpful in educating me on how to do this. (Informant 32)

As part of the interactions, there were conversations about implementing specific challenges.

However in contrast to the inertia group, these were described by informants as suggestions

rather than directives, because these were tied to specific objectives. The following quotes

illustrate:

The idea was suggested by Aneesh Chopra, who was at the time the Chief Technology

Officer of the United States. He suggested that we do this challenge. (Informant 21)

Right after that conference, some colleagues and I, went and we met with Todd Park

(OSTP), because we loved this idea of opening up data and making it available. And so

he said well why don’t you consider doing a challenge? (Informant 12)

This exposure or personal encounters triggered an interest within the agency, motivating them to

act.

5.3.3.2 Organization Mission and Experiences

This represented the largest group of agencies representing varying missions and

operation. This group was not as distinct as the inertia and change groups. Many had operations

which produced large quantities of data related to public issues such as health, education and the

environment.

5.3.3.3 Internal Motivation (Experiment)

Exposure to the challenge program triggered the desire within agencies to experiment.

Program officers and managers were curious to explore to see whether it fit their needs. For

example informant 11 spoke of the agency’s desire to try a challenge because they had not done

one before:

Yeah, the whole point was like we need to do a challenge. We haven’t done a challenge at

all. We haven’t leveraged this platform. (Informant 11)

The desire to do a challenge preceded the identification of how exactly it would be used.

It was just getting at that’s what we’re going to use. So I have a little bit of a different

history with challenges. I actually remember the very first apps challenge done here in

D.C., called [challenge name]. I even know the guy [name] who did this from his

marketing firm, who ran the first one. So I’d always known about challenges, knew about

challenge.gov. So this was like, hey we need to do the same thing, we need something like

that, and so let’s run something. (Informant 29)

However the motivation got a boost when they were able to capitalize on the opportunity and

match the tool with an existing issue:

Well we looked at what we were trying to get, what our end result was going to be, and

we thought it was an effective tool, rather than us trying to build some tool inside to get

what we need. (Informant 4)

5.3.3.4 Challenge Characteristics

Stemming from the desire to experiment, some challenge objectives were not related to

specific problems, but were used as an outreach and engagement tool. However this form of

engagement required tasks which were slightly more complex than awareness challenges. This is

illustrated in the following example:

So our goal was really to excite the students. So we were not really so much interested

in the end result, even though that’s a desirable outcome. But we were really interested in

the process of engaging the students to push themselves and to expand their boundaries

and get them excited about the topic…. (Informant 10)

Others saw the opportunity to go beyond engagement, and incorporated an issue which needed

addressing. For example, as part of the Open Data initiative, agencies were required to make

their data publicly available. This opened up new opportunities for creating software applications

to enable public consumption of the data, as well as opportunities for entrepreneurship. The

following quotes illustrate:

So it wasn’t a specific problem that we had, rather it was an opportunity. We had a lot of

data and there were a lot of software developers who might want to use it. And he said

let’s take advantage of that opportunity. (Informant 21)

more than anything else was just marketing the fact that we did have a set of web API,’s

or web services that was available. There wasn’t really a problem, it was more used as a

marketing tool (Informant 11)

The types of challenge were mainly service involving software applications, but also included

some research and a few awareness types.

5.3.3.5 Preparation

The decision to run a challenge preceded the selection of a problem or issue. Adoption

was seen as an opportunity to experiment with something extra, not necessarily critical to the

internal operations. Similar to inertia enactment, initial challenges undertaken by an agency

focused on ad-hoc and stand-alone issues. Implementation was not preceded by long term

preparation and learning, but rather was more improvised. Challenge champions had to take the

approach of learning by doing, and had the task of convincing management and colleagues.

In setting up the first big challenge the [agency name] did, I am very inclusive and I

brought everyone to the table to understand what I was doing, so I brought in people from

our Office of General Counsel, I brought in people from our Policy Office, to let them

know exactly what it was that I was planning on doing, and for them to be able to give me

any input, or to share any concerns they might have.[…] So during all the planning of

the [name]competition, I had all of the stakeholders in conversation with me about what I

was doing. (Informant 32)

Some organizations were able to learn during the process and use that new knowledge to inform

future attempts.

5.3.3.6 Enablers/Hurdles

Initiatives were bottom-up and decentralized, resulting in an absence of official

organization policies, guidelines, and training initiatives. In the absence of a guiding framework,

challenge managers and champions were forced to learn as they went along, encountering many

hurdles along the way.

Lack of Internal Processes

With the practice still emerging, many agencies lacked documented instructions on the

mechanics and sequence of tasks necessary for achieving milestones such as getting the project

approved, and the allocation and disbursement of cash prizes. Several challenge managers spoke

of uncertainties, and recounted having to learn through trial and error with no one to turn to for

help. While some saw these as part of the learning process, others blamed it for lengthy delays in

execution.

…the first time anyone in our agency had ever done one of these, and… part of me was

(thinking) I don’t know what I’m doing, and I don’t know who to ask for help. So we just

kind of figured it out as we went along (Informant 23)

it was a very big ordeal, because we didn’t even know who had to approve what, and so

the procedure was not defined. We didn’t know if it could be approved at our agency, or

the head agency. So, this was really challenging. (Informant 10)

Though the process for the first challenge was extremely difficult, the experience improved with

subsequent projects as agencies were able to learn and improve. Thus agencies were now able to

set policies and guidelines based on earlier experiences, making it easier for other individuals

and internal implementing units. This is illustrated in the account given by informant 10:

I was one of the first at [agency name] to do a challenge, and the [agency head office]

really worked with me, and that kind of set precedent. And that’s how they established

rules for challenges. So now there is a very organized way to help people who want to do

other kind of challenges, to guide them. (Informant 10)

Absence of dedicated staff resources

The challenge effort was initiated by curious program managers and professional staff

who acted as pioneers within their agencies. They voluntarily took on duties of challenge

manager or champion, in addition to their regular duties. However they soon realized the extent

of time commitment required:

From the first thought, it wasn’t clear to me how much time commitment this takes on the

challenge manager’s part. But it really takes time from establishing the competition and

running it, and staying on top and answering the questions, and making sure that the

website works properly and all these things. So that would be one thing that I would

really make sure somebody who’s considering a challenge is aware of… (Informant 10)

In the absence of an appointed challenge manager, the situation was improved with someone who

takes on special projects or innovation:

having someone who’s willing to experiment like myself, is important because that’s kind

of my job is to take on special projects (Informant 32)

Management endorsed but not management driven

Challenge champions were often successful in educating top management and their

colleagues, resulting in management endorsement. While management was generally supportive,

that support was limited to endorsement rather than driven from the top.

Our associate administrator was very enthusiastic about doing this from the beginning.

And so beyond that, we had to do some education, even up to the administrative level, just

to let them know what we were doing (Informant 11)

Though not common, there were a few cases where risk-averse management hindered the process

of challenge execution.

Before we launch a challenge, we put it through our whole vetting process, all the way up

to the, the head of the agency, to make sure that they’re okay with it. (Informant 4)

and continued…

It’s just like they’re afraid, they like all the safety with the grant competition. There are

more steps and more checks or balances. And they’re afraid of what’s going to be posted,

what’s going to be submitted, especially with public voting they’re a little nervous.

They’re more nervous the less control we have over the results. They know that we’re

supposed to be turning towards challenges, and we’ve structured these in a way that

we’ve gotten it past them. It’s been challenging to us internally. We can’t give out cash

awards outside of the grant mechanism, is a view of our legal office. (Informant 4)

Lack of funding for cash prizes

Because of the spontaneous manner in which these challenges emerged, there was often no

funding in the current budget allocated for cash awards. However, there were positive reports

that challenges could be run successfully in the absence of monetary prizes.

So there’s no monetary…we didn’t have funding for this. And, the experience we had

with the [challenge name] challenge was that if you do it the right way, you don’t need

money. That people are using this because they want to be part of something larger, it’s a

resource for them. And having the recognition from the White House makes a big

difference. (Informant 5)

5.3.3.7 Execution and feedback

The hurdles encountered during implementation made the experience difficult. Some

agencies were able to learn from the process and make improvements for the future. However

without all the required enablers, the rate of adoption was still intermittent and did not cause a

transformation in the agency’s existing practices. Other agencies were not able to learn from the

initial experiences, and abandoned after the first few initial attempts. In response to the question

of whether there were any challenges that had been planned prior, or were there future plans, this

informant responded in the negative for both questions:

This is our first foray… we don’t have any (challenges) in the queue right now.

(Informant 5)

5.3.3.8 Application Enactment Summary

The following table provides a summary for characteristics of application enactment.

Application Enactment

External trigger -Exposure

Internal

motivation

-Experiment (curious about challenges)

Challenge

objective and type

-Problems were not very critical, not taken directly from work plan

-Service, research

Agency Mission

and experiences

primarily service

Approach to

implementation

-Did not spend time to build internal processes

-Encountered difficulties during implementation

-After running the first or second challenge some agencies decided to

build organizational procedures so program could be continued

Enablers/Hurdles -Absence of internal processes

-Lack of dedicated human resources

Execution and

Feedback

Two sub groups

- One group experienced characteristics similar to inertia, and

abandoned after a small number of challenges

- One group learned during the process, and have started to

institute internal processes for program continuation

Table 8: Application Enactment Summary

5.3.4 Change Enactment

This is the most innovative use of challenges, as it enables transformation of the agency’s

approach to problem solving and innovation. Challenges implemented in this category were

problem-driven and initiated at the program level, with a long term focus on integration as a

problem solving tool. Generally these initiatives were strategic, and linked to the agency

mission. Many of the agencies within this group had existing Research and Development

portfolios and budgets, and were used to engaging external providers for solutions. Some had

previously experimented with challenges and internal pilot programs, enabling a smooth

implementation process. There was a high repetition rate with agencies in this group averaging

over ten (10) per year.

5.3.4.1 External Trigger (Policy/Legislation)

Introduction of the Open Government Initiative and subsequent passage of the America

Competes Reauthorization Act acted as the primary trigger within this group. Through

interaction with industry, they became familiar with the emerging trends involving open

innovation and crowdsourcing and had an early start. Based on their early knowledge of the

mechanisms, some of these agencies ventured to pilot innovation initiatives even before the

passage of the America Competes Reauthorization Act of 2010. Thus while the policy and

legislation was the primary external triggers, these agencies were poised to adopt because of their

knowledge and history with other open innovation practices.

We started Open Innovation in 2006, the first government agency to do so. Before that

Open Innovation was primarily done in industry. We started crowdsourcing in 2006/2007

(Informant 2)

And later continued…

We started much earlier (than other organizations). We understood, we reached out to

Proctor and Gamble to understand what worked and what wouldn’t. We benefitted from

experience. Plus the push from OSTP. (Informant 2)

Well in parallel, we became aware of open innovation techniques, case studies through

Harvard Business School. And, we thought that shed our need to open up our problem

solving space. And so, we did pilot projects back in 2009, 2010, using open innovation

platforms. (Informant 27)

There was a push on in 2004 to get [agency name] the ability to do these kinds of things.

The people recognized back in the 1999, 2000, 2001 timeframe that this was an

interesting pathway that was not available to federal agencies. And so there was about a

3 year process that [our agency] went through to build the case, and present it to

Congress to get the authority to do these kind of competitions and award cash prizes.

(Informant 26)

Others that did not run early pilots were poised to respond to the policy by matching the

challenge tool to appropriate problems. This is illustrated in the following quote:

So, we were trying out the use of the Prize Authority and the America Competes

Act for the first time. (Informant 17)

5.3.4.2 Organizational Mission and experiences

While the policy was directed at all federal agencies, only a small number were able to

respond in a strategic and deliberate manner. These agencies were poised to respond to the policy

pronouncements due to previous experiences with pilot programs, or their active and anticipatory

pursuit of new approaches to problem solving. Some had major research and development

portfolios and it was common practice to engage the services of external partners, providers, and

contractors.

Organizational missions covered areas related to research and development primarily in

areas related to science and technology. Due to the strategic selection of challenge objectives,

they were closely linked to the agency mission and organizational strategy. For example

informant 26 explained the deliberate alignment with agency mission and strategy:

First and foremost it has to be something that was relevant to our mission. Things that

would bring about new capabilities that we don’t currently have, or any existing

capabilities (Informant 26)

That particular challenge is directly connected to the strategy that had been developed.

There’s direct connection for the output from that particular challenge to be fed into a

future project (Informant 26)

Linking with mission and operations was not limited to science and technology agencies. Two

agencies with a regulatory focus were able to use their innovation portfolios in soliciting

software application challenges to empower their constituents to make decisions based on their

policies and legislation.

We do our own research and development, and issue grants through various mechanisms

to outside investigators, and we use contracts for technical development, depending, and

small business proposals as well. So we had used a variety of tools, we always have, for

new ideas. (Informant 27)

5.3.4.3 Internal Motivation (problem solving)

Generally these agencies embraced the policy, as they were motivated by the real or

perceived advantages of challenges in problem solving and innovation. Some had positive

experiences from early pilot programs and easily embraced the new policy. Others had found

existing methods inadequate in fulfilling their innovation needs, and embraced the alternative

provided by the new policy.

Agencies that frequently outsource to external contractors usually draw from the same

regular pool of bidders with similar background and skill sets. Challenges offer a way to reach

wider audience with diverse backgrounds, offering the possibility of revolutionary approaches to

problem solving and innovation. For example one informant spoke of the desire to reach outside

their usual list of contractors to get a more innovative approach to problem- solving:

We’re interested in getting as many options demonstrated for consideration and ultimate

use, as possible. And, we were looking for some out of the box solutions, trying to be

different than what we would get from our typical group of people who propose to our

contract and grant opportunities. (Informant 26)

And continued…

The reasons that we decided to use the challenge as opposed to setting up some other

mechanism, is that one could envision that there’s a whole lot of different ways that one

could design a software for [the solution]. It was deemed to be a good candidate for

doing it as a challenge, rather than doing it by contract, where you would basically be

forced to choose amongst all these possible ways of doing it. When you really don’t have

a good basis for making that particular choice and you only get one possible solution at

best for the amount of money that would be invested, as opposed to potentially multiple

solutions. (Informant 26)

Some attempts at innovation require a more open ended approach than the typical

requests for proposals (RFP) allow. It may be more fruitful to simply state the desired end results

without specifying the approach. Informant 17 spoke of a situation where the typical RFP would

have been too restrictive because they did not know how the problem should be solved.

Informant 17 articulated that challenges provided the perfect approach for this level of flexibility,

since it absolved them of obligations to pay if they did not get the desired end result:

We didn’t know how to solve the problem, and not such that we could have

created RFP, where we were confident would have been the right way to solve

the problem. We knew the problem that needed to be solved was […]. What

we didn’t know was what the best way to go about doing that. And so we

wouldn’t have been able to say to industry here’s what we’re looking for […].

Instead we just said here’s the end result, and, the beauty of the prize challenge

is that you only pay for results. If you’re just interested in the result, the prize

challenge is perfect for this. (Informant 17)

Also compared to procurement through RFPs, informant (18) underscored the potential

cost savings when using challenges, because you only pay for the outcome:

First of all, imagine the dire alternative which is that we would have put out

through a conventional procurement process an RFP, that would have been bid

on by dozens of vendors, one of whom we would have picked in the hope of

being able to actually execute the expectations. And they would have or they

wouldn’t have, but it would have been a lot more expensive, and taken a lot

longer. Instead, the government takes some risk in this, in the sense that we

don’t even know if anybody’s going to respond to the contest. But we

minimize the possibility of a failed outcome after a lot of money has been

spent... We know what the outcome is before we pay. And we’re paying a lot

less than I think anybody would have estimated if we had gone through the

more conventional process. (Informant 18)

5.3.4.4 Challenge Characteristics

Agencies within this group were not forced into compliance with the policy, but rather

self selected based on their innovation needs. Problems were therefore strategically selected and

objectives were skewed toward problem solving and innovation. Agencies within this group had

research and development portfolios, and had a common practice of reaching out to external

contractors. Some objectives had very practical application such as the development of tools or

technologies:

We’ve run a number of different challenges across a wide variety of topics. These include

tools or apps or technologies that we want to have developed. (Informant 9)

The main challenge types in this category were service and operational. The service challenges

solicited software solutions such as mobile and institutional applications meant to provide a

service to the agency’s constituents. The operational solicited development of solutions more

directly related to the core operations of the agency and usually involved some form of scientific

or technological innovation.

5.3.4.5 Preparation

One of the major distinguishing factors of this group was their strategic approach to

implementation. The first step was the basic understanding of the nature and role of challenges

as articulated by informant (Informant 27):

I think these are really crucial to be rooted in strategic planning. That, the organization

in my opinion, needs to fully understand why they’re moving in a particular direction,

and if they choose new problem solving tools like this, why they are choosing them. And

then to understand where are they applying them for the most effect. (Informant 27)

They pursued goals of long term integration by first setting up internal policies and guidelines, in

direct contrast to ad-hoc trial and error approach used by other agencies.

What we’re trying to do is more systemic and long term, where we’re taking advantage of

the support from top leadership, but also trying to do cultural change so that prizes and,

and competitions become part of the routine set of tools that one considers in order to

accomplish the task that one is assigned. In the long run, eventually you’d like everybody

to think, do I want to do this as a contract, or as a grant, or do I want to use this as a

prize competition. (Informant 26)

They used a variety of preparatory initiatives and events such as pilot programs, staff education

and training, and documenting procedures. The positive impact of those preparatory steps is

evident from the enablers described below.

5.3.4.6 Enablers/Hurdles

The implementation experience was aided by presence of a variety of enablers; the major

ones being internal processes, management support, and resources.

Internal Processes

Early pilot projects were used as learning experiences to help the lay out an internal

mechanism to guide implementation. For example informant 26 described how his agency

collected data from pilot programs to inform decisions and provide guidance on use of the tool.

There were about a half a dozen challenges that were initiated with the specific intent of

trying to work with different sizes of organizations, different sizes of prize purses. We

collect data that you could use to make decisions later on, and for wide dissemination

and utilization of the tool. (Informant 26)

As challenge programs grew, project ideas were being submitted from different parts of the

agency. Not all ideas were viable, and failed projects could mean wasted time and money. Some

agencies instituted internal mechanisms to weed out potential failures, as illustrated in the

following quotes:

Not every problem is meant for a Challenge. Agencies have failed when they have not

done upfront homework (Informant 2)

Vetting process is used to weed out inappropriate problems. We stopped several because

they were not viable (Informant 3)

We have several layers of review. I give it that initial eye test, or schedule, if the idea is

completely unworkable, then I will ask that submitter to put a little bit more thought into

it, and respond to a specific area depending on what they send me. After that, there is a

steering committee made up of individuals from numerous different units within the

agency. After that, we can go into full challenge development, and make certain key

people at [agency name] are aware of what we’re doing. We need to make sure that it’s

not something that the agency is already trying to do. So we take in all that feedback,

and then finally the challenge is approved by [agency head]. (Informant 9)

For some leading agencies the internal implementation process could be easily identified and

articulated:

The first process is problem definition. The second process is prize feasibility, and the

third process or stage is prize design. So you don’t even really get to prize design until

you’ve answered two questions. First, what problem are you trying to solve, and second,

is this problem suitable to be solved through a prize? (Informant 28)

Management Support

Informants spoke of receiving support from the highest levels of their agencies and the

importance of that support in moving the program forward.

We have a high level executive understanding and support of prize competitions, which is

very, very important, and not all agencies have. (Informant 28)

Positive effects of this support were even more evident when management was directly involved

in implementation decisions:

We brief top level management at the broad level on the plans for the year. However we

do not have to get approval for individual challenges. I can make the decision.

(Informant 2)

Now in my case, I’m a senior manager at [agency name], and so after securing the

funding, it could be my decision to go ahead. (Informant 27)

By contrast, the following quote from informant 9 illustrates how the lack of leadership support

in a partner agency hindered the whole project:

You definitely have to have leadership that is willing to move forward with the challenge

process. We worked with another department on a challenge for about a month, and it

was pretty well developed. But finally, it appeared that their Secretary’s office got cold

feet and wanted to pursue solutions through more traditional means. So actually, you

have to have buy-in at the executive level. (Informant 9)

Resources

Agencies executing change enactment had established challenge programs which were

running in excess of ten (10) challenges a year. They utilized services of external contractors and

non-profit organizations to manage some or all aspects of the implementation:

…part of what the contractor does for us, is handle a lot of the challenge

logistics, and running the review panels that we use to evaluate the

submissions. They already had a platform when they won the bid. And their use

of that platform eases the whole panel management process (Informant 9)

From a contractor’s perspective:

As part of our relationship with [agency name], not only are we running the competitions

for them, but we are also advising them on areas where they can pursue competitions and

they can run competitions (Informant 33)

Financial resources also enabled them to have a dedicated challenge manager to coordinate all

implementation events.

5.3.4.7 Execution and Feedback

The challenges in this group generally addressed mission related issues. Early

preparation and a strategic approach paved the way for presence of internal processes and other

critical enablers. This led to smooth implementation, prompting greater frequency in launching

challenges. These agencies had on average over ten (10) challenges per year and advanced

towards integration in regular operations.

5.3.4.8 Change Enactment Summary

The following table provides a summary for characteristics of change enactment.

Change Enactment

External trigger -Government policy and legislation to introduce this new tool

Internal motivation -New way of addressing existing problems

Challenge objective

and type

-Objective: Directly related to mission/work plan, involved R&D -Type:

Operational, Service

Agency Mission and

Experiences

-Mission: Emphasis on Science, Technology, R&D, innovation

-Experiences: pilot innovation programs

Approach to

implementation

-Went through process of learning and documenting steps as an agency

Enablers/Hurdles - Financial resources: tend to have larger R&D budgets, had

funding to do internal pilot programs and offer large cash prizes

- Management directly involved in process

- Resources: had dedicated challenge manager and also hired

third-party providers to run challenge

Execution and

feedback

-Established internal processes, so did not encounter many difficulties

with challenge execution

-Large numbers of challenges, established program

Table 9: Change Enactment Summary

5.3.5 Summary of different enactment types

The table below compares the main characteristics of the three enactment types.

Stages Inertia Application Change

External Trigger Mandate Exposure to practice Policy and Legislation

Internal Motivation Expedience Experiment Problem Solving

Agency Mission public outreach,

some research

Primarily service Science, Tech, R&D

Agency Experiences

(with innovation)

limited Some experience Experience with prior pilot

programs and innovation

portfolio

Challenge Objective Not directly

related to core

mission

Mission related

Portfolio of problems

directly related to mission

Challenge Type Mainly awareness

(a few research)

Research and Service Service and Operational

Preparation Ad-hoc Ad-hoc but sometimes Strategic learn-first

approach

Enablers/Hurdles Many hurdles Many initial hurdles

Many enablers

Execution Difficult

Difficult at start

Gets easier if repeated

Smooth

Feedback Not often repeated Sometimes repeated Often repeated

Table 10: Summary of enactment types

6 Discussion

The objective of this study was to help us understand the motivations, conditions and

processes that influence and characterize adoption of challenges in US federal agencies.

Challenges fall under the broad umbrella of organizational innovation. Challenges also overlap

with open innovation practices because they offer a more open and collaborative path to problem

solving and innovation. Concepts from organizational innovation and open innovation literature

provided initial direction for the study, while public sector innovation literature was used to

highlight organizational characteristics for special consideration.

In this chapter I recap the various findings and discuss to what extent they reflect existing

literature, and where they extend the boundaries. The discussion is ordered based on findings for

the three research questions.

6.1 Challenge Types and Level of Innovation

The first research question examined the manner in which challenges were being used by

the various agencies and sought to establish patterns and explanations. Following analysis and

classification of challenges listed on the web platform, four (4) categories emerged namely:

awareness, research, service, and operational. These categories were mainly descriptive and

grouped based on similarity in objectives, beneficiary type, and submission format. More

indepth analysis revealed different levels of innovativeness, suggesting an innovation continuum

among categories.

The Office of Management and Budget (OMB) used the classification by McKinsey and

Company (2009) in providing guidelines to agencies on the different ways in which challenges

can be employed. McKinsey’s grouping identifies six types of prize contests, where each type is

based on a combination of two or more objectives termed change levers. For example, exemplar

prizes focus attention on particular areas by recognizing accomplishments, and are based on two

change levers: i) identifying excellence and ii) influencing public perception. Another prize

typeexposition, also focuses attention to ideas and is based on the change levers i) identifying

excellence and ii) mobilizing capital. Therefore while change levers help define a particular

type, they are not unique to any particular type. By contrast the challenge types emerging from

this study are grouped based on primary objectives, and each grouping is formed around one

particular objective. For example awareness challenges are grouped together because their main

objective is to raise awareness on particular topics primarily through creativity contests (such as

logos, posters, videos, slogans and photos). While challenges falling under other categories may

have some awareness aspect, they would not fall in the awareness category because it is not their

primary purpose.

There is a high degree of overlap between McKinsey’s point solution prizes which aim to

find the solution to a well-defined problem, and the newly identified category operational

challenges which seek to provide a functional solution to a specific problem. The problem in this

case is usually science and technology related, requiring a technical solution. While the type

descriptions demonstrate similarity between the two, the focus of point solution prizes as

specified by the change levers is i) to focus a community and ii) to mobilize talent. On the other

hand the primary focus of operational challenges is to find the solution to a specific problem.

Identification of challenge categories confirms observations from the preliminary analysis

of differentiated uses among agencies. As stated in McKinsey (2009), the six prize types are not

exhaustive, and new variations were expected with the growing prize industry. While

McKinsey’s types were based on prizes from the private and philanthropic sectors, the four

emergent types offer a more appropriate representation for challenges because they were derived

directly from the data. Thus identification of the four (4) new prize types extends the general

literature on prizes, as well as the growing literature on challenges in US government agencies.

Additional findings on the continuum of innovation levels among types helped highlight

that different challenge types promote different levels of innovation. While the analysis was not

able to offer a specific measure for innovation levels, other factors such as the level of

specialized skills, complexity and novelty of solution, served as indicators. This is a key finding

because it highlights the extent to which the various challenge types meet the innovation

objective of the Open Government initiative. This is reflected in the objectives of the America

Competes Act which are “to invest in innovation through research and development…” (America

Competes Reauthorization Act, 2010), and also to help agencies “spur innovation, solve tough

problems, and advance their core missions” (White House, 2012).

The innovation continuum also confirms preliminary observations articulated in the

problem statement, concerning disparities in the types and level of problems tackled by the

various agencies. Concern was expressed that some challenges addressed topics which did not

facilitate innovation, but essentially maintained the status quo; while others fulfilled the true

mandate of tackling difficult and complex problems. The ordering of challenge types as an

innovation continuum goes a step further than McKinsey’s (2009) grouping which projects the

assumption of equality among the various prize types and change levers. No mention is made of

different levels of complexity or innovation in McKinsey’s grouping. Thus the four challenge

types presented in the previous chapter provide a useful tool to help agencies evaluate to what

extent their challenges approach the innovation goal.

The placement along an innovation continuum is not meant to suggest that challenges at

the lower end should not be pursued. For example awareness challenges may be a good fit for

agencies with outreach as a big part of their mission, due to the ability to extend the level of

participation and outreach. For example in cases where agencies normally held public contests

or solicited nominations for awards, challenges could be employed to help the agency better

manage the outreach activity. In this scenario, challenge.gov provided a central repository to

accept public submissions while incorporating the newer feature of public viewing and voting on

submissions. Awareness challenges also have the advantage of less organizational effort and

resources, and can be used by agencies as a learning experience and a stepping stone to

launching more complex challenges. Unfortunately however, the evidence did not demonstrate

that organizations starting with less innovative challenges later move to more innovative ones.

This is in line with concerns expressed by administrators of the program that some agencies miss

opportunities for innovation by not moving beyond less innovative challenge types. However

after just two years of the platform, it may be too early to make a determination of whether

implementation of awareness challenges eventually leads to the more innovative categories.

This can be monitored in a future study.

The next part of the discussion examines the stages and conditions related to challenge

adoption.

6.2 Adoption Model

Response to the second research question involved reconstructing the various adoption

and implementation activities and events as reported by informants. The outcome of this analysis

is a challenge adoption model consisting of components and relationships that make up the

adoption process. The model is data driven and therefore provides a closer representation of the

phenomenon under study. It extends the organizational innovation literature by offering a

representation specifically relevant to challenge adoption as part of the US Open Government

initiative. Model components include stages in the adoption process, as well as variable

conditions which influence the stages. The model combines stage model and process research

with variance research, to present a more holistic picture of the process. This is a departure from

the literature where these two elements are usually explored in separate research streams.

Challenges form a subset of open innovation practices, and this model also extends the open

innovation and prize literature.

The emergent model breaks down the adoption process into stages, following the trend of

stage model research (Wolfe, 1994). Stage models generally present a series of stages connected

by unidirectional arrows, suggesting a rigid order from start to end. This level of rigidity is not

always realistic in such a dynamic process where there are likely to be variations in the order and

timing of activities. The challenge model eliminates the rigidity of stage models by replacing the

directional arrows with simple connections indicating relationships between stages. This

provides a more dynamic and realistic representation of adoption related activities and events.

6.2.1 Stages in the adoption process

Over the years researchers have offered several variations in classification of innovation

stages (e.g. Hage and Aiken, 1970; Klein and Sorra, 1996; Angle and Van de Van, 2000). With

such propensity for variation, it is no surprise that a different set of stages emerged in the

challenge model. However the question remains, do these differences exist only in terminology,

or do they reflect true differences in activities and events? For this part of the discussion a

comparison will be made between the stages of the challenge model, and the major adoption

subprocesses put forward by Rogers (2003) and others (discussed in chapter 2). These sub-

processes are initiation and implementation, and the comparison is illustrated in Figure 4 below.

Initiation

Initiation in the organization innovation literature represents the initial stages where the

organization becomes aware of the innovation, matches it with a problem or opportunity, and

evaluates costs and benefits in preparation for the adoption decision (Wolfe, 1994, Rogers, 2003).

Though individual stages of the challenge model differ from existing models, the initiation sub-

process can be mapped to combined activities in two stages of the model: external trigger and

motivation. The external trigger is comparable to the awareness stage (different from awareness

type challenge) in many of the stage models, and represents how the organization encountered

the notion of challenges following the policy pronouncement. As discussed in the previous

chapter, organizations encountered challenges through three different external triggers namely: i)

mandate ii) exposure to the practice in outside organizations or other government agencies and

iii) the policy and legislation. The internal motivation in the challenge model is comparable to

the matching and evaluation stages from the literature, where the organization examines the

options and develops the motivation to adopt by matching with a problem or opportunity. Again

there are three possible values for motivation namely: i) expedience ii) experiment and iii)

problem solving. Thus there are identifiable stages within the challenge model which are

comparable with the initiation sub-process from the literature. However the main difference is

that while models from the organization innovation literature assume a singular adoption path,

the challenge model provides alternate values for each step, setting the stage for different

enactment types.

Figure 4: Comparison between Challenge adoption model and Rogers (2003) adoption sub-processes

Implementation

Similarities can also be drawn between the adoption model and the organization literature

for the implementation phase. Implementation from the organizational innovation literature

covers activities related to modifying the innovation to fit the organization, or within the

organization to accommodate the innovation (Rogers, 2003). Challenge implementation starts

with selection of challenge characteristics such as the type and problem/issue that will be

addressed. Rogers (2003) also indicates that flexible innovations are more compatible with the

organization as they allow significant amounts of re-invention by employees. The typology of

challenges presented in Research Question 1 gives an indication of their flexibility to

accommodate a wide range of problems from simple to the most complex. Thus the organization

can potentially benefit more where challenges are modified to address complex mission related

problems, rather than the more generic public outreach situations. The next part of

implementation as described by Rogers (2003) is the clarifying process during which employees

Enablers/

Hurdles

Comparable to implementation

Comparable to initiation

Mission/

Experiences

Experiment Exposure Challenge

Characteristics

Preparation

mainly

Ad-hoc

Execution

get a better understanding of the innovation and clear up uncertainties around questions such as

how it works, who is responsible, and who it affects. A parallel can be drawn with the preparation

stage in the adoption model where the organization takes steps in paving the way for

implementation activities. As stated earlier, some organizations take time upfront to allow

employees to learn about the process and set out procedures from smooth implementation, while

others approach it in an ad-hoc manner of learning while doing. Here again the challenge model

makes a departure from the literature by providing two distinct paths which separates

organizations that undertake significant preparation from those which do not. The paths continue

to define the organization’s adoption journey, and are reflected in the degree to which critical

enabling factors are present. Inclusion of these variable enabling factors in the adoption model is

another departure from the literature which simply focuses on the stages.

One of the motives of the America Competes Act is the long term integration of

challenges as a standard problem-solving mechanism alongside more established methods like

grants and contracts (White House, 2012). Thus while there is a common objective of

routinization and long term integration, these are represented differently in the two models.

Rogers (2003) describes routinizing as the last step in implementation where the innovation

eventually loses its separate identity and becomes incorporated into regular organizational

activities. For the challenge model, implementation ends when the first challenge is executed (i.e

an award is made), and routinization comes after the implementation process has been repeated

several times. Thus the number of challenges executed can be used as a proxy for routization in

challenge adoption and implementation.

Decision

In the literature the decision is marked by one clear step or go/no-go choice. It is usually

made following initiation and before implementation, and is applicable to all adoption scenarios.

With challenge adoption there were two decisions. First an authority decision was made by an

entity external to the organization (White House OSTP) followed by a contingency decision

made at the organization level. Since all agencies received the same authority decision, the

differentiating factor would be the contingency decision which was within organizational control.

However, this study only focused on adopting organizations, and it was not possible to know

what a negative decision might look like, or where exactly the decision point would fall. Thus the

challenge adoption model differs from the existing model because it does not contain a decision

point.

Judging from the reasons given by informants for their adoption decision, it is tempting to

draw a parallel between the agency’s internal motivation to adopt and the adoption decision.

In that scenario, the motivation provides the closest insight into the driving reasons for adoption.

For example inertia enactment was triggered by the directive to execute a challenge. For

application enactment the motivation was to experiment with challenges mainly to see what they

were like and how they worked. For change enactment the main motivation was the need to

solve particular problems, or innovate in areas where there were gaps. However, before forming

any conclusions about the link between the motivation and adoption decision, further study

should be conducted to include non-adopters.

6.2.2 Enabling Conditions for challenge adoption

The conceptual framework presented at the beginning of the study (section 2.5) used

sensitizing concepts from the three literature streams considered relevant to the study, namely

organization innovation, open innovation (and related concepts), and public sector organizations.

The concepts selected from the literature were: management support, employee attitudes (NIH

syndrome), organizational commitment, nature of problem faced, use of intermediaries and

organizational red tape. This conceptual framework provided an important starting point which

gave general direction to the study, and specific direction to data collection and analysis. In light

of the broad findings which include a challenge adoption model and a typology of enactment, the

initial framework diminished in theoretical importance. Notwithstanding, its practical

importance cannot be ignored, since it highlights factors within the direct control of the adopting

organization which can be adjusted for a more effective adoption process.

While there may be a long list of potential enablers, the top four which emerged from the

data are: internal processes, management support, financial resources and human resources. The

presence of these factors was found to help implementation, while their absence (or limited

presence) was found to hinder the process. Now for a closer look at those factors.

Management Support

The emergence of management support as a critical enabler is consistent with the

literature on organizational innovation (Damanpour, 1991, Crossan and Apaydin, 2010) and

public sector innovation (Vigoda-Gadot et al., 2005). In both the data and literature, support

from management has been identified as the most critical organizational factor, since it has direct

consequences on the remainder of the adoption and implementation experience. Crossan and

Apaydin (2010) also underscored the importance of getting support from all levels of

management to lead and maintain momentum for the process. The data also gave an indication

of differing levels of support at different agencies ranging from leading and driving the effort, to

merely approving the process. There were no reported cases of outright opposition but rather

skepticism in a few instances. In one case the departure and subsequent replacement of a

supportive administrator resulted in changing organizational priorities, and the challenge

program was placed on the back burner. On the other hand, initiatives that were management

led or driven had greater likelihood of getting organizational resources to enable the process.

Thus management support is closely linked with the availability of resources, as discussed below.

Organization Commitment

In the open innovation literature, Chesbrough and Crowther (2006) identified

organization commitment as a critical enabler of open innovation initiatives. This broad term

referred to the general level of commitment the organization displayed towards the initiative, and

included a wide range of indicators such as senior management support, funding, innovation

champions, revised internal processes, metrics and incentives. In a general sense the data

supports the literature in highlighting the importance of having the entire organization prepare to

accommodate the adoption process. More specifically the data helped identify which of the list

of indicators are the most important to the study context – challenge adoption in public sector

organizations.

The specific organization commitment factors which emerged from the data were the

availability of human and financial processes, as well as internal processes. Human Resources

referred to the presence of specially designated staff with temporary or longer term appointments

to oversee the running of challenges. Financial resources refer to funding for challenge-related

activities such as prize awards and hiring contractors or intermediaries to undertake the extra

steps required for implementation. The emergence of financial and human resources as critical

enablers is an indication that challenges have not been fully integrated and routinized into

organizational practices, and are still viewed as extra work, requiring extra personnel and extra

funding. Therefore it would be interesting to follow whether these factors continue being an

issue after a few additional years of challenge use.

Internal Processes

The presence of internal processes did not appear in the initial conceptual framework, but

has emerged as important enabler of challenge adoption. Internal processes refer to the

establishment and presence of organization policies and mechanisms to guide employees

undertaking challenge adoption activities. The absence of adequate internal processes was often

a hindrance to implementation. Respondents recounted difficulties caused by the lack of

understanding of financial and legal procedures, compounded by an absence organization

policies and mechanisms.

The Not-Invented-Here syndrome (NIH)

This factor was included in the initial framework as a major potential hindrance to the

adoption process, but did not appear in the data as expected. In the open innovation literature the

NIH syndrome was used to refer to internal resistance to outside knowledge (Chesbrough and

Crowther, 2006; Huston and Sakkab, 2006). While the existence of NIH was seen as a hindrance

to open innovation practices, its existence was not insurmountable. In one of the studies Huston

and Sakkab (2006) reported that the company was able to counter negative effects of the NIH

syndrome by shifting the organization attitudes to get buy-in and support from employees.

The data indicated a high level of support from colleagues and non-management

employees. This finding is in direct contradiction to open innovation literature where

Chesbrough and Crowther (2006) found resistance among employees in private enterprises. The

negative NIH syndrome in private enterprises is plausible specifically in the manufacturing and

production sector where Research and Development (R&D) employees or entire organizations

may be compensated for income generating innovations. On the other hand only a very small

number of public sector agencies have R&D staff, and these functions are usually outsourced to

private enterprises. Therefore organization culture may already be favorable to external input in

the form of grants, contracts, and outsourcing. Further, there is no compensation to individuals

or agencies for ground-breaking innovation, which may explain the absence of NIH syndrome.

Organizational red-tape

Organizational red-tape refers to rules that create unnecessary compliance burdens for the

organization without contributing to functional objectives (Bozeman and Feeney, 2011). This

factor was thought worthy of exploration since public sector organizations are often viewed as

bureaucratic and not receptive to change. While some informants mentioned that they were

burdened with following specific rules and regulations, organizational red-tape did not stand out

as a major hindrance.

This was a study of early adopters, and it is possible that these agencies were able to

adopt early on because they have moved away from the typical bureaucratic structures. This is in

line with an observation by Vigoda-Gadot et al. (2005) that in recent decades some public

organizations have moved away from the classic bureaucratic structures, making them more

receptive to innovation. An alternative explanation is that these agencies were motivated to

adopt challenges early because they viewed existing paths to innovation and problem solving as

extremely burdensome. This is in line with the finding by Pandey and Bretschneider (1997) that

agencies with high levels of red-tape were motivated to innovate as a way of minimizing

burdensome effects of the red-tape. While the findings show that red-tape was not a significant

hindrance, the data did not reveal the underlying reasons for this. Future studies could try to

measure and collect data on the level of red-tape in the organization prior to challenge adoption.

The related concept organizational processes (discussed above) stood out as a major

enabler (or potential hindrance in its absence). While regular organizational rules did not pose a

big threat, the need to follow rules and procedures specifically related to the process of challenge

implementation was indeed a burden. An important consideration is the passage of the America

Competes Act of 2010 which gave agencies legal authority to run challenges and pay out cash

rewards. Prior to this legislation, rules and budgetary regulations of many agencies restricted

their ability to run competitions and pay our awards. Prior to the passage of America Competes,

other agencies like NASA and Department of Defense modified their governing Acts to give

them authority to run challenges. Other agencies which tried to implement challenges prior to

passage of appropriate legislation encountered difficulties with existing budgetary rules. This

seems to indicate the existence of special legislation including America Competes had gone a

long way in mitigating restrictions of these regulations.

6.3 Types of Enactment

The third research question explored the influence of the various adoption and

implementation factors on the manner in which the agency enacted challenges. The variations at

each stage of the challenge model created the possibility for multiple paths for adoption and

implementation. Orlikowski’s (2000) framework provided a basis for a typology of enactment

which featured three types – inertia, application and change. Though Orlikowski’s framework is

based on the study of technology adoption in organizations, it is used here to extend the

organization innovation literature to the adoption of challenges. Thus a framework developed to

explain adoption patterns of a technological innovation has been employed to explain similar

patterns in an administrative innovation. Orlikowski’s framework recognizes that when the same

technology (and by extension innovation) is adopted under different contexts, it results in

different adoption experiences and paths. This is a departure from the adoption models presented

in the organization innovation literature (see chapter 2), which refer to just one adoption path.

This typology of enactment also has practical significance since it can provide insight into which

combination of circumstances and conditions result in desired outcomes, and which

combinations produce less desired outcomes. It is worth noting that enactment types may change

if organizational conditions change.

6.4 Considerations for public sector organizations

Innovation and open innovation practices started in the private business sector before

being adopted by public organizations. By the same token, research studies from the private

sector are constantly being applied to study these phenomena in public organizations. There are

ongoing debates about the merits and shortcomings of applying concepts across sectors which

cannot be solved here. However the unique attributes of public sector organizations cannot be

ignored.

The findings highlight diversity in adoption patterns and the consequent enactment types.

This diversity is a demonstration of the variation in organizational characteristics such as

mission, goals, and experiences in public sector institutions. This is in agreement with

observations by Caudle et al (1991) of the multiple, often intangible and sometimes conflicting

goals of public sector organizations as opposed to the single profit motive in private institutions.

The effects of introducing the innovation through a government-wide policy mandate cannot be

ignored. In some cases the need to comply with the policy superseded all other motivations,

leading to less innovative implementations. In keeping with the profit motive of private sector

organizations, it is reasonable to suggest that these less innovative or noncompetitive

implementations be more difficult to justify and less likely to be implemented in private firms.

At the individual level the study explored attitudes to embracing outside knowledge, and

found that the idea was widely embraced, in contrast to the not-invented-here syndrome. One

other potentially important aspect of individual attitudes not explored during this study was the

individual motivations to adopt. For example Potts & Kastelle (2010) suggest that sometimes

there is a structural disconnect between the political mandate to introduce new initiatives and the

employees whose general approach may be less than enthusiastic. Public sector employees may

not be sufficiently motivated to innovate, as there is often no reward at the individual or

organization level for being innovative. This is also in keeping with Damanpour & Schneider's

(2009) identification of common barriers to innovation in the public sector which include the

lack of incentives to innovate, and lack of funding. This is in contrast to private institutions

where resources are usually allocated for innovation, as well as financial rewards for innovative

organization or group.

7 Conclusion

This was a study of early adopters of challenges within the US Federal Government. The

organization innovation literature and open innovation literature provided the initial conceptual

direction for the study. Data was collected from the challenge.gov web platform and interviews

with challenge managers and administrators from adopting agencies. This chapter presents a

summary of contributions to theory and practice, limitations of the study, and directions for

future study.

7.1 Contributions to Theory

This study makes several contributions to theory:

1. The development of a typology of challenges consisting four groups based on similar

objectives. Though there may be other groupings for prize contests in general, this

represents the only grouping to date which is specific to challenges implemented in US

federal agencies. The groups have been arranged along a continuum of increasing levels

of innovation, indicating that all challenges did not necessarily meet the innovation

objectives of the Open Government initiative.

2. The challenge adoption model combines innovation stages with variable conditions

impacting the stages. Stage model research and organizational innovativeness research

usually represent separate streams in the organizational innovation literature. In the

challenge adoption model they are combined to give a more realistic representation.

While the model is based on the organization innovation literature, it highlights stages

and conditions which are unique to challenge adoption in US federal government

agencies. Unlike many existing adoption models which display a rigid sequence of

stages, this model focuses on the relationships between the various components.

3. This study extends and enriches Orlikowski’s (2000) typology of enactment for

technology adoption by combining it with factors from a new domain. The incorporation

of factors from the challenge adoption model (e.g. external trigger, internal motivation,

etc) into each of the three enactment types adds richness to the framework by making it

actionable and extending its practical applicability. The incorporation of factors from

challenge adoption also boosts the explanatory power of the framework by helping to

explain why some enactments lead to major change while others lead to no change. The

grouping of agencies based on the three enactment types is one demonstration of the

practical applicability of this enhanced framework.

4. Though Orlikowski’s framework was developed around the adoption of technological

innovations, the findings show that it can be successfully applied to characterize adoption

patterns in an administrative innovation. This is another instance where the framework is

made more actionable by applying it across the various dimensions of organizational

innovation.

5. The use of challenges and prize contests and other open innovation practices started in

private organizations and are still relatively new to the public sector. This study helps

explain how a private sector innovation has been adopted in the public sector by

highlighting the various stages, conditions, and organization reactions. By focusing on

early adopters, the study leads the way in exploring open innovation practices in the

public sector prior to more widespread adoption.

The findings indicate that the organizational innovation literature is an appropriate

perspective to study challenge adoption in public sector organizations. The resulting challenge

adoption model matched some foundational aspects from more general adoption models such as

the initiation and implementation stages, but diverged in places pertaining to the feedback loop.

The similarities with other adoption models developed outside the public sector supports the

view expressed by Vigoda-Gadot et al (2005), that existing knowledge from the private business

sector can be used to inform public sector innovation research. However there is a need to extend

that knowledge to consider specific characteristics of the public sector.

The greatest overall contribution to theory is the recognition that the adoption of

challenges differed across agencies based on varying organizational scenarios. Thus even though

all agencies faced the same policy pronouncement and type of innovation (challenges), the

variations among stages and conditions in the emergent model were sufficient to yield different

enactment types. The findings also reflect recommendations by De Vries et al (2014) that the

environmental context of external push such as policy and political pressures need to be

considered in public sector innovation research. Further, the various factors should not be looked

at in isolation, but should also consider linkages and interrelationships among them.

7.2 Contributions to practice

The research findings confirm legitimacy of concerns expressed in the problem statement

about disparities in challenge types and innovation levels. They also bring to light the conditions

which promote true change and those which simply maintain the status quo (inertia). This

provides a starting point for discussion among policy makers and administrators about how the

conditions can be maximized to promote true innovation and change. The findings provide a

perspective on when and where challenge adoption contributes to the highest level of innovation,

as well as an understanding of the enabling and hindering conditions. At the level of the

adopting unit, results can be used to inform decisions and alert prospective challenge managers

on critical areas to be addressed.

Based on the interest expressed by the GSA program managers, I compiled the following

recommendations which can be used to inform policy and practice. The recommendations

underscore the importance of purpose, planning, and support as important precursors to a smooth

adoption and implementation process.

1. Challenges do not fit all situations

Challenges were introduced as a tool to spur innovation and should only be used in

situations where there is an innovation goal. Before a decision is made to use a

challenge, there should be a clear and specific purpose why a challenge will be used and

what the agency intends to get out of it. Generally, challenges are most appropriate when

there is a clear intended outcome with some uncertainty as to how best that outcome can

be achieved. For this reason, agencies seeking to innovate should avoid implementing the

common types of awareness challenges involving creativity contests.

2. Clear decision point needed

The internal motivation to adopt is often indistinguishable from the decision to adopt.

That means that too often agencies go ahead with challenges once there is sufficient

desire (motivation), without going through the requisite evaluation and decision making

process. A clear decision point is needed after the motivation surfaces, where agencies

examine the feasibility of the idea, weigh the alternatives, and make a definite go or nogo

decision.

3. Management Support

The data showed that management support was the single biggest enabler for challenge

adoption and implementation. Challenge initiatives originating or strongly supported at

top management level are more likely to benefit from resource allocation and

organization commitment. For initiatives originating at the program level, it is essential

to get buy-in and support from top management in order to secure approval and

allocation of resources. Program managers need to sensitize top management from early

on the value and potential benefits of this mechanism.

4. Planning at the strategic level

Advance planning and preparation is a critical prerequisite for smooth adoption and

implementation. However some agencies made the mistake of quickly rolling out their

first challenge without spending time upfront to lay out plans for a sustainable program.

At the strategic level, agencies should view the challenge mechanism as a new approach

to problem solving and innovation which can potentially replace existing methods. Thus,

rather than experimenting with isolated efforts, initiatives should be linked to the

organization’s mission and long term goals. For example NASA’s thriving challenge

program began by tackling areas in their strategic plan where they had been unable to

make progress over a number of years.

5. Planning at the operational level

This involves working out the details of challenges upfront to avoid uncertainty and

ambiguity in later stages. Before launching, each challenge needs to be tested to ensure it

is feasible and that objectives are attainable. Staff should be trained in the procedural and

legal requirements, and legal counsel should be brought on board early in the process.

Human and financial resources need to be identified for tasks such as formulating contest rules,

allocation of prizes, marketing and publicity, responding to queries and judging. To be included

in the overall plan is a strategy for how the solutions will be rolled out or deployed after selection

of winning entries. Plans should also be made for documenting and debriefing on challenge

implementation procedures, with the objective of learning and improving current procedures.

6. Specially Designated Staff

An important output of the planning process should be the identification of human

resources to perform challenge duties. From all reports, tasks are time consuming and

burdensome when combined with regular duties. These include tasks related to outreach,

problem definition, responding to public queries, judging, awarding prizes, and

consultation with internal and external partners. While a handful of agencies have

specially assigned challenge managers, the majority rely on program managers and

regular staff to perform these additional duties. For agencies planning to establish a

sustainable challenge program, the appointment of specially designated staff is

recommended. An alternative arrangement currently utilized by agencies with larger

challenge programs (e.g NASA, ONCHIT) is to form partnerships with third party

(forprofit or non-profit) providers. Thus agency staff can focus on planning and

coordinating while other tasks can be offloaded onto the external partner.

7. Documenting and Sharing best practices

Challenge execution is an emerging practice with many unknowns. Procedural

difficulties encountered in the early stages reportedly became easier with subsequent

attempts, underscoring the importance of continuous learning. Agencies can learn from

their own experiences as well as the experiences of others by documenting and sharing

best practices. So far the GSA, White House OSTP, and HHS have online resources

which other agencies are able to access. There have been calls for a central repository

containing resources such as rules, procedures, templates and sample documents which

can provide guidance at each stage. Others may benefit from the formation of formal or

informal partnerships among staff.

7.3 Limitations

As in any research on real life situations, there were conditions which were out of my

control. I had to make decisions and compromises to enhance feasibility of the study, which had

consequences for how the results are presented and interpreted. In the interest of transparency I

would like to acknowledge the limitations and allow the reader to place the results in context.

One of the limitations is the short time frame covered by the study. The challenge.gov

platform was legislated and put into practice just two years before the start of the study, and

current users are viewed as early adopters. Huizingh (2010) contends that though lessons have

been learned by studying early adopters of open innovation, these lessons are not always

applicable to those who adopt later. He theorized that some of the later adopters may not be as

excited about the new concepts and practices used by the early adopters, and may not be as

willing to change. Therefore at this point it is difficult to predict whether findings will still be

applicable a few years from now when new agencies come on board.

The study only presents the perspective of adopting agencies. It would have been

interesting to get the perspective of non-adopters as well, particularly concerning the reasons for

delayed or non-adoption. This was tentatively added to the initial research design, but presented

challenges in identifying and classifying non-adopters. With only 47 implementing units and

agencies, the remaining majority fell into the category of non-adopters. At that point it was

difficult to identify significant criteria besides non-adoption which could be used to guide

theoretical sampling. An alternative approach was to identify those who had considered or

initiated adoption but did not carry out the project. However it was impossible to identify these

cases since there was no requirement to inform the GSA during early planning stages.

Responsibility for challenge management and adoption varied widely among agencies, and it was

difficult to speculate within which program or portfolio it may have fallen within nonadopting

agencies. Therefore even if a non-adopting agency was identified, it was not clear which units or

individuals should be targeted. In light of these difficulties a decision was taken to focus on

adopting agencies only.

Interviews were the primary form of data collection. However interviews rely on the

ability of informants to recall past events and are subject to problems of bias and poor recall

(Yin, 2009). Recognizing this shortcoming, interview data was corroborated with information

from other sources where feasible (Yin, 2009) such as other personnel from the same agency,

documentation, and the challenge.gov web platform.

Collecting data from listings on the challenge.gov platform presented some logistical

challenges. In most cases challenge listings identified executive departments and parent agencies

but not implementing units. For example the listings may indicate that Agency A hosted 10

challenges, giving the impression of an established challenge program. However in reality this

scenario may turn out to be a number of unrelated and sometimes isolated implementations

undertaken in different units of the agency. In some cases this made it difficult to identify the

persons responsible for a particular challenge, and also created difficulties in accurately

compiling lists for smaller implementing units.

Students also viewed