Review on Energy Resilience

Technological Forecasting & Social Change 103 (2016) 21–33

Contents lists available at ScienceDirect

Technological Forecasting & Social Change

A holistic framework for building critical infrastructure resilience

Leire Labaka ⁎, Josune Hernantes, Jose M. Sarriegi TECNUN, University of Navarra, Paseo Manuel Lardizabal, 13, 20018 San Sebastian, Spain

⁎ Corresponding author. E-mail addresses: [email protected] (L. Labaka), jhern

[email protected] (J.M. Sarriegi).

http://dx.doi.org/10.1016/j.techfore.2015.11.005 0040-1625/© 2015 Elsevier Inc. All rights reserved.

a b s t r a c t

a r t i c l e i n f o

Article history: Received 1 April 2015 Received in revised form 26 August 2015 Accepted 3 November 2015 Available online 28 November 2015

Keywords: Crisis management Critical Infrastructures Resilience Resilience policies Delphi process Case studies

The welfare of society is more and more dependent on the proper functioning of Critical Infrastructures (CIs), and crises that affect CIs usually aggravate their impact on society. Therefore, improving the resilience of CIs is the most important objective of today's crisis managers. Although several resilience frameworks can be found in the literature, their implementation is still incipient and detailed prescriptions for their implementation are lack- ing. Moreover, some frameworks are only limited to describing the activities performed within the boundaries of the CI, neglecting the role of external agents. This research describes a practical and holistic resilience framework for improving the resilience of CIs taking into account the external agents. The framework is composed of three elements: a set of resilience policies; an influence table that assesses the influence of policies on prevention, ab- sorption and recovery stages; and an implementation methodology that defines the temporal order in which the policies should be implemented. Two empirical studies were undertaken in two CIs to implement this frame- work. The studies show that the resilience framework helps CIs to diagnose their resilience level, detect areas of potential improvement and complement their risk management approach with a transversal approach to be better prepare to deal with crises.

© 2015 Elsevier Inc. All rights reserved.

1. Introduction

Crises derived from similar triggering events occur continuously, but despite the efforts of governments and organizations to develop lessons learned reports and suggest best practices, our ability to effectively im- plement these lessons learned and best practices seems limited. Al- though crisis management for future events is improved based on learning from previous incidents, the particular characteristics of every crisis cannot be foreseen. How can crisis managers enhance their preparedness for unexpected events? Crises such as the earth- quake in Japan and the subsequent Fukushima nuclear accident (Broad, 2011; Dempsey and LaFraniere, 2011), several power cuts in Western Europe (Andersson et al., 2005; Union for the Coordination of Transmission of Electricity (UCTE), 2004; US–Canada power system outage task force. 2004; Larsson and Danell, 2006), and the eruption of Iceland's Eyjafjallajökull volcano and the resulting air traffic crisis (Hall, 2010; Barr, 2010) have warned us that it is still not possible to an- ticipate how a crisis may evolve and what protective measures should be established in order to avoid its occurrence. Can we forecast what could occur in the future? Are the characteristics of today's world the same as in the past? Is the current crisis management approach ade- quate for dealing with today's world crises?

Crisis can be defined as a consequence of an unexpected triggering event that suddenly or by an accumulative process of near misses strikes the entire system (Mitroff and Anagnos, 2000; Pearson and Clair, 1998;

[email protected] (J. Hernantes),

Coleman, 2004). Preventing and preparing for something which is un- expected is almost impossible since nobody knows when or how a crisis will occur or what would be affected by the crisis.

To date, crisis managers have been focused on developing specific preparation and response procedures for already identified risks, but they lack sufficient preparation for unexpected situations (Boin et al., 2003; Boin, 2004; Lagadec, 2007). Risk analysis is built on the premise that hazards are identifiable (Risk and Resilience Research Group and Center for Security Studies, 2011; Park et al., 2013). However, nowa- days, as it has been shown in the previous crises examples, it is almost impossible to forecast when a crisis would occur and how it would evolve. Furthermore, as today's world is more complex and intercon- nected than ever before, interactions among Critical Infrastructures (CIs) are more unfamiliar and complex than before, and this makes it difficult to anticipate how an incident that occurs in a CI may affect the rest of the CI network (Gilpin and Murphy, 2008; Turner, 1976; Perrow, 1984). CIs are essential systems for the safety and economic and social welfare of modern society (Min et al., 2007; Oliva et al., 2010; Katina et al., 2014), and therefore crises compound their effects if they affect one or more CIs (Min et al., 2007; Oliva et al., 2010; Laugé et al., 2014; Chang et al., 2007). It is therefore really important that crisis management focuses on improving the safety and reliability levels of CIs (Hämmerli and Renda, 2010).

Several national approaches have been established worldwide such as European Programme for Critical Infrastructure Protection (EPCIP) in Europe (CEU, 2008), National Infrastructure Protection Plan (NIPP) in the US (NIPP. National Infraestructure Protection Plan, 2009), and Critical Infrastructure Resilience Strategy in Australia (Australian

22 L. Labaka et al. / Technological Forecasting & Social Change 103 (2016) 21–33

Government, 2010). Furthermore, several studies have attempted to an- alyze the CIs interdependences. Diverse modeling techniques and simu- lation approaches such as empirical, agent based, system dynamics and, network based approach among others have been used to analyze the CIs interdependences (Ouyang, 2014; Yusta et al., 2011). These model- ing approaches have their own particular advantages and disadvantages but all of them have several challenges that they need to overcome. All of them need data to feed into the models but the access to this data is limited and often the accuracy is not sufficient. Most of the approaches limit to model the interdependence between two or a proportion of CIs without taking into account the interdependences among all of them (Ouyang, 2014). The validation is mostly based on feedback from ex- perts and historical data, which might not reflect the reality in the fu- ture. In light of these challenges, Ouyang and Yusta et al. (Ouyang, 2014; Yusta et al., 2011) propose to enhance the collaboration and infor- mation sharing and to joint effort from the government agencies, re- search communities and the utility companies to provide the required data and to face the current challenges.

Given the unforeseen nature of crises and the complex structure of CI networks, mitigation efforts established beforehand may often be in- adequate or not even desirable when dealing with crises and their cas- cading effects (Gilpin and Murphy, 2008; Wardekker et al., 2010). Assessing the magnitude of the hazard may be unknowable and fore- casting the joint probability of the occurrence of two or more major events simultaneously may be even harder (Park et al., 2013). There- fore, improving cooperation between the CI and external stakeholders and developing a crisis awareness culture within the organizations have become the most promising alternatives for crisis managers (Van de Walle and Turoff, 2008). To be able to face crisis situations, it is essen- tial that their occurrence be prevented and specific response plans be developed; it is equally crucial that an adaptive behavior plan also be adopted (Gilpin and Murphy, 2008; Wardekker et al., 2010; Weick and Sutcliffe, 2007; Elwood, 2009; Boin and McConnell, 2007). Lindblom (1959) outlined this approach in a 1959 paper. He explained that decisions cannot always be made using a “scientific” process where complete knowledge of all relevant variables is known nor can the optimized solution be obtained. As Turoff et al. (2009) point out, when unexpected events occur and there is not enough information or a previously established plan is not adequate for handling the situa- tion, decisions made by crisis managers are subjective and involve a lim- ited number of alternatives that rely on expert knowledge and past experience.

In this context, resilience has become an essential concept in the field of crisis management and critical infrastructure protection (Hämmerli and Renda, 2010; Boin and McConnell, 2007; De Bruijne, 2006; De Bruijne and Van Eeten, 2007). Resilience goes beyond tradi- tional risk management methods by not only defining policies for facing expected events but also by taking into account unexpected events (Risk and Resilience Research Group and Center for Security Studies, 2011). Both approaches, risk management and resilience, must be com- bined to adequately cope with crises (Park et al., 2013). Although there are several definitions in the literature regarding the concept of resil- ience (Manyena, 2006; Moteff, 2012), our research defines resilience as the ability of a system to prevent the occurrence of a crisis and the ca- pacity to absorb the impact and recover to the normal state rapidly and efficiently when a crisis does occur.

In operationalizing resilience, this paper presents a holistic resilience framework for CIs that supports crisis managers in diagnosing and im- proving a CI's resilience level. Although there are several studies regard- ing the analysis of CI interdependences (Ouyang, 2014; Yusta et al., 2011; Eusgeld et al., 2011), only a few address the relationships that exist between a single CI and external response stakeholders such as first responders, government and society (Yusta et al., 2011). Therefore, this framework focuses on a single CI, including the external stake- holders potentially involved in a crisis at this particular CI. This research does not analyze the interdependences that exist among different CIs.

The framework is composed of three main elements: a set of resilience policies, an influence table where the influence of each resilience policy on the three resilience lifecycle stages (prevention, absorption and re- covery) is assessed, and an implementation methodology which iden- tifies the temporal order in which the resilience policies should be implemented to achieve the highest effectiveness in the implementa- tion process. Moreover, two case studies were carried out in order to implement this framework in practice. We conclude by drawing some conclusions about the experiences of applying the framework.

2. Resilience dimensions, characteristics and principles

The literature contains several definitions of resilience as well as sev- eral dimensions, characteristics and principles that define this concept. Some authors break resilience down into four dimensions (Bruneau et al., 2003; Multidisciplinary Center for Earthquake Engineering Research (MCEER), 2008; Zobel, 2010; Gibson and Tarrant, 2010):

• Technical resilience: this refers to the ability of the organization's physical system to perform properly when subject to a crisis.

• Organizational resilience: this refers to the capacity of crisis managers to make decisions and take actions that lead to a crisis being avoided or to at least reducing its impact.

• Economic resilience: this refers to the ability of the entity to face the extra costs that arise from a crisis.

• Social resilience: this refers to the ability of society to lessen the im- pact of a crisis by helping first responders or acting as volunteers.

Alternatively, some authors set the following characteristics as the main features of resilience (Bruneau et al., 2003; Multidisciplinary Center for Earthquake Engineering Research (MCEER), 2008; Zobel, 2010): robustness, redundancy, resourcefulness, and rapidity. Brunsdon and Dalziell (2005) propose that resilience can be broken down into two components: vulnerability and adaptive capacity. Vul- nerability refers to the ease with which an organization is pushed into a new state and adaptive capacity is the ability to cope with that change. In turn, McEntire (2001) defines vulnerability as “the degree of risk, sus- ceptibility, resistance and resilience level of the system”. Vulnerability is dependent not only on the magnitude of the hazard and exposure of the system to an event, but also on the capacity of the system to resist and absorb the impact (McEntire, 2001; Francis and Bekera, 2014).

The literature also presents several resilience frameworks and prin- ciples for improving the resilience level of CIs. High Reliability Organiza- tions (HROs) have been defined as those organizations that operate complex and high-risk technologies and manage to remain accident free for long periods of time (Roberts and Rousseau, 1989; Roberts, 1990). HROs are defined by several characteristics and processes that help them reach and maintain high reliability levels (Weick and Sutcliffe, 2007; Lekka, 2011). More recently, a research group in New Zealand called “Resilient Organisations” developed a framework to build up organizations' resilience level. This framework is composed of thirteen indicators grouped under three attributes: leadership and cul- ture, networks, and change ready (Resilient Organisations. Resilience Indicators. 2012). In the same vein, Parsons describes eight key attri- butes of organizations that are resilient based on a workshop conducted by Trusted Information Sharing Network's Community of Interests (Parsons, 2007). However, all frameworks focus on organizational resil- ience, without providing any information about how to improve the rest of the resilience dimensions (technical, economic, and social).

Johnsen (2010) takes a step forward and describes seven principles based on the organizational and technical aspects that organizations need to fulfill to be resilient. Francis and Bekera (2014) propose a resil- ience assessment framework based on the three resilience capacities: absorptive capacity, adaptive capacity and restorative capacity. A four step process (system identification, vulnerability analysis, resilience

23L. Labaka et al. / Technological Forecasting & Social Change 103 (2016) 21–33

objective setting, and stakeholder engagement) together with a resil- ience measurement formula that leads to the assessment of the follow- ing resilience capacities is proposed. Nonetheless, in this two frameworks, as in the earlier cases, the processes, the order, and trans- formations required to create resilience building activities are not spec- ified. Other authors (Cutter et al., 2010; Cohen et al., 2013) define a set of indicators to evaluate disaster resilience levels and in turn evaluate the efficiency of the established policies that support the resilience level. However, these policies focus on social resilience and do not pro- vide specific policies for CI operators. Furthermore, little is stated about how to improve these indicators.

The literature provides a broad set of works discussing ways to build CI resilience levels. However, as we have already noted, these frame- works and principles present certain difficulties when they are imple- mented in practice due to their theoretical nature (Boin and Van Eeten, 2013; Prior and Hagmann, 2014; Lekka and Sugden, 2011; Waller and Roberts, 2003): the principles are only theoretically ex- plained and the frameworks do not provide the activities or actions that need to be carried out to implement these principles in practice (Boin and Van Eeten, 2013; Lekka and Sugden, 2011). Furthermore, the literature contains little on the way of empirical research and case studies that address the implementation of these principles in CIs. In ad- dition, most of the frameworks only focus on developing the resilience level of the CI or even only on improving organizational resilience, underestimating the role of the external stakeholders (such as first re- sponders, government and society), which are also of utmost impor- tance in response and recovery activities. Thus, the aim of this paper is to provide a resilience framework for CIs that improves their resilience level and overcomes the limitations found in the frameworks defined in the literature to date.

3. Research methodology for the development of the resilience framework

The research methodology used to develop the resilience framework consists of three main phases based on the three main elements of the framework: (1) identification of the resilience policies, (2) development of the influence table, and (3) development of the methodology for implementing the resilience policies.

Fig. 1. Research m

Several research methods have been applied over three distinct phases to develop the final version of the resilience framework for CIs (see Fig. 1). The methods applied in each phase will be explained below.

3.1. Phase 1: Identification of the resilience policies

To obtain the final list of resilience policies (see Fig. 1), three differ- ent research methodologies were applied sequentially. First, a collabo- rative method called Group Model Building (GMB) was used to gather knowledge from experts. GMB is a collaborative methodology that en- ables fragmented knowledge, initially residing in the minds of different agents, to be integrated into aggregated models (Richardson and Andersen, 1995). Three workshops were arranged in San Sebastian (Spain) within the context of the SEMPOC (Simulation Exercise to Man- age Power Cut Crises) European project in the field of Critical Infrastruc- ture Protection (CIP) during 2009–2011. Fifteen experts from different arenas such as energy companies, first responders, and organizations for civil protection, health care and CI protection took part in the pro- cess. The workshops provided a wealth of information about the variety of perspectives on crisis management as well as the policies that build up a system's resilience level (Hernantes et al., 2012a; Hernantes et al., 2012b). Furthermore, the resilience policies defined in the GMB workshops were integrated based on previous resilience frameworks defined in the literature review. As a result of this research, the prelim- inary list of resilience policies was created.

In order to provide more confidence in the initial list of policies, sev- eral previous large-scale crises were analyzed using the multiple case studies method (Yin, 1994). Major nuclear accidents, blackouts, oil spills, mining accidents and air traffic accidents were studied to obtain evidence of the consequences of having a low or high degree of effective implementation of each policy and to complete the initial list of policies. Through this study, the initial version of the resilience policies for CIs was improved (Labaka et al., 2013).

Finally, a Delphi method was applied in order to complete and vali- date the list of resilience policies. Because the experts who took part in the GMB workshops were mainly concerned with energetic crises and the aim of this framework was to be suitable for all kind of CIs, a Delphi process was carried out with experts from diverse CI sectors. Delphi is a systematic and iterative process for structuring group communication in order to develop a commonly agreed framework that includes

ethodology.

Table 1 Resilience policies classified by the resilience type and resilience dimension.

Internal resilience External resilience

Technical resilience

CI safety design and construction

External crisis response equipment

CI maintenance CI data acquisition and monitoring system CI crisis response equipment

Organizational resilience

CI organizational procedures for crisis management

First responder preparation

CI top management commitment

Government preparation

CI crisis manager preparation Trusted network community CI operator preparation Crisis regulation and

legislation Economic resilience

CI crisis response budget Public crisis response budget

Social resilience Societal situation awareness

24 L. Labaka et al. / Technological Forecasting & Social Change 103 (2016) 21–33

convergent as well as divergent views of the participants about a com- plex problem (Linstone and Turoff, 2011; Linstone and Turoff, 1975; Dalkey, 1969; Okoli and Pawlowski, 2004) via multiple rounds of ques- tionnaires and feedback among them (Skulmoski et al., 2007).

Fifteen experts from different sectors (academia, transport, energy, and first responders) took part in the process. Although initially twenty-one experts agreed to take part in this Delphi process, finally fif- teen experts completed the entire process. Most of the experts who par- ticipated in the Delphi method did not take part in the GMB process (only three took part in both). In total, two rounds were carried out. The same questionnaire was used in both rounds and in the second round the experts had to reevaluate their initial answers based on the other experts' answers from the first iteration. The experts were asked to assess from 0 to 5 (0 being the lowest value and 5 the highest value) the appropriateness of each policy within the corresponding re- silience dimension (technical, organizational, economic and social) as well as to suggest other policies to build up each resilience dimension. As a result, an improved version of the list of policies was obtained. Section 4.1 presents the final list of resilience policies.

3.2. Phase 2: Development of the Influence table

The aim of this second phase of the research methodology was to as- sess the influence of each resilience policy on the three resilience lifecycle stages (prevention, absorption and recovery). In order to gath- er this information, we used the same Delphi process that was used to validate the resilience policies; the same group of experts participated but they were given different questionnaires. Once the final list of resil- ience policies was defined, the experts were asked to assess from 0 to 5 the influence of each policy on the three resilience lifecycle stages, where 0 indicates no influence and 5 indicates strong influence. Again, two additional rounds were carried out to assess the influence level of the policies, and the final results from the questionnaire are shown in section 4.2.

3.3. Phase 3: Development of the implementation methodology

In the third phase, a methodology was defined in order to help crisis managers implement the resilience policies in the most effective way, meaning that CIs are able to receive the full benefit that each policy pro- vides. In this case, we chose to use a survey (Marsden and Wright, 2010; Forza, 2002) to collect information from experts regarding the most beneficial order in which to implement the policies.

A questionnaire was sent to a sample of forty-five experts in the field of crisis management and CIs from all over the world and covering a range of sectors, from energy, transport, telecommunications and water to academia, first responders, and national civil protection. We selected a web-based questionnaire since it is inexpensive, it is easy for respondents to use, it keeps data from being lost, and it is the fastest way to receive responses and the easiest way to involve experts. The ex- perts were asked to rank 16 resilience policies in terms of the order in which they should be implemented in order to get the most effective re- sult from each policy. In total, twenty-five experts responded to the questionnaire. The results obtained from the survey as well as the im- plementation methodology developed from the survey are presented in section 4.3.

Based on the results of the three research phases described in this section (see Fig. 1), we developed the resilience framework for CIs, which is presented and explained in detail in the next section.

4. Resilience framework for critical infrastructures

The framework is composed of three main elements: a list of resil- ience policies that have to be implemented in a CI in order to improve its resilience level, an influence table where the influence of each resil- ience policy on the three resilience lifecycle stages (prevention,

absorption, and recovery) is assessed, and finally, an implementation methodology where the temporal order in which the resilience policies should be implemented in practice is presented.

4.1. Resilience policies

As stated above, CIs depend on external stakeholders such as gov- ernment, first responders and society when coping with crises (Yusta et al., 2011; Hernantes et al., 2013). Because crises initiated in a CI may become serious and affect a large number of people, external stake- holders need to cooperate with the CI or even lead the crisis resolution in the most appropriate way. Therefore, our approach considers rele- vant to take into account the resilience level of external stakeholders when analyzing the resilience level of a CI.

Given that the resilience level of the focal CI, which is where the trig- gering event occurs, may be different from the resilience level of the rest of the external entities, we divide the resilience level of the overall sys- tem (including the CI and external stakeholders involved) into two dif- ferent resilience types (Labaka et al., 2015): internal resilience, which refers to the resilience level of the focal CI and, external resilience which corresponds to the resilience level of the rest of external stakeholders.

We identified dimensions within each of the four resilience dimen- sions defined in the literature (technical resilience, organizational resil- ience, economic resilience, and social resilience (Bruneau et al., 2003; Multidisciplinary Center for Earthquake Engineering Research (MCEER), 2008; Zobel, 2010; Gibson and Tarrant, 2010)). In addition, we divided internal resilience into three dimensions—technical, organi- zational and economic—while we broke external resilience down into four dimensions—technical, organizational, economic, and social resilience.

Once the resilience types and dimensions were identified based on a literature review, the resilience policies that should be implemented in order to improve the resilience level of CIs were identified based on the knowledge gathered from our various experts.

The data collected in Section 3.1 yielded a total of sixteen resilience policies. Table 1 presents the complete list of resilience policies classi- fied by resilience type and resilience dimension, followed by a descrip- tion of each policy.

4.1.1. CI safety design and construction This policy refers to the safety level of a CI and its ability to prevent a

crisis from occurring and to efficiently absorb the magnitude of the im- pact. Having safety sub-systems and redundant components and sub- systems allow crises to be prevented and ensures the functioning of the CI (Bruneau et al., 2003; Johnsen, 2010). However, having a complex system with many additional redundant and safety systems makes it

25L. Labaka et al. / Technological Forecasting & Social Change 103 (2016) 21–33

difficult to manage and control its functioning (Perrow, 1984; Leveson, 2004; Sagan, 2004). Therefore, when designing the CI, it is important to reduce complexity as well as loosen the tight integration between sys- tems, which will prevent a triggering event from expanding to other systems. Internal and external audits should also be carried out to en- sure the proper functioning of the CI.

4.1.2. CI maintenance Not only should a CI be well designed and built, but high quality

maintenance activities must also be performed periodically in order to guarantee a high level of reliability. Having a good level of maintenance helps to withstand incidents and also reduces the magnitude of the im- pact and the time to recover.

4.1.3. CI data acquisition and monitoring system Having systems to monitor the state of the CI helps ensure its proper

state. Setting up the sensors needed to gather information from the CI and installing adequate software to monitor the CI's performance are some of the main activities that should be carried out in order to achieve a high implementation level of this policy.

4.1.4. CI crisis response equipment This policy refers to the emergency equipment that a CI should have

when a crisis occurs to absorb the impact and ensure the safety of the workers at the CI. Emergency equipment should be reliable so that it functions correctly when it is needed, and it should be available for use when a crisis occurs.

4.1.5. CI organizational procedures for crisis management This policy corresponds to the preparation and the capacity of an or-

ganization to deal with crises and incidents as well as the ability to co- ordinate with external stakeholders such as government and first responders. CIs should develop crisis management and coordination procedures with external stakeholders in order to fully define the re- sponse actions and the responsibilities of each worker before a crisis occurs.

4.1.6. CI top management commitment Top managers should be committed to the resilience building pro-

cess, and they must promote a resilience-based culture, attitudes and values within the CI. They are responsible for deploying resources to promote worker commitment and training and to establish the techni- cal measures required to prevent a crisis occurrence and absorb the impact.

4.1.7. CI crisis manager preparation Crisis manager preparation involves the capacity of crisis managers

to detect early warning signals, communicate to the stakeholders and analyze triggering events to propose new preventive measures for the future. In addition, managers also have to develop their sensemaking capacity (Gilpin and Murphy, 2008), which is the ability to understand an unexpected event, adapt to it, and make the correct decisions in a stressful situation and without complete information.

4.1.8. CI operator preparation Operators at the CI must be adequately trained prior to the occur-

rence of a crisis so they know how to respond when a crisis does occur. Operators should take training courses to learn the response pro- cedures and protocols and develop their response and coordination abilities (Resilient Organisations. Resilience Indicators. 2012). Operators should also be committed to the safety of the company since they can help detect early warning signals and prevent a crisis occurrence (Resilient Organisations. Resilience Indicators. 2012).

4.1.9. CI crisis response budget When a triggering event occurs, monetary resources are needed to

absorb the impact and recover the initial state as soon as possible. CIs should have monetary resources set aside in order to cover repairs and replacements just after the triggering event happens and until an acceptable level of performance that guarantees society's welfare is attained (Resilient Organisations. Resilience Indicators. 2012).

4.1.10. External crisis response equipment External stakeholders such as first responders, government and soci-

ety should also have reliable and adequate equipment to cope with a crisis. Furthermore, having redundant equipment would ensure the availability of this equipment when a component or a subsystem gets damaged. CIs should advise external stakeholders about the equipment required, especially when specific equipment is needed. In case of a se- vere crisis, equipment could also be gathered from foreign countries.

4.1.11. First responder preparation This policy refers to how first responders (fire fighters, emergency

units, policemen, military, etc.) are prepared to face a crisis. Prior to the occurrence of a crisis, they should be trained to absorb and bounce back from a crisis and learn about the special characteristics of the CIs near them in order to be able to properly respond when a crisis occurs. How they should act in dangerous environments and how they should organize themselves and coordinate with each other need to be defined before a critical event takes place.

4.1.12. Government preparation Governments should be well prepared for crisis management. Gov-

ernments should be aware of the possible incidents that could lead to a big crisis and they should be committed to the crisis management pro- cess. Government agencies should develop response procedures and ac- quire the leadership and communication skills needed to properly manage and inform in case of a crisis (Carrel, 2000; Boin, 2009). Further- more, government agencies are also responsible for efficiently coordi- nating the network of stakeholders involved in the absorption and recovery activities (Carrel, 2000; Boin, 2009).

4.1.13. Trusted network community Creating a network of stakeholders (CI owners, regulators, govern-

ment, etc.) in which agents involved in a crisis can trust each other to share experiences and lessons learned may improve crisis management knowledge and increase the number of collaboration agreements that provide help with crisis prevention and resolution (Resilient Organisations. Resilience Indicators. 2012; Ruffner et al., 2010; Snyder and de Souza Briggs, 2003; Wenger et al., 2002). These networks should promote research in the field of CI protection and safety to improve CI resilience levels.

4.1.14. Crisis regulation and legislation This policy refers to the maturity level and compliance level of regu-

lations and laws. Having well defined and updated regulation and legis- lation results in infrastructures that are safer and better prepared to prevent a crisis occurrence and that are to better able to handle one when it does occur. Furthermore, the regulations and laws should be regularly updated and reviewed to identify who is in charge in case a crisis occurs.

4.1.15. Public crisis response budget As in the case of the CI crisis budget, public institutions should have a

pool of money set aside in case a crisis occurs, in order to help stake- holders and society. This extra funding allows organizations, society and first responders to obtain resources within a reasonable time. Mon- etary resources will allow response and recovery activities to be per- formed, such as repairing and rebuilding physical systems, and disbursement of compensations to affected CIs and people.

26 L. Labaka et al. / Technological Forecasting & Social Change 103 (2016) 21–33

4.1.16. Societal situation awareness Not only should governments and first responders be prepared to

handle crises, but society can also play an important role in resolving a crisis. Society's situation awareness, which refers to the degree that the general public is aware of the levels of risk and vulnerability it faces in a crisis, and its commitment to avoiding a crisis occurrence re- duce the probability that a crisis will occur and the magnitude of the im- pact and improve the ability of society to respond (Resilient Organisations. Resilience Indicators. 2012; Shaw et al., 2009). Further- more, the collaboration and information that society can provide may be crucial to enhancing crisis management.

4.2. Influence table: influence of resilience policies on resilience lifecycle stages

All the policies have different influences on the different stages of the resilience lifecycle. There are some policies that are more effective in preventing a crisis occurrence, and there are others that are more im- portant in the absorption and recovery phases. Therefore, the aim of the

Table 2 Results of the Delphi process classified by sectors.

Resilience policies Resilience stages Academia (5 exp.)

CI safety design and construction Preven. 3.6 Absorp. 4.6 Recove. 4.6

CI maintenance Preven. 3 Absorp. 3.6 Recove. 3.4

CI data acquisition and monitoring sys. Preven. 3.4 Absorp. 3.8 Recove. 3.8

CI crisis response equipment Preven. 2.4 Absorp. 4.4 Recove. 4

CI org. procedures for crisis mgmt. Preven. 3.8 Absorp. 4.6 Recove. 4.2

CI top management commitment Preven. 4.4 Absorp. 4.6 Recove. 4

CI crisis manager preparation Preven. 3.8 Absorp. 4.6 Recove. 4.2

CI operator preparation Preven. 4 Absorp. 4.2 Recove. 4.2

CI crisis response budget Preven. 2.8 Absorp. 3.4 Recove. 4.4

External crisis response equipment Preven. 3 Absorp. 3.4 Recove. 2.6

First responder preparation Preven. 3.8 Absorp. 4.2 Recove. 4.2

Government preparation Preven. 4.2 Absorp. 4 Recove. 3.8

Trusted network community Preven. 3.2 Absorp. 4 Recove. 4

Crisis regulation and legislation Preven. 4.2 Absorp. 3.8 Recove. 4

Public crisis response budget Preven. 2.2 Absorp. 3.6 Recove. 4

Societal situation awareness Preven. 4 Absorp. 4.2 Recove. 4.6

second part of the resilience framework is to assess the influence of each resilience policy on the three resilience lifecycle stages: prevention, ab- sorption, and recovery.

Based on the information gathered from the experts in the Delphi pro- cess, an influence table was developed. The experts were asked to evalu- ate the influence of each policy on the three resilience lifecycle stages on a scale of 0 to 5 (0 being no influence and 5 being strong influence). Table 2 shows the answers given by the experts, classified by the sector they be- long to.

After analyzing the answers, data were ordered in a way that allows the results to be interpreted more easily. As the range of the mean values is concentrated in the middle values (between 2.3 and 4.4), in order to de- termine more precisely the influence of each resilience policy this re- search defines a new scale with a more suitable range of values as an approach to analyze the data. The new scale was divided into seven levels, with the following ranges for the arithmetic mean values (see Table 3). Since all the mean values were concentrated within the values 2.3 and 4.4, we define tighter ranges between these centric values, and wider ranges for the extreme values. Table 3 shows the detailed values of the new levels.

Transport (2 exp.)

Energy (4 exp.)

First responders (4 exp.)

Arithmetic mean

4 4.8 5 4.3 3.5 4 4 4.1 3.5 3.3 3.3 3.8 4.5 3.8 4.8 3.9 2.5 2.8 3.3 3.1 2 2.5 3 2.9 3.5 3.5 5 3.9 4 3.3 4.8 3.9 3.5 3.8 4 3.8 3.5 3 2.8 2.8 4.5 4.3 4.5 4.4 4 3.8 4 3.9 5 3.8 4 4.0 4 3.8 4.3 4.2 4.5 3.5 3.3 3.8 5 4.3 4.5 4.5 4.5 4 4.5 4.4 4 3.8 3 3.7 5 4 4 4.1 4 4 4.8 4.4 4.5 3.5 3.5 3.7 4 2.5 2.8 3.2 3.5 4 4.3 4.1 3 4.5 4.3 4.1 4 2.3 2.5 2.7 3.5 3 4 3.5 3.5 4.5 4.5 4.3 3.5 1.3 1.8 2.3 3.5 3.3 4.5 3.7 3.5 4 4.5 3.6 1.5 1.8 1.5 2.3 4 3.3 4.5 4.0 3.5 4 3.5 3.9 3 2.3 2 2.9 3.5 3.8 4.5 4.0 3 4.3 4.5 4.0 3.5 2.5 3.3 3.0 3 3 4 3.6 2.5 3.3 4.3 3.7 3.5 4.8 4 4.3 2.5 3.8 2.8 3.3 2 3 2.8 3.1 2.5 1.5 3 2.3 4 3.3 3.3 3.5 3 4.5 4.5 4.1 2.5 1.3 3.8 3.0 4 3 4 3.8 4 4 4 4.2

Table 3 Range of arithmetic mean values for the levels of the new scale.

Levels in the new scale Range of arithmetic mean values

Extremely low 0–2.0 Very low 2.1–2.5 Low 2.6–3.0 Regular 3.1–3.5 High 3.6–4.0 Very high 4.1–4.5 Extremely high 4.6–5.0

Table 5 Relationship between the order provided by the experts and the new stages defined for the analysis of the data.

Order provided by the experts New stages for the analysis of the data

1, 2, 3 1st stage 4, 5, 6 2nd stage 7, 8, 9 3rd stage 10, 11, 12 4th stage 13, 14, 15, 16 5th stage

27L. Labaka et al. / Technological Forecasting & Social Change 103 (2016) 21–33

Based on this new scale, the final influence table that defines the in- fluence of each resilience policy on the three resilience lifecycle stages was developed (see Table 4).

Results in Table 4 show that during the prevention stage, internal policies rather than external policies most contribute to the prevention of a crisis occurrence. The CI holds the primary responsibility for keep- ing a crisis from occurring by ensuring a robust and safely maintained infrastructure and raising the crisis awareness level of the workers as well as establishing the adequate procedures for incident management. On the other hand, during the absorption and recovery stages, the influ- ence of external stakeholders becomes fundamental. CIs require a good level of internal resilience as well as external resilience in order to prop- erly cope with crises. There are a few exceptions in which the influence of the policy does not follow the influence patterns just noted. Crisis

Table 4 Influence of the resilience policies on the three resilience lifecycle stages.

Resilience dimensions Resilience policies

In te

rn a

l re

si li

e n

c e

Technical resilience

CI Safety Design and Construction

CI Maintenance

CI Data Acquisition and Monitoring System

CI Crisis Response Equipment

Organizational resilience

CI Organizational Procedures for Crisis Management

CI Top Management Commitment

CI Crisis Manager Preparation

CI Operator Preparation

Economic resilience CI Crisis Response Budget

E x

te rn

a l

re si

li e

n c e

Technical resilience

External Crisis Response Equipment

Organizational resilience

First Responder Preparation

Government Preparation

Trusted Network Community

Crisis Regulation and Legislation

Economic resilience Public Crisis Response Budget

Social resilience Societal Situation Awareness

regulation and legislation policy, for instance, has the greatest influence during the prevention stage despite being an external policy. Moreover, policies related to economic resilience mainly influence the recovery stage by providing resources to bounce back to the normal situation, but their influence on the other two stages is low.

Focusing on the results classified by the sectors the experts belong to (see Table 2), it can be seen that there are minor disagreements among experts regarding some policies. For example, if we look at CI mainte- nance, academics think that its influence in preventing a crisis occur- rence is low, believing that maintenance activities are more important during the absorption and recovery phases. This is contrary to the opin- ion of the non-academic (practitioners) experts. This difference might be because practitioners might have experienced a crisis situation due to a low maintenance level. Another slight difference can be noticed

Prevention Absorption Recovery

Very high Very high High

High Regular Low

High High High

Low Very high High

High Very high High

Very high Very high High

Very high Very high High

Regular Very high Very high

Low Regular Very high

Very low High High

Very low High High

Low High High

Low High High

Very high Regular Regular

Very low Regular Very high

Low High Very high

Table 7 Range of mean stages in the new scale.

Range of mean stages Stage

1–2.4 1st 2.4–2.8 2nd 2.8–3.2 3rd 3.2–3.6 4th 3.6–5 5th

28 L. Labaka et al. / Technological Forecasting & Social Change 103 (2016) 21–33

with respect to Crisis Regulation and Legislation policy. Academics be- lieve that this policy helps in recovery whereas the rest of the sectors do not concur. The reason for this difference may lie in the fact that prac- titioners mostly use the regulation and the law to define prevention measures and preparedness activities. Finally, experts from the trans- port field agree that the trusted network community does not influence the recovery stage, whereas the rest of the experts think that it does. The last difference could lie in the maturity level of these communities in the area of crisis management, meaning they lack the communities of prac- tice that allow them to share knowledge and lessons learned. However, the differences are very small, and therefore it is hard to reach general- izable conclusions.

4.3. Implementation methodology for the resilience policies

The resilience policies are closely related to each other in that some of the policies require others to have been implemented beforehand to be effective. Additionally, some policies take longer to have an effect than others, so not all the policies should be implemented at the same moment.

For this reason, the third element of the framework is an implementa- tion methodology based on the order of implementation that the experts recommended in section 3.3. The aim of the implementation methodolo- gy is to provide guidelines about the temporal order in which the sixteen resilience policies should be implemented in order to attain the maxi- mum degree of effectiveness of the CI resilience framework.

After analyzing the results provided by the experts we concluded that there are some policies that need to be implemented at the begin- ning of the process since they are required for the implementation of others. In turn, others are placed in the last positions, as they necessarily must be built on previous policies. Finally, there are also some policies that require that other policies be implemented beforehand, but they also affect the effectiveness of others. Therefore, in order to facilitate the implementation of the framework we grouped these sixteen steps into a more practical number of stages, creating a new scale where the temporal order is maintained between stages. In total, we defined five stages in order to represent the most efficient temporal order in which the policies should be implemented based on the relationship presented in Table 5. The first three steps defined by the experts were grouped into the first stage in the new scale. The following three steps defined by the experts were classified in the second stage of the new

Table 6 Percentages of how many times each policy has been placed in each stage. The last column represents the mean stage for each policy.

Resil. Types

Resilience dimension Resilience policies Mean stage

Internal resilience

Technical resilience

CI safety design and construction

1.73

CI maintenance 2.59 CI data acquisition and monitoring system

2.91

CI crisis response equipment 2.95

Organizational resilience

CI organizational procedures for crisis management

2.41

CI top management commitment

2.05

CI crisis manager preparation 3 CI operator preparation 3.36

Economic resilience CI crisis response budget 3.23

External resilience

Technical resilience External crisis response equipment

4.41

Organizational resilience

First responder preparation 3.32 Government preparation 3.05 Trusted network community 4.14 Crisis regulation and legislation

2.95

Economic resilience Public crisis response budget 4.18 Social resilience Societal situation awareness 3.73

scale. Steps 7, 8 and 9 defined by the experts were grouped into the third stage of the new scale. Steps 10, 11 and 12 were classified into the fourth stage of the new scale and finally, the last four steps were classified into the fifth stage of the new scale.

Table 6 summarizes the results obtained from the survey based on the new stages defined for the analysis of the data. The numbers in the last column represent the mean stage for each policy.

Similar to the Delphi process carried out to define the influence table (see section 3.2), mean stages are concentrated in the middle values. Thus, a new scale based on the mean stages in Table 6 was defined (see Table 7).

Based on the new scale, the implementation methodology for the re- silience framework was defined, determining the stage in which each policy should be implemented (see Table 8). For what follows, keep in mind that by implement we mean that each policy's implementation is initiated during a specified stage, but that the policy is not completely implemented during that same stage; instead it is expected that the pol- icies will continue developing over time.

4.3.1. First stage There are two policies that are the driving forces in beginning, pro-

moting, and encouraging the improvement of resilience in CIs. First, hav- ing a safely designed and built infrastructure is essential to improving the resilience of CIs. Second, the commitment of top management towards the resilience-building process is vital in allocating resources, promoting a resilience-based culture, and increasing the engagement of the workers.

4.3.2. Second stage In the second stage, two new policies would be added to the ones

from the first stage. Not only do CIs need to be well designed and

Table 8 The stage in which each policy is implemented in the implementation methodology.

Resil. types

Resilience dimension

Resilience policies 1st

stage 2nd

stage 3rd

stage 4th

stage 5th

stage

In te

rn a

l re

si li

e n

c e

Technical resilience

CI Safety Design and Construction

CI Maintenance

CI Data Acquisition and Monitoring System

CI Crisis Response Equipment

Organization– al resilience

CI Organizational Procedures for Crisis

Management

CI Top Management Commitment

CI Crisis Manager Preparation

CI Operator Preparation

Economic resilience

CI Crisis Response Budget

E x

te rn

a l

re si

li e

n c e

Technical resilience

External Crisis Response Equipment

Organization– al resilience

First Responder Preparation

Government Preparation

Trusted Network Community

Crisis Regulation and Legislation

Economic resilience

Public Crisis Response Budget

Social resilience

Societal Situation Awareness

29L. Labaka et al. / Technological Forecasting & Social Change 103 (2016) 21–33

built, but maintenance activities should also be carried out to ensure the reliability of the components and the CIs and to prevent the accumula- tion of errors. Therefore, the CI maintenance policy should be imple- mented in this second stage. CI organizational procedures for crisis management should also be developed to properly manage crises. This means that internally, the CI should prepare so it is able to deal with a crisis. Guidelines should be drafted, and the responsibilities of each worker should be established along with the coordination procedures with external stakeholders so crises are better handled.

4.3.3. Third stage In this stage, five new policies are introduced. First, CI data acquisi-

tion and monitoring systems should be implemented throughout the in- frastructure to gather information about the state of the infrastructure and to be able to anticipate an incident. Second, CI crisis response equip- ment also has to be acquired in order to be able to absorb the impact and ensure the safety of the workers. Third, CI crisis manager preparation is introduced since managers are the ones responsible for detecting early warning signals, analyzing them and communicating them to the inter- nal staff and if necessary to external stakeholders. They are aware of all possible incidents and they have the responsibility of preparing the or- ganization to perform efficiently during a crisis. Fourth, government preparation should be addressed since governments also play an impor- tant role in crisis management. They have the authority and the capacity to increase external entities' awareness of and commitment towards the resilience-building process and they can provide the resources to ac- quire equipment and help with the crisis resolution. Fifth, together with government preparation policy, governments and their public enti- ties should develop crisis regulations and laws in order to establish the minimum requirements that CIs need to ensure their safety and high re- liability. It is worth noting that these last two policies should be con- stantly improved and provided with feedback.

4.3.4. Fourth stage The CI operator preparation, CI crisis response budget, and first re-

sponder preparation policies are implemented in this stage. Once top management is committed, the crisis management procedures are established and crisis managers are well prepared, and operators should be prepared to face crises. They should be given training courses and do some table-top exercises and emergency drills to improve their crisis management skills and awareness. Furthermore, CIs have to set aside some monetary resources or take out insurance so they are able to ab- sorb the extra costs that arise from a crisis. Externally, the preparation of first responders must be addressed to ensure they respond properly to a crisis.

4.3.5. Fifth stage In this final stage, the remaining external policies are implemented.

In order to be able to respond appropriately, it is important that external entities have reliable and sufficient response equipment (external crisis response equipment). Furthermore, a trusted network community has to be created, where stakeholders share information and experiences with other agents involved and improve their crisis management knowledge. Public crisis response budget is also improved in order to have the monetary resources needed to respond to crises. Finally, socie- tal situation awareness is enhanced since society can help handle a crisis, either by preventing its occurrence or at least not making it worse. Soci- ety should be aware of a crisis occurrence and prepared to cope with cri- ses in the most efficient way.

5. Case studies

Crisis management and the resilience-building process in particular are primarily applied disciplines, and thus empirical research is needed to add more relevance to the proposed framework (McLachlin, 1997). The framework was implemented in two critical infrastructures in

order to diagnose the resilience level of CIs and identify improvement points for enhancing their resilience level. Furthermore, the case studies ensured that the framework was complete and useful in helping crisis managers build resilience.

A nuclear plant and a water distribution company were chosen since they have different characteristics and present diverse challenges when building up resilience. The first study was carried out at the nuclear plant, where a researcher stayed on-site for six months full-time. Vari- ous information sources such as interviews, internal and external docu- ments and procedures, archival records and direct observation were used to gather information for the framework. The second case study was carried out in a water distribution company but in this case, due to limited access to the company, the information was only gathered through four interviews with the general manager of the company.

The framework helped estimate the resilience level of the CI by iden- tifying improvement opportunities. The methodology followed to im- plement the framework consisted of four steps: first, the resilience policies were divided into resilience sub-policies in order to better de- fine each resilience policy. Second, information was gathered about the different actions and measures already implemented for each resil- ience policy and sub-policy. Third, interviews and direct observations were conducted at the plant in order to verify the information obtained. Finally, the information gathered was assessed by researchers and im- provement opportunities for different resilience policies were sug- gested to both CIs. In what follows, the analysis carried out within the case studies is presented for each resilience policy.

5.1. CI safety design and construction

In the case of the nuclear plant, the most critical element of the nu- clear plant is the core of the reactor, and therefore all the safety systems are geared towards preventing or absorbing any event affecting it. How- ever, there are not any systems that help to bouncing back during the recovery phase. The nuclear plant has thirteen safety systems that are activated over time to mitigate and absorb impacts to the core of the re- actor and eight support systems that provide support to safety systems. Furthermore, the nuclear plant has duplicated all the critical systems and several safety systems are implemented to stop incidents.

In the case of the water company, there are deficiencies in design and construction in that there is no redundancy in the distribution network's supply tubes that run from the dams to the purification plants and from the purification centers to the water tanks since duplicating the whole system is costly. However, both CIs have several redundant power supply systems to deal with a power outage and help absorbing the impact. With regard to audits, both CIs conduct internal and external audits, not only the ones required by law but also the ones voluntarily accepted by the CI in order to verify the fulfillment of all the require- ments. These audits help especially avoiding a crisis occurrence and ab- sorbing triggering events.

5.2. CI maintenance

Both companies have preventive as well as corrective maintenance activities in place. They basically help ensuring the reliability level of the security systems and avoiding the escalation of little incidents into major events. In the case of the nuclear plant, the frequency of preven- tive maintenance activities is defined based on experience (through the analysis of historical data) and based on a database from the Electric Power Research Institute (EPRI). The target of the nuclear plant is to have at least 60% preventive maintenance and at most 40% corrective maintenance. Therefore, the nuclear plant schedules is maintenance ac- tivities as a function of how well it is meeting this target.

Regarding the water distribution company, using a computer-based information system they establish the frequency, the quantity, and the points where preventive maintenance activities should be performed. Unlike the nuclear plant, the water distribution company does not

30 L. Labaka et al. / Technological Forecasting & Social Change 103 (2016) 21–33

have any clear objective about the distribution of preventive and correc- tive maintenance activities.

5.3. CI data acquisition and monitoring system

The nuclear plant has different types of instrumentation in place to gather data such as pressure, flow, temperature, water level, radioactivity and electrical power supply, and the information is displayed in different places such as control panels in the plant and control room. They also save the data in the plant data information system. In order to evaluate the ap- propriateness of the collected data there are some systems in place that check whether the data is within the proper range of values. Alarms are triggered and workers are notified in case a value is outside the limits.

Similarly, the water distribution company also gathers data, saves data and displays data in two places: on the control panels in the water purifi- cation centers and in the Control and Monitoring Center. Furthermore, alarms that give alerts when there is a problem are classified in two groups based on the severity of the problem. As already defined in the in- fluence table of the resilience framework for critical infrastructures (Table 4), these systems allow collecting data about the CI status during the prevention stage, absorbing failures when an incident occurs, and collecting information about the system improvement during the recov- ery stage.

5.4. CI crisis response equipment

Both companies have several pieces of emergency equipment on site in order to respond to an incident and ensure the safety of the workers. This equipment helps mostly during the absorption and recovery stages bouncing back to the normal situation. In the case of the nuclear plant, the material is distributed within the nuclear plant and outside the physical boundaries of the nuclear plant, whereas in the case of the water distribution company, fire hydrants are the only elements that are distributed through the whole network for the firefighters.

5.5. CI organizational procedures for crisis management

Both CIs have several defined operating and organizational procedures in order to help avoid the escalation of incidents and to know how to re- spond when a crisis occurs and the responsibility that each worker has. Furthermore, both CIs establish an information system to track all the in- cidents and to gather lessons learned. However, while in the case of the nuclear plant all the workers can report an incident in the system, in the case of the water distribution company, only authorized people have the authority to do so. Finally, both companies have coordination procedures with external stakeholders within the off-site emergency plan.

5.6. CI top management commitment

In both cases, management seems to be completely committed to the resilience improvement process. The commitment of the top man- agement is essential during the three resilience lifecycle stages (preven- tion, absorption and recovery). The nuclear plant affirmed that the safety of the plant is a daily concern. Similarly, the general manager at the water distribution company assured us that the safety of the workers is the top priority for the company.

In order to promote a resilience-based culture within the company, the nuclear plant has a reporting system to collect workers' proposals to improve the safety of the nuclear plant in place. This system provides the nuclear plant with a high number of proposals to enhance resilience. In contrast, although the water distribution company affirmed that top managers evaluate and appreciate the contributions of the workers, it did not specify how this is promoted in the company.

5.7. CI crisis manager preparation

The role of the crisis managers is essential in all the resilience lifecycle stages. In both cases the responsibility for crisis management changes from the prevention stage to the absorption and recovery stages. Moreover, both CIs establish training programs depending on the crisis manager type. In the nuclear plant, crisis managers receive three types of training activities: training activities in the nuclear plant, seminars, and training with the simulator. In the water distribu- tion company, crisis managers are trained to gather data, detect early warning signals, and know how they should respond in case of a crisis.

Nevertheless, there was an improvement opportunity in both CIs. Both CIs were mainly focused on improving the management of already identified hazards and neither of them performed training activities to address how to handle unexpected and unplanned situations. Both CIs aimed to reduce the risk probability by improving technical aspects and preparing and establishing well defined response and protection procedures. Their crisis management approach was mainly based on risk management without taking into account the concept of resilience, which also focuses on the capacity to deal with unknown crises.

Finally, based on the direct observations carried out in the CIs and the interviews with managers, we concluded that, in both cases, crisis managers are constantly aware of possible incidents.

5.8. CI operator preparation

In the nuclear plant all the workers receive training courses on the emergency plans and procedures to be followed if a crisis occurs. As de- fined in the influence table of the resilience framework (Table 4), their influence is higher once the triggering event has occurred. In the case of the water distribution company, the operators do not receive training and are unaware of the emergency plan; only managers are trained to know how the organization should act when a crisis occurs.

In order to improve the awareness and commitment level of the workers, the nuclear plant encourages its workers to suggest new ideas or measures to help enhance the resilience level of the CI. Around 30% of the workers propose new improvement measures every year. In the case of the water distribution company, the general manager ensures that operators are committed and aware of the need for the improvement of resilience but they do not have any system in place to specifically pro- mote that.

5.9. CI crisis response budget

The nuclear plant takes out insurance policies to cover the costs that arise when a crisis occurs during the recovery stage. Almost the entire CI is covered by insurance policies. The case of the water distribution com- pany is different since this company is public. When a crisis occurs, re- sources are gathered from the local government. However, most of the essential elements at the distribution network are covered by insurance companies.

5.10. External crisis response equipment

Having adequate and reliable crisis response equipment is essen- tial in order to absorb the impact and recover efficiently. In the nu- clear plant, the collection and coordination of the external equipment are carried out from the operations coordination center with the local government's external relations department. Further- more, the nuclear plant collects more equipment through the nation- al Nuclear Plants Association. At the water distribution company, the collection and coordination of equipment are handled by the local first responders group.

31L. Labaka et al. / Technological Forecasting & Social Change 103 (2016) 21–33

5.11. First responder preparation

Both CIs have strong relationships with first responders. They are of utmost importance once the crisis has occurred. In the case of the nucle- ar plant some of them (six firefighters, one doctor, four nurses, and one specialist in radiological protection) are even on site in order to be able to respond immediately. Furthermore, the plant periodically trains mil- itary, police, firefighters and civil protection workers in order to show them the layout of the CI and the procedures to follow (the off-site emergency plan). In the water distribution company, although there are no first responders on site, they have direct contact with them. Fire- fighters have the plans and all necessary information for the distribution network and know how they need to act in case a crisis occurs.

With respect to emergency drills with external stakeholders, in the case of the water distribution company, they do not perform table-top exercises or simulations with first responders, whereas in the case of the nuclear plant they do them every year.

5.12. Government preparation

The influence of the government's preparation is higher in the crisis absorption and recovery phases. Due to the severity level of the nuclear industry, the national government of the country where the nuclear plant is located has a specific group called the national Nuclear Security Council (NSC) to control and manage the safety of the nuclear industry. The NSC is responsible for training the heads of government agencies and also the CIs and to develop off site emergency plans. When a crisis occurs, the NSC is accountable for managing the crisis effectively and for giving advice to the government regarding communication strate- gies and leadership issues.

However, the preparation level of the government regarding the water industry is different. In the water sector, the relevant national government does not have a similar specific group for controlling the safety of the water distribution companies and managing crises. How- ever, as the water distribution company is a public entity, they are in di- rect contact with the government and the water distribution company provides advice and knowledge about the decisions and actions that should be taken.

5.13. Trusted network community

The nuclear plant is involved in a community in which many nation- al and international nuclear associations also take part, such as the na- tional Nuclear Plants Association, the World Association of Nuclear Operators (WANO), and the International Atomic Energy Agency (IAEA). The international nuclear associations share information through their websites and periodical workshops and the national nu- clear plants association works through the NSC. Within the nuclear plant there is a department called Operational Experience, which is re- sponsible for sharing and gathering information and lessons learned with external stakeholders and other nuclear plants.

The water distribution company is member of the Association of Water Supply and Sanitation, which collaborates with other national water distribution companies. This association shares information and knowledge through their websites and by telephone in case specific in- formation is needed. However, the water distribution company does not usually share any experience or lessons learned with other companies publicly. However, they are in permanent contact with first responders so they can share all the information about the distribution network and its updates. These relationships help them during the absorption and re- covery stages obtaining help from others.

5.14. Crisis regulation and legislation

Having well defined and updated regulation and legislation is vital to have safe and reliable systems and prevent a crisis occurrence. In the

case of the nuclear plant, the NSC is responsible for developing regula- tions and updating them. The nuclear sector is a high-risk sector and thus safety is of primer importance. NCS is constantly updating the laws and regulations based on lessons learned from incidents and crises. In order to ensure that CIs comply with regulations and laws, an NSC in- spector works full-time in the nuclear plant, checking the plant's com- pliance level.

On the contrary, in the case of the water distribution company, there is no a specific group of experts that develop regulations and laws for the water industry. Nevertheless, laws and regulations are also updated and reviewed periodically, taking into account lessons learned from in- cidents and crises. Regarding inspections, unlike the nuclear industry, there are no on-site inspectors and the general manager tells us that in principle there is no penalty for not complying with laws or regulation.

5.15. Public crisis budget

Due to the CIs' lack of contact with the public, we were not able to gather evidence about this policy since economic issues were unknown to the members of the CIs.

5.16. Societal situation awareness

Both companies have a plan established to improve the preparation of the surrounding population as well as to make society aware of the safety and reliability level of the CIs. The nuclear plant has a plan to im- prove communication with the surrounding community once the crisis has occurred. They provide evidence about meeting all the regulatory and legislative requirements and help with social activities and the eco- nomic development of the surrounding community. Furthermore, they provide training and education seminars and perform some general emergency drills in order to improve the general public's crisis response capacity.

Lately, the water distribution company has made a significant effort to change the general public's awareness about the need to save water. The water company provides some training courses about the scarcity and importance of this resource and as a result society is more aware and is more careful about its consumption, thus reducing the number of incidents and crises. The general manager confirmed that the number of incidents have considerably reduced in recent years.

5.17. Discussion of the case studies

Based on the case studies carried out in the CIs, we analyzed the re- silience level of both CIs and we concluded that the resilience level of the nuclear plant was higher than the resilience level of the water distri- bution company, as might have been expected. Although both CIs pre- sented several improvement opportunities (as illustrated below), in the water distribution company most of the policies had incipient im- plementation levels.

Prior to our study, the crisis management approach of both CIs was mainly based on risk management and it did not take into account the perspective of resilience, which emphasizes the capacity to deal with unknown crises (Risk and Resilience Research Group and Center for Security Studies, 2011; Park et al., 2013). The application of our frame- work allowed them to recognize this weakness and increase their awareness and use some specific training plans.

Furthermore, most of the crisis management activities and systems in both CIs were focused on preventing the triggering event and the es- calation of incidents. They have few systems and procedures for the ab- sorption and recovery periods. The reason for this was that they were almost sure that nothing could happen with their prevention systems. However, our approach helped them to develop higher awareness about any near miss or unexpected event that could occur.

32 L. Labaka et al. / Technological Forecasting & Social Change 103 (2016) 21–33

Unlike the nuclear plant, for the water distribution company the reg- ulations and legislation governing this sector were incipient and there was little control regarding compliance with the rules and law. Further- more, the external audits were only performed when the company was established.

Finally, in both cases the internal resilience level was higher than the external resilience level. Most of the CIs' efforts were concentrated on improving the internal resilience level and little was done in order to enhance external resilience. The implementation of the framework highlighted the importance of the external part and made them detect areas of improvement.

The results of the study and the improvement opportunities found for each CI were communicated to both CIs. Furthermore, both CIs con- firmed that they found the framework to be very useful because it made them consider their current situation and discover new areas where their resilience level can be enhanced. In addition, in order to help the CIs improve their resilience, we provided them with the implementa- tion methodology so they could make sure their future actions were ef- fective and yielded the desired resilience level.

6. Conclusions, limitations and future research

Our research is a comprehensive advance in the implementation of resilience in critical infrastructures. Several authors (Boin and Van Eeten, 2013; Lekka and Sugden, 2011) argue that the literature provides very few empirical and case studies on the implementation of resilience principles in CIs. This research aims to overcome this gap and presents a preliminary step toward improving the resilience of CIs. Our work with the nuclear plant and the water distribution company, however, shows that critical infrastructures need to complement their approach and be aware that a transversal preparation is required in order to be capable of dealing with any kind of event, whether expected or unexpected.

Our resilience framework, which was based on empirical findings and expert knowledge, attempts to provide CIs with a tool that im- proves their resilience. The framework allows crisis managers to diag- nose the resilience level of their CI, to detect improvement opportunities and to project future actions or policies to enhance the CI's resilience level.

This framework was implemented in two CIs (a nuclear plant and a water distribution company) where the aim was to confirm that the framework provides value to CIs in by helping them improve their resil- ience level. In the case studies, the resilience levels of the two CIs were estimated and opportunities to enhance the CIs' resilience levels were identified. The diagnosis of the resilience level was based on the resil- ience policies included in the framework. Finally, the usefulness of the framework to improve the resilience level of the CI was reassured through the empirical research.

However, our research presents two major limitations. This frame- work aims to be applicable to all CIs, and thus it presents aggregated re- silience policies. When implementing this framework in a specific CI in a specific country, however, it needs to be adjusted to the specific sector and country where more detailed policies should be defined. Further- more, the influence of the resilience policies on the three resilience lifecycle stages can differ from one CI to another.

In addition, the resilience framework presents a qualitative ap- proach to improve resilience. The resilience policies qualitatively define the areas which should be improved and the activities that should be carried out to increase the resilience level. The lack of precise metrics is a drawback of our method which remains to be addressed in future work.

Based on these limitations, our research suggests that several steps will be performed in the future to improve the resilience framework for CIs. In order to provide a more quantitative approach to diagnosing and improving the resilience level of CIs, several metrics and indicators should be defined. Metrics and indicators common to all CIs will be de- fined, together with other specific metrics identified for each particular

CI adapted to the different nature of CIs. This approach will serve for the evaluation of current resilience level of CIs and so assess the enhance- ment provided by the applied measures. Furthermore, in order to ex- tend the validation process and increase crisis managers' confidence in this framework, more implementation case studies should be undertak- en in other CI sectors and in different countries.

Acknowledgements

The SEMPOC project was supported by the European Commission's Directorate-General of Justice, Freedom and Security as part of the framework for the European CIPS strategic objective (Grant no. JLS/ 2008/CIPS/024). The authors are thankful to the European Commission for providing finance for the SEMPOC project and also to the nuclear plant that participated in our study for allowing us to carry out this research and providing the necessary information.

References

Broad, W.J., 2011. Scientists project path of radiation plume. http://www.nytimes.com/ 2011/03/17/science/17plume.html?_r=0 (N. Y.).

Dempsey, J., LaFraniere, S., 2011. In Europe and China, Japan's crisis renews fears about nuclear power. http://www.nytimes.com/2011/03/17/business/global/17atomic. html?pagewanted=all (N. Y.).

Andersson, G., Donalek, P., Farmer, R., Hatziargyriou, N., Kamwa, I., Kundur, P., Martins, N., Paserba, J., Pourbeik, P., Sanchez-Gasca, J., Schulz, R., Stankovic, A., Taylor, C., Vittal, V., 2005. Causes of the 2003 major grid blackouts in North America and Europe, and rec- ommended means to improve system dynamic performance. IEEE Trans. Power Syst. 20, 1922–1928.

Union for the Coordination of Transmission of Electricity (UCTE), 2004U. Final report of the investigation committee on the 28 September 2003 blackout in Italy.

US–Canada power system outage task force, 2004. Final report on the August 14, 2003 blackout in the United States and Canada: causes and recommendations.

Larsson, S., Danell, A., 2006. The black-out in southern Sweden and eastern Denmark Sep- tember 23, 2003 .

Hall, J., 2010. Volcanic ash cloud leaves shops facing shortages of fruit. vegetables and medicine http://www.telegraph.co.uk/finance/newsbysector/retailandconsumer/ 7599042/Volcanic-ash-cloud-leaves-shops-facing-shortages-of-fruit-vegetables- and-medicine.html (United Kingdom).

Barr, R., 2010. Iceland's volcanic ash halts flights across Europe. http://www.guardian.co. uk/world/feedarticle/9032564 (Great Britain).

Mitroff, I., Anagnos, G., 2000. Managing Crises Before They Happen: What Every Executive And Manager Needs to Know About Crisis Management. AMACOM, New York.

Pearson, C.M., Clair, J.A., 1998. Reframing crisis management. Acad. Manag. Rev. 23, 59–76.

Coleman, L., 2004. The frequency and cost of corporate crises. J. Conting. Crisis Manag. 12, 2–13.

Boin, A., Lagadec, P., Michel-Kerjan, E., Overdijk, W., 2003. Critical infrastructures under threat: Learning from the anthrax scare. J. Conting. Crisis Manag. 11, 99–104.

Boin, A., 2004. Lessons from crisis research. Int. Stud. Rev. 6, 165–194. Lagadec, P., 2007. Crisis management in the twenty-first century:“unthinkable” events in

“inconceivable” contexts. Anonymous, editors. Handbook of Disaster Research, Springer, pp. 489–507.

Risk and Resilience Research Group—Center for Security Studies. Resilience and Risk Man- agement in Critical Infrastructure Protection Policy: Exploring the Relationship and Comparing its Use. 2011.

Park, J., Seager, T.P., Rao, P.S.C., Convertino, M., Linkov, I., 2013. Integrating risk and resil- ience approaches to catastrophe management in engineering systems. Risk Anal. 33, 356–367.

Gilpin, D.R., Murphy, P.J., 2008. Crisis Management in a Complex World. Oxford University Press, Oxford.

Turner, B.A., 1976. The organisational and inter-organisational development of disasters. Adm. Sci. Q. 21, 378–397.

Perrow, C., 1984. Normal Accidents: Living with High Risk Technologies. Princeton Uni- versity Press, New Jersey, USA.

Min, H.J., Beyeler, W., Brown, T., Son, Y.J., Jones, A.T., 2007. Toward modeling and simula- tion of critical national infrastructure interdependencies. IIE Trans. 39, 57–71.

Oliva, G., Panzieri, S., Setola, R., 2010. Agent-based input–output interdependency model. Int. J. Crit. Infrastruct. Prot. 3, 76–82.

Katina, P.F., Pinto, C.A., Bradley, J.M., Hester, P.T., 2014. Interdependency-induced risk with applications to healthcare. Int. J. Crit. Infrastruct. Prot. 7, 12–26.

Laugé, A., Hernantes, J., Sarriegi, J.M., 2014. Critical infrastructure dependencies: a holistic, dynamic and quantitative approach. Int. J. Crit. Infrastruct. Prot. 8, 16–23.

Chang, S.E., McDaniels, T.L., Mikawoz, J., Peterson, K., 2007. Infrastructure failure interde- pendencies in extreme events: power outage consequences in the 1998 ice storm. Nat. Hazards 41, 337–358.

Hämmerli, B., Renda, A., 2010. Protecting Critical Infrastructure in the EU. CEU, 2008. On the identification and designation of European critical infrastructures and

the assessment of the need to improve their protection. Off. J. Eur. Communities 114, 75p (Directive).

33L. Labaka et al. / Technological Forecasting & Social Change 103 (2016) 21–33

NIPP. National Infraestructure Protection Plan, 2009. US Department of Homeland Securi- ty. U.S.Department of Home Security. p. 175.

Australian Government, 2010. Critical infrastructure resilience strategy. Common Wealth of Australia.

Ouyang, M., 2014. Review on modeling and simulation of interdependent critical infra- structure systems. Reliab. Eng. Syst. Saf. 121, 43–60.

Yusta, J.M., Correa, G.J., Lacal-Arántegui, R., 2011. Methodologies and applications for crit- ical infrastructure protection: state-of-the-art. Energ Policy 39, 6100–6119.

Wardekker, J.A., de Jong, A., Knoop, J.M., van der Sluijs, J.P., 2010. Operationalising a resil- ience approach to adapting an urban delta to uncertain climate changes. Tech. Forcasting Soc. Chang. 77, 987–998.

Van de Walle, B., Turoff, M., 2008. Decision support for emergency situations. Information Systems and E-Business Management. 6, 295–316.

Weick, K.E., Sutcliffe, K.M., 2007. Managing the Unexpected: Resilient Performance in an Age of Uncertainty. 2nd ed. Jossey-Bass, San Francisco: Calif.

Elwood, A., 2009. Using the disaster crunch/release model in building organisational resil- ience. J. Bus. Contin. Emerg. Plan. 3, 241–247.

Boin, A., McConnell, A., 2007. Preparing for critical infrastructure breakdowns: the limits of crisis management and the need for resilience. J. Conting. Crisis Manag. 15, 50–59.

Lindblom, C.E., 1959. The science of “muddling through”. Public Adm. Rev. 19, 79–88. Turoff, M., Hiltz, S.R., White, C., Plotnick, L., Hendela, A., Yoa, X., 2009. The past as the fu-

ture of emergency preparedness and management. International Journal of Informa- tion Systems for Crisis Response and Management (IJISCRAM) 1, 12–28.

De Bruijne, M.L.C., 2006. Networked Reliability: Institutional Fragmentation and the Reli- ability of Service Provision in Critical Infrastructures. Faculty of Technology, Policy and Management, Delft University of Technology, Netherlands.

De Bruijne, M., Van Eeten, M., 2007. Systems that should have failed: critical infrastruc- ture protection in an institutionally fragmented environment. J. Conting. Crisis Manag. 15, 18–29.

Manyena, S.B., 2006. The concept of resilience revisited. Disasters 30, 434–450. Moteff, J.D., 2012. Critical infrastructure resilience: the evolution of policy and programs

and issues for congress. Congressional Research Service. Eusgeld, I., Nan, C., Dietz, S., 2011. “System-of-systems” approach for interdependent crit-

ical infrastructures. Reliab. Eng. Syst. Saf. 96, 679–686. Bruneau, M., Chang, S., Eguchi, R., Lee, G., O'Rourke, T., Reinhorn, A., Shinozuka, M.,

Tierney, K., Wallace, W., von Winterfelt, D., 2003. A framework to quantitatively as- sess and enhance seismic resilience of communities. Earthquake Spectra 19, 733–752.

Multidisciplinary Center for Earthquake Engineering Research (MCEER), 2008c. Engineer- ing Resilience Solutions. University of Buffalo.

Zobel, C.W., 2010. Representing perceived tradeoffs in defining disaster resilience. Decis. Support. Syst. 50, 394–403.

Gibson, C.A., Tarrant, M., 2010. A ‘conceptual models’ approach to organisational resil- ience. Australian Journal of Emergency Management 25, 6–12.

Brunsdon, D., Dalziell, E., 2005. Making organisations resililent: understanding the reality of the challenge. Proceedings of the Resilient Infrastructure Conference. Rotorua, pp. 27–34.

McEntire, D.A., 2001. Triggering agents, vulnerabilities and disaster reduction: towards a holistic paradigm. Disaster Prevention and Management: An International Journal. 10, 189–196.

Francis, R., Bekera, B., 2014. A metric and frameworks for resilience analysis of engineered and infrastructure systems. Reliab. Eng. Syst. Saf. 121, 90–103.

Roberts, K.H., Rousseau, D.M., 1989. Research in nearly failure-free, high-reliability orga- nizations: having the bubble. Eng. Manag. IEEE Trans. On. 36, 132–139.

Roberts, K.H., 1990. Some characteristics of one type of high reliability organization. Organ. Sci. 1, 160–176.

Lekka, C., 2011. High reliability organizations: a review of the literature. Health and Safety Executive.

Resilient Organisations, 2012. Resilience Indicators from http://www.resorgs.org.nz/ Content/what-is-organisational-resilience.html.

Parsons, D., 2007. National Organisational Resilience Framework Workshop: The Outcomes.

Johnsen, S.O., 2010. Resilience in risk analysis and risk assessment. In: Palmer, C., Shenoi, S. (Eds.), Critical Infrastructure Protection. Springer, Berlin, pp. 215–227.

Cutter, S.L., Burton, C.G., Emrich, C.T., 2010. Disaster resilience indicators for benchmarking baseline conditions. Journal of Homeland Security and Emergency Management. 7, 1–22.

Cohen, O., Leykin, D., Lahad, M., Goldberg, A., Aharonson-Daniel, L., 2013. The conjoint community resiliency assessment measure as a baseline for profiling and predicting community resilience for emergencies. Technol. Forecast. Soc. Chang. 80, 1732–1741.

Boin, A., Van Eeten, M.J.G., 2013. The resilient organization. Public Management Review. 15, 429–445.

Prior, T., Hagmann, J., 2014. Measuring resilience: methodological and political challenges of a trend security concept. Journal of Risk Research. 17, 281–298.

Lekka, C., Sugden, C., 2011. The successes and challenges of implementing high reliability principles: a case study of a UK oil refinery. Process. Saf. Environ. Prot. 89, 443–451.

Waller, M.J., Roberts, K.H., 2003. High reliability and organizational behavior: finally the twain must meet. J. Organ. Behav. 24, 813–814.

Richardson, G.P., Andersen, D.F., 1995. Teamwork in group model building. Syst. Dyn. Rev. 11, 113–137.

Hernantes, J., Labaka, L., Laugé, A., Sarriegi, J.M., 2012a. Group model building: a collabo- rative modelling methodology applied to critical infrastructure protection. Int. J. Organ. Des. Eng. 2, 41–60.

Hernantes, J., Labaka, L., Laugé, A., Sarriegi, J.M., 2012b. Three complementary approaches for crisis management. Int. J. Emerg. Manag. 8, 245–263.

Yin, R.K., 1994. Case Study Research: Design and Methods. 2nd ed. Sage publications, Thousand Oaks, CA.

Labaka, L., Hernantes, J., Rich, E., Sarriegi, J.M., 2013. Resilience building policies and their influence in crisis prevention, absorption and recovery. J. Homel. Secur. Emerg. Manag. 10, 1–29.

Linstone, H.A., Turoff, M., 2011. Delphi: a brief look backward and forward. Technol. Fore- cast. Soc. Chang. 78, 1712–1719.

Linstone, H.A., Turoff, M., 1975. The Delphi Method: Techniques and Applications. Addison-Wesley Pub. Co., Boston, M.A., USA.

Dalkey, N., 1969. An experimental study of group opinion. Futures 408–426. Okoli, C., Pawlowski, S.D., 2004. The Delphi method as a research tool: an example, design

considerations and applications. Inf. Manag. 42, 15–29. Skulmoski, G.J., Hartman, F.T., Krahn, J., 2007. The Delphi method for graduate research.

J. Inf. Technol. Educ. 6, 1–21. Marsden, P.V., Wright, J.D., 2010. Handbook of Survey Research. 2nd ed. Emerald Group

Publishing, United Kingdom. Forza, C., 2002. Survey research in operations management: a process-based perspective.

International Journal of Operations & Production Management. 22, 152–194. Hernantes, J., Rich, E., Laugé, A., Labaka, L., Sarriegi, J.M., 2013. Learning before the storm:

modeling multiple stakeholder activities in support of crisis management, a practical case. Tech. Forcasting Soc. Chang. 80, 1742–1755.

Labaka, L., Hernantes, J., Sarriegi, J.M., 2015. Resilience framework for critical infrastruc- tures: an empirical study in a nuclear plant. Reliab. Eng. Syst. Saf. 141, 92–105.

Leveson, N., 2004. A new accident model for engineering safer systems. Saf. Sci. 42, 237–270.

Sagan, S.D., 2004. The problem of redundancy problem: why more nuclear security forces may produce less nuclear security. Risk Anal. 24, 935–946.

Carrel, L.F., 2000. Training civil servants for crisis management. J. Conting. Crisis Manag. 8, 192–196.

Boin, A., 2009. The new world of crises and crisis management: implications for policymaking and research. Review of Policy Research. 26, 367–377.

Ruffner, J.W., Brodie, A.C., Holiday, C.L., Isenberg, T.H., 2010. Selecting and utilizing metrics for an internet-based community of practice. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp. 1254–1258.

Snyder, W.M., de Souza Briggs, X., 2003. Communities of Practice: A New Tool for Govern- ment Managers. IBM Center for The Business of Government.

Wenger, E., McDermott, R.A., Snyder, W.M., 2002. Cultivating Communities of Practice: A Guide to Managing Knowledge. Harvard Business School Press, Boston, Massachusetts.

Shaw, R.S., Chen, C.C., Harris, A.L., Huang, H.J., 2009. The impact of information richness on information security awareness training effectiveness. Comput. Educ. 52, 92.

McLachlin, R., 1997. Management initiatives and just-in-time manufacturing. J. Oper. Manag. 15, 271–292.

Dr. Leire Labaka, Industrial Engineer (2009, PhD 2013), is a professor of Operational Re- search, Modelling and Simulation and Accounting and Finance at TECNUN. Her research interests include complex system modelling, critical infrastructures protection and resil- ience. She has taken part in SEMPOC and ELITE European projects. She has published sev- eral papers in conference proceedings as well as in journals such as Journal of Homeland Security and Emergency Management, Reliability Engineering and System Safety, Journal of Technological Forecast and Social Change, and International Journal of Critical Infra- structures.

Dr. Josune Hernantes, Computer Science Engineer (2003, PhD 2008) is a professor of Computer Science at TECNUN. Her research interests include information systems, com- plex systems modelling and crisis management. She has taken part in research projects about crisis management such as SEMPOC and ELITE European projects. She has published several papers in conference proceedings as well as in journals such as Journal of Home- land Security and Emergency Management, Journal of Technological Forecast and Social Change, International Journal of Critical Infrastructures and IEEE Software.

Prof. Dr. Jose Mari Sarriegi, Industrial Engineer (1994, PhD 1999) is a professor of Infor- mation Systems, Knowledge Management and Modelling and Simulation at TECNUN. His research interests include security management, knowledge management and complex systems modelling. He has coordinated the SEMPOC European project and ELITE European project. He has published in journals such as IEEE Software, International Journal of Computer Integrated Manufacturing, IEEE Internet Computing and International Journal of Industrial Ergonomics as well as in conference proceedings such as in the Lecture Notes in Computer Science.