Review on Energy Resilience

Reliability Engineering and System Safety 150 (2016) 11–21

Contents lists available at ScienceDirect

Reliability Engineering and System Safety

http://d 0951-83

n Corr E-m

lambert kashbar

1 Te 2 Te 3 Te

journal homepage: www.elsevier.com/locate/ress

Resilience analytics with disruption of preferences and lifecycle cost analysis for energy microgrids

Michelle C. Hamilton a,1, James H. Lambert a,n, Elizabeth B. Connelly a,2, Kash Barker b,3

a Department of Systems & Information Engineering, University of Virginia, 151 Engineer’s Way, P.O. Box 400747, Charlottesville, VA 22904-4747, USA b School of Industrial & Systems Engineering, University of Oklahoma, 202 W. Boyd St., Room 124, Norman, OK 73019-1022, USA

a r t i c l e i n f o

Article history: Received 3 January 2014 Received in revised form 4 January 2016 Accepted 5 January 2016 Available online 13 January 2016

Keywords: Infrastructure resilience Scenario analysis Robust decisions Energy systems Strategic planning Systems engineering

x.doi.org/10.1016/j.ress.2016.01.005 20/& 2016 Elsevier Ltd. All rights reserved.

esponding author. Tel.: þ1 434 531 4529; fax ail addresses: [email protected] (M.C. Ham @virginia.edu (J.H. Lambert), [email protected] [email protected] (K. Barker). l.: þ1 717 860 6092; fax: þ1 434 924 0865. l.: þ1 864 905 6962. l.: þ1 405 325 3721.

a b s t r a c t

Innovative technologies are presenting opportunities to improve resilience of energy plans for industrial and military installations. The investment rationale is complicated by uncertain future conditions across the system lifecycle, including technology, climate, economy, and others. This paper introduces resilience analytics with scenario-based preferences as follows. Risk is addressed here as the degree of disruption of priorities for investments in engineering systems. The particular concern of this paper is disruption from shifts in public values, and to evaluate the resilience of investment plans to such shifts. It recognizes resilience models as compilations of instantaneous framings of initiatives, objectives, stakeholder pre- ferences, and uncertainties. Problem frames can be considered in series, where inputs to frames are the outputs of previous frames. Or frames can be considered in parallel, featuring joint inputs while addressing differing questions. This paper presents a case study of resilience analytics focusing on two quantitative frames. In the first frame, scenario-based preferences are used to identify combinations of factors disruptive to energy innovation at installations. In the second frame, estimation of lifecycle costs is performed with respect to factors that were identified as influential in the previous frame.

& 2016 Elsevier Ltd. All rights reserved.

1. Introduction

Resilience of energy systems of military and industrial instal- lations for a range of emergent and future conditions is important to national security and homeland defense, as well as economic development. Reliance on a publicly owned and aging grid infra- structure is a key challenge of the coming decade, and various agencies are encouraging the development of technologies that can help to assure that energy supplies meet critical demands at all times. Currently, fixed installations are 99% dependent on the commercial power grid to meet their electricity needs. The Defense Science Board [7] noted, “Critical national security and Homeland defense missions are at an unacceptably high risk of extended outage from failure of the grid,” thus relating the resi- lience of missions to the resilience of underlying infrastructure. In this work, we emphasize energy resilience, or the assured access

: þ1 434 924 0865. ilton), du (E.B. Connelly),

to reliable supplies of energy and the ability to protect and deliver sufficient energy to meet operational needs.

Resilience is generally defined as an ability to withstand, adapt to, and recover from a disruption. Resilience of a system or orga- nization is generally measured as some function of the adverse change in performance it experiences after a disruption (its vul- nerability) and its subsequent return to a desired performance level (its recoverability) [15,28]. Francis and Bekera [10] discuss the diverse definitions and usage of the term resilience and con- clude that resilience analytics must include a focus on deep uncertainties and emergent conditions that bring about shifts in stakeholder preferences, methods for which are described in [39,17,13,23]. We consider resilience analytics as the descriptive, predictive, and prescriptive analytics to understand, design, and manage system performance across disruptive events to enhance their resilience.

When designing energy microgrids, stakeholders must prioritize the many potential design alternatives such as including natural gas microturbines, solar photovoltaics, biofuels, batteries and many others while considering multiple objectives such as reducing costs, reducing environmental impacts, and others. In this paper, we introduce resi- lience analytics of engineering systems to describe the disruption of prioritized alternatives by scenarios of emergent and future condi- tions. First, the focus of this paper is the risk defined as the degree of disruption of priorities for strategic planning. The disruption is caused

M.C. Hamilton et al. / Reliability Engineering and System Safety 150 (2016) 11–2112

by shifts in the preferences of stakeholders due to uncertain emergent conditions, which can be at least equal a challenge for engineering systems as physical, software, and other hazards [25,38,46]. In other words, risk is the influence of scenarios to priorities, referring to the ordering of initiatives or investments. A scenario is comprised of one or more uncertain emergent conditions. A condition refers to external or internal sources of risk to systems that may either adversely or favorably affect the performance of the system against a set of obje- ctives. While it would be ideal to investigate the consequences of all of these conditions, limited time and budget necessitates the need for addressing conditions with the highest influence to prioritization. In this way, risk arises from shifts in weights of objectives, which in turn disrupts the prioritization of initiatives. For example, when con- sidering energy investments consideration is given to multiple objectives such as reducing costs, increasing power reliability, and reducing environmental impacts, among others. The emergent con- ditions of stricter federal energy efficiency, conservation, and renewable energy requirements can be considered a renewable poli- cies scenario under which stakeholders more strongly prefer the objective of reducing environmental impacts as compared to other objectives such as reducing costs. Resilience analytics thus identifies the scenarios that are potentially most disruptive to priorities. We represent the disruptiveness of scenarios through a simple metric of the shifts in rank across a set of initiatives. Thus, this paper does not deal with uncertainty in the usual way. Here, we chose to filter risk not by probability and severity but by the sensitivity of priorities to scenarios.

Second, a particular focus is considering multiple problem frames [2,36]. The paper describes how several published case studies are used to frame the same problem, and develops two additional frames (Frame I and Frame II, below) for illustration. The overlapping of frames is demonstrated to contribute to the toolkit of reliability engineering, in particular for the identification and refinement of system requirements supported by resilience analytics.

The overall direction of the paper is thus resilience analytics with multiple problem frames as a support for decision-making for reliability and safety of engineering systems.

2. Background

Historically, energy investment decisions were based on least-cost alternatives. However, the military now recognizes that it must incorporate other objectives or criteria into the priority-setting pro- cess. There have been several policy directives and Federal mandates in recent years that have moved Federal agencies towards energy security and environmental sustainability. These include the Energy Policy Act of 2005 [8], the Executive Order (EO) 13423 [9], the Energy Independence and Security Act of 2007 (EISA 2007), and most recently the Executive Order (EO) 13693. Some of the requirements in these policy directives include reducing facility energy intensity 30% by 2015 compared to 2003 baseline [EO 13423], reducing water consumption intensity relative to the 2007 baseline by 2% annually through the end of fiscal year 2015 [EO 13423] [9], and requiring renewable electricity consumption by the Federal government to not be less than 7.5% after 2013 [8]. To address both energy security and sustainability concerns, the 2009 Army Energy Security Imple- mentation Strategy (AESIS) identifies the following five strategic objectives: (i) Reduce energy consumption, (ii) Increase energy effi- ciency across platforms and facilities, (iii) Increase use of renewable/ alternative energy, (iv) Assure access to sufficient energy supplies, (v) Reduce adverse impacts on the environment [44]. Renewable and alternative energy sources such as biomass, landfill gas, municipal solid waste, hydrogen, hydropower, geo-thermal/pressure, micro- turbines, fuel cells, wind, tidal and solar are some examples of the

many initiatives that military and industrial installations use to meet their energy security and sustainability objectives.

Even as energy planners for military installations address a plethora of resource options and multiple objectives, they face a dynamic, complex, and uncertain future. Emergent and future conditions related to technologies, political and regulatory chan- ges, resource demand and supply shifts, and climatic changes, might significantly impact the resilience of energy initiatives. Of particular concern to energy planners is how current investment decisions will be impacted by future uncertainties. Changing fed- eral, military, and business requirements will influence high-level energy security objectives, and thus the type of technologies that the installations will pursue. For example, an increase in the number and enforcement of renewable energy requirements at Federal buildings will direct attention more towards investigating renewable technologies that are feasible in the region. Changing state regulations such as renewable portfolio standards, carbon taxes, and utility tariff regulations influences the feasibility of various energy systems. Supply and demand shifts of oil and natural gas might affect availability and price of these resources. In general, emergent and future conditions can constitute deep uncertainties [17]. Deep uncertainty includes when parties to a decision do not know or cannot agree on the system model and event probabilities [24].

The above characteristics add to contentiousness of priorities across stakeholders. Recent literature has advocated the use of scenario planning and multicriteria analysis to seek resilient initiatives across scenarios of epistemic or deep uncertainty when probabilistic analysis is not practical [12,25,26,29,37]. Schroeder and Lambert [35] refine this concept with a focus on risk identi- fication; identifying combinations of factors that are the most influential to prioritization. Their work builds on the 2009 ISO definition of risk as the impact of uncertainties on objectives [16]. Specifically, previous work has considered risk as how uncertain future conditions change the relative preference of objectives and consequently the prioritization of initiatives.

Catrinu and Nordgard [5] apply multicriteria decision analysis for risk analysis of electric system management subject to aleatory and epistemic uncertainties. Karvetski and Lambert [17] perform a scenario-based multicriteria analysis for selecting energy systems at a particular military base. Hamilton et al. [13] illustrate a scenario- based multicriteria analysis for evaluating research and development priorities at military installations. Tylock et al. [40] evaluate energy technology alternatives for a representative military base using a stochastic multi-attribute analytic approach to explore different priorities or weighting schemes in combination with uncertainties related to technology performance. So far, these analyses have been static views of a priority-setting problem. While they provide a systematic methodological process for identifying scenarios and resilient initiatives for a particular problem frame in time, most priority-setting situations do not start with a well-defined set of candidate initiatives, criteria, and emergent conditions. Thus existing methods need to be extended with iterations and continuity of analyses that reflect insight gained from prior iterations.

Emergent conditions for infrastructure plans can lead to changes in stakeholder preferences in several modeling frames. Examples include changes in preferences realized after major events including 9–11 (improve homeland security), Katrina (improve conditions of underrepresented groups), 2008 recession (improve fiscal accountability), Sandy (improve human migrations with respect to climate change), among others. Brito and de Almeida [3] recommend incorporating decision maker preferences into risk assessment for prioritizing energy investments.

M.C. Hamilton et al. / Reliability Engineering and System Safety 150 (2016) 11–21 13

3. Methods

3.1. Transitional models for resilience analytics

The following sections introduce resilience analytics at multiple overlapping, interrelated problem frames as illustrated in Fig. 1. Each frame provides information that can be used to (i) refine the set of initiative for some subsequent analysis, (ii) refine criteria, (iii) identify a set of scenarios for further study, and (iv) recommend and inform other analytical process such as lifecycle cost analysis.

There has been considerable interest over the past two decades in prioritization for long-term investments when probabilistic analysis is lacking information and consensus. Scenario analysis has been applied to many complex decision environments including to energy systems [13,17,25]. In long-term investment settings, the descriptions of possible futures are seldom collec- tively exhaustive [17]. Scenario planning addresses uncertainty without forecasting or probabilities and enables the observation of joint impacts of various uncertainties, simultaneous changes in various variables, and uses subjective interpretations beyond the reach of objective analysis [34]. Although it is not possible to fully characterize the potential futures, scenario planning provides aid to reduce the uncertainties to a smaller number of states that most matter to decision making. Ultimately, the decision maker is able to define strategies that are resilient over a range of different possible outcomes [1,12,17–19,22,26,29].

Scenarios in this paper are comprised of one or more emergent and future conditions. A condition refers to external or internal sources of risk to systems that may either adversely or favorably affect the

Fig. 1. Transitional models for identifying risk and uncertainty factor

Table 1 Frame I – Description of energy investment initiatives that will be considered in priorit

Investment initiative Description

a1 No-action Maintain the current states quo gri heat.

a2 Bury lines Bury power lines supplied to the fa a3 Critical load microturbines with cogeneration Microturbines distributed generatio a4 Base load microturbines with cogeneration Microturbines distributed generatio

enables removal of steam plant. a5 Microgrid with existing generators Create a microgrid for using existin a6 Photovoltaic Solar Install solar PV on rooftops

performance of the system as a whole. While it would be ideal to investigate the consequences of all of these conditions, limited time and budget necessitates the need for addressing conditions with the highest disruption to prioritization. Recent literature has suggested a multi-criteria framework as a quantitative approach for performing scenario analysis [17,18] [19,12–14,26,32]. However, these models are best viewed as isolated, single frames of an inevitably more complex modeling challenge.

Systems analysis literature has long recognized that models are transient instances of the current knowledge of possible initiatives, criteria, measured outcomes, stakeholder preferences, information inputs, and uncertainties. Systems analysis is broadly defined as holistic, top-down thinking to include problem formulation and interpretation of proposed initiatives in terms of the value systems of the stakeholders [11]. Systems analysis requires both taking apart of constituent elements of the problem and “synthesis” or com- bining elements into functional new whole in order to recommend solutions [11]. Thus, in order to prioritize initiatives when faced with uncertainty, it is necessary to both identify uncertain factors for the a priori set of initiatives and creatively design new initiatives that mitigate negative risks or take advantage of opportunistic risk. Each decision model instance represents a particular problem frame. These problem frames can be in series, where model inputs are updated due to learning that occurs from interacting with the model. When problem frames are in series, one frame identifies opportunities to revise and reconstruct scenarios and/or initiatives for subsequent problem frames. They can also be in parallel as auxiliary problems where models of different system levels interact with and inform other levels. To illustrate, this paper explores

s in the context of energy investments for military installations.

ization.

d supplied energy for power and cooling needs, and natural gas steam plant for

cility n of prime power including cogeneration of electricity and thermal energy. n of prime power including cogeneration of electricity and thermal energy. This

g generators as backup power.

M.C. Hamilton et al. / Reliability Engineering and System Safety 150 (2016) 11–2114

models impacting lifecycle costs. Montibeller and Franco [27] dis- cuss how decision models can be viewed as transitional objects to share concerns, increase individual understandings of issues, appreciate the potential impact of different choices, and negotiate action that is politically feasible. Participants’ interaction with the model reshapes the analysis, and the model analysis reshapes the group discussion [27]. Phillips [30] suggests that that these models are requisite in that they are at best conditionally prescriptive, indicating what could be done considering only the given frame, assumptions, data and judgments. Similarly, Wright and Goodwin [45] suggest that values are not sufficiently well formed in early stages of engineering design and that one needs some interaction with alternatives to understand values, and values and criteria are formed out of experience with alternatives. The consideration of multiple problem frames, especially those in series, thus contributes to resilience analysis by exploring how iterative updating of the model impacts the strategic prioritization of initiatives.

Corner et al. [6] suggest a dynamic approach to problem structuring and that thinking about values helps generate alter- natives and thinking about alternatives helps generate values. This paper asserts that the same philosophy applies to understanding the disruption of scenarios which are combinations of uncertain emergent and future conditions. Some interaction with perform- ing scenario analysis is needed to understand changeable pre- ferences across criteria. This paper adds to the approach of Corner et al. [6] that thinking about criteria helps generate scenarios, with thinking about scenarios helps generate criteria and thinking about scenarios helps generate initiatives. With each execution (or frame) of the priority-setting model, some initiatives are eliminated, new ones are created, some criteria become more or less relevant, the decision maker’s preferences change based on insight gathered in previous iterations, and the information inputs change. Decision elements mutually interact with each other through time both in series and in parallel. Parallel problem frames are considerations from different systems-of-systems level perspectives such as strategic, operational, research and development, lifecycle cost,

Table 2 Frame I – Criteria and criteria weighting for energy security at the facility of the case study (criteria weights were assessed by Karvetski and Lambert [17]).

Criteria Baseline weight

z1 Reduce costs 0.27 z2 Quality, prime power 0.12 z3 Achieve AESIS goals 0.12 z4 Provide proof-of-principle, lessons learned 0.27 z6 Procure funding 0.12 z7 Reduce physical/non-physical vulnerability 0.05 z8 Reduce environmental impact 0.05

Table 3 Frame I – Normalized criterion value scores vk aið Þ of the six investment initiatives for e

Criteria Investment Initiatives

No action

Bury power line

Microturbines with trigenerat critical load

z1 Reduce costs 0.33 0.67 z2 Quality prime power 0 0.33 0.67 z3 Achieve AESIS goals 0 0.33 1 z4 Provide proof-of-principle, les- sons learned

0 0 0.67

z5 Procure funding 0 0.67 0.67 z6 Reduce physical/non-physical vulnerability

0 0.33 0.67

z7 Reduce environmental impact 0 0.33 0.67

etc. The following two sections demonstrate resilience analytics for energy systems at military installations under multiple pro- blem frames.

3.2. Demonstration of Frame I: Resilience analytics with scenario- based preferences

The first frame applies resilience analytics to a prioritization of initiatives for a major military installation, hereafter referred to as the “facility”, that is considering various energy system modifica- tions to improve energy security. This facility consists of forty-six buildings requiring approximately four megawatts (MW) of power. Within the facility, there are several scientific laboratories that are often running multi-week experiments on high technology equipment. Currently, the facility receives its electricity from the public grid. It also uses electricity for its cooling needs via electric coolers. The heating needs are met mainly through a natural gas steam plant, which is aged and requires frequent maintenance each year. The facility experiences many outages a year, often due to weather, trees falling, and animals. Recently, the facility experienced over forty hours of outages in a single year. This poses a problem to the entire facility and particularly for the research and scientific laboratories. Whenever the power is lost or inter- mittent, these laboratories can lose days or weeks of work and data collection, and the experiment often has to be reinitiated. Also, the humidity, temperature, and air quality of the laboratories must be kept strictly regulated, and any power outage can cause contamination of materials and equipment. Currently, the utility supplied power is not of consistent quality, and these variations in voltages cause a problem for some of the electronic equipment. Thus, the installation is investigating diverse initiatives to address the operational need for consistent, reliable, and quality electricity.

The approach of this frame involves several elements: A set of initiatives, a set of evaluation criteria, and a set of scenarios comprised of emergent and future conditions that ought to influence priority-setting for energy security investments. Sce- narios are integrated into the evaluation of the initiatives by eli- citing what performance criteria become more or less relevant under each scenario. The approach identifies a prioritized subset of high performing initiatives and a set of scenarios (reflecting deep uncertainties) that are in need of additional modeling and other investigative resources. A set A ¼ a1; …; anf g of n investment initiatives is selected to be prioritized. To evaluate and compare of each initiative, a set Z ¼ Z1; …; Zmf g of m criteria is used. In the case study, these criteria describe several energy security and opera- tional goals such as providing high quality power or reducing emissions. A multiple criteria value function is constructed for prioritization of the initiatives. A linear additive value function is

ach of the seven criteria.

ion for Microturbines with trigeneration for base load

Microgrid of backup generators

Solar PV

1 0 0 1 0.67 0 1 0.33 0.33 1 1 0.33

0.33 1 0.33 0.67 1

1 0 0.33

M.C. Hamilton et al. / Reliability Engineering and System Safety 150 (2016) 11–21 15

used in this demonstration [1,20]. The additive multi-criteria value function V(ai) has the form

V aið Þ ¼ 100 � Pm

k ¼ 1 wkvkðaiÞ with Pm

k ¼ 1 wk ¼ 1 and 0rvk aið Þ r1 for all k ¼ 1; …; m.

The quantity vk aið Þ is the assessed value of initiative ai with respect to criterion Zk. The weights wk

� � represent the tradeoffs

across the criteria. In general, V aið ÞZV aj � �

implies initiative ai is at least as preferred as initiative aj. Table 1 provides a set of energy investments for the initiative set A. The initiatives range from investing in solar photovoltaic systems to natural gas powered microturbines with electricity, heating, and cooling abilities (tri- generation) to an electricity microgrid of existing diesel backup generators. The initiatives evaluated in [17] were updated in this model. For example, the concept of a microgrid was combined with backup generators. The solar photovoltaic initiative replaced another initiative that was lowly prioritized in the previous iteration. Table 2 provides six performance criteria for the criteria set Z, derived from both the guiding Federal policy directives, as well as discussions of the facility’s working group that consists of multiple diverse stakeholders. The criteria and associated weights are the same as used in [17], which were derived from stakeholder input. Table 3 provides an assessment of how well each alterative satisfies each criterion. These represent the criterion value scores, vkðaiÞ, assigned to each initiative-criteria combination.

Next, a set of scenarios defined S ¼ s1; …; sp � �

is constructed and used to reflect future uncertainty and the viewpoints of the multiple stakeholders serving as decision makers. These scenarios are used to analyze the resilience of priorities across uncertain preference schemes. These scenarios are different than those used in [17]. The previous analysis showed little change in the prior- itization of initiatives across scenarios. However, new scenarios were envisioned based on the relationship with an R&D problem frame as described in [13]. Thus, a new set of five scenarios is described for this paper as below. Each scenario can increase the preference levels of some criteria among the others while possibly decreasing preferences levels of other criteria. The baseline sce- nario, where there are no dominating emergent conditions, is indexed as s0. The additional scenarios are indexed as follows:

Table 4 Frame I – The adjusted importance of each criterion is updated for each of the five scen

Criteria Scenarios

S1. Islanding S2. Renewable poli

z1 Reduce costs Minor decrease z2 Quality prime power Major increase z3 Achieve AESIS goals Major increase z4 Provide proof-of-principle, lessons learned z5 Procure funding z6 Reduce physical/non-physical vulnerability Major increase z7 Reduce environmental impact Major increase

Table 5 Frame I – Corresponding normalized adjusted weight given in Table 4.

Criteria Scenarios

S1. Islanding S2. Renewable polic

z1. Reduce costs 0.041 0.114 z2. Quality prime power 0.495 0.051 z3. Achieve AESIS goals 0.055 0.458 z4. Provide proof-of-principle, lessons learned 0.124 0.114 z5. Procure funding 0.055 0.051 z6. Reduce physical/non-physical vulnerability 0.206 0.021 z7. Reduce environmental impact 0.023 0.191

� s1_Islanding-Deterioration and vulnerability of commercial power grid infrastructure, increase in storm frequency and duration, increase in brownouts and longer duration blackouts.

� s2_Renewable Policies-Stricter federal energy efficiency, conser- vation, and renewable energy requirements.

� s3_Cyber and Terrorism-Increase in cyber threats, increase in terrorism threats.

� s4_Economic-Slow economic development, cutbacks in govern- ment spending.

� s5_Technology innovation-Lack of private investment in R&D, increased pressure for the government to provide proof-of- principle for emerging technology.

The stakeholders reassess the baseline criteria preferences or weights under each scenario and a new score is generated for each initiative under each scenario. The ratios of the weights {wk} are adjusted relative to each other (normalizing or dividing each by a constant still maintains the same preference information) [17]. Table 4 displays how the baseline criteria coefficients are adjusted. These adjustments answer the following question: “Compared to a baseline scenario, which of the criteria becomes more important or less important under scenario sk?” For example, the first sce- nario s1 describes that deterioration and vulnerability of the electric power grid to long term outages is becoming more of a concern, thus the islanding ability and reduction of physical and nonphysical vulnerabilities is more of a priority and reducing costs is less of a priority given the baseline criteria coefficients. This method of processing scenarios focuses stakeholder discussion on the criteria and not on the plausibility of scenarios. Table 5 dis- plays the associated normalized adjustments in the weight of each criterion given the responses in Table 4.

Fig. 2 describes results of the analysis. Each energy security initiative is listed along the top axis and the priority score of the initiative is represented on the left axis (100 being the highest performing). The diamond represents the baseline scenario and the range bar represents the highest and lowest scores of each initiative across all considered scenarios. For example, the initia- tive X05, microgrid of backup generators has a high value score

arios of deep uncertainty, representing scenario-based preference changes.

cies S3. Cyber and terrorism S4. Economic S5. Technology innovation

Minor decrease Major increase

Minor decrease Major increase

Major increase Minor decrease

ies S3. Cyber and terrorism S4. Economic S5. Technology innovation

0.074 0.80 0.0854 0.098 0.039 0.0380 0.098 0.013 0.0380 0.221 0.089 0.7690 0.098 0.039 0.0380 0.369 0.016 0.0158 0.041 0.005 0.0158

M.C. Hamilton et al. / Reliability Engineering and System Safety 150 (2016) 11–2116

under the baseline scenario; however, the prioritization is highly disrupted by several of the scenarios, as shown by the large range above and below the baseline scenario diamond. On the other hand, initiatives X04, microturbines with trigeneration for base load is relatively resilient, i.e., has a high relative value score under all scenarios. Tables 6 and 7 show the value scores and prioritizations, respectively, of the each initiative under each scenario. Notice that initiative X04, microturbines with trigeneration for base load is the highest performing initiative under all scenarios. However, the value score drops significantly under the cyber and terrorism scenario and is very close to the value score for initiative X05, microgrid of backup diesel generators. Under this perspective, the reliance on the natural gas utility and/or storage of large amounts of natural gas provides additional points of vulnerability compared to the microgrid of diesel generators.

Fig. 2. Frame I – Resilience of priority scores for several energy security investment initiatives under scenarios of deep uncertainty. The diamond represents the baseline score.

Table 6 Frame I – Value score V aið Þ of initiatives across scenarios of deep uncertainty.

Scenarios Investment Initiatives

No action Bury power line

Microturbines with trigenerati critical load

Baseline 3 23 70 S1. Islanding 0 31 68 S2. Renewable Policies 2 28 81 S3. Cyber and Terrorism 1 27 70 S4. Economic 18 11 67 S5. Technology Innovation

1 7 68

Table 7 Frame I – Prioritization of investment initiatives across scenarios.

Scenarios Initiatives

No action Bury power line

Microturbines with trigenerati critical load

Baseline 6th 4th 2nd S1. Islanding 6th 4th 3rd S2. Renewable Policies 6th 4th 2nd S3. Cyber and Terrorism 6th 4th 3rd S4. Economic 4th 5th 2nd S5. Technology Innovation

6th 5th 3rd

A scenario is considered disruptive if it changes the prior- itization of initiatives. Table 8 shows the sum of the squares (of the differences in priority order) representing how much each sce- nario disrupts the overall prioritization of initiatives relative to the baseline scenario. Notice that s4 economic is the most diruptive scenario. Under this scenario, the alterative, X01, do nothing increases in prioritization by two spots. The next most influential scenario is s5 technology innovation, where the initiatives micro- grids of backup generators and solar PV increase in prioritization.

Table 9 summarizes the key results for this frame. The results of this analysis can be used to creatively design initiatives that are more resilient across scenarios. For example, a new initiative that could be considered is a combination of two of the initiatives – a microgrid of backup generators combined with microturbines for base load trigeneration. This would remove a layer of vulnerability so that if there is a disruption in the natural gas supply, the backup generators could still provide power to critical loads. There will be less of a need to store natural gas onsite, which is a potential source of vulnerability to terrorist attack. Importantly, this com- bined initiative would satisfy the need to demonstrate an inno- vative use of technology. These results inform the next iteration and additional frame (below).

3.3. Demonstration of Frame II: Resilience analytics with lifecycle cost analysis

Each of the scenarios of Frame I above describes combinations of factors that affect the lifecycle cost analysis of the initiatives. There are numerous uncertain parameters that contribute to estimation of the lifecycle costs. These parameters include: installation, operation, and maintenance costs, the price of fuel and electricity, the energy profile of the buildings such as thermal and electrical loads, the energy system availability and reliability, the cost of lost mission, and any possible environmental revenue streams such as renewable energy credits (RECs), emissions offsets, and tax incentives. Thus it is important to perform a sensitivity analysis of lifecycle costs in parallel to explore whether the variety of cost uncertainties implied by the

on for Microturbines with trigeneration for base load

Microgrid of backup generators

Solar PV

92 69 18 95 71 5 96 46 26 86 80 15 95 38 9 98 91 29

on for Microturbines with trigeneration for base load

Microgrid of backup generators

Solar PV

1st 3rd 5th 1st 2nd 5th 1st 3rd 5th 1st 2nd 5th 1st 3rd 6th 1st 2nd 4th

M.C. Hamilton et al. / Reliability Engineering and System Safety 150 (2016) 11–21 17

scenarios are influential, or alternatively, whether priorities are resi- lient to cost uncertainties.

The facility requires electrical energy for lighting, office equip- ment, ventilation, and other laboratory equipment; and thermal energy for space and water heating and cooling. Under the current system, electrical power is supplied by a utility and the electricity is distributed through the power grid. Also, the space cooling is gen- erated from electricity through electric chillers. The thermal energy is supplied through mostly through a centrally located, on-site natural gas steam plant and several smaller natural gas boilers and oil fur- naces. Quantitative data were collected from electricity and natural gas utility providers in order to characterize the electrical and ther- mal loads of the facility. The electrical load information was based on the yearly total consumption, as monthly data were not available. Monthly natural gas usage was collected. In Frame II, the following three initiatives have been adapted from Frame I for detailed analysis of lifecycle costs:

� Critical load microturbines: 1.8 MW of microturbine installed capacity. This would be enough to cover the electrical and heating load of the critical scientific buildings of the facility. In this initia- tive, the natural gas steam plant would still be used for the majority of the facility and electricity from the utility would still be used to provide the majority of the electrical and cooling needs.

� Base load microturbines: 4 MW of microturbines installed capacity to cover both the thermal and electrical load of the all the buildings in the facility. Under this initiative, the old natural gas steam plant can be decommissioned.

� Solar PV: 0.6 MW of solar photovoltaic plans on building rooftops.

Table 8 Frame I – Degree of disruptiveness of scenarios of deep uncertainty (sum of squared changes of priority order in comparison to baseline prioritization) on prioritization of energy security investment initiatives under consideration at the facility.

Scenario Sum of squared changes in priority order

S1. Islanding 2 S2. Renewable policies 0 S3. Cyber and terrorism 2 S4. Economic 6 S5. Technology innovation 4

Table 9 Frame I – Summary of key results of the resilience analysis for the case study.

Type of result Description

Resilient initiative The initiative a4 microturbines w the baseline scenario and rema

Nearly resilient initiatives The initiatives a3 microturbines robust initiatives. They are in th scenarios. They alternate for the

Influential scenarios of deep uncertainties The Economic scenario provides combination of conditions repre comparison to the baseline ran bury power lines and a6 solar PV scenario as it raises in priority

Initiatives that fall in priority relative to the baseline scenario

The initiative a3 microturbines w decreases to third priority unde The initiative a2 bury power lines scenarios The initiative a6 solar PV is rank scenario

Initiatives that rise in priority relative to the baseline scenario

a5 microgrid of backup generator three scenarios a1 No action is ranked sixth in t

The total annual value or savings of the distributed generation initiatives is calculated as the cost of producing the equivalent amount of energy that these systems produce using the current energy system. It is assumed that microturbines will be run at full capacity for ninety-eight percent of the year and that solar PV has a fourteen percent capacity factor for the region. For the dis- tributed generation systems it is assumed that the energy not supplied by the distributed generation system will come from traditional sources. Any electricity demand that is not generated by the distributed generation technology will be purchased from the utility. Similarly, if there is excess electricity produced by the distributed generation, it is assumed that it can be sold back to the utility. If there is excess thermal energy produced by the dis- tributed generation, it is assumed that this has no value. Also, under the partial investment alterative, it is assumed that the current steam plant for heat and utility electricity for cooling will supply any thermal demand not met by the microturbines.

Estimates for the relevant cost parameters were collected from a variety of sources including vendor technical specifications, utility historical data, industry standards, and weather-based assumptions. Table 10 describes the facility annual loads and costs under the current energy system. Tables 11 and 12 describe the estimates of the cost parameters that were aggregated in the baseline lifecycle cost estimate. The electrical output, chilled water output and hot water output listed in this table produced from vendor specification sheets were used to estimate the total yearly output of the microturbines under each initiative, for 98% of the year. The fuel input rate was used to estimate the total amount of fuel required to run the microturbines for 98% (358 days) of a year. Annual maintenance savings were also considered and added to the annual energy cost savings. Finally, the net present value of the installation costs and annual savings were calculated over a ten- year period. A ten-year net present value is used because the facility stakeholders expressed interest in projects that had a ten- year payback period or less. Table 13 provides more details on the equations used to derive the total annual energy savings. A three percent discount factor was used based on recommendations from the Federal Energy Management Plan’s manual entitled “Energy Price Indices and Discount Factors for Lifecycle Cost Analysis” [33].

Results for the baseline lifecycle cost assessment are as follows. The base load microturbines initiative provides the greatest savings of $7.8 million over a ten-year horizon. This is followed by the critical load microturbine initiative worth $2.9 million. The solar PV

ith trigeneration for base load is the most robust initiative. It is highly prioritized in ins the a top priority for all scenarios with trigeneration for critical load and a5 microgrid of backup generator are nearly e top three of priority in the baseline scenario and remain in the top three for all second priority position in the various scenarios. the most changes to prioritization of initiatives. This is based on scores for each senting the sum of square ranking (SSRC) changes of the ranking values in king. Under this scenario, a1 no action increases two positions in priority, and a2 initiatives drop in priority. Technology innovation is the second most influential

a5 microgrid of backup generator and a6 solar PV ith trigeneration for critical load is ranked second in the baseline scenario but r three scenarios is ranked fourth in the baseline scenario but decreases to fifth priority under two

ed fifth in the baseline scenario but decreses in priority to sixth in the economic

is ranked third in in the baseline scenario but increases to second priority under

he baseline scenario but increases to fourth priority under the economic scenario

M.C. Hamilton et al. / Reliability Engineering and System Safety 150 (2016) 11–2118

investment initiative produces a net loss of $1.5 million. The base load investment in microturbines is the preferred initiative in terms of lifecycle costs. However, as mentioned previously, many of the cost parameters are subject to scenarios consisting of one or more emergent conditions. The scenarios originate from the various stakeholder perspectives as described in the earlier sections. The following effects of four main scenarios on the lifecycle cost ana- lysis of initiatives for energy security were explored:

Natural gas prices. The sensitivity of the lifecycle cost assessment to the price of natural gas was explored due to concerns of economic slowdown as articulated in s4_Economic in Frame I. As the cost savings greatly depends on this parameter, it is important to understand how sensitive the cost savings are to fluctuations in the price of natural gas. In the baseline scenario, the price of natural gas is assumed to be $8/MMBTU. The price of natural gas is varied from $8/MMBTU to $25/ MMBTU. Fig. 3 shows how the present value changes with the price of natural gas. Notice that if the price increases to $13/MMBTU, from its current value of $8/MMBTU, the net present value of the microturbine initiatives become negative. The full investment in microturbine initiative is more sensitive to the price of natural gas, than the partial investment. Because of this extreme sensitivity, before making a decision, the stakeholders should be confident that the price would not go above $13/MMBTU. As recently as 2008, commercial prices of natural gas reached above $15/MMBTU [42]. While the recent developments in hydraulic fracturing has lead to an increased supply and decreased price of natural gas, this industry may come under increased pressure for additional regulation in the future [31].

Storms. Due to concerns from the scenario s1_Islanding in Frame I that that there will be an increase in frequency and duration of

Table 10 Frame II – Current energy system annual loads and costs for the facility.

Cost source Energy consumption

Cost/unit Annual cost

Oil (gallons) 117,291 $2.25/gallon $263,905 Natural gas steam plant (MMBTU)

124,918 $8/MMBTU $999,344

Natural gas other (MMBTU)

30,851 $8/MMBTU $246,808

Electricity (kwh) 35, 172, 980 $0.0889/kwh $3,126,878 Total

Table 11 Frame II – Energy initiative cost parameter values for baseline lifecycle cost savings est

Symbol Parameters Current system Critical load micr

A Capacity (kW) 1800 B Annual capacity factor % 98% [4]

C Electricity price 0.0889 (historical data)

0.0889 (historical

D Fuel rate ($/MMBTU) 8 (historical data) 8 (historical data) E Fuel input (MMBTU/hr) 0.0114 [4] F Hot water output (MMBTU/hr) 0.004715 [4] G fuel input/steam output

(MMBTU/klb steam) 1.6 (Utility supplied historical data)

1.2, (Industry stan

H Efficiency conversion factor for microturbine verses steam plant

1.33 (To get the in ciency of microtur 1.6 by 1.2)

I Chilled water output (tons/hour/ kw installed capacity)

0.3 [4]

J kwh/ton of chilled water 1 (Based on age of electric chillers)

N/A

K Capital Costs ($/kW) N/A 300 (estimate) L O*M ($/yr) 500 (Utility supplied

historical data) 500 (Utility suppl data)

M REC ($/kwh) 0 0

electricity outages due to storms, the value of lost mission savings was explored. This is the value that can be attributed to reducing the amount of electricity outages that occur in the facility area as a result of implementing the various initiatives. This scenario explores the lifecycle cost savings from the perspective of the installation owner. While, historically, installation decisions on energy system expenditures have been made based on “first costs”, there is an increasing emphasis from the Department of Defense to create new business models that include the value of mission requirements as a parameter in the decision process [21]. It can be challenging to quantify the cost of lost mission when performing lifecycle cost analysis. When power goes out in the facility, there is lost labor productivity, science experiments are interrupted, and occasionally, electronic equipment is ruined due to power surges. Data to quantify many of these consequences are not readily available; therefore, this analysis uses the labor cost of downtime as a proxy for the cost of lost mission. This analysis assumes that for each hour of downtime, one hour of the labor force productivity is lost. The science lab portion of the facility is frequently running science experiments that last for up two weeks or more. If the power is interrupted, the entire two weeks of efforts can be lost and equipment can be damaged. This analysis does not incorporate those costs, but they may be substantially higher than the cost of wages for the time electricity was dis- rupted. The net present value of the initiatives under this scenario are $24.1 million, $56.1 million, and �$1.5 million for critical load microturbine, base load microturbine, and solar PV respectively. The top rightmost portion of Fig. 3b, displays the ten-year present value of energy savings with the included new cost parameter of

imate for the facility.

oturbine Base load microturbine Solar PV

4000 600 98% [4] 14% (geographical

estimate) data) 0.0889 (historical data) 0.04 (utility contract)

8 (historical data) 8 (historical data) 0.0114 [4] N/A 0.004715 [4] N/A

dard) 1.2, (Industry standard) N/A

creased effi- bine divide

1.33 (To get the increased effi- ciency of microturbine divide 1.6 by 1.2)

N/A

0.3 [4] N/A

N/A N/A

300 (estimate) 3000 (estimate) ied historical 0 500 (Utility supplied his-

torical data) 0 0

Table 12 Frame II – Facility cost parameter values for baseline lifecycle cost savings estimate for the facility.

Symbol Facility cost parameter Value

N Heating yearly demand (MMBTU)

172,189 (Utility supplied historical data)

O Chilled water months 6 (Weather based) P Electricity yearly demand 35,172,980 (utility supplied historical

data) Q Facility Labor Rate ($/h) $127,870 (Facility supplied data) R Outages per year (h/year) 40 (Facility supplied data)

Table 13 Frame II – Calculation methodology for uncertainty analysis of the lifecycle cost savings for the facility.

Information needed Calculation

Ten year NPV of cost savings ¼P10 t ¼ 0

VTotal ð1þ0:03Þ�K*A

Total annual savings (VTotal) – baseline and natural gas scenarios VelectricityþVheatingþVcoolingþVO&M�Vfuel Total annual savings (VTotal) – storms scenario VelectricityþVheatingþVcoolingþVO&M�VfuelþVoutage Total annual savings (VTotal) – renewable policies and biofuels scenarios VelectricityþVheatingþ VcoolingþVO&M�VfuelþVRECs Annual value of electricity (Velectricity) A � B � 8760 � C Annual value of heating produced (Vheating) Minimum {A � N � B � 8760 � D, N} Annual value of cooling produced (Vcooling) A � I � J � O/12 � B � 8760 � C Annual operation and maintenance savings (Vcooling) 500-L Annual value of the cost of fuel (Vfuel) A � E � B � 8760 � D Annual facility outage (Voutage). A � B � 8760/P � Q � R Annual value of RECs (VRECs). A � B � 8760 � P

Fig. 3. Frame II – Ten-year NPV of lifecycle cost savings verses fuel price for several initiatives of energy security at the facility (millions $). (a) Natural gas scenario; (b) Storms scenario; (c) Renewable policies scenario; (d) Biofuels scenario.

M.C. Hamilton et al. / Reliability Engineering and System Safety 150 (2016) 11–21 19

the value of lost mission savings. The results under this scenario reveal the total lifecycle cost savings to be much more robust than to changes in the price of natural gas. This result reemphasizes the importance of reliability in the energy system plans. Due to the unreliable nature of the local grid, it would be very valuable to the installation to install a highly reliable distributed generation energy technology to protect their sensitive scientific laboratories.

Renewable policies. This scenario explores a combination of emer- gent regulatory and technology emergent conditions that might result

from the scenario s2_Renewable Policies. It assumes that Virginia implements a mandatory renewable portfolio standard and includes combined heat and power in the as eligible sources of renewable energy credits. Currently 12 states include CHP or waste heat recovery as eligible resources [43]. This emergent condition would add a new environmental revenue cost parameter for the kilowatt-hours of elec- tricity generated by the distributed generation systems. It is assumed that the renewable energy credits (RECs) created by the microturbines would be similar to that of Tier II RECs in other states that include

M.C. Hamilton et al. / Reliability Engineering and System Safety 150 (2016) 11–2120

combined heat and power applications in their RPS, and a value of $2/ MWh is assigned to the electricity generated from the microturbines. The RECs generated by the solar PV cells are assumed to be much more valuable, which is typical in most states, and a value of $200/MWh is assigned to the electricity generated from solar. A second emergent condition included in this scenario is that the demand charge portion of the utility electricity charges becomes more favorable towards dis- tributed generation. This could be implemented in a variety of ways, including changing the demand charges from an eleven-month ratchet to a daily ratchet. Under this emergent condition, the value of electricity generated by the solar PV cells would increase from the assumed $0.04/ kwh–$0.0889/kwh. The net present value of the initiatives under this scenario are $3.3 million, $8.6 million, and �$0.7 million for critical load microturbine, base load microturbine, and solar PV respectively. Fig. 3c shows the ten-year present value of the three initiative energy systems with the changes and additions to the lifecycle cost parameters described above. Notice that these only very slightly increase the price at which the net present value of saving becomes negative. The biggest change is for solar PV, which now has a positive ten-year net present value. The results from this scenario indicate that RECs for solar and revised utility demand charges would make the solar initiative much more economically viable in the region.

Biofuels. This scenario makes the same assumptions as the renewable policies scenario with the additional assumption that an affordable source of biofuel is accessible as feedstock for the microturbines. This may be in the form of biogas from a nearby waste facility. Under this scenario, the price of the renewable energy credits from the microturbines would increase to a value similar to Tier I RECs. The 2010 national average spot price for RECs varied between $28/MWh and $45/MWh. A price of $34/MWh is used in this scenario. The net present values of the initiatives under this scenario are $8.9 million, $21 million, and $0.7 million for critical load microturbine, base load microturbine, and solar PV respectively. The Fig. 3d displays the ten-year present value of the several initiatives for this scenario verses the price of fuel. Under this scenario the lifecycle cost savings remains positive if the price of fuel, in this case, biogas, is below $20/MMBTU for the base load microturbines and $25/MMBTU for the critical load microturbines.

There are several implications of Frame II. First, it may be worth- while, in the future, to pursue negotiations with the local waste facility to procure biogas. Second, if obtaining biogas, as feedstock for the microturbines is a serious future consideration, then the instal- lation should pay close attention to the future consequences of the various financing mechanism contract options. If the installation chooses to use a PPA to finance the microturbines now, the PPA would most likely get rights to the RECs, unless explicitly worked out dif- ferently in the contract. Also, if the agency wanted to use the RECs to meet the EPAct 2005 requirements instead of selling them for profit, they would be unable to unless they owned the right to the RECs. The EPAct 2005 provides a bonus to Federal agencies by allowing them to double count renewable energy towards the EPAct 2005 require- ments if it is produced on-site and used at a Federal facility. Renew- able energy from a project on Federal land is only available if an agency retains the RECs associated with the generation from the project. However, agencies can “swap” RECs purchased from another source to replace RECs sold to finance an on-site project and still receive the bonus [41].

3.4. Resilience analytics across frames

The analysis in two frames presented above is an example of managing deep uncertainties for energy investments, addressing key epistemic or knowledge uncertainties in multiple, overlapping problem frames. An aim is to encourage stakeholders to creatively address how to combine the best aspects of several initiatives to be resilient to deep uncertainties. For example, insights gained

from considering deep uncertainties across frames led to the development of the following plan. The facility plans to pursue detailed design of the critical load microturbines combined with a microgrid of backup generators in the first phase of development. This will provide protection against interruptions of both the electrical grid and the natural gas system for the mission critical buildings, and it would lessen the need to store large amounts of natural gas on site. It would also satisfy the criteria for an inno- vative use of technology. A second phase of initiatives may include migrating to a base load of microturbines in order to decommis- sion the steam plant. A third phase of initiatives includes inte- grating solar PV or working with the local landfill to procure biofuels to further satisfy the AESIS goals and environmental impact criteria. The third phase could wait until the capital costs of solar PV decrease, Virginia state or national renewable portfolio standards are implemented, and utility demand charges are renegotiated. It has thus been shown that iterative analysis, con- sidering different problem frames, is able to inform decision makers as to initiatives that are most resilient across scenarios.

4. Conclusions

There is an urgent need to understand the influence of deep uncertainties, including those related to stakeholder preferences, to the resilience of large-scale systems. Risk and uncertainty analysis in such systems tends to proceed ad hoc with insufficient regard to the influence of uncertainties to system resilience. In particular, energy managers require consideration of lifecycle costs, environmental impacts, and energy security goals, etc., as well as understanding how deep uncertainties (technology, environment, political, regulatory, socio-demographic, behavioral, economic, others) are manifested in scenario-based preferences. The present method of resilience analysis considers the degree of disruptiveness of scenario-based preferences on priorities. Frame 1 explores the resilience of initiatives to changes in preferences due to emergent conditions, while Frame 2 uses sce- narios of emergent conditions from the preceding frame to explore the impacts of lifecycle cost of key initiatives. This paper highlights the approach and outcomes of resilience analytics to identify influ- ential uncertainties and develop creative solutions.

The key insights of the two frames of the case study are as follows. Frame I of the approach identified several emergent conditions that should influence the prioritization of initiatives, including: Economic conditions, technology innovation, cyber and terrorism, and islanding scenarios. Frame II quantified the impacts of scenarios for the lifecycle costs of energy security initiatives, and thus, added critical evidence for or against initiatives in the negotiating process. Frame II performed a net present value cost analysis to understand the sensitivity of lifecycle valuations to (i) fuel prices; (ii) the cost of lost mission; (iii) availability of renewable energy credits and utility contract changes; and (iv) availability of biofuels. First, this analysis shows that if the price of natural gas increases by fifty-percent than the ten-year present value of all initiatives under consideration will be negative. In considering the resilience of initiatives to emergent conditions, the stakeholders should be confident that natural gas could be procured at an affordable long- term price. Second, the scenario of cost of lost mission savings indicates that this cost parameter far exceeds traditional lifecycle cost savings. This re-emphasizes the importance of prioritization of energy initiatives for resilience. Third, the lifecycle cost analysis shows that renewable energy credits can produce a significant improvement in savings. This indicates the benefits to system resilience of pursuing negotiations with the local waste facility to procure biogas. It would be worthwhile to negotiate ownership of the future REC rights under various financing mechanism contracts if the installation wants to use the RECs towards meeting its EPAct 2005 renewable energy requirements. Furthermore, retaining the rights to the RECs produced by the solar technology would

M.C. Hamilton et al. / Reliability Engineering and System Safety 150 (2016) 11–21 21

also be a worthwhile avenue to pursue, as these very valuable RECs could be sold and cheaper ones could be purchased in replacement and the installation would still be able to get credit towards EPAct 2005 renewable requirements.

References

[1] Belton V, Stewart TJ. Multiple criteria decision analysis: an integrated approach. Boston: Kluwer Academic; 2002.

[2] Bohnenblust H, Slovic P. Integrating technical analysis and public values in risk-based decision making. Reliab Eng Syst Saf 1998;59(1):151–9.

[3] Brito AJ, de Almeida AT. Multi-attribute risk assessment for risk ranking of natural gas pipelines. Reliab Eng Syst Saf 2009;94:187–98.

[4] Capstone, Inc. P0410 C1000 Megawatt power package HP natural gas data sheet CAP148. Capstone P/N 331044D.

[5] Catrinu MD, Nordgard DE. Integrating risk analysis and multi-criteria decision support under uncertainty in electricity distribution system asset manage- ment. Reliab Eng Syst Saf 2011;96:663–70.

[6] Corner J, Buchanan J, Henig M. Dynamic decision problem structuring. J Multi- criteria Decis Anal 2001;10:129–41.

[7] Defense Science Board. Report of the defense science board task force on DoD energy strategy. “More Fight – Less Fuel” Office of the Under Secretary of Defense For Acquisition. Technology, Logist Wash, DC 2008:20301–3140.

[8] Energy Policy Act of 2005. Public law 109-58; AUG. 8, 2005. [9] Executive Order 13423. Strengthening federal environmental, energy, and

transportation management; 2007, January, 24. [10] Francis R, Bekera B. A metric and frameworks for resilience analysis of engineered

and infrastructure systems. Reliab Eng Syst Saf 2014;121(2014):90–103. [11] Gibson JE, Scherer WT, Gibson WF. How to do systems analysis. . John Wiley &

Sons; 2007. [12] Goodwin P, Wright G. Enhancing strategy evaluation in scenario planning: a

role for decision analysis. J Manag Stud 2001;38(1):1–16. [13] Hamilton MC, Lambert JH, Keisler JM, Holcomb FH, Linkov I. Research and

development priorities for energy islanding of military and industrial instal- lations. J Infrastruct Syst 2013;19(3):297–305.

[14] Hamilton MC, Thekdi SA, Jenicek EM, Harmon RS, Goodsite ME, Case MP, Karvetski CW, Lambert JH. Case studies of scenario analysis for adaptive management of natural resource and infrastructure systems. Environ Syst Decis 2013;33:89–103.

[15] Henry D, Ramirez-Marquez JE. Generic metrics and quantitative approaches for system resilience as a function of time. Reliab Eng Syst Saf 2012;99(1):114–22.

[16] ISO 31000:2009, Risk management – Principles and guidelines. [17] Karvetski CW, Lambert JH. Evaluating deep uncertainties in strategic priority

setting with an application to facility energy investments. Syst Eng 2012;15 (4):483–93.

[18] Karvetski CW, Lambert JH, Linkov I. Scenario and multiple criteria decision analysis for energy and environmental security of military and industrial installations. Integr Environ Assess Manag 2011;7(2):228–36.

[19] Karvetski CW, Lambert JH, Linkov I. Emergent conditions and multiple criteria analysis in infrastructure prioritization for developing countries. J Multicriteria Decis Anal 2009;16:125–37.

[20] Keeney RL, Raiffa H. Decisions with multiple objectives–preferences and value tradeoffs. Cambridge & New York: Cambridge University Press; 1993.

[21] Kleber D. The US department of defense: valuing energy security. J Energy Secur 2009.

[22] Lambert JH, Karvetski CW, Linkov I, Abdallah T. Energy security of military and industrial facilities: a scenario-based multiple criteria decision analysis to identify threats and opportunities. In: Proceedings of the tenth international conference on probabilistic safety assessment and management (IAPSAM), Seattle; 2010.

[23] Lambert JH, Karvetski CW, Spencer DK, Sotirin BJ, Liberi DM, Zaghloul HH, Koogler JB, et al. Prioritizing infrastructure investments in afghanistan with

multiagency stakeholders and deep uncertainty of emergent conditions. ASCE J Infrastruct Syst, 18; 2012. p. 155–66.

[24] Lempert R, Popper S, Bankes S. Shaping the next one hundred years, New methods for quantitative Long-term policy analysis, MR-1626-RPC, Santa Monica, The RAND Pardee Centre; 2003.

[25] Martinez LJ, Lambert JH, Karvetski CW. Scenario-informed multiple criteria analysis for prioritizing investments in electricity capacity expansion. Reliab Eng Syst Saf 2011;96:883–91.

[26] Montibeller G, Gummer H, Tumidei D. Combining scenario planning and multicriteria decision analysis in practice. J Multi-criteria Decis Anal 2006;14:5–20.

[27] Montibeller G, Franco A. Multi-criteria decision analysis for strategic decision making. Handbook of multicriteria analysis. Berlin Heidelberg: Springer; 2010. p. 25–48.

[28] Pant R, Barker K, Ramirez-Marquez JE, Rocco S CM. Stochastic measures of resilience and their application to container terminals. Comput Ind Eng 2014;70(1):183–94.

[29] Parnell GS, Jackson JA, Burk RC, Lehmkuhl LJ, Engelbrecht JA. R&D concept decision analysis: using alternate futures for sensitivity analysis. J Multicriteria Decis Anal 1999;8:119–27.

[30] Phillips LD. A theory of requisite decision models. Acta Psychol 1984;56 (1):29–48.

[31] Pless J. Natural gas development and hydraulic fracturing: a policymaker’s guide. isbn 978-1-58024-662-0. National conference of state legislatures; 2012. [retrieved 19.03.13] ⟨www.ncsl.org/documents/energy/frackingguide_ 060512.pdf⟩.

[32] Ram C, Montibeller G, Morton A. Extending the use of scenario planning and mcda for the evaluation of strategic options. J Oper Res Soc 2011;62(5):817–29.

[33] Rushing A, Kneifel J, Lippiatt B. Energy price indices and discount factors for lifecycle cost analysis. S Dep Commer Technol Adm 2010.

[34] Schoemaker PJH. Scenario planning: a tool for strategic thinking. Sloan Manag Rev 1995;36(2):25–40.

[35] Schroeder MJ, Lambert JH. Scenario-based multiple criteria analysis for infrastructure policy impacts and planning. J Risk Res 2011;14(2):191–214.

[36] Shafieezadeh A, Burden LI. Scenario-based resilience assessment framework for critical infrastructure systems: case study for seismic resilience of seaports. Reliab Eng Syst Saf 2014;132:207–19.

[37] Stewart TJ, French S, Rios J. Integrating multicriteria decision analysis and scenario planning: review and extension. Omega -Int J Manag Sci 2013;41 (4):679–88.

[38] Teng K, Thekdi SA, Lambert JH. Identification and evaluation of priorities in the business process of a risk or safety organization. Reliab Eng Syst Saf 2012;99 (2012):74–86.

[39] Thekdi SA, Lambert JH. Integrated risk management of safety and develop- ment on transportation corridors. Reliab Eng Syst Saf 2015;138(2015):1–12.

[40] Tylock SM, Seager Thomas P, Snell Jeff, Bennett Erin R, Sweet Don. Energy management under policy and technology uncertainty. Energy Policy 2012;47:156–63.

[41] U.S. Department of Energy Federal Energy Management Program (U.S. DoE FEMP). Federal renewable production and consumption requirements; 2010. [retrieved 01.11.10] ⟨http://www1.eere.energy.gov/femp/technologies⟩.

[42] U.S. Energy Information Association (EIA). Natural gas historical data. [retrieved 01.03.13] ⟨http://www.eia.gov/dnav/ng/hist/n3020us3m.htm⟩.

[43] U.S. Environmental Protection Agency. Combined heat and power partnership. environmental revenue streams for combined heat and power; 2008. Retrieved from ⟨www.epa.gov/chp/documents/ers_program_details.pdf⟩.

[44] United Stated Army. Army energy security implementation plan (AESIS); 2009. [retrieved 01.05.10] ⟨www.asaie.army.mil/…/AESIS_13JAN09_Approved%204- 03-09.pdf⟩.

[45] Wright G, Goodwin P. Future‐focussed thinking: combining scenario planning with decision analysis. J Multi‐Criteria Decis Anal 1999;8(6):311–21.

[46] You H, Lambert JH, Clarens AF, McFarlane BJ. Quantifying the influence of climate change to priorities for infrastructure projects. IEEE Trans Syst Man Cybern: Part A 2013:1–13.