Critical Infrastructure

ElectricPower.pdf

Home >Government homework help >Critical Infrastructure

236

ElEctric PowEr

In 2000, the National Academy of Engineering named modern power grids—those vast electrical power genera- tion, transmission, and distribution networks that span the country—the top engineering technology of the twentieth century. According to the Academy’s opinion, the Power Grid surpassed the invention of the automobile, airplane, moon shot, atomic bomb, delivery of safe and abundant water, and electronics as the most important engineering accomplishment. Electrical power is what makes modern society tick. It is essential. So it comes as no surprise that the grid is one of the fundamental infrastructures of the United States.

In this chapter, you will learn the following concepts and be able to apply them to the challenge of electrical power grid risk analysis:

1. Blackouts are increasing: The frequency and size of power outages have been rising exponentially since deregulation in 1992. This increase is traced to a number of factors, including, but not limited to, under- investment in transmission and distribution, deregula- tion of utilities resulting in loss of control, and network topology—rising self-organized criticality (SOC) due to the Grid’s wiring diagram.

2. Deregulated utilities: Historically, the components of power—generation, transmission, distribution, load (consumption), and SCADA control—have been owned and operated by vertically integrated utility companies. Since 1992, these vertically integrated monopolies have been disaggregated and decoupled from generation, transmission, and distribution of

power through deregulation legislation. Unfortunately, deregulation has brought with it economic and control vulnerabilities that are still being worked out. By sep- arating key components of the Grid into competing companies, regulation has introduced instabilities in command and control of the Grid.

3. Deregulation and physics: The power Grid has been and continues to be shaped by a combination of gov- ernmental regulation and the laws of physics—these two do not always work together. Physics demands rigorous control of complex electrical circuits. Deregulation often ignores this requirement by sepa- rating control from the operators, thus introducing instability. A deregulated Grid is like a highway net- work with thousands of vehicles going in different directions: accidents are bound to happen.

4. There is no shortage of power, but there is a shortage of transmission and distribution capacity. The U.S. produces approximately 15% more power than it con- sumes. But it cannot always deliver power to where it is needed, when it is needed, because of inadequate transmission and distribution capacity. This occasion- ally leads to blackouts—massive normal accidents that start small and spread to far points of the Grid.

5. Criticality of power plants: No single power gener- ator is critical—the largest source of power provides less than 1% of the national capacity. It is a myth that the most vital components of the nation’s power sector are power plants. This points once again to the “middle” of the Grid as the most likely place for failures to occur.

Lewis, T. G., & Lewis, T. G. (2014). Critical infrastructure protection in homeland security : Defending a networked nation. ProQuest Ebook Central <a onclick=window.open('http://ebookcentral.proquest.com','_blank') href='http://ebookcentral.proquest.com' target='_blank' style='cursor: pointer;'>http://ebookcentral.proquest.com</a> Created from apus on 2020-11-12 09:54:47.

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

THE GrID 237

6. Topology matters: Grid topology magnifies the fre- quency and size of blackouts. The Grid has com- paratively low spectral radius, so it should be resilient against cascade failures, but it is self-organized around high betweenness critical paths. These critical paths exist along transmission lines connecting power plants and major population centers. resiliency can only be improved by reducing this form of self-organization.

7. Resiliency and congestion. The western power grid (Western Electricity Coordinating Council, WECC) is used to illustrate the relationship between critical points called hot spots and congestion. It is shown that high betweenness and degree are correlated with known congestion points in the WECC. Congestion cannot be removed by increasing the capacity of one or more transmission line. Instead, the Grid must be re-wired to lower betweenness. resiliency and congestion are related to network topology.

8. Human threats: Major threat-asset pairs are traced to fuel supply chains, destabilizing physical and cyber

attacks, and attacks on critical components such as transformers and major transmission lines. Threat analysis shows that it is easier to reduce risk than vulnerability because of the Or-gate relationship in the fault tree model of terrorist threats.

9. Distributed generation: An alternative topology that solves many of the problems facing the Grid is distrib- uted generation—colocating power generation near its load. This can be achieved in two ways: (1) by switching to solar, wind, or alternative sources of power or (2) by adding storage to the Grid. Both of these reduce reliance on long-haul transmission lines.

13.1 thE Grid

The Grid, as the collection of electric power networks across the country is called, is a complex Critical Infrastructure and Key resources (CIKr) system consisting of four major components as shown in Figure 13.1. Power is generated by

Generation

Transmission

SCADA

Distribution

Load Residential Commercial Industrial

Distribution substation

Subtransmission substation

Transmission substation

FiGurE 13.1 The five major components of the power grid are generation, transmission, distribution, load, and ICS-SCADA, typically called an Energy Management System (EMS).

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

238 ElECTrIC POWEr

burning a fossil fuel or turning a turbine by wind, water, or tidal action, and then put into a vast transmission system that transports the electrons long distances to local distribution networks. Transmission is more economical if done at high voltages, so it must be stepped down and redirected by sub- stations along the way. Metropolitan level distribution net- works further distribute stepped-down power to residential, commercial, and industrial customers called the load.

The entire generation, transmission, and distribution process is monitored by an industrial control SCADA, typ- ically called an EMS. This ICS-SCADA system is typically out of band, which means it is a separate communications network running parallel to the Internet and power lines, although this practice is changing as SmartGrid technol- ogies are adopted. SmartGrid is the name given by Massoud Amin to describe the convergence of electric power SCADA with information technology and communications [1].1

Amin’s idea came from years of work on stability of aircraft and other complex systems. He was impressed by a pilot who landed a jet fighter after one wing was blown off and control surfaces were damaged. The clever pilot used thrust control to land the badly damaged airplane. This got Amin to thinking: if a pilot can control a badly damaged aircraft, then it ought to be possible for a computer to control the unruly electric power grid. To do so, operators need real-time information about the state of the grid at any instant in time. The challenge is to complete the loop from power plant to transmission, distribution, load, and back—quickly enough to head off impending disaster… computers connected to embedded sensors in power lines, transformers and electricity meters can be programmed to overcome instabil- ities created by unpredictable faults in the network.

The “unruly electric power grid” has become increasingly unruly over the past several decades due to a number of economic and regulatory missteps. Figure 13.2 documents the trend—both size and number of customers affected have increased exponentially since deregulation began in earnest in the late 1970s and continued on to the penultimate legis- lation in 1992. (Fig. 3.6 of Chapter 3 documents the decline in investment since reaching a peak in the early 1970s.)

According to Amin,

In the electricity sector, outages and power quality dis- turbances cost the economy, on average, more than US$80 billion annually and sometimes as much as US$188 billion in a single year. Due to heavier use of transmission and dis- tribution systems and more frequent congestion, T&D [Transmission and Distribution] losses almost doubled between 1970 and 2001, rising from about 5 percent to 9.5 percent. That 4.5 percentage point increase trans- lates to 184 million MWh, or electrical power for about 13 percent of U.S. households. Since 1995, the amortization

and depreciation rate on old transmission investments has exceeded new construction expenditures.

With utility construction expenditures lagging behind asset depreciation, a mode of grid operation has ensued that is analogous to harvesting crops more rapidly than replacement seeds are planted. As a result, it has been apparent for a decade that the grid is increasingly stressed and that the carrying capacity or safety margin to support anticipated demand is seriously in question.

reliability According to data assembled by the U.S. Energy Information Administration (EIA) for most of the past decade, there were 156 outages of 100 megawatts or more during 2000–2004; such outages increased to 264 during 2005–2009. The number of U.S. power outages affecting 50,000 or more consumers increased from 149 during 2000–2004 to 349 during 2005–2009, according to EIA. [2]

Amin has set the stage for the technical and policy chal- lenges facing the Grid. In the following analysis, attention will be focused on the main problem—lack of transmission capacity. How did the Grid become a victim of the tragedy of commons?

13.2 From dEath rays to VErtical intEGration

The Grid has its historical roots in the famous Pearl Street New York utility created by Thomas Edison in the 1880s. This first utility supplied direct current (DC) electrical power to 59 Manhattan customers. Edison was convinced that DC was the best way to deliver electricity, but Serbian immigrant Nikola Tesla had a better idea: alternating current (AC). Tesla was Edison’s rival in all things having

300

250

200

150

100

N um

be r

of o

ut ag

0 2000–2004 2005–2009

Number of outages affecting 50,000 or more customers

Historical analysis of US outages (2000–2009) (Adjusted for 0.9%/year increase in load to 2000 levels)

Number of outages of 100 MW or more

FiGurE 13.2 Power outages in the United States have increased in size over the past decade due to regulation and tragedy of the commons (underinvestment in transmission).

1A personal communication with Massoud Amin.

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

FrOM DEATH rAYS TO VErTICAl INTEGrATION 239

to do with harnessing the power of the electron. He is the father of all modern electric power generation technology (generators), distribution (transmission lines and substa- tions), and appliances (motors).

A titanic power struggle between Tesla and Edison ensued over the advantages of AC versus DC. When Tesla sold his patent rights to George Westinghouse, Edison’s feud shifted from Tesla to Westinghouse. Edison derided AC. At one point, he used the electric chair to convince consumers that AC was unsafe. Tesla countered with daring demonstrations of his own:

Tesla gave exhibitions in his laboratory in which he lighted lamps without wires by allowing electricity to flow through his body, to allay fears of alternating current. He was often invited to lecture at home and abroad. The Tesla coil, which he invented in 1891, is widely used today in radio and televi- sion sets and other electronic equipment. That year also marked the date of Tesla’s United States citizenship.

Westinghouse used Tesla’s system to light the World’s Columbian Exposition at Chicago in 1893. His success was a factor in winning him the contract to install the first power machinery at Niagara Falls, which bore Tesla’s name and patent numbers. The project carried power to Buffalo by 1896.

In 1898 Tesla announced his invention of a tele-automatic boat guided by remote control. When skepticism was voiced, Tesla proved his claims for it before a crowd in Madison Square Garden.

In Colorado Springs, Colo., where he stayed from May 1899 until early 1900, Tesla made what he regarded as his most important discovery—terrestrial stationary waves. By this discovery he proved that the Earth could be used as a conductor and would be as responsive as a tuning fork to electrical vibrations of a certain frequency. He also lighted 200 lamps without wires from a distance of 25 miles (40 kilometers) and created man-made lightning, producing flashes measuring 135 feet (41 meters). At one time he was certain he had received signals from another planet in his Colorado laboratory, a claim that was met with derision in some scientific journals.

Tesla was a godsend to reporters who sought sensational copy but a problem to editors who were uncertain how seri- ously his futuristic prophecies should be regarded. Caustic criticism greeted his speculations concerning communica- tion with other planets, his assertions that he could split the Earth like an apple, and his claim of having invented a death ray capable of destroying 10,000 airplanes at a distance of 250 miles (400 kilometers).2

Eventually, the Tesla–Westinghouse approach won out and established AC as the standard technology for power gener- ation and distribution. AC could be transmitted over longer distances than DC, easily powered motors used in factories and homes, and could be voltage-stepped up/down to accom- modate different needs for a diverse consumer.

By 1896, the Tesla–Westinghouse collaboration resulted in hydroelectric power generation at Niagara Falls and AC transmission to Buffalo 20 miles away. Edison’s DC power networks were limited to 1 mile. This was the first Grid. It showed the technical and economic feasibilities of electric power. Soon, privately owned and operated “power com- panies” sprang up across the nation. These companies were vertically integrated as shown in Figure 13.3. But over the course of a century, these vertically integrated power companies would be broken up into nonvertical oligopolies. The business of power distribution would take another 100 years to perfect.

13.2.1 Early regulation

The first modern governmental regulator of all things having to do with energy and power—the Federal Power Commission (FPC)—was set up by Congress to coordinate hydroelectric projects in 1920. FPC grew over the decades and eventually has become FErC (Federal Energy regulatory Commission), with a budget exceeding US$200 million and vast regulatory powers over natural gas and electrical power. But in the 1920s, electrical power generation, transmission, and distri- bution were owned by large interstate holding companies that optimized the flow of power from fuels, such as coal, or hydroelectric generators. They exercised control of their regions of the country by vertically integrating all aspects of production, distribution, and marketing. See the regulated model of Figure 13.3.

The vertical monopolies standardized on 60 Hz (cycles/s) and 240-/120-V current, but they were stove-piped islands when it came to interoperability. Two AC signals have to be synchronized before they can be combined across vertical monopolies. Synchronization would remain a technical challenge into the twenty-first century, including problems integrating solar and wind power into the Grid.

Standardization and synchronization were needed before privately held vertical monopolies could interoperate. Universal access—the ability for anyone in the United States

ConsumersConsumers

DeregulatedRegulated

Distribution Distribution

TransmissionTransmission

Generation

ISO

FiGurE 13.3 Vertically integrated power companies have been broken into oligopolies over the past 100 years.2http://www.neuronet.pitt.edu/~bogdan/tesla/bio.htm

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

240 ElECTrIC POWEr

to get electrical power service—had not yet arrived. It would require interoperability, a technical capability that was lack- ing among the local monopolies.

The Federal Power Act of 1920, the Natural Gas Act of 1938, and the Public Utility Holding Company Act (PUHCA) of 1935 changed the landscape by empowering the FPC to regulate the sale and transportation of natural gas and electricity across state borders. Together, these laws defined power and energy transmission as interstate commerce, which is the exclusive purview of the legislative Branch of Government. Thus, a state could not directly regulate that commerce—but Congress could. PUHCA shaped the electric power industry until 1992.

A series of legal modifications to PUHCA expanded the power of Congress to regulate power and energy com- panies. For example, the Natural Gas Act was amended in 1940 to charge the FPC with responsibility for certifying and regulating natural gas facilities—going beyond simply regulating the sale of power across interstate boundaries.

The Northeast Blackout of 1965 highlighted the vulnera- bility of the vertically integrated power grid. As local holding companies were encouraged to interoperate and borrow power from one another to accommodate surges in demand, they also became more fragile. A loss of capacity in one region could lead to a series of failures that could collapse entire regions. Thus, the cascade failure was born. Significantly, it forced a shift in federal regulatory legisla- tion from pure regulation and universal access to an emphasis on safety and reliability.

The first prerequisite for prevention of cascade failures is that the power grid must be extremely reliable. Even a relatively insignificant component such as a power line must not fail. Thus, the North American Electric reliability Council (NErC) was formed shortly after the blackout in 1965. NErC is a not-for-profit company formed to promote the reliability of bulk electric systems that serve North America.3

The energy crisis of the 1970s brought fuel price inflation, conservation, and a growing concern for the environment. Congress began to shift its emphasis once again from reli- ability to clean and inexpensive power. The Public Utilities regulatory Policies Act (PUrPA) was enacted in 1978 to promote conservation of energy. But it had an important side effect: it opened the vertically integrated monopolies to com- petitors. PUrPA required the electric utilities to buy power from “qualified facilities” (QFs). Thus was born the non- utility generator and independent power producer. This side

effect was expanded in 1992 when the vertical monopolies were broken up by deregulation legislation.

In 1977, Congress transferred the powers of FPC into FErC—an independent agency that regulates the interstate transmission of natural gas, oil, and electricity. FErC main- tained the shape of the electrical power sector during the 1980s and early 1990s. (For details on FErC’s regulatory powers, see Chapter 12.)

FErC interprets and implements regulatory statutes that grant an exclusive franchise to electric utilities in exchange for low-cost universal access by all consumers. Universal access means that a lone farmer in a relatively sparse part of the country has access to electric power at the same cost as a city dweller surrounded by thousands of ratepayers. The cost of providing universal access was amortized over all con- sumers. This forced the monopolies into a “cost plus” business model rather than a model that encouraged innovation and expansion of power options. Universal access and regulation produced highly efficient, reliable, environmentally sensitive power, at the expense of technological advancement.

13.2.2 deregulation and EPact 1992

The era of regulated, layered vertical monopolies shown as in the regulated model of Figure 13.3 came to an end in 1992 with the enactment of the Energy Policy Act (EPACT). EPACT dramatically changed the industry once again. In addition to retaining clean, environmentally safe, reliable power, Congress now required utilities to provide “nondiscrimina- tory” transmission access to the transmission and distribution layers as shown in the deregulation model of Figure 13.3. Deregulation essentially replaced monopolies by oligopolies.

Under PUrPA 1978, any qualifying facility (QF) can use any part of the power grid to deliver its power to consumers. Under EPACT 1992, utilities are required to divest their interest in generation and open their transmission networks to any competitor. The intention paralleled other deregu- lations such as the 1996 Telecommunications Act, which promoted innovation by creating competition. Unfortunately, EPACT 1996 plunged the grid into chaos. According to one industrial expert, the modern deregulated power industry is like a gasoline industry that fixes the price of oil at US$30/ barrel but allows the retail price of gasoline to rise up to US$450/gallon!

A particularly extreme example of the new sensitivity of prices occurred during the latter part of June 1998. For several days, spot-market prices for electricity in the Midwest experi- enced almost unheard-of volatility, soaring from typical values of about US$25 per megawatt-hour (2.5 cents per kilowatt-hour) up to US$7,500 per megawatt-hour (US$7.50 per kilowatt-hour). Because the affected utilities were selling the power to their customers at fixed rates of less than 10 cents per kilowatt-hour, they lost a lot of money very quickly.

3“NErC’s members are the 10 regional reliability Councils whose members come from all segments of the electric industry: investor-owned utilities; federal power agencies; rural electric cooperatives; state, municipal, and provincial utilities; independent power producers; power marketers; and end-use customers. These entities account for virtually all the electricity supplied in the United States, Canada, and a portion of Baja California Norte, Mexico,” http://www.nerc.com.

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

OUT OF OrDErS 888 AND 889 COMES CHAOS 241

The run-up in prices was so staggering that it might take an everyday analogy to appreciate it. In the 1970s, drivers howled when the price of gasoline tripled.

Imagine your consternation if, one day, you pulled into a gas station and discovered the price had increased three hun- dredfold, from US$1.50 per gallon to US$450 per gallon.

Most of us would look for alternative transportation. But with electricity you do not have options. With no way to store it, the affected utilities had a choice of either paying the going rate, or pulling the plug on their customers on the hottest day of the year. The total additional charges incurred by the utilities as a result of the price spike were estimated to be US$500 million. [3]

As we shall see, this peculiar mixture of physics and eco- nomics will lead to vulnerabilities in the grid that must be considered when establishing policies for the protection of this very critical infrastructure. In particular, the grid has been made more vulnerable at the point in history when it should be made less vulnerable. Economics has been given precedence over security. Deregulation encourages competi- tion, but it discourages investment in the grid, itself. The Grid currently suffers from the tragedy of the commons—a phenomenon described in greater detail in Chapter 3.

13.2.3 Energy sector isac

The Electricity Sector (ES)-Information Sharing and Analysis Center (ISAC) should not be confused with EISAC—the Energy ISAC that deals with oil and natural gas information sharing.4 ES-ISAC is run by the NErC and serves the electricity sector. It provides sharing among its electric sector members, federal government, and other critical infrastruc- ture industries. Specifically, the mission of ES-ISAC is to collect and analyze security data and disseminate its analysis and warnings to its members, the FBI, and the Department of Homeland Security (DHS).

According to the ES-ISAC web site:

The Electricity Sector Information Sharing and Analysis Center (ES-ISAC) establishes situational awareness, incident management, coordination and communication capabilities within the electricity sector through timely, reliable and secure information exchange. The ES-ISAC, in collaboration with the Department of Energy and the Electricity Sector Coordinating Council (ESCC), serves as the primary security communications channel for the electricity sector and enhances the ability of the sector to prepare for and respond to cyber and physical threats, vulnerabilities and incidents.

The ES-ISAC engages in the following activities:

• Identifies, prioritizes, and coordinates the protection of critical power services, infrastructure service, and key resources

• Facilitates sharing of information pertaining to physical and cyber threats, vulnerabilities, incidents, potential protective measures, and practices

• Provides rapid response through the ability to effec- tively contact and coordinate with member companies as required

• Provides and shares campaign analysis that includes capturing, correlating and trending data for historical analysis, and sharing that information within the sector

• receives incident data from private and public entities

• Assists the Department of Energy, the Federal Energy regulatory Commission, and the DHS in analyzing event data to determine threat, vulnerabilities, trends, and impacts for the sector, as well as interdependencies with other critical infrastructures (This includes integration into DHS’ National Cyber security and Communications Integration Center.)

• Analyzes incident data and prepares reports based on subject matter expertise in security and the bulk power system

• Shares threat alerts, warnings, advisories, notices, and vulnerability assessments with the industry

• Works with other ISACs to share information and provide assistance during actual or potential sector disruptions whether caused by intentional, accidental, or natural events

• Develops and maintains an awareness of private and government infrastructure interdependencies

• Provides an electronic, secure capability for the ES-ISAC participants to exchange and share information on all threats to defend critical infrastructure

• Participates in government critical infrastructure exercises

• Conducts outreach to educate and inform the electricity sector5

13.3 out oF ordErs 888 and 889 comEs chaos

The EPACT of 1992 opened up the formerly closed trans- mission and distribution grid to all comers (FErC order 889). The power companies of the vertically integrated era are now required to buy power from QFs and allow com- petitors to use their transmission and distribution lines. But they can only charge consumers a usage fee set by state regulators—not by them. retail prices are fixed while wholesale prices are allowed to float. The new grid is a com- petitive marketplace—almost. Floating wholesale prices can be inflated to the advantage of the seller, but each state sets retail prices as low as possible for political reasons. This

4http://www.esisac.com/ 5http://www.esisac.com/SitePages/Home.aspx

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

242 ElECTrIC POWEr

has created chaotic economic shockwaves in states like California where power brokers have been allowed to “game the system” through predatory pricing contracts. Enron was perhaps the most notorious example of this practice.

The modern deregulated grid is still regulated for the purpose of encouraging innovation through competition. Still, it is a regulated industry with layers of regulators as shown in Figure 13.4. By Order 888, FErC created ISOs (Independent System Operators) that essentially replaced the monopolistic utilities with nonprofit “broker” companies. ISOs are nonprofit markets where buyers meet sellers. According to Overbye [3]:

In a bid to ensure open and fair access by all to the transmis- sion system, in Order 888 FErC envisioned the establishment of several region wide entities known as ISOs, or Independent System Operators. The purpose of the ISO is to replace the local utility’s operation of the grid by a private, not-for-profit organization with no financial interest in the economic performance of any market players. In short, the job of the ISO is to keep the lights on, staying indepen dent of and there- fore impartial to the market players. As of the end of 1999 ISOs were operating the electrical grid in California, New England, New York, Texas and the coordinated power market known as PJM (Pennsylvania–New Jersey–Maryland).

Under EPACT 1992, the responsibilities of an ISO are to:

• Control the transmission system

• Maintain system reliability

• Provide ancillary services such as system and voltage control, regulation, spinning reserve, supplemental operating reserve, and energy imbalance

• Administer transmission tariff

• Manage transmission constraints

• Provide transmission system information (Open Access Same-Time Information System, OASIS)

• Operate a power exchange (optional)

Sometimes, the ISO separates the buying and selling activ- ities from the regulation and reliability activities. In this case, they set up a separate power exchange. These are trading centers where utilities and other electricity suppliers submit price and quantity bids to buy and sell energy or ser- vices. Enron Online was one such exchange. It bought power on contract and resold its contracts to utility companies like PG&E in California. At one time, Enron Online cornered enough of the California market that it could charge what- ever it wanted. This led to the California energy crisis in the late 1990s, which in turn led to the downfall of California’s Governor.

FErC requires an ISO to monitor its energy market for manipulation or abuses by the participants. This require- ment covers both the power exchange (auction-based) market and bilateral transactions in the region (wheeling). An ISO’s authority to take corrective action when market abuses are identified depends on the nature of the abuse. In the case of abuses by Enron in 2002, the Department of Justice—not the ISO—pursued malfeasance charges against Enron executives.

Congress legislates and FErC regulates through coop- eration with NErC. NErC has divided the United States and Canada into geographical areas called Reliability Coordinators. Each reliability coordinator oversees the operation of a number of Reliability Assessment Areas sometimes called “wheels.” reliability coordinators mon- itor and adjust the flow of electrons throughout their region of responsibility. Buying and selling across control areas is called “wheeling” in the terminology of grid operators. The major reliability coordinators and control areas of North America are shown in Figure 13.5. Alphabetically, they are as follows:

NERC Reliability Assessment Areas

BASN: Basin (WECC)

CAlN: California, North (WECC)

CAlS: California, South (WECC)

DSW: Desert Southwest (WECC)

ErCOT: Electric reliability Organization of Texas (TrE)

FrCC: Florida reliability Coordinating Council

ISO-NE: ISO New England Inc (NPCC)

Congress

FERC

NERC

Reliability coordinators

Control areas (wheels)

Generation transmission & distribution

Load SCADA

FiGurE 13.4 Many layers of regulation shape the Grid: Congress, FErC, NErC, reliability coordinators, control areas, and finally the laws of physics.

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

OUT OF OrDErS 888 AND 889 COMES CHAOS 243

MAPP: Mid-Continent Area Power Pool

MISO: Midwest Independent Transmission System Operator, Inc

NOrW: Northwest (WECC)

NYISO: New York Independent System Operator (NPCC)

PJM: PJM Interconnection

rOCK: rockies (WECC)

SErC-E: SErC, East

SErC-N: SErC, North

SErC-SE: SErC, Southeast

SErC-W: SErC, West

SPP: Southwest Power Pool regional Entity

Power is moved back and forth across these areas to balance supply and demand, but this balancing act is not always easy to do, because economics and physics do not always coop- erate with one another. A surplus of electrons in one area may occur because of low demand, weather conditions, faults in transmission, or an overflow from another area. Operators have very limited options—they must either sell

the surplus to an adjacent area or shut down generation. These options are not easy to achieve in a timely manner. Hence, physics often gets in the way of economics.

13.3.1 Economics versus Physics

The economics of the deregulated grid often conflicts with the laws of physics because of the following:

• Electrons cannot be easily stored or inventoried—hence spot markets can be volatile, thus encouraging gaming of the system.

• The grid cannot easily redirect power to where it is needed—this foils demand and supply economics with both short-term and long-term implications.

• It is difficult to quantify the exact amount of power available at any point in time, which introduces human errors in the process of stabilizing the grid.

• A certain portion of the gird is “down” at any point in time because of maintenance, which makes it difficult for operators to estimate transmission and distribution capacity.

FiGurE 13.5 Major power grid interconnect components, reliability coordinators, and assessment areas of NErC include Canada and portions of Mexico [4]. Source: North American Electric reliability Corporation.

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

244 ElECTrIC POWEr

Economics and physics further clashed with politics as the Grid was deregulated throughout the 1990s and 2000s. A subtle SOC began building as a consequence—the physical distance separating power generator and customer began to increase. Power plants and solar farms were pushed away from populations to satisfy NIMBY, which required more transmission lines. It is politically easier to obtain permis- sion to put solar farms in the desert, but politicians ignored the physics of transmission. NIMBY increases the load on transmission lines because remote power generation requires more transmission capacity. Thus, EPACT 1992 increased the load on an already overtaxed transmission network. Overloaded lines tend to burn out sooner, especially during the warm summer months. Taken together, NIMBY and EPACT have steadily increased SOC.

A subtle economic SOC also began to take over: reactive power began to go away, because power companies could no longer make money from it, and yet smooth operation of the grid depends on it. reactive power is a form of electrical energy that sloshes back and forth between generator and load. Sloshing cancels the net–net transfer so there is no associated billing, hence there is no profit in maintaining reactive power. However, utilities must install heavier wires to handle the excess current—an added cost that saps profits. If power producers cannot profit from it, and utilities cannot charge for it, then why to produce it? Without reactive power, the grid became less stable.

A fourth conflict between economics, physics, and politics is emerging. Optimizing the grid by centralizing substations and power stations increases the network’s spectral radius—another step toward the critical point. Distributing control among a handful of ISOs makes things even worse. In 2000, loss of grid capacity and control cost consumers US$20 billion, but state public utility commis- sions refused to increase rates. Something had to give, so in the first few years leading up to the 2003 blackout, 150,000 skilled utility workers were let go. By August 14, 2003, the over-extended operators of the Ohio portion of the Eastern Grid had inadequate situation awareness and inadequate options for handling a normal accident. As a consequence, a tripped line in Ohio toppled the Northern portion of the Eastern Grid, leaving 55 million people without electricity for more than 2 days. Airports, railroads, factories, hospitals, highways, Internet service providers, and emergency services were shut down across portions of northeastern Canada and the United States. At least 11 people died.

Note that Edison believed in distributed generation that requires shorter transmission lines. Today’s policies push in the wrong direction as they lead to more long-haul trans- mission—the opposite of Edison’s design. As transmission lines become longer, the grid becomes less stable. Either we need to bolster long-haul transmission or return to Edison’s original design. But then Edison never had to deal with NIMBY.

13.3.2 Betweenness increases soc

In 2005, a group of researchers constructed a network model of the U.S. high-voltage transmission grid consisting of over 14,000 generators, transmission substations, and distribution substations, and over 19,000 links representing transmission lines [5]. They then identified the betweener nodes—the nodes with the largest number of shortest paths passing to- and-from other nodes. These betweeners were the critical nodes of the grid. Out of the 14,000 nodes, only 140 were important enough to bring down the entire grid. Criticality in the power grid is highly correlated with node betweenness, and cascade fragility is highly correlated with large-scale outages. This is the clue we need to fully understand and analyze the Grid.

13.4 thE north amErican Grid

The North American Electric Grid is one of the largest and most complex man-made objects ever been created. It con- sists of four large 60 Hz AC synchronous subsystems called the Eastern Interconnect, Western Interconnect (WSCC), Texas (ErCOT), and Quebec Interconnect. Figure 13.5 shows the four interconnects plus some of the subdivisions of each.6

The Eastern Interconnect has about 670,000 MW of capacity and a maximum demand of about 580,000 MW. The Western Interconnect has about 166,000 MW of capacity and a maximum demand of about 135,000 MW. ErCOT has 69,000 MW of capacity and maximum demand of 57,000 MW. Thus, there is approximately 15% more generation capacity than demand at peak levels. The North American Electric Grid has sufficient power, but it lacks the transmission and distribution capacity needed to meet surge demand. This is a consequence of the historical development of vertical monopolies and the regulatory policies of Congress. It is also the grid’s major weakness.

Theoretically, the grid is able to move power from one place to another to meet demand. For example, peak power consumption in the Eastern Interconnect occurs 3 h before peak demand in the Western Interconnect simply because of time zones. In addition, weather conditions ameliorate the demand for power. During the winter, los Angeles sends power to heat homes in the Northwest, and during the summer, Bonneville Power transmits power to southern California to run air conditioners.

But this is theory. In reality, the grid is not robust enough to transmit power to where it is needed the most. Instead, the grid has to be constantly monitored to meet demand, guard against cascading events such as tripped lines or power plants that are taken off line for maintenance. This challenge is mediated by SCADA/EMS at all levels throughout the grid.

6To see an animation of real-time flow of electricity in the Eastern Power Grid, visit: http://powerworld.com/Java/Eastern/

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

THE NOrTH AMErICAN GrID 245

The grid is made up of four major components: SCADA/ EMS, generation, transmission and distribution, and consumer load. The last three are managed by various SCADA/EMS systems. Figure 13.1 illustrates this as a unified system of components called the grid:

1. Generation—source of electric energy: coal serves as fuel for over half of the U.S. electric power generators. There are more than 10,000 different generating units with a total capacity of about 800,000 MW in the United States. The largest generation plant is Grand Coulee Dam, Washington (7000 MW from hydro), and the next largest plants are Palo Verde, Arizona (3700 MW from nuclear), W.A Parish, Texas (3600 MW of coal), and Scherer, Georgia (3400 MW from coal). Generation is fueled 56% by coal, 21% by nuclear fuel, 9.6% by natural gas, 9.5% by hydroelectric power, and 3.4% by petroleum. Most hydroelectric generators are in the East and West, most nuclear generators are in the Mid- West and East, and thermal electric generation plants are spread throughout the United States.

2. Transmission and distribution—the substations, trans- formers, and wires that carry the power from genera- tion to load. There are more than 150,000 miles of high-voltage transmission lines in the United States. High-voltage lines operate at voltages up to 765 kV (kilovolt), with many 500, 345, and 230 kV lines. Higher voltage lines typically consist of three wires attached to poles and towers by large conspicuous insu- lators. They are easy to identify from a passing automo- bile, bus, or train. Generally, they are in the open and unprotected. When a transmission line becomes too hot or short circuits, it is said to have “tripped.” Perhaps the most common fault in the grid stems from tripped lines. Often a line is overloaded in an attempt to shift power to where it is needed. The line heats up, sags, and touches a tree or the ground. Contact causes the circuit to short into the ground, and the line has to be shut down. Thus, a series of cascade failures can begin with a tripped high-power line. The greatest vulnerabilities of the grid are in the middle of the grid—its transmis- sion and distribution network. The state of the transmis- sion and distribution network is maintained by regional ISOs and the OASIS database. OASIS is an Internet- based database used by transmission providers and cus- tomers. It provides capacity reservation information, transmission prices, and ancillary services.

3. Power lines have varying capacities: The higher the voltage, the more efficient it is to transmit power. So generators deliver power to large-capacity, long-haul transmission lines (e.g., 733,000 V), which in turn deliver power to substations. The substations step the voltage down, say to 230,000 V, and then transmit to other substations, which do the same. Finally, when

the electricity reaches your backyard, it is reduced to 240/120 V. This is the idea behind the grid—use high voltage lines to move power over long distances, and small voltage lines to move power around the consumer’s home, factory, etc.

4. Load: consumers are in complete control of demand; utilities must supply enough power to meet the load at all times. Total peak demand is about 710,000 MW, but the peaks occur at different times in different regions. In addition, demand can make dramatic swings—from 20,000 to 35,000 MW over a 1 week period and as much as 8,000–20,000 MW on an hourly basis. This means the SCADA/EMS system must be highly responsive and the operators must be alert. Gas-fired peaker plants are commonly used to meet surges in demand, but it may not be possible to dis- tribute the additional capacity to where it is needed, because of inadequate transmission and distribution capacity. Hence, there is no shortage of power, but there is a shortage of transmission and distribution capacity. This, and the wild swings in demand, is the major reason for blackouts.

5. SCADA and other control systems— the control of all components of the grid. This component includes EMS and Power Plant Automation hardware and soft- ware. The main measure of how well the grid is doing is called the ACE—area control error. It is the difference between the actual flow of electricity into an area and the scheduled flow. Ideally, ACE should always be zero, but due to changing load conditions, it varies. Most wheels use automatic generation control (AGC) to adjust ACE. The goal of AGC is to keep ACE close to zero. loss of a generator, transmission line, tower, or transformer can cause abrupt changes in ACE. It can take several minutes for AGC to rectify the loss and bring ACE back to zero. This is done by a complicated series of steps involving simulation of the intended change (say to increase the power from a generator or buy power from an adjacent QF). Power control systems work much like other sec- tor’s SCADA systems. Many rTUs (remote terminal units) in the field collect data and control switches, turning them on and off. The rTU data goes into a data- base, where EMS software calculates the next setting of the switches. And like other control systems, the control network sometimes hangs from the same towers and poles as the power lines themselves.

13.4.1 acE and Kirchhoff’s law

The Grid is most vulnerable in the middle—the transmission and distribution layers—because of the insufficient capacity to deliver all the available power generated by the major interconnects. But more importantly, the Grid is a complex CIKr system subject to complex interactions due to physics.

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

246 ElECTrIC POWEr

Kirchhoff’s law says that at every point in an electrical circuit, the amount of electricity flowing in must equal the amount flowing out. Kirchhoff’s law is another way of stating “ACE must equal zero.” System operators must chase ACE to meet unpredictable demand. This means either producing more power near the load or buying power from other parts of the grid. Under deregulation, they are encour- aged to buy and sell from each other to drive ACE to zero. They are also required to allow competitors to use the old vertical monopoly’s transmission and distribution layer. Add Kirchhoff’s law to this dynamic balancing act and you risk the destabilization of the Grid.

Consider the simple hypothetical grid shown in Figure 13.6 before and after a transmission line is dropped. Figure 13.6a models a simple generator-transmission-load- SCADA feed back network. In the lower right hand corner is a node representing generators. At the top is a node repre- senting the load, and in between are substations and trans- mission lines. One link from the load back to the generators represents a feedback signal that tells the generator operators to increase or decrease power in order to guarantee ACE = 0.

Every node in the network attempts to balance inflow with outflow. The inflows from all incoming links is summed, and then apportioned equally to all outgoing links in honor of Kirchhoff’s law. The network of Figure 13.6a will self- synchronize no matter what the inflows are. The network of Figure 13.6b, will never synchronize, regardless. Why?

Note that the link between Calvert and Minor (the middle link) has been removed in Figure 13.6b to simulate a dropped transmission line. This destabilizes the network so that it is impossible to obey Kirchhoff’s law. Instead, electrical current oscillates forever as it flows through the damaged network. The reason is Figure 13.6a contains an aperiodic network while Figure 13.6b contains a periodic network. An aperiodic network self-synchronizes, whereas a periodic one does not.

An aperiodic network contains cycles of length m and n, where m and n are relatively prime. A cycle is a path from one node to other node that returns to the starting node. Two integers, m and n, are relatively prime, if one divides the other with a remainder. For example, m = 4, n = 3 are relatively prime, because 4/3 = 1 with a remainder of 1.

There are five cycles in Figure 13.6a starting from and returning to the generator node:

1. G e n e r a t o r ⟶ P o s u m ⟶ S S 1 3 ⟶ r e d Bluff ⟶ load ⟶ Generator: 5 hops

2. Generator ⟶ Calvert ⟶ SS5 ⟶ Minor ⟶ load ⟶ Generator: 5 hops

3. Generator ⟶ Calvert ⟶ Minor ⟶ load ⟶ Gener ator: 4 hops

4. Generator ⟶ Calvert ⟶ SS4 ⟶ Minor ⟶ load ⟶ Generator: 5 hops

5. Generator ⟶ Calvert ⟶ SS4 ⟶ Annapolis ⟶ loa d ⟶ Generator: 5 hops

Therefore, the Kirchhoff network in Figure 13.6a is aperi- odic, because m = 5, n = 4 are relatively prime. But the Kirchhoff network of Figure 13.6b is periodic, because removal of cycle #3 leaves four cycles, each of length equal to 5 hops, and m = 5, n = 5 are non-prime relative to each other. This means instability in Figure 13.6b may not die out.

Kirchhoff Stability: A Kirchhoff network is stable if it is aperiodic. Departures from Kirchhoff ’s Law eventu- ally die out and the network self-synchronizes.

This illustrates the subtle complexities inherent in even the simplest power grid. Operators at each node (substation, power plant) may inadvertently destabilize a Kirchhoff net- work by attempting to balance ACE. Only a global under- standing of the network’s topology can overcome this error.

13.5 anatomy oF a BlacKout

According to Massoud Amin, both size and frequency of Grid outages—called brownouts and blackouts—are on the increase. They occur for a number of reasons, but in hindsight they are typically normal accidents. They start out as relatively insignificant faults or errors that spread like a contagion to adjacent power lines, substations, and power plants. Consequences grow as the outage sweeps across part, or all, of the Grid. The 2003 Blackout is a classic example of a normal accident.

The infamous Eastern Interconnection blackout of August 14, 2003, cut off power to millions of Americans and Canadians in eight states and one province. lasting 2 days, the blackout shed 12% of NErC capacity. The economic costs—based on the loss of electricity sales to consumers—range from US$7 to US$10 billion. The insurance industry lost US$3 billion. Compare this with the damages and cleanup costs of TMI-2 (Three Mile Island Nuclear Power Plant #2) that melted down in 1979. TMI-2 incurred damages of US$973 million—one-tenth the damage done by the Blackout of 2003.

According to the Final Report of the US–Canada Power System Outage Task Force, “The initiating events of the blackout involved two critical utilities—First Energy (FE) and American Electric Power (AEP)—and their respective reliability coordinators, MISO and PJM” [6]. See Figure 13.7. AEP (American Electric Power, Inc.) is an area within MISO. AEP of Columbus, Ohio, owns and operates more than 42,000 MW of generating capacity in the United States and in some international markets.7 It is one of the largest electric utilities in the country, with almost 5 million customers linked to its 11-state electricity transmission and distribution network.

7http://www.aep.com

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

ANATOMY OF A BlACKOUT 247

FiGurE 13.6 The Grid must obey Kirchhoff’s law by adjusting inflows to equal outflows at all points in the network. A dropped link can destabilize the Grid so that it is impossible to make ACE equal to zero, as this hypothetical grid illustrates. (a) This hypothetical grid forms an aperiodic network that self-stabilizes. (b) This damaged grid forms a periodic network that is inherently unstable.

N = Possum PWR

L = 0

L = 1

L = 2

L = 8

L = 10 L = 12

L = 11

L = 9

L = 13

L = 3

L = 5

L = 7L = 6

D = 2

N = SS5

N = SS13

N = SS4

N = Minor

N = Annapolis N = Red bluff

N = Load

N = Calvert-PWR

N = Generators

D = 2 D = 2

D = 3

D = 4

D = 4L = 4

(a)

N = Possum PWR

L = 0

L = 1

L = 2

L = 8

L = 10 L = 12

L = 11

L = 9

L = 13

L = 3

L = 4

L = 7L = 6

S = 0.00 S = 0.00

S = 0.00

N = SS5

N = SS13

(b)

N = SS4

N = Minor

N = Annapolis N = Red bluff

N = Load

N = Calvert-PWR

N = Generators

S = 0.00 S = 0.00

S = 0.00

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

248 ElECTrIC POWEr

FirstEnergy Corp. of Akron, Ohio, is the fourth largest investor-owned electric power network in the United States. Its seven electric utility operating companies serve 4.3 million customers within 92,400 km2 of Ohio, Pennsylvania, and New Jersey.8 It also provides natural gas service to approxi- mately 150,000 customers in the Midwest.

13.5.1 what happened on august 14th, 2003

The following sequence of events are broken down in stages so you can follow what happened and how cascade failures start out small and insignificant and grow,9 eventually over- whelming the entire grid. The sequence of events leading to the outage is documented in greater detail in the Interim and Final reports produced by the U.S.–Canada Power System Outage Task Force.

• FirstEnergy’s control-room alarm system was not working, which meant operators did not know trans-

mission lines had gone down, did not take any action to keep the problem from spreading, and did not alert anyone else in a timely fashion.

• MISO’s tools for analyzing the system were also malfunctioning, and its reliability coordinators were using outdated data for monitoring—all of which kept MISO from noticing what was happening with FirstEnergy in time to avert the cascading.

• MISO and PJM Interconnection, the neighboring reliability coordinator, had no procedures to coordinate their reactions to transmission problems.

The colossal collapse started out small with an error in control software and tripping of an obscure power line in Ohio. Then the cascade unraveled in the following phases:

Phase i: Power degradation 12:15 EDT: MISO SCADA/EMS state estimator software

has high error—it is turned off.

13:31 EDT: Eastlake Unit #5 generation tripped in Ohio.

14:02 EDT: Stuart-Atlanta 345 kV line tripped in Ohio due to contact with a tree.

FiGurE 13.7 The 2003 Blackout started with reports from FirstEnergy (red), which is part of the AEP (green) wheel at the epicenter of the blackout.

8http://www.firstenergycorp.com 9“Final report on the August 14, 2003 Blackout in the United States and Canada: Causes and recommendations”, April 2004. U.S.-Canada Power System Outage Task Force, pp. 12, (dated January 12, 2004) at https:// reports.energy.gov.

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

THrEAT ANAlYSIS 249

Phase ii: computer Failure 14:14: FE SCADA/EMS alarm software fails.

14:20: FE SCADA rTUs fail.

14:27: Star-South Canton 345 kV line tripped.

14:32: AEP called FE regarding Star-South Canton line.

14:41: FE transfers software applications to backup computer.

14:54: FE backup computer failed.

Phase iii: cascade line Failures Begin 15:05: Harding-Chamberlin 345 kV line overheats, shorts

with tree.

15:31: MISO called PJM to confirm Stuart-Atlanta line was out.

15:32: Hanna-Juniper 345 kV overheated, sags, and shorts out.

15:35: AEP unaware of Hanna-Juniper failure.

15:36: MISO unaware of Hanna-Juniper failure.

15:41: Star-South Canton tripped, closed, tripped again, unknown to AEP & PJM.

Phase iV: cascading collapses transmission 15:39–15:58: Seven 138 kV lines trip.

15:59: loss of the West Akron bus causes five more 138 kV lines to trip.

16:00–16:08: four more 138 kV lines trip; Sammis-Star 345 kV line overheats and trips.

Phases V, Vi, and Vii 16:10–16:12: Transmission lines disconnect and form

isolated islands in Northeast United States and Canada.

When it was all over, 263 of the 531 generators were shut down in the United States and Canada. The cascade that began in MISO spread to other regions: Quebec, Ontario, New England, New York, and PJM (Pennsylvania–Jersey– Maryland). The 2003 Blackout was a normal accident that started with operator errors and tripped distribution lines, and propagated to transmission lines and power plants. Eventually, 55 million people were without power.

This blackout qualified as a “1000 year flood,” because of its size, measured along a number of metrics—it covered a large geographical area, affected a large population, and had a large economic impact. However, it was typical of many smaller-consequence outages that happen every day, because of the following:

• Power lines making contact with trees and shorting

• Underestimation of generator output

• Inability of operators to visualize the entire system

• Failure to ensure operation within safe limits

• lack of coordination

• Ineffective communication

• lack of “safety nets”

• Inadequate training of operators

This handful of possible causes of blackouts ignores the potential for widespread and longer-term outages if the per- petrators are human. What if terrorists attempt to take down the Grid?

13.6 thrEat analysis

One way to identify human threats to the grid is to create “red team” scenarios by pretending to be a terrorist or criminal.10 Maj. Warren Aronson, U.S. Army, and Maj. Tom Arnold, U.S. Marine Corps, prepared the following four attack scenarios while playing the role of red team.11 They focused their attention on power plant fuel supply, transmis- sion line transformers, transmission substations including towers, SCADA, and power generators. These targets were chosen because they cost little to attack and yet they can create enormous damage or destabilize the Grid. If the red team can create an unstable grid, argued the red team, NErC rules require operators to propagate the instability across the entire control area, and perhaps create a blackout across the entire interconnection.

13.6.1 attack scenario 1: disruption of Fuel supply to Power Plants

The process of supplying electricity begins with the transpor- tation of power plant fuel by water, rail, road, or pipe to power generation plants. The largest source of North American electricity comes from coal-fired, thermal-generating plants. These plants have historically maintained a reserve supply of 60–90 days of coal near each generator complex. Gas and fuel oil–fired plants generally have little, if any, fuel on-site, because of variance in seasonal demand for coal and dependence on just-in-time inventory to reduce storage costs and environ- mental impact. This is an opportunity for an attacker.

A red team might disable, or at least significantly degrade, a major portion of regional power generation by attacking key components in the fuel supply chain. A specific example might be the Powder river Basin in Wyoming, described in the Chapter 12. Only three railroad lines serve the region, carrying 305 million tons of coal annually to generation plants in more than a dozen states. Moving the same volume of coal by truck—currently the only alternative to rail—is both prohibitively expensive and restricted by available trucks and drivers, which currently support other consumers.

The destruction of an important bridge, like the High Triple Bridge over Antelope Creek, would stop coal transport

10red teams are attackers and blue teams are defenders. 11CS 3660 projects, summer 2002.

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

250 ElECTrIC POWEr

on one of the two primary lines feeding rail hubs for distribu- tion to multiple states. Destruction of one to three similar targets immediately before peak periods of seasonal electricity demand could disable much of the country’s gen- eration capacity for periods of weeks to months. Vulnerability to this type of attack is high, say 75%, and economic cost alone would exceed perhaps US$2000 million.

13.6.2 attack scenario 2: destruction of major transformers

Transformers are the key links between generation and transmission substations as well as between transmission and distribution subsystems. Most transformers are mechani- cally simple, consisting of wound copper coils encased in tanks of oil. The oil cools the coils to prevent the high-voltage current from melting the copper, breaking the wire, and opening the transformer circuit. Step-up transformers ser- vicing generation plants are very large and heavy, with some weighing hundreds of tons. Their size makes movement from the manufacturer to installation locations slow and expensive. Some transformers are made only in foreign countries and take months to replace. As a consequence, utility owners/operators and manufacturers do not maintain a large inventory. Step-down transformers can be equally difficult to replace and also represent choke points between transmission and distribution networks.

A devastating attack against step-up and step-down trans- formers is relatively simple. A single person can accomplish outright destruction quickly and inexpensively by planting explosives or driving a vehicle or material handling equip- ment into the side of a transformer. An even easier attack may be possible without entering the facility; puncturing the side of a transformer with a weapon like a rifle would cause coolant oil to leak resulting in overheating before the attack is detected. Although heat sensors might shut down the transformer before fatal overheating, the loss of oil would temporarily stop the flow of electricity while the substation was shut down and transformer isolated and repaired. The consequences of a major transformer outage including economic consequences could exceed US$100 million, and the probability of success is rather high, say 95%.

13.6.3 attack scenario 3: disruption of scada communications

SCADA OCCs (Operation Control Centers) provide constant monitoring and adjustment to all subsystems of the electrical power system. Electric utility companies recognize the importance of these sites and have taken measures to protect them from physical attack. They are normally well protected and located in hardened structures, often behind layered security or below ground. Attackers could be insiders or mil- itants that launch a direct assault on the facilities. However,

recruiting existing employees sympathetic to the attacker’s cause or placing a team member in a trusted position in such a facility requires total faith in that individual and may take considerable time. In addition, direct assault against a facility requires information on facility configuration, extended surveillance to discover security procedures, well-trained assault forces, overt action, and relative strength favoring the attacker, which is not typical of an asymmetric attack.

The weakest points in a control system are usually the communication networks themselves, rather than the OCC facilities. Although communication links normally have some form of redundancy, they are still susceptible to attack. Some components of the communication system will likely be exposed to observation and thus vulnerable to physical attack—for example, telephone wires strung on poles and externally mounted antennas. More sophisticated terrorists might use directed energy weapons or other forms of electronic warfare to damage SCADA without entering buildings.

Cyber exploits could target the published protocols used by SCADA systems, much like Stuxnet targeted the Siemens control system protocol. regardless of the method chosen, the goal of these attacks is to both seize control of a power system and cause operations to occur outside safe operating parameters, destabilizing the ACE or disrupting recovery efforts following other events. The likelihood of this type of attack is comparatively low, say 10%, but the consequences could be comparatively high, say US$1000 million.

13.6.4 attack scenario 4: creation of a cascading transmission Failure

In accordance with NErC rules, generators and switching circuits are designed to automatically go off-line when they operate outside safe operating ranges. Control circuits usu- ally make automatic adjustments before the system exceeds these limits, but fail-safe devices will shut down generators and wheeling in the absence of external commands. When multiple failures occur nearly simultaneously, it is possible that the cumulative effect is an artificially induced normal accident. The attacker’s strategy is to induce a cascade by carefully chosen targets.

Here is how the normal accident might unfold. By design, if a major transmission line trips or substation fails, current surges and voltage transients trip circuit breakers. In reaction to the circuit breakers being tripped, the protective circuits at the affected generation plants shut down the turbines to prevent them from over-speeding due to the loss of load. As the load increases on the remaining generators, the rotation of the generator turbines decreases. As the turbine genera- tors slow down, the power company must start to shut off some customers or bring more power online. If more power is not added or the load decreases too quickly, other genera- tors will also start to shut down. This cascade will continue until the entire Grid is stopped.

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

rISK ANAlYSIS 251

The attacker must have enough expertise to know which substations and transmission lines to attack, simultaneously. Therefore, the probability of success is low, say 5%, and the consequences are uncertain, say US$500 million in equip- ment and economic loss.

13.7 risK analysis

The forgoing red team scenarios are incorporated into the hypothetical grid of Figure 13.6a and input into the MBrA fault tree risk analysis tool as shown in Figure 13.8. For sim- plicity, assume all threats are 100% and all elimination costs

are US$100 million. Therefore, the only differences among threat-asset pairs are consequences and vulnerabilities as estimated by the red team.

Figure 13.9 shows the results of return on Investment (rOI) analysis. risk declines much faster than vulnerability because of the Or-gate logic of the fault tree. An investment of US$200 million nearly eliminates risk but lowers vulnerability to approximately 33%. The reason for the low vulnerability reduction rOI is traced to the high cost of protecting transformers. Transformer rOI is less than US$1.00/$ invested. Therefore, transformer vulnera- bility remains high no matter how much of the limited budget is invested in protection of transformers.

Grid fault Transmission

Generator

SCADA

Transformers

Substations

OROR

FiGurE 13.8 MBrA fault tree analysis of red team threats invests most in protecting scenario “Attack scenario 1: Disruption of fuel supply to power plants.”

100% Grid: Risk, vulnerability vs. Investment

90%

80%

70%

60%

50%

40%

30%

20%

R is

k, v

ul ne

ra bi

li ty

10%

0% $0 $50 $100 $150 $200 $250 $300 $350 $400

Risk

Vulnerability

Investment $

FiGurE 13.9 Threat analysis risk declines faster than vulnerability because of the Or-gate logic of Figure 13.8.

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

252 ElECTrIC POWEr

13.8 analysis oF wEcc

The U.S. power grid is vulnerable in the middle where trans- mission and distribution take place. The importance of sub- stations and transmission lines was underscored in 1996 when the Western Interconnect—also known as WECC— was disrupted by a single failure in a transmission line connecting Oregon and California. This small fault spread throughout the 11 Western states pulling down the entire grid in a spectacular demonstration of normal accident theory.

Barabási describes this spectacular failure in terms of network science:

On a day of record temperatures, at 15:42:37 on August 10, 1996, the Allison-Keeler line in Oregon expanded and sagged close to a tree. There was a huge flash and the 1,300-megawatt line went dead. Because electricity cannot be stored, this enormous amount of power had to be suddenly shifted to neighboring lines. The shift took place automatically, funneling the current over to lower- voltage lines of 115 and 232 kilovolts, east of the Cascade Mountains. These power lines were not designed, however, to carry this excess power for an extended time. loaded up to 115% of their thermal ratings, they too failed. A relay broke down in the 115-kilovolt line, and the excess current overheated the overloaded ross-lexington line, causing it too to drop into a tree. From this moment things could only keep deteriorating. Thirteen generators at the McNary Dam malfunctioned, causing power and voltage oscillations, effectively separating the North-South Pacific Intertie near the California-Oregon border. This shattered the Western Interconnected Network into isolated pieces, creating a blackout in eleven U.S. states and two Canadian provinces. [8]

The 1996 power outage in the 11 Western states and Canada was prophetic. A similar failure on a relatively minor portion of the Eastern Grid led to massive outages in 2003. Knowledge of the 1996 outage did not prevent the 2003 blackout. This is because power engineers and politicians lack a thorough understanding of complexity theory as it applies to grid networks. Nonetheless, a number of com- plexity theory researchers identified the problem—complex CIKr network cascades are magnified by self-organized topology. In the case of the Grid, self-organization is found in networks with high betweenness and high degree. The more connections a substation has, and the more paths passing through a substation or transformer, the more likely are disastrous cascade failures.

Power Grid Resilience: High values of spectral radius and betweenness in the network formed by substa- tions, transmission lines, and interconnections decrease network resilience. To make the Grid more resilient spectral radius and betweenness must be minimized.

The Western Interconnect shown in Figure 13.10 illustrates this principle. A network model of the WECC grid contains 181 nodes and 232 links, so node and link robustness are relatively high (spectral radius is 3.46):

N by simulation

= −( )

= − = − −( )

232 181

232 22

1 1 3 46 71 77

/ . % %

Indeed, this network is comparatively resilient against cas- cades, in theory. Over (0.22)(232) = 51 links have to be removed to separate the network into disjoint components, and (0.77)(181) = 139 nodes can be removed—one at a time— without separating the network by single-node depercolation. This means that (0.23)(181) = 41 blocking nodes form the critical nodes necessary to halt cascading. Figure 13.11 shows the results of using the blocking node algorithm to identify which nodes to harden. Without these nodes, the western power grid would not work, and with them, catastrophic cascade failures are possible. Therefore, they must be pro- tected from failure and block cascading normal accidents at the same time.

Furthermore, the fundamental resiliency equation, obtained by assuming random single node failures, indicates a high critical vulnerability. The WECC network transitions from low- to high risk at critical vulnerability point 0.54:

log . . ; %q( ) = − =0 91 0 54 540γρ γ

With such a low spectral radius and apparent high critical point, we would expect this network to be extremely resilient against cascade failures. However, it contains nodes and links with high combinations of connectivity (degree) and between- ness (4027 paths run through its hub). Connectivity measured by node degree promotes cascading, and congestion mea- sured by betweenness promotes vulnerability. Together, these metrics signal potential weaknesses—hot spots—along paths between generator and load. The hot spots—areas in the net- work, which are prone to failure because of high normalized degree times betweenness—are indicated with darkly shaded and numbered squares in Figure 13.10.

This analysis generally agrees with historical data indi- cating congestion zones, shown in Figure 13.10 as rectangular roadblocks. Square hot spots and rectangular congestion zones fall on major transmission paths running North and South between Arizona and Washington. During the warm season, power flows from the North to the South; during the cold months, flow reverses—from the South to the North. los Angeles depends on Washington in the summer months and Seattle depends on the Palo Verdes nuclear power plant in Arizona in the winter. Hot spots also lie on paths connect- ing power sources such as the dams at Bonneville to highly populated areas such as Seattle and los Angeles.

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

ANAlYSIS OF WECC 253

PRINCE RUPERT PEACE CANYON

SUNDANCE

MICA

LANGDON

CANADA

UNITED STATES

SEATTLE AREA

FT. PECK SPRINGS

COLSTRIP

CHIEF JOSEPH

VANCOUVER AREA

PORTLAND AREA

BUCKLEY HELLS

BOISE BURNS

MALIN

SHASTA

ROUND MTN

TABLE MTN

SAN FRANCISCO AREA

SALT LAKE CITY AREA

DENVER AREA

PINTO

ALBUQUERQUE AREA

EL PASO AREA

MEXICO

DEVERS

MOJAVE

1 8

4 6

45 6 18

20 16

24 32

30 36

46 22

48 51

HOO

PHOEN

NAVAJO

UGO

LOS ANGE

AREA

LANGDON

CANYON

FiGurE 13.10 The Western power grid contains a number of congestion zones as indicated by rectangular shapes and a number of high betweenness and degree hot spots as indicated by large (darkly shaded) numbered squares [7].

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

254 ElECTrIC POWEr

The largest nuclear power plant is in southern Arizona and the largest hydroelectric generator in the United States is located on the Columbia river separating Washington and Oregon. High betweenness links connect these large power sources to large population centers—Phoenix, Denver, los Angeles, Seattle, and points in between. If we want this Grid to be more resilient, the hot spots in between generation and load must be eliminated. The way to do this is to rewire the WECC grid such that node degree and betweenness are reduced.

13.9 analysis

Theoretically, a large power grid can shift power from one end of the country to the other because it can be extremely adaptable to changing demand and localized faults. Even if the largest dozen or so centralized power plants fail, power can theoretically be transferred from somewhere else. The largest dozen power plants supply less that 5% of national power demand, and there is a 15% surplus at any moment in time. In addition, different regions of the grid reach peak load at different times, so when demand peaks in one part

of the Grid, the demand can be satisfied by a demand valley in another part.

So, the larger the Grid, the more adaptable it is—theoret- ically. But in practice, the Grid is too complex to guarantee isolation of faults (versus cascading), and vertical integration over the past century has led to regional interoperability problems. Simply put, it is still impossible for the Grid to adapt to demand on a national scale because there is insuffi- cient capacity in the transmission and distribution network. To make matters worse, the SCADA/EMS systems are not sophisticated enough to properly automate the regulation of ACE. While there is no shortage of power, there is a shortage of distribution capability, SCADA sophistication, and trained operators.

The 1992 EPACT was aimed at decoupling the layers of the old vertical monopolies, but at the present time, this has increased, rather than decreased, the vulnerability of the grid. In addition to economic vulnerability (the “gaming of prices” by predator utility brokers like Enron), the network is vul- nerable to technical vulnerabilities (SCADA software errors, complex interdependencies that are not fully understood). EPACT has deregulated the generation and load components, but left the middle component on its own in a world that

Canada

Washington Oregon

Idaho

Montana

Colorado

Nevada

Arizona

Mexico

California

N. California

Langdon0

Langdon1

Sundance

Vancouver

V5 V1 B0

PC2 PR2

PR3

Round Mountain

Mid point Table Mountain

HS2

C5 C6

D1W1

HS3

FP1

D7 D6SL2

SH1

SF1

San Francisco

Coyote HO2

HO1

Phoenix

AZ3

Mex2Mex0

SD0

LA0

Reno

N32

S34

FiGurE 13.11 The Western power grid’s blocking nodes hold the grid together and also have the potential to stop cascade failures if hardened.

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

ExErCISES 255

views the transmission and distribution component “someone else’s problem.” There is no money to be made from the middle. Thus, economic forces are working against the pro- tection of the most vulnerable part of the Grid.

The Grid may simply be too big and complex to fully control. In fact, the grid may not be entirely necessary. In 1902, there were 50,000 independent, isolated power- generating plants in the United States and only 3,624 central power plants [9]. The grid started out as a decen- tralized, distributed generation network. Immediately after World War I, the price of coal soared and urbanization favored centralization of generation. Technology advanced rapidly during the 1920s, which lowered the cost of building centralized plants. In addition, their owners drove the inde- pendents out of the market by lowering monthly bills to consumers and diversifying applications to make up for the loss of revenue during nonpeak periods of the day. The vertically integrated and centralized power companies sold their nonpeak power to electrified train systems (sub- ways and commuter trains), factories, and large building owners to raise elevators in tall buildings. Thus, central- ized generation won out, and today we have a Grid with a high degree of SOC.

But the Grid does not have to remain the way it is today. If it was redesigned and regulatory legislation was to favor distributed generation (wind, solar, and fuel cell generators at factories, shopping malls, neighborhoods), the Grid would be made almost invincible because it would truly be adapt- able. In distributed generation systems, most of the time most of the power comes from only a few yards away. Solar generators do not produce during the night and wind power does not produce during periods of calm weather, so the Grid might still be needed. But it would be needed less of the time, and when it fails, the local generation facility would provide enough power to keep critical services like hospitals operating.

In addition, large storage cells located close to metropol- itan centers would further alleviate the burden on transmis- sion lines. During off-peak periods, generators could use the transmission lines to charge up batteries, flywheels, or reservoirs. During peak demand periods, power could be drawn from local storage rather than power plants located hundreds of miles away. Inadequate transmission and distri- bution capacity would become less critical.

This leaves SCADA/EMS as the vulnerability of great- est concern. And the cyber threat to power is real. The SQlSlammer worm penetrated a private computer network at Ohio’s Davis-Besse nuclear power plant in January 2003. It disabled a safety monitoring system for nearly 5 h and shut down a critical control network after moving from a corpo- rate network, through a remote computer onto the local area network that was connected to the control center network. SQlSlammer could have affected critical control systems at Davis-Besse. As it turned out, the affected systems were

used to monitor, not control the reactor. The safety of Davis- Besse was not jeopardized.

By 2005, more than 60 cyber security events impacted power control systems, including three nuclear plants.12 This number is likely to grow as the Internet becomes intertwined with non-Internet control networks. Unfortunately, SCADA/ EMS components—computers, networks, and software— will remain complex and unreliable for a long time because securing an information system is well known to be prob- lematic. Thus far, it has been impossible to build software that is guaranteed to be bug-free. These software flaws lead to networks becoming disconnected, data being lost, and computers being disabled. As long as software is flawed, there will be faults in industrial control systems such as SCADA and EMS. And, as long as software is designed and written by humans, it will be flawed.

13.10 ExErcisEs

1. Why are there high and low voltage lines? a. Cities need more power than farms b. Electrons travel farther on high voltage c. Electricity travels more efficiently at high voltage d. Electrons travel faster at high voltage e. Electricity has lower resistance at low voltage

2. AC won over DC because AC: a. Works better in radios and TVs b. Is an international standard c. Operates at 60 cycles/s, which is compatible with clocks d. Can be transmitted at high voltages e. Can be switched like the Internet

3. Before it was called FErC, it was called: a. FPC b. CIA c. NErC d. MISO e. NCS

4. The PUHCA of 1935 established federal regulatory con- trol over power because: a. It was the right thing to do b. rural areas needed power, too c. The Great Depression was in full effect d. Congress wanted to establish power over power e. Interstate commerce allows the federal government

to regulate sales

5. NErC and load sharing through wheeling was established soon after: a. Enactment of the Federal Power Act of 1920 b. The Northeast Blackout of 1965 c. Soon after the problem of synchronization was solved

in 1970s

12A personal communication with Joe Weiss.

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

256 ElECTrIC POWEr

d. Enactment of PUrPA in 1978 e. Soon after deregulation in 1992

6. Electrical power was deregulated by enactment of: a. EPACT 1992 b. PUrPA in 1992 c. Establishment of ISOs in 1992 d. Tragedy of the Commons Act of 1992 e. ISOs

7. ISOs were authorized by FErC to: a. Monitor the operators b. look for abuses by participants c. run power exchange markets d. Maintain their independence e. All of the above

8. The Electricity Sector ISAC (ES-ISAC) is: a. The same as EISAC b. run by NErC c. run by FErC d. run by ISO e. run by the DHS

9. Which one of the following is NOT a power grid within NErC? a. ErCOT b. Western Interconnect c. Quebec Interconnect d. Midwestern Interconnect e. Eastern Interconnect

10. Which of the following is NOT a component of the U.S. electrical power grid? a. Dams b. Power Plants c. Transmission d. Distribution networks e. SCADA

11. Which of the following makes the electrical power grid particularly vulnerable? a. Most power comes from a few central power plants b. Coal fuel supplies in the United States depend on

critical railroad links c. large transformers never break d. SmartGrid technology e. Hydroelectric dams are vulnerable

12. In the United States, power outages have been: a. Constant b. Increasing c. Decreasing d. Smaller than in other countries e. None of the above

13. The most asymmetric attack on power generation would be: a. Bombing of Grand Coolee dam b. Attacking a nuclear power plant

c. Coordinated attack on substations using fault trees d. Cyber attack on SCADA systems that control

power e. Bombing of Hoover Dam on the Colorado river

14. Critical transmission paths are defined by: a. Towers carrying high voltage power b. Interstate tie lines c. local distribution network transformers d. Transmission lines supplying power to major areas

such as Chicago e. High degree and betweenness hot spots

15. large transformers are considered critical, because they are: a. Difficult to transport from manufacturing to

installation b. Easy to destroy c. Cause power outages d. Expensive e. All of the above

16. Why does ACE deviate from zero? a. The load is constantly changing b. Generators generate unpredictable output c. SCADA/EMS software often fails d. The weather is constantly changing e. The Grid is too big and complicated to understand

17. Why is the Grid vulnerable in the middle? a. There is insufficient transmission and distribution

capacity b. Transformers are critical and unprotected c. Substations are unreliable d. Everything depends on generators e. Fuel is in short supply

18. Deregulation under EPACT 1992 allowed: a. Utilities to make more money b. Competing utilities to use the transmission

infrastructure c. Increased build out of more transmission lines d. reduced blackouts e. Higher penalties for letting ACE deviate from zero

19. A Kirchhoff grid can be destabilized if it forms a: a. Periodic network b. Aperiodic network c. Scale-free network d. Clustered network e. Separated network

20. Distributed generation solves the problem of: a. renewable energy sources b. Underutilized transmission c. Transmission capacity d. NIMBY e. regulation

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .

rEFErENCES 257

rEFErEncEs

[1] lewis, T. G. Bak’s Sand Pile, Monterey: AgilePress, 2011.

[2] Amin, M. Toward a More Secure, Strong and Smart Electric Power Grid, IEEE Smart Grid, January 2011. Available at http://smartgrid.ieee.org/january-2011/67-toward-a-more- secure-strong-and-smart-electric-power-grid. Accessed June 20, 2014.

[3] Overbye, T. reengineering the Electric Grid, American Scientist, 88, 3, May-June 2000, pp. 220.

[4] U.S.-Canada Power System Outage Task Force. Interim report: Causes of the August 14th Blackout in the United States and Canada, November 2003, pp. 134.

[5] Kinney, r., Crucitti, P., Albert, r., and latora, V. Modeling Cascading Failures in the North American Power Grid, European Physical Journal B, 46, 1, 2005, pp. 101–107.

[6] U.S.-Canada Power System Outage Task Force. Final report on the August 14, 2003, Blackout in the United States and Canada: Causes and recommendations, April 2004, pp. 12.

[7] US Department of Energy. National Electric Transmission Congestion Study, August 2006, pp. 32.

[8] Barabási, A.-l. Linked: How Everything is Connected to Everything Else and What it Means for Business, Science, and Everyday Life, New York: PlUME, 2003, pp. 119.

[9] Friedlander, A. Power and Light: Electricity in the U.S. Energy Infrastructure 1870–1940, reston: Corporation for National research Initiatives, 1996, pp. 51.

C o p yr

ig h t ©

2 0 1 4 . Jo

h n W

ile y

& S

o n s,

I n co

rp o ra

te d . A

ll ri g h ts

r e se

rv e d .