Disaster Recovery Tabletop Exercise Plan.

All organizations experience unexpected and unwanted disruptions to their day-to-day operations. Too often organizations view an IT emergency as something solely handled by their IT unit. However as more and more of the University’s mission requires a working IT infrastructure, it becomes increasingly important to look at the broader impact of an IT systems or infrastructure disaster on Brandeis’ operations.

Fortunately while it is impossible to predict when and what sort of emergency will occur, it is possible to prepare in advance. Only by regularly practicing responding to a simulated disaster can an organization gain confidence that when a real incident occurs, it’ll be prepared to respond.

A tabletop exercise is a review of the processes and procedures that would generally be used during a real crisis. The goal of this exercise is to detect issues that may interfere with response and recovery during an actual emergency.

The scope of this exercise should be strictly limited to online education and specifically the impact of the Latte system being unavailable. Do not spend time discussing how to recover from additional systems that would (in a real event) also be disabled by the simulated incident. Aspects of the exercise are necessarily contrived – some suspension of belief is always required.

Note to all participants and facilitator: due to the compressed nature of the ‘incident time’ vs. actual time, it will be necessary to treat incident time in an elastic fashion. Once the exercise begins the facilitator will start the ‘incident clock’ and we will attempt to work to the degree possible in real-time. However the facilitator should feel free to move forward in incident time if necessary to push the discussion forward.

The primary objective of this exercise is to explore many of the issues that will arise during an IT disaster scenario, some technical, some mission related. This is the first step to the creation of a rigorous disaster recovery plan and thus to provide Brandeis with the capabilities to respond and recover effectively. We want to identify gaps and establish best practices that should be addressed when creating a disaster recovery plan. Although this is a timed event, our goal is not to race to some arbitrary point of resolution.

· Exercise teamwork: focus on relationship and team building

· Provide us tools for crisis response, and a forum for discussing and developing emergency plans

· Test assumptions

· Enhance Brandeis emergency resiliency

The participants in this exercise will be separated into two teams, one operations and one leadership. The operations team should focus on returning impacted services to availability. The leadership team will be discussing questions related to general emergency response (such as the availability of an emergency operations center) and addressing questions related to policy or resources beyond the capacity or authority of the operations team. Both rooms will have phones in them, though participants are free to communicate with others as desired and within the constraints of the scenario.

This will be a facilitated tabletop exercise (TTX). Players will participate in the following three distinct modules:

· Module 1: Incident + Initial Response

· Module 2: Secondary Impact

· Module 3: Tertiary Impact

Each module begins with an update that summarizes the key events occurring within a specific time period. Following the updates, participants review the situation and engage in a plenary group discussion of appropriate response issues.

Questions have been included after each module to stimulate discussion and the flow of information around departmental procedures and encourage interdepartmental collaboration.

Each exercise participant will receive this Exercise Plan (ExPlan), which provides a written scenario and situation updates. Following each module is a series of questions that highlight pertinent issues for consideration. These questions are supplied as catalysts for the group discussions; participants are not required to answer every question, nor are they limited to those topics. Participants are encouraged to use this ExPlan as a reference throughout the exercise.

Although you may look ahead in this plan, it is important to address only the current and prior events in each module. You may not move forward or discuss items that have not yet occurred.

This is a time to discuss the specific actions you will—or be assigned—to undertake. Always consider how long each action might take. Take whatever time is necessary to discuss your process, procedures and protocol.

The follow ground rules will apply to this exercise:

· The exercise setting is the ideal opportunity to consider different approaches and suggest improvements to current resources, plans, and training.

· Responses should be based on current capabilities.

· Fight the problems, not the scenario.

· Respect the speaker.

· Start on time, end on time, and use the timers.

· Look through the windshield and not the rear view mirror.

· Enough, Let’s Move On (E.L.M.O.) will be used to keep the group moving forward and avoid becoming entrenched in the minutiae

· There are no “hidden agendas” or trick questions intended to mislead participants.

· All participants will receive the same information at the same time.

Incident Background

Incident

Monday, November 2nd 2015 at 3:00am

Event 1: 3am November 2nd 2015

At 3am a disgruntled ex-employee entered Feldberg – he were terminated on October 30th and his card access had not yet be terminated so he was able to enter the building and all LTS communications rooms and data centers.

Once in the building he took a crow bar and smashes the CISCO ACE 30 load balancer impacting Moodle services and then he pulled the alarm bar and turned off building power (by pressing the circuit disconnect in room 104A).

Event 2: 3:15am November 2nd

Brandeis University police arrive and seeing the smashed equipment quickly disable the alarm and declare the data center a crime scene. The police do not allow anyone to touch the core power switch for the building until a fingerprint expert arrives and tests the switch for fingerprints.

Event 3: 5am November 2nd

After hiding in the Library for the last couple of hours, the ex-employee made his way to the Goldfarb data center and physically removes the CISCO ACE 30 in this data center. This load balancer is also crushed and left on the floor in pieces.

Current Situation

Anyone who feels they would have already been engaged in the incident should summarize what they believe their actions would have been.

Inject 1, 9am: The LTS Helpdesk opens to a queue of 100 messages from students reporting that they are unable to log into Latte. 30 similar messages are from faculty who have early morning classes and are unable to access Latte.

Group

1. In an actual incident, what would have taken place by the time of the exercise kick-off?

2. Based on the information presented, what are your top priorities at this time?

University Services

1. What processes or procedures would you implement in response to the situation presented? What procedures are in place to access the environmental hazard from the liquid in Goldfarb?

1. Who would you look to coordinate your response?

2. Who or when would you engage the University’s leadership?

Library and Technology

1. What alarms or monitoring would have been triggered by the incident as described?

2. What plans, policies, and/or procedures are in place to prevent or respond to a large-scale service interruption?

3. What information sources could you contact to get further information about this service interruption?

4. Due to the information presented, would there be any immediate operational changes in your department? Would this involve a change in security protocol, either physical or logical?

Academic Units

1. How would you expect to first hear about the incident?

2. What procedures or communications might you undertake once learning about the incident?

Communications

1. When would you expect to be notified?

2. Is this protocol discussed in the Brandeis Crisis Communications Plan? Has this plan been provided to communications liaisons university-wide? Are they aware of the protocol?

Public Safety

1. What coordination among departments is necessary at this point?

2. Due to the information presented, would there be any immediate operational changes in your department? Would this involve a change in security protocol, either physical or logical?