Instruction for Review 2

International Journal of Disaster Risk Reduction 48 (2020) 101617

Available online 21 April 2020 2212-4209/© 2020 Elsevier Ltd. All rights reserved.

Building preparedness in response to active shooter incidents: Results of focus group interviews

Runhe Zhu a, Gale M. Lucas b, Burcin Becerik-Gerber a, *, Erroll G. Southers c

a Sonny Astani Department of Civil and Environmental Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA, USA b USC Institute for Creative Technologies, University of Southern California, Los Angeles, CA, USA c Sol Price School of Public Policy, University of Southern California, Los Angeles, CA, USA

A R T I C L E I N F O

Keywords: Active shooter incidents Focus group interviews Building preparedness Countermeasures Evaluation

A B S T R A C T

Active shooter incidents present an increasing threat to the American society. Many of these incidents occur in building environments, therefore, it is important to consider design and security elements in buildings to decrease the risk of active shooter incidents. This study aims to assess current security countermeasures and identify varying considerations associated with implementing these countermeasures. Fifteen participants, with expertise and experience in a diverse collection of operational and organizational backgrounds, including se- curity, engineering, law enforcement, emergency management and policy making, participated in three focus group interviews. The participants identified a list of countermeasures that have been used for active shooter incidents. Important determinants for the effectiveness of countermeasures include their influence on occupants’ behavior during active shooter incidents, and occupants’ and administrators’ awareness of how to use them effectively. The nature of incidents (e.g., internal vs. external threats), building type (e.g., office buildings vs. school buildings), and occupants (e.g., students of different ages) were also recognized to affect the selection of appropriate countermeasures. The nexus between emergency preparedness and normal operations, and the importance of tradeoffs such as the ones between cost, aesthetics, maintenance needs and the influence on oc- cupants’ daily activities were also discussed. To ensure the effectiveness of countermeasures and improve safety, the participants highlighted the importance of both training and practice, for occupants and administrators (e.g., first responder teams). The interview results suggested that further study of the relationship between security countermeasures and occupants’ and administrators’ responses, as well as efficient training approaches are needed.

1. Introduction

A wide spectrum of emergencies, either man-made or natural, could occur in our built environment, causing economic damages and jeop- ardizing human safety. Compared with other types of emergencies, active shooter incidents present a critical homeland security threat to public safety and human life in the U.S. According to the definition in the Federal Bureau of Investigation (FBI) report, an active shooter is “an individual actively engaged in killing or attempting to kill people in a confined and populated area, typically through the use of firearms.” [1]. A total of 277 active shooter incidents, according to the FBI, have occurred in the U.S. between 2000 and 2018, resulting in 884 deaths and 1546 injuries. Among these casualties, 25% of the deaths and 47% of the injuries occurred in 2017 and 2018 alone [2], which shows an increasing threat

of active shooter incidents in the recent years. Among all the active shooter incidents occurred in the U.S. between

2000 and 2018, only 14% took place in open spaces, while the remaining incidents happened in various built environments, including commercial, educational, and government buildings [2]. The fact that active shooters frequently target buildings clearly demonstrates the necessity of implementing security countermeasures to protect buildings and occupants. It has been shown that buildings and human behavior are two correlating factors in emergency situations [3–5]. How human behavior is influenced by various building attributes, such as signage [6], corridors [7], exits [8], elevators [9], and visual access [10], have been examined in different types of emergencies (e.g., fires, earth- quakes, and unspecified emergencies [3,4,11]). However, compared with other types of emergencies, active shooter incidents have several

* Corresponding author. E-mail address: [email protected] (B. Becerik-Gerber).

Contents lists available at ScienceDirect

International Journal of Disaster Risk Reduction

journal homepage: http://www.elsevier.com/locate/ijdrr

https://doi.org/10.1016/j.ijdrr.2020.101617 Received 13 October 2019; Received in revised form 17 February 2020; Accepted 18 April 2020

International Journal of Disaster Risk Reduction 48 (2020) 101617

2

distinct characteristics (e.g., shorter incident duration, presence of ad- versaries and weapons, etc. [12]), hence whether findings from other emergency contexts could be applied to active shooter incidents is un- certain. Additionally, how certain countermeasures implemented in buildings, such as access control and safe rooms, would affect human behavior and building preparedness remains underexplored. With this motivation, the present study aims to assess the effectiveness of different security countermeasures, and how they would impact human behavior, safety, and building preparedness in response to active shooter in- cidents. A qualitative approach (i.e., focus group interview) was adop- ted, owing to its advantage in providing an in-depth exploration of a topic in which many factors are still unknown [13].

2. Background

To mitigate the risks that active shooter incidents impose on build- ings and occupants, several public agencies have published guidelines and recommendations on preparedness, response, and management related to active shooter incidents. For example, the Interagency Secu- rity Committee released a document for planning and responding to an active shooter incident and highlighted the importance of preparedness (e.g., establishing threat assessment teams) and training (e.g., con- ducting drills) [14]. The FBI recommended that while there is no ab- solute best response strategy during active shooter incidents, maintaining a “run, hide, fight” mindset can increase occupant safety [15]. Another active shooter/hostile event guide compiled by the Interagency Board also underlined that an Incident Command System (ICS) is important to foster incident management and coordination when multiple agencies are involved and suggested a bottom-up approach to build ICS [16].

Research efforts have also been made to improve building pre- paredness and investigate the relationship between buildings and oc- cupants during active shooter incidents. Kuligowski presented guidance on the creation and dissemination of emergency information in both audible and visual means in response to active shooter incidents [17]. Based on the results of agent-based simulations, Cho et al. showed that with the presence of human-sensing technology and building informa- tion, the efficiency of safe evacuation can be significantly improved during active shooter incidents [18], and Lee et al. demonstrated that alert systems in public buildings, as well as quick responses of occupants and first responders are helpful to decrease the casualty rate of active shooter incidents [19]. Considering the relationship between building layouts and occupant responses, Kellom and Nubani found that the simplicity of building layouts and the ease of wayfinding could facilitate occupants to execute the “run” and “hide” action [20]. Similarly, providing guidance and safe rooms was found effective for evacuation during active shooter incidents [21]. That being said, it was also pointed out that certain building environments could inhibit occupant responses during active shooter incidents. For instance, the openness of university buildings may compromise occupants’ ability to hide [22]. Public address systems and mass notifications (e.g., text and email messages) were suggested to be deployed in classrooms, dormitories and outdoor environments, in order to address active shooter incidents in university campuses [23,24]. Fox and Savage also stressed that the effectiveness of countermeasures may be different on university campuses as compared with high schools [25]. Furthermore, Addington discussed various countermeasures implemented in response to active shooter incidents, including limiting access, locking or monitoring doors, and employing security officers, and stressed that further investigations are still needed to assess the effectiveness of these countermeasures [26].

Apart from the above-mentioned efforts, several documents have been developed for the purpose of guiding building design in prepara- tion for active shooter incidents and other types of attacks (e.g., explo- sive blasts, chemical, biological, and radiological attacks). These documents include Primer to Design Safe School Projects in Case of Terrorist Attacks [27], Primer for Design of Commercial Buildings to Mitigate

Terrorist Attacks [28], Reference Manual to Mitigate Potential Terrorist Attacks Against Buildings [29], and Minimum Antiterrorism Standards for Buildings [30]. In these guidelines, lists of countermeasures have been proposed for protecting buildings and occupants against various attacks. Nevertheless, these countermeasures are frequently aimed at deterring attacks and mitigating the effect of attacks on buildings and occupants, whereas when an attack (e.g., active shooter incident) occurs, the effectiveness of these countermeasures, particularly how they influence the actions of occupants and administrators, is yet to be investigated.

The balance of the paper is organized as follows: Section 3 describes the methodology of the study, including participant recruitment, inter- view guide and procedure, and data analysis. The findings from the focus group interviews are presented in Section 4. We further discuss implications of our findings in Section 5 and conclude the paper in Section 6.

3. Methodology

As mentioned in Section 1, focus group interviews were conducted in this study. A focus group interview refers to a discussion, in which an interviewer asks a set of open-ended questions to a small group of target population [31]. The name is derived from the fact that the selected groups are “focused” on a given topic [32]. Therefore, the selection criteria for participants in focus group interviews include: having domain knowledge on the topic, being within the age-range, and comfortable talking to the interviewer and other participants [33]. Compared to individual interviews, the main advantage of focus group interviews lies in that the group could foster a synergy, which results in more than the sum of each individual’s output [32,34,35]. In order to achieve the objective of this study, a series of online focus group in- terviews were conducted, which could accommodate geographically distributed populations in a synchronous virtual shared space [36]. This study was approved by the University Park Institutional Review Board (UPIRB) of University of Southern California. Institutional Review Board is the main administrative body in the U.S. that protects the rights of human subjects participating in research studies. More details of the focus group interviews are provided in the following subsections.

3.1. Participants

The participants of the focus group interviews were recruited through professional organizations and societies that focus on building safety and emergency management, as well as experts from leading engineering and design firms. Purposive sampling was used to select participants with expertise in the two key areas, namely building design/engineering and building security [37,38]. To be included in the study, participants must (1) be between 18 and 85 years old, (2) be fluent in English speaking, (3) be able to hear and talk remotely, and (4) have expertise in relevant areas, such as terrorism/active shooter in- cidents, school/office building design, security engineering in buildings, and risk and emergency management. A minimum of 4 participants in each group is generally accepted [39]. Krueger and Casey suggested between six and eight participants in a group, but at the same time stated that a smaller group might be preferable, which would allow partici- pants to contribute more freely [40]. Thus, a total of 15 participants were recruited in this study, and three focus group interviews were conducted, with each group consisting of 5 participants. The partici- pants’ professions were security engineer (2), architect (1), emergency and security manager (2), security consultant (5), and police officer (5). All of the participants had professional experience and expertise closely related to active shooter incidents. Specifically, 8 participants have been involved in the law enforcement, management and training for active shooter incidents, and the other 7 participants mainly focus on building security, including electronics security, anti-terrorism design, and crime prevention through environmental design. The total number of the participants (i.e., 15) and focus groups (i.e., 3) in this study was

R. Zhu et al.

International Journal of Disaster Risk Reduction 48 (2020) 101617

3

determined by analyzing the interview results until no more themes emerged from the focus group interviews [40].

3.2. Interview guide

A semi-structured interview guide was developed to direct the focus group interviews. Following the suggestion in Ref. [32], the guide consists of four parts. The first part covers the beginning of the inter- view, including welcome messages, explanation of the interview pur- pose and use of data, obtainment of participants’ verbal consent to participate in the study and be recorded, and self-introduction of the participants. In the second part, the interviewer asks the participants a warm-up question: “What are the countermeasures in your experience that have been used to proof buildings against active shooter incidents?” The purpose of this question is to motivate the participants to share their perspectives based on their work experience and get them involved in the discussion. Subsequently, the third part contains a series of questions to further provoke participants’ thoughts on the topic. The purposes of these questions are to create discussions around the effectiveness and the pros and cons of current countermeasures for active shooter in- cidents. The general list of questions is illustrated in Table 1.

It is important to note that while the questions listed in Table 1 acted as guidance for the interviews, they might not be asked exactly as they are presented in Table 1. Depending on the progress of the interview, some questions might be asked in another way or some questions might be raised and discussed among the participants before being asked by the interviewer. Finally, the last part of the interview includes summa- rizing the discussion and thanking the participants for their contribution to the study.

3.3. Procedure

Emails were used to send invitations to potential participants, in which the purpose of the study was mentioned. After the participants agreed to contribute to this study, the authors sent a Doodle (an online calendar tool for time management and coordination of meetings) link for the participants to provide their available timeslots [41]. Based on the participants’ availability, they were assigned to different groups and received BlueJeans (a cloud-based video conferencing service) in- vitations [42]. The interviews were video and audio-recorded using the built-in function of BlueJeans, and each interview lasted between one and one-and-a-half hours. Before the interview, the authors also sent the UPIRB-approved consent form to the participants via email, which presented the purpose of this study, procedures, and confidentiality (i.e., the use of audio and video dada). The interviews were conducted by two

of the authors: one acted as a moderator and the other acted as an observer [40]. The moderator led the discussion, while the observer was responsible for taking notes and did not actively participate in the dis- cussion. Prior to the start of discussions, the moderator introduced the interview purpose and obtained participants’ verbal consent to partici- pate and be recorded during the interview, as approved by the UPIRB. The moderator then initiated the discussion around the topic and asked questions based on the interview guide. Upon completion of the dis- cussion, the moderator concluded the interview and asked if the par- ticipants had any additional thoughts or considerations. Finally, the participants were thanked and dismissed from the interview.

3.4. Data analysis

Upon completion of the focus group sessions, the interviews were reviewed, and the conversations were transcribed verbatim. The tran- scription process was based on the audio data, while the video data was used merely to verify who was speaking in the focus group discussion. The authors reviewed the transcripts repeatedly to get familiar with the data and obtain a sense of interviews as a whole, which served as the preparation phase for analyzing the data [43]. Based on the transcript, the authors used qualitative analysis to interpret the interview data [44]. The purpose of the data analysis was to identify agreements and controversies over the topic, instead of just presenting numbers and percentages of participants’ responses [32,45]. Three major stages were involved in the analysis. First, the text in the transcripts were divided into meaning units, which could be individual word or some words in a sentence or several sentences that share similar content (i.e., the open coding stage [46]). The meaning units were labeled with different codes that could represent the content of the text. For instance, “One of the most impactful things that we have been able to do and justify is reducing the amount of entrances into offices.” was labeled as “access control.” Second, these generated codes were compared with each other and those that were related in their content were grouped into subthemes, which is denoted as axial coding in the literature [46]. For instance, “access control” generated in the last stage was categorized as “countermeasures for building design and facility management.” Finally, more abstract high-level themes were created to organize the subthemes in a hierar- chical structure (i.e., the selective coding stage [46]). For instance, the subtheme “countermeasures for building design and facility manage- ment” created in the last stage was organized under the theme “coun- termeasures to protect buildings and occupants from active shooter incidents.” One of the authors conducted the transcribing, coding and data analysis. The results, together with the transcript and audio/video data were reviewed by all of the authors to verify that the outcome was accurate and reliable. However, as only one author actually coded the transcripts, no quantitative measures of agreement (e.g., interrater reliability) were included. The data analysis pipeline is shown in Fig. 1.

4. Findings OF FOCUS group interviews

The interviews were focused on evaluating the effectiveness of countermeasures to safeguard buildings from active shooter incidents, accompanied by various considerations when implementing these de- fense strategies. Through the analysis of the interviews, four themes emerged, which are summarized along with their subthemes in Fig. 2. The themes are presented in the following subsections, and participants’ quotes during the interviews are used for illustrative purposes. It is important to point out that the findings are based on the participants’ perspectives and should not be considered as factually defined.

4.1. Countermeasures to protect buildings and occupants from active shooter incidents

With regard to protecting buildings and occupants from active shooter incidents, one of the most commonly mentioned

Table 1 General list of questions in the focus group interviews.

Questions

1. What are the countermeasures in your experience that have been used to proof buildings against active shooter incidents?

2. How does the effectiveness of countermeasures change when the shooter is an internal threat versus an external threat?

3. Do you see any difference in terms of these countermeasures’ implementation in different types of buildings, for example, in school buildings vs. office buildings? If yes, how do different building types differ in terms of countermeasures?

4. Do you have a sense of which countermeasure might affect building occupant behavior during an active shooter incident? If so, which measures might change occupant behavior the most?

5. Do you have a sense of the effectiveness of which countermeasures might be affected by occupant behavior during an active shooter incident? If so, the effectiveness of which countermeasure might be changed by occupant behavior the most?

6. Are there differences or conflicts between countermeasures that try to prevent risk from other emergencies (e.g., fires, earthquakes) in contrast to the countermeasures that are used for active shooter incidents?

7. What are the impacts of the countermeasures for active shooter incidents on day-to- day activities?

R. Zhu et al.

International Journal of Disaster Risk Reduction 48 (2020) 101617

4

countermeasures by the participants was access control, such as reducing the amount of entrances, posting someone physically at the reception/security desk, having proximity access control badges and walk-through metal detectors. An illustrative claim was: “If I am a bad guy, that [access control] is going to keep me from coming into the building. [If there is strong access control], I might set up [the attack] outside the building, …but I think that access control, in whatever it looks like for that facility, is the first step.” Some participants pointed out that many active shooters target a soft target (i.e., a person or thing that is relatively unprotected or vulnerable) rather than a hard target (i.e., a highly defended target), thus the presence of access control could be a deterrent by reducing the shooter’s expectations: “They [shooters] are not going to waste their time on trying to gain entry into an area, where they will have a much more difficult time reaching that goal.” Nevertheless, maintaining egress during active shooter incidents was also identified as an impor- tant factor, as an example statement being: “There is definitely a balance between public access and security.”

Having multiple layers of security was another identified counter- measure. The primary justification for this security design, as stated by one of the participants, is that: “You cannot ensure that every single inch is not going to have any person that could cause harm with any weapons.” Hence, how to draw the lines for different layers of security and build different zones is essential, as mentioned during the interviews. A

participant illustrated the use of this countermeasure: having walk through metal detectors at the main entrance of buildings, and then anti- tailgating device and inner hardened lobby doors. In this way, it can be detected if someone comes in with a weapon and the inner bulletproof doors can be locked, so that occupants inside could have time to react, as the first responder teams respond. Nevertheless, concerns about imple- menting this countermeasure were voiced during the interviews. A participant stated that in a campus environment, if a shooter breaches the first line (e.g., the entrance to the campus) and starts to attack where students are, whether or not to lock the doors of buildings where more students are would be a difficult decision. This further relates to another countermeasure that was discussed: remote locking control mecha- nisms, which could secure a building or a section of a building by acti- vating a switch. A participant mentioned that “It [remote locking control mechanism] is generally for an imminent threat … once the threat [a shooter] is inside, the last thing you want to do is to lock the building, because it locks everybody [all the occupants] in.” However, a participant commented that “When people are under stress, they might not necessarily make rational decisions and therefore might not be able to make the right call, which might make things worse,” referring to an accidental locking. Therefore, training for the administrators who make these decisions was also identified as the crucial component for employing the countermeasures correctly.

Fig. 1. The data analysis pipeline.

Fig. 2. Themes and subthemes of the focus group interviews.

R. Zhu et al.

International Journal of Disaster Risk Reduction 48 (2020) 101617

5

Another frequently mentioned countermeasure addressing the issue of communication was mass notification, which could be conveyed through social media, text message, and email notifications, etc. An illustrative comment was: “In a campus environment or an area where you may not hear the shots being fired in one location, that [notification] is going to be very important if someone is on the run and has not heard the gun shots.” Apart from that, some other countermeasures were mentioned during the interviews, including using ballistic-resistant materials, video surveillance, safe rooms, etc.

While the topic was mainly focused on countermeasures related to building design and facility management, the participants highlighted the importance of training and regular drills. It was mentioned that many shooters tend to surveil the location and study their target in advance, thus the countermeasures in place cannot eliminate attacks completely. Under this circumstance, the participants suggested that training for building occupants is a decisive factor. One example state- ment was: “The fact that you have thought it [how to respond] through once is the first step, and the fact you have actually done it [practice appropriate responses] once is the second step to knowing how you should respond in an actual active shooter incident.” Similar principles were also mentioned for law enforcement and first responder teams. Illustrative statements include: “We let them [police agencies] conduct training in our office buildings. Therefore, when something [an active shooter incident] does happen, it is not new to them and they have proper access to the buildings and all [response procedures] is pre-established.” and “I always think that we could be a bit blas�e about the fact that somebody is going to make the call, and then we do not necessarily prepare them to make that call.” Establishing working relationships and communications among the diverse first responder agencies was suggested as well. One participant argued that: “I think the leadership of municipal agencies and campus agencies is very important. They have to break down barriers and build avenues by which the officers can become familiar with the geography of the campus.”

4.2. Interactions between countermeasures and human behavior

In conjunction with identifying countermeasures for deterring active shooter incidents and improving security, how these countermeasures impact the behavior of occupants and first responder teams and vice versa were acknowledged as important issues during the interviews.

First, from the perspective of occupant response, while mass notifi- cation was commonly mentioned as a countermeasure, concerns about its application were mentioned as well. Participants underlined that the level of information conveyed by mass notification needs to be very clear, and it should be communicated in such a way that occupants know how to respond to the information. Otherwise, as claimed by a partici- pant, “Mass notification could be a detriment if appropriate response is not conveyed when occupants receive the notification, because then that [not conveying appropriate information] just becomes chaos.” Additionally, participants mentioned that access control could have a negative effect as well, because it sometimes acts as a chokepoint, which could lead to large concentrations of people, therefore causing delay of evacuation and creating a target of opportunity for shooters. It was also mentioned that not having an area with concealment has benefits for active shooter incidents as well as general safety. However, other participants expressed concerns about eliminating concealment or hiding places: “Eliminating hiding places has a flipside as well, as the run-hide-fight program requires having the ability to hide, so I think there should be a balance.” Another countermeasure that fell into this category was related to line of sight. A participant commented that having open space could enable first responder teams to find and stop the shooter more easily, but at the same time, shooters would have line of sight as well to facilitate their attack. In contrast, the effectiveness of certain countermeasures is also contingent upon occupant behavior. For mass notification, it was sug- gested that occupants’ training and drills play significant roles. An example statement was: “Occupants need to know how to respond appro- priately to the advice that is being given [in mass notification], and they need

to know which exits or evacuation routes, or which zone of the building they should be seeking refuge in.” Meanwhile, the participants stated that oc- cupants’ natural responses during active shooter incidents consist of seeking concealment, getting behind an object or curling down under the desk and hide. Thus, the use of ballistic-resistant materials was identified as being effective in this situation.

Second, from the perspective of first responder teams, one counter- measure identified that could impact first responder teams was access control. A participant commented that: “If you create a fortress-style building and a bad guy [a shooter] gets in, it would prevent or delay us [first responder teams] from getting in, as we have to call the fire department to provide an access to the building.” The effect of remote locking control mechanism, as mentioned in the last subsection, is also up to the au- thorities to use at their discretion in response to active shooter incidents. Along this line, some participants highlighted countermeasures should be somewhat tolerant to potential errors in their usage, as participants stated that: “Even if somebody [an administrator] does not follow up on something [a certain procedure], such as forgetting to turn on a switch or reviewing the camera footage, it [a countermeasure] still offers some level of safeguard.” and “It is important to achieve the balance between providing the capability [for administrators to take actions during active shooter incidents] and giving them the fewest number of options to make sure they make the right decisions.”

4.3. Contextual influences on countermeasure implementations

The first context that was recognized to influence the implementa- tion of countermeasures was whether it is an internal or an external threat. Some participants argued that if it is an internal threat, many of the countermeasures could be immediately circumvented, as shooters may not be subject to the same access control and they would have fa- miliarity with the environment, including the location of occupants and rooms, and so on. Some countermeasures, however, were identified as being immune to internal threats. An example was the remote locking control mechanism. A participant commented that: “It [remote locking control mechanism] does not matter who you are, if you are outside the door that is locked, you are not going to be able to get in.” Another one that was mentioned was maintaining multiple egress routes, for which the inherited assumption was that: “The shooter can only be in one place at a time, so if there are multiple egress routes, they would provide evacuation opportunities for occupants, no matter who the shooter is.” Moreover, the countermeasures for handling internal threats were discussed. It was pointed out that there are software applications that manage electronic security systems, which could be used to detect gunshots and deactivate certain badges in certain sections of the building. Nevertheless, chal- lenges are associated with this countermeasure as well, as a participant commented that: “[To implement this countermeasure], you have to think about relocating occupants out of the incident, as well as where the incident is occurring in relationship to other sections of the building.” Similarly, to deal with internal threat, another countermeasure mentioned was separating the main building with less secured areas, such as food delivery places, and employing monitoring and alerting systems at these locations.

Building types were acknowledged as an influencing factor in countermeasure implementations. It was stated that compared with school buildings, many office buildings consist of a lobby or atrium areas. Thus, the participants mentioned that minimizing any “high ground” vantage points (e.g., views from a mezzanine down to an atrium where events are held) was a common countermeasure applied to office buildings. Moreover, it was pointed out that: “I think in office buildings, … it is culturally more acceptable to have security desks for checking in and turnstiles and access control for entering the building.” Regarding school buildings, it was mentioned that high school design goes more towards open and flexible environments, instead of being in the traditional classroom style. Thus, a participant suggested that in this scenario: “Those barriers [used in high schools] might be clear or translucent, but they should still afford deterrence to active shooter incidents.” In addition, it was

R. Zhu et al.

International Journal of Disaster Risk Reduction 48 (2020) 101617

6

suggested that certain buildings, such as houses of worship or very old buildings, might not even have countermeasures implemented (e.g., access control), which imposes greater security risks.

During the interviews, the participants emphasized the significance of occupant characteristics. For example, one participant explained: “I know that lower grade students have to follow the instructions of their teachers, but high school students, probably middle school students as well, are at an appropriate age for the run-hide-fight program.” Another partic- ipant analogized lower grade students to patients in hospitals, because they must be supervised to a certain extent. In addition, for the occu- pants that have limited mobility, such as those sitting in a wheelchair, the participants suggested the use of safe rooms, so that: “Somebody who is disabled could get into the safe room, lock it from the inside and be safe inside there, because running might not be a feasible option.” Aside from occupants’ physical characteristics, the training that occupants receive also has an impact. The participants mentioned that different schools, for example, have their own training protocols and emphasis. Some may only conduct fire drills, some only teach students “shelter in place”, and some focus more on the “run-hide-fight” program. The different training that students receive, as the participants suggested, could largely impact occupant behavior and the use of countermeasures.

4.4. Preparedness for different types of emergencies and normal operations

The participants identified that the cost associated with imple- menting countermeasures as an important consideration. Whether it is employing security staff, using electronic security system, or adding video surveillance, it was acknowledged that there is always going to be additional costs. One participant argued that: “The question that should be asked is: have the client, the school district or whoever owns the facility, done the analysis to see whether the costs outweigh the risk?” However, another participant stated that “The problem with these very low likelihood high consequence events is that nobody ever really thinks it will happen to them to a degree, or that the risk reward ratio does not necessarily make sense in their minds.” Along this line, a similar issue suggested by the participants was the cost associated with applying certain countermeasures during an active shooter incident. For example, locking the entire building down could lead to loss of revenue for a company, thus people might be afraid of using certain countermeasures. It was further mentioned, “There needs to be a culture of not being afraid to make the decision [of using a counter- measure].” On the other hand, while countermeasures cause additional costs, one participant mentioned that parents are more likely to send their children to the schools with appropriate security systems, which indicates that implementing countermeasures could be beneficial in the economical aspect as well.

Besides the identified costs, other factors related to preparedness for active shooter incidents and normal operation were aesthetics and people’s psychological perception of the building design. One partici- pant opined that buildings should at least appear to be welcoming and inviting instead of looking like fortresses, and thus a balance between openness and security should be taken into consideration. Moreover, people’s psychological reactions to the building design were mentioned as well. An illustrative example was: “You might create an environment where people do not feel safe to come to work, if there is a big sign that says: if an active shooter comes, hide here.”

Another operational factor that should be considered together with countermeasures was recognized as occupants’ daily activities. One particular example mentioned by the participants was that: “In our office building, we need to use badges to get in, however, everyone holds the door open for anyone behind them, and this is a huge security risk.” Moreover, it was pointed out that occupants might attempt to compromise certain countermeasures for the convenience of their daily activities (e.g., propping a door open because it is too heavy). In this regard, a partic- ipant highlighted that: “It is rarely any one thing that results in something going very badly wrong, it is usually a sequence of events, which might have

unintended consequences.” Given that multiple types of emergencies can occur in buildings, the

approaches to ensure occupant safety are different. Thus, the interde- pendency of preparedness for different emergencies was discussed during the interviews. For example, one participant claimed that: “Fire alarms are good for fires, but they are not necessarily good for responding to an active shooter. Because fire alarms would have occupants evacuate the building, going to a safe refuge area, which could be a place where a lot of people gather, which could make it easier for an active shooter [to attack occupants].” Other participants further illustrated this challenge by pointing out that even if there are two separate security systems, shooters may still have a way to set off the fire alarm and have every- body come out of their rooms. The complexity would further increase if there were multiple types of emergencies (e.g., active shooter incidents, fires, earthquakes). A participant exemplified that: “When the alarm goes off, I need to detect which type of alarms it is, and then I need to work out how I should be respond[ing] to each type of the alarms”. Apart from occupant behavior, countermeasures for different emergencies could also influ- ence the building itself, as an example statement being: “Ballistic resis- tant material is very heavy. In smaller buildings, adding that weight to a building could impact how the structure performs under an earthquake.” That being said, the participants also mentioned that there are similar- ities among different emergencies. For example, knowing the location of building exits was considered important across different emergency situations.

5. Discussions

In this study, we conducted focus group interviews to develop an understanding of current countermeasures being used to proof buildings from active shooter incidents and protect building occupants. The countermeasures mentioned during the interviews were primarily consistent with the prevalent recommendations published by public agencies and research focus in prior studies [22–26,29], which indicates that results of the focus group interviews reflect the state-of-the-art of this area and have practical implications.

5.1. Influence of countermeasures on occupant behavior during active shooter incidents

The behavioral consequences (e.g., influence on human behavior) of certain countermeasures were part of the main discussion during the interviews. In contrast with other building emergencies such as fires, the duration of active shooter incidents is typically very short. Between 2000 and 2013, 69.8% of the active shooter incidents ended within 5 min [1]. Thus, before the arrival of first responder teams, occupants need to largely rely on themselves to ensure their safety [4], hence more emphasis should be placed on the influence of countermeasures on occupant behavior. In particular, occupant behavior during building emergencies could be categorized into three phases: perception of environmental cues, decision-making and performing an action [47]. How current countermeasures influence occupant behavior in these phases, based on the interview results, is illustrated in Fig. 3.

As shown in Fig. 3, it was noted that mass notification might impact occupants’ perception of environmental cues and decision-making during active shooter incidents either positively or negatively, based on the information conveyed in the notification. This is in alignment with previous research, which found that during emergency situations, providing accurate and timely notifications would motivate occupants to take appropriate actions, whereas ambiguous information might result in the opposite [24,48,49]. Meanwhile, participants mentioned that in other emergency situations (e.g., fires and earthquakes), many occupants tend to ignore the notification or alarms and continue pre-event behavior, because they may be perceived as false alarms. This phenomenon is also reflected in the literature [50]. However, given the different nature of emergency situations, whether occupants would still

R. Zhu et al.

International Journal of Disaster Risk Reduction 48 (2020) 101617

7

ignore notifications/alarms during active shooter incidents is yet to be answered. Similarly, while mass notification provides audio cues, visual access offers visual cues and could also influence occupants’ perception and decision-making [51]. Compared with other emergencies (e.g., fires) where visual access mainly affects occupants’ evacuation process [10], during active shooter incidents, visual access also influences the behavior of first responder teams and shooters. Other countermeasures that affect occupants’ decision-making include access control, elimi- nating hiding places, etc., as these countermeasures could change available options for occupants to respond. For example, access control was acknowledged as a potentially negative factor because it limits the number of evacuation routes and hence generates chokepoints at the main exit. Such effect was also shown in prior studies that studied building emergency evacuations [52,53]. Therefore, it was underlined that maintaining emergency evacuation exits at the same time could alleviate the negative effects of access control. A significant factor, however, is that people would usually evacuate through the exits that they are familiar with, such as the main exits [4]. Hence even though providing emergency exits could add additional evacuation capacity, whether it will be sufficiently utilized remains uncertain and should be assessed. Other countermeasures were considered to affect occupants’ action phase by influencing their performance when carrying out a certain action (e.g., the presence of multiple evacuation routes). As discussed above, countermeasures affect occupant behavior from different perspectives (perceptual versus physical influences). There- fore, to assess the correlation between occupant responses and coun- termeasures, a mixed approach should be used. For those countermeasures that have physical impact (e.g., restricting the flow rates), simulation-based methods could be used [52,54]; and those countermeasures that influence occupant’s decision-making process, human-subjects experiments could be more effective [55].

5.2. Training and drills for occupants and first responder teams

While the initial aim of this study was to assess building-related countermeasures, training and drills were the top influencing factors that emerged during the interviews. It was emphasized that counter- measures cannot perfectly deter active shooter incidents from happening or protect occupants entirely. Thus, enhancing occupants’ and first responder teams’ preparedness to use the countermeasures and respond to the incidents appropriately is a decisive factor, which has also been widely suggested in the literature [56–58]. However, the participants also mentioned that in many locations across the U.S.,

building occupants, especially students, are improperly trained. While it was mentioned that some high schools have started to teach their stu- dents, faculty and staff the run-hide-fight program, the participants pointed out that more students only practice fire drills and are not aware of the response procedures recommended for active shooter incidents. It was suggested that the training on responses, such as the run-hide-fight program, should trickle down to middle and primary schools as well, but could be in different forms. For example, it was mentioned that for lower grade students, the information conveyed during their training could be: “There might be a bad person outside, and we want to hide from him or her,” instead of: “There may be somebody that is trying to kill us.” Moreover, other forms of training, such as serious games that have been used in other emergencies [59,60], could be considered as well. More recently, virtual reality (VR) technology has been used to prepare occupants, teachers and administrators for active shooter incidents [61]. More research could be done to test the effectiveness of VR training in this area. Apart from occupants’ usage of countermeasures during active shooter incidents, their behavior during daily life also impacts the effectiveness of these countermeasures. For example, occupants may (un)intentionally get around certain countermeasures (e.g., propping security doors open for their convenience), which compromises the function of countermeasures. Thus, it is also necessary to train occupants to increase their awareness of how to use countermeasures properly in their daily life. In addition, training was found to be as important an issue for first responder teams and building administrators, including both their responses to active shooter incidents as well as coordination and communication among different agencies, which have also been reported in prior studies [23,62]. Gaining familiarity with the building was identified to be a crucial element for the training of first responder teams. While the first responder teams may not have the accessibility to on-site training in every building in a region or district, it was mentioned that leveraging technologies, such as creating databases of building in- formation on mobile devices that first responder teams can access would be helpful for their operation.

5.3. Practical considerations for implementing countermeasures

While the primary goal of implementing countermeasures is to improve building security and occupant safety in active shooter in- cidents, their influence on daily operations were also stressed. Typical practical factors that impact the implementation of countermeasures include: (1) cost efficiency (e.g., cost of implementing certain counter- measures and the influence on the revenue of a company), (2) building

Fig. 3. Influence of countermeasures on occupant behavior during active shooter incidents.

R. Zhu et al.

International Journal of Disaster Risk Reduction 48 (2020) 101617

8

aesthetics and attractiveness, and (3) risk level of different types of at- tacks (e.g., external versus internal threats). Meanwhile, it was mentioned that providing necessary information to occupants regarding the purpose of countermeasures is very important, which highlights the importance of educating occupants via an appropriate approach when implementing a countermeasure. A similar strategy has also been sug- gested in the literature [63]. In addition, different activities are corre- lated with building types (e.g., commercial activities in office buildings and educational activities in school buildings). Previous studies have illustrated that occupant responses are correlated with building func- tions as well as activities conducted in buildings [3,25,64,65], hence a balance should be achieved between safety during active shooter in- cidents and building functions.

5.4. Limitations and implications for future research

While this study presented interesting findings on the effectiveness of security countermeasures and building preparedness for active shooter incidents, there still exist limitations that could be studied in future research. First, the findings of this study, as with any other focus group studies, were based on the participants’ pronouncements rather than defined facts, hence further studies are needed to justify some of the results. In particular, one main finding from this study was the influence of countermeasures on occupants, building administrators and first responder teams, which is directly related to human behavior. There- fore, more empirical behavioral data could be collected from human- subjects experiments to verify the findings. Second, while we included participants with expertise in building security, emergency manage- ment, and active shooter incidents in the focus group interviews, some topics emerged during the interviews, such as cost efficiency of coun- termeasures and building aesthetics, could be further explored from the perspectives of other roles, such as building owners and occupants. Thus, further studies that involve participants in more roles could be helpful.

6. Conclusions

In response to the risk of active shooter incidents, a variety of se- curity countermeasures have been proposed and used in buildings. While these countermeasures are intended to proof buildings against active shooter incidents, challenges still exist in their implementation. In this study, we conducted three focus group interviews with fifteen participants who have expertise in the area of building security and active shooter incidents. We found that some of the countermeasures (e. g., mass notification, access control) can be a double-edged sword, especially when it comes to their influence on the behavior of occupants and first responder teams. Moreover, it was suggested that training and drills on how to respond and how to use the countermeasures during active shooter incidents are of critical importance. Further research in these areas is necessary. Practical factors that affect the use of coun- termeasures were also discussed, including cost efficiency and occu- pants’ psychological perception of the building environment. We also revealed that the use of countermeasures largely depends on the nature of the building (e.g., different building types, different risk levels) and occupant characteristics (e.g., students of different ages). Therefore, there is no universal solution that works for the whole spectrum of en- vironments, and thorough investigations are needed to develop coun- termeasures for active shooter incidents that cater to different environments.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This material is based upon the work funded by the National Science Foundation under Grant No. 1826443. Any opinions, findings, or rec- ommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. The advice and contributions of Arup Inc., Facebook Inc., DLR Group, Building Owners and Managers Association (BOMA), American Society for Industrial Security (ASIS), and Los Angeles Police Department (LAPD) are greatly appreciated.

Appendix A. Supplementary data

Supplementary data to this article can be found online at https://doi. org/10.1016/j.ijdrr.2020.101617.

References

[1] J.P. Blair, K.W. Schweit, A Study of Active Shooter Incidents in the United States between 2000 and 2013, 2014. Washington, DC.

[2] Federal Bureau of Investigation, Active Shooter Incidents in the United States from 2000-2018, 2018.

[3] J. Lin, R. Zhu, N. Li, B. Becerik-gerber, How occupants respond to building emergencies: a systematic review of behavioral characteristics and behavioral theories, Saf. Sci. 122 (2020) 104540, https://doi.org/10.1016/j. ssci.2019.104540.

[4] M. Kobes, I. Helsloot, B. de Vries, J.G. Post, Building safety and human behaviour in fire: a literature review, Fire Saf. J. 45 (2010) 1–11, https://doi.org/10.1016/j. firesaf.2009.08.005.

[5] E. Ronchi, D. Nilsson, Fire evacuation in high-rise buildings: a review of human behaviour and modelling research, Fire Sci. Rev. 2 (2013) 7, https://doi.org/ 10.1186/2193-0414-2-7.

[6] C.H. Tang, W.T. Wu, C.Y. Lin, Using virtual reality to determine how emergency signs facilitate way-finding, Appl. Ergon. 40 (2009) 722–730, https://doi.org/ 10.1016/j.apergo.2008.06.009.

[7] E. Vilar, F. Rebelo, P. Noriega, J. Teles, C. Mayhorn, The influence of environmental features on route selection in an emergency situation, Appl. Ergon. 44 (2013) 618–627, https://doi.org/10.1016/j.apergo.2012.12.002.

[8] M.T. Kinateder, B. Comunale, W.H. Warren, Exit choice in an emergency evacuation scenario is influenced by exit familiarity and neighbor behavior, Saf. Sci. 106 (2018) 170–175, https://doi.org/10.1016/j.ssci.2018.03.015.

[9] K. Andr�ee, D. Nilsson, J. Eriksson, Evacuation experiments in a virtual reality high- rise building: exit choice and waiting time for evacuation elevators, Fire Mater. 40 (2016) 554–567, https://doi.org/10.1002/fam.2310.

[10] R. Zhu, J. Lin, B. Becerik-Gerber, N. Li, Influence of architectural visual access on emergency wayfinding: a cross-cultural study in China, United Kingdom and United States, Fire Saf. J. (2020) 102963, https://doi.org/10.1016/j. firesaf.2020.102963.

[11] G. Bernardini, E. Quagliarini, M. D’Orazio, Towards creating a combined database for earthquake pedestrians’ evacuation models, Saf. Sci. 82 (2016) 77–94, https:// doi.org/10.1016/j.ssci.2015.09.001.

[12] R. Zhu, B. Becerik-Gerber, G. Lucas, E. Southers, D.V. Pynadath, Information requirements for virtual environments to study human-building interactions during active shooter incidents. Comput. Civ, . Eng., Atlanta, GA, 2019, pp. 188–195.

[13] D. Stewart, P. Shamdasani, D. Rook, Analyzing Focus Group Data, Sage publications, 2007, https://doi.org/10.4135/9781412991841.

[14] Interagency Security Committee, Planning and Response to an Active Shooter: an Interagency Security Committee Policy and Best Practices Guide, 2015.

[15] Federal Bureau of Investigation, Responding to an Active Shooter Crisis Situation, 2016. https://www.fbi.gov/about/partnerships/office-of-partner-engagement/acti ve-shooter-resources/responding-to-an-active-shooter-crisis-situation. (Accessed 21 July 2019).

[16] InterAgency Board, Active Shooter/Hostile Event, ASHE) Guide, 2016. [17] E.D. Kuligowski, Guidance Document: Emergency Communication Strategies for

Buildings, 2014. [18] C. Cho, J.W. Park, S. Sakhakarmi, Emergency response: effect of human detection

resolution on risks during indoor mass shooting events, Saf. Sci. 114 (2019) 160–170, https://doi.org/10.1016/j.ssci.2019.01.021.

[19] J.Y. Lee, J. Eric Dietz, K. Ostrowski, Agent-based modeling for casualty rate assessment of large event active shooter incidents, Proc. - Winter Simul. Conf. (2019) 2737–2746, https://doi.org/10.1109/WSC.2018.8632535.

[20] K. Kellom, L. Nubani, One step ahead of active shooters: are our university buildings ready? Buildings 8 (2018) 173, https://doi.org/10.3390/ buildings8120173.

[21] S. Gunn, P.B. Luh, X. Lu, B. Hotaling, Optimizing guidance for an active shooter event, Proc. - IEEE Int. Conf. Robot. Autom. (2017) 4299–4304, https://doi.org/ 10.1109/ICRA.2017.7989495.

[22] D.A. Drysdale, W. Modzeleski, A.B. Simons, Campus Attacks: Targeted Violence Affecting Institutions of Higher Education, 2010.

[23] S.F. Greenberg, Active shooters on college campuses: conflicting advice, roles of the individual and first responder, and the need to maintain perspective, Disaster

R. Zhu et al.

International Journal of Disaster Risk Reduction 48 (2020) 101617

9

Med. Public Health Prep. 1 (2007) S57–S61, https://doi.org/10.1097/ dmp.0b013e318149f492.

[24] D.N. Sattler, K. Larpenteur, G. Shipley, Active shooter on campus: evaluating text and E-mail warning message effectiveness, J. Homel. Secur. Emerg. Manag. 8 (2011), https://doi.org/10.2202/1547-7355.1826.

[25] J.A. Fox, J. Savage, Mass murder goes to college: an examination of changes on college campuses following Virginia tech, Am. Behav. Sci. 52 (2009) 1465–1485, https://doi.org/10.1177/0002764209332558.

[26] L.A. Addington, Cops and cameras: public school security as a policy response to columbine, Am. Behav. Sci. 52 (2009) 1426–1446, https://doi.org/10.1177/ 0002764209332556.

[27] Federal Emergency Management Agency, Primer to Design Safe School Projects in Case of Terrorist Attacks, 2003.

[28] Federal Emergency Management Agency, Primer for Design of Commercial Buildings to Mitigate Terrorist Attacks, 2003.

[29] Federal Emergency Management Agency, Reference Manual to Mitigate Potential Terrorist Attacks against Buildings, 2011.

[30] Department of Defense, DoD Minimum Antiterrorism Standards for Buildings, 2013.

[31] S.M. Van Es, E.M. Le Coq, A.I. Brouwer, I. Mesters, A.F. Nagelkerke, V.T. Colland, Adherence-related behavior in adolescents with asthma: results from focus group interviews, J. Asthma 35 (1998) 637–646, https://doi.org/10.3109/ 02770909809048966.

[32] L.C. Lederman, Assessing educational effectiveness: the focus group interview as a technique for data collection, Commun. Educ. 39 (1990) 117–127, https://doi.org/ 10.1080/03634529009378794.

[33] C.A. Richardson, F. Rabiee, Question of access : an exploration of the factors that influence the health of young use of health care services, Health Educ. J. 60 (2001) 3–16.

[34] D.W. Stewart, P.N. Shamdasani, Focus Groups: Theory and Practice, Sage publications, 2014.

[35] A. Howard, K. Agllias, M. Bevis, T. Blakemore, “They’ll tell us when to evacuate”: the experiences and expectations of disaster-related communication in vulnerable groups, Int. J. Disaster Risk Reduct. 22 (2017) 139–146, https://doi.org/10.1016/ j.ijdrr.2017.03.002.

[36] C.A. Tuttas, Lessons learned using web conference technology for online focus group interviews, Qual. Health Res. 25 (2015) 122–133, https://doi.org/10.1177/ 1049732314549602.

[37] M.Q. Patton, Qualitative Evaluation and Research Methods, Sage publications, 1990.

[38] H.W. Asfaw, Sandy lake first nation, T.K. McGee, A.C. Christianson, evacuation preparedness and the challenges of emergency evacuation in indigenous communities in Canada: the case of sandy lake first nation, northern ontario, Int. J. Disaster Risk Reduct 34 (2019) 55–63, https://doi.org/10.1016/j. ijdrr.2018.11.005.

[39] R.W. Mcdaniel, C.A. Bach, Focus groups: a data-gathering strategy for nursing research, Nurs. Sci. Q. 7 (1994) 4–5, https://doi.org/10.1177/ 089431849400700103.

[40] R.A. Krueger, M.A. Casey, Focus Groups: a Practical Guide for Applied Research, Sage publications, 2014.

[41] Doodle Doodle, accessed, https://doodle.com, 2019. (Accessed 25 July 2019). [42] BlueJeans Network, BlueJeans, accessed, https://www.bluejeans.com, 2018.

(Accessed 25 July 2019). [43] S. Elo, H. Kyng€as, The qualitative content analysis process, J. Adv. Nurs. 62 (2008)

107–115, https://doi.org/10.1111/j.1365-2648.2007.04569.x. [44] A.J. Onwuegbuzie, W.B. Dickinson, N.L. Leech, A.G. Zoran, A qualitative

framework for collecting and analyzing data in focus group research, Int. J. Qual. Methods 8 (2009) 1–21, https://doi.org/10.1177/160940690900800301.

[45] P.S. Kidd, M.B. Parshall, Getting the focus and the group: enhancing analytical rigor in focus group research, Qual. Health Res. 10 (2000) 293–308.

[46] J. Corbin, A. Strauss, Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, Sage publications, 2014.

[47] E. Kuligowski, Predicting human behavior during fires, Fire Technol. 49 (2013) 101–120, https://doi.org/10.1007/s10694-011-0245-6.

[48] G. Proulx, Occupant behaviour and evacuation, in: Proc. 9th Int. Fire Prot. Symp., 2001, pp. 219–232.

[49] R.W. Perry, M.K. Lindell, Understanding citizen response to disasters with implications for terrorism, J. Contingencies Crisis Manag. 11 (2003) 49–60, https://doi.org/10.1111/1468-5973.1102001.

[50] G. Hofinger, R. Zinke, L. Künzer, Human factors in evacuation simulation, planning, and guidance, Transp. Res. Procedia. 2 (2014) 603–611, https://doi.org/ 10.1016/j.trpro.2014.09.101.

[51] S. Cao, X. Liu, M. Chraibi, P. Zhang, W. Song, Characteristics of pedestrian’s evacuation in a room under invisible conditions, Int. J. Disaster Risk Reduct. 41 (2019) 101295, https://doi.org/10.1016/j.ijdrr.2019.101295.

[52] V. Ha, G. Lykotrafitis, Agent-based modeling of a multi-room multi-floor building emergency evacuation, Phys. A Stat. Mech. Its Appl. 391 (2012) 2740–2751, https://doi.org/10.1016/j.physa.2011.12.034.

[53] R. Nagai, T. Nagatani, M. Isobe, T. Adachi, Effect of exit configuration on evacuation of a room without visibility, Phys. A Stat. Mech. Its Appl. 343 (2004) 712–724.

[54] V.A. Oven, N. Cakici, Modelling the evacuation of a high-rise office building in Istanbul, Fire Saf. J. 44 (2009) 1–15, https://doi.org/10.1016/j. firesaf.2008.02.005.

[55] M. Haghani, M. Sarvi, Following the crowd or avoiding it? Empirical investigation of imitative behaviour in emergency escape of human crowds, Anim. Behav. 124 (2017) 47–56, https://doi.org/10.1016/j.anbehav.2016.11.024.

[56] Department of Homeland Security, Active Shooter Preparedness, 2019 accessed, https://www.dhs.gov/cisa/active-shooter-preparedness. (Accessed 21 August 2019).

[57] J.P. O’Neill, J.J. Miller, J.R. Waters, Active Shooter Recommendations and Analysis for Risk Mitigation, 2016.

[58] E.A. Skryabina, N. Betts, G. Reedy, P. Riley, R. Amlôt, The role of emergency preparedness exercises in the response to a mass casualty terrorist incident: a mixed methods study, Int. J. Disaster Risk Reduct. (2020) 101503, https://doi.org/ 10.1016/j.ijdrr.2020.101503.

[59] M. Cha, S. Han, J. Lee, B. Choi, A virtual reality based fire training simulator integrated with fire dynamics data, Fire Saf. J. 50 (2012) 12–24, https://doi.org/ 10.1016/j.firesaf.2012.01.004.

[60] R. Lovreglio, V. Gonzalez, Z. Feng, R. Amor, M. Spearpoint, J. Thomas, M. Trotter, R. Sacks, Prototyping virtual reality serious games for building earthquake preparedness: the Auckland City Hospital case study, Adv. Eng. Inf. 38 (2018) 670–682, https://doi.org/10.1016/j.aei.2018.08.018.

[61] T. Griffith, J. Ablanedo, T. Dwyer, Leveraging a virtual environment to prepare for school shootings. Int. Conf. Virtual, Augment. Mix. Real., Springer, Cham, 2017, pp. 325–338, https://doi.org/10.1007/978-3-319-57987-0_26.

[62] J. Jones, R. Kue, P. Mitchell, S.G. Eblan, K.S. Dyer, Emergency medical services response to active shooter incidents: provider comfort level and attitudes before and after participation in a focused response training program, Prehospital Disaster Med. 29 (2014) 350–357, https://doi.org/10.1017/S1049023X14000648.

[63] M.C. Weber, S.E. Schulenberg, E.C. Lair, University employees’ preparedness for natural hazards and incidents of mass violence: an application of the extended parallel process model, Int. J. Disaster Risk Reduct. 31 (2018) 1082–1091, https:// doi.org/10.1016/j.ijdrr.2018.03.032.

[64] M. Gerges, M. Mayouf, P. Rumley, D. Moore, Human behaviour under fire situations in high-rise residential building, Int. J. Build. Pathol. Adapt. 35 (2017) 90–106, https://doi.org/10.1108/IJBPA-09-2016-0022.

[65] G. Proulx, J. Pineau, Differences in the evacuation behaviour of office and apartment building occupants, Proc. Hum. Factors Ergon. Soc. Annu. Meet. 40 (1996) 825–829, https://doi.org/10.1177/154193129604001607.

R. Zhu et al.