A Quantitative Study Assessing the Impact of Financial Sector Saf-11-85.pdf

1

CHAPTER I: INTRODUCTION

Cybersecurity failure in the private sector is a national problem and a growing federal policy

priority. Diminishing the frequency and severity of successful cybersecurity breaches has

emerged as a “national security and economic security imperative” (The White House, 2022, 2nd

paragraph). Nevertheless, US policy makers continue to struggle with which policy tools are

most effective to address this policy problem. The federal government is now embracing a

regulation heavy approach. President Biden’s 2023 National Cybersecurity Strategy states “new

authorities will be required to set regulations that can drive better cybersecurity practices at

attributes and policy tool selection to policy outcome achievement. Howlett & Fraser (2018) and

Barak - Corren & Kariv – Teitelbaum (2020), and Jing & Graham (2008) assert key policy target

behavioral attributes that align with regulation policy tools are a culture of deference to

government regulatory activities, a culture shaped by regulatory activity, and regulator / regulate

interaction during regulation development.

The financial sector is one of the most essential critical infrastructure sectors, and

cybersecurity failures in the financial sector can have great cascading societal impacts (Atkins &

Lawson, 2020). Diminishing breach frequency and severity in the financial sector is nationally

important. Current behavioral public policy theory suggests that an appropriate tool selection for

2

achieving cybersecurity policy outcomes in the financial sector are regulatory authority tools.

This is because the financial sector has key behavioral attributes as noted by Howlett & Fraser

(2018) and Barak - Corren & Kariv – Teitelbaum (2020) and (Jing & Graham, 2008). Financial

companies have a long- established culture of willingness to respond to “government invoked

penalties and proscriptions” (Howlett & Fraser, 2018, p. 111) and financial sector regulators

have often “interacted with regulates” (Howlett & Fraser, 2018, p. 5169) in the development of

regulation (Barak – Corren & Kariv – Teitelbaum, 2020). In 2000, The Gramm - Leach Bliley

Act (1999) allowed financial regulators to adopt management based regulation to compel

financial sector cybersecurity regulations to diminish breach frequency and severity.

Although cybersecurity public policy scholars have investigated the impact of regulation on

government sector breach reduction (Curti et. al, 2023) and private company proclivity to invest

in cybersecurity (Lam and Seifert, 2023) as a result of breach notification regulations, little

empirical study exists that tests the relationship of Safeguards rules on reducing actual breach

frequency and severity for private companies in the financial sector.

The purpose of this study is to quantitatively test behavioral public policy theory in the

cybersecurity policy domain by assessing if Safeguards rules have served as an effective tool for

3

achieving cybersecurity policy outcomes in the financial sector – reduction of breach severity

and frequency through implementation of administrative, technical, and physical safeguards.

For policy practitioners, the study sheds light on the effectiveness of policy instrumentation that

heavily focuses on regulation as a tool for facilitating private sector cybersecurity. The United

States government’s use of federal cybersecurity regulation is primed for growth. Empirical

study illuminating the effectiveness of cybersecurity regulation in the financial sector can help

shape the impending regulatory evolution for other industry sectors that are not currently subject

to cybersecurity regulation. When discussing the state of behavioral public policy theory,

The current behavioral public policy research offers that a regulation tool heavy cybersecurity

policy instrumentation should be effective in reducing breach severity and frequency in financial

sector companies. This work poses the question - have safeguards regulatory rules reduced data

breach frequency and severity on private sector US financial institutions from 2005 - 2020?

This study hypothesizes that these regulations have significantly helped reduce successful breach

severity and frequency during this time period. The study’s indictor variable is the presence of

Safeguards regulations. The study’s outcome variables are successful breach severity and

frequency which are measured with a mixture of numerical and dichotomous categorical

variables. Breach frequency is measured as the percentage of the total number of breaches that

4

occur on financial sector companies each year. A successful breach is characterized as a cyber

enabled intrusion into a company’s network enabling expose of consumer records. Breach

severity is measured as the number of consumer records exposed during the breach and the

sensitivity of the information exposed.

The remainder of this chapter provides necessary background context by outlining the

financial sector regulatory environment and the evolution of financial sector cybersecurity

with a discussion of the work’s philosophical approach. It subsequently enumerates the study’s

key variables, data collection procedures, and data analysis technique. In this part of the chapter,

variable operational measurement and statistical analysis procedures are explained. The chapter

concludes with a discussion of measurement reliability and study validity. Chapter IV revisits

the research question and hypothesis, reports the results of analysis, and provides commentary on

the result’s statistical and practical significance. Finally, Chapter V, discusses analysis results

and provides implications for both behavioral public policy theory and practice in the

cybersecurity policy domain.

5

delving into the literature review of theoretical and empirical works, it is helpful to provide

background context for the US financial sector cybersecurity regulatory environment. The US

financial sector does not have a single regulator. Its regulatory environment has been shaped by

“responses to various financial crises” and is composed of “multiple overlapping regulators”

(Congressional Research Office, 2020, p. Summary). The table below provides a crosswalk of

financial regulators to their covered entities. The regulator column is the regulatory agency. The

broad area of regulation illustrates the major category of regulatory grouping for the regulator,

and the covered entity column depicts the type of financial business that falls under the purview

of the associated regulator.

6

7

The Laws, the Rules, and the Guidances

A prerequisite to any federal regulatory discussion is a delineation of regulatory laws, rules,

and guidances. There are significant differences between federal laws, rules, and guidance.

Laws are pieces of legislation that are passed by both houses of Congress and signed by the

President. Regulations (also referred to as rules) are a set of requirements issued by a federal

government agency to implement laws passed by Congress. Regulations are published through a

rule – making process that is made available for public comment through the Federal Register.

Once a final rule is issued, it has the force of law because it is implementing a law. Safeguards

addresses the safeguarding of consumer information confidentiality through prevention of

unauthorized access and unauthorized use of consumer personal information.

Gramm – Leach – Bliley (1999) asserts that financial institutions must “protect the security

and confidentiality of its customers’ nonpublic personal information” (p. 15 USC 6801) and that

regulatory agencies must establish standards for financial institutions relating to “administrative,

technical, and physical safeguards” to protect confidentiality, integrity, and unauthorized access

of personal information (p. 15 USC 6801). From 2003 – 2005, the various financial regulators

imposed Safeguards rules on their financial sector regulated entities.

8

Safeguards Rules

The Safeguards regulations are a goals – based regulatory regime enacted from 2003 – 2005

by financial regulators initiating implementation of GLBA stipulations. Goals - based

regulations establish regulatory aims, but do not specify the exact means to achieve these aims;

whereas, rules – based regimes establish specific standards, protocols, and tool (Decker, 2018).

Safeguards regulations established a regulatory environment that compelled financial sector

companies to aim for the outcome of “protection of customer records and information” by

“adopt[ing] written policies and procedures that address administrative, technical, and physical

technical, and physical safeguards for the protection of customer records and information by: (1)

Ensuring security, confidentiality, and integrity of customer records and defend against (2)

Mitigating unauthorized access to records and information that could result in “substantial harm

or inconvenience to any customer” (Taft et al., 2019, p. 2). Although at the time of adoption, the

primary focus of these Safeguards rules were administrative and physical protections from loss,

the growing maturity of the digital ecosystem in the 2000s ushered in an increased focus on the

technical aspect of the regulation – what we now refer to as cybersecurity.

9

Intensifying Covered Entity Regulation Enforcement

Financial regulators issued Safeguards rules in the early 2000s, but enforcement actions

remained sparse for the better part of the next decade. From 2000 - 2010, the SEC infrequently

brought cybersecurity focused enforcement actions. Although in the early 2000s financial

regulators remained a bit shy in wading in the cybersecurity enforcement waters, toward the

2014 - 2015 time frame, financial regulators intensified emphasis on cybersecurity regulation

audit and enforcement. This period marks a dramatic elevation in SEC, FTC, and CFTC

cybersecurity focus. In April 2014, the SEC heralded its first cybersecurity sweep examination

as offering six key take - aways that entities could contemplate when executing Safeguards

requirements.

Financial regulators were becoming much more aggressive in enforcing Safeguards rule

requirements. From 2015 - 2018, the SEC brought forth three enforcement actions against R.T.

Jones Capital Equities, Craig Scott Capital, and Morgan Stanley. The Morgan Stanley action is

particularly noteworthy. In this action the SEC stated that "despite having written policies and

procedures, Morgan Stanley had breached the Safeguards rule. The safeguards were not

reasonably designed to protect customers" (Pierotti, 2018, p. 413). With this action, the SEC sent

a message to the financial sector that both the quality of cybersecurity planning and the

16

assertiveness is exemplified by financial sector cybersecurity regulation with the actual demand

of actions. Describing the authoritarian tool, Hood introduces the term “constraints” (Hood &

Margetts, 2007, p. 62). The term constraint usually implies prohibition of an activity, but Hood

discusses both positive and negative constraints. Positive constraints are when government

requires and action. Negative constraints are when government prohibits an action. Financial

sector cybersecurity regulations (and presumably expanded regulation that will impact other

sectors) are effect based positive constraints because they direct a specific action through their

regulatory token of authority to shape the behavior of a specific population.

tool past financial incentives to "more subtle applications" such as establishment of

organizational connections (Hood & Margetts, 2007, p. 78). In fulfilling detection functions,

government may use rewards or government purchases for information. When using treasure to

effect public behavior, government can execute a quid pro quo for a specific population or it can

execute without requiring an affirmative exchange. Some examples of quid pro quo are

government tax breaks or loans for specific endeavors, grants, loans, and tax breaks. A common

form of non quid pro quo use of treasure to effect policy outcomes is through public money

transfers. It is not uncommon for government to try and bind policy outcomes to public money

17

transfer mechanisms such as pensions or welfare programs to shape opinions or actions (Hood &

Margetts, 2007).

Organization

Organization is the last tool Hood discusses in his tools of government theory. When

employing the organization tool, the government acts with its organic capability. Hood &

Margetts (2007) characterize organization “as stock of land, buildings, and equipment, and a

collection of individuals” (p. 102). Like the other tools, organization serves both detecting and

organization for cybersecurity is the creation of the Cybersecurity and Infrastructure Security

Agency (CISA). Over the past five years, CISA has grown into one of the most influential

federal agencies. With a project 2024 budget of $3.1 billion, it sways considerable influence

shaping both government and US private sector cybersecurity endeavors.

18

Behavioral Public Policy Based Tool Selection

specific instruments, his position was largely that the instrumentation process “cannot be

systemically appraised” (Howlett, 1991, p. 10) and that instrumentation is largely an iterative

learning process based on “past presents and experiences” (Howlett, 1991, p. 11).

Nevertheless, policy scientist continued to seek systematic approaches to explain and guide

policy instrumentation. Throughout the early part of the 21st century, efforts to determine how to

select policy tools largely focused on direct resource based instrumentation. In this era,

instrumentation was often mechanical and an exercise of aligning the perceived right instruments

for the job. Van Duen et., al (2018) note that during this era, policy makers assumed a rationality

of thought by policy targets and that policy targets would make decisions that “maximize

welfare” (p. 3). However, toward the second decade of the 21st century, policy instrumentation

choose alternatives. Behavioral public policy moved beyond exclusively considering the

feasibility and acceptability of policy instruments, but included considerations of how certain

instruments might cognitively shape the policy targets decision.

19

Nudging, made famous by Richard Thaler and Cass Sunstein, was one of the first attempts to

operationalize behavioral public policy. Nudging attempts to “change the choice architecture of

citizens, i.e, the informational or physical structure of the [policy] environment” (Van Deun, et.,

al, 2018, p. 6) and “change individual behavior with coercion (Martin & Mikolajczak, 2023, p.

363). Nudging frequently aligns with use of Hood’s nodal and organizational instruments.

There are two critical premises behind nudging: (1) Policy targets’ desires are aligned with the

policy ends and (2) The government uses policy instruments to set the conditions to make it

easier for policy targets to make decisions that are aligned with policy ends (Lawlor & Hopkins,

As the policy science discipline progressively integrated more thoughts from behavioral

scientists, a view that an understanding of why policy targets may respond to one instrument or

set of instruments versus another was severely lacking. To address this, policy scholars began to

aggressively revisit the likes of Schneider and Ingram. An early vanguard in forging connections

between policy instrumentation and behavioral science, Anne Schneider and Helen Ingram

(1990) argued that "existing concepts had not been helpful in understanding choices among

policy instruments" (p. 511). While they didn't disagree with resource - based taxonomies, they

argued that these approaches were limited because they did not account for behavioral

characteristics that would shape policy target adoption. To that end, Schneider and Ingram

20

(1990) introduced a framework to "capture the behavioral attributes of policy content" (p. 511).

The policy instruments included in their framework were: authority tools, incentive tools,

capacity tools, symbolic / hortatory tools, and learning tools. Interesting to the current cyber

policy domain, they offered that authority tools could be helpful when incentives are lacking and

that policy targets are indoctrinated in a culture of regulation adherence. When bemoaning the

lack of effectiveness of PPPs and information distribution in facilitating private sector critical

infrastructure cybersecurity, Madeline Carr (2018) referred to the challenge of lack of incentives.

In these areas, Carr's insights align with some existing behavioral policy based study that

based "calibration of policy tools" (Howlett & Fraser, 2018, p. 101). Hood's resource-based

taxonomy continues to provide a widely used scaffolding for understanding and discussing

various policy instruments, but policy scholars are integrating behavior concepts as discussed by

Schneider and Ingram in instrumentation design. Howlett and Fraser (2018) discuss the concept

of the supply and demand side of policy tools. Tool instruments such as Hood's NATO constitute

the supply side whereas, the policy target constitutes the demand side. The behavioral bridge

between the supply and demand sides is that the effectiveness of instrumentation and instrument

selection is "linked not just to resource availability", but to an ability of policy instrumentation to

shape behavioral "receptors on the part of policy targets which make them respond in predictable

21

ways" (Howlett & Fraser, 2018, p. 110). Howlett further enumerated policy target behavioral

perquisites for consideration for each of Hood's policy instruments. Table 2 below (Howlett &

Fraser, 2018, p. 111).

22

Connected to these concepts, scholars in the economics and legal disciplines have devoted

time to specific focus on the authority tools and the types of regulation that is most conducive to

target population compliance. Because this research focuses on the authority tool in the form of

regulation, it is helpful to introduce a distinction between command and control direct regulation

and incentive based regulation. When employing direct regulation, government creates policy

requirements and frequently relies on penalties for regulatory compliance; when relying on

incentive based regulation, government seeks to create opportunities to entice compliance

through things such as reduced costs (Malloy, 2002). Both of these approaches leverage what

– policy targets will seek the least expensive way to meet regulatory compliance (Malloy, 2002).

In order to mitigate this byproduct, it may be better crafted as management – based regulation or

performance – based regulation versus directive requirement regulations. In the next section of

this literature review, this distinction is highlighted in a work by David Thaw (2014).

It is important to remember that although behavior based policy instrumentation offers great

promise, it is not a policy science panacea. Governance is full of failed policies because of an

inability of policy makers to account for key obstacles to behavior based policy design. Holger

Strassheim (2019) argue that prominent obstacles that we should note are that: One, behavioral

based policy design is often ill - shaped because policy makers misinterpret or misapply target

23

feedback. The misinterpretation can pollute the policy instrumentation learning environment and

result in a failed contamination; Two, behavioral based policy design approaches can be very

culturally contextual. As discussed by Howlett and Fraser (2018), there may be specific set of

target behavioral prerequisites that enable the effectiveness of instrumentation. A lack of

consideration for the cultural differences between one target group and another can instrument

success for one group and failure for another.

24

cybersecurity regulation comparing the impact of directive regulations akin to state breach

reporting laws vs federal management – based regulations (p. 294). He characterizes directive

regulations as “laying out express performance standards and / or means of achieving

performance” (Thaw, 2014, p. 310) whereas he characterizes management – based regulations as

“modes of regulation under which administrative agencies, through legislatives mandated

collaboration with regulatory stakeholders, promulgate regulations requiring regulated entities to

develop plans designed to achieve certain aspirational goals” (Thaw, 2014, p. 324). His period

of study is from 2000 – 2010. His key findings were that regulatory tools can be effective in

Another interesting perspective along this topic area is a study by Massacci et al. (2016).

Their work does not seek to evaluate the effectiveness of regulation vs other policy tools but

provides interesting insights as to which regulatory models may be more effective. Rules - based

regulation applies mandated security provisions and is the dominant form for the US bulk energy

sector. Risk - based regulation applies mandates for regulatory outcomes. For example, rules -

based regulation may dictate specific cybersecurity controls, whereas risk - based regulation may

dictate thresholds of impact to the electric grid. Leveraging the Institutional Analysis and

Development (IAD) framework, Massacci et al. found that: (1) The effectiveness of rules - based

regulation is predicated upon the regulators knowledge of the security architectures, (2) A clear

25

understanding of the regulator's desired outcome from regulation is essential, and (3) Culture

matters and that the appropriate regulatory approach may hinge on the cultural dynamics of the

regulated ecosystem (Massacci et al., 2016).

Loren Selznick and Carolyn LaMacchia (2016) investigate if SEC cybersecurity actions are

leading to desired outcomes. However, their focus is SEC CF disclosure guidance and not

financial sector Safeguard rules. They ultimately assert that SEC disclosure guidance has done

little to impact the likelihood of cybersecurity breaches and efforts to incentivize breach

deterrence through investor awareness have failed because most company disclosures are little

contributes little to this study’s research question. CF Disclosure guidance is expressly focused

on reporting breaches as part of traditional investor risk notifications. Unlike Safeguards Rule

that are applicable to financial sector companies, CF Disclosure guidance does not institute

cybersecurity control activities. Their findings align with conclusions by Michael Ferraro (2013)

who contends that CF Disclosure guidance “substantially underachieves” (p. 297).

As opposed to measuring reductions in breach frequency and severity, Lam and Seifert (2023)

use financial investment as a proxy for cybersecurity regulation effectiveness. Lam and Seifert

(2023) highlight "important market failures" because companies under-invest in cybersecurity (p.

146). They offer that a contributing factor to this dynamic is that the private sector overly

32

be reluctant to report breaches in SEC risk disclosures. However, much empirical research

argues that companies are actually incentivized not to hide breaches from the public. Jumah and

Alasour (2020) discovered that there is not a significant negative effect of breach notification on

company value. Liune and Eli’s (2018) study discovered that withholding breach information is

associated with a “decline of approximately 3.6% in equity values in the month the attack is

discovered” (p. 1177). This insight is echoed by Gordon and Loeb (2010) who reported that

there is “strong evidence” that breach disclosure results in an increase in company market value

because public disclosure facilitates public trust and mitigates potential litigation costs (p. 568).

out data collection and analysis techniques.

33

Philosophical Approach

The work’s philosophical world view flows from a positivist and empirical perspective. A

positivist approach is characterized with objectivism, impartiality, and a worldview of theory

based deductive analysis. When discussing the positivist research philosophy, James Scotland

states that “the researcher and the researched are independent entities” (Scotland, 2012, p. 10).

John and J. David Creswell note that hallmarks of positivist research world views are

determination, reductionism, empirical observation, and theory verification (Creswell &

Creswell, 2018). In this condition, outcome variables can be inferred from indicator variables

The question is - have safeguards regulatory rules reduced data breach severity and frequency

on private sector US financial institutions from 2005 - 2020? Further explanation of breach

frequency and severity is provided in the subsequent key variable subsection. This study

hypothesizes that Safeguards rules have significantly helped reduce successful breach frequency

and severity.

34

Research Design

Research design is based upon selection of methods that provide direction and azimuth for the

research study protocol (Creswell & Creswell, 2019). Well considered design enables

researchers to effectively operationalize the theoretical framework. My theoretical framework

derives from behavioral public policy theory which contends that when there is alignment of

policy target behavioral attributes and selected policy tools, desired policy outcomes are more

likely. The financial sector maintains many of the behavioral attributes that behavioral public

policy theory asserts are conducive for use of regulatory tools to achieve desired policy

regulations on breach frequency and severity, there is engagement with policy evaluation phase

activities. In his book Studying Public Policy: Principles and Processes, Howlett et. al (2020)

provides key concepts for policy outcome evaluation design. The first of these concepts is a

distinction between efficiency based output assessment and impact based outcome assessment.

Efficiency and outputs are tied to products, whereas, impacts and outcomes are tied to changes in

the environment. This study focuses on desired regulatory outcomes – protection of consumer

information through reduction of frequency and severity of breach in financial sector companies.

As a result, selected outcome variables should measure frequency and severity of breach. The

second concept regards selection of the type of experiment. Howlett et al (2020) note that an

35

advantage of quasi – experiments is that “they occur in the real world which makes the findings

more robust and generalizable” (p. 261). I adopt this form of experiment, but instead of using

the term quasi – experiment, I use the term nonexperimental. The key hallmarks of

nonexperimental research are a lack of manipulation of the independent or indicator variable, and

a lack of participant random assignment. Nonexperimental design is appropriate when the

researcher’s hypothesis includes a “relationship between two variables” and it is not “feasible

[or] ethical to manipulate the independent variable” (University of Minnesota Libraries, no date,

1st and 2nd paragraph). Such is the case in this study. It is not feasible to manipulate the indicator

In the most basic terms, a study’s unit of analysis is the entity that the analysis is focused and

the main parameter of investigation (Columbia University Libraries, no date). Tying this

concept to the theoretical framework, this work’s unit of analysis is successful breaches on

private sector companies from 2005 – 2020 and delineation if the victim was a member of the

financial sector. STATA SE 18 allows time effect included difference in difference analysis of

pooled cross section data of individual breach events to assess the impact of the financial sector’s

Safeguards regulations as a treatment to breach severity and frequency.

36

Indicator Variables

In positivist studies, indictor variables are derived from theoretical suppositions and provide a

predictor of outcome variables. In this study, the indicator variable is a policy treatment –

whether a breached company was subject to financial sector Safeguards regulations. This is

categorically measured for each successful breach attempts from 2005 – 2020, indicating

whether the breached company was a member of the financial sector and subject to financial

sector cybersecurity regulations or not a member of the financial sector. A determination of

whether a breached company is a member of the US financial sector is based on the original

the company’s four digit North American Industrial Classification System (NAICS) code. Refer

to Appendix A for a NAICS code decomposition table extracted from the NAICS association (no

date).

37

Outcome Variables

Because this is a positivist study, an assumption is that there is an impact of indicator

variables on outcome variables. As previously discussed, behavioral public policy theory

suggests that authority tools should be effective means to achieve policy outcomes in the

financial sector. Assessing the impact of Safeguards regulations on breach frequency and

successful breach severity is an effective way to evaluate whether these regulations facilitated

desired cybersecurity policy outcomes. Safeguards rules were implemented as management -

based regulation to compel financial sector companies to adopt “administrative, technical, and

study incidents that involve stolen computers, accidental emails, lost network recovery tapes, or

accidental postings of information on a public webpage are not characterized as breach incidents.

policy instrumentation on the private sector, victim entities such as governmental organizations

and not for profit organizations are excluded. Additionally, the health care and bulk energy

sectors are excluded because these sectors are subject to other sets of cybersecurity regulations as

per the Health Insurance Portability and Accountability Act of 1996 (HIPAA) and the North

American Electric Reliability Corporate Corporation Critical Infrastructure Protection (NERC

CIP).

38

Breach severity is measured using two elements – number of records exposed during a

successful cybersecurity breach and the sensitivity of the information contained in exposed

records. Both of these elements were prominently included in IT – Harvest and SafeNet’s Breach

Level Index (BLI). In 2013, IT-Harvest and SafeNet released its BLI in a presentation at the

globally influential RSA Cyber conference. They stated that breaches are not "binary

proposition[s] where an organization either has or has not been breached" (Stiennon, 2013, p. 1).

Instead, breaches are "wildly variable in their severity and ramifications" (Stiennon, 2013, p. 1).

To help organizations better measure the impact of their cybersecurity breaches, IT - Harvest and

individual to whom the information applies to be reasonably inferred by either direct or indirect

means” (Department of Labor, nd., first paragraph). However, this definition is not helpful in

delineating sensitivity of extracted information because it lumps social security numbers in the

same characterization as names. This study adopt two categories of sensitivity: nonsensitive and

sensitive. A nonsensitive record is characterized with attributes such as names, email address,

addresses. A nonsensitive record does not offer account authentication and access information to

on – line accounts or any financial information. Sensitive records are characterized as records

that allow account authentication and access information, provide financial information, provide

39

social security numbers or other data that could be expected to cause significant consumer

impact.

Covariates

In this study, covariates are variables other than the examined policy treatment variable that

could impact breach severity and frequency outside of Safeguards rules. The variables are

helpful in isolating the impact of Safeguards by controlling for the impacts of other deployed

governmental policy tools. The federal government has, and continues, to leverage policy tools

cybersecurity threat information. The federal government has a long standing view that

information sharing is foundational to private sector cybersecurity (GAO 04 – 780, 2004). To

that end, the federal government has “emphasized the importance of ISACs” and have worked

closely with ISACs to provide cyber threat information to industry sectors. (GAO 04 – 780,

2004, p. intro). Industry sectors with an ISAC may be more apt to address emerging

cybersecurity risks. Over the last 25 years, ISACs have emerged independently in various

industry sectors. Appendix B provides a listing of the year each ISAC was established. Another

covariate is whether a breached company was in a critical infrastructure sector with an assigned

sector risk management agency (SRMA). In 2013, Presidential Directive – 21, assigned the 16

40

designated critical infrastructure sectors a federal SRMA to coordinate cybersecurity defense.

SRMAs often recommend cybersecurity standards, proliferate threat information, and partner

with ISACs. It is reasonable to assume that both of these covariates could impact breach

frequency and severity absent of federal regulations. For each breach from 2005 – 2020, an

assessment is made as to whether the breached company was in a sector with an active ISAC and

whether the company was aligned to a SRMA.

capture breach events and build a reliable dataset, this study fuses information from two

respected sources: The Audit Analytics database and Privacy Right’s Data Breach Chronology.

The company Audit Analytics maintains comprehensive databases that are tailored for a

range of financial sector investigations. The company’s databases are used to support research by

many R1 universities such as Cornell, University of Florida, Duke, and the University of Kansas

and are cited in over 1500 academic papers (Audit Analytics for Academia, no date). One of its

databases is the cybersecurity database which contains information about cybersecurity incidents

impacting all SEC registered companies. The company builds the database through a

combination of both SEC filings and open-source reporting of cybersecurity breaches.

41

PrivacyRights.org is a not-for-profit organization (privacyrights.org, 2023) largely funded by

The Rose Foundation for Communities and the Environment Consumer Products Fund. Drawing

data from the U.S. Department of Health and Human Services, public press breach reporting and

US states with breach reporting requirement laws, it assembled a database titled The Privacy

Right’s Data Breach Chronology that captures cybersecurity breach information for the majority

of the last twenty years. The database is a well-regarded source of data and was prominently

featured in the Journal of Accountancy (Collins, September 1 2019).

not feasible. Such is the case with the current research. In these circumstances, difference in

difference analysis is very constructive in estimating the effect of policy intervention (Wing, et.

al, 2018). Difference in difference allows an evaluation of the “effect of a specific intervention

or treatment (such as a passage of law, enactment of policy, or large-scale program

implementation)” by contrasting the “changes in outcomes over time” between a control and

subject group after the intervention. (Columbia University Mailman School of Public Heath, no

date, description paragraph). Difference in difference methods allow a focus on the policy effect

outcomes prevalent at the unit of analysis over time under the alternative levels of treatment

(Wing et al. 2018). It aligns well with the policy evaluation requirement of a counterfactual

48

The purpose of this chapter is to report the results of the statistical analysis. Discussions

regarding the result’s implications for practitioners and theory, recommendations, and

conclusions are in Chapter V. After conducting three regressions, the analysis does substantiate

the hypothesis that the financial sector Safeguards regulations have significantly impacted

cybersecurity breach severity and frequency in financial sector private companies relative to

companies that are not subject to federal cybersecurity regulations. The remainder of this

chapter will revisit the research question and key variables, discuss descriptive statistics, report

financial sector and subject to Safeguards regulations. The outcome variables are breach severity,

measured as the numbers of records extracted in each breach and whether these records contain

sensitive PII, and breach frequency, measured as the total number of breaches in a given year

divided by the number of breaches that occur on financial sector companies. Two covariates are

whether the breached company’s industry sector has an active ISAC, and whether the breached

company has an assigned SRMA. Both of these covariates represent government policy tools,

other than regulation, that could affect private sector breach frequency and severity outside

enactment of Safeguards regulations. Inclusion of these covariates provides an ability to control

for government employed nodality and organization tools and isolate the impact of Safeguards

49

regulations. Please refer back to Chapter III for a more detailed discussion regarding key

variables.

After considering the American Statistical Association’s (ASA) statement about p – value

significance which contends that researchers should not base statistical significant conclusions

on whether a p – value exceeds a specific value and that significance must be considered in

greater context (Bentensky, 2019), this research adopts a (P ≤ 0.10) significance value; but,

assertions regarding the significance of findings is based on a variety of measures that include

coefficient values and whether the difference between the treatment and control group is

Descriptive Statistics

population of breach events (Fulk, 2023, p. 1). The study’s population is the total number of

successful breaches from 2005 – 2020; this number is 746 (N = 746). Because the research

focuses on cybersecurity regulatory impacts on the private sector, entities such as government

organizations, not for profits, and universities are omitted from the population. Additionally,

private sector companies that are subject to cybersecurity regulations other than Safeguards rules

are also omitted. Entities of this type include companies subject to medical HIPAA and bulk

energy NERC CIP regulations. Finally, recorded breaches without an associated number of

records extracted or clearly defined sensitivity of records extracted are omitted from the study.

Table four provides some key descriptive statistics.

50

Total Population (N) = 746

breached companies. This is consistent with data presented by private sector cybersecurity

researchers. In Kroll Cybersecurity Risk Management firm’s annual Data Breach Outlook, David

White (2024) reports that in 2023 the financial sector accounted for 27% of breaches.

Additionally, this ratio is very similar to the ratio of mean number of breaches per year between

these groups in this study’s descriptive statistics. The mean number of breaches per year for

financial sector companies is 12, and the mean number of breaches per year for non-financial

sector companies is 38. Interestingly however, is that this ratio does not hold when examining

the median number of records extracted per breach in each group; the ratio of median number of

51

records extracted per breach closes considerably between the two groups. The median number of

extracted record per breach in financial sector companies is 1,300, whereas, the median number

of extracted records per breach in non-financial sector companies is 3,657. Furthermore, when

examining the percentage of breaches that exposed sensitive records, the ratio reverses with

financial sector company breaches exposing sensitive records 74% of the time and mon-financial

sector company breaches exposing sensitive records 69% of the time.

University Mailman School of Public Heath, no date, description paragraph). It provides an

ability to integrate both the effects of a specific policy intervention and the effects of time on

outcome variables. I execute two difference in difference regressions that address severity of

breaches. The first regression evaluates the impact of Safeguards regulations on the number of

records extracted in breaches on financial versus non-financial sector companies from 2005 –

2020. The second regression evaluates the impact of the Safeguards regulations on the sensitivity

of the records extracted in breaches on financial versus non-financial sector companies from

2005 – 2020. For a third regression, I employ a linear model to evaluate the likelihood of a

52

breach being a financial sector company by examining the annual percentage of total breach in

the financial sector from 2005 – 2020.

For all three regressions, the indicator variable is whether a breached company was a member

of the financial sector and subject to Safeguards regulations. The outcome variables are breach

severity and likelihood. In regression one and two, severity is measured as the numbers of

records extracted in each breach and the sensitivity of these records - whether these records

contained sensitive PII. In regression three, frequency is measured as the total number of

breaches in a given year divided by the number of breaches that occur in financial sector

dissertation prose variable name.

53

Number of Records Extracted Regression

The first regression uses the outcome variable the number of records extracted per breach.

This regression includes the following variables: (1) The Equal Effect of Time (2) Differences

Prior to Safeguards and (3) Difference After Safeguards. While, financial sector regulators

formalized Safeguards in formal regulatory rule – making from 2000 – 2005, this study marks

2007 as the post treatment variable because we would expect a several year lag in regulation

enactment and implementation by private sector companies. The graph below shows a

correlation exists between implementation of Safeguards regulations and a relative reduction of

records extracted in the financial sector vs the non-financial sector. Please note the divergent

trajectories between financial sector companies and non – financial sector companies. As

54

The difference in difference regression output reveals a statistically and practically significant

impact on the number of records extracted correlated to the implementation of Safeguards

cybersecurity regulations. In terms of statistical significance, the regression meets the

established p – value significance threshold F (3, 742) = 4.27, p = .065, there is also a non-

significant difference between the financial sector and non-financial sector prior to

implementation of Safeguards regulation p = .308. In terms of practical significance, the

Difference After Safeguards variable coefficient indicates that companies in the financial sector

only experienced an increase of records extracted by approximately 3 million, compared to

roughly an increase of 20 million records extracted in the non – financial sector. The Equal

55

in non – financial sector company breaches. The Difference Prior to Safeguards coefficient

shows us that prior to implementation of Safeguards regulations the difference in records

extracted from financial sector companies versus non – financial sector companies was 4.2

million records. The Difference After Safeguards coefficient shows us that from 2005 – 2020,

16.7 million less records were extracted from financial sector versus non – financial sector

companies during cybersecurity breaches.

Although the initial regression proved statistically and practically significant, it does not

account for omitted variable bias because the two covariates were not included. To validate these

findings and mitigate the potential of omitted variable bias, I regressed the number of records

extracted two more times – first, including the covariate active ISAC and second, including the

covariate assigned SRMA. Finally, I executed a Joint F – Test regressing with both covariates to

validate a lack of significance for the combined effect of both covariates.

When regressing with the covariate active ISAC, the impact of Safeguards regulations,

56

note, the covariate Active ISAC proved not significant F (4, 741) = 3.27, p = .108. The

Treatment Post’s coefficient increased from -16.7662 to -17.1560 and robust standard errors only

slightly increased from 9.0626 to 9.6657. When regressing with the covariate assigned SRMA,

the impacts of Safeguards regulations also remained significant F (4, 740) = 3.51, p = 0.095. Of

additional note, the covariate assigned SRMA also proved not significant F (4, 740) = 3.51, p =

0.285. The magnitude of the Treatment Post’s coefficient increased from -16.7662 to -27.6123.

However, robust standard errors notably increased from 9.0626 to 16.5316. This increase is

likely because SRMAs did not exist until 2013. When regressing with this covariate, all breaches

combined impact of both covariates was not significant F (2, 739) = 1.30, p = 0.2726.

Sensitivity of Records Extracted Regression

The second regression uses the outcome variable the sensitivity of information in the records

extracted per breach. This regression includes the following variables: (1) The Equal Effect of

Time. (2) Differences Prior to Safeguards (3) Difference After Safeguards. In the graph below

note the parallel trajectory of the financial and non – financial sectors after the implementation of

Safeguards regulations in 2007. This demonstrates that implementation of the Safeguards

57

regulation does not create a significant difference of the sensitivity of records extracted between

financial sector companies breached and non – financial sector companies breached.

established p – value significance threshold F (3, 676) = 4.27, p = .785, there is not a significant

difference between the financial sector and non-financial sector prior to implementation of

Safeguards regulation p = .303, nor does the treatment post coefficient indicate a difference in

the sensitivity of records extract between financial companies and non-financial companies.

64

CHAPTER V: IMPLICATIONS AND CONCLUSIONS

The previous chapter provided the analysis’s findings and assessed the validity of the

hypothesis that the financial sector Safeguards regulations have significantly reduced cyber

breach severity and frequency in financial firms in comparison to other industries not subject to

cyber security focused regulations. Leveraging difference in difference and linear model

regressions, Chapter III substantiated this hypothesis. This chapter operationalizes the findings

from Chapter IV into implications for behavioral public policy theory and national policy

practice in the cyber domain. Additionally, this chapter will offer recommendations for further

and, as a result, it is incumbent on researchers to convey the research’s caveats (Botterill &

Hindmoor, 2012, p. 370). To that end, key caveats are addressed in discussions regarding

implications.

The terms research caveats and research limitations are often used interchangeably. This

study views them differently. This chapter provides separate discussions regarding caveats and

limitations. Research caveats are qualifiers of the results that require contemplation when

considering the implications of the work. An example of a caveat is if a study found that some

caffeine may be beneficial, but too much caffeine may be harmful. Research limitations are

65

boundaries of scope, or constraints to the study. Using the same caffeine example, a limitation

could be that the study did not consider the benefit of caffeine for males versus females.

Study Limitations

There are three notable potential study limitations. The first limitation gravitates around data

collection. A lack of an authoritative breach data is a limitation that challenges all studies using

cybersecurity breaches as a key variable. There is no authoritative source for breach information.

Various companies, academic institutions, and not for profits have built databases that capture

selection to policy outcome achievement. This study focuses on the alignment of behavioral

attributes that are conducive to regulatory tools in a specific industry sector. Other industry

sectors may not have the prerequisite behavioral attributes that are conducive for authority tools.

As a result, this study’s findings are generalizable to policy targets with similar behavioral

attributes, but scholar and practitioners should take heed in generalizing findings to all industry

sectors. Finally, this study’s analysis includes data from 2005 – 2020. In 2020 the FTC revised

its Safeguards rule and extended it to companies that were originally excluded. The 2020

modifications opened the covered firm aperture to include companies such as personal property

and real estate appraisers, automobile manufacturers and dealers that extend credit (Federal

66

Trade Commission, 2022). Additionally in 2023, the SEC proposed a modified Safeguards rule

which included additional cybersecurity requirements. The impacts from FTC and SEC rule

expansions may further impact firm breach frequency and severity, but it will take several years

for firms to implement these regulatory changes and additional time for accurate impact

measurement. The impact of these rule expansions are outside the scope of this study. Further

study expanding the scope of the current work is recommended in the years to come.

discipline and behavioral economics, which emerged as behavioral public policy (Starassheim,

2019). Behavioral public policy theory moved beyond a traditional resource – based policy

instrumentation approach of balancing the feasibility and acceptability of policy instruments. It

moved to deliberately considering the potential effectiveness of a policy tool by evaluating the

policy target population’s cultural norms and behavioral attributes.

The current state of behavioral public policy research assumes that "policy tool use and

behavioral expectations" (Howlett & Fraser, 2018, p. 101) are linked and that a correlational

relationship exists between alignment of policy target behavioral attributes and policy tool

selection to policy outcome achievement. Behavioral public policy posits that the effectiveness

67

of policy instrumentation and instrument selection is "linked not just to resource availability",

but to an ability of policy instrumentation to shape behavioral "receptors on the part of policy

targets which make them respond in predictable ways" (Howlett & Fraser, 2018, p. 110).

However, even the leading scholars in the behavioral public policy field contend that more

research, in a wider spectrum of policy domains, is needed to validate the theory.

The US financial private sector enjoys the cultural and behavioral attributes that are perceived

as conducive to use of regulatory policy tools. The financial sector is ingrained over much time

with a willingness to respond to “government invoked penalties and proscriptions” (Howlett &

National Cyber Director’s (2024) report on cybersecurity regulation harmonization, stated that

the financial inquiry respondents “broadly supported the need for enhanced cybersecurity

regulation” (p. 19).

For all of these reasons, we should not be surprised that the findings in Chapter IV seem to

substantiate the validity of behavioral public policy theory in the cybersecurity policy domain.

The financial sector, which maintains the cultural and behavioral attributes that behavioral public

policy theory asserts are conducive to use of regulatory authority tools, experienced a significant

breach frequency and severity decrease in the two decades following implementation of

Safeguards regulations relative to other cybersecurity unregulated sectors. An important insight

68

is that this correlation existed while controlling for government use of information and

organization based policy tools in the form of ISAC support and SRMA creation. There are

several implications for theory based on these findings. Supporting the central premise behind

behavioral public policy theory.

The first implication is that a continuum – based perspectives to policy instrumentation seems

more appropriate than an exclusively resource – based perspectives. This is a long standing

debate that dominated policy science circles in the later part of twentieth century. Resource

based approaches, which were more popular in United Kingdom scholarly works, offered that

which instrument was most contextually appropriate given the social, political, and economic

environment (Howlett, 1991). Although the concept of behavioral public policy were not

meaningfully discussed at this time, a key implication from this study is that behavioral public

policy can be an effective way to operationalize continuum – based tool section.

A second implication for theory is that policy targets may not be strictly rational actors and

that Van Duen (2018) was correct in contending that that policy targets are “subject to systemic

bias” and use “cognitive shortcuts or rules of thumb” when reacting to policy instruments (p. 3).

This premise is foundational to behavioral public policy theory and separates it from historic

resource – based policy instrumentation approaches. It reaffirms the nexus of bounded rationality

69

and behavioral public policy theory – that there are “limits to human cognition” and “limits to

our knowledge of the social world”. (Sidney, 2007, p. 80). These limits shape policy target’s

internal calculus when evaluating their propensity for policy compliance.

However, there is a notable study caveat to these implications. The results of this study lend

credence to the idea that when specific cultural and behavioral attributes exist, regulation – based

authority tools are an appropriate policy tool selection. However, the study does not provide

insights into if regulation may also be effective when focused on policy targets that do not have

these same cultural and behavioral attributes. A key caveat is that is it possible that enactment of

For most of the 21st century, federal cyber policy makers outside of the financial sector heavily

relied on a combination of nodal and organizational policy tools to enhancing private sector

cybersecurity. Cybersecurity information campaigns and voluntary opt – in cybersecurity

focused public private partnerships (PPP) have often been the North Star for federal cyber policy

instrumentation. Since the George W. Bush administration, democrat and republican

administrations alike, primarily relied on nodality and organization policy tools to facilitate

cybersecurity in the private sector. The 2003 National Strategy to Secure Cyberspace stated that

“federal regulation will not become a primary means of securing cyberspace” (Healy, 2023a,

Getting Markets to Work Section). This sentiment was subsequently echoed during President

70

Obama’s administration. Hesitance to employ regulatory tools was largely because the majority

of digital infrastructure in the United States is owned and operated by the private sector.

However, federal policy makers are now increasingly adopting regulatory approaches

traditionally leveraged by the financial sector. In March 2022, President Biden signed into law

the most sweeping piece of cybersecurity legislation yet. The Cyber Incident Reporting for

Critical Infrastructure Act requires all private sector critical infrastructure providers to report

cybersecurity incidents within specified time frames (Office of the Federal Register, 2024).

Additionally, Biden administration’s 2023 National Cybersecurity Strategy states “new

streak” (Healy, 2023b, in the title of the article).

However, there is little empirical study to illuminate the effectiveness of the federal embrace

of federal cybersecurity regulation. By focusing on the effectiveness of long – standing

cybersecurity regulation in the financial sector, this work offers important implications for

federal cybersecurity practitioners. The most significant implication is that early 21st century

assumptions about cybersecurity regulatory ineffectiveness because the majority of digital

infrastructure is owned and operated by the private sector is incorrect. The financial sector

demonstrates that regulation – based authority tools can be an effective federal policy tool for

reducing breach frequency and severity in the private sector. This study empirically

71

demonstrated that through use of cybersecurity regulations, financial sector regulators were able

to reduce the severity and frequency of breach in financial sector companies. This implication

validates the new federal approach to private sector cybersecurity and substantiates Jason

Healy’s claim.

A second implication is that Hood and Margetts (2007) seem to be correct when arguing

authority tools are necessary when “there are no clear incentives” to adhere to government’s

policy desires (p. 150). Renowned cybersecurity scholar Madeline Car (2016) contends that one

of the biggest reasons for the ineffectiveness of exclusive use of nodal and organization based

domains.

However, these implications must be digested with an awareness of several of caveats. The

first of these caveats is that the Safeguards regulatory regime analyzed in this work is

management – based versus directive – based. This distinction is not trivial. The financial

sector’s management – based Safeguards regulations establish overarching regulatory objectives,

but leave implementation details to the financial sector firm. In other words, Safeguards

regulations tell firms what they must achieve and firms determine how to achieve it. This is

different than directive – based regulatory regimes where regulators mandate not only specifics

of what needs to be achieved, but also the how of it must be achieved. In a dynamic and ever –

72

changing domain such as cybersecurity, it is not reasonable to assume that this study’s findings

about the impacts of cybersecurity regulation on breach severity and frequency can be blindly

transferred to a directive – based regulatory regime. Potential cybersecurity regulations that

mandate specific technical controls may be limited to short –term success because cyber

adversary’s tools and technologies continue to evolve. Cybersecurity policy makers leveraging

authority tools for instrumentation should be optimistic of regulatory potential, but weary of too

specifically crafting regulation. This sentiment also applies to an overly compliance - based risk

management approach. Safeguards cybersecurity regulations are specific enough to provide

Stanley. In the Morgan Stanley action, the SEC stated that "despite having written policies and

procedures, Morgan Stanley had breached the Safeguards rule. The safeguards were not

reasonably designed to protect customers" (Pierotti, 2018, p. 413). This sent a clear message to

financial sector companies that enforcement of cybersecurity regulation was now a SEC priority

and that light-heartened attempts to pay homage to the regulations would not be tolerated. In

2020, the SEC announced enforcement action against eight firms that included Cetera Advisor

Networks LLC, Cetera Investment Services LLC, Cetera Financial Specialists LLC, Cetera

Advisors LLC, and Cetera Investment Advisers LLC (collectively, the Cetera Entities);

Cambridge Investment Research Inc. and Cambridge Investment Research Advisors Inc.

73

(collectively, Cambridge); and KMS Financial Services Inc. (KMS). When addressing these

enforcement actions, the SEC asserted that "it is not enough to write a policy requiring enhanced

security measures if those requirements are not implemented or are only partially implemented,

especially in the face of known attacks" (Securities and Exchange Commission, 2020, 5th

paragraph).

Cybersecurity policy – makers including regulation – based authority tools in their

instrumentation calculus should be weary of assuming this study’s findings are equally

applicable without a credible threat of enforcement. It is likely that the financial sector

gap, the courts must defer to the regulatory agency's interpretation if the interpretation is

reasonable” (Public Policy Institute of California, 2024, first paragraph). This study’s findings

may not be transferable to an environment where regulator’s ability to execute enforcement

actions are limited as a result of the court’s decision in Chevron v. Natural Resources Defense

Council.