Quizz

Chapter 2

Why Measure Information Security?

PRAGMATIC

security metrics

Chapter 1 Introduction

Chapter 13 Conclusion

Chapter 2

Why measure? Chapter 12 Case study

Chapter 3 Art and science

Chapter 11

Using metrics Chapter 4 Audiences

Chapter 10 Downsides Chapter 5

Finding metrics

Chapter 9 Advanced metrics

Chapter 6

PRAGMATIC

Chapter 8 Measurement system Chapter 7

Example metrics

Appendices

Answering

questions

Strategy,

mgmt, ops

Compliance

For profit

Every CSO should have half a dozen dials to watch on a regular basis. ese indicators could be “survival metrics,” the hot buttons on a dash- board you are expected to address that monitor the wellness of your organization or an issue of particular concern to management.

George K. Campbell (2006)

Given that so many organizations evidently cope much in the way of infowithout mation security metrics, it seems reasonable to explore the reasons why we belie

f h h l l h h b l l

measuring information security is worthwhile although not absolutely essenti

14  ◾  PRAGMATIC Security Metrics

Good practices may, in fact, suffice in some circumstances, but a one-size-fits- approach will never be optimal and inevitably will result in overprotection of som assets and under-protection of others.

From our experience, we believe there is a genuine and increasingly urgent ne for viable metrics in information security. While, to date, the profession has gen ally muddled through with almost no rational, sound, and defensible security me surements, the situation is simply not sustainable over the long term. We are fa approaching and, in some cases, already exceeding the limits of the informati security manager’s gut feeling, qualifications, and experience, coupled with the u of ill-defined and generic good or so-called best practices, as a basis for extreme important security and risk management decisions. While not so common the days, there are still those who contend that as long as you implement best pra tices, you don’t need extensive metrics. However, best practices are an inadequa substitute for genuine knowledge. What may be best in one organization may too costly and excessive in another or, in some cases, wholly inadequate. Witho metrics, how would you ever know?*

Improving information security is becoming ever harder given that we ha already, to a fair degree, harvested the low-hanging fruit. And, unfortunately, our rate of improvement declines, there are clear signs that organized crimina hackers, saboteurs, industrial spies, fraudsters, malware authors, and terrorists a gaining the upper hand, perceptibly raising the stakes. It is not far off the mark suggest that the profession is in, or is fast approaching, a crisis of confidence. We winning occasional battles but losing the war. When experienced security profe sionals turn from being just ordinarily pessimistic and risk-averse to jaundiced a cynical and retiring or leaving the profession for less stressful occupations, is it a wonder that business managers and stakeholders begin to lose faith in our abilitie

e bulk of this chapter consists of a string of rhetorical questions or issues th raise their ugly heads in some form in most organizations at some point. Cou yourself lucky if you haven’t been asked them yet: it’s just a matter of time.

e points that follow would form the basis on which one might justify t

investment needed to specify, design, and use an information security measu

ment system, leading to (we hope) a convincing business case for such a system an potentially, justifications supporting at least the initial suite of security metrics th populate it. Don’t fret: we will discuss the measurement system, the selection metrics, and all that in later chapters, but let’s start by considering the fundamen requirement for security metrics.

* We hear, “You can’t manage what you can’t measure” quite often. It’s an old saw. e phr

has a ring of truth to it, but actually we manage unmeasured things all the time, just ndo particularly well! We contend that a lot of information security managers have been struggl

to manage information securit ith inadequate measures because the had no alternati e

to manage information security with inadequate measures because they had no alternative until now.

Why Measure Information Security?  ◾ 

2.1 To Answer Awkward Management Questions

Your organization is not ready for a metrics program if you do not have a clear, formal understanding of your goals; strategic plans; policies, procedures and guidelines; existing, repeatable processes; and open lines of communication with stakeholders.

Samuel A. Merrell

We opened this book with an imaginary internal memo from the CEO to t information security manager, urgently seeking answers to a bunch of questio that turn out to be rather difficult to answer without the ability to measure vario aspects of information security. Let’s now explore some more of those awkwa questions:

◾ Are we secure enough? Realistic managers should not actually anticipate t organization being perfectly secure and free of all information security in dents, but it is perfectly reasonable for them to seek assurance that avoidab incidents are (mostly) being avoided while any incidents that do occur cau minimal (ideally negligible) or, at least, manageable impacts. Manageme also wants to be reasonably confident that the information security measu in place are adequate to address the risks. is is a rational—if naive—qu tion for management to pose, yet it is fiendishly difficult to answer witho metrics and, to be frank, still tricky to address even with solid metrics. “A we secure enough?” is arguably the $6 million question, the elephant in t security metrics room.

◾ Are we secure than our peers?more or less Assuming that our organiz tion is, in fact, comparable with industry peers, we don’t want to overspe

Tip: is is a deceptively important chapter, more than just a way to introduce the book. Please don’t speed-read too quickly in an effort to get to the real meat, or you might just miss the target completely. While you consider the questions we raise here, ask yourself other similar questions that are relevant to you and your organization, your management, your information security situation. Better yet, consider the way in which we have phrased the questions as, in so doing, we are subtly framing the problem space. Posing appropriate questions is the real art to information security metrics. Compared to that, answering them is the easy part! If you gain nothing else from this book, we sincerely hope you learn the value of gently guiding your management into posing better questions of information security. If they are questions you can answer, so much the better! And if you are a manager, pose away!

, , p du y p , d p or underspend on security, so they could be our benchmark. On the oth

16  ◾  PRAGMATIC Security Metrics

hand, appearing to be more secure than them may actually be worthwhi in terms of both the brand value of security and in deterring potential adv saries, encouraging them to divert their attentions to our competitors (an by the way, the same point applies in reverse: are our peers truly as secure

they appear to be?). e fact is that the perception of security can be nearly important as the . And if we are in a highly regulated industry subje reality to punitive sanctions, we clearly don’t want to be at the bottom of the he presenting a prime target for enforcement actions.

◾ Which are our strongest and weakest security points? What are the thin we can and should build upon, respectively? Note that there is more to th than just identifying and addressing information security vulnerabiliti Security strengths and capabilities (such as multifactor authentication) c be leveraged to develop new lines of business that would be reckless for o less capable competitors.

◾ What are our biggest security threats or concerns? is is important bo in the present context and in the future, for instance, in connection wi new business ventures or relationships, product lines, processes, or system Depending on the business sector, these threats can differ greatly in terms potential impact.

◾ Are we spending (investing) too much or too little on information sec rity, or do we have it about right? Are we investing wisely, spending on t things that will benefit the organization the most and support its strateg goals? Would additional investment in certain areas be cost-effective (e. enabling business processes or opportunities that would otherwise be t risky), and, if so, which are the preferred security investment options? Wh times are hard, in which areas of information security would it be safest make cutbacks, and, conversely, which ones would we be foolish or ev

reckless to cut? Security metrics can be used both to demonstrate the val of information security and the ongoing investment, two activjustify ties that challenge even the most seasoned and battle-scarred informati security veterans.

◾ Are our security resources allocated optimally? Do we implement the sec rity technologies simply because others do? Have we truly considerdu jour the return on investment for major security investments, such as IDS/IP Are we perhaps missing out on opportunities to implement more cost-effe tive information security controls, such as security awareness and training, generally accepted standards of good security practice?

◾ Have we properly and adequately treated all reasonably foreseeable info mation security risks? Are any nasty surprises lurking around the corn is is a complex question because definitions of “properly and adequatel may be called into question if there are incidents, while “reasonably forese able” gives management plenty of wiggle room to deny its accountability f

able gives management plenty of wiggle room to deny its accountability f poor management decisions.

Why Measure Information Security?  ◾ 

◾ Can we handle compromises, breaches, and other information security in dents effectively and efficiently? What about more serious ones, includi outright disasters? Are we sufficiently well prepared to cope with unknow difficult situations that may arise, or are we operating on a knife edge whe one more serious incidents may tip us over?

◾ Are we (sufficiently) compliant? Are we fully compliant with the obligatio that really matter? While we must comply fully with many of our secur obligations, on some, we may wish to defer full compliance until a mo appropriate time, and for a few, we may make a rational management de sion that it will be less costly to accept the consequences of noncomplian than to implement security controls purely for the sake of compliance; other words, full compliance may not be in the organization’s best interes Will the auditors, regulators, and business partners/customers give us a cle bill of health if they review our information security and related matters, su as risk management, governance, and compliance?

◾ Are we best in class? Are we perhaps overdoing it, or are we lagging the fie in information security? Which parts of our information security manag ment system are performing relatively weakly, and which are leading the wa Would we be able to defend our position on information security to stak holders, the stock markets, and the news media if probed or if a serious sec rity incident occurred? Can we genuinely claim to have done everything w could to secure customer data? As a number of court cases involving a ban responsibility for protecting customers’ accounts have demonstrated, this far from a straightforward issue. Legal decisions have differed significan with nearly identical cases being decided both for and against the bank Having credible metrics demonstrating not only that the organization ha decent set of security controls in place, but that management takes inform tion security seriously enough to invest and take an interest in the measur would, we feel, bolster their case. It will not inspire comfort, confidence, a credibility if the CEO or CIO gets all flustered on the witness stand wh asked basic questions, such as “When was the last time this happened?” “How many times has this kind of incident happened before, exactly?”*

In this context, we are amused to note that fully 43% of the ~10,000 respo dents to the Global State of Information Security 2012† survey consider they a not merely “strategists” but “front-runners” with respect to their approach towa information security (Figure 2.1).

* ere’s a sting in the tail here, however. Management has no excuse for failing to act on se ous security issues that were clearly evident from the metrics. We’ll return to that depress

thought in Chapter 10

thought in Chapter 10. † See PwC (2011).