Security Architecture & Design

www.it-ebooks.info

fl ast.indd 11:57:23:AM 01/17/2014 Page xx

www.it-ebooks.info

ffi rs.indd 12:57:18:PM 01/17/2014 Page i

Threat Modeling

Designing for Security

Adam Shostack

www.it-ebooks.info

ffi rs.indd 12:57:18:PM 01/17/2014 Page ii

Threat Modeling: Designing for Security

Published by John Wiley & Sons, Inc. 10475 Crosspoint BoulevardIndianapolis, IN 46256 www.wiley.com

Copyright © 2014 by Adam Shostack

Published by John Wiley & Sons, Inc., Indianapolis, Indiana Published simultaneously in Canada

ISBN: 978-1-118-80999-0 ISBN: 978-1-118-82269-2 (ebk) ISBN: 978-1-118-81005-7 (ebk)

Manufactured in the United States of America

10 9 8 7 6 5 4 3 2 1

No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permis- sion of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley .com/go/permissions.

Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifi cally disclaim all warranties, including without limitation warranties of fi tness for a particular purpose. No warranty may be created or extended by sales or promotional materials. The advice and strategies contained herein may not be suitable for every situation. This work is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional services. If professional assistance is required, the services of a competent professional person should be sought. Neither the publisher nor the author shall be liable for damages arising herefrom. The fact that an organization or Web site is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organization or website may provide or recommendations it may make. Further, readers should be aware that Internet websites listed in this work may have changed or disappeared between when this work was written and when it is read.

For general information on our other products and services please contact our Customer Care Department within the United States at (877) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002.

Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com.

Library of Congress Control Number: 2013954095

Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affi liates, in the United States and other countries, and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book.

www.it-ebooks.info

ffi rs.indd 12:57:18:PM 01/17/2014 Page iii

For all those striving to deliver more secure systems

www.it-ebooks.info

iv

ffi rs.indd 12:57:18:PM 01/17/2014 Page iv

Credits

Executive Editor Carol Long

Project Editors Victoria Swider Tom Dinse

Technical Editor Chris Wysopal

Production Editor Christine Mugnolo

Copy Editor Luann Rouff

Editorial Manager Mary Beth Wakefi eld

Freelancer Editorial Manager Rosemarie Graham

Associate Director of Marketing David Mayhew

Marketing Manager Ashley Zurcher

Business Manager Amy Knies

Vice President and Executive Group Publisher Richard Swadley

Associate Publisher Jim Minatel

Project Coordinator, Cover Todd Klemme

Technical Proofreader Russ McRee

Proofreader Nancy Carrasco

Indexer Robert Swanson

Cover Image Courtesy of Microsoft

Cover Designer Wiley

www.it-ebooks.info

v

ffi rs.indd 12:57:18:PM 01/17/2014 Page v

Adam Shostack is currently a program manager at Microsoft. His security roles there have included security development processes, usable security, and attack modeling. His attack- modeling work led to security updates for Autorun being delivered to hundreds of millions of computers. He shipped the SDL Threat Modeling Tool and the Elevation of Privilege threat modeling game. While doing security development process work, he delivered threat modeling training across Microsoft and its partners and customers.

Prior to Microsoft, he has been an executive at a number of successful information security and privacy startups. He helped found the CVE, the Privacy Enhancing Technologies Symposium and the International Financial Cryptography Association. He has been a consultant to banks, hospitals and startups and established software companies. For the fi rst several years of his career, he was a systems manager for a medical research lab. Shostack is a prolifi c author, blogger, and public speaker. With Andrew Stewart, he co-authored The New School of Information Security (Addison-Wesley, 2008).

About the Author

www.it-ebooks.info

vi

ffi rs.indd 12:57:18:PM 01/17/2014 Page vi

Chris Wysopal, Veracode’s CTO and Co-Founder, is responsible for the company’s software security analysis capabilities. In 2008 he was named one of InfoWorld’s Top 25 CTO’s and one of the 100 most infl uential people in IT by eWeek. One of the original vulnerability researchers and a member of L0pht Heavy Industries, he has testifi ed on Capitol Hill in the US on the subjects of government computer security and how vulnerabilities are discovered in software. He is an author of L0phtCrack and netcat for Windows. He is the lead author of The Art of Software Security Testing (Addison-Wesley, 2006).

About the Technical Editor

www.it-ebooks.info

vii

ffi rs.indd 12:57:18:PM 01/17/2014 Page vii

First and foremost, I’d like to thank countless engineers at Microsoft and else- where who have given me feedback about their experiences threat modeling. I wouldn’t have had the opportunity to have so many open and direct conversa- tions without the support of Eric Bidstrup and Steve Lipner, who on my fi rst day at Microsoft told me to go “wallow in the problem for a while.” I don’t think either expected “a while” to be quite so long. Nearly eight years later with countless deliverables along the way, this book is my most complete answer to the question they asked me: “How can we get better threat models?”

Ellen Cram Kowalczyk helped me make the book a reality in the Microsoft context, gave great feedback on both details and aspects that were missing, and also provided a lot of the history of threat modeling from the fi rst security pushes through the formation of the SDL, and she was a great manager and mentor. Ellen and Steve Lipner were also invaluable in helping me obtain permission to use Microsoft documents.

The Elevation of Privilege game that opens this book owes much to Jacqueline Beauchere, who saw promise in an ugly prototype called “Threat Spades,” and invested in making it beautiful and widely available.

The SDL Threat Modeling Tool might not exist if Chris Peterson hadn’t given me a chance to build a threat modeling tool for the Windows team to use. Ivan Medvedev, Patrick McCuller, Meng Li, and Larry Osterman built the fi rst version of that tool. I’d like to thank the many engineers in Windows, and later across Microsoft, who provided bug reports and suggestions for improvements in the beta days, and acknowledge all those who just fl amed at us, reminding us of the importance of getting threat modeling right. Without that tool, my experience and breadth in threat modeling would be far poorer.

Larry Osterman, Douglas MacIver, Eric Douglas, Michael Howard, and Bob Fruth gave me hours of their time and experience in understanding threat

Acknowledgments

www.it-ebooks.info

viii Acknowledgments

ffi rs.indd 12:57:18:PM 01/17/2014 Page viii

modeling at Microsoft. Window Snyder’s perspective as I started the Microsoft job has been invaluable over the years. Knowing when you’re done . . . well, this book is nearly done.

Rob Reeder was a great guide to the fi eld of usable security, and Chapter 15 would look very different if not for our years of collaboration. I can’t discuss usable security without thanking Lorrie Cranor for her help on that topic; but also for the chance to keynote the Symposium on Usable Privacy and Security, which led me to think about usable engineering advice, a perspective that is now suffused throughout this book.

Andy Stiengrubel, Don Ankney, and Russ McRee all taught me important lessons related to operational threat modeling, and how the trade-offs change as you change context. Guys, thank you for beating on me—those lessons now permeate many chapters. Alec Yasinac, Harold Pardue, and Jeff Landry were generous with their time discussing their attack tree experience, and Chapters 4 and 17 are better for those conversations. Joseph Lorenzo Hall was also a gem in helping with attack trees. Wendy Nather argued strongly that assets and attackers are great ways to make threats real, and thus help overcome resistance to fi xing them. Rob Sama checked the Acme fi nancials example from a CPA’s perspective, correcting many of my errors. Dave Awksmith graciously allowed me to include his threat personas as a complete appendix. Jason Nehrboss gave me some of the best feedback I’ve ever received on very early chapters.

I’d also like to acknowledge Jacob Appelbaum, Crispin Cowan, Dana Epp (for years of help, on both the book and tools), Jeremi Gosney, Yoshi Kohno, David LeBlanc, Marsh Ray, Nick Mathewson, Tamara McBride, Russ McRee, Talhah Mir, David Mortman, Alec Muffet, Ben Rothke, Andrew Stewart, and Bryan Sullivan for helpful feedback on drafts and/or ideas that made it into the book in a wide variety of ways.

Of course, none of those acknowledged in this section are responsible for the errors which doubtless crept in or remain.

Writing this book “by myself” (an odd phrase given everyone I’m acknowl- edging) makes me miss working with Andrew Stewart, my partner in writing on The New School of Information Security. Especially since people sometimes attribute that book to me, I want to be public about how much I missed his collaboration in this project.

This book wouldn’t be in the form it is were it not for Bruce Schneier’s will- ingness to make an introduction to Carol Long, and Carol’s willingness to pick up the book. It wasn’t always easy to read the feedback and suggested changes from my excellent project editor, Victoria Swider, but this thing is better where I did. Tom Dinse stepped in as the project ended and masterfully took control of a very large number of open tasks, bringing them to resolution on a tight schedule.

Lastly, and most importantly, thank you to Terri, for all your help, support, and love, and for putting up with “it’s almost done” for a very, very long time.

—Adam Shostack

www.it-ebooks.info

ix

ftoc.indd 11:56:7:AM 01/17/2014 Page ix

Introduction xxi

Part I Getting Started 1

Chapter 1 Dive In and Threat Model! 3 Learning to Threat Model 4

What Are You Building? 5 What Can Go Wrong? 7 Addressing Each Threat 12 Checking Your Work 24

Threat Modeling on Your Own 26 Checklists for Diving In and Threat Modeling 27 Summary 28

Chapter 2 Strategies for Threat Modeling 29 “What’s Your Threat Model?” 30 Brainstorming Your Threats 31

Brainstorming Variants 32 Literature Review 33 Perspective on Brainstorming 34

Structured Approaches to Threat Modeling 34 Focusing on Assets 36 Focusing on Attackers 40 Focusing on Software 41

Models of Software 43 Types of Diagrams 44 Trust Boundaries 50 What to Include in a Diagram 52 Complex Diagrams 52 Labels in Diagrams 53

Contents

www.it-ebooks.info

x Contents

ftoc.indd 11:56:7:AM 01/17/2014 Page x

Color in Diagrams 53 Entry Points 53 Validating Diagrams 54

Summary 56

Part II Finding Threats 59

Chapter 3 STRIDE 61 Understanding STRIDE and Why It’s Useful 62 Spoofing Threats 64

Spoofing a Process or File on the Same Machine 65 Spoofing a Machine 66 Spoofing a Person 66

Tampering Threats 67 Tampering with a File 68 Tampering with Memory 68 Tampering with a Network 68

Repudiation Threats 68 Attacking the Logs 69 Repudiating an Action 70

Information Disclosure Threats 70 Information Disclosure from a Process 71 Information Disclosure from a Data Store 71 Information Disclosure from a Data Flow 72

Denial-of-Service Threats 72 Elevation of Privilege Threats 73

Elevate Privileges by Corrupting a Process 74 Elevate Privileges through Authorization Failures 74

Extended Example: STRIDE Threats against Acme-DB 74 STRIDE Variants 78

STRIDE-per-Element 78 STRIDE-per-Interaction 80 DESIST 85

Exit Criteria 85 Summary 85

Chapter 4 Attack Trees 87 Working with Attack Trees 87

Using Attack Trees to Find Threats 88 Creating New Attack Trees 88

Representing a Tree 91 Human-Viewable Representations 91 Structured Representations 94

Example Attack Tree 94 Real Attack Trees 96

Fraud Attack Tree 96 Election Operations Assessment Threat Trees 96 Mind Maps 98

www.it-ebooks.info

Contents xi

ftoc.indd 11:56:7:AM 01/17/2014 Page xi

Perspective on Attack Trees 98 Summary 100

Chapter 5 Attack Libraries 101 Properties of Attack Libraries 101

Libraries and Checklists 103 Libraries and Literature Reviews 103

CAPEC 104 Exit Criteria 106 Perspective on CAPEC 106

OWASP Top Ten 108 Summary 108

Chapter 6 Privacy Tools 111 Solove’s Taxonomy of Privacy 112 Privacy Considerations for

Internet Protocols 114 Privacy Impact Assessments (PIA) 114 The Nymity Slider and the Privacy Ratchet 115 Contextual Integrity 117

Contextual Integrity Decision Heuristic 118 Augmented Contextual Integrity Heuristic 119 Perspective on Contextual Integrity 119

LINDDUN 120 Summary 121

Part III Managing and Addressing Threats 123

Chapter 7 Processing and Managing Threats 125 Starting the Threat Modeling Project 126

When to Threat Model 126 What to Start and (Plan to) End With 128 Where to Start 128

Digging Deeper into Mitigations 130 The Order of Mitigation 131 Playing Chess 131 Prioritizing 132 Running from the Bear 132

Tracking with Tables and Lists 133 Tracking Threats 133 Making Assumptions 135 External Security Notes 136

Scenario-Specifi c Elements of Threat Modeling 138

Customer/Vendor Trust Boundary 139 New Technologies 139 Threat Modeling an API 141

Summary 143

www.it-ebooks.info

xii Contents

ftoc.indd 11:56:7:AM 01/17/2014 Page xii

Chapter 8 Defensive Tactics and Technologies 145 Tactics and Technologies for Mitigating Threats 145

Authentication: Mitigating Spoofi ng 146 Integrity: Mitigating Tampering 148 Non-Repudiation: Mitigating Repudiation 150 Confi dentiality: Mitigating Information Disclosure 153 Availability: Mitigating Denial of Service 155 Authorization: Mitigating Elevation of Privilege 157 Tactic and Technology Traps 159

Addressing Threats with Patterns 159 Standard Deployments 160 Addressing CAPEC Threats 160

Mitigating Privacy Threats 160 Minimization 160 Cryptography 161 Compliance and Policy 164

Summary 164

Chapter 9 Trade-Off s When Addressing Threats 167 Classic Strategies for Risk Management 168

Avoiding Risks 168 Addressing Risks 168 Accepting Risks 169 Transferring Risks 169 Ignoring Risks 169

Selecting Mitigations for Risk Management 170 Changing the Design 170 Applying Standard Mitigation Technologies 174 Designing a Custom Mitigation 176 Fuzzing Is Not a Mitigation 177

Threat-Specifi c Prioritization Approaches 178 Simple Approaches 178 Threat-Ranking with a Bug Bar 180 Cost Estimation Approaches 181

Mitigation via Risk Acceptance 184 Mitigation via Business Acceptance 184 Mitigation via User Acceptance 185

Arms Races in Mitigation Strategies 185 Summary 186

Chapter 10 Validating That Threats Are Addressed 189 Testing Threat Mitigations 190

Test Process Integration 190 How to Test a Mitigation 191 Penetration Testing 191

www.it-ebooks.info

Contents xiii

ftoc.indd 11:56:7:AM 01/17/2014 Page xiii

Checking Code You Acquire 192 Constructing a Software Model 193 Using the Software Model 194

QA’ing Threat Modeling 195 Model/Reality Conformance 195 Task and Process Completion 196 Bug Checking 196

Process Aspects of Addressing Threats 197 Threat Modeling Empowers Testing;

Testing Empowers Threat Modeling 197 Validation/Transformation 197 Document Assumptions as You Go 198

Tables and Lists 198 Summary 202

Chapter 11 Threat Modeling Tools 203 Generally Useful Tools 204

Whiteboards 204 Offi ce Suites 204 Bug-Tracking Systems 204

Open-Source Tools 206 TRIKE 206 SeaMonster 206 Elevation of Privilege 206

Commercial Tools 208 ThreatModeler 208 Corporate Threat Modeller 208 SecurITree 209 Little-JIL 209 Microsoft’s SDL Threat Modeling Tool 209

Tools That Don’t Exist Yet 213 Summary 213

Part IV Threat Modeling in Technologies and Tricky Areas 215

Chapter 12 Requirements Cookbook 217 Why a “Cookbook”? 218 The Interplay of Requirements, Threats,

and Mitigations 219 Business Requirements 220

Outshining the Competition 220 Industry Requirements 220 Scenario-Driven Requirements 221

Prevent/Detect/Respond as a Frame for Requirements 221

Prevention 221 Detection 225

www.it-ebooks.info

xiv Contents

ftoc.indd 11:56:7:AM 01/17/2014 Page xiv

Response 225 People/Process/Technology as a Frame

for Requirements 227 People 227 Process 228 Technology 228

Development Requirements vs. Acquisition Requirements 228 Compliance-Driven Requirements 229

Cloud Security Alliance 229 NIST Publication 200 230 PCI-DSS 231

Privacy Requirements 231 Fair Information Practices 232 Privacy by Design 232 The Seven Laws of Identity 233 Microsoft Privacy Standards for Development 234

The STRIDE Requirements 234 Authentication 235 Integrity 236 Non-Repudiation 237 Confi dentiality 238 Availability 238 Authorization 239

Non-Requirements 240 Operational Non-Requirements 240 Warnings and Prompts 241 Microsoft’s “10 Immutable Laws” 241

Summary 242

Chapter 13 Web and Cloud Threats 243 Web Threats 243

Website Threats 244 Web Browser and Plugin Threats 244

Cloud Tenant Threats 246 Insider Threats 246 Co-Tenant Threats 247 Threats to Compliance 247 Legal Threats 248 Threats to Forensic Response 248 Miscellaneous Threats 248

Cloud Provider Threats 249 Threats Directly from Tenants 249 Threats Caused by Tenant Behavior 250

Mobile Threats 250 Summary 251

www.it-ebooks.info

Contents xv

ftoc.indd 11:56:7:AM 01/17/2014 Page xv

Chapter 14 Accounts and Identity 253 Account Life Cycles 254

Account Creation 254 Account Maintenance 257 Account Termination 258 Account Life-Cycle Checklist 258

Authentication 259 Login 260 Login Failures 262 Threats to “What You Have” 263 Threats to “What You Are” 264 Threats to “What You Know” 267 Authentication Checklist 271

Account Recovery 271 Time and Account Recovery 272 E-mail for Account Recovery 273 Knowledge-Based Authentication 274 Social Authentication 278 Attacker-Driven Analysis of

Account Recovery 280 Multi-Channel Authentication 281 Account Recovery Checklist 281

Names, IDs, and SSNs 282 Names 282 Identity Documents 285 Social Security Numbers and Other

National Identity Numbers 286 Identity Theft 289 Names, IDs, and SSNs Checklist 290

Summary 290

Chapter 15 Human Factors and Usability 293 Models of People 294

Applying Behaviorist Models of People 295 Cognitive Science Models of People 297 Heuristic Models of People 302

Models of Software Scenarios 304 Modeling the Software 304 Diagramming for Modeling the Software 307 Modeling Electronic Social Engineering Attacks 309

Threat Elicitation Techniques 311 Brainstorming 311 The Ceremony Approach to Threat Modeling 311 Ceremony Analysis Heuristics 312 Integrating Usability into the Four-Stage Framework 315

www.it-ebooks.info

xvi Contents

ftoc.indd 11:56:7:AM 01/17/2014 Page xvi

Tools and Techniques for Addressing Human Factors 316

Myths That Inhibit Human Factors Work 317 Design Patterns for Good Decisions 317 Design Patterns for a Kind Learning

Environment 320 User Interface Tools and Techniques 322

Confi guration 322 Explicit Warnings 323 Patterns That Grab Attention 325

Testing for Human Factors 327 Benign and Malicious Scenarios 328 Ecological Validity 328

Perspective on Usability and Ceremonies 329 Summary 331

Chapter 16 Threats to Cryptosystems 333 Cryptographic Primitives 334

Basic Primitives 334 Privacy Primitives 339 Modern Cryptographic Primitives 339

Classic Threat Actors 341 Attacks against Cryptosystems 342 Building with Crypto 346

Making Choices 346 Preparing for Upgrades 346 Key Management 346 Authenticating before Decrypting 348

Things to Remember about Crypto 348 Use a Cryptosystem Designed by Professionals 348 Use Cryptographic Code Built and Tested by Professionals 348 Cryptography Is Not Magic Security Dust 349 Assume It Will All Become Public 349 You Still Need to Manage Keys 349

Secret Systems: Kerckhoffs and His Principles 349 Summary 351

Part V Taking It to the Next Level 353

Chapter 17 Bringing Threat Modeling to Your Organization 355 How To Introduce Threat Modeling 356

Convincing Individual Contributors 357 Convincing Management 358

Who Does What? 359 Threat Modeling and Project Management 359

www.it-ebooks.info

Contents xvii

ftoc.indd 11:56:7:AM 01/17/2014 Page xvii

Prerequisites 360 Deliverables 360 Individual Roles and Responsibilities 362 Group Interaction 363 Diversity in Threat Modeling Teams 367

Threat Modeling within a Development Life Cycle 367 Development Process Issues 368 Organizational Issues 373 Customizing a Process for Your Organization 378

Overcoming Objections to Threat Modeling 379 Resource Objections 379 Value Objections 380 Objections to the Plan 381

Summary 383

Chapter 18 Experimental Approaches 385 Looking in the Seams 386 Operational Threat Models 387

FlipIT 388 Kill Chains 388

The “Broad Street” Taxonomy 392 Adversarial Machine Learning 398 Threat Modeling a Business 399 Threats to Threat Modeling Approaches 400

Dangerous Deliverables 400 Enumerate All Assumptions 400 Dangerous Approaches 402

How to Experiment 404 Defi ne a Problem 404 Find Aspects to Measure and Measure Them 404 Study Your Results 405

Summary 405

Chapter 19 Architecting for Success 407 Understanding Flow 407

Flow and Threat Modeling 409 Stymieing People 411 Beware of Cognitive Load 411 Avoid Creator Blindness 412 Assets and Attackers 412

Knowing the Participants 413 Boundary Objects 414 The Best Is the Enemy of the Good 415 Closing Perspectives 416

“The Threat Model Has Changed” 417

www.it-ebooks.info

xviii Contents

ftoc.indd 11:56:7:AM 01/17/2014 Page xviii

On Artistry 418 Summary 419

Now Threat Model 420

Appendix A Helpful Tools 421 Common Answers to “What’s Your Threat Model?” 421

Network Attackers 421 Physical Attackers 422 Attacks against People 423 Supply Chain Attackers 423 Privacy Attackers 424 Non-Sentient “Attackers” 424 The Internet Threat Model 424 Assets 425 Computers as Assets 425 People as Assets 426 Processes as Assets 426 Intangible Assets 427 Stepping-Stone Assets 427

Appendix B Threat Trees 429 STRIDE Threat Trees 430

Spoofi ng an External Entity (Client/ Person/Account) 432 Spoofi ng a Process 438 Spoofi ng of a Data Flow 439 Tampering with a Process 442 Tampering with a Data Flow 444 Tampering with a Data Store 446 Repudiation against a Process (or by an External Entity) 450 Repudiation, Data Store 452 Information Disclosure from a Process 454 Information Disclosure from a Data Flow 456 Information Disclosure from a Data Store 459 Denial of Service against a Process 462 Denial of Service against a Data Flow 463 Denial of Service against a Data Store 466 Elevation of Privilege against a Process 468

Other Threat Trees 470 Running Code 471 Attack via a “Social” Program 474 Attack with Tricky Filenames 476

Appendix C Attacker Lists 477 Attacker Lists 478

Barnard’s List 478 Verizon’s Lists 478 OWASP 478 Intel TARA 479

www.it-ebooks.info

Contents xix

ftoc.indd 11:56:7:AM 01/17/2014 Page xix

Personas and Archetypes 480 Aucsmith’s Attacker Personas 481 Background and Defi nitions 481 Personas 484

David “Ne0phyate” Bradley – Vandal 484 JoLynn “NightLily” Dobney – Trespasser 486 Sean “Keech” Purcell – Defacer 488 Bryan “CrossFyre” Walton – Author 490 Lorrin Smith-Bates – Insider 492 Douglas Hite – Thief 494 Mr. Smith – Terrorist 496 Mr. Jones – Spy 498

Appendix D Elevation of Privilege: The Cards 501 Spoofi ng 501 Tampering 503 Repudiation 504 Information Disclosure 506 Denial of Service 507 Elevation of Privilege (EoP) 508

Appendix E Case Studies 511 The Acme Database 512

Security Requirements 512 Software Model 512 Threats and Mitigations 513

Acme’s Operational Network 519 Security Requirements 519 Operational Network 520 Threats to the Network 521

Phones and One-Time Token Authenticators 525 The Scenario 526 The Threats 527 Possible Redesigns 528

Sample for You to Model 528 Background 529 The iNTegrity Data Flow Diagrams 530 Exercises 531

Glossary 533

Bibliography 543

Index 567

www.it-ebooks.info

fl ast.indd 11:57:23:AM 01/17/2014 Page xx

www.it-ebooks.info

xxi

fl ast.indd 11:57:23:AM 01/17/2014 Page xxi

All models are wrong, some models are useful. — George Box

This book describes the useful models you can employ to address or mitigate these potential threats. People who build software, systems, or things with software need to address the many predictable threats their systems can face.

Threat modeling is a fancy name for something we all do instinctively. If I asked you to threat model your house, you might start by thinking about the precious things within it: your family, heirlooms, photos, or perhaps your collec- tion of signed movie posters. You might start thinking about the ways someone might break in, such as unlocked doors or open windows. And you might start thinking about the sorts of people who might break in, including neighborhood kids, professional burglars, drug addicts, perhaps a stalker, or someone trying to steal your Picasso original.

Each of these examples has an analog in the software world, but for now, the important thing is not how you guard against each threat, but that you’re able to relate to this way of thinking. If you were asked to help assess a friend’s house, you could probably help, but you might lack confi dence in how complete your analysis is. If you were asked to secure an offi ce complex, you might have a still harder time, and securing a military base or a prison seems even more diffi cult. In those cases, your instincts are insuffi cient, and you’d need tools to help tackle the questions. This book will give you the tools to think about threat modeling technology in structured and effective ways.

In this introduction, you’ll learn about what threat modeling is and why indi- viduals, teams, and organizations threat model. Those reasons include fi nding security issues early, improving your understanding of security requirements, and being able to engineer and deliver better products. This introduction has

Introduction

www.it-ebooks.info

xxii Introduction

fl ast.indd 11:57:23:AM 01/17/2014 Page xxii

fi ve main sections describing what the book is about, including a defi nition of threat modeling and reasons it’s important; who should read this book; how to use it, and what you can expect to gain from the various parts, and new lessons in threat modeling.

What Is Threat Modeling?

Everyone threat models. Many people do it out of frustration in line at the airport, sneaking out of the house or into a bar. At the airport, you might idly consider how to sneak something through security, even if you have no intent to do so. Sneaking in or out of someplace, you worry about who might catch you. When you speed down the highway, you work with an implicit threat model where the main threat is the police, who you probably think are lurking behind a billboard or overpass. Threats of road obstructions, deer, or rain might play into your model as well.

When you threat model, you usually use two types of models. There’s a model of what you’re building, and there’s a model of the threats (what can go wrong). What you’re building with software might be a website, a downloadable program or app, or it might be delivered in a hardware package. It might be a distributed system, or some of the “things” that will be part of the “Internet of things.” You model so that you can look at the forest, not the trees. A good model helps you address classes or groups of attacks, and deliver a more secure product.

The English word threat has many meanings. It can be used to describe a person, such as “Osama bin Laden was a threat to America,” or people, such as “the insider threat.” It can be used to describe an event, such as “There is a threat of a hurricane coming through this weekend,” and it can be used to describe a weakness or possibility of attack, such as “What are you doing about confi dentiality threats?” It is also used to describe viruses and malware such as “This threat incorporates three different methods for spreading.” It can be used to describe behavior such as “There’s a threat of operator error.”

Similarly, the term threat modeling has many meanings, and the term threat model is used in many distinct and perhaps incompatible ways, including:

■ As a verb—for example, “Have you threat modeled?” That is, have you gone through an analysis process to fi gure out what might go wrong with the thing you’re building?

■ As a noun, to ask what threat model is being used. For example, “Our threat model is someone in possession of the machine,” or “Our threat model is a skilled and determined remote attacker.”

■ It can mean building up a set of idealized attackers.

■ It can mean abstracting threats into classes such as tampering.

There are doubtless other defi nitions. All of these are useful in various sce- narios and thus correct, and there are few less fruitful ways to spend your time

www.it-ebooks.info

Introduction xxiii

fl ast.indd 11:57:23:AM 01/17/2014 Page xxiii

than debating them. Arguing over defi nitions is a strange game, and the only way to win is not to play. This book takes a big tent approach to threat model- ing and includes a wide range of techniques you can apply early to make what you’re designing or building more secure. It will also address the reality that some techniques are more effective than others, and that some techniques are more likely to work for people with particular skills or experience.

Threat modeling is the key to a focused defense. Without threat models, you can never stop playing whack-a-mole.

In short, threat modeling is the use of abstractions to aid in thinking about risks.

Reasons to Threat Model

In today’s fast-paced world, there is a tendency to streamline development activ- ity, and there are important reasons to threat model, which are covered in this section. Those include fi nding security bugs early, understanding your security requirements, and engineering and delivering better products.

Find Security Bugs Early

If you think about building a house, decisions you make early will have dramatic effects on security. Wooden walls and lots of ground-level windows expose you to more risks than brick construction and few windows. Either may be a reason- able choice, depending on where you’re building and other factors. Once you’ve chosen, changes will be expensive. Sure, you can put bars over your windows, but wouldn’t it be better to use a more appropriate design from the start? The same sorts of tradeoffs can apply in technology. Threat modeling will help you fi nd design issues even before you’ve written a line of code, and that’s the best time to fi nd those issues.

Understand Your Security Requirements

Good threat models can help you ask “Is that really a requirement?” For example, does the system need to be secure against someone in physical possession of the device? Apple has said yes for the iPhone, which is different from the tradi- tional world of the PC. As you fi nd threats and triage what you’re going to do with them, you clarify your requirements. With more clear requirements, you can devote your energy to a consistent set of security features and properties.

There is an important interplay between requirements, threats, and mitiga- tions. As you model threats, you’ll fi nd that some threats don’t line up with your business requirements, and as such may not be worth addressing. Alternately, your requirements may not be complete. With other threats, you’ll fi nd that addressing them is too complex or expensive. You’ll need to make a call between

www.it-ebooks.info

xxiv Introduction

fl ast.indd 11:57:23:AM 01/17/2014 Page xxiv

addressing them partially in the current version or accepting (and communicat- ing) that you can’t address those threats.

Engineer and Deliver Better Products

By considering your requirements and design early in the process, you can dramatically lower the odds that you’ll be re-designing, re-factoring, or facing a constant stream of security bugs. That will let you deliver a better product on a more predictable schedule. All the effort that would go to those can be put into building a better, faster, cheaper or more secure product. You can focus on whatever properties your customers want.

Address Issues Other Techniques Won’t

The last reason to threat model is that threat modeling will lead you to catego- ries of issues that other tools won’t fi nd. Some of these issues will be errors of omission, such as a failure to authenticate a connection. That’s not something that a code analysis tool will fi nd. Other issues will be unique to your design. To the extent that you have a set of smart developers building something new, you might have new ways threats can manifest. Models of what goes wrong, by abstracting away details, will help you see analogies and similarities to problems that have been discovered in other systems.

A corollary of this is that threat modeling should not focus on issues that your other safety and security engineering is likely to fi nd (except insofar as fi nding them early lets you avoid re-engineering). So if, for example, you’re building a product with a database, threat modeling might touch quickly on SQL injection attacks, and the variety of trust boundaries that might be injectable. However, you may know that you’ll encounter those. Your threat modeling should focus on issues that other techniques can’t fi nd.

Who Should Read This book?

This book is written for those who create or operate complex technology. That’s primarily software engineers and systems administrators, but it also includes a variety of related roles, including analysts or architects. There’s also a lot of information in here for security professionals, so this book should be useful to them and those who work with them. Different parts of the book are designed for different people—in general, the early chapters are for generalists (or special- ists in something other than security), while the end of the book speaks more to security specialists.

www.it-ebooks.info

Introduction xxv

fl ast.indd 11:57:23:AM 01/17/2014 Page xxv

You don’t need to be a security expert, professional, or even enthusiast to get substantial benefi t from this book. I assume that you understand that there are people out there whose interests and desires don’t line up with yours. For example, maybe they’d like to take money from you, or they may have other goals, like puffi ng themselves up at your expense or using your computer to attack other people.

This book is written in plain language for anyone who can write or spec a program, but sometimes a little jargon helps in precision, conciseness, or clarity, so there’s a glossary.

What You Will Gain from This Book

When you read this book cover to cover, you will gain a rich knowledge of threat modeling techniques. You’ll learn to apply those techniques to your projects so you can build software that’s more secure from the get-go, and deploy it more securely. You’ll learn to how to make security tradeoffs in ways that are considered, measured, and appropriate. You will learn a set of tools and when to bring them to bear. You will discover a set of glamorous distractions. Those distractions might seem like wonderful, sexy ideas, but they hide an ugly inte- rior. You’ll learn why they prevent you from effectively threat modeling, and how to avoid them.

You’ll also learn to focus on the actionable outputs of threat modeling, and I’ll generally call those “bugs.” There are arguments that it’s helpful to consider code issues as bugs, and design issues as fl aws. In my book, those arguments are a distraction; you should threat model to fi nd issues that you can address, and arguing about labels probably doesn’t help you address them.

Lessons for Diff erent Readers

This book is designed to be useful to a wide variety of people working in tech- nology. That includes a continuum from those who develop software to those who combine it into systems that meet operational or business goals to those who focus on making it more secure.

For convenience, this book pretends there is a bright dividing line between development and operations. The distinction is used as a way of understand- ing who has what capabilities, choices, and responsibilities. For example, it is “easy” for a developer to change what is logged, or to implement a different authentication system. Both of these may be hard for operations. Similarly, it’s “easy” for operations to ensure that logs are maintained, or to ensure that a computer is in a locked cage. As this book was written, there’s also an important

www.it-ebooks.info

xxvi Introduction

fl ast.indd 11:57:23:AM 01/17/2014 Page xxvi

model of “devops” emerging. The lessons for developers and operations can likely be applied with minor adjustments. This book also pretends that security expertise is separate from either development or operations expertise, again, simply as a convenience.

Naturally, this means that the same parts of the book will bring different les- sons for different people. The breakdown below gives a focused value proposi- tion for each audience.

Software Developers and Testers

Software developers—those whose day jobs are focused on creating software— include software engineers, quality assurance, and a variety of program or project managers. If you’re in that group, you will learn to fi nd and address design issues early in the software process. This book will enable you to deliver more secure software that better meets customer requirements and expecta- tions. You’ll learn a simple, effective and fun approach to threat modeling, as well as different ways to model your software or fi nd threats. You’ll learn how to track threats with bugs that fi t into your development process. You’ll learn to use threats to help make your requirements more crisp, and vice versa. You’ll learn about areas such as authentication, cryptography, and usability where the interplay of mitigations and attacks has a long history, so you can understand how the recommended approaches have developed to their current state. You’ll learn about how to bring threat modeling into your development process. And a whole lot more!

Systems Architecture, Operations, and Management

For those whose day jobs involve bringing together software components, weaving them together into systems to deliver value, you’ll learn to fi nd and address threats as you design your systems, select your components, and get them ready for deployment. This book will enable you to deliver more secure systems that better meet business, customer, and compliance requirements. You’ll learn a simple, effective, and fun approach to threat modeling, as well as different ways to model the systems you’re building or have built. You’ll learn how to fi nd security and privacy threats against those systems. You’ll learn about the building blocks which are available for you to operationally address those threats. You’ll learn how to make tradeoffs between the threats you face, and how to ensure that those threats are addressed. You’ll learn about specifi c threats to categories of technology, such as web and cloud systems, and about threats to accounts, both of which are deeply important to those in operations. It will cover issues of usability, and perhaps even change your perspective on

www.it-ebooks.info

Introduction xxvii

fl ast.indd 11:57:23:AM 01/17/2014 Page xxvii

how to infl uence the security behavior of people within your organization and/ or your customers. You will learn about cryptographic building blocks, which you may be using to protect systems. And a whole lot more!

Security Professionals

If you work in security, you will learn two major things from this book: First, you’ll learn structured approaches to threat modeling that will enhance your productivity, and as you do, you’ll learn why many of the “obvious” parts of threat modeling are not as obvious, or as right, as you may have believed. Second, you’ll learn about bringing security into the development, operational and release processes that your organization uses.

Even if you are an expert, this book can help you threat model better. Here, I speak from experience. As I was writing the case study appendix, I found myself turning to both the tree in Appendix B and the requirements chapter, and fi nding threats that didn’t spring to mind from just considering the models of software.

TO MY COLLEAGUES IN INFORMATION SECURITY

I want to be frank. This book is not about how to design abstractly perfect software. It is a practical, grounded book that acknowledges that most software is built in some business or organizational reality that requires tradeoff s. To the dismay of purists, software where tradeoff s were made runs the world these days, and I’d like to make such software more secure by making those tradeoff s better. That involves a great many elements, two of which are making security more consistent and more acces- sible to our colleagues in other specialties.

This perspective is grounded in my time as a systems administrator, deploying security technologies, and observing the issues people encountered. It is grounded in my time as a startup executive, learning to see security as a property of a system which serves a business goal. It is grounded in my responsibility for threat model- ing as part of Microsoft’s Security Development Lifecycle. In that last role, I spoke with thousands of people at Microsoft, its partners, and its customers about our approaches. These individuals ranged from newly hired developers to those with decades of experience in security, and included chief security offi cers and Microsoft’s Trustworthy Computing Academic Advisory Board. I learned that there are an awful lot of opinions about what works, and far fewer about what does not. This book aims to convince my fellow security professionals that pragmatism in what we ask of devel- opment and operations helps us deliver more secure software over time. This perspec- tive may be a challenge for some security professionals. They should focus on Parts II, IV, and V, and perhaps give consideration to the question of the best as the enemy of the good.

www.it-ebooks.info

xxviii Introduction

fl ast.indd 11:57:23:AM 01/17/2014 Page xxviii

How To Use This Book

You should start at the very beginning. It’s a very good place to start, even if you already know how to threat model, because it lays out a framework that will help you understand the rest of the book.

The Four-Step Framework

This book introduces the idea that you should see threat modeling as composed of steps which accomplish subgoals, rather than as a single activity. The essential questions which you ask to accomplish those subgoals are:

1. What are you building?

2. What can go wrong with it once it’s built?

3. What should you do about those things that can go wrong?

4. Did you do a decent job of analysis?

The methods you use in each step of the framework can be thought of like Lego blocks. When working with Legos, you can snap in other Lego blocks. In Chapter 1, you’ll use a data fl ow diagram to model what you’re building, STRIDE to help you think about what can go wrong and what you should do about it, and a checklist to see if you did a decent job of analysis. In Chapter 2, you’ll see how diagrams are the most helpful way to think about what you’re building. Different diagram types are like different building blocks to help you model what you’re building. In Chapter 3, you’ll go deep into STRIDE (a model of threats), while in Chapter 4, you’ll learn to use attack trees instead of STRIDE, while leaving everything else the same. STRIDE and attack trees are different building blocks for considering what can go wrong once you’ve built your new technology.

Not every approach can snap with every other approach. It takes crazy glue to make an Erector set and Lincoln logs stick together. Attempts to glue threat modeling approaches together has made for some confusing advice. For example, trying to consider how terrorists would attack your assets doesn’t really lead to a lot of actionable issues. And even with building blocks that snap together, you can make something elegant, or something confusing or bizarre.

So to consider this as a framework, what are the building blocks? The four- step framework is shown graphically in Figure I-1.

The steps are:

1. Model the system you’re building, deploying, or changing.

2. Find threats using that model and the approaches in Part II.

www.it-ebooks.info

Introduction xxix

fl ast.indd 11:57:23:AM 01/17/2014 Page xxix

3. Address threats using the approaches in Part III.

4. Validate your work for completeness and effectiveness (also Part III).

Model System

Find Threats

Address Threats

Validate

Figure I-1: The Four-Step Framework

This framework was designed to align with software development and opera- tional deployment. It has proven itself as a way to structure threat modeling. It also makes it easier to experiment without replacing the entire framework. From here until you reach Part V, almost everything you encounter is selected because it plugs into this four-step framework.

This book is roughly organized according to the framework: Part I “Getting Started” is about getting started. The opening part of the book

(especially Chapter 1) is designed for those without much security expertise. The later parts of the book build on the security knowledge you’ll gain from this material (or combined with your own experience). You’ll gain an understanding of threat modeling, and a recommended approach for those who are new to the discipline. You’ll also learn various ways to model your software, along with why that’s a better place to start than other options, such as attackers or assets.

Part II “Finding Threats” is about fi nding threats. It presents a collection of techniques and tools you can use to fi nd threats. It surveys and analyzes the dif- ferent ways people approach information technology threat modeling, enabling you to examine the pros and cons of the various techniques that people bring to bear. They’re grouped in a way that enables you to either read them from start to fi nish or jump in at a particular point where you need help.

Part III “Managing and Addressing Threats” is about managing and address- ing threats. It includes processing threats and how to manage them, the tactics and technologies you can use to address them, and how to make risk tradeoffs

www.it-ebooks.info

xxx Introduction

fl ast.indd 11:57:23:AM 01/17/2014 Page xxx

you might need to make. It also covers validating that your threats are addressed, and tools you can use to help you threat model.

Part IV “Threat Modeling in Technologies and Tricky Areas” is about threat modeling in specifi c technologies and tricky areas where a great deal of threat modeling and analysis work has already been done. It includes chapters on web and cloud, accounts and identity, and cryptography, as well as a requirements “cookbook” that you can use to jump-start your own security requirements analysis.

Part V “Taking it to the Next Level” is about taking threat modeling to the next level. It targets the experienced threat modeler, security expert, or process designer who is thinking about how to build and customize threat modeling processes for a given organization.

Appendices include information to help you apply what you’ve learned. They include sets of common answers to “what’s your threat model,” and “what are our assets”; as well as threat trees that can help you fi nd threats, lists of attackers and attacker personas; and details about the Elevation of Privilege game you’ll use in Chapter 1; and lastly, a set of detailed example threat models. These are followed by a glossary, bibliography, and index.

Website: This book’s website, www.threatmodelingbook.com will contain a PDF of some of the fi gures in the book, and likely an errata list to mitigate the errors that inevitably threaten to creep in.

What This Book Is Not

Many security books today promise to teach you to hack. Their intent is to teach you what sort of attacks need to be defended against. The idea is that if you have an empirically grounded set of attacks, you can start with that to create your defense. This is not one of those books, because despite millions of such books being sold, vulnerable systems are still being built and deployed. Besides, there are solid, carefully considered defenses against many sorts of attacks. It may be useful to know how to execute an attack, but it’s more important to know where each attack might be executed, and how to effectively defend against it. This book will teach you that.

This book is not focused on a particular technology, platform, or API set. Platforms and APIs infl uence may offer security features you can use, or mitigate some threats for you. The threats and mitigations associated with a platform change from release to release, and this book aims to be a useful reference volume on your shelf for longer than the release of any particular technology.

This book is not a magic pill that will make you a master of threat modeling. It is a resource to help you understand what you need to know. Practice will

www.it-ebooks.info

Introduction xxxi

fl ast.indd 11:57:23:AM 01/17/2014 Page xxxi

help you get better, and deliberative practice with feedback and hard problems will make you a master.

New Lessons on Threat Modeling

Most experienced security professionals have developed an approach to threat modeling that works for them. If you’ve been threat modeling for years, this book will give you an understanding of other approaches you can apply. This book also give you a structured understanding of a set of methods and how they inter-relate. Lastly, there are some deeper lessons which are worth bringing to your attention, rather than leaving them for you to extract.

There’s More Than One Way to Threat Model

If you ask a programmer “What’s the right programming language for my new project?” you can expect a lot of clarifying questions. There is no one ideal pro- gramming language. There are certainly languages that are better or worse at certain types of tasks. For example, it’s easier to write text manipulation code in Perl than assembler. Python is easier to read and maintain than assembler, but when you need to write ultra-fast code for a device driver, C or assembler might be a better choice. In the same way, there are better and worse ways to threat model, which depend greatly on your situation: who will be involved, what skills they have, and so on.

So you can think of threat modeling like programming. Within programming there are languages, paradigms (such as waterfall or agile), and practices (pair programming or frequent deployment). The same is true of threat modeling. Most past writing on threat modeling has presented “the” way to do it. This book will help you see how “there’s more than one way to do it” is not just the Perl motto, but also applies to threat modeling.

The Right Way Is the Way That Finds Good Threats

The right way to threat model is the way that empowers a project team to fi nd more good threats against a system than other techniques that could be employed with the resources available. (A “good threat” is a threat that illuminates work that needs to be done.) That’s as true for a project team of one as it is for a project team of thousands. That’s also true across all levels of resources, such as time, expertise, and tooling. The right techniques empower a team to really fi nd and address threats (and gain assurance that they have done so).

www.it-ebooks.info

xxxii Introduction

fl ast.indd 11:57:23:AM 01/17/2014 Page xxxii

There are lots of people who will tell you that they know the one true way. (That’s true in fi elds far removed from threat modeling.) Avoid a religious war and fi nd a way that works for you.

Threat Modeling Is Like Version Control

Threat modeling is sometimes seen as a specialist skill that only a few people can do well. That perspective holds us back, because threat modeling is more like version control than a specialist skill. This is not intended to denigrate or minimize threat modeling; rather, no professional developer would think of building software of any complexity without a version control system of some form. Threat modeling should aspire to be that fundamental.

You expect every professional software developer to know the basics of a version control system or two, and similarly, many systems administrators will use version control to manage confi guration fi les. Many organizations get by with a simple version control approach, and never need an expert. If you work at a large organization, you might have someone who manages the build tree full time. Threat modeling is similar. With the lessons in this book, it will become reasonable to expect professionals in software and operations to have basic experience threat modeling.

Threat Modeling Is Also Like Playing a Violin

When you learn to play the violin, you don’t start with the most beautiful violin music ever written. You learn to play scales, a few easy pieces, and then progress to trickier and trickier music.

Similarly, when you start threat modeling, you need to practice to learn the skills, and it may involve challenges or frustration as you learn. You need to understand threat modeling as a set of skills, which you can apply in a variety of ways, and which take time to develop. You’ll get better if you practice. If you expect to compete with an expert overnight, you might be disappointed. Similarly, if you threat model only every few years, you should expect to be rusty, and it will take you time to rebuild the muscles you need.

Technique versus Repertoire

Continuing with the metaphor, the most talented violinist doesn’t learn to play a single piece, but they develop a repertoire, a set of knowledge that’s relevant to their fi eld.

As you get started threat modeling, you’ll need to develop both techniques and a repertoire—a set of threat examples that you can build from to imagine how new systems might be attacked. Attack lists or libraries can act as a partial

www.it-ebooks.info

Introduction xxxiii

fl ast.indd 11:57:23:AM 01/17/2014 Page xxxiii

substitute for the mental repertoire of known threats an expert knows about. Reading about security issues in similar products can also help you develop a repertoire of threats. Over time, this can feed into how you think about new and different threats. Learning to think about threats is easier with training wheels.

“Think Like an Attacker” Considered Harmful

A great deal of writing on threat modeling exhorts people to “think like an attacker.” For many people, that’s as hard as thinking like a professional chef. Even if you’re a great home cook, a restaurant-managing chef has to wrestle with problems that a home cook does not. For example, how many chickens should you buy to meet the needs of a restaurant with 78 seats, each of which will turn twice an evening? The advice to think like an attacker doesn’t help most people threat model.

Worse, you may end up with implicit or incorrect assumptions about how an attacker will think and what they will choose to do. Such models of an attacker’s mindset may lead you to focus on the wrong threats. You don’t need to focus on the attacker to fi nd threats, but personifi cation may help you fi nd resources to address them.

The Interplay of Attacks, Mitigations, & Requirements

Threat modeling is all about making more secure software. As you use models of software and threats to fi nd potential problems, you’ll discover that some threats are hard or impossible to address, and you’ll adjust requirements to match. This interplay is a rarely discussed key to useful threat modeling.

Sometimes it’s a matter of wanting to defend against administrators, other times it’s a matter of what your customers will bear. In the wake of the 9/11 hijackings, the US government reputedly gave serious consideration to banning laptops from airplanes. (A battery and a mass of explosives reportedly look the same on the x-ray machines.) Business customers, who buy last minute expen- sive tickets and keep the airlines aloft, threatened to revolt. So the government implemented other measures, whose effectiveness might be judged with some of the tools in this book.

This interplay leads to the conclusion that there are threats that cannot be effectively mitigated. That’s a painful thought for many security professionals. (But as the Man in Black said, “Life is pain, Highness! Anyone who says differ- ently is selling something.”) When you fi nd threats that violate your requirements and cannot be mitigated, it generally makes sense to adjust your requirements. Sometimes it’s possible to either mitigate the threat operationally, or defer a decision to the person using the system.

With that, it’s time to dive in and threat model!

www.it-ebooks.info

www.it-ebooks.info

c01.indd 11:33:50:AM 01/17/2014 Page 1

This part of the book is for those who are new to threat modeling, and it assumes no prior knowledge of threat modeling or security. It focuses on the key new skills that you’ll need to threat model and lays out a methodology that’s designed for people who are new to threat modeling.

Part I also introduces the various ways to approach threat modeling using a set of toy analogies. Much like there are many children’s toys for modeling, there are many ways to threat model. There are model kits with precisely molded parts to create airplanes or ships. These kits have a high degree of fi delity and a low level of fl exibility. There are also numerous building block systems such as Lincoln Logs, Erector Sets, and Lego blocks. Each of these allows for more fl exibility, at the price of perhaps not having a propeller that’s quite right for the plane you want to model.

In threat modeling, there are techniques that center on attackers, assets, or software, and these are like Lincoln Logs, Erector Sets, and Lego blocks, in that each is powerful and fl exible, each has advantages and disadvantages, and it can be tricky to combine them into something beautiful.

Par t

I Getting Started

www.it-ebooks.info

2 Part I ■ Getting Started

c01.indd 11:33:50:AM 01/17/2014 Page 2

Part I contains the following chapters:

■ Chapter 1: Dive In and Threat Model! contains everything you need to get started threat modeling, and does so by focusing on four questions:

■ What are you building?

■ What can go wrong?

■ What should you do about those things that can go wrong?

■ Did you do a decent job of analysis?

These questions aren’t just what you need to get started, but are at the heart of the four-step framework, which is the core of this book.

■ Chapter 2: Strategies for Threat Modeling covers a great many ways to approach threat modeling. Many of them are “obvious” approaches, such as thinking about attackers or the assets you want to protect. Each is explained, along with why it works less well than you hope. These and others are contrasted with a focus on software. Software is what you can most reasonably expect a software professional to understand, and so models of software are the most important lesson of Chapter 2. Models of software are one of the two models that you should focus on when threat modeling.

www.it-ebooks.info

3

c01.indd 11:33:50:AM 01/17/2014 Page 3

Anyone can learn to threat model, and what’s more, everyone should. Threat modeling is about using models to fi nd security problems. Using a model means abstracting away a lot of details to provide a look at a bigger picture, rather than the code itself. You model because it enables you to fi nd issues in things you haven’t built yet, and because it enables you to catch a problem before it starts. Lastly, you threat model as a way to anticipate the threats that could affect you.

Threat modeling is fi rst and foremost a practical discipline, and this chapter is structured to refl ect that practicality. Even though this book will provide you with many valuable defi nitions, theories, philosophies, effective approaches, and well-tested techniques, you’ll want those to be grounded in experience. Therefore, this chapter avoids focusing on theory and ignores variations for now and instead gives you a chance to learn by experience.

To use an analogy, when you start playing an instrument, you need to develop muscles and awareness by playing the instrument. It won’t sound great at the start, and it will be frustrating at times, but as you do it, you’ll fi nd it gets easier. You’ll start to hit the notes and the timing. Similarly, if you use the simple four- step breakdown of how to threat model that’s exercised in Parts I-III of this book, you’ll start to develop your muscles. You probably know the old joke about the person who stops a musician on the streets of New York and asks “How do I get to Carnegie Hall?” The answer, of course, is “practice, practice, practice.” Some of that includes following along, doing the exercises, and developing an

C H A P T E R

1

Dive In and Threat Model!

www.it-ebooks.info

4 Part I ■ Getting Started

c01.indd 11:33:50:AM 01/17/2014 Page 4

understanding of the steps involved. As you do so, you’ll start to understand how the various tasks and techniques that make up threat modeling come together.

In this chapter you’re going to fi nd security fl aws that might exist in a design, so you can address them. You’ll learn how to do this by examining a simple web application with a database back end. This will give you an idea of what can go wrong, how to address it, and how to check your work. Along the way, you’ll learn to play Elevation of Privilege, a serious game designed to help you start threat modeling. Finally you’ll get some hands-on experience building your own threat model, and the chapter closes with a set of checklists that help you get started threat modeling.

Learning to Threat Model

You begin threat modeling by focusing on four key questions:

1. What are you building?

2. What can go wrong?

3. What should you do about those things that can go wrong?

4. Did you do a decent job of analysis?

In addressing these questions, you start and end with tasks that all technolo- gists should be familiar with: drawing on a whiteboard and managing bugs. In between, this chapter will introduce a variety of new techniques you can use to think about threats. If you get confused, just come back to these four questions.

Everything in this chapter is designed to help you answer one of these ques- tions. You’re going to fi rst walk through these questions using a three-tier web app as an example, and after you’ve read that, you should walk through the steps again with something of your own to threat model. It could be software you’re building or deploying, or software you’re considering acquiring. If you’re feeling uncertain about what to model, you can use one of the sample systems in this chapter or an exercise found in Appendix E, “Case Studies.”

The second time you work through this chapter, you’ll need a copy of the Elevation of Privilege threat-modeling game. The game uses a deck of cards that you can download free from http://www.microsoft.com/security/sdl/ adopt/eop.aspx. You should get two–four friends or colleagues together for the game part.

You start with building a diagram, which is the fi rst of four major activities involved in threat modeling and is explained in the next section. The other three include fi nding threats, addressing them, and then checking your work.

www.it-ebooks.info

Chapter 1 ■ Dive In and Threat Model! 5

c01.indd 11:33:50:AM 01/17/2014 Page 5

What Are You Building?

Diagrams are a good way to communicate what you are building. There are lots of ways to diagram software, and you can start with a whiteboard diagram of how data fl ows through the system. In this example, you’re working with a simple web app with a web browser, web server, some business logic and a database (see Figure 1-1).

Web browser Web server Business Logic Database

Figure 1-1: A whiteboard diagram

Some people will actually start thinking about what goes wrong right here. For example, how do you know that the web browser is being used by the person you expect? What happens if someone modifi es data in the database? Is it OK for information to move from one box to the next without being encrypted? You might want to take a minute to think about some things that could go wrong here because these sorts of questions may lead you to ask “is that allowed?” You can create an even better model of what you’re building if you think about “who controls what” a little. Is this a website for the whole Internet, or is it an intranet site? Is the database on site, or at a web provider?

For this example, let’s say that you’re building an Internet site, and you’re using the fi ctitious Acme storage-system. (I’d put a specifi c product here, but then I’d get some little detail wrong and someone, certainly not you, would get all wrapped around the axle about it and miss the threat modeling lesson. Therefore, let’s just call it Acme, and pretend it just works the way I’m saying. Thanks! I knew you’d understand.)

Adding boundaries to show who controls what is a simple way to improve the diagram. You can pretty easily see that the threats that cross those bound- aries are likely important ones, and may be a good place to start identifying threats. These boundaries are called trust boundaries, and you should draw

www.it-ebooks.info

6 Part I ■ Getting Started

c01.indd 11:33:50:AM 01/17/2014 Page 6

them wherever different people control different things. Good examples of this include the following:

■ Accounts (UIDs on unix systems, or SIDS on Windows)

■ Network interfaces

■ Different physical computers

■ Virtual machines

■ Organizational boundaries

■ Almost anywhere you can argue for different privileges

TRUST BOUNDARY VERSUS ATTACK SURFACE

A closely related concept that you may have encountered is attack surface. For example, the hull of a ship is an attack surface for a torpedo. The side of a ship presents a larger attack surface to a submarine than the bow of the same ship. The ship may have inter- nal “trust” boundaries, such as waterproof bulkheads or a Captain’s safe. A system that exposes lots of interfaces presents a larger attack surface than one that presents few APIs or other interfaces. Network fi rewalls are useful boundaries because they reduce the attack surface relative to an external attacker. However, much like the Captain’s safe, there are still trust boundaries inside the fi rewall. A trust boundary and an attack surface are very similar views of the same thing. An attack surface is a trust boundary and a direction from which an attacker could launch an attack. Many people will treat the terms are inter- changeable. In this book, you’ll generally see “trust boundary” used.

In your diagram, draw the trust boundaries as boxes (see Figure 1-2), show- ing what’s inside each with a label (such as “corporate data center”) near the edge of the box.

Web browser Web server

Corporate data center Web storage (offsite)

Business Logic Database

Figure 1-2: Trust boundaries added to a whiteboard diagram

www.it-ebooks.info

Chapter 1 ■ Dive In and Threat Model! 7

c01.indd 11:33:50:AM 01/17/2014 Page 7

As your diagram gets larger and more complex, it becomes easy to miss a part of it, or to become confused by labels on the data fl ows. Therefore, it can be very helpful to number each process, data fl ow, and data store in the diagram, as shown in Figure 1-3. (Because each trust boundary should have a unique name, representing the unique trust inside of it, there’s limited value to numbering those.)

Web browser Web server 1 2 3 4 5 6 7

Corporate data center Web storage (offsite)

Business Logic Database

Figure 1-3: Numbers and trust boundaries added to a whiteboard diagram

Regarding the physical form of the diagram: Use whatever works for you. If that’s a whiteboard diagram and a camera phone picture, great. If it’s Visio, or OmniGraffl e, or some other drawing program, great. You should think of threat model diagrams as part of the development process, so try to keep it in source control with everything else.

Now that you have a diagram, it’s natural to ask, is it the right diagram? For now, there’s a simple answer: Let’s assume it is. Later in this chapter there are some tips and checklists as well as a section on updating the diagram, but at this stage you have a good enough diagram to get started on identifying threats, which is really why you bought this book. So let’s identify.

What Can Go Wrong?

Now that you have a diagram, you can really start looking for what can go wrong with its security. This is so much fun that I turned it into a game called, Elevation of Privilege. There’s more on the game in Appendix D, “Elevation of Privilege: The Cards,” which discusses each card, and in Chapter 11, “Threat Modeling Tools,” which covers the history and philosophy of the game, but you can get started playing now with a few simple instructions. If you haven’t already done so, download a deck of cards from http://www.microsoft.com/security/sdl/ adopt/eop.aspx. Print the pages in color, and cut them into individual cards. Then shuffl e the deck and deal it out to those friends you’ve invited to play.

www.it-ebooks.info

8 Part I ■ Getting Started

c01.indd 11:33:50:AM 01/17/2014 Page 8

N O T E Some people aren’t used to playing games at work. Others approach new games with trepidation, especially when those games involve long, complicated instruc-

tions. Elevation of Privilege takes just a few lines to explain. You should give it a try.

How To Play Elevation of Privilege

Elevation of Privilege is a serious game designed to help you threat model. A sample card is shown in Figure 1-4. You’ll notice that like playing cards, it has a number and suit in the upper left, and an example of a threat as the main text on the card. To play the game, simply follow the instructions in the upcoming list.

Tampering An attacker can take advantage of your custom key exchange or integrity control which you built instead of using standard crypto.

Figure 1-4: An Elevation of Privilege card

1. Deal the deck. (Shuffl ing is optional.)

2. The person with the 3 of Tampering leads the fi rst round. (In card games like this, rounds are also called “tricks” or “hands.”)

www.it-ebooks.info

Chapter 1 ■ Dive In and Threat Model! 9

c01.indd 11:33:50:AM 01/17/2014 Page 9

3. Each round works like so:

A. Each player plays one card, starting with the person leading the round, and then moving clockwise.

B. To play a card, read it aloud, and try to determine if it affects the system you have diagrammed. If you can link it, write it down, and score yourself a point. Play continues clockwise with the next player.

C. When each player has played a card, the player who has played the highest card wins the round. That player leads the next round.

4. When all the cards have been played, the game ends and the person with the most points wins.

5. If you’re threat modeling a system you’re building, then you go fi le any bugs you fi nd.

There are some folks who threat model like this in their sleep, or even have trouble switching it off. Not everyone is like that. That’s OK. Threat modeling is not rocket science. It’s stuff that anyone who participates in software devel- opment can learn. Not everyone wants to dedicate the time to learn to do it in their sleep.

Identifying threats can seem intimidating to a lot of people. If you’re one of them, don’t worry. This section is designed to gently walk you through threat identifi cation. Remember to have fun as you do this. As one reviewer said: “Playing Elevation of Privilege should be fun. Don’t downplay that. We play it every Friday. It’s enjoyable, relaxing, and still has business value.”

Outside of the context of the game, you can take the next step in threat model- ing by thinking of things that might go wrong. For instance, how do you know that the web browser is being used by the person you expect? What happens if someone modifi es data in the database? Is it OK for information to move from one box to the next without being encrypted? You don’t need to come up with these questions by just staring at the diagram and scratching your chin. (I didn’t!) You can identify threats like these using the simple mnemonic STRIDE, described in detail in the next section.

Using the STRIDE Mnemonic to Find Threats

STRIDE is a mnemonic for things that go wrong in security. It stands for Spoofi ng, Tampering, Repudiation, Information Disclosure, Denial of Service, and Elevation of Privilege:

■ Spoofi ng is pretending to be something or someone you’re not.

www.it-ebooks.info

10 Part I ■ Getting Started

c01.indd 11:33:50:AM 01/17/2014 Page 10

■ Tampering is modifying something you’re not supposed to modify. It can include packets on the wire (or wireless), bits on disk, or the bits in memory.

■ Repudiation means claiming you didn’t do something (regardless of whether you did or not).

■ Denial of Service are attacks designed to prevent a system from provid- ing service, including by crashing it, making it unusably slow, or fi lling all its storage.

■ Information Disclosure is about exposing information to people who are not authorized to see it.

■ Elevation of Privilege is when a program or user is technically able to do things that they’re not supposed to do.

N O T E This is where Elevation of Privilege, the game, gets its name. This book uses Elevation of Privilege, italicized, or abbreviated to EoP, for the game—to avoid confusion

with the threat.

Recall the three example threats mentioned in the preceding section:

■ How do you know that the web browser is being used by the person you expect?

■ What happens if someone modifi es data in the database? ■ Is it ok for information to go from one box to the next without being encrypted?

These are examples of spoofi ng, tampering, and information disclosure. Using STRIDE as a mnemonic can help you walk through a diagram and select example threats. Pair that with a little knowledge of security and the right techniques, and you’ll fi nd the important threats faster and more reliably. If you have a process in place for ensuring that you develop a threat model, document it, and you can increase confi dence in your software.

Now that you have STRIDE in your tool belt, walk through your diagram again and look for more threats, this time using the mnemonic. Make a list as you go with the threat and what element of the diagram it affects. (Generally, the software, data fl ow, or storage is affected, rather than the trust boundary.) The following list provides some examples of each threat.

■ Spoofi ng: Someone might pretend to be another customer, so you’ll need a way to authenticate users. Someone might also pretend to be your website, so you should ensure that you have an SSL certifi cate and that you use a single domain for all your pages (to help that subset of customers who read URLs to see if they’re in the right place). Someone might also place a deep link to one of your pages, such as logout.html or placeorder.aspx. You should be checking the Referrer fi eld before taking action. That’s not a complete solution to what are called CSRF (Cross Site Request Forgery) attacks, but it’s a start.

www.it-ebooks.info

Chapter 1 ■ Dive In and Threat Model! 11

c01.indd 11:33:50:AM 01/17/2014 Page 11

■ Tampering: Someone might tamper with the data in your back end at Acme. Someone might tamper with the data as it fl ows back and forth between their data center and yours. A programmer might replace the operational code on the web front end without testing it, thinking they’re uploading it to staging. An angry programmer might add a coupon code “PayBobMore” that offers a 20 percent discount on all goods sold.

■ Repudiation: Any of the preceding actions might require digging into what happened. Are there system logs? Is the right information being logged effectively? Are the logs protected against tampering?

■ Information Disclosure: What happens if Acme reads your database? Can anyone connect to the database and read or write information?

■ Denial of Service: What happens if a thousand customers show up at once at the website? What if Acme goes down?

■ Elevation of Privilege: Perhaps the web front end is the only place customers should access, but what enforces that? What prevents them from connecting directly to the business logic server, or uploading new code? If there’s a fi rewall in place, is it correctly confi gured? What controls access to your database at Acme, or what happens if an employee at Acme makes a mistake, or even wants to edit your fi les?

The preceding possibilities aren’t intended to be a complete list of how each threat might manifest against every model. You can fi nd a more complete list in Chapter 3, “STRIDE.” This shorter version will get you started though, and it is focused on what you might need to investigate based on the very simple diagram shown in Figure 1-2. Remember the musical instrument analogy. If you try to start playing the piano with Ravel’s Gaspard (regarded as one of the most complex piano pieces ever written), you’re going to be frustrated.

Tips for Identifying Threats

Whether you are identifying threats using Elevation of Privilege, STRIDE, or both, here are a few tips to keep in mind that can help you stay on the right track to determine what could go wrong:

■ Start with external entities: If you’re not sure where to start, start with the external entities or events which drive activity. There are many other valid approaches though: You might start with the web browser, look- ing for spoofi ng, then tampering, and so on. You could also start with the business logic if perhaps your lead developer for that component is in the room. Wherever you choose to begin, you want to aspire to some level of organization. You could also go in “STRIDE order” through the diagram. Without some organization, it’s hard to tell when you’re done, but be careful not to add so much structure that you stifl e creativity.

www.it-ebooks.info

12 Part I ■ Getting Started

c01.indd 11:33:50:AM 01/17/2014 Page 12

■ Never ignore a threat because it’s not what you’re looking for right now. You might come up with some threats while looking at other categories. Write them down and come back to them. For example, you might have thought about “can anyone connect to our database,” which is listed under information disclosure, while you were looking for spoofi ng threats. If so, that’s awesome! Good job! Redundancy in what you fi nd can be tedious, but it helps you avoid missing things. If you fi nd yourself asking whether “someone not authorized to connect to the database who reads informa- tion” constitutes spoofi ng or information disclosure, the answer is, who cares? Record the issue and move along to the next one. STRIDE is a tool to guide you to threats, not to ask you to categorize what you’ve found; it makes a lousy taxonomy, anyway. (That is to say, there are plenty of security issues for which you can make an argument for various different categorizations. Compare and contrast it with a good taxonomy, such as the taxonomy of life. Does it have a backbone? If so, it’s a vertebrae.)

■ Focus on feasible threats: Along the way, you might come up with threats like “someone might insert a back door at the chip factory,” or “someone might hire our janitorial staff to plug in a hardware key logger and steal all our passwords.” These are real possibilities but not very likely com- pared to using an exploit to attack a vulnerability for which you haven’t applied the patch, or tricking someone into installing software. There’s also the question of what you can do about either, which brings us to the next section.

Addressing Each Threat

You should now have a decent-sized list or lists of threats. The next step in the threat modeling process is to go through the lists and address each threat. There are four types of action you can take against each threat: Mitigate it, eliminate it, transfer it, or accept it. The following list looks briefl y at each of these ways to address threats, and then in the subsequent sections you will learn how to address each specifi c threat identifi ed with the STRIDE list in the “What Can Go Wrong” section. For more details about each of the strategies and techniques to address these threats, see Chapters 8 and 9, “Defensive Building Blocks” and “Tradeoffs When Addressing Threats.”

■ Mitigating threats is about doing things to make it harder to take advan- tage of a threat. Requiring passwords to control who can log in mitigates the threat of spoofi ng. Adding password controls that enforce complex- ity or expiration makes it less likely that a password will be guessed or usable if stolen.

■ Eliminating threats is almost always achieved by eliminating features. If you have a threat that someone will access the administrative function of

www.it-ebooks.info

Chapter 1 ■ Dive In and Threat Model! 13

c01.indd 11:33:50:AM 01/17/2014 Page 13

a website by visiting the /admin/URL, you can mitigate it with passwords or other authentication techniques, but the threat is still present. You can make it less likely to be found by using a URL like /j8e8vg21euwq/, but the threat is still present. You can eliminate it by removing the interface, handling administration through the command line. (There are still threats associated with how people log in on a command line. Moving away from HTTP makes the threat easier to mitigate by controlling the attack surface. Both threats would be found in a complete threat model.) Incidentally, there are other ways to eliminate threats if you’re a mob boss or you run a police state, but I don’t advocate their use.

■ Transferring threats is about letting someone or something else handle the risk. For example, you could pass authentication threats to the operating system, or trust boundary enforcement to a fi rewall product. You can also transfer risk to customers, for example, by asking them to click through lots of hard-to-understand dialogs before they can do the work they need to do. That’s obviously not a great solution, but sometimes people have knowledge that they can contribute to making a security tradeoff. For example, they might know that they just connected to a coffee shop wireless network. If you believe the person has essential knowledge to contribute, you should work to help her bring it to the decision. There’s more on doing that in Chapter 15, “Human Factors and Usability.”

■ Accepting the risk is the fi nal approach to addressing threats. For most organizations most of the time, searching everyone on the way in and out of the building is not worth the expense or the cost to the dignity and job satisfaction of those workers. (However, diamond mines and some- times government agencies take a different approach.) Similarly, the cost of preventing someone from inserting a back door in the motherboard is expensive, so for each of these examples you might choose to accept the risk. And once you’ve accepted the risk, you shouldn’t worry over it. Sometimes worry is a sign that the risk hasn’t been fully accepted, or that the risk acceptance was inappropriate.

The strategies listed in the following tables are intended to serve as examples to illustrate ways to address threats. Your “go-to” approach should be to miti- gate threats. Mitigation is generally the easiest and the best for your customers. (It might look like accepting risk is easier, but over time, mitigation is easier.) Mitigating threats can be hard work, and you shouldn’t take these examples as complete. There are often other valid ways to address each of these threats, and sometimes trade-offs must be made in the way the threats are addressed.

Addressing Spoofi ng

Table 1-1 and the list that follows show targets of spoofi ng, mitigation strategies that address spoofi ng, and techniques to implement those mitigations.

www.it-ebooks.info

14 Part I ■ Getting Started

c01.indd 11:33:50:AM 01/17/2014 Page 14

Table 1-1: Addressing Spoofi ng Threats

THREAT TARGET MITIGATION STRATEGY MITIGATION TECHNIQUE

Spoofi ng a person

Identifi cation and authen- tication (usernames and something you know/have/ are)

Usernames, real names, or other identifi ers:

❖ Passwords

❖ Tokens

❖ Biometrics

Enrollment/maintenance/expiry

Spoofi ng a “fi le” on disk

Leverage the OS ❖ Full paths

❖ Checking ACLs

❖ Ensuring that pipes are created properly

Cryptographic authenticators

Digital signatures or authenticators

Spoofi ng a net- work address

Cryptographic ❖ DNSSEC

❖ HTTPS/SSL

❖ IPsec

Spoofi ng a program in memory

Leverage the OS Many modern operating systems have some form of application identifi er that the OS will enforce.

■ When you’re concerned about a person being spoofed, ensure that each person has a unique username and some way of authenticating. The tradi- tional way to do this is with passwords, which have all sorts of problems as well as all sorts of advantages that are hard to replicate. See Chapter 14, “Accounts and Identity” for more on passwords.

■ When accessing a fi le on disk, don’t ask for the fi le with open(file). Use open(/path/to/file). If the fi le is sensitive, after opening, check vari- ous security elements of the fi le descriptor (such as fully resolved name, permissions, and owner). You want to check with the fi le descriptor to avoid race conditions. This applies doubly when the fi le is an executable, although checking after opening can be tricky. Therefore, it may help to ensure that the permissions on the executable can’t be changed by an attacker. In any case, you almost never want to call exec() with ./file.

■ When you’re concerned about a system or computer being spoofed when it connects over a network, you’ll want to use DNSSEC, SSL, IPsec, or a combination of those to ensure you’re connecting to the right place.

www.it-ebooks.info

Chapter 1 ■ Dive In and Threat Model! 15

c01.indd 11:33:50:AM 01/17/2014 Page 15

Addressing Tampering

Table 1-2 and the list that follows show targets of tampering, mitigation strate- gies that address tampering, and techniques to implement those mitigations.

Table 1-2: Addressing Tampering Threats

THREAT TARGET MITIGATION STRATEGY MITIGATION TECHNIQUE

Tampering with a fi le Operating system ACLs

Cryptographic ❖ Digital Signatures

❖ Keyed MAC

Racing to create a fi le (tampering with the fi le system)

Using a directory that’s protected from arbitrary user tampering

ACLs

Using private directory structures

(Randomizing your fi le names just makes it annoying to execute the attack.)

Tampering with a net- work packet

Cryptographic ❖ HTTPS/SSL

❖ IPsec

Anti-pattern Network isolation (See note on network isolation anti-pattern.)

■ Tampering with a fi le: Tampering with fi les can be easy if the attacker has an account on the same machine, or by tampering with the network when the fi les are obtained from a server.

■ Tampering with memory: The threats you want to worry about are those that can occur when a process with less privileges than you, or that you don’t trust, can alter memory. For example, if you’re getting data from a shared memory segment, is it ACLed so only the other process can see it? For a web app that has data coming in via AJAX, make sure you validate that the data is what you expect after you pull in the right amount.

■ Tampering with network data: Preventing tampering with network data requires dealing with both spoofi ng and tampering. Otherwise, someone who wants to tamper can simply pretend to be the other end, using what’s called a man-in-the-middle attack. The most common solution to these problems is SSL, with IP Security (IPsec) emerging as another possibility. SSL and IPsec both address confi dentiality and tampering, and can help address spoofi ng.

www.it-ebooks.info

16 Part I ■ Getting Started

c01.indd 11:33:50:AM 01/17/2014 Page 16

■ Tampering with networks anti-pattern: It’s somewhat common for peo- ple to hope that they can isolate their network, and so not worry about tampering threats. It’s also very hard to maintain isolation over time. Isolation doesn’t work as well as you would hope. For example, the isolated United States SIPRNet was thoroughly infested with malware, and the operation to clean it up took 14 months (Shachtman, 2010).

N O T E A program can’t check whether it’s authentic after it loads. It may be possible for something to rely on “trusted bootloaders” to provide a chain of signatures,

but the security decisions are being made external to that code. (If you’re not familiar

with the technology, don’t worry, the key lesson is that a program cannot check its own

authenticity.)

Addressing Repudiation

Addressing repudiation is generally a matter of ensuring that your system is designed to log and ensuring that those logs are preserved and protected. Some of that can be handled with simple steps such as using a reliable trans- port for logs. In this sense, syslog over UDP was almost always silly from a security perspective; syslog over TCP/SSL is now available and is vastly better.

Table 1-3 and the list that follows show targets of repudiation, mitigation strate- gies that address repudiation, and techniques to implement those mitigations.

Table 1-3: Addressing Repudiation Threats

THREAT TARGET MITIGATION STRATEGY MITIGATION TECHNIQUE

No logs means you can’t prove anything.

Log Be sure to log all the security- relevant information.

Logs come under attack Protect your logs. ❖ Send over the network.

❖ ACL

Logs as a channel for attack Tightly specifi ed logs Documenting log design early in the development process

■ No logs means you can’t prove anything: This is self-explanatory. For example, when a customer calls to complain that they never got their order, how will this be resolved? Maintain logs so that you can investigate what happens when someone attempts to repudiate something.

www.it-ebooks.info

Chapter 1 ■ Dive In and Threat Model! 17

c01.indd 11:33:50:AM 01/17/2014 Page 17

■ Logs come under attack: Attackers will do things to prevent your logs from being useful, including fi lling up the log to make it hard to fi nd the attack or forcing logs to “roll over.” They may also do things to set off so many alarms that the real attack is lost in a sea of troubles. Perhaps obviously, sending logs over a network exposes them to other threats that you’ll need to handle.

■ Logs as a channel for attack: By design, you’re collecting data from sources outside your control, and delivering that data to people and systems with security privileges. An example of such an attack might be sending mail addressed to "</html> [email protected]", causing trouble for web-based tools that don’t expect inline HTML.

You can make it easier to write secure code to process your logs by clearly communicating what your logs can’t contain, such as “Our logs are all plaintext, and attackers can insert all sorts of things,” or “Fields 1–5 of our logs are tightly controlled by our software, fi elds 6–9 are easy to inject data into. Field 1 is time in GMT. Fields 2 and 3 are IP addresses (v4 or 6)...” Unless you have incredibly strict control, documenting what your logs can contain will likely miss things. (For example, can your logs contain Unicode double-wide characters?)

Addressing Information Disclosure

Table 1-4 and the list which follows show targets of information disclosure, mitigation strategies that address information disclosure, and techniques to implement those mitigations.

Table 1-4: Addressing Information Disclosure Threats

THREAT TARGET MITIGATION STRATEGY MITIGATION TECHNIQUE

Network monitoring Encryption ❖ HTTPS/SSL

❖ IPsec

Directory or fi lename (for example layoff-letters/ adamshostack.docx)

Leverage the OS. ACLs

File contents Leverage the OS. ACLS

Cryptography File encryption such as PGP, disk encryption (FileVault, BitLocker)

API information disclosure

Design Careful design control

Consider pass by reference or value.

www.it-ebooks.info

18 Part I ■ Getting Started

c01.indd 11:33:50:AM 01/17/2014 Page 18

■ Network monitoring: Network monitoring takes advantage of the archi- tecture of most networks to monitor traffi c. (In particular, most networks now broadcast packets, and each listener is expected to decide if the packet matters to them.) When networks are architected differently, there are a variety of techniques to draw traffi c to or through the monitoring station.

If you don’t address spoofi ng, much like tampering, an attacker can just sit in the middle and spoof each end. Mitigating network information disclosure threats requires handling both spoofi ng and tampering threats. If you don’t address tampering, then there are all sorts of clever ways to get information out. Here again, SSL and IP Security options are your simplest choices.

■ Names reveal information: When the name of a directory or a fi lename itself will reveal information, then the best way to protect it is to create a parent directory with an innocuous name and use operating system ACLs or permissions.

■ File content is sensitive: When the contents of the fi le need protection, use ACLs or cryptography. If you want to protect all the data should the machine fall into unauthorized hands, you’ll need to use cryptography. The forms of cryptography that require the person to manually enter a key or passphrase are more secure and less convenient. There’s fi le, fi lesystem, and database cryptography, depending on what you need to protect.

■ APIs reveal information: When designing an API, or otherwise passing information over a trust boundary, select carefully what information you disclose. You should assume that the information you provide will be passed on to others, so be selective about what you provide. For example, website errors that reveal the username and password to a database are a common form of this fl aw, others are discussed in Chapter 3.

Addressing Denial of Service

Table 1-5 and the list that follows show targets of denial of service, mitigation strategies that address denial of service, and techniques to implement those mitigations.

www.it-ebooks.info

Chapter 1 ■ Dive In and Threat Model! 19

c01.indd 11:33:50:AM 01/17/2014 Page 19

Table 1-5: Addressing Denial of Service Threats

THREAT TARGET

MITIGATION STRATEGY MITIGATION TECHNIQUE

Network fl ooding

Look for exhaustible resources.

❖ Elastic resources

❖ Work to ensure attacker resource consumption is as high as or higher than yours.

Network ACLS

Program resources

Careful design Elastic resource management, proof of work

Avoid multipliers. Look for places where attackers can multiply CPU consumption on your end with minimal eff ort on their end: Do something to require work or enable distinguishing attackers, such as client does crypto fi rst or login before large work factors (of course, that can’t mean that logins are unencrypted).

System resources

Leverage the OS. Use OS settings.

■ Network flooding: If you have static structures for the number of connections, what happens if those fi ll up? Similarly, to the extent that it’s under your control, don’t accept a small amount of network data from a possibly spoofed address and return a lot of data. Lastly, fi rewalls can provide a layer of network ACLs to control where you’ll accept (or send) traffi c, and can be useful in mitigating network denial-of-service attacks.

■ Look for exhaustible resources: The fi rst set of exhaustible resources are network related, the second set are those your code manages, and the third are those the OS manages. In each case, elastic resourcing is a valuable technique. For example, in the 1990s some TCP stacks had a hardcoded limit of fi ve half-open TCP connections. (A half-open connection is one in the process of being opened. Don’t worry if that doesn’t make sense, but rather ask yourself why the code would be limited to fi ve of them.) Today, you can often obtain elastic resourcing of various types from cloud providers.

www.it-ebooks.info

20 Part I ■ Getting Started

c01.indd 11:33:50:AM 01/17/2014 Page 20

■ System resources: Operating systems tend to have limits or quotas to control the resource consumption of user-level code. Consider those resources that the operating system manages, such as memory or disk usage. If your code runs on dedicated servers, it may be sensible to allow it to chew up the entire machine. Be careful if you unlimit your code, and be sure to document what you’re doing.

■ Program resources: Consider resources that your program manages itself. Also, consider whether the attacker can make you do more work than they’re doing. For example, if he sends you a packet full of random data and you do expensive cryptographic operations on it, then your vulnerability to denial of service will be higher than if you make him do the cryptography fi rst. Of course, in an age of botnets, there are limits to how well one can reassign this work. There’s an excellent paper by Ben Laurie and Richard Clayton, “Proof of work proves not to work,” which argues against proof of work schemes (Laurie, 2004).

Addressing Elevation of Privilege

Table 1-6 and the list that follows show targets of elevation of privilege, mitiga- tion strategies that address elevation of privilege, and techniques to implement those mitigations.

Table 1-6: Addressing Elevation of Privilege Threats

THREAT TARGET MITIGATION STRATEGY MITIGATION TECHNIQUE

Data/code confusion

Use tools and architectures that separate data and code.

❖ Prepared statements or stored procedures in SQL

❖ Clear separators with canonical forms

❖ Late validation that data is what the next func- tion expects

Control fl ow/ memory corrup- tion attacks

Use a type-safe language.

Writing code in a type-safe language protects against entire classes of attack.

Leverage the OS for memory protection.

Most modern operating systems have memory- protection facilities.

www.it-ebooks.info

Chapter 1 ■ Dive In and Threat Model! 21

c01.indd 11:33:50:AM 01/17/2014 Page 21

THREAT TARGET MITIGATION STRATEGY MITIGATION TECHNIQUE

Use the sandbox. ❖ Modern operating systems support sand- boxing in various ways (AppArmor on Linux, AppContainer or the MOICE pattern on Windows, Sandboxlib on Mac OS).

❖ Don’t run as the “nobody” account, create a new one for each app. Postfi x and QMail are examples of the good pattern of one account per function.

Command injec- tion attacks

Be careful. ❖ Validate that your input is the size and form you expect.

❖ Don’t sanitize. Log and then throw it away if it’s weird.

■ Data/code confusion: Problems where data is treated as code are common. As information crosses layers, what’s tainted and what’s pure can be lost. Attacks such as XSS take advantage of HTML’s freely interweaving code and data. (That is, an .html fi le contains both code, such as Javascript, and data, such as text, to be displayed and sometimes formatting instructions for that text.) There are a few strategies for dealing with this. The fi rst is to look for ways in which frameworks help you keep code and data separate. For example, prepared statements in SQL tell the database what statements to expect, and where the data will be.

You can also look at the data you’re passing right before you pass it, so you know what validation you might be expected to perform for the func- tion you’re calling. For example, if you’re sending data to a web page, you might ensure that it contains no <, >, #, or & characters, or whatever.

In fact, the value of “whatever” is highly dependent on exactly what exists between “you” and the rendition of the web page, and what security checks it may be performing. If “you” means a web server, it may be very important to have a few < and > symbols in what you produce. If “you” is something taking data from a database and sending it to, say PHP, then the story is quite different. Ideally, the nature of “you” and the additional steps are clear in your diagrams.

■ Control fl ow/memory corruption attacks: This set of attacks generally takes advantage of weak typing and static structures in C-like languages to enable an attacker to provide code and then jump to that code. If you

www.it-ebooks.info

22 Part I ■ Getting Started

c01.indd 11:33:50:AM 01/17/2014 Page 22

use a type-safe language, such as Java or C#, many of these attacks are harder to execute.

Modern operating systems tend to contain memory protection and random- ization features, such as Address Space Layout Randomization (ASLR). Sometimes the features are optional, and require a compiler or linker switch. In many cases, such features are almost free to use, and you should at least try all such features your OS supports. (It’s not completely effortless, you may need to recompile, test, or make other such small investments.)

The last set of controls to address memory corruption are sandboxes. Sandboxes are OS features that are designed to protect the OS or the rest of the programs running as the user from a corrupted program.

N O T E Details about each of these features are outside the scope of this book, but searching on terms such as type safety, ASLR, and sandbox should provide a plethora of

details.

■ Command injection attacks: Command injection attacks are a form of code/data confusion where an attacker supplies a control character, fol- lowed by commands. For example, in SQL injection, a single quote will often close a dynamic SQL statement; and when dealing with unix shell scripts, the shell can interpret a semicolon as the end of input, taking anything after that as a command.

In addition to working through each STRIDE threat you encounter, a few other recurring themes will come up as you address your threats; these are covered in the following two sections.

Validate, Don’t Sanitize

Know what you expect to see, how much you expect to see, and validate that that’s what you’re receiving. If you get something else, throw it away and return an error message. Unless your code is perfect, errors in sanitization will hurt a lot, because after you write that sanitize input function you’re going to rely on it. There have been fascinating attacks that rely on a sanitize function to get their code into shape to execute.

Trust the Operating System

One of the themes that recurs in the preceding tables is “trust the operating system.” Of course, you may want to discount that because I did much of this

www.it-ebooks.info

Chapter 1 ■ Dive In and Threat Model! 23

c01.indd 11:33:50:AM 01/17/2014 Page 23

work while working for Microsoft, a purveyor of a variety of fi ne operating system software, so there might be some bias here. It’s a valid point, and good for you for being skeptical. See, you’re threat modeling already!

More seriously, trusting the operating system is a good idea for a number of reasons:

■ The operating system provides you with security features so you can focus on your unique value proposition.

■ The operating system runs with privileges that are probably not available to your program or your attacker.

■ If your attacker controls the operating system, you’re likely in a world of hurt regardless of what your code tries to do.

With all of that “trust the operating system” advice, you might be tempted to ask why you need this book. Why not just rely on the operating system?

Well, many of the building blocks just discussed are discretionary. You can use them well or you can use them poorly. It’s up to you to ensure that you don’t set the permissions on a fi le to 777, or the ACLs to allow Guest accounts to write. It’s up to you to write code that runs well as a normal or even sandboxed user, and it’s certainly up to you in these early days of client/server, web, distributed systems, web 2.0, cloud, or whatever comes next to ensure that you’re building the right security mechanisms that these newfangled widgets don’t yet offer.

File Bugs

Now that you have a list of threats and ways you would like to mitigate them, you’re through the complex, security-centered parts of the process. There are just a few more things to do, the fi rst of which is to treat each line of the preceding tables as a bug. You want to treat these as bugs because if you ship software, you’ve learned to handle bugs in some way. You presumably have a way to track them, prioritize them, and ensure that you’re closing them with an appropriate degree of consistency. This will mean something very different to a three-person start-up versus a medical device manufacturer, but both organizations will have a way to handle bugs. You want to tap into that procedure to ensure that threat modeling isn’t just a paper exercise.

You can write the text of the bugs in a variety of ways, based on what your organization does. Examples of fi ling a bug might include the following:

■ Someone might use the /admin/ interface without proper authorization.

■ The admin interface lacks proper authorization controls,

■ There’s no automated security testing for the /admin/ interface.

www.it-ebooks.info

24 Part I ■ Getting Started

c01.indd 11:33:50:AM 01/17/2014 Page 24

Whichever way you go, it’s great if you can include the entire threat in the bug, and mark it as a security bug if your bug-tracking tool supports that. (If you’re a super-agile scrum shop, use a uniquely colored Post-it for security bugs.)

You’ll also have to prioritize the bugs. Elevation-of-privilege bugs are almost always going to fall into the highest priority category, because when they’re exploited they lead to so much damage. Denial of service often falls toward the bottom of the stack, but you’ll have to consider each bug to determine how to rank it.

Checking Your Work

Validation of your threat model is the last thing you do as part of threat model- ing. There are a few tasks to be done here, and it is best to keep them aligned with the order in which you did the previous work. Therefore, the validation tasks include checking the model, checking that you’ve looked for each threat, and checking your tests. You probably also want to validate the model a second time as you get close to shipping or deploying.

Checking the model

You should ensure that the fi nal model matched what you built. If it doesn’t, how can you know that you found the right, relevant threats? To do so, try to arrange a meeting during which everyone looks at the diagram, and answer the following questions:

■ Is this complete?

■ Is it accurate?

■ Does it cover all the security decisions we made?

■ Can I start the next version with this diagram without any changes?

If everyone says yes, your diagram is suffi ciently up to date for the next step. If not, you’ll need to update it.

Updating the Diagram

As you went through the diagram, you might have noticed that it’s missing key data. If it were a real system, there might be extra interfaces that were not drawn in, or there might be additional databases. There might be details that you jumped to the whiteboard to draw in. If so, you need to update the diagram with those details. A few rules of thumb are useful as you create or update diagrams:

■ Focus on data fl ow, not control fl ow.

■ Anytime you need to qualify your answer with “sometimes” or “also,” you should consider adding more detail to break out the various cases. For example, if you say, “Sometimes we connect to this web service via SSL,

www.it-ebooks.info

Chapter 1 ■ Dive In and Threat Model! 25

c01.indd 11:33:50:AM 01/17/2014 Page 25

and sometimes we fall back to HTTP,” you should draw both of those data fl ows (and consider whether an attacker can make you fall back like that).

■ Anytime you fi nd yourself needing more detail to explain security-relevant behavior, draw it in.

■ Any place you argued over the design or construction of the system, draw in the agreed-on facts. This is an important way to ensure that everyone ended that discussion on the same page. It’s especially important for larger teams when not everyone is in the room for the threat model discussions. If they see a diagram that contradicts their thinking, they can either accept it or challenge the assumptions; but either way, a good clear diagram can help get everyone on the same page.

■ Don’t have data sinks: You write the data for a reason. Show who uses it.

■ Data can’t move itself from one data store to another: Show the process that moves it.

■ The diagram should tell a story, and support you telling stories while pointing at it.

■ Don’t draw an eye chart (a diagram with so much detail that you need to squint to read the tiny print).

Diagram Details

If you’re wondering how to reconcile that last rule of thumb, don’t draw an eye chart, with all the details that a real software project can entail, one technique is to use a sub diagram that shows the details of one particular area. You should look for ways to break things out that make sense for your project. For example, if you have one hyper-complex process, maybe everything in that process should be covered in one diagram, and everything outside it in another. If you have a dispatcher or queuing system, that’s a good place to break things up. Your databases or the fail-over system is also a good split. Maybe there’s a set of a few elements that really need more detail. All of these are good ways to break things out.

The key thing to remember is that the diagram is intended to help ensure that you understand and can discuss the system. Recall the quote that opens this book: “All models are wrong. Some models are useful.” Therefore, when you’re adding additional diagrams, don’t ask, “Is this the right way to do it?” Instead, ask, “Does this help me think about what might go wrong?”

Checking Each Threat

There are two main types of validation activities you should do. The fi rst is checking that you did the right thing with each threat you found. The other is asking if you found all the threats you should fi nd.

www.it-ebooks.info

26 Part I ■ Getting Started

c01.indd 11:33:50:AM 01/17/2014 Page 26

In terms of checking that you did the right thing with each threat you did fi nd, the fi rst and foremost question here is “Did I do something with each unique threat I found?” You really don’t want to drop stuff on the fl oor. This is “turning the crank” sort of work. It’s rarely glamorous or exciting until you fi nd the thing you overlooked. You can save a lot of time by taking meeting minutes and writing a bug number next to each one, checking that you’ve addressed each when you do your bug triage.

The next question is “Did I do the right something with each threat?” If you’ve fi led bugs with some sort of security tag, run a query for all the security bugs, and give each one a going-over. This can be as lightweight as reading each bug and asking yourself, “Did I do the right thing?” or you could use a short checklist, an example of which (“Validating threats”) is included at the end of this chapter in the “Checklists for Diving in and Threat Modeling” section.

Checking Your Tests

For each threat that you address, ensure you’ve built a good test to detect the problem. Your test can be a manual testing process or an automated test. Some of these tests will be easy, and others very tricky. For example, if you want to ensure that no static web page under /beta can be accessed without the beta cookie, you can build a quick script that retrieves all the pages from your source repository, constructs a URL for it, and tries to collect the page. You could extend the script to send a cookie with each request, and then re-request with an admin cookie. Ideally, that’s easy to do in your existing web testing framework. It gets a little more complex with dynamic pages, and a lot more complex when the security risk is something such as SQL injection or secure parsing of user input. There are entire books written on those subjects, not to mention entire books on the subject of testing. The key question you should ask is something like “Are my security tests in line with the other software tests and the sorts of risks that failures expose?”

Threat Modeling on Your Own

You have now walked through your fi rst threat model. Congratulations! Remember though: You’re not going to get to Carnegie Hall if you don’t practice, practice, practice. That means it is time to do it again, this time on your own, because doing it again is the only way to get better. Pick a system you’re working on and threat model it. Follow this simplifi ed, fi ve-step process as you go:

1. Draw a diagram.

2. Use the EoP game to fi nd threats.

www.it-ebooks.info

Chapter 1 ■ Dive In and Threat Model! 27

c01.indd 11:33:50:AM 01/17/2014 Page 27

3. Address each threat in some way.

4. Check your work with the checklists at the end of this chapter.

5. Celebrate and share your work.

Right now, if you’re new to threat modeling, your best bet is to do it often, applying it to the software and systems that matters to you. After threat model- ing a few systems, you’ll fi nd yourself getting more comfortable with the tools and techniques. For now, the thing to do is practice. Build your fi rst muscles to threat model with.

This brings up the question, what should you threat model next? What you’re working on now is the fi rst place to look for the next system to

threat model. If it has a trust boundary of some sort, it may be a good candidate. If it’s too simple to have trust boundaries, threat modeling it probably won’t be very satisfying. If it has too many boundaries, it may be too big a project to chew on all at once. If you’re collaborating closely on it with a few other people who you trust, that may be a good opportunity to play EoP with them. If you’re working on a large team, or across organizational boundaries, or things are tense, then those people may not be good fi rst collaborators on threat modeling. Start with what you’re working on now, unless there are tangible reasons to wait.

Checklists for Diving In and Threat Modeling

There’s a lot in this chapter. As you sit down to really do the work yourself, it can be tricky to assess how you’re doing. Here are some checklists that are designed to help you avoid the most common problems. Each question is designed to be read aloud and to have an affi rmative answer from everyone present. After read- ing each question out loud, encourage questions or clarifi cation from everyone else involved.

Diagramming

1. Can we tell a story without changing the diagram?

2. Can we tell that story without using words such as “sometimes” or “also”?

3. Can we look at the diagram and see exactly where the software will make a security decision?

4. Does the diagram show all the trust boundaries, such as where different accounts interact? Do you cover all UIDs, all application roles, and all network interfaces?

5. Does the diagram refl ect the current or planned reality of the software?

www.it-ebooks.info

28 Part I ■ Getting Started

c01.indd 11:33:50:AM 01/17/2014 Page 28

6. Can we see where all the data goes and who uses it?

7. Do we see the processes that move data from one data store to another?

Threats

1. Have we looked for each of the STRIDE threats?

2. Have we looked at each element of the diagram?

3. Have we looked at each data fl ow in the diagram?

N O T E Data fl ows are a type of element, but they are sometimes overlooked as people get started, so question 3 is a belt-and-suspenders question to add redundancy. (A belt-

and-suspenders approach ensures that a gentleman’s pants stay up.)

Validating Threats

1. Have we written down or fi led a bug for each threat?

2. Is there a proposed/planned/implemented way to address each threat?

3. Do we have a test case per threat?

4. Has the software passed the test?

Summary

Any technical professional can learn to threat model. Threat modeling involves the intersection of two models: a model of what can go wrong (threats), applied to a model of the software you’re building or deploying, which is encoded in a diagram. One model of threats is STRIDE: spoofi ng, tampering, repudiation, information disclosure, denial of service, and elevation of privilege. This model of threats has been made into the Elevation of Privilege game, which adds struc- ture and hints to the model.

With a whiteboard diagram and a copy of Elevation of Privilege, developers can threat model software that they’re building, systems administrators can threat model software they’re deploying or a system they’re constructing, and security professionals can introduce threat modeling to those with skillsets outside of security.

It’s important to address threats, and the STRIDE threats are the inverse of properties you want. There are mitigation strategies and techniques for devel- opers and for systems administrators.

Once you’ve created a threat model, it’s important to check your work by making sure you have a good model of the software in an up-to-date diagram, and that you’ve checked each threat you’ve found.

www.it-ebooks.info

29

c02.indd 11:35:5:AM 01/17/2014 Page 29

The earlier you fi nd problems, the easier it is to fi x them. Threat modeling is all about fi nding problems, and therefore it should be done early in your develop- ment or design process, or in preparing to roll out an operational system. There are many ways to threat model. Some ways are very specifi c, like a model air- plane kit that can only be used to build an F-14 fi ghter jet. Other methods are more versatile, like Lego building blocks that can be used to make a variety of things. Some threat modeling methods don’t combine easily, in the same way that Erector set pieces and Lego set blocks don’t fi t together. This chapter covers the various strategies and methods that have been brought to bear on threat modeling, presents each one in depth, and sets the stage for effectively fi nding threats.

You’ll start with very simple methods such as asking “what’s your threat model?” and brainstorming about threats. Those can work for a security expert, and they may work for you. From there, you’ll learn about three strategies for threat modeling: focusing on assets, focusing on attackers, and focusing on software. These strategies are more structured, and can work for people with different skillsets. A focus on software is usually the most appropriate strategy. The desire to focus on assets or attackers is natural, and often presented as an unavoidable or essential aspect of threat modeling. It would be wrong not to present each in its best light before discussing issues with those strategies. From there, you’ll learn about different types of diagrams you can use to model your system or software.

C H A P T E R

2

Strategies for Threat Modeling

www.it-ebooks.info

30 Part I ■ Getting Started

c02.indd 11:35:5:AM 01/17/2014 Page 30

N O T E This chapter doesn’t include the specifi c threat building blocks that discover threats, which are the subject of the next few chapters.

“What’s Your Threat Model?”

The question “what’s your threat model?” is a great one because in just four words, it can slice through many conundrums to determine what you are worried about. Answers are often of the form “an attacker with the laptop” or “ insiders,” or (unfortunately, often) “huh?” The “huh?” answer is useful because it reveals how much work would be needed to fi nd a consistent and structured approach to defense. Consistency and structure are important because they help you invest in defenses that will stymie attackers. There’s a compendium of standard answers to “what’s your threat model?” in Appendix A, “Helpful Tools,” but a few examples are listed here as well:

■ A thief who could steal your money

■ The company stakeholders (employees, consultants, shareholders, etc.) who access sensitive documents and are not trusted

■ An untrusted network

■ An attacker who could steal your cookie (web or otherwise)

N O T E Throughout this book, you’ll visit and revisit the same example for each of these approaches. Your main targets are the fi ctitious Acme Corporation’s “Acme/SQL,”

which is a commercial database server, and Acme’s operational network. Using Acme

examples, you can see how the diff erent approaches play out against the

same systems.

Applying the question “what’s your threat model?” to the Acme Corporation example, you might get the following answers:

■ For the Acme SQL database, the threat model would be an attacker who wants to read or change data in the database. A more subtle model might also include people who want to read the data without showing up in the logs.

■ For Acme’s fi nancial system, the answers might include someone getting a check they didn’t deserve, customers who don’t make a payment they owe, and/or someone reading or altering fi nancial results before reporting.

If you don’t have a clear answer to the question, “what’s your threat model?” it can lead to inconsistency and wasted effort. For example, start-up Zero-Knowledge

www.it-ebooks.info

Chapter 2 ■ Strategies for Threat Modeling 31

c02.indd 11:35:5:AM 01/17/2014 Page 31

Systems didn’t have a clear answer to the question “what’s your threat model?” Because there was no clear answer, there wasn’t consistency in what security features were built. A great deal of energy went into building defenses against the most complex attacks, and these choices to defend against such attackers had performance impacts on the whole system. While preventing governments from spying on customers was a fun technical challenge and an emotionally resonant goal, both the challenge and the emotional impact made it hard to make techni- cal decisions that could have made the business more successful. Eventually, a clearer answer to “what’s your threat model?” let Zero-Knowledge Systems invest in mitigations that all addressed the same subset of possible threats.

So how do you ensure you have a clear answer to this question? Often, the answers are not obvious, even to those who think regularly about security, and the question itself offers little structure for fi guring out the answers. One approach, often recommended is to brainstorm. In the next section, you’ll learn about a variety of approaches to brainstorming and the tradeoffs associated with those approaches.

Brainstorming Your Threats

Brainstorming is the most traditional way to enumerate threats. You get a set of experienced experts in a room, give them a way to take notes (white- boards or cocktail napkins are traditional) and let them go. The quality of the brainstorm is bounded by the experience of the brainstormers and the amount of time spent brainstorming.

Brainstorming involves a period of idea-generation, followed by a period of analyzing and selecting the ideas. Brainstorming for threat modeling involves coming up with possible attacks of all sorts. During the idea generation phase, you should forbid criticism. You want to explore the space of possible threats, and an atmosphere of criticism will inhibit such idea generation. A moderator can help keep brainstorming moving.

During brainstorming, it is key to have an expert on the technology being modeled in the room. Otherwise, it’s easy to make bad assumptions about how it works. However, when you have an expert who’s proud of their technology, you need to ensure that you don’t end up with a “proud parent” offended that their software baby is being called ugly. A helpful rule is that it’s the software being attacked, not the software architects. That doesn’t always suffi ce, but it’s a good start. There’s also a benefi t to bringing together a diverse grouping of experts with a broader set of experience.

Brainstorming can also devolve into out-of-scope attacks. For example, if you’re designing a chat program, attacks by the memory management unit against the CPU are probably out of scope, but if you’re designing a motherboard,

www.it-ebooks.info

32 Part I ■ Getting Started

c02.indd 11:35:5:AM 01/17/2014 Page 32

these attacks may be the focus of your threat modeling. One way to handle this issue is to list a set of attacks that are out of scope, such as “the administrator is malicious” or “an attacker edits the hard drive on another system,” as well as a set of attack equivalencies, like, “an attacker can smash the stack and execute code,” so that those issues can be acknowledged and handled in a consistent way. A variant of brainstorming is the exhortation to “think like an attacker,” which is discussed in more detail in Chapter 18, “Experimental Approaches.”

Some attacks you might brainstorm in threat modeling the Acme’s fi nancial statements include breaking in over the Internet, getting the CFO drunk, bribing a janitor, or predicting the URL where the fi nancials will be published. These can be a bit of a grab bag, so the next section provides somewhat more focused approaches.

Brainstorming Variants

Free-form or “normal” brainstorming, as discussed in the preceding sec- tion, can be used as a method for threat modeling, but there are more specifi c methods you can use to help focus your brainstorming. The following sections describe variations on classic brainstorming: scenario analyses, pre-mortems, and movie-plotting.

Scenario Analysis

It may help to focus your brainstorming with scenarios. If you’re using written scenarios in your overall engineering, you might start from those and ask what might go wrong, or you could use a variant of Chandler’s law (“When in doubt, have a man come through a door with a gun in his hand.”) You don’t need to restrict yourself to a man with a gun, of course; you can use any of the attackers listed in Appendix C, “Attacker Lists.”

For an example of scenario-specifi c brainstorming, try to threat model for handing your phone to a cute person in a bar. It’s an interesting exercise. The recipient could perhaps text donations to the Red Cross, text an important person to “stop bothering me,” or post to Facebook that “I don’t take hints well” or “I’m skeevy,” not to mention possibilities of running away with the phone or dropping it in a beer.

Less frivolously, your sample scenarios might be based on the product scenarios or use cases for the current development cycle, and therefore cover failover and replication, and how those services could be exploited when not properly authenticated and authorized.

Pre-Mortem

Decision-sciences expert Gary Klein has suggested another brainstorming tech- nique he calls the pre-mortem (Klein, 1999). The idea is to gather those involved

www.it-ebooks.info

Chapter 2 ■ Strategies for Threat Modeling 33

c02.indd 11:35:5:AM 01/17/2014 Page 33

in a decision and ask them to assume that it’s shortly after a project deadline, or after a key milestone, and that things have gone totally off the rails. With an “assumption of failure” in mind, the idea is to explore why the participants believe it will go off the rails. The value to calling this a pre-mortem is the fram- ing it brings. The natural optimism that accompanies a project is replaced with an explicit assumption that it has failed, giving you and other participants a chance to express doubts. In threat modeling, the assumption is that the product is being successfully attacked, and you now have permission to express doubts or concerns.

Movie Plotting

Another variant of brainstorming is movie plotting. The key difference between “normal brainstorming” and “movie plotting” is that the attack ideas are intended to be outrageous and provocative to encourage the free fl ow of ideas. Defending against these threats likely involves science-fi ction-type devices that impinge on human dignity, liberty, and privacy without actually defending anyone. Examples of great movies for movie plot threats include Ocean’s Eleven, The Italian Job, and every Bond movie that doesn’t stink. If you’d like to engage in more structured movie plotting, create three lists: fl awed protagonists, brilliant antagonists, and whiz-bang gadgetry. You can then combine them as you see fi t.

Examples of movie plot threats include a foreign spy writing code for Acme SQL so that a fourth connection attempt lets someone in as admin, a scheming CFO stealing from the fi rm, and someone rappelling from the ceiling to avoid the pressure mats in the fl oor while hacking into the database from the console. Note that these movie plots are equally applicable to Acme and its customers.

The term movie plotting was coined by Bruce Schneier, a respected security expert. Announcing his contest to elicit movie plot threats, he said: “The purpose of this contest is absurd humor, but I hope it also makes a point. Terrorism is a real threat, but we’re not any safer through security measures that require us to correctly guess what the terrorists are going to do next” (Schneier, 2006). The point doesn’t apply only to terrorism; convoluted but vividly described threats can be a threat to your threat modeling methodology.

Literature Review

As a precursor to brainstorming (or any other approach to fi nding threats), reviewing threats to systems similar to yours is a helpful starting point in threat modeling. You can do this using search engines, or by checking the academic literature and following citations. It can be incredibly helpful to search on competitors or related products. To start, search on a competitor, appending terms such as “security,” “security bug,” “penetration test,” “pwning,” or “Black Hat,” and use your creativity. You can also review common threats in this book,

www.it-ebooks.info

34 Part I ■ Getting Started

c02.indd 11:35:5:AM 01/17/2014 Page 34

especially Part III, “Managing and Addressing Threats” and the appendixes. Additionally, Ross Anderson’s Security Engineering is a great collection of real world attacks and engineering lessons you can draw on, especially if what you’re building is similar to what he covers (Wiley, 2008).

A literature review of threats against databases might lead to an understanding of SQL injection attacks, backup failures, and insider attacks, suggesting the need for logs. Doing a review is especially helpful for those developing their skills in threat modeling. Be aware that a lot of the threats that may come up can be super-specifi c. Treat them as examples of more general cases, and look for variations and related problems as you brainstorm.

Perspective on Brainstorming

Brainstorming and its variants suffer from a variety of problems. Brainstorming often produces threats that are hard or impossible to address. Brainstorming inten- tionally requires the removal of scoping or boundaries, and the threats are very dependent on the participants and how the session happens to progress. When experts get together, unstructured discussion often ensues. This can be fun for the experts and it usually produces interesting results, but oftentimes, experts are in short supply. Other times, engineers get frustrated with the inconsistency of “ask two experts, get three answers.”

There’s one other issue to consider, and that relates to exit criteria. It’s diffi cult to know when you’re done brainstorming, and whether you’re done because you have done a good job or if everyone is just tired. Engineering management may demand a time estimate that they can insert into their schedule, and these are diffi cult to predict. The best approach to avoid this timing issue is simply to set a meeting of defi ned length. Unfortunately, this option doesn’t provide a high degree of confi dence that all interesting threats have been found.

Because of the diffi culty of addressing threats illuminated with a limitless brainstorming technique and the poorly defi ned exit criteria to a brainstorming session, it is important to consider other approaches to threat modeling that are more prescriptive, formal, repeatable, or less dependent on the aptitudes and knowledge of the participants. Such approaches are the subject of the rest of this chapter and also discussed in the rest of Part II, “Finding Threats.”

Structured Approaches to Threat Modeling

When it’s hard to answer “What’s your threat model?” people often use an approach centered on models of their assets, models of attackers, or models of their software. Centering on one of those is preferable to using approaches that attempt to combine them because these combinations tend to be confusing.

www.it-ebooks.info

Chapter 2 ■ Strategies for Threat Modeling 35

c02.indd 11:35:5:AM 01/17/2014 Page 35

Assets are the valuable things you have. The people who might go after your assets are attackers, and the most common way for them to attack is via the software you’re building or deploying.

Each of these is a natural place to start thinking about threats, and each has advantages and disadvantages, which are covered in this section. There are people with very strong opinions that one of these is right (or wrong). Don’t worry about “right” or “wrong,” but rather “usefulness.” That is, does your approach help you fi nd problems? If it doesn’t, it’s wrong for you, however forcefully someone might argue its merits.

These three approaches can be thought of as analogous to Lincoln Log sets, Erector sets, and Lego sets. Each has a variety of pieces, and each enables you to build things, but they may not combine in ways as arbitrary as you’d like. That is, you can’t snap Lego blocks to an Erector set model. Similarly, you can’t always snap attackers onto a software model and have something that works as a coherent whole.

To understand these three approaches, it can be useful to apply them to some- thing concrete. Figure 2-1 shows a data fl ow diagram of the Acme/SQL system.

Web Clients

SQL Clients

Acme Front End(s)

External Entity

Key:

Process Data Store

DB Admin

Data Management Logs

Log Analysis

Original SQL Account

DB Cluster

DBA (Human) DB

Users (Human)

Database

data flow Trust

Boundary

Figure 2-1: Data flow diagram of the Acme/SQL database

Looking at the diagram, and reading from left to right, you can see two types of clients accessing the front ends and the core database, which manages

www.it-ebooks.info

36 Part I ■ Getting Started

c02.indd 11:35:5:AM 01/17/2014 Page 36

transactions, access control, atomicity, and so on. Here, assume that the Acme/ SQL system comes with an integrated web server, and that authorized clients are given nearly raw access to the data. There could simultaneously be web servers offering deeper business logic, access to multiple back ends, integration with payment systems, and so on. Those web servers would access Acme/SQL via the SQL protocol over a network connection.

Back to the diagram, Figure 2-1 also shows there is also a set of DB Admin tools that the DBA (the human database administrator) uses to manage the system. As shown in the diagram, there are three conceptual data stores: Data, Management (including metadata such as locks, policies, and indices), and Logs. These might be implemented in memory, as fi les, as custom stores on raw disk, or delegated to network storage. As you dig in, the details matter greatly, but you usually start modeling from the conceptual level, as shown.

Finally, there’s a log analysis package. Note that only the database core has direct access to the data and management information in this design. You should also note that most of the arrows are two-way, except Database ➪ Logs and Logs ➪ Log Analysis. Of course, the Log Analysis process will be query- ing the logs, but because it’s intended as a read-only interface, it is represented as a one-way arrow. Very occasionally, you might have strictly one-way fl ows, such as those implemented by SNMP traps or syslog. Some threat modelers prefer two one-way arrows, which can help you see threats in each direction, but also lead to busy diagrams that are hard to label or read. If your diagrams are simple, the pair of one way arrows helps you fi nd threats, and is therefore better than two-way. If your diagram is complex, either approach can be used.

The data fl ow diagram is discussed in more detail later in this chapter, in the section “Data Flow Diagrams.” In the next few sections, you’ll see how to apply asset, attacker, and software-centric models to fi nd threats against Acme/SQL.

Focusing on Assets

It seems very natural to center your approach on assets, or things of value. After all, if a thing has no value, why worry about how someone might attack it? It turns out that focusing on assets is less useful than you may hope, and is therefore not the best approach to threat modeling. However, there are a small number of people who will benefi t from asset-centered threat modeling. The most likely to benefi t are a team of security experts with experience structuring their thinking around assets. (Having found a way that works for them, there may be no reason to change.) Less technical people may be able to contribute to threat modeling by saying “focus on this asset.” If you are in either of these groups, or work with them, congratulations! This section is for you. If you aren’t one of those people, however, don’t be too quick to skip ahead. It is still important

www.it-ebooks.info

Chapter 2 ■ Strategies for Threat Modeling 37

c02.indd 11:35:5:AM 01/17/2014 Page 37

to have a good understanding of the role assets can play in threat modeling, even if they’re not in a starring role). It can also help for you to understand why the approach is not as useful as it may appear, so you can have an informed discussion with those advocating it.

The term asset usually refers to something of value. When people bring up assets as part of a threat modeling activity, they often mean something an attacker wants to access, control, or destroy. If everyone who touches the threat modeling process doesn’t have a working agreement about what an asset is, your process will either get bogged down, or participants will just talk past each other.

There are three ways the term asset is commonly used in threat modeling:

■ Things attackers want

■ Things you want to protect

■ Stepping stones to either of these

You should think of these three types of assets as families rather than categories because just as people can belong to more than one family at a time, assets can take on more than one meaning at a time. In other words, the tags that apply to assets can overlap, as shown in Figure 2-2. The most common usage of asset in discussing threat models seems to be a marriage of “things attackers want” and “things you want to protect.”

Stepping stonesThings youprotect

Things attackers want

Figure 2-2: The overlapping definitions of assets

N O T E There are a few other ways in which the term asset is used by those who are threat modeling—such as a synonym for computer, or a type of computer (for

example, “Targeted assets: mail server, database”). For the sake of clarity, this book

only uses asset with explicit reference to one or more of the three families previously

defi ned, and you should try to do the same.

www.it-ebooks.info

38 Part I ■ Getting Started

c02.indd 11:35:5:AM 01/17/2014 Page 38

Things Attackers Want

Usually assets that attackers want are relatively tangible things such as “this database of customer medical data.” Good examples of things attackers want include the following:

■ User passwords or keys

■ Social security numbers or other identifi ers

■ Credit card numbers

■ Your confi dential business data

Things You Want to Protect

There’s also a family of assets you want to protect. Unlike the tangible things attackers want, many of these assets are intangibles. For example, your com- pany’s reputation or goodwill is something you want to protect. Competitors or activists may well attack your reputation by engaging in smear tactics. From a threat modeling perspective, your reputation is usually too diffuse to be able to technologically mitigate threats against it. Therefore, you want to protect it by protecting the things that matter to your customers.

As an example of something you want to protect, if you had an empty safe, you intuitively don’t want someone to come along and stick their stethoscope to it. But there’s nothing in it, so what’s the damage? Changing the combination and letting the right folks (but only the right folks) know the new combination requires work. Therefore you want to protect this empty safe, but it would be an unlikely target for a thief. If that same safe has one million dollars in it, it would be much more likely to pique a thief’s interest. The million dollars is part of the family of things you want that attackers want, too.

Stepping Stones

The fi nal family of assets is stepping stones to other assets. For example, every- thing drawn in a threat model diagram is something you want to protect because it may be a stepping stone to the targets that attackers want. In some ways, the set of stepping stone assets is an attractive nuisance. For example, every com- puter has CPU and storage that an attacker can use. Most also have Internet connectivity, and if you’re in systems management or operational security, many of the computers you worry most about will have special access to your organization’s network. They’re behind a fi rewall or have VPN access. These are stepping stones. If they are uniquely valuable stepping stones in some way, note that. In practice, it’s rarely helpful to include “all our PCs” in an asset list.

www.it-ebooks.info

Chapter 2 ■ Strategies for Threat Modeling 39

c02.indd 11:35:5:AM 01/17/2014 Page 39

N O T E Referring back to the safe example in the previous section, the safe combina- tion is a member of the stepping stone family. It may well be that stepping-stones and

things you protect are, in practice, very similar. The list of technical elements you

protect that are not members of the stepping-stone family appears fairly short.

Implementing Asset-Centric Modeling

If you were to threat model with an asset-focused approach, you would make a list of your assets and then consider how an attacker could threaten each. From there, you’d consider how to address each threat.

After an asset list is created, you should connect each item on the list to particular computer systems or sets of systems. (If an asset includes something like “Amazon Web Services” or “Microsoft Azure,” then you don’t need to be able to point to the computer systems in question, you just need to understand where they are—eventually you’ll need to identify the threats to those systems and determine how to mitigate them.)

The next step is to draw the systems in question, showing the assets and other components as well as interconnections, until you can tell a story about them. You can use this model to apply either an attack set like STRIDE or an attacker-centered brainstorm to understand how those assets could be attacked.

Perspective on Asset-Centric Threat Modeling

Focusing on assets appears to be a common-sense approach to threat model- ing, to the point where it seems hard to argue with. Unfortunately, much of the time, a discussion of assets does not improve threat modeling. However, the misconception is so common that it’s important to examine why it doesn’t help.

There’s no direct line from assets to threats, and no prescriptive set of steps. Essentially, effort put into enumerating assets is effort you’re not spending fi nd- ing or fi xing threats. Sometimes, that involves a discussion of what’s an asset, or which type of asset you’re discussing. That discussion, at best, results in a list of things to look for in your software or operational model, so why not start by creating such a model? Once you have a list of assets, that list is not (ahem) a stepping stone to fi nding threats; you still need to apply some methodology or approach. Finally, assets may help you prioritize threats, but if that’s your goal, it doesn’t mean you should start with or focus on assets. Generally, such informa- tion comes out naturally when discussing impacts as you prioritize and address threats. Those topics are covered in Part III, “Managing and Addressing Threats.”

How you answer the question “what are our assets?” should help focus your threat modeling. If it doesn’t help, there is no point to asking the question or spending time answering it.

www.it-ebooks.info

40 Part I ■ Getting Started

c02.indd 11:35:5:AM 01/17/2014 Page 40

Focusing on Attackers

Focusing on attackers seems like a natural way to threat model. After all, if no one is going to attack your system, why would you bother defending it? And if you’re worried because people will attack your systems, shouldn’t you understand them? Unfortunately, like asset-centered threat modeling, attacker- centered threat modeling is less useful than you might anticipate. But there are also a small number of scenarios in which focusing on attackers can come in handy, and they’re the same scenarios as assets: experts, less-technical input to your process, and prioritization. And similar to the “Focusing on Assets” section, you can also learn for yourself why this approach isn’t optimal, so you can discuss the possibility with those advocating this approach.

Implementing Attacker-Centric Modeling

Security experts may be able to use various types of attacker lists to fi nd threats against a system. When doing so, it’s easy to fi nd yourself arguing about the resources or capabilities of such an archetype, and needing to fl esh them out. For example, what if your terrorist is state-sponsored, and has access to govern- ment labs? These questions make the attacker-centric approach start to resemble “personas,” which are often used to help think about human interface issues. There’s a spectrum of detail in attacker models, from simple lists to data-derived personas, and examples of each are given in Appendix C, “Attacker Lists” That appendix may help security experts and will help anyone who wants to try attacker-centric modeling and learn faster than if they have to start by creating a list.

Given a list of attackers, it’s possible to use the list to provide some structure to a brainstorming approach. Some security experts use attacker lists as a way to help elicit the knowledge they’ve accumulated as they’ve become experts. Attacker-driven approaches are also likely to bring up possibilities that are human-centered. For example, when thinking about what a spy would do, it may be more natural (and fun) to think about them seducing your sysadmin or corrupting a janitor, rather than think about technical attacks. Worse, it will probably be challenging to think about what those human attacks mean for your system’s security.

Where Attackers Can Help You

Talking about human threat agents can help make the threats real. That is, it’s sometimes tough to understand how someone could tamper with a confi gura- tion fi le, or replace client software to get around security checks. Especially when dealing with management or product teams who “just want to ship,”

www.it-ebooks.info

Chapter 2 ■ Strategies for Threat Modeling 41

c02.indd 11:35:5:AM 01/17/2014 Page 41

it’s helpful to be able to explain who might attack them and why. There’s real value in this, but it’s not a suffi cient argument for centering your approach on those threat agents; you can add that information at a later stage. (The risk associated with talking about attackers is the claim that “no one would ever do that.” Attempting to humanize a risk by adding an actor can exacerbate this, especially if you add a type of actor who someone thinks “wouldn’t be interested in us.”).

You were promised an example, and the spies stole it. More seriously, carefully walking through the attacker lists and personas in Appendix C likely doesn’t help you (or the author) fi gure out what they might want to do to Acme/SQL, and so the example is left empty to avoid false hope.

Perspective on Attacker-Centric Modeling

Helping security experts structure and recall information is nice, but doesn’t lead to reproducible results. More importantly, attacker lists or even personas are not enough structure for most people to fi gure out what those people will do. Engineers may subconsciously project their own biases or approaches into what an attacker might do. Given that the attacker has his own motivations, skills, background, and perspective (and possibly organizational priorities), avoiding such projection is tricky.

In my experience, this combination of issues makes attacker-centric approaches less effective than other approaches. Therefore, I recommend against using attackers as the center of your threat modeling process.

Focusing on Software

Good news! You’ve offi cially reached the “best” structured threat modeling approach. Congrats! Read on to learn about software-centered threat modeling, why it’s the most helpful and effective approach, and how to do it.

Software-centric models are models that focus on the software being built or a system being deployed. Some software projects have documented models of various sorts, such as architecture, UML diagrams, or APIs. Other projects don’t bother with such documentation and instead rely on implicit models.

Having large project teams draw on a whiteboard to explain how the software fi ts together can be a surprisingly useful and contentious activity. Understandings differ, especially on large projects that have been running for a while, but fi nd- ing where those understandings differ can be helpful in and of itself because it offers a focal point where threats are unlikely to be blocked. (“I thought you were validating that for a SQL call!”)

The same complexity applies to any project that is larger than a few people or has been running longer than a few years. Projects accumulate complexity,

www.it-ebooks.info

42 Part I ■ Getting Started

c02.indd 11:35:5:AM 01/17/2014 Page 42

which makes many aspects of development harder, including security. Software- centric threat modeling can have a useful side effect of exposing this accumu- lated complexity.

The security value of this common understanding can also be substantial, even before you get to looking for threats. In one project it turned out that a library on a trust boundary had a really good threat model, but unrealistic assumptions had been made about what the components behind it were doing. The work to create a comprehensive model led to an explicit list of common assumptions that could and could not be made. The comprehensive model and resultant understanding led to a substantial improvement in the security of those components.

N O T E As complexity grows, so will the assumptions that are made, and such lists are never complete. They grow as experience requires and feedback loops allow.

Threat Modeling Diff erent Types of Software

The threat discovery approaches covered in Part II, can be applied to models of all sorts of software. They can be applied to software you’re building for others to download and install, as well as to software you’re building into a larger operational system. The software they can be applied to is nearly endless, and is not dependent on the business model or deployment model associated with the software.

Even though software no longer comes in boxes sold on store shelves, the term boxed software is a convenient label for a category. That category is all the software whose architecture is defi nable, because there’s a clear edge to what is the software: It’s everything in the box (installer, application download, or open source repository). This edge can be contrasted with the deployed systems that organizations develop and change over time.

N O T E You may be concerned that the techniques in this book focus on either boxed software or deployed systems, and that the one you’re concerned about isn’t

covered. In the interests of space, the examples and discussion only cover both when

there’s a clear diff erence and reason. That’s because the recommended ways to model

software will work for both with only a few exceptions.

The boundary between boxed software models and network models gets blurrier every year. One important difference is that the network models tend to include more of the infrastructural components such as routers, switches, and data circuits. Trust boundaries are often operationalized by these components, or by whatever group operates the network, the platforms, or the applications.

www.it-ebooks.info

Chapter 2 ■ Strategies for Threat Modeling 43

c02.indd 11:35:5:AM 01/17/2014 Page 43

Data fl ow models (which you met in Chapter 1, “Dive In and Threat Model!” and which you’ll learn more about in the next section) are usually a good choice for both boxed software and operational models. Some large data center operators have provided threat models to teams, showing how the data center is laid out. The product group can then overlay its models “on top” of that to align with the appropriate security controls that they’ll get from operations. When you’re using someone else’s data center, you may have discussions about their infrastructure choices that make it easy to derive a model, or you might have to assume the worst.

Perspective on Software-Centric Modeling

I am fond of software-centric approaches because you should expect software developers to understand the software they’re developing. Indeed, there is nothing else you should expect them to understand better. That makes software an ideal place to start the threat-modeling tasks in which you ask developers to participate. Almost all software development is done with software models that are good enough for the team’s purposes. Sometimes they require work to make them good enough for effective threat modeling.

In contrast, you can merely hope that developers understand the business or its assets. You may aspire to them understanding the people who will attack their product or system. But these are hopes and aspirations, rather than reasonable expectations. To the extent that your threat modeling strategy depends on these hopes and aspirations, you’re adding places where it can fail. The remainder of this chapter is about modeling your software in ways that help you fi nd threats, and as such enabling software centric-modeling. (The methods for fi nding these threats are covered in the rest of Part II.)

Models of Software

Making an explicit model of your software helps you look for threats without getting bogged down in the many details that are required to make the software function properly. Diagrams are a natural way to model software.

As you learned in Chapter 1, whiteboard diagrams are an extremely effective way to start threat modeling, and they may be suffi cient for you. However, as a system hits a certain level of complexity, drawing and redrawing on white- boards becomes infeasible. At that point, you need to either simplify the system or bring in a computerized approach.

In this section, you’ll learn about the various types of diagrams, how they can be adapted for use in threat modeling, and how to handle the complexities of larger systems. You’ll also learn more detail about trust boundaries, effective labeling, and how to validate your diagrams.

www.it-ebooks.info

44 Part I ■ Getting Started

c02.indd 11:35:5:AM 01/17/2014 Page 44

Types of Diagrams

There are many ways to diagram, and different diagrams will help in differ- ent circumstances. The types of diagrams you’ll encounter most frequently are probably data fl ow diagrams (DFDs). However, you may also see UML, swim lane diagrams, and state diagrams. You can think of these diagrams as Lego blocks, looking them over to see which best fi ts whatever you’re building. Each diagram type here can be used with the models of threats in Part II.

The goal of all these diagrams is to communicate how the system works, so that everyone involved in threat modeling has the same understanding. If you can’t agree on how to draw how the software works, then in the process of getting to agreement, you’re highly likely to discover misunderstandings about the security of the system. Therefore, use the diagram type that helps you have a good conversation and develop a shared understanding.

Data Flow Diagrams

Data fl ow models are often ideal for threat modeling; problems tend to follow the data fl ow, not the control fl ow. Data fl ow models more commonly exist for network or architected systems than software products, but they can be created for either.

Data fl ow diagrams are used so frequently they are sometimes called “threat model diagrams.” As laid out by Larry Constantine in 1967, DFDs consist of numbered elements (data stores and processes) connected by data fl ows, interacting with external entities (those outside the developer’s or the organization’s control).

The data fl ows that give DFDs their name almost always fl ow two ways, with exceptions such as radio broadcasts or UDP data sent off into the Ethernet. Despite that, fl ows are usually represented using one-way arrows, as the threats and their impact are generally not symmetric. That is, if data fl owing to a web server is read, it might reveal passwords or other data, whereas a data fl ow from the web server might reveal your bank balance. This diagramming convention doesn’t help clarify channel security versus message security. (The channel might be something like SMTP, with messages being e-mail messages.) Swim lane diagrams may be more appropriate as a model if this channel/message distinction is important. (Swim lane diagrams are described in the eponymous subsection later in this chapter.)

The main elements of a data fl ow diagram are shown in Table 2-1.

www.it-ebooks.info

Chapter 2 ■ Strategies for Threat Modeling 45

c02.indd 11:35:5:AM 01/17/2014 Page 45

Table 2-1: Elements of a Data Flow Diagram

ELEMENT APPEARANCE MEANING EXAMPLES

Process Rounded rect- angle, circle, or concentric circles

Any running code Code written in C, C#, Python, or PHP

Data fl ow Arrow Communication between processes, or between processes and data stores

Network connec- tions, HTTP, RPC, LPC

Data store Two parallel lines with a label between them

Things that store data Files, databases, the Windows Registry, shared memory segments

External entity

Rectangle with sharp corners

People, or code outside your control

Your customer, Microsoft.com

Figure 2-3 shows a classic DFD based on the elements from Table 2-1; how- ever, it’s possible to make these models more usable. Figure 2-4 shows this same model with a few changes, which you can use as an example for improving your own models.

1 Web Clients

2 SQL Clients

3 Front End(s)

External Entity

Key:

Process Data Store

5 DB Admin

9 Data 10 Management 11 Logs

8 Log Analysis

6 DBA (Human) 7 DBUsers

4 Database

data flow

Figure 2-3: A classic DFD model

www.it-ebooks.info

46 Part I ■ Getting Started

c02.indd 11:35:5:AM 01/17/2014 Page 46

Web Clients

SQL Clients

Acme Front End(s)

External Entity

Key:

Process Data Store

DB Admin

Data Management Logs

Log Analysis

Original SQL Account

DB Cluster

DBA (Human) DB

Users (Human)

Database

data flow Trust

Boundary

Figure 2-4: A modern DFD model (previously shown as Figure 2-1)

The following list explains the changes made from classic DFDs to more modern ones:

■ The processes are rounded rectangles, which contain text more effi ciently than circles.

■ Straight lines are used, rather than curved, because straight lines are easier to follow, and you can fi t more in larger diagrams.

Historically, many descriptions of data flow diagrams contained both “process” elements and “complex process” elements. A process was depicted as a circle, a complex process as two concentric circles. It isn’t entirely clear, however, when to use a normal process versus a complex one. One possible rule is that anything that has a subdiagram should be a complex process. That seems like a decent rule, if (ahem) a bit circular.

DFDs can be used for things other than software products. For example, Figure 2-5 shows a sample operational network in a DFD. This is a typical model for a small to mid-sized corporate network, with a representative sampling

www.it-ebooks.info

Chapter 2 ■ Strategies for Threat Modeling 47

c02.indd 11:35:5:AM 01/17/2014 Page 47

of systems and departments shown. It is discussed in depth in Appendix E, “Case Studies.”

Acme Corporate Network

Internet

Payroll

HR

Directory

Sales/CRM

Operations

Production

Desktop & Mobile

E-mail & Intranet Servers

Development Servers

HR Mgmt

Figure 2-5: An operational network model

UML

UML is an abbreviation for Unifi ed Modeling Language. If you use UML in your software development process, it’s likely that you can adapt UML diagrams for threat modeling, rather than redrawing them. The most impor- tant way to adapt UML for threat modeling diagrams is the addition of trust boundaries.

UML is fairly complex. For example, the Visio stencils for UML offer roughly 80 symbols, compared to six for DFDs. This complexity brings a good deal of nuance and expressiveness as people draw structure diagrams, behavior diagrams, and interaction diagrams. If anyone involved in the threat model- ing isn’t up on all the UML symbols, or if there’s misunderstanding about what those symbols mean, then the diagram’s effectiveness as a tool is greatly diminished. In theory, anyone who’s confused can just ask, but that requires them to know they’re confused (they might assume that the symbol for fi sh

www.it-ebooks.info

48 Part I ■ Getting Started

c02.indd 11:35:5:AM 01/17/2014 Page 48

excludes sharks). It also requires a willingness to expose one’s ignorance by asking a “simple” question. It’s probably easier for a team that’s invested in UML to add trust boundaries to those diagrams than to create new diagrams just for threat modeling.

Swim Lane Diagrams

Swim lane diagrams are a common way to represent fl ows between various participants. They’re drawn using long lines, each representing participants in a protocol, with each participant getting a line. Each lane edge is labeled to identify the participant; each message is represented by a line between participants; and time is represented by fl ow down the diagram lanes. The diagrams end up looking a bit like swim lanes, thus the name. Messages should be labeled with their contents; or if the contents are complex, it may make more sense to have a diagram key that abstracts out some details. Computation done by the parties or state should be noted along that partici- pant’s line. Generally, participants in such protocols are entities like comput- ers; and as such, swim lane diagrams usually have implicit trust boundaries between each participant. Cryptographer and protocol designer Carl Ellison has extended swim lanes to include the human participants as a way to structure discussion of what people are expected to know and do. He calls this extension ceremonies, which is discussed in more detail in Chapter 15, “Human Factors and Usability.”

A sample swim lane diagram is shown in Figure 2-6.

SYN

SYN-ACK

ACK

Data

Client Server

Figure 2-6: Swim lane diagram (showing the start of a TCP connection)

www.it-ebooks.info

Chapter 2 ■ Strategies for Threat Modeling 49

c02.indd 11:35:5:AM 01/17/2014 Page 49

State Diagrams

State diagrams represent the various states a system can be in, and the transi- tions between those states. A computer system is modeled as a machine with state, memory, and rules for moving from one state to another, based on the valid messages it receives, and the data in its memory. (The computer should course test the messages it receives for validity according to some rules.) Each box is labeled with a state, and the lines between them are labeled with the conditions that cause the state transition. You can use state diagrams in threat modeling by checking whether each transition is managed in accordance with the appropriate security validations.

A very simple state machine for a door is shown in Figure 2-7 (derived from Wikipedia). The door has three states: opened, closed, and locked. Each state is entered by a transition. The “deadbolt” system is much easier to draw than locks on the knob, which can be locked from either state, creating a more complex diagram and user experience. Obviously, state diagrams can become complex quickly. You could imagine a more complex state diagram that includes “ajar,” a state that can result from either open or closed. (I started drawing that but had trouble deciding on labels. Obviously, doors that can be ajar are poorly specifi ed and should not be deployed.) You don’t want to make architectural decisions just to make modeling easier, but often simple models are easier to work with, and refl ect better engineering.

Opened

Closed Locked

State

Transition

Open doorClose door

Unlock deadbolt

Lock deadbolt

Transition condition

Figure 2-7: A state machine diagram

www.it-ebooks.info

50 Part I ■ Getting Started

c02.indd 11:35:5:AM 01/17/2014 Page 50

Trust Boundaries

As you saw in Chapter 1, a trust boundary is anyplace where various principals come together—that is, where entities with different privileges interact.

Drawing Boundaries

After a software model has been drawn, there are two ways to add boundaries: You can add the boundaries you know and look for more, or you can enumerate principals and look for boundaries. To start from boundaries, add any sorts of enforced trust boundary you can. Boundaries between unix UIDs, Windows sessions, machines, network segments, and so on should be drawn in as boxes, and the principal inside each box should be shown with a label.

To start from principals, begin from one end or the other of the privilege spectrum (often that’s root/admin or anonymous Internet users), and then add boundaries each time they talk to “someone else.”

You can always add at least one boundary, as all computation takes place in some context. (So you might criticize Figure 2-1 for showing Web Clients and SQL Clients without an identifi ed context.)

If you don’t see where to draw trust boundaries of any sort, your diagram may be detailed as everything is inside a single trust boundary, or you may be missing boundaries. Ask yourself two questions. First, does everything in the system have the same level of privilege and access to everything else on the system? Second, is everything your software communicates with inside that same boundary? If either of these answers are a no, then you should now have clarifi ed either a missing boundary or a missing element in the diagram, or both. If both are yes, then you should draw a single trust boundary around everything, and move on to other development activities. (This state is unlikely except when every part of a development team has to create a software model. That “bottom up” approach is discussed in more detail in Chapter 7, “Processing and Managing Threats.”)

A lot of writing on threat modeling claims that trust boundaries should only cross data fl ows. This is useful advice for the most detailed level of your model. If a trust boundary crosses over a data store (that is, a database), that might indicate that there are different tables or stored procedures with different trust levels. If a boundary crosses over a given host, it may refl ect that members of, for example, the group “software installers,” have different rights from the “web content updaters.” If you fi nd a trust boundary crossing an element of a diagram other than a data fl ow, either break that element into two (in the model, in reality, or both), or draw a subdiagram to show them separated into multiple entities. What enables good threat models is clarity about what boundaries exist and how those boundaries are to be protected. Contrariwise, a lack of clarity will inhibit the creation of good models.

www.it-ebooks.info

Chapter 2 ■ Strategies for Threat Modeling 51

c02.indd 11:35:5:AM 01/17/2014 Page 51

Using Boundaries

Threats tend to cluster around trust boundaries. This may seem obvious: The trust boundaries delineate the attack surface between principals. This leads some to expect that threats appear only between the principals on the boundary, or only matter on the trust boundaries. That expectation is sometimes incorrect. To see why, consider a web server performing some complex order processing. For example, imagine assembling a computer at Dell’s online store where thousands of parts might be added, but only a subset of those have been tested and are on offer. A model of that website might be constructed as shown in Figure 2-8.

Web browser TCP/IP stack

Kernel

Platform

Application

Web server

Sales

Order processing

Marketing

Figure 2-8: Trust boundaries in a web server

The web server in Figure 2-8 is clearly at risk of attack from the web browser, even though it talks through a TCP/IP stack that it presumably trusts. Similarly, the sales module is at risk; plus an attacker might be able to insert random part numbers into the HTML post in which the data is checked in an order processing module. Even though there’s no trust boundary between the sales module and the order processing module, and even though data might be checked at three boundaries, the threats still follow the data fl ows. The client is shown simply as a web browser because the client is an external entity. Of course, there are many other components around that web browser, but you can’t do anything about threats to them, so why model them?

Therefore, it is more accurate to say that threats tend to cluster around trust boundaries and complex parsing, but may appear anywhere that information is under the control of an attacker.

www.it-ebooks.info

52 Part I ■ Getting Started

c02.indd 11:35:5:AM 01/17/2014 Page 52

What to Include in a Diagram

So what should be in your diagram? Some rules of thumb include the following:

■ Show the events that drive the system.

■ Show the processes that are driven.

■ Determine what responses each process will generate and send.

■ Identify data sources for each request and response.

■ Identify the recipient of each response.

■ Ignore the inner workings, focus on scope.

■ Ask if something will help you think about what goes wrong, or what will help you fi nd threats.

This list is derived from Howard and LeBlanc’s Writing Secure Code, Second Edition (Microsoft Press, 2009).

Complex Diagrams

When you’re building complex systems, you may end up with complex diagrams. Systems do become complex, and that complexity can make using the diagrams (or understanding the full system) diffi cult.

One rule of thumb is “don’t draw an eye chart.” It is important to balance all the details that a real software project can entail with what you include in your actual model. As mentioned in Chapter 1, one technique you can use to help you do this is a subdiagram showing the details of one particular area. You should look for ways to break out highly-detailed areas that make sense for your project. For example, if you have one very complex process, maybe everything inside it is one diagram, and everything outside it is another. If you have a dispatcher or queuing system, that might be a good place to break things up. Maybe your databases or the fail over system is a good place to split. Maybe there are a few elements that really need more detail. All of these are good ways to break things out.

One helpful approach to subdiagrams is to ensure that there are not more subdiagrams than there are processes. Another approach is to use different diagrams to show different scenarios.

Sometimes it’s also useful to simplify diagrams. When two elements of the diagram are equivalent from a security perspective, you can combine them. Equivalent means inside the same trust boundary, relying on the same technol- ogy, and handling the same sort of data.

The key thing to remember is that the diagram is intended to help ensure that you understand and can discuss the system. Remember the quote that opens this book: “All models are wrong, some models are useful.” Therefore, when

www.it-ebooks.info

Chapter 2 ■ Strategies for Threat Modeling 53

c02.indd 11:35:5:AM 01/17/2014 Page 53

you’re adding additional diagrams, don’t ask “is this the right way to do it?” Instead, ask “does this help us think about what might go wrong?”

Labels in Diagrams

Labels in diagrams should be short, descriptive, and meaningful. Because you want to use these names to tell stories, start with the outsiders who are driving the system; those are nouns, such as “customer” or “vibration sensor.” They communicate information via data fl ows, which are nouns or noun phrases, such as “books to buy” or “vibration frequency.” Data fl ows should almost never be labeled using verbs. Even though it can be hard, you should work to fi nd more descriptive labels than “read” or “write,” which are implied by the direction of the arrows. In other words, data fl ows communicate their information (nouns) to processes, which are active: verbs, verb phrases, or verb/noun chains.

Many people fi nd it helpful to label data fl ows with sequence numbers to help keep track of what happens in what order. It can also be helpful to number ele- ments within a diagram to help with completeness or communication. You can number each thing (data fl ow 1, a process 1, et cetera) or you can have a single count across the diagram, with external entity 1 talking over data fl ows 2 and 3 to process 4. Generally, using a single counter for everything is less confusing. You can say “number 1” rather than “data fl ow 1, not process 1.”

Color in Diagrams

Color can add substantial amounts of information without appearing over- whelming. For example, Microsoft’s Peter Torr uses green for trusted, red for untrusted and blue for what’s being modeled (Torr, 2005). Relying on color alone can be problematic. Roughly one in twelve people suffer from color blindness, the most common being red/green confusion (Heitgerd, 2008). The result is that even with a color printer, a substantial number of people are unable to easily access this critical information. Box boundaries with text labels address both problems. With box trust boundaries, there is no reason not to use color.

Entry Points

One early approach to threat modeling was the “asset/entry point” approach, which can be effective at modeling operational systems. This approach can be partially broken down into the following steps:

1. Draw a DFD.

2. Find the points where data fl ows cross trust boundaries.

3. Label those intersections as “entry points.”

www.it-ebooks.info

54 Part I ■ Getting Started

c02.indd 11:35:5:AM 01/17/2014 Page 54

N O T E There were other steps and variations in the approaches, but we as a com- munity have learned a lot since then, and a full explanation would be tedious and

distracting.

In the Acme/SQL example (as shown in Figure 2-1) the entry points are the “front end(s)” and the “database admin” console process. “Database” would also be an entry point, because nominally, other software could alter data in the databases and use failures in the parsers to gain control of the system. For the fi nancials, the entry points shown are “external reporting,” “fi nancial plan- ning and analysis,” “core fi nance software,” “sales” and “accounts receivable.”

Validating Diagrams

Validating that a diagram is a good model of your software has two main goals: ensuring accuracy and aspiring to goodness. The fi rst is easier, as you can ask whether it refl ects reality. If important components are missing, or the diagram shows things that are not being built, then you can see that it doesn’t refl ect reality. If important data fl ows are missing, or nonexistent fl ows are shown, then it doesn’t refl ect reality. If you can’t tell a story about the software without editing the diagram, then it’s not accurate.

Of course, there’s that word “important” in there, which leads to the second criterion: aspiring to goodness. What’s important is what helps you fi nd issues. Finding issues is a matter of asking questions like “does this element have any security impact?” and “are there things that happen sometimes or in special circumstances?” Knowing the answers to these questions is a matter of expe- rience, just like many aspects of building software. A good and experienced architect can quickly assess requirements and address them, and a good threat modeler can quickly see which elements will be important. A big part of gain- ing that experience is practice. The structured approaches to fi nding threats in Part II, are designed to help you identify which elements are important.

How To Validate Diagrams

To best validate your diagrams, bring together the people who understand the system best. Someone should stand in front of the diagram and walk through the important use cases, ensuring the following:

■ They can talk through stories about the diagram.

■ They don’t need to make changes to the diagram in its current form.

■ They don’t need to refer to things not present in the diagram.

www.it-ebooks.info

Chapter 2 ■ Strategies for Threat Modeling 55

c02.indd 11:35:5:AM 01/17/2014 Page 55

The following rules of thumb will be useful as you update your diagram and gain experience:

■ Anytime you fi nd someone saying “sometimes” or “also” you should consider adding more detail to break out the various cases. For example, if you say, “Sometimes we connect to this web service via SSL, and some- times we fall back to HTTP,” you should draw both of those data fl ows (and consider whether an attacker can make you fall back like that).

■ Anytime you need more detail to explain security-relevant behavior, draw it in.

■ Each trust boundary box should have a label inside it.

■ Anywhere you disagreed over the design or construction of the system, draw in those details. This is an important step toward ensuring that everyone ended that discussion on the same page. It’s especially important for larger teams where not everyone is in the room for the threat model discussions. If anyone sees a diagram that contradicts their thinking, they can either accept it or challenge the assumptions; but either way, a good clear diagram can help get everyone on the same page.

■ Don’t have data sinks: You write the data for a reason. Show who uses it.

■ Data can’t move itself from one data store to another: Show the process that moves it.

■ All ways data can arrive should be shown.

■ If there are mechanisms for controlling data fl ows (such as fi rewalls or permissions) they should be shown.

■ All processes must have at least one entry data fl ow and one exit data fl ow.

■ As discussed earlier in the chapter, don’t draw an eye chart.

■ Diagrams should be visible on a printable page.

N O T E Writing Secure Code author David LeBlanc notes that “A process without input is a miracle, while one without output is a black hole. Either you’re missing

something, or have mistaken a process for people, who are allowed to be black holes

or miracles.”

When to Validate Diagrams

For software products, there are two main times to validate diagrams: when you create them and when you’re getting ready to ship a beta. There’s also a third triggering event (which is less frequent), which is if you add a security boundary.

www.it-ebooks.info

56 Part I ■ Getting Started

c02.indd 11:35:5:AM 01/17/2014 Page 56

For operational software diagrams, you also validate when you create them, and then again using a sensible balance between effort and up-to-dateness. That sensible balance will vary according to the maturity of a system, its scale, how tightly the components are coupled, the cadence of rollouts, and the nature of new rollouts. Here are a few guidelines:

■ Newer systems will experience more diagram changes than mature ones.

■ Larger systems will experience more diagram changes than smaller ones.

■ Tightly coupled systems will experience more diagram changes than loosely coupled systems.

■ Systems that roll out changes quickly will likely experience fewer diagram changes per rollout.

■ Rollouts or sprints focused on refactoring or paying down technical debt will likely see more diagram changes. In either case, create an appropriate tracking item to ensure that you recheck your diagrams at a good time. The appropriate tracking item is whatever you use to gate releases or rollouts, such as bugs, task management software, or checklists. If you have no formal way to gate releases, then you might focus on a clearly defi ned release process before worrying about rechecking threat models. Describing such a process is beyond the scope of this book.

Summary

There’s more than one way to threat model, and some of the strategies you can employ include modeling assets, modeling attackers, or modeling software. “What’s your threat model” and brainstorming are good for security experts, but they lack structure that less experienced threat modelers need. There are more structured approaches to brainstorming, including scenario analysis, pre-mortems, movie plotting, and literature reviews, which can help bring a little structure, but they’re still not great.

If your threat modeling starts from assets, the multiple overlapping defi ni- tions of the term, including things attackers want, things you’re protecting, and stepping stones, can trip you up. An asset-centered approach offers no route to fi gure out what will go wrong with the assets.

Attacker modeling is also attractive, but trying to predict how another person will attack is hard, and the approach can invite arguments that “no one would do that.” Additionally, human-centered approaches may lead you to human- centered threats that can be hard to address.

www.it-ebooks.info

Chapter 2 ■ Strategies for Threat Modeling 57

c02.indd 11:35:5:AM 01/17/2014 Page 57

Software models are focused on that what software people understand. The best models are diagrams that help participants understand the software and fi nd threats against it. There are a variety of ways you can diagram your soft- ware, and DFDs are the most frequently useful.

Once you have a model of the software, you’ll need a way to fi nd threats against it, and that is the subject of Part II.

www.it-ebooks.info

www.it-ebooks.info

c03.indd 07:51:54:AM 01/15/2014 Page 59

At the heart of threat modeling are the threats. There are many approaches to fi nding threats, and they are the subject of

Part II. Each has advantages and disadvantages, and different approaches may work in different circumstances. Each of the approaches in this part is like a Lego block. You can substitute one for another in the midst of this second step in the four-step framework and expect to get good results.

Knowing what aspects of security can go wrong is the unique element that makes threat modeling threat modeling, rather than some other form of mod- eling. The models in this part are abstractions of threats, designed to help you think about these security problems. The more specifi c models (such as attack libraries) will be more useful to those new to threat modeling, and are less freewheeling. As you become more experienced, the less structured approaches such as STRIDE become more useful.

In this part, you’ll learn about the following approaches to fi nding threats:

■ Chapter 3: STRIDE covers the STRIDE mnemonic you met in chapter 1, and its many variants.

■ Chapter 4: Attack Trees are either a way for you to think through threats against your system, or a way to help others structure their thinking about those threats. Both uses of attack trees are covered in this chapter.

Par t

II Finding Threats

www.it-ebooks.info

60 Part II ■ Finding Threats

c03.indd 07:51:54:AM 01/15/2014 Page 60

■ Chapter 5: Attack Libraries are libraries constructed to track and organize threats. They can be very useful to those new to security or threat modeling.

■ Chapter 6: Privacy Tools covers a collection of tools for fi nding privacy threats.

Part II focuses on the second question in the four-step framework: What can go wrong? As you’ll recall from Part I, before you start fi nding threats with any of the techniques in this part, you should fi rst have an idea of scope: where are you looking for threats? A diagram, such as a data fl ow diagram discussed in Part I, can help scope the threat modeling session, and thus is an excellent input condition. As you discuss threats, however, you’ll likely fi nd imperfec- tions in the diagram, so it isn’t necessary to “perfect” your diagram before you start fi nding threats.

www.it-ebooks.info

61

c03.indd 07:51:54:AM 01/15/2014 Page 61

As you learned in Chapter 1, “Dive in and Threat Model!,” STRIDE is an acro- nym that stands for Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, and Elevation of Privilege. The STRIDE approach to threat modeling was invented by Loren Kohnfelder and Praerit Garg (Kohnfelder, 1999). This framework and mnemonic was designed to help people developing software identify the types of attacks that software tends to experience.

The method or methods you use to think through threats have many differ- ent labels: fi nding threats, threat enumeration, threat analysis, threat elicitation, threat discovery. Each connotes a slightly different fl avor of approach. Do the threats exist in the software or the diagram? Then you’re fi nding them. Do they exist in the minds of the people doing the analysis? Then you’re doing analysis or elicitation. No single description stands out as always or clearly preferable, but this book generally talks about fi nding threats as a superset of all these ideas. Using STRIDE is more like an elicitation technique, with an expectation that you or your team understand the framework and know how to use it. If you’re not familiar with STRIDE, the extensive tables and examples are designed to teach you how to use it to discover threats.

This chapter explains what STRIDE is and why it’s useful, including sections covering each component of the STRIDE mnemonic. Each threat-specifi c sec- tion provides a deeper explanation of the threat, a detailed table of examples for that threat, and then a discussion of the examples. The tables and examples are designed to teach you how to use STRIDE to discover threats. You’ll also

C H A P T E R

3

STRIDE

www.it-ebooks.info

62 Part II ■ Finding Threats

c03.indd 07:51:54:AM 01/15/2014 Page 62

learn about approaches built on STRIDE: STRIDE-per-element, STRIDE-per- interaction, and DESIST. The other approach built on STRIDE, the Elevation of Privilege game, is covered in Chapters 1, “Dive In and Threat Model!” and 12, “Requirements Cookbook,” and Appendix C, “Attacker Lists.”

Understanding STRIDE and Why It’s Useful

The STRIDE threats are the opposite of some of the properties you would like your system to have: authenticity, integrity, non-repudiation, confidentiality, availability, and authorization. Table 3-1 shows the STRIDE threats, the cor- responding property that you’d like to maintain, a defi nition, the most typical victims, and examples.

Table 3-1: The STRIDE Threats

THREAT PROPERTY VIOLATED

THREAT DEFINITION

TYPICAL VICTIMS EXAMPLES

Spoofi ng Authentication Pretending to be something or some- one other than yourself

Processes, external entities, people

Falsely claiming to be Acme.com, winsock .dll, Barack Obama, a police offi cer, or the Nigerian Anti-Fraud Group

Tampering Integrity Modifying some- thing on disk, on a network, or in memory

Data stores, data fl ows, processes

Changing a spread- sheet, the binary of an important program, or the contents of a database on disk; modifying, adding, or removing packets over a network, either local or far across the Internet, wired or wireless; chang- ing either the data a program is using or the running program itself

www.it-ebooks.info

Chapter 3 ■ STRIDE 63

c03.indd 07:51:54:AM 01/15/2014 Page 63

THREAT PROPERTY VIOLATED

THREAT DEFINITION

TYPICAL VICTIMS EXAMPLES

Repudiation Non- Repudiation

Claiming that you didn’t do some- thing, or were not responsible. Repudiation can be honest or false, and the key question for system designers is, what evidence do you have?

Process Process or system: “I didn’t hit the big red button” or “I didn’t order that Ferrari.” Note that repudia- tion is somewhat the odd-threat-out here; it transcends the technical nature of the other threats to the business layer.

Information Disclosure

Confi dentiality Providing informa- tion to someone not authorized to see it

Processes, data stores, data fl ows

The most obvious example is allowing access to fi les, e-mail, or databases, but information disclosure can also involve fi le- names (“Termination for John Doe.docx”), packets on a network, or the contents of program memory.

Denial of Service

Availability Absorbing resources needed to provide service

Processes, data stores, data fl ows

A program that can be tricked into using up all its memory, a fi le that fi lls up the disk, or so many net- work connections that real traffi c can’t get through

Elevation of Privilege

Authorization Allowing someone to do something they’re not autho- rized to do

Process Allowing a normal user to execute code as admin; allowing a remote person with- out any privileges to run code

In Table 3-1, “typical victims” are those most likely to be victimized: For example, you can spoof a program by starting a program of the same name, or

www.it-ebooks.info

64 Part II ■ Finding Threats

c03.indd 07:51:54:AM 01/15/2014 Page 64

by putting a program with that name on disk. You can spoof an endpoint on the same machine by squatting or splicing. You can spoof users by capturing their authentication info by spoofing a site, by assuming they reuse credentials across sites, by brute forcing (online or off) or by elevating privilege on their machine. You can also tamper with the authentication database and then spoof with falsified credentials.

Note that as you’re using STRIDE to look for threats, you’re simply enumerat- ing the things that might go wrong. The exact mechanisms for how it can go wrong are something you can develop later. (In practice, this can be easy or it can be very challenging. There might be defenses in place, and if you say, for example, “Someone could modify the management tables,” someone else can say, “No, they can’t because...”) It can be useful to record those possible attacks, because even if there is a mitigation in place, that mitigation is a testable feature, and you should ensure that you have a test case.

You’ll sometimes hear STRIDE referred to as “STRIDE categories” or “the STRIDE taxonomy.” This framing is not helpful because STRIDE was not intended as, nor is it generally useful for, categorization. It is easy to find things that are hard to categorize with STRIDE. For example, earlier you learned about tampering with the authentication database and then spoofi ng. Should you record that as a tam- pering threat or a spoofi ng threat? The simple answer is that it doesn’t matter. If you’ve already come up with the attack, why bother putting it in a category? The goal of STRIDE is to help you find attacks. Categorizing them might help you figure out the right defenses, or it may be a waste of effort. Trying to use STRIDE to categorize threats can be frustrating, and those efforts cause some people to dismiss STRIDE, but this is a bit like throwing out the baby with the bathwater.

Spoofing Threats

Spoofing is pretending to be something or someone other than yourself. Table 3-1 includes the examples of claiming to be Acme.com, winsock.dll, Barack Obama, or the Nigerian Anti-Fraud Offi ce. Each of these is an example of a different subcategory of spoofing. The first example, pretending to be Acme. com (or Google.com, etc.) entails spoofing the identity of an entity across a net- work. There is no mediating authority that takes responsibility for telling you that Acme.com is the site I mean when I write these words. This differs from the second example, as Windows includes a winsock.dll. You should be able to ask the operating system to act as a mediating authority and get you to winsock. If you have your own DLLs, then you need to ensure that you’re open- ing them with the appropriate path (%installdir%\dll); otherwise, someone might substitute one in a working directory, and get your code to do what they want. (Similar issues exist with unix and LD_PATH.) The third example, spoofing Barack Obama, is an instance of pretending to be a specific person. Contrast that

www.it-ebooks.info

Chapter 3 ■ STRIDE 65

c03.indd 07:51:54:AM 01/15/2014 Page 65

with the fourth example, pretending to be the President of the United States or the Nigerian Anti-Fraud Offi ce. In those cases, the attacker is pretending to be in a role. These spoofi ng threats are laid out in Table 3-2.

Table 3-2: Spoofi ng Threats

THREAT EXAMPLES WHAT THE ATTACKER DOES NOTES

Spoofi ng a process on the same machine

Creates a fi le before the real process

Renaming/linking Creating a Trojan “su” and alter- ing the path

Renaming Naming your process “sshd”

Spoofi ng a fi le Creates a fi le in the local directory

This can be a library, execut- able, or confi g fi le.

Creates a link and changes it From the attacker’s perspec- tive, the change should hap- pen between the link being checked and the link being accessed.

Creates many fi les in the expected directory

Automation makes it easy to cre- ate 10,000 fi les in /tmp, to fi ll the space of fi les called /tmp /”pid.NNNN, or similar.

Spoofi ng a machine ARP spoofi ng

IP spoofi ng

DNS spoofi ng Forward or reverse

DNS Compromise Compromise TLD, registrar or DNS operator

IP redirection At the switch or router level

Spoofi ng a person Sets e-mail display name

Takes over a real account

Spoofi ng a role Declares themselves to be that role

Sometimes opening a special account with a relevant name

Spoofing a Process or File on the Same Machine

If an attacker creates a file before the real process, then if your code is not care- ful to create a new file, the attacker may supply data that your code interprets, thinking that your code (or a previous instantiation or thread) wrote that data,

www.it-ebooks.info

66 Part II ■ Finding Threats

c03.indd 07:51:54:AM 01/15/2014 Page 66

and it can be trusted. Similarly, if file permissions on a pipe, local procedure call, and so on, are not managed well, then an attacker can create that endpoint, confusing everything that attempts to use it later.

Spoofi ng a process or fi le on a remote machine can work either by creating spoofed fi les or processes on the expected machine (possibly having taken admin rights) or by pretending to be the expected machine, covered next.

Spoofing a Machine

Attackers can spoof remote machines at a variety of levels of the network stack. These spoofing attacks can influence your code’s view of the world as a client, server, or peer. They can spoof ARP requests if they’re local, they can spoof IP packets to make it appear that they’re coming from somewhere they are not, and they can spoof DNS packets. DNS spoofing can happen when you do a forward or reverse lookup. An attacker can spoof a DNS reply to a forward query they expect you to make. They can also adjust DNS records for machines they control such that when your code does a reverse lookup (translating IP to FQDN) their DNS server returns a name in a domain that they do not control—for example, claiming that 10.1.2.3 is update.microsoft.com. Of course, once attackers have spoofed a machine, they can either spoof or act as a man-in-the-middle for the processes on that machine. Second-order variants of this threat involve stealing machine authenticators such as cryptographic keys and abusing them as part of a spoofing attack.

Attackers can also spoof at higher layers. For example, phishing attacks involve many acts of spoofi ng. There’s usually spoofi ng of e-mail from “your” bank, and spoofi ng of that bank’s website. When someone falls for that e-mail, clicks the link and visits the bank, they then enter their credentials, sending them to that spoofed website. The attacker then engages in one last act of spoofi ng: They log into your bank account and transfer your money to themselves or an accomplice. (It may be one attacker, or it may be a set of attackers, contracting with one another for services rendered.)

Spoofing a Person

Major categories of spoofing people include access to the person’s account and pretending to be them through an alternate account. Phishing is a common way to get access to someone else’s account. However, there’s often little to prevent anyone from setting up an account and pretending to be you. For example, an attacker could set up accounts on sites like LinkedIn, Twitter, or Facebook and pretend to be you, the Adam Shostack who wrote this book, or a rich and deposed prince trying to get their money out of the country.

www.it-ebooks.info

Chapter 3 ■ STRIDE 67

c03.indd 07:51:54:AM 01/15/2014 Page 67

Tampering Threats

Tampering is modifying something, typically on disk, on a network, or in memory. This can include changing data in a spreadsheet (using either a program such as Excel or another editor), changing a binary or configuration file on disk, or modifying a more complex data structure, such as a database on disk. On a network, packets can be added, modified, or removed. It’s sometimes easier to add packets than to edit them as they fly by, and programs are remarkably bad about handling extra copies of data securely. More examples of tampering are in Table 3-3.

Table 3-3: Tampering Threats

THREAT EXAMPLES WHAT THE ATTACKER DOES NOTES

Tampering with a fi le Modifi es a fi le they own and on which you rely

Modifi es a fi le you own

Modifi es a fi le on a fi le server that you own

Modifi es a fi le on their fi le server Loads of fun when you include fi les from remote domains

Modifi es a fi le on their fi le server Ever notice how much XML includes remote schemas?

Modifi es links or redirects

Tampering with memory Modifi es your code Hard to defend against once the attacker is run- ning code as the same user

Modifi es data they’ve supplied to your API

Pass by value, not by refer- ence when crossing a trust boundary

Tampering with a network Redirects the fl ow of data to their machine

Often stage 1 of tampering

Modifi es data fl owing over the network

Even easier and more fun when the network is wire- less (WiFi, 3G, et cetera)

Enhances spoofi ng attacks

www.it-ebooks.info

68 Part II ■ Finding Threats

c03.indd 07:51:54:AM 01/15/2014 Page 68

Tampering with a File

Attackers can modify fi les wherever they have write permission. When your code has to rely on fi les others can write, there’s a possibility that the fi le was written maliciously. While the most obvious form of tampering is on a local disk, there are also plenty of ways to do this when the file is remotely included, like most of the JavaScript on the Internet. The attacker can breach your security by breaching someone else’s site. They can also (because of poor privileges, spoofing, or elevation of privilege) modify files you own. Lastly, they can modify links or redirects of various sorts. Links are often left out of integrity checks. There’s a somewhat subtle variant of this when there are caches between things you control (such as a server) and things you don’t (such as a web browser on the other side of the Internet). For example, cache poisoning attacks insert data into web caches through poor security controls at caches (OWASP, 2009).

Tampering with Memory

Attackers can modify your code if they’re running at the same privilege level. At that point, defense is tricky. If your API handles data by reference (a pat- tern often chosen for speed), then an attacker can modify it after you perform security checks.

Tampering with a Network

Network tampering often involves a variety of tricks to bring the data to the attacker’s machine, where he forwards some data intact and some data modi- fi ed. However, tricks to bring you the data are not always needed; with radio interfaces like WiFi and Bluetooth, more and more data flow through the air. Many network protocols were designed with the assumption you needed spe- cial hardware to create or read arbitrary packets. The requirement for special hardware was the defense against tampering (and often spoofi ng). The rise of software-defined radio (SDR) has silently invalidated the need for special hard- ware. It is now easy to buy an inexpensive SDR unit that can be programmed to tamper with wireless protocols.

Repudiation Threats

Repudiation is claiming you didn’t do something, or were not responsible for what happened. People can repudiate honestly or deceptively. Given the increas- ing knowledge often needed to understand the complex world, those honestly repudiating may really be exposing issues in your user experiences or service architectures. Repudiation threats are a bit different from other security threats,

www.it-ebooks.info

Chapter 3 ■ STRIDE 69

c03.indd 07:51:54:AM 01/15/2014 Page 69

as they often appear at the business layer. (That is, above the network layer such as TCP/IP, above the application layer such as HTTP/HTML, and where the business logic of buying products would be implemented.)

Repudiation threats are also associated with your logging system and process. If you don’t have logs, don’t retain logs, or can’t analyze logs, repudiation threats are hard to dispute. There is also a class of attacks in which attackers will drop data in the logs to make log analysis tricky. For example, if you display your logs in HTML and the attacker sends </tr> or </html>, your log display needs to treat those as data, not code. More repudiation threats are shown in Table 3-4.

Table 3-4: Repudiation Threats

THREAT EXAMPLES WHAT THE ATTACKER DOES NOTES

Repudiating an action Claims to have not clicked Maybe they really did

Claims to have not received Receipt can be strange; does mail being downloaded by your phone mean you’ve read it? Did a network proxy pre-fetch images? Did some- one leave a package on the porch?

Claims to have been a fraud victim

Uses someone else’s account

Uses someone else’s pay- ment instrument without authorization

Attacking the logs Notices you have no logs

Puts attacks in the logs to con- fuse logs, log-reading code, or a person reading the logs

Attacking the Logs

Again, if you don’t have logs, don’t retain logs, or can’t analyze logs, repudiation actions are hard to dispute. So if you aren’t logging, you probably need to start. If you have no log centralization or analysis capability, you probably need that as well. If you don’t properly define what you will be logging, an attacker may be able to break your log analysis system. It can be challenging to work through the layers of log production and analysis to ensure reliability, but if you don’t, it’s easy to have attacks slip through the cracks or inconsistencies.

www.it-ebooks.info

70 Part II ■ Finding Threats

c03.indd 07:51:54:AM 01/15/2014 Page 70

Repudiating an Action

When you’re discussing repudiation, it’s helpful to discuss “someone” rather than “an attacker.” You want to do this because those who repudiate are often not actually attackers, but people who have been failed by technology or pro- cess. Maybe they really didn’t click (or didn’t perceive that they clicked). Maybe the spam filter really did eat that message. Maybe UPS didn’t deliver, or maybe UPS delivered by leaving the package on a porch. Maybe someone claims to have been a victim of fraud when they really were not (or maybe someone else in a household used their credit card, with or without their knowledge). Good technological systems that both authenticate and log well can make it easier to handle repudiation issues.

Information Disclosure Threats

Information disclosure is about allowing people to see information they are not authorized to see. Some information disclosure threats are shown in Table 3-5.

Table 3-5: Information Disclosure Threats

THREAT EXAMPLES WHAT THE ATTACKER DOES NOTES

Information dis- closure against a process

Extracts secrets from error messages

Reads the error messages from username/passwords to entire database tables

Extracts machine secrets from error cases Can make defense against memory corruption such as ASLR far less useful

Extracts business/personal secrets from error cases

Information dis- closure against data stores

Takes advantage of inappropriate or missing ACLs

Takes advantage of bad database permissions

Finds fi les protected by obscurity

Finds crypto keys on disk (or in memory)

Sees interesting information in fi lenames

Reads fi les as they traverse the network

Gets data from logs or temp fi les

Gets data from swap or other temp storage

Extracts data by obtaining device, changing OS

www.it-ebooks.info

Chapter 3 ■ STRIDE 71

c03.indd 07:51:54:AM 01/15/2014 Page 71

THREAT EXAMPLES WHAT THE ATTACKER DOES NOTES

Information dis- closure against a data fl ow

Reads data on the network

Redirects traffi c to enable reading data on the network

Learns secrets by analyzing traffi c

Learns who’s talking to whom by watching the DNS

Learns who’s talking to whom by social network info disclosure

Information Disclosure from a Process

Many instances in which a process will disclose information are those that inform further attacks. A process can do this by leaking memory addresses, extracting secrets from error messages, or extracting design details from error messages. Leaking memory addresses can help bypass ASLR and similar defenses. Leaking secrets might include database connection strings or passwords. Leaking design details might mean exposing anti-fraud rules like “your account is too new to order a diamond ring.”

Information Disclosure from a Data Store

As data stores, well, store data, there’s a profusion of ways they can leak it. The first set of causes are failures to properly use security mechanisms. Not setting permis- sions appropriately or hoping that no one will find an obscure file are common ways in which people fail to use security mechanisms. Cryptographic keys are a special case whereby information disclosure allows additional attacks. Files read from a data store over the network are often readable as they traverse the network.

An additional attack, often overlooked, is data in filenames. If you have a directory named “May 2013 layoffs,” the fi lename itself, “Termination Letter for Alice.docx,” reveals important information.

There’s also a group of attacks whereby a program emits information into the operating environment. Logs, temp files, swap, or other places can contain data. Usually, the OS will protect data in swap, but for things like crypto keys, you should use OS facilities for preventing those from being swapped out.

Lastly, there is the class of attacks whereby data is extracted from the device using an operating system under the attacker’s control. Most commonly (in 2013), these attacks affect USB keys, but they also apply to CDs, backup tapes, hard drives, or stolen laptops or servers. Hard drives are often decommissioned without full data deletion. (You can address the need to delete data from hard

www.it-ebooks.info

72 Part II ■ Finding Threats

c03.indd 07:51:54:AM 01/15/2014 Page 72

drives by buying a hard drive chipper or smashing machine, and since such machines are awesome, why on earth wouldn’t you?)

Information Disclosure from a Data Flow

Data flows are particularly susceptible to information disclosure attacks when information is flowing over a network. However, data flows on a single machine can still be attacked, particularly when the machine is shared by cloud co-tenants or many mutually distrustful users of a compute server. Beyond the simple reading of data on the network, attackers might redirect traffi c to themselves (often by spoofing some network control protocol) so they can see it when they’re not on the normal path. It’s also possible to obtain information even when the network traffi c itself is encrypted. There are a variety of ways to learn secrets about who’s talking to whom, including watching DNS, friend activity on a site such as LinkedIn, or other forms of social network analysis.

N O T E Security mavens may be wondering if side channel attacks and covert channels are going to be mentioned. These attacks can be fun to work on (and side channels are

covered a bit in Chapter 16, “Threats to Cryptosystems”), but they are not relevant until

you’ve mitigated the issues covered here.

Denial-of-Service Threats

Denial-of-service attacks absorb a resource that is needed to provide service. Examples are described in Table 3-6.

Table 3-6: Denial-of-Service Threats

THREAT EXAMPLES WHAT THE ATTACKER DOES NOTES

Denial of service against a process

Absorbs memory (RAM or disk)

Absorbs CPU

Uses process as an amplifi er

Denial of service against a data store

Fills data store up

Makes enough requests to slow down the system

Denial of service against a data fl ow

Consumes network resources

www.it-ebooks.info

Chapter 3 ■ STRIDE 73

c03.indd 07:51:54:AM 01/15/2014 Page 73

Denial-of-service attacks can be split into those that work while the attacker is attacking (say, filling up bandwidth) and those that persist. Persistent attacks can remain in effect until a reboot (for example, while(1){fork();}), or even past a reboot (for example, filling up a disk). Denial-of-service attacks can also be divided into amplified and unamplified. Amplified attacks are those whereby small attacker effort results in a large impact. An example would take advantage of the old unix chargen service, whose purpose was to generate a semi-random character scheme for testing. An attacker could spoof a single packet from the chargen port on machine A to the chargen port on machine B. The hilarity continues until someone pulls a network cable.

Elevation of Privilege Threats

Elevation of privilege is allowing someone to do something they’re not autho- rized to do—for example, allowing a normal user to execute code as admin, or allowing a remote person without any privileges to run code. Two important ways to elevate privileges involve corrupting a process and getting past autho- rization checks. Examples are shown in Table 3-7.

Table 3-7: Elevation of Privilege Threats

THREAT EXAMPLES WHAT THE ATTACKER DOES NOTES

Elevation of privilege against a process by corrupting the process

Send inputs that the code doesn’t handle properly

These errors are very com- mon, and are usually high impact.

Gains access to read or write memory inappropriately

Writing memory is (hope- fully obviously) bad, but reading memory can enable further attacks.

Elevation through missed authorization checks

Elevation through buggy authorization checks

Centralizing such checks makes bugs easier to manage

Elevation through data tampering

Modifi es bits on disk to do things other than what the authorized user intends

www.it-ebooks.info

74 Part II ■ Finding Threats

c03.indd 07:51:54:AM 01/15/2014 Page 74

Elevate Privileges by Corrupting a Process

Corrupting a process involves things like smashing the stack, exploiting data on the heap, and a whole variety of exploitation techniques. The impact of these techniques is that the attacker gains influence or control over a program’s control flow. It’s important to understand that these exploits are not limited to the attack surface. The first code that attacker data can reach is, of course, an important target. Generally, that code can only validate data against a limited subset of purposes. It’s important to trace the data flows further to see where else elevation of privilege can take place. There’s a somewhat unusual case whereby a program relies on and executes things from shared memory, which is a trivial path for elevation if everything with permissions to that shared memory is not running at the same privilege level.

Elevate Privileges through Authorization Failures

There is also a set of ways to elevate privileges through authorization failures. The simplest failure is to not check authorization on every path. More complex for an attacker is taking advantage of buggy authorization checks. Lastly, if a program relies on other programs, configuration files, or datasets being trust- worthy, it’s important to ensure that permissions are set so that each of those dependencies is properly secured.

Extended Example: STRIDE Threats against Acme-DB

This extended example discusses how STRIDE threats could manifest against the Acme/SQL database described in Chapter 1, “Dive In and Threat Model!” and 2, “Strategies for Threat Modeling,” and shown in Figure 2-1. You’ll fi rst look at these threats by STRIDE category, and then examine the same set according to who can address them.

Spoofi ng

■ A web client could attempt to log in with random credentials or stolen credentials, as could a SQL client.

■ If you assume that the SQL client is the one you wrote and allow it to make security decisions, then a spoofed (or tampered with) client could bypass security checks.

■ The web client could connect to a false (spoofed) front end, and end up disclosing credentials.

■ A program could pretend to be the database or log analysis program, and try to read data from the various data stores.

www.it-ebooks.info

Chapter 3 ■ STRIDE 75

c03.indd 07:51:54:AM 01/15/2014 Page 75

Tampering

■ Someone could also tamper with the data they’re sending, or with any of the programs or data files.

■ Someone could tamper with the web or SQL clients. (This is nominally out of scope, as you shouldn’t be trusting external entities anyway.)

N O T E These threats, once you consider them, can easily be addressed with operating system permissions. More challenging is what can alter what data within the database.

Operating system permissions will only help a little there; the database will need to

implement an access control system of some sort.

Repudiation

■ The customers using either SQL or web clients could claim not to have done things. These threats may already be mitigated by the presence of logs and log analysis. So why bother with these threats? They remind you that you need to configure logging to be on, and that you need to log the “right things,” which probably include successes and failures of authen- tication attempts, access attempts, and in particular, the server needs to track attempts by clients to access or change logs.

Information Disclosure

■ The most obvious information disclosure issues occur when confidential information in the database is exposed to the wrong client. This informa- tion may be either data (the contents of the salaries table) or metadata (the existence of the termination plans table). The information disclosure may be accidental (failure to set an ACL) or malicious (eavesdropping on the network). Information disclosure may also occur by the front end(s)— for example, an error message like “Can’t connect to database foo with password bar!”

■ The database files (partitions, SAN attached storage) need to be protected by the operating system and by ACLs for data within the files.

■ Logs often store confidential information, and therefore need to be protected.

Denial of Service

■ The front ends could be overwhelmed by random or crafted requests, especially if there are anonymous (or free) web accounts that can craft requests designed to be slow to execute.

■ The network connections could be overwhelmed with data.

■ The database or logs could be filled up.

www.it-ebooks.info

76 Part II ■ Finding Threats

c03.indd 07:51:54:AM 01/15/2014 Page 76

■ If the network between the main processes, or the processes and databases, is shared, it may become congested.

Elevation of Privilege

■ Clients, either web or SQL, could attempt to run queries they’re not autho- rized to run.

■ If the client is enforcing security, then anyone who tampers with their client or its network stream will be able to run queries of their choice.

■ If the database is capable of running arbitrary commands, then that capa- bility is available to the clients.

■ The log analysis program (or something pretending to be the log analysis program) may be able to run arbitrary commands or queries.

N O T E The log analysis program may be thought of as trusted, but it’s drawn outside the trust boundaries. So either the thinking or the diagram (in Figure 2-1) is incorrect.

■ If the DB cluster is connected to a corporate directory service and no action is taken to restrict who can log in to the database servers (or file servers), then anyone in the corporate directory, including perhaps employees, contractors, build labs, and partners can make changes on those systems.

N O T E The preceding lists in this extended example are intended to be illustrative; other threats may exist.

It is also possible to consider these threats according to the person or team that must address them, divided between Acme and its customers. As shown in Table 3-8, this illustrates the natural overlap of threat and mitigation, fore- shadowing the Part III, “Managing and Addressing Threats” on how to mitigate threats. It also starts to enumerate things that are not requirements for Acme/ SQL. These non-requirements should be documented and provided to customers, as covered in Chapter 12. In this table, you’re seeing more and more actionable threats. As a developer or a systems administrator, you can start to see how to handle these sorts of issues. It’s tempting to start to address threats in the table itself, and a natural extension to the table would be a set of ways for each actor to address the threats that apply.

www.it-ebooks.info

Chapter 3 ■ STRIDE 77

c03.indd 07:51:54:AM 01/15/2014 Page 77

Table 3-8: Addressing Threats According to Who Handles Them

THREAT INSTANCES THAT ACME MUST HANDLE

INSTANCES THAT IT DEPARTMENTS MUST HANDLE

Spoofi ng Web/SQL/other client brute forcing logins

DBA (human)

DB users

Web client

SQL client

DBA (human)

DB users

Tampering Data

Management

Logs

Front end(s)

Database

DB admin

Repudiation Logs (Log analysis must be protected.)

Certain actions from web and SQL clients will need careful logging.

Certain actions from DBAs will need careful logging.

Logs (Log analysis must be protected.)

If DBAs are not fully trusted, a system in another privilege domain to log all commands might be required.

Information disclosure

Data, management, and logs must be protected.

Front ends must implement access control.

Only the front ends should be able to access the data.

ACLs and security groups must be managed.

Backups must be protected.

Denial of service

Front ends must be designed to minimize DoS risks. The system must be deployed with suf- fi cient resources.

Continues

www.it-ebooks.info

78 Part II ■ Finding Threats

c03.indd 07:51:54:AM 01/15/2014 Page 78

THREAT INSTANCES THAT ACME MUST HANDLE

INSTANCES THAT IT DEPARTMENTS MUST HANDLE

Elevation of privilege

Trusting client

The DB should support prepared statements to make injection harder.

No “run this command” tools should be in the default install.

No default way to run commands on the server, and calls like exec()and system() must be permis- sioned and confi gurable if they exist.

Inappropriately trusting clients that are written locally

Confi gure the DB appropriately.

STRIDE Variants

STRIDE can be a very useful mnemonic when looking for threats, but it’s not perfect. In this section, you’ll learn about variants of STRIDE that may help address some of its weaknesses.

STRIDE-per-Element

STRIDE-per-element makes STRIDE more prescriptive by observing that certain threats are more prevalent with certain elements of a diagram. For example, a data store is unlikely to spoof another data store (although running code can be confused as to which data store it’s accessing.) By focusing on a set of threats against each element, this approach makes it easier to find threats. For example, Microsoft uses Table 3-9 as a core part of its Security Development Lifecycle threat modeling training.

Table 3-9: STRIDE-per-Element

S T R I D E

External Entity x x

Process x x x x x x

Data Flow x x x

Data Store x ? x x

Table 3-8 (continued)

www.it-ebooks.info

Chapter 3 ■ STRIDE 79

c03.indd 07:51:54:AM 01/15/2014 Page 79

Applying this chart, you can focus threat analysis on how an attacker might tamper with, read data from, or prevent access to a data flow. For example, if data is flowing over a network such as Ethernet, it’s trivial for someone attached to that same Ethernet to read all the content, modify it, or send a flood of packets to cause a TCP timeout. You might argue that you have some form of network segmentation, and that may mitigate the threats suffi ciently for you. The ques- tion mark under repudiation indicates that logging data stores are involved in addressing repudiation, and sometimes logs will come under special attack to allow repudiation attacks.

The threat is to the element listed in Table 3-9. Each element is the victim, not the perpetrator. Therefore, if you’re tampering with a data store, the threat is to the data store and the data within. If you’re spoofing in a way that affects a process, then the process is the victim. So, spoofing by tampering with the network is really a spoof of the endpoint, regardless of the technical details. In other words, the other endpoint (or endpoints) are confused about what’s at the other end of the connection. The chart focuses on spoofing of a process, not spoofing of the data flow. Of course, if you happen to find spoofing when looking at the data flow, obviously you should record the threat so you can address it, not worry about what sort of threat it is. STRIDE-per-element has the advantage of being prescriptive, helping you identify what to look for where without being a checklist of the form “web component: XSS, XSRF...” In skilled hands, it can be used to find new types of weaknesses in components. In less skilled hands, it can still find many common issues.

STRIDE-per-element does have two weaknesses. First, similar issues tend to crop up repeatedly in a given threat model; second, the chart may not represent your issues. In fact, Table 3-9 is somewhat specifi c to Microsoft. The easiest place to see this is “information disclosure by external entity,” which is a good description of some privacy issues. (It is by no means a complete description of privacy.) However, the table doesn’t indicate that this could be a problem. That’s because Microsoft has a separate set of processes for analyzing privacy problems. Those privacy processes are outside the security threat modeling space. Therefore, if you’re going to adopt this approach, it’s worth analyzing whether the table covers the set of issues you care about, and if it doesn’t, create a version that suits your scenario. Another place you might see the specifi city is that many people want to discuss spoofi ng of data fl ows. Should that be part of STRIDE-per-element? The spoofi ng action is a spoofi ng of the endpoint, but that description may help some people to look for those threats. Also note that the more “x” marks you add, the closer you come to “consider STRIDE for each element of the diagram.” The editors ask if that’s a good or bad thing, and it’s a fi ne question. If you want to be comprehensive, this is helpful; if you want to focus on the most likely issues, however, it will likely be a distraction.

So what are the exit criteria for STRIDE-per-element? When you have a threat per checkbox in the STRIDE-per-element table, you are doing reasonably well.

www.it-ebooks.info

80 Part II ■ Finding Threats

c03.indd 07:51:54:AM 01/15/2014 Page 80

If you circle around and consider threats against your mitigations (or ways to bypass them) you’ll be doing pretty well.

STRIDE-per-Interaction

STRIDE-per-element is a simplified approach to identifying threats, designed to be easily understood by the beginner. However, in reality, threats don’t show up in a vacuum. They show up in the interactions of the system. STRIDE-per- interaction is an approach to threat enumeration that considers tuples of (origin, destination, interaction) and enumerates threats against them. Initially, another goal of this approach was to reduce the number of things that a modeler would have to consider, but that didn’t work out as planned. STRIDE-per-interaction leads to the same number of threats as STRIDE-per-element, but the threats may be easier to understand with this approach. This approach was developed by Larry Osterman and Douglas MacIver, both of Microsoft. The STRIDE- per-interaction approach is shown in Tables 3-10 and 3-11. Both reference two processes, Contoso.exe and Fabrikam.dll. Table 3-10 shows which threats apply to each interaction, and Table 3-11 shows an example of STRIDE per interaction applied to Figure 3-1. The relationships and trust boundaries used for the named elements in both tables are shown in Figure 3-1.

Browser database

widgetsCreate(widgets)

Write

Results

Commands

Responses

Fabrikam.dll

Contoso.exe

Figure 3-1: The system referenced in Table 3-10

In Table 3-10, the table columns are as follows:

■ A number for referencing a line (For example, “Looking at line 2, let’s look for spoofi ng and information disclosure threats.”)

www.it-ebooks.info

Chapter 3 ■ STRIDE 81

c03.indd 07:51:54:AM 01/15/2014 Page 81

■ The main element you’re looking at

■ The interactions that element has

■ The STRIDE threats applicable to the interaction

Table 3-10: STRIDE-per-Interaction: Threat Applicability

# ELEMENT INTERACTION S T R I D E

1 Process (Contoso)

Process has outbound data fl ow to data store.

x x

2 Process sends output to another process.

x x x x x

3 Process sends output to external interactor (code).

x x x x

4 Process sends output to external interactor (human).

x

5 Process has inbound data fl ow from data store.

x x x x

6 Process has inbound data fl ow from a process.

x x x x

7 Process has inbound data fl ow from external interactor.

x x x

8 Data Flow (com- mands/ responses)

Crosses machine boundary x x x

9 Data Store (database)

Process has outbound data fl ow to data store.

x x x x

10 Process has inbound data fl ow from data store.

x x x

11 External Interactor (browser)

External interactor passes input to process.

x x x

12 External interactor gets input from process.

x

www.it-ebooks.info

c03.indd 07:51:54:AM 01/15/2014 Page 82

Ta b

le 3

-1 1:

S TR

ID E-

p er

-I nt

er ac

ti on

(E xa

m p

le )

E LE

M E

N T

IN TE

R A

C TI

O N

S T

R I

D E

1 Pr

oc es

s (C

on to

so )

Pr oc

es s

ha s

ou t-

bo un

d da

ta fl

ow

to d

at a

st or

e.

“D at

ab as

e” is

s po

of ed

, an

d Co

nt os

o w

rit es

to th

e w

ro ng

p la

ce .

P2 : C

on to

so

w rit

es in

fo rm

a- tio

n in

“d at

a- ba

se w

hi ch

sh

ou ld

n ot

b e

in d

at ab

as e”

(e

.g .,

pa ss

w or

ds ).

2 Pr

oc es

s se

nd s

ou tp

ut to

o th

er

pr oc

es s.

Fa br

ik am

is s

po of

ed , a

nd

Co nt

os o

w rit

es to

th e

w ro

ng p

la ce

.

Fa br

ik am

cl

ai m

s no

t t o

ha ve

b ee

n ca

lle d

by

Co nt

os o.

P2 : F

ab rik

am is

no

t a ut

ho riz

ed

to re

ce iv

e da

ta .

N on

e un

le ss

ca

lls a

re

sy nc

hr on

ou s

Fa br

ik am

ca

n im

pe r-

so na

te

Co nt

os o

an d

us e

its

pr iv

ile ge

s.

3 Pr

oc es

s se

nd s

ou tp

ut to

e xt

er -

na l i

nt er

ac to

r (w

ith th

e in

te ra

c- to

r b ei

ng c

od e)

.

Co nt

os o

is c

on fu

se d

ab ou

t t he

id en

tit y

of th

e br

ow se

r.

Br ow

se r

di sc

la im

s an

d do

es n’

t ac

kn ow

le dg

e th

e ou

tp ut

.

P2 : B

ro w

se r

ge ts

d at

a it’

s no

t a ut

ho riz

ed

to g

et .

N on

e un

le ss

ca

lls a

re

sy nc

hr on

ou s

4 Pr

oc es

s se

nd s

ou tp

ut to

e xt

er -

na l i

nt er

ac to

r (fo

r a h

um an

in

te ra

ct or

).

H um

an d

is- cl

ai m

s se

ei ng

th

e ou

tp ut

.

www.it-ebooks.info

c03.indd 07:51:54:AM 01/15/2014 Page 83

E LE

M E

N T

IN TE

R A

C TI

O N

S T

R I

D E

5 Pr

oc es

s ha

s in

bo un

d da

ta

fl o w

fr om

d at

a st

or e.

“D at

ab as

e” is

s po

of ed

, a nd

Co

nt os

o re

ad s

th e

w ro

ng

da ta

.

Co nt

os o

st at

e is

c or

- ru

pt ed

b y

da ta

re ad

fro

m th

e da

ta s

to re

.

Pr oc

es s

st at

e is

c or

ru pt

ed

by th

e da

ta

re tr

ie ve

d fro

m th

e da

ta s

to re

.

Pr oc

es s

in te

rn al

st

at e

is

co rr

up te

d

ba se

d on

da

ta re

ad

fro m

th e

fi l e,

le

ad in

g to

co

de e

xe cu

- tio

n .

6 Pr

oc es

s ha

s in

bo un

d da

ta

fl o w

fr om

a

pr oc

es s.

Co nt

os o

be lie

ve s

it’ s

ge t-

tin g

da ta

fr om

F ab

rik am

. Co

nt os

o de

ni es

g et

- tin

g da

ta fr

om

Fa br

ik am

.

Co nt

os o

cr as

he s/

st op

s du

e to

F ab

rik am

in

te ra

ct io

n.

Fa br

ik am

pa

ss es

d at

a or

a rg

s th

at

al lo

w it

to

ch an

ge fl

ow

of e

xe cu

tio n

of C

on to

so .

7 Pr

oc es

s ha

s in

bo un

d da

ta

fl o w

fr om

e xt

er -

na l i

nt er

ac to

r.

Co nt

os o

be lie

ve s

it’ s

ge tt

in g

da ta

fr om

th e

br ow

se r,

w he

n in

fa ct

it ’s

a ra

nd om

a tt

ac ke

r.

Co nt

os o

cr as

he s/

st op

s du

e to

b ro

w se

r in

te ra

ct io

n.

Br ow

se r

pa ss

es d

at a

or a

rg s

th at

al

lo w

it to

ch

an ge

fl ow

of

e xe

cu tio

n of

C on

to so

.

Co nt in ue s

Ta b

le 3

-1 1

(c o

n ti

n u

ed )

www.it-ebooks.info

c03.indd 07:51:54:AM 01/15/2014 Page 84

E LE

M E

N T

IN TE

R A

C TI

O N

S T

R I

D E

8 D

at a

Fl ow

(c

om m

an ds

/ re

sp on

se s)

C ro

ss es

m ac

hi ne

bo

un da

ry D

at a

fl o w

is

m od

ifi ed

by

M IT

M

at ta

ck .

Th e

co nt

en ts

o f

th e

da ta

fl ow

ar

e sn

iff ed

o n

th e

w ire

.

Th e

da ta

fl o

w is

in te

r- ru

pt ed

b y

an e

xt er

na l

en tit

y (e

.g .,

m es

si ng

w

ith T

CP

se qu

en ce

nu

m be

rs .)

9 D

at a

St or

e (d

at ab

as e)

Pr oc

es s

ha s

ou t-

bo un

d da

ta fl

ow

to d

at a

st or

e.

D at

ab as

e is

co

rr up

te d.

Co nt

os o

cl ai

m s

no t t

o ha

ve w

rit te

n to

d at

ab as

e.

D at

ab as

e re

ve al

s in

fo rm

at io

n.

D at

ab as

e ca

nn ot

b e

w rit

te n

to .

10 Pr

oc es

s ha

s in

bo un

d da

ta

fl o w

fr om

d at

a st

or e.

Co nt

os o

cl ai

m s

no t t

o ha

ve re

ad fr

om

da ta

ba se

.

D at

ab as

e di

sc lo

se s

in fo

rm at

io n.

D at

ab as

e ca

nn ot

b e

re ad

fr om

.

11 Ex

te rn

al

In te

ra ct

or

(b ro

w se

r)

Ex te

rn al

in te

ra c-

to r p

as se

s in

pu t

to p

ro ce

ss .

Co nt

os o

is c

on fu

se d

ab ou

t t he

id en

tit y

of th

e br

ow se

r.

Co nt

os o

cl ai

m s

no t t

o ha

ve re

ce iv

ed

th e

da ta

.

P2 : p

ro ce

ss n

ot

au th

or iz

ed to

re

ce iv

e th

e da

ta

(W e

ca n’

t s to

p it.

)

12 Ex

te rn

al in

te ra

c- to

r g et

s in

pu t

fro m

p ro

ce ss

.

Br ow

se r i

s co

nf us

ed a

bo ut

th

e id

en tit

y of

C on

to so

. Co

nt os

o cl

ai m

s no

t t o

ha ve

s en

t t he

da

ta

(N ot

o ur

pr

ob le

m .)

Ta b

le 3

-1 1

(c o n ti n u ed

)

www.it-ebooks.info

Chapter 3 ■ STRIDE 85

c03.indd 07:51:54:AM 01/15/2014 Page 85

When you have a threat per checkbox in the STRIDE-per-interaction table, you are doing reasonably well. If you circle through and consider threats against your mitigations (or ways to bypass them) you’ll be doing pretty well.

STRIDE-per-interaction is too complex to use without a reference chart handy. (In contrast, STRIDE is an easy mnemonic, and STRIDE-per-element is simple enough that the chart can be memorized or printed on a wallet card.)

DESIST

DESIST is a variant of STRIDE created by Gunnar Peterson. DESIST stands for Dispute, Elevation of privilege, Spoofing, Information disclosure, Service denial, and Tampering. (Dispute replaces repudiation with a less fancy word, and Service denial replaces Denial of Service to make the acronym work.) Starting from scratch, it might make sense to use DESIST over STRIDE, but after more than a decade of STRIDE, it would be expensive to displace at Microsoft. (CEO of Scorpion Software, Dana Epp, has pointed out that acronyms with repeated letters can be challenging, a point in STRIDE’s favor.) Therefore, STRIDE-per- element, rather than DESIST-per-element, exists as the norm. Either way, it’s always useful to have mnemonics for helping people look for threats.

Exit Criteria

There are three ways to judge whether you’re done fi nding threats with STRIDE. The easiest way is to see if you have a threat of each type in STRIDE. Slightly harder is ensuring you have one threat per element of the diagram. However, both of these criterion will be reached before you’ve found all threats. For more comprehensiveness, use STRIDE-per-element, and ensure you have one threat per check.

Not having met these criteria will tell you that you’re not done, but having met them is not a guarantee of completeness.

Summary

STRIDE is a useful mnemonic for fi nding threats against all sorts of techno- logical systems. STRIDE is more useful with a repertoire of more detailed threats to draw on. The tables of threats can provide that for those who are new to security, or act as reference material for security experts (a function also served by Appendix B, “Threat Trees”). There are variants of STRIDE that attempt to add focus and attention. STRIDE-per-element is very useful for

www.it-ebooks.info

86 Part II ■ Finding Threats

c03.indd 07:51:54:AM 01/15/2014 Page 86

this purpose, and can be customized to your needs. STRIDE-per-interaction provides more focus, but requires a crib sheet (or perhaps software) to use. If threat modeling experts were to start over, perhaps DESIST would help us make better ... progress in fi nding threats.

www.it-ebooks.info

87

c04.indd 11:38:53:AM 01/17/2014 Page 87

As Bruce Schneier wrote in his introduction to the subject, “Attack trees provide a formal, methodical way of describing the security of systems, based on vary- ing attacks. Basically, you represent attacks against a system in a tree structure, with the goal as the root node and different ways of achieving that goal as leaf nodes” (Schneier, 1999).

In this chapter you’ll learn about the attack tree building block as an alterna- tive to STRIDE. You can use attack trees as a way to fi nd threats, as a way to organize threats found with other building blocks, or both. You’ll start with how to use an attack tree that’s provided to you, and from there learn various ways you can create trees. You’ll also examine several example and real attack trees and see how they fi t into fi nding threats. The chapter closes with some additional perspective on attack trees.

Working with Attack Trees

Attack trees work well as a building block for threat enumeration in the four- step framework. They have been presented as a full approach to threat modeling (Salter, 1998), but the threat modeling community has learned a lot since then.

There are three ways you can use attack trees to enumerate threats: You can use an attack tree someone else created to help you fi nd threats. You can create

C H A P T E R

4

Attack Trees

www.it-ebooks.info

88 Part II ■ Finding Threats

c04.indd 11:38:53:AM 01/17/2014 Page 88

a tree to help you think through threats for a project you’re working on. Or you can create trees with the intent that others will use them. Creating new trees for general use is challenging, even for security experts.

Using Attack Trees to Find Threats

If you have an attack tree that is relevant to the system you’re building, you can use it to fi nd threats. Once you’ve modeled your system with a DFD or other diagram, you use an attack tree to analyze it. The attack elicitation task is to iterate over each node in the tree and consider if that issue (or a variant of that issue) impacts your system. You might choose to track either the threats that apply or each interaction. If your system or trees are complex, or if process documentation is important, each interaction may be help- ful, but otherwise that tracking may be distracting or tedious. You can use the attack trees in this chapter or in Appendix B “Threat Trees” for this purpose.

If there’s no tree that applies to your system, you can either create one, or use a different threat enumeration building block.

Creating New Attack Trees

If there are no attack trees that you can use for your system, you can create a project-specifi c tree. A project-specifi c tree is a way to organize your thinking about threats. You may end up with one or more trees, but this section assumes you’re putting everything in one tree. The same approach enables you to create trees for a single project or trees for general use.

The basic steps to create an attack tree are as follows:

1. Decide on a representation.

2. Create a root node.

3. Create subnodes.

4. Consider completeness.

5. Prune the tree.

6. Check the presentation.

Decide on a Representation

There are AND trees, where the state of a node depends on all of the nodes below it being true, and OR trees, where a node is true if any of its subnodes are true. You need to decide, will your tree be an AND or an OR tree? (Most will be OR trees.) Your tree can be created or presented graphically or as an outline. See the section “Representing a Tree” later in this chapter for more on the various forms of representation.

www.it-ebooks.info

Chapter 4 ■ Attack Trees 89

c04.indd 11:38:53:AM 01/17/2014 Page 89

Create a Root Node

To create an attack tree, start with a root node. The root node can be the com- ponent that prompts the analysis, or an adversary’s goal. Some attack trees use the problematic state (rather than the goal) as the root. Which you should use is a matter of preference. If the root node is a component, the subnodes should be labeled with what can go wrong for the node. If the root node is an attacker goal, consider ways to achieve that goal. Each alternative way to achieve the goal should be drawn in as a subnode.

The guidance in “Toward a Secure System Engineering Methodology” (Salter, 1999) is helpful to security experts; however, it doesn’t shed much light on how to actually generate the trees, comparative advice about what a root node should be (in other words, whether it’s a goal or a system component and, most important, when one is better than the other), or how to evaluate trees in a structured fashion that would be suitable for those who are not security experts. To be prescriptive:

■ Create a root node with an attacker goal or high-impact action.

■ Use OR trees.

■ Draw them into a grid that the eye can track linearly.

Create Subnodes

You can create subnodes by brainstorming, or you can look for a structured way to fi nd more nodes. The relation between your nodes can be AND or OR, and you’ll have to make a choice and communicate it to those who are using your tree. Some possible structures for first-level subnodes include:

■ Attacking a system:

■ physical access

■ subvert software

■ subvert a person

■ Attacking a system via:

■ People

■ Process

■ Technology

■ Attacking a product during:

■ Design

■ Production

■ Distribution

■ Usage

■ Discard

www.it-ebooks.info

90 Part II ■ Finding Threats

c04.indd 11:38:53:AM 01/17/2014 Page 90

You can use these as a starting point, and make them more specifi c to your system. Iterate on the trees, adding subnodes as appropriate.

N O T E Here the term subnode is used to include leaf (end) nodes and nodes with children, because as you create something you may not always know whether it is a

leaf or whether it has more branches.

Consider Completeness

For this step, you want to determine whether your set of attack trees is complete enough. For example, if you are using components, you might need to add addi- tional trees for additional components. You can also look at each node and ask “is there another way that could happen?” If you’re using attacker motivations, consider additional attackers or motivations. The lists of attackers in Appendix C “Attacker Lists” can be used as a basis.

An attack tree can be checked for quality by iterating over the nodes, look- ing for additional ways to reach the goal. It may be helpful to use STRIDE, one of the attack libraries in the next chapter, or a literature review to help you check the quality.

Prune the Tree

In this step, go through each node in the tree and consider whether the action in each subnode is prevented or duplicative. (An attack that’s worth putting in a tree will generally only be prevented in the context of a project.) If an attack is prevented, by some mitigation you can mark those nodes to indicate that they don’t need to be analyzed. (For example, you can use the test case ID, an “I” for impossible, put a slash through the node, or shade it gray.) Marking the nodes (rather than deleting them) helps people see that the attacks were considered. You might choose to test the assumption that a given node is impossible. See the “Test Process Integration” section in Chapter 10 “Validating That Threats Are Addressed” for more details.

Check the Presentation

Regardless of graphical form, you should aim to present each tree or subtree in no more than a page. If your tree is hard to see on a page, it may be help- ful to break it into smaller trees. Each top level subnode can be the root of a new tree, with a “context” tree that shows the overall relations. You may also be able to adjust presentation details such as font size, within the constraints of usability.

The node labels should be of the same form, focusing on active terms. Finally, draw the tree on a grid to make it easy to track. Ideally, the equivalent level subnodes will show on a single line. That becomes more challenging as you go deeper into a tree.

www.it-ebooks.info

Chapter 4 ■ Attack Trees 91

c04.indd 11:38:53:AM 01/17/2014 Page 91

Representing a Tree

Trees can be represented in two ways: as a free-form (human-viewable) model without any technical structure, or as a structured representation with variable types and/or metadata to facilitate programmatic analysis.

Human-Viewable Representations

Attack trees can be drawn graphically or shown in outline form. Graphical representations are a bit more work to create but have more potential to focus attention. In either case, if your nodes are not all related by the same logic (AND/OR), you’ll need to decide on a way to represent the relationship and communicate that decision. If your tree is being shown graphically, you’ll also want to decide if you use a distinct shape for a terminal node: The labels in a node should be carefully chosen to be rich in information, especially if you’re using a graphical tree. Words such as “attack” or “via” can distract from the key information. Choose “modify fi le” over “attack via modifying fi le.” Words such as “weak” are more helpful when other nodes say “no.” So “weak cryptography” is a good contrast to “no cryptography.”

As always, care should be taken to ensure that the graphics are actually information-rich and communicative. For instance, consider the three repre- sentations of a tree shown in Figure 4–1.

Asset/Revenue Overstatement

Asset/Revenue Overstatement

Asset/Revenue Overstatement

Timing Differences

Fictitious Revenue

Timing Differences

Fictitious Revenue

Timing Differences

Fictitious Revenue

Figure 4–1: Three representations of a tree

The left tree shows an example of a real tree that simply uses boxes. This rep- resentation does not clearly distinguish hierarchy, making it hard to tell which nodes are at the same level of the tree. Compare that to the center tree, which uses a tree to show the equivalence of the leaf nodes. The rightmost tree adds the “OR gate” symbol from circuit design to show that any of the leaf nodes lead to the parent condition.

Additionally, tree layout should make considered use of space. In the very small tree in Figure 4–2, note the pleasant grid that helps your eye follow the layout. In contrast, consider the layout of Figure 4–3, which feels jumbled. To focus your attention on the layout, both are shown too small to read.

www.it-ebooks.info

92 Part II ■ Finding Threats

c04.indd 11:38:53:AM 01/17/2014 Page 92

Conflicts of Interest Bribery

Corruption AssetMisappropriation Fraudulent Statements

Purchases Schemes

Sales Schemes

Other Other

Cash

Fraudulent Disbursements

Inventory and all

Other Assets

Larceny Skimming Misuse Larceny

Bid Rigging

Invoice Kickbacks

Illegal Gratuities

Economic Extortion Financial

Non- Financial

Employment Credentials

Asset/Revenue Overstatements

Asset/Revenue Understatements

Timing Differences

Fictitious Revenues

Concealed Liabilities

Unconcealed Unconcealed

Larceny

Billing Schemes

Shell Company

Non-Accomplice Vendor

Personal Purchases

Workers Compensation

Falsified Wages

Ghost Employees

Commission Schemes

Mischaracterized Expenses

Overstated Expenses

Fictitious Expenses

Multiple Reimbursements

Altered Payee

Authorized Maker

Forged Endorsement

Concealed Checks

Forged Maker

Payroll Schemes

Expense Reimbursement

Schemes

Check Tampering

Register Disbursements

False Voids

False Refunds

Improper Disclosures

Improper Asset

Valuations

Sales Receivables Refunds &other Asset Req. &

Transfers

Fake Sales & Shipping

Purchasing & Receiving

Write-off Schemes

Lapping Schemes

Of Cash on Hand

From the Deposit

Other Understated

Unrecorded

Internal Documents

External Documents

Figure 4–2: A tree drawn on a grid

Repudiate message

Weak signature system

Replay attacks

Weak logging

Spoofing external entity

Spoofing external entity

Logs weaker than authentication system

Logging unauthenticated or weakly authenticated

data

No logs Logging

insufficient data

Tampering threats

against logs

Repudiate transaction

Repudiation, external entity

or process

Figure 4–3: A tree drawn without a grid

www.it-ebooks.info

Chapter 4 ■ Attack Trees 93

c04.indd 11:38:53:AM 01/17/2014 Page 93

N O T E In Writing Secure Code 2 (Microsoft Press, 2003), Michael Howard and David LeBlanc suggest the use of dotted lines for unlikely threats, solid lines for likely

threats, and circles to show mitigations, although including mitigations may make

the trees too complex.

Outline representations are easier to create than graphical representations, but they tend to be less attention-grabbing. Ideally, an outline tree is shown on a single page, not crossing pages. The question of how to effectively represent AND/OR is not simple. Some representations leave them out, others include an indicator either before or after a line. The next three samples are modeled after the trees in “Election Operations Assessment Threat Trees” later in this chapter. As you look at them, ask yourself precisely what is needed to achieve the goal in node 1, “Attack voting equipment.”

1. Attack voting equipment

1.1 Gather knowledge

1.1.1 From insider

1.1.2 From components

1.2 Gain insider access

1.2.1 At voting system vendor

1.2.2 By illegal insider entry

The preceding excerpt isn’t clear. Should the outline be read as a need to do each of these steps, or one or the other to achieve the goal of attacking voting equipment? Contrast that with the next tree, which is somewhat better:

1. Attack voting equipment

1.1 Gather knowledge (and)

1.1.1 From insider (or)

1.1.2 From components

1.2 Gain insider access (and)

1.2.1 At voting system vendor (or)

1.2.2 By illegal insider entry

This representation is useful at the end nodes: It is clearly 1.1.1 or 1.1.2. But what does the “and” on line 1.1 refer to? 1.1.1 or 1.1.2? The representation is not clear. Another possible form is shown next:

www.it-ebooks.info

94 Part II ■ Finding Threats

c04.indd 11:38:53:AM 01/17/2014 Page 94

1. Attack voting equipment

O 1.1 Gather knowledge

T 1.1.1 From insider

O 1.1.2 From components

O 1.2 Gain insider access

T 1.2.1 At voting system vendor

T 1.2.2 By illegal insider entry

This is intended to be read as “AND Node: 1: Attack voting equipment, involves 1.1, gather knowledge either from insider or from components AND 1.2, gain insider access . . .” This can be confusing if read as the children of that node are to be ORed, rather than being ORed with its sibling nodes. This is much clearer in the graphical presentation. Also note that the steps are intended to be sequential. You must gather knowledge, then gain insider access, then attack the components to pull off the attack.

As you can see from the preceding examples, the question of how to use an out- line representation of a tree is less simple than you might expect. If you are using someone else’s tree, be sure you understand their intent. If you are creating a tree, be sure you are clear on your intent, and clear in your communication of your intent.

Structured Representations

Graphical and outline presentation of trees are useful for humans, but a tree is also a data structure, and a structured representation of a tree makes it pos- sible to apply logic to the tree and in turn, the system you’re modeling. Several software packages enable you to create and manage complex trees. One such package allows the modeler to add costs to each node, and then assess what attacks an attacker with a given budget can execute. As your trees become more complex, such software is more likely to be worthwhile. See Chapter 11 “Threat Modeling Tools” for a list of tree management software.

Example Attack Tree

The following simple example of an attack tree (and a useful component for other attack tree activity) models how an attacker might get into a building. The entire tree is an OR tree; any of the methods listed will achieve the goal. (This tree is derived from “An Attack Tree for the Border Gateway Protocol” [Convery, 2004].)

www.it-ebooks.info

Chapter 4 ■ Attack Trees 95

c04.indd 11:38:53:AM 01/17/2014 Page 95

Goal: Access to the building

1. Go through a door

a. When it’s unlocked:

i. Get lucky.

ii. Obstruct the latch plate (the “Watergate Classic”).

iii. Distract the person who locks the door at night.

b. Drill the lock.

c. Pick the lock.

d. Use the key.

i. Find a key.

ii. Steal a key.

iii. Photograph and reproduce the key.

iv. Social engineer a key from someone.

1. Borrow the key.

2. Convince someone to post a photo of their key ring.

e. Social engineer your way in.

i. Act like you’re authorized and follow someone in.

ii. Make friends with an authorized person.

iii. Carry a box, a cup of coffee in each hand, etc.

2. Go through a window.

a. Break a window.

b. Lift the window.

3. Go through a wall.

a. Use a sledgehammer or axe.

b. Use a truck to go through the wall.

4. Gain access via other means.

a. Use a fi re escape.

b. Use roof access from a helicopter (preferably black) or adjacent building.

c. Enter another part of the building, using another tenant’s access.

www.it-ebooks.info

96 Part II ■ Finding Threats

c04.indd 11:38:53:AM 01/17/2014 Page 96

Real Attack Trees

A variety of real attack trees have been published. These trees may be helpful to you either directly, because they model systems like the one you’re model- ing, or as examples of how to build an attack tree. The three attack trees in this section show how insiders commit fi nancial fraud, how to attack elections, and threats against SSL.

Each of these trees has the nice property of being available now, either as an extended example, as a model for you to build from, or (if you’re working around fraud, elections, or SSL), to use directly in analyzing a system which matters to you.

The fraud tree is designed for you to use. In contrast, the election trees were developed to help the team think through their threats and organize the possibilities.

Fraud Attack Tree

An attack tree from the Association of Certifi ed Fraud Examiners is shown with their gracious permission in Figure 4–4, and it has a number of good qualities. First, it’s derived from actual experience in fi nding and exposing fraud. Second, it has a structure put together by subject matter experts, so it’s not a random collection of threats. Finally, it has an associated set of mitigations, which are discussed at great length in Joseph Wells’ Corporate Fraud Handbook (Wiley, 2011).

Election Operations Assessment Threat Trees

The largest publicly accessible set of threat trees was created for the Elections Assistance Commission by a team centered at the University of Southern Alabama. There are six high-level trees. They are useful both as an example and for you to use directly, and there are some process lessons you can learn.

N O T E This model covers a wider scope of attacks than typical for software threat models, but is scoped like many operational threat models.

1. Attack voting equipment.

2. Perform an insider attack.

3. Subvert the voting process.

4. Experience technical failure.

5. Attack audit.

6. Disrupt operations.

www.it-ebooks.info

Chapter 4 ■ Attack Trees 97

c04.indd 11:38:53:AM 01/17/2014 Page 97

Conflicts of Interest Bribery

Corruption AssetMisappropriation Fraudulent Statements

Purchases Schemes

Sales Schemes

Other Other

Cash

Fraudulent Disbursements

Inventory and all

Other Assets

Larceny Skimming Misuse Larceny

Bid Rigging

Invoice Kickbacks

Illegal Gratuities

Economic Extortion Financial

Non- Financial

Employment Credentials

Asset/Revenue Overstatements

Asset/Revenue Understatements

Timing Differences

Fictitious Revenues

Concealed Liabilities

Unconcealed Unconcealed

Larceny

Billing Schemes

Shell Company

Non-Accomplice Vendor

Personal Purchases

Workers Compensation

Falsified Wages

Ghost Employees

Commission Schemes

Mischaracterized Expenses

Overstated Expenses

Fictitious Expenses

Multiple Reimbursements

Altered Payee

Authorized Maker

Forged Endorsement

Concealed Checks

Forged Maker

Payroll Schemes

Expense Reimbursement

Schemes

Check Tampering

Register Disbursements

False Voids

False Refunds

Improper Disclosures

Improper Asset

Valuations

Sales Receivables Refunds &other Asset Req. &

Transfers

Fake Sales & Shipping

Purchasing & Receiving

Write-off Schemes

Lapping Schemes

Of Cash on Hand

From the Deposit

Other Understated

Unrecorded

Internal Documents

External Documents

Figure 4–4: The ACFE fraud tree

www.it-ebooks.info

98 Part II ■ Finding Threats

c04.indd 11:38:53:AM 01/17/2014 Page 98

If your system is vulnerable to threats such as equipment attack, insider attack, process subversion or disruption, these attack trees may work well to help you fi nd threats against those systems.

The team created these trees to organize their thinking around what might go wrong. They described their process as having found a very large set of issues via literature review, brainstorming, and original research. They then broke the threats into high-level sets, and had individuals organize them into trees. An attempt to sort the sets into a tree in a facilitated group process did not work (Yanisac, 2012). The organization of trees may require a single person or a very close-knit team; you should be cautious about trying for consensus trees.

Mind Maps

Application security specialist Ivan Ristic (Ristić, 2009) conducted an interesting experiment using a mind map for what he calls an SSL threat model, as shown in Figure 4–5.

This is an interesting way to present a tree. There are very few mind-map trees out there. This tree, like the election trees, shows a set of editorial decisions and those who use mind maps may fi nd the following perspective on this mind map helpful:

■ The distinction between “Protocols/Implementation bugs” and “End points/Client side/secure implementation” is unclear.

■ There’s “End points/Client side/secure implementation” but no “server side” counterpart to that.

■ Under “End points/server side/server confi g” there’s a large subtree. Compare that to Client side where there’s no subtree at all.

■ Some items have an asterisk (*) but it’s unclear what that means. After discussion with Ivan, it turns out that those “may not apply to everyone.”

■ There’s an entire set of traffi c analytic threats that allow you to see where on a secure site someone is. These issues are made worse by AJAX, but more important here, how should they fi t into this mind map? Perhaps under “Protocols/specifi cations/scope limits”?

■ It’s hard to fi nd elements of the map, as it draws the eye in various direc- tions, some of which don’t align with the direction of reading.

Perspective on Attack Trees

Attack trees can be a useful way to convey information about threats. They can be helpful even to security experts as a way to quickly consider possible attack types. However, despite their surface appeal, it is very hard to create attack trees.

www.it-ebooks.info

c04.indd 11:38:53:AM 01/17/2014 Page 99

Trust path validation bugs

NUL-byte Certificates

Leaked CA Certificates

Rogue CA Certificates Rogue Sysadmin

Server Compromise

Backup Compromise

Attacks against sysadmins

Social engineering

Validation software subversion

Forgery

BriberyFailure to enforce SSL

Expired certificate

Incorrectly configured chain

Invalid hostname

Not valid for all required hostnames

Insufficient assurance (*)

Self-signed Certificates

Unprotected Private Key

Private Key Duplication (*)

Private Key reuse Lack of trust validation

Validation against other root certs

Lack of revocation checking

Client Authentication

Use of weak protocols

Weak key exchange (*)

Weak ciphers (*)

Non-FIPS approved ciphers (*)

Anonymous key exchange

Invalid Certificates

Configuration errors

Server Configuration Server-side

End Points

SSL Threat Model

Protocols Specifications

Scope limitations

No IP layer protection

Not end-to-end

No certificate information protection

Hostname leakage (via SNI)

Downgrade attack (SSLv2)

Truncation attack (SSLv2)

Bleichenbacher adaptive chosen-ciphertext attack

Klima-Pokorny-Rosa adaptive chosen-ciphertext attack

etc.. Implementation bugs

Usability

Prevalence of self-signed certificates

Domain name spoofing Internationalised domain names

Similar domain names DNS Cache Poisoning

MITM LAN

Wireless

Route hijacking (BGP)

Phishing

Corporate interception

XSS

Weaknesses

Users

Attacks

Configuration Weaknesses

Use of unpactched SSL libraries

Mixed SSL/Non-SSL Areas

Insecure cookies

Site Implementation

User Interface (Usability)

Client Configuration

Secure Implementation Lack of revocation checking

Client Side

Validation errors

Theft Site certificate attacks

Trust (PKI)

Certificate Validation Bugs

CA Certificate Attacks

Figure 4–5: Ristic’s SSL mind map

www.it-ebooks.info

100 Part II ■ Finding Threats

c04.indd 11:38:53:AM 01/17/2014 Page 100

I hope that we’ll see experimentation and perhaps progress in the quality of advice. There are also a set of issues that can make trees hard to use, including completeness, scoping, and meaning:

■ Completeness: Without the right set of root nodes, you could miss entire attack groupings. For example, if your threat model for a safe doesn’t include “pour liquid nitrogen on the metal, then hit with a hammer,” then your safe is unlikely to resist this attack. Drawing a tree encourages specifi c questions, such as “how could I open the safe without the combination?” It may or may not bring you to the specifi c threat. Because there’s no way of knowing how many nodes a branch should have, you may never reach that point. A close variant of the this is how do you know that you’re done? (Schneier’s attack tree article alludes to these problems.)

■ Scoping: It may be unreasonable to consider what happens when the computer’s main memory is rapidly cooled and removed from the moth- erboard. If you write commercial software for word processing, this may seem like an operating system issue. If you create commercial operating systems, it may seem like a hardware issue. The nature of attack trees means many of the issues discovered will fall under the category of “there’s no way for us to fi x that.”

■ Meaning: There is no consistency around AND/OR, or around sequence, which means that understanding a new tree takes longer.

Summary

Attack trees fi t well into the four-step framework for threat modeling. They can be a useful tool for fi nding threats, or a way to organize thinking about threats (either for your project or more broadly).

To create a new attack tree to help you organize thinking, you need to decide on a representation, and then select a root node. With that root node, you can brainstorm, use STRIDE, or use a literature review to fi nd threats to add to nodes. As you iterate over the nodes, consider if the tree is complete or overly- full, aiming to ensure the right threats are in the tree. When you’re happy with the content of the tree, you should check the presentation so others can use it. Attack trees can be represented as graphical trees, as outlines, or in software.

You saw a sample tree for breaking into a building, and real trees for fraud, elections, and SSL. Each can be used as presented, or as an example for you to consider how to construct trees of your own.

www.it-ebooks.info

101

c05.indd 07:52:5:AM 01/15/2014 Page 101

Some practitioners have suggested that STRIDE is too high level, and should be replaced with a more detailed list of what can go wrong. Insofar as STRIDE being abstract, they’re right. It could well be useful to have a more detailed list of common problems.

A library of attacks can be a useful tool for fi nding threats against the system you’re building. There are a number of ways to construct such a library. You could collect sets of attack tools; either proof-of-concept code or fully developed (“weaponized”) exploit code can help you understand the attacks. Such a collec- tion, where no modeling or abstraction has taken place, means that each time you pick up the library, each participant needs to spend time and energy creat- ing a model from the attacks. Therefore, a library that provides that abstraction (and at a more detailed level than STRIDE) could well be useful. In this chapter, you’ll learn about several higher-level libraries, including how they compare to checklists and literature reviews, and a bit about the costs and benefi ts of creating a new one.

Properties of Attack Libraries

As stated earlier, there are a number of ways to construct an attack library, so you probably won’t be surprised to learn that selecting one involves trade-offs,

C H A P T E R

5

Attack Libraries

www.it-ebooks.info

102 Part II ■ Finding Threats

c05.indd 07:52:5:AM 01/15/2014 Page 102

and that different libraries address different goals. The major decisions to be made, either implicitly or explicitly, are as follows:

■ Audience

■ Detail versus abstraction

■ Scope

Audience refers to whom the library targets. Decisions about audience dra- matically infl uence the content and even structure of a library. For example, the “Library of Malware Traffi c Patterns” is designed for authors of intrusion detection tools and network operators. Such a library doesn’t need to spend much, if any, time explaining how malware works.

The question of detail versus abstraction is about how many details are included in each entry of the library. Detail versus abstraction is, in theory, simple. You pick the level of detail at which your library should deliver, and then make sure it lands there. Closely related is structure, both within entries and between them. Some libraries have very little structure, others have a great deal. Structure between entries helps organize new entries, while structure within an entry helps promote consistency between entities. However, all that structure comes at a cost. Elements that are hard to categorize are inevitable, even when the things being categorized have some form of natural order, such as they all descend from the same biological origin. Just ask that egg-laying mammal, the duck- billed platypus. When there is less natural order (so to speak), categorization is even harder. You can conceptualize this as shown in Figure 5-1.

DetailedAbstract

STRIDE OWASP Top 10 CAPEC Checklist

Figure 5-1: Abstraction versus detail

Scope is also an important characteristic of an attack library. If it isn’t shown by a network trace, it probably doesn’t fi t the malware traffi c attack library. If it doesn’t impact the web, it doesn’t make sense to include it in the OWASP attack libraries.

There’s probably more than one sweet spot for libraries. They are a balance of listing detailed threats while still being thought provoking. The thought- provoking nature of a library is important for good threat modeling. A thought- provoking list means that some of the engineers using it will find interesting and different threats. When the list of threats reaches a certain level of granularity, it stops prompting thinking, risks being tedious to apply, and becomes more and more of a checklist.

www.it-ebooks.info

Chapter 5 ■ Attack Libraries 103

c05.indd 07:52:5:AM 01/15/2014 Page 103

The library should contain something to help remind people using it that it is not a complete enumeration of what could go wrong. The precise form of that reminder will depend on the form of the library. For example, in Elevation of Privilege, it is the ace card(s), giving an extra point for a threat not in the game.

Closely related to attack libraries are checklists and literature reviews, so before examining the available libraries, the following section looks at checklists and literature reviews.

Libraries and Checklists

Checklists are tremendously useful tools for preventing certain classes of prob- lems. If a short list of problems is routinely missed for some reason, then a checklist can help you ensure they don’t recur. Checklists must be concise and actionable.

Many security professionals are skeptical, however, of “checklist security” as a substitute for careful consideration of threats. If you hate the very idea of checklists, you should read The Checklist Manifesto by Atul Gawande. You might be surprised by how enjoyable a read it is. But even if you take a big-tent approach to threat modeling, that doesn’t mean checklists can replace the work of trained people using their judgment.

A checklist helps people avoid common problems, but the modeling of threats has already been done when the checklist is created. Therefore, a checklist can help you avoid whatever set of problems the checklist creators included, but it is unlikely to help you think about security. In other words, using a checklist won’t help you fi nd any threats not on the list. It is thus narrower than threat modeling.

Because checklists can still be useful as part of a larger threat modeling process, you can fi nd a collection of them at the end of Chapter 1, “Dive In and Threat Model!” and throughout this book as appropriate. The Elevation of Privilege game, by the way, is somewhat similar to a checklist. Two things distinguish it. The fi rst is the use of aces to elicit new threats. The second is that by making threat modeling into a game, players are given social permission to playfully analyze a system, to step beyond the checklist, and to engage with the security questions in play. The game implicitly abandons the “stop and check in” value that a checklist provides.

Libraries and Literature Reviews

A literature review is roughly consulting the library to learn what has happened in the past. As you saw in Chapter 2, “Strategies for Threat Modeling,” reviewing threats to systems similar to yours is a helpful starting point in threat modeling. If you write up the input and output of such a review, you may have the start of

www.it-ebooks.info

104 Part II ■ Finding Threats

c05.indd 07:52:5:AM 01/15/2014 Page 104

an attack library that you can reuse later. It will be more like an attack library if you abstract the attacks in some way, but you may defer that to the second or third time you review the attack list.

Developing a new library requires a very large time investment, which is probably part of why there are so few of them. However, another reason might be the lack of prescriptive advice about how to do so. If you want to develop a literature review into a library, you need to consider how the various attacks are similar and how they differ. One model you can use for this is a zoo. A zoo is a grouping—whether of animals, malware, attacks, or other things—that taxonomists can use to test their ideas for categorization. To track your zoo of attacks, you can use whatever form suits you. Common choices include a wiki, or a Word or Excel document. The main criteria are ease of use and a space for each entry to contain enough concrete detail to allow an analyst to dig in.

As you add items to such a zoo, consider which are similar, and how to group them. Be aware that all such categorizations have tricky cases, which sometimes require reorganization to refl ect new ways of thinking about them. If your categorization technique is intended to be used by multiple independent people, and you want what’s called “inter-rater consistency,” then you need to work on a technique to achieve that. One such technique is to create a fl owchart, with specifi c questions from stage to stage. Such a fl owchart can help produce consistency.

The work of grouping and regrouping can be a considerable and ongoing investment. If you’re going to create a new library, consider spending some time fi rst researching the history and philosophy of taxonomies. Books like Sorting Things Out: Classifi cation and Its Consequences (Bowker, 2000) can help.

CAPEC

The CAPEC is MITRE’s Common Attack Pattern Enumeration and Classifi cation. As of this writing, it is a highly structured set of 476 attack patterns, organized into 15 groups:

■ Data Leakage

■ Attacks Resource Depletion

■ Injection (Injecting Control Plane content through the Data Plane)

■ Spoofi ng

■ Time and State Attacks

■ Abuse of Functionality

www.it-ebooks.info

Chapter 5 ■ Attack Libraries 105

c05.indd 07:52:5:AM 01/15/2014 Page 105

■ Probabilistic Techniques

■ Exploitation of Authentication

■ Exploitation of Privilege/Trust

■ Data Structure Attacks

■ Resource Manipulation

■ Network Reconnaissance

■ Social Engineering Attacks

■ Physical Security Attacks

■ Supply Chain Attacks

Each of these groups contains a sub-enumeration, which is available via MITRE (2013b). Each pattern includes a description of its completeness, with values ranging from “hook” to “complete.” A complete entry includes the following:

■ Typical severity

■ A description, including:

■ Summary

■ Attack execution fl ow

■ Prerequisites

■ Method(s) of attack

■ Examples

■ Attacker skills or knowledge required

■ Resources required

■ Probing techniques

■ Indicators/warnings of attack

■ Solutions and mitigations

■ Attack motivation/consequences

■ Vector

■ Payload

■ Relevant security requirements, principles and guidance

■ Technical context

■ A variety of bookkeeping fi elds (identifi er, related attack patterns and vulnerabilities, change history, etc.)

www.it-ebooks.info

106 Part II ■ Finding Threats

c05.indd 07:52:5:AM 01/15/2014 Page 106

An example CAPEC is shown in Figure 5-2. You can use this very structured set of information for threat modeling in

a few ways. For instance, you could review a system being built against either each CAPEC entry or the 15 CAPEC categories. Reviewing against the individual entries is a large task, however; if a reviewer averages fi ve minutes for each of the 475 entries, that’s a full 40 hours of work. Another way to use this information is to train people about the breadth of threats. Using this approach, it would be possible to create a training class, probably taking a day or more.

Exit Criteria

The appropriate exit criteria for using CAPEC depend on the mode in which you’re using it. If you are performing a category review, then you should have at least one issue per categories 1–11 (Data Leakage, Resource Depletion, Injection, Spoofi ng, Time and State, Abuse of Functionality, Probabilistic Techniques, Exploitation of Authentication, Exploitation of Privilege/Trust, Data Structure Attacks, and Resource Manipulation) and possibly one for categories 12–15 (Network Reconnaissance, Social Engineering, Physical Security, Supply Chain).

Perspective on CAPEC

Each CAPEC entry includes an assessment of its completion, which is a nice touch. CAPECs include a variety of sections, and its scope differs from STRIDE in ways that can be challenging to unravel. (This is neither a criticism of CAPEC, which existed before this book, nor a suggestion that CAPEC change.)

The impressive size and scope of CAPEC may make it intimidating for people to jump in. At the same time, that specifi city may make it easier to use for someone who’s just getting started in security, where specifi city helps to identify attacks. For those who are more experienced, the specifi city and apparent completeness of CAPEC may result in less creative thinking. I personally fi nd that CAPEC’s impressive size and scope make it hard for me to wrap my head around it.

CAPEC is a classifi cation of common attacks, whereas STRIDE is a set of security properties. This leads to an interesting contrast. CAPEC, as a set of attacks, is a richer elicitation technique. However, when it comes to addressing the CAPEC attacks, the resultant techniques are far more complex. The STRIDE defenses are simply those approaches that preserve the property. However, looking up defenses is simpler than fi nding the attacks. As such, CAPEC may have more promise than STRIDE for many populations of threat modelers. It would be fascinating to see efforts made to improve CAPEC’s usability, perhaps with cheat sheets, mnemonics, or software tools.

www.it-ebooks.info

Chapter 5 ■ Attack Libraries 107

c05.indd 07:52:5:AM 01/15/2014 Page 107

Figure 5-2: A sample CAPEC entry

www.it-ebooks.info

108 Part II ■ Finding Threats

c05.indd 07:52:5:AM 01/15/2014 Page 108

OWASP Top Ten

OWASP, The Open Web Application Security Project, offers a Top Ten Risks list each year. In 2013, the list was as follows:

■ Injection

■ Broken Authentication and Session Management

■ Cross-Site Scripting

■ Insecure Direct Object References

■ Security Misconfi guration

■ Sensitive Data Exposure

■ Missing Function Level Access Control

■ Cross Site Request Forgery

■ Components with Known Vulnerabilities

■ Invalidated Requests and Forwards [sic]

This is an interesting list from the perspective of the threat modeler. The list is a good length, and many of these attacks seem like they are well-balanced in terms of attack detail and its power to provoke thought. A few (cross-site scripting and cross-site request forgery) seem overly specifi c with respect to threat modeling. They may be better as input into test planning.

Each has backing information, including threat agents, attack vectors, security weaknesses, technical and business impacts, as well as details covering whether you are vulnerable to the attack and how you prevent it.

To the extent that what you’re building is a web project, the OWASP Top Ten list is probably a good adjunct to STRIDE. OWASP updates the Top Ten list each year based on the input of its volunteer membership. Over time, the list may be more or less valuable as a threat modeling attack library.

The OWASP Top Ten are incorporated into a number of OWASP-suggested methodologies for web security. Turning the Top Ten into a threat modeling methodology would likely involve creating something like a STRIDE-per-element approach (Top Ten per Element?) or looking for risks in the list at each point where a data fl ow has crossed a trust boundary.

Summary

By providing mode specifi cs, attack libraries may be useful to those who are not deeply familiar with the ways attackers work. It is challenging to fi nd generally useful sweet spots between providing lots of details and becoming tedious. It

www.it-ebooks.info

Chapter 5 ■ Attack Libraries 109

c05.indd 07:52:5:AM 01/15/2014 Page 109

is also challenging to balance details with the threat of fooling a reader into thinking a checklist is comprehensive. Performing a literature review and cap- turing the details in an attack library is a good way for someone to increase their knowledge of security.

There are a number of attack libraries available, including CAPEC and the OWASP Top Ten. Other libraries may also provide value depending on the technology or system on which you’re working.

www.it-ebooks.info

www.it-ebooks.info

111

c06.indd 02:15:11:PM 01/16/2014 Page 111

Threat modeling for privacy issues is an emergent and important area. Much like security threats violate a required security property, privacy threats are where a required privacy property is violated. Defi ning privacy requirements is a delicate balancing act, however, for a few reasons: First, the organization offering a service may want or even need a lot of information that the people using the service don’t want to provide. Second, people have very different perceptions of what privacy is, and what data is private, and those perceptions can change with time. (For example, someone leaving an abusive relation- ship should be newly sensitive to the value of location privacy, and perhaps consider their address private for the fi rst time.) Lastly, most people are “privacy pragmatists” and will make value tradeoffs for personal information.

Some people take all of this ambiguity to mean that engineering for privacy is a waste. They’re wrong. Others assert that concern over privacy is a waste, as consumers don’t behave in ways that expose privacy concerns. That’s also wrong. People often pay for privacy when they understand the threat and the mitigation. That’s why advertisements for curtains, mailboxes, and other privacy- enhancing technologies often lead with the word “privacy.”

Unlike the previous three chapters, each of which focused on a single type of tool, this chapter is an assemblage of tools for fi nding privacy threats. The approaches described in this chapter are more developed than “worry about privacy,” yet they are somewhat less developed than security attack libraries such as CAPEC (discussed in Chapter 5, “Attack Libraries”). In either event, they

C H A P T E R

6

Privacy Tools

www.it-ebooks.info

112 Part II ■ Finding Threats

c06.indd 02:15:11:PM 01/16/2014 Page 112

are important enough to include. Because this is an emergent area, appropriate exit criteria are less clear, so there are no exit criteria sections here.

In this chapter, you’ll learn about the ways to threat model for privacy, includ- ing Solove’s taxonomy of privacy harms, the IETF’s “Privacy Considerations for Internet Protocols,” privacy impact assessments (PIAs), the nymity slider, contextual integrity, and the LINDDUN approach, a mirror of STRIDE created to fi nd privacy threats. It may be reasonable to treat one or more of contextual integrity, Solove’s taxonomy or (a subset of) LINDDUN as a building block that can snap into the four-stage model, either replacing or complementing the security threat discovery.

N O T E Many of these techniques are easier to execute when threat modeling operational systems, rather than boxed software. (Will your database be used to

contain medical records? Hard to say!) The IETF process is more applicable than other

processes to “boxed software” designs.

Solove’s Taxonomy of Privacy

In his book, Understanding Privacy (Harvard University Press, 2008), George Washington University law professor Daniel Solove puts forth a taxonomy of privacy harms. These harms are analogous to threats in many ways, but also include impact. Despite Solove’s clear writing, the descriptions might be most helpful to those with some background in privacy, and challenging for tech- nologists to apply to their systems. It may be possible to use the taxonomy as a tool, applying it to a system under development, considering whether each of the harms presented is enabled. The following list presents a version of this taxonomy derived from Solove, but with two changes. First, I have added “iden- tifi er creation,” in parentheses. I believe that the creation of an identifi er is a discrete harm because it enables so many of the other harms in the taxonomy. (Professor Solove and I have agreed to disagree on this issue.) Second, exposure is in brackets, because those using the other threat modeling techniques in this Part should already be handling such threats.

■ (Identifi er creation)

■ Information collection: surveillance, interrogation

■ Information processing: aggregation, identifi cation, insecurity, secondary use, exclusion

■ Information dissemination: breach of confi dentiality, disclosure, increased accessibility, blackmail, appropriation, distortion, [exposure]

■ Invasion: intrusion, decisional interference

www.it-ebooks.info

Chapter 6 ■ Privacy Tools 113

c06.indd 02:15:11:PM 01/16/2014 Page 113

Many of the elements of this list are self-explanatory, and all are explained in depth in Solove’s book. A few may benefi t from a brief discussion. The harm of surveillance is twofold: First is the uncomfortable feeling of being watched and second are the behavioral changes it may cause. Identifi cation means the association of information with a fl esh-and-blood person. Insecurity refers to the psychological state of a person made to feel insecure, rather than a techni- cal state. The harm of secondary use of information relates to societal trust. Exclusion is the use of information provided to exclude the provider (or others) from some benefi t.

Solove’s taxonomy is most usable by privacy experts, in the same way that STRIDE as a mnemonic is most useful for security experts. To make use of it in threat modeling, the steps include creating a model of the data fl ows, paying particular attention to personal data.

Finding these harms may be possible in parallel to or replacing security threat modeling. Below is advice on where and how to focus looking for these.

■ Identifi er creation should be reasonably easy for a developer to identify.

■ Surveillance is where data is collected about a broad swath of people or where that data is gathered in a way that’s hard for a person to notice.

■ Interrogation risks tend to correlate around data collection points, for example, the many “* required” fi elds on web forms. The tendency to lie on such forms may be seen as a response to the interrogation harm.

■ Aggregation is most frequently associated with inbound data fl ows from external entities.

■ Identifi cation is likely to be found in conjunction with aggregation or where your system has in-person interaction.

■ Insecurity may associate with where data is brought together for decision purposes.

■ Secondary use may cross trust boundaries, possibly including boundaries that your customers expect to exist.

■ Exclusion happens at decision points, and often fraud management decisions.

■ Information dissemination threats (all of them) are likely to be associated with outbound data fl ows; you should look for them where data crosses trust boundaries.

■ Intrusion is an in-person intrusion; if your system has no such features, you may not need to look at these.

■ Decisional interference is largely focused on ways in which information collection and processing may infl uence decisions, and as such it most likely plays into a requirements discussion.

www.it-ebooks.info

114 Part II ■ Finding Threats

c06.indd 02:15:11:PM 01/16/2014 Page 114

Privacy Considerations for Internet Protocols

The Internet Engineering Task Force (IETF) requires consideration of security threats, and has a process to threat model focused on their organizational needs, as discussed in Chapter 17, “Bringing Threat Modeling to Your Organization.” As of 2013, they sometimes require consideration of privacy threats. An infor- mational RFC “Privacy Considerations for Internet Protocols,” outlines a set of security-privacy threats, a set of pure privacy threats, and offers a set of mitiga- tions and some general guidelines for protocol designers (Cooper, 2013). The combined security-privacy threats are as follows:

■ Surveillance

■ Stored data compromise

■ Mis-attribution or intrusion (in the sense of unsolicited messages and denial-of-service attacks, rather than break-ins)

The privacy-specifi c threats are as follows:

■ Correlation

■ Identifi cation

■ Secondary use

■ Disclosure

■ Exclusion (users are unaware of the data that others may be collecting)

Each is considered in detail in the RFC. The set of mitigations includes data minimization, anonymity, pseudonymity, identity confi dentiality, user participa- tion and security. While somewhat specifi c to the design of network protocols, the document is clear, free, and likely a useful tool for those attempting to threat model privacy. The model, in terms of the abstracted threats and methods to address them, is an interesting step forward, and is designed to be helpful to protocol engineers.

Privacy Impact Assessments (PIA)

As outlined by Australian privacy expert Roger Clarke in his “An Evaluation of Privacy Impact Assessment Guidance Documents,” a PIA “is a systematic process that identifi es and evaluates, from the perspectives of all stakeholders, the potential effects on privacy of a project, initiative, or proposed system or scheme, and includes a search for ways to avoid or mitigate negative privacy impacts.” Thus, a PIA is, in several important respects, a privacy analog to security threat modeling. Those respects include the systematic tools for identifi cation

www.it-ebooks.info

Chapter 6 ■ Privacy Tools 115

c06.indd 02:15:11:PM 01/16/2014 Page 115

and evaluation of privacy issues, and the goal of not simply identifying issues, but also mitigating them. However, as usually presented, PIAs have too much integration between their steps to snap into the four-stage framework used in this book.

There are also important differences between PIAs and threat modeling. PIAs are often focused on a system as situated in a social context, and the evaluation is often of a less technical nature than security threat modeling. Clarke’s evalu- ation criteria include things such as the status, discoverability, and applicability of the PIA guidance document; the identifi cation of a responsible person; and the role of an oversight agency; all of which would often be considered out of scope for threat modeling. (This is not a critique, but simply a contrast.) One sample PIA guideline from the Offi ce of the Victorian Privacy Commissioner states the following:

“Your PIA Report might have a Table of Contents that looks something like this:

1. Description of the project

2. Description of the data fl ows

3. Analysis against ‘the’ Information Privacy Principles

4. Analysis against the other dimensions to privacy

5. Analysis of the privacy control environment

6. Findings and recommendations”

Note that step 2, “description of the data fl ows,” is highly reminiscent of “data fl ow diagrams,” while steps 3 and 4 are very similar to the “threat fi nding” building blocks. Therefore, this approach might be highly complementary to the four-step model of threat modeling.

The appropriate privacy principles or other dimensions to consider are some- what dependent on jurisdiction, but they can also focus on classes of intrusion, such as those offered by Solove, or a list of concerns such as informational, bodily, territorial, communications, and locational privacy. Some of these documents, such as those from the Offi ce of the Victorian Privacy Commissioner (2009a), have extensive lists of common privacy threats that can be used to support a guided brainstorming approach, even if the documents are not legally required. Privacy impact assessments that are performed to comply with a law will often have a formal structure for assessing suffi ciency.

The Nymity Slider and the Privacy Ratchet

University of Waterloo professor Ian Goldberg has defi ned a measurement he calls nymity, the “amount of information about the identity of the participants that is revealed [in a transaction].” Nymity is from the Latin for name, from which anonymous (“without a name”) and pseudonym (“like a name”) are derived.

www.it-ebooks.info

116 Part II ■ Finding Threats

c06.indd 02:15:11:PM 01/16/2014 Page 116

Goldberg has pointed out that you can graph nymity on a continuum (Goldberg, 2000). Figure 6-1 shows the nymity slider. On the left-hand side, there is less privacy than on the right-hand side. As Goldberg points out, it is easy to move towards more nymity, and extremely diffi cult to move away from it. For example, there are protocols for electronic cash that have most of the privacy-preserving properties of physical cash, but if you deliver it over a TCP connection you lose many of those properties. As such, the nymity slider can be used to examine how privacy-threatening a protocol is, and to compare the amount of nymity a system uses. To the extent that it can be designed to use less identifying infor- mation, other privacy features will be easier to achieve.

Verinymity Persistent Pseudonym Pen name

Linkable anonymity Prepaid phone cards

Unlinkable anonymity Cash paymentsGovernment ID

Credit Card # Address

Figure 6-1: The nymity slider

When using nymity privacy in threat modeling, the goal is to measure how much information a protocol, system, or design exposes or gathers. This enables you to compare it to other possible protocols, systems, or designs. The nymity slider is thus an adjunct to other threat-fi nding building blocks, not a replace- ment for them.

Closely related to nymity is the idea of linkability. Linkability is the ability to bring two records together, combining the data in each into a single record or virtual record. Consider several databases, one containing movie preferences, another containing book purchases, and a third containing telephone records. If each contains an e-mail address, you can learn that [email protected] likes religious movies, that he’s bought books on poison, and that several of the people he talks with are known religious extremists. Such intersections might be of interest to the FBI, and it’s a good thing you can link them all together! (Unfortunately, no one bothered to include the professional database showing he’s a doctor, but that’s beside the point!) The key is that you’ve engaged in link- ing several datasets based on an identifi er. There is a set of identifi ers, including e-mail addresses, phone numbers, and government-issued ID numbers, that are often used to link data, which can be considered strong evidence that multiple records refer to the same person. The presence of these strongly linkable data points increases linkability threats.

Linkability as a concept relates closely to Solove’s concept of identifi cation and aggregation. Linkability can be seen as a spectrum from strongly linkable with multiple validated identifi ers to weakly linkable based on similarities in the data.

www.it-ebooks.info

Chapter 6 ■ Privacy Tools 117

c06.indd 02:15:11:PM 01/16/2014 Page 117

(“John Doe and John E. Doe is probably the same person.”) As data becomes richer, the threat of linkage increases, even if the strongly linkable data points are removed. For example, Harvard professor Latanya Sweeney has shown how data with only date of birth, gender, and zip code uniquely identifi es 87 percent of the U.S. population (Sweeney, 2002). There is an emergent scientifi c research stream into “re-identifi cation” or “de-anonymization,” which discloses more such results on a regular basis. The release of anonymous datasets carries a real threat of re-identifi cation, as AOL, Netfl ix, and others have discovered. (McCullagh, 2006; Narayanan, 2008; Buley, 2010).

Contextual Integrity

Contextual integrity is a framework put forward by New York University professor Helen Nissenbaum. It is based on the insight that many privacy issues occur when information is taken from one context and brought into another. A context is a term of art with a deep grounding in discussions of the spheres, or arenas, of our lives. A context has associated roles, activities, norms, and val- ues. Nissenbaum’s approach focuses on understanding contexts and changes to those contexts. This section draws very heavily from Chapter 7 of her book Privacy in Context, (Stanford Univ. Press, 2009) to explain how you might apply the framework to product development.

Start by considering what a context is. If you look at a hospital as a context, then the roles might include doctors, patients, and nurses, but also family mem- bers, administrators, and a host of other roles. Each has a reason for being in a hospital, and associated with that reason are activities that they tend to perform there, norms of behavior, and values associated with those norms and activities.

Contexts are places or social areas such as restaurants, hospitals, work, the Boy Scouts, and schools (or a type of school, or even a specifi c school). An event can be “in a work context” even if it takes place somewhere other than your normal offi ce. Any instance in which there is a defi ned or expected set of “normal” behaviors can be treated as a context. Contexts nest and overlap. For example, normal behavior in a church in the United States is infl uenced by the norms within the United States, as well as the narrower context of the parishioners. Thus, what is normal at a Catholic Church in Boston or a Baptist Revival in Mississippi may be inappropriate at a Unitarian Congregation in San Francisco (or vice versa). Similarly, there are shared roles across all schools, those of student or teacher, and more specifi c roles as you specify an elementary school versus a university. There are specifi c contexts within a university or even the particular departments of a university.

Contextual integrity is violated when the informational norms of a context are breached. Norms, in Nissenbaum’s sense, are “characterized by four key parameters: context, actors, attributes, and transmission principles.” Context

www.it-ebooks.info

118 Part II ■ Finding Threats

c06.indd 02:15:11:PM 01/16/2014 Page 118

is roughly as just described. Actors are senders, recipients, and information subjects. Attributes refer to the nature of the information—for example, the nature or particulars of a disease from which someone is suffering. A transmission principle is “a constraint on the fl ow (distribution, dissemination, transmission) of information from party to party.” Nussbaum fi rst provides two presentations of contextual integrity, followed by an augmented contextual integrity heuristic. As the technique is new, and the “augmented” approach is not a strict superset of the initial presentation, it may help you to see both.

Contextual Integrity Decision Heuristic

Nissenbaum fi rst presents contextual integrity as a post-incident analytic tool. The essence of this is to document the context as follows:

1. Establish the prevailing context.

2. Establish key actors.

3. Ascertain what attributes are affected.

4. Establish changes in principles of transmission.

5. Red fl ag

Step 5 means “if the new practice generates changes in actors, attributes, or transmission principles, the practice is fl agged as violating entrenched infor- mational norms and constitutes a prima facie violation of contextual integrity.” You might have noticed a set of interesting potential overlaps with software development and threat modeling methodologies. In particular, actors over- lap fairly strongly with personas, in Cooper’s sense of personas (discussed in Appendix B, “Threat Trees”). A contextual integrity analysis probably does not require a set of personas for bad actors, as any data fl ow outside the intended participants (and perhaps some between them) is a violation. The information transmissions, and the associated attributes are likely visible in data fl ow or swim lane diagrams developed for normal security threat modeling.

Thus, to the extent that threat models are being enhanced from version to version, a set of change types could be used to trigger contextual integrity analysis. The extant diagram is the “prevailing context.” The important change types would include the addition of new human entities or new data fl ows.

Nissenbaum takes pains to explore the question of whether a violation of contextual integrity is a worthwhile reason to avoid the change. From the perspective of threat elicitation, such discussions are out of scope. Of course, they are in scope as you decide what to do with the identifi ed privacy threats.

www.it-ebooks.info

Chapter 6 ■ Privacy Tools 119

c06.indd 02:15:11:PM 01/16/2014 Page 119

Augmented Contextual Integrity Heuristic

Nissenbaum also presents a longer, ‘augmented’ heuristic, which is more pre- scriptive about steps, and may work better to predict privacy issues.

1. Describe the new practice in terms of information fl ows.

2. Identify the prevailing context.

3. Identify information subjects, senders, and recipients.

4. Identify transmission principles.

5. Locate applicable norms, identify signifi cant changes.

6. Prima facie assessment

7. Evaluation

a. Consider moral and political factors.

b. Identify threats to autonomy and freedom.

c. Identify effects on power structures.

d. Identify implications for justice, fairness, equality, social hierarchy, democracy and so on.

8. Evaluation 2

a. Ask how the system directly impinges on the values, goals, and ends of the context.

b. Consider moral and ethical factors in light of the context.

9. Decide.

This is, perhaps obviously, not an afternoon’s work. However, in considering how to tie this to a software engineering process, you should note that steps 1, 3, and 4 look very much like creating data fl ow diagrams. The context of most organizations is unlikely to change substantially, and thus descriptions of the context may be reusable, as may be the work products to support the evalua- tions of steps 7 and 8.

Perspective on Contextual Integrity

I very much like contextual integrity. It strikes me as providing deep insight into and explanations for a great number of privacy problems. That is, it may be pos- sible to use it to predict privacy problems for products under design. However, that’s an untested hypothesis. One area of concern is that the effort to spell out

www.it-ebooks.info

120 Part II ■ Finding Threats

c06.indd 02:15:11:PM 01/16/2014 Page 120

all the aspects of a context may be quite time consuming, but without spelling out all the aspects, the privacy threats many be missed. This sort of work is challenging when you’re trying to ship software and Nissenbaum goes so far as to describe it as “tedious” (Privacy In Context, page 142). Additionally, the act of fi xing a context in software or structured defi nitions may present risks that the fi xed representation will deviate as social norms evolve.

This presents a somewhat complex challenge to the idea of using contextual integrity as a threat modeling methodology within a software engineering process. The process of creating taxonomies or categories is an essential step in structuring data in a database. Software engineers do it as a matter of course as they develop software, and even those who are deeply cognizant of taxonomies often treat it as an implicit step. These taxonomies can thus restrict the evolution of a context—or worse; generate dissonance between the software-engineered version of the context or the evolving social context. I encourage security and privacy experts to grapple with these issues.

LINDDUN

LUNDDUN is a mnemonic developed by Mina Deng for her PhD at the Katholieke Universiteit in Leuven, Belgium (Deng, 2010). LINDDUN is an explicit mirroring of STRIDE-per-element threat modeling. It stands for the following violations of privacy properties:

■ Linkability

■ Identifi ability

■ Non-Repudiation

■ Detectability

■ Disclosure of information

■ Content Unawareness

■ Policy and consent Noncompliance

LINDDUN is presented as a complete approach to threat modeling with a process, threats, and requirements discovery method. It may be reasonable to use the LINDDUN threats or a derivative as a tool for privacy threat enumeration in the four-stage framework, snapping it either in place of or next to STRIDE security threat enumeration. However, the threats in LINDDUN are somewhat unusual terminology; therefore, the training requirements may be higher, or the learning curve steeper than other privacy approaches.

www.it-ebooks.info

Chapter 6 ■ Privacy Tools 121

c06.indd 02:15:11:PM 01/16/2014 Page 121

N O T E LINDDUN leaves your author deeply confl icted. The privacy terminology it relies on will be challenging for many readers. However, it is, in many ways, one of

the most serious and thought-provoking approaches to privacy threat modeling, and

those seriously interested in privacy threat modeling should take a look. As an aside,

the tension between non-repudiation as a privacy threat and repudiation as a security

threat is delicious.

Summary

Privacy is no less important to society than security. People will usually act to protect their privacy given an understanding of the threats and how they can address them. As such, it may help you to look for privacy threats in addition to security threats. The ways to do so are less prescriptive than ways to look for security threats.

There are many tools you can use to fi nd privacy issues, including Solove’s taxonomy of privacy harms. (A harm is a threat with its impact.) Solove’s taxonomy helps you understand the harm associated with a privacy violation, and thus, perhaps, how best to prioritize it. The IETF has an approach to privacy threats for new Internet protocols. That approach may complement or substitute Privacy Impact Assessments. PIAs and the IETF’s processes are appropriate when a regu- latory or protocol design context calls for their use. Both are more prescriptive than the nymity slider, a tool for assessing the amount of personal information in a system and measuring privacy invasion for comparative purposes. They are also more prescriptive than contextual integrity, an approach which attempts to tease out the social norms of privacy. If your goal is to identify when a design is likely to raise privacy concerns, however, then contextual integrity may be the most helpful. Far more closely related to STRIDE-style threat identifi cation is LINDDUN, which considers privacy violations in the manner that STRIDE considers security violations.

www.it-ebooks.info

www.it-ebooks.info

c07.indd 09:44:3:AM 01/09/2014 Page 123

Par t

III Managing and Addressing

Threats

Part III is all about managing threats and the activities involved in threat mod- eling. While threats themselves are at the heart of threat modeling, the reason you threat model is so that you can deliver more secure products, services, or technologies. This part of the book focuses on the third step in the four-step framework, what to do after you’ve found threats and need to do something about them; but it also covers the fi nal step: validation.

Chapters in this part include the following:

■ Chapter 7: Processing and Managing Threats describes how to start a threat modeling project, how to iterate across threats, the tables and lists you may want to use, and some scenario-specifi c process elements.

■ Chapter 8: Defensive Tactics and Technologies are tools you can use to address threats, ranging from simple to complex. This chapter focuses on a STRIDE breakdown of security threats and a variety of ways to address privacy.

■ Chapter 9: Trade-Offs When Addressing Threats includes risk manage- ment strategies, how to use those strategies to select mitigations, and threat-modeling specifi c prioritization approaches.

www.it-ebooks.info

124 Part III ■ Managing and Addressing Threats

c07.indd 09:44:3:AM 01/09/2014 Page 124

■ Chapter 10: Validating That Threats Are Addressed includes how to test your threat mitigations, QA’ing threat modeling, and process aspects of addressing threats. This is the last step of the four-step approach.

■ Chapter 11: Threat Modeling Tools covers the various tools that you can use to help you threat model, ranging from the generic to the specifi c.

www.it-ebooks.info

125

c07.indd 09:44:3:AM 01/09/2014 Page 125

Finding threats against arbitrary things is fun, but when you’re building some- thing with many moving parts, you need to know where to start, and how to approach it. While Part II is about the tasks you perform and the methodolo- gies you can use to perform them, this chapter is about the processes in which those tasks are performed. Questions of “what to do when” naturally come up as you move from the specifi cs of looking at a particular element of a system to looking at a complete system. To the extent that these are questions of what an individual or small team does, they are addressed in this chapter; questions about what an organization does are covered in Chapter 17, “Bringing Threat Modeling to Your Organization.”

Each of the approaches covered here should work with any of the “Lego blocks” covered in Part II. In this chapter, you’ll learn how to get started look- ing for threats, including when and where to start and how to iterate through a diagram. The chapter continues with a set of tables and lists that you might use as you threat model, and ends with a set of scenario-specifi c guidelines, including the importance of the vendor-customer trust boundary, threat model- ing new technologies, and how to threat model an API.

C H A P T E R

7

Processing and Managing Threats

www.it-ebooks.info

126 Part III ■ Managing and Addressing Threats

c07.indd 09:44:3:AM 01/09/2014 Page 126

Starting the Threat Modeling Project

The basic approach of “draw a diagram and use the Elevation of Privilege game to fi nd threats” is functional, but people prefer different amounts of prescrip- tiveness, so this section provides some additional structure that may help you get started.

When to Threat Model

You can threat model at various times during a process, with each choice having a different value. Most important, you should threat model as you get started on a project. The act of drawing trust boundaries early on can greatly help you improve your architecture. You can also threat model as you work through fea- tures. This allows you to have smaller, more focused threat modeling projects, keeping your skills sharp and reducing the chance that you’ll fi nd big problems at the end. It is also a good idea to revisit the threat model as you get ready to deliver, to ensure that you haven’t made decisions which accidentally altered the reality underlying the model.

Starting at the Beginning

Threat modeling as you get started involves modeling the system you’re plan- ning or building, fi nding threats against the model, and fi ling bugs that you’ll track and manage, such as other issues discovered throughout the develop- ment process. Some of those bugs may be test cases, some might be feature work, and some may be deployment decisions. It depends on what you’re threat modeling.

Working through Features

As you develop each feature, there may be a small amount of threat modeling work to do. That work involves looking deeply at the threats to that feature (and possibly refreshing or validating your understanding of the context by checking the software model). As you start work on a feature or component, it can also be a good time to work through second- or third-order threats. These are the threats in which an attacker will try to bypass the features or design elements that you put in place to block the most immediate threats. For example, if the primary threat is a car thief breaking a window, a secondary threat is them jumping the ignition. You can mitigate that with a steering-wheel lock, which is thus a second-order mitigation. There’s more on this concept of ordered threats in the “Digging Deeper into Mitigations” section later in this chapter,

www.it-ebooks.info

Chapter 7 ■ Processing and Managing Threats 127

c07.indd 09:44:3:AM 01/09/2014 Page 127

as well as more on considering planned mitigations, and how an attacker might work around them.

Threat modeling as you work through a feature has several important value propositions. One is that if you do a small threat model as you start a component or feature, the design is probably closer to mind. In other words, you’ll have a more detailed model with which to look for threats. Another is that if you fi nd threats, they are closer to mind as you’re working on that feature. Threat model- ing as you work through features can also help you maintain your awareness of threats and your skills at threat modeling. (This is especially true if your project is a long one.)

Close to Delivery

Lastly, you should threat model as you get ready to ship by reexamining the model and checking your bugs. (Shipping here is inclusive of delivering, deploy- ing, or going live.) Reexamining the model means ensuring that everyone still agrees it’s a decent model of what you’re building, and that it includes all the trust boundaries and data fl ows that cross them. Checking your bugs involves checking each bug that’s tagged threat modeling (or however else you’re track- ing them), and ensuring it didn’t slip through the cracks.

Time Management

So how long should all this take? The answer to that varies according to system size and complexity, the familiarity of the participants with the system, their skill in threat modeling, and even the culture of meetings in an organization. Some very rough rules of thumb are that you should be able to diagram and fi nd threats against a “component” and decide if you need to do more enumeration in a one-hour session with an experienced threat modeler to moderate or help. For the sort of system that a small start-up might build, the end-to-end threat modeling could take a few hours to a week, or possibly longer if the data the system holds is particularly sensitive. At the larger end of the spectrum, a project to diagram the data fl ows of a large online service has been known to require four people working for several months. That level of effort was required to help fi nd threat variations and alternate routes through a system that had grown to serve millions of people.

Whatever you’re modeling, familiarity with threat modeling helps. If you need to refer back to this book every few minutes, your progress will be slower. One of the reasons to threat model regularly is to build skill and familiarity with the tasks and techniques. Organizational culture also plays a part. Organizations that run meetings with nowhere to sit will likely create a list of threats faster

www.it-ebooks.info

128 Part III ■ Managing and Addressing Threats

c07.indd 09:44:3:AM 01/09/2014 Page 128

than a consensus-oriented organization that encourages exploring ideas. (Which list will be better is a separate and fascinating question.)

What to Start and (Plan to) End With

When you start looking for threats, a diagram is something between useful and essential input. Experienced modelers may be able to start without it, but will likely iterate through creating a diagram as they fi nd threats. The diagram is likely to change as you use it to fi nd threats; you’ll discover things you missed or don’t need. That’s normal, and unless you run a strict waterfall approach to engineering, it’s a process that evolves much like the way requirements evolve as you discover what’s easy or hard to build.

Use the following two testable states to help assess when you’re done:

■ You have fi led bugs.

■ You have a diagram or diagrams that everyone agrees represents the system.

To be more specifi c, you should probably have a number of bugs that’s roughly scaled to the number of things in your diagram. If you’re using a data fl ow diagram and STRIDE, expect to have about fi ve threats per diagram element.

N O T E Originally, this text suggested that you should have: (# of processes * 6) + (# of data fl ows * 3) + (# of data stores * 3.5) + (# of distinct external entities *2) threats,

but that requires keeping four separate counts, and is thus more work to get approxi-

mately the same answer.

You might notice that says “STRIDE” rather than “STRIDE-per-element” or “STRIDE-per-interaction,” and fi ve turns out to match the number you get if you tally up the checkmarks in those charts. That’s because those charts are derived from where the threats usually show up.

Where to Start

When you are staring at a blank whiteboard and wondering where to start, there are several commonly recommended places. Many people have recom- mended assets or attackers, but as you learned in Chapter 2, “Strategies for Threat Modeling,” the best starting place is a diagram that covers the system as a whole, and from there start looking at the trust boundaries. For advice on how to create diagrams, see Chapter 2.

When you assemble a group of people in a room to look for threats, you should include people who know about the software, the data fl ows and (if possible) threat modeling. Begin the process with the component(s) on which the participants in the room are working. You’ll want to start top-down, and

www.it-ebooks.info

Chapter 7 ■ Processing and Managing Threats 129

c07.indd 09:44:3:AM 01/09/2014 Page 129

then work across the system, going “breadth fi rst,” rather than delving deep into any component (“depth fi rst”).

Finding Threats Top Down

Almost any system should be modeled from the highest-level view you can build of the entire system, for some appropriate value of “the entire system”. What constitutes an entire system is, of course, up for debate, just like what constitutes the entire Internet, the entirety of (say) Amazon’s website, and so on, isn’t a simple question. In such cases more scoping is needed. The ideal is probably what is within an organization’s control and, to the extent possible, cross-reviews with those responsible for other components.

In contrast, bottom-up threat modeling starts from features, and then attempts to derive a coherent model from those feature-level models. This doesn’t work well, but advice that implies you should do this is common, so a bit of discussion may be helpful. The reason this doesn’t work is because it turns out to be very challenging to bring threat models together when they are not derived from a system-level view. As such, you should start from the highest-level view you can build of the entire system.

MICROSOFT’S BOTTOMUP EXPERIENCE

It may help to understand the sorts of issues that can lead to a bottom-up approach. At Microsoft in the mid-2000s, there was an explosion of bottom-up threat model- ing. There were three drivers for this: specifi c words in the Security Development Lifecycle (SDL) threat model requirement, aspects of Microsoft’s approach to func- tion teams, and the work involved in creating top-level models. The SDL required “all new features” be threat modeled. This intersected with an approach to features whereby a particular team of developer, tester, and program manager owns a feature and collaborates to ship it. Because the team owned its feature, it was natural to ask it to add threat models to the specifi cations involved in producing it. As Microsoft’s approach to security evolved, product security teams had diverse sets of important tasks to undertake. Creating all-up threat models was usually not near the top of the list. (Many large product diagrams have now done that work and found it worthwhile. Some of these diagrams require more than one poster-size sheet of paper.)

Finding Threats “Across”

Even with a top-down approach, you want to go breadth fi rst, and there are three different lists you can iterate “across”: A list of the trust boundaries, a list of diagram elements, or a list of threats. A structure can help you look

www.it-ebooks.info

130 Part III ■ Managing and Addressing Threats

c07.indd 09:44:3:AM 01/09/2014 Page 130

for threats, either because you and your team like structure, or because the task feels intimidating and you want to break it down, Table 7-1 shows three approaches.

Table 7-1: Lists to Iterate Across

METHOD SAMPLE STATEMENT COMMENTS

Start from what goes across trust boundaries.

“What can go wrong as foo comes across this trust boundary?

This is likely to identify the highest-value threats.

Iterate across diagram elements.

“What can go wrong with this data- base fi le?” “What can go wrong with the logs?”

Focusing on diagram ele- ments may work well when a lot of teams are collaborating.

Iterate across the threats.

“Where are the spoofi ng threats in this diagram?” “Where are the tam- pering threats?”

Making threats the focus of discussion may help you fi nd related threats.

Each of these approaches can be a fi ne way to start, as long as you don’t let them become straightjackets. If you don’t have a preference, try starting from what crosses trust boundaries, as threats tend to cluster there. However you iterate, ensure that you capture each threat as it comes up, regardless of the planned approach.

Digging Deeper into Mitigations

Many times threats will be mitigated by the addition of features, which can be designed, developed, tested and delivered much like other features. (Other times, mitigation might be a confi guration change, or at the other end of the effort scale, require re-design.) However, mitigations are not quite like other features. An attacker will rarely try to work around the bold button and fi nd an unintended, unsupported way to bold their text.

Finding threats is great, and to the extent that you plan to be attacked only by people who are exactly lazy enough to fi nd a threat but not enthusiastic enough to try to bypass your mitigation, you don’t need to worry about going deeper into the mitigations. (You may have to worry about a new job, but that, as they say, is beyond the scope of this book. I recommend Mike Murray’s Forget the Parachute: Let Me Fly the Plane.) In this section, you’ll learn about how to go deeper into the interplay of how attackers can attempt to bypass the design choices and features you put in place to make their lives harder.

www.it-ebooks.info

Chapter 7 ■ Processing and Managing Threats 131

c07.indd 09:44:3:AM 01/09/2014 Page 131

The Order of Mitigation

Going back to the example of threat modeling a home from the introduction, it’s easy and fun for security experts to focus on how attackers could defeat the security system by cutting the alarm wire. If you consider the window to be the attack surface, then threats include someone smashing through it and someone opening it. The smashing is addressed by re-enforced glass, which is thus “first-order” mitigation. The smashing threat is also addressed by an alarm, which is a second-order defense. But, oh no! Alarms can be defeated by cutting power. To address that third-level threat, the system designer can add more defenses. For example, alarm systems can include an alert if the line is ever dropped. Therefore, the defender can add a battery, or perhaps a cell phone or some other radio. (See how much fun this is?) These multiple layers, or orders, of attack and defense are shown in Table 7-2.

N O T E If you become obsessed with the window-smashing threat and forget to put a lock on the window, or you never discover that there’s a door key under the mat, you

have unaddressed problems, and are likely mis-investing your resources.

Table 7-2: Threat and Mitigation “Orders” or “Layers”

ORDER THREAT MITIGATION

1st Window smashing Reinforced glass

2nd Window smashing Alarm

3rd Cut alarm wire Heartbeat

4th Fake heartbeat Cryptographic signal integrity

Threat modeling should usually proceed from attack surfaces, and ensure that all fi rst-order threats are mitigated before attention is paid to the second- order threats. Even if a methodology is in place to ensure that the full range of fi rst-order threats is addressed, a team may run out of time to follow the methodology. Therefore, you should fi nd threats breadth-fi rst.

Playing Chess

It’s also important to think about what an attacker will do next, given your mitigations. Maybe that means following the path to faking a heartbeat on the alarm wire, but maybe the attacker will fi nd that door key, or maybe they’ll move on to another victim. Don’t think of attacks and mitigations as static. There’s a

www.it-ebooks.info

132 Part III ■ Managing and Addressing Threats

c07.indd 09:44:3:AM 01/09/2014 Page 132

dynamic interplay between them, usually driven by attackers. You might think of threats and mitigations like the black and white pieces on a chess board. The attacker can move, and when they move, their relationship to other pieces can change. As you design mitigation, ask what an attacker could do once you deliver that mitigation. How could they work around it? (This is subtly different from asking what they will do. As Nobel-prize winning physicist Niels Bohr said, “Prediction is very diffi cult, especially about the future.”)

Generally, attackers look for the weakest link they can easily fi nd. As you consider your threats and where to go into depth, start with the weakest links. This is an area where experience, including a repertoire of real scenarios, can be very helpful. If you don’t have that repertoire, a literature review can help, as you saw in Chapter 2. This interplay is a place where artful threat modeling and clever redesign can make a huge difference.

Believing that an attacker will stop because you put a mitigation in place is optimistic, or perhaps naive would be a better word. What happens several moves ahead can be important. (Attackers are tricky like that.) This differs from thinking through threat ordering in that your attacker will likely move to the “next easiest” attack available. That is, an attacker doesn’t need to stick to the attack you’re planning for or coding against at the moment, but rather can go anywhere in your system. The attacker gets to choose where to go, so you need to defend everywhere. This isn’t really fair, but no one promised you fair.

Prioritizing

You might feel that the advice in this section about the layers of mitigations and the suggestion to fi nd threats across fi rst is somewhat contradictory. Which should you do fi rst? Consider the chess game or cover everything? Covering breadth fi rst is probably wise. As you manage the bugs and select ways to mitigate threats, you can consider the chess game and deeper threat variants. However, it’s important to cover both.

The unfortunate reality is that attackers with enough interest in your technology will try to fi nd places where you didn’t have enough time to investigate or build defenses. Good requirements, along with their interplay with threats and mitigations, can help you create a target that is consistently hard to attack.

Running from the Bear

There’s an old joke about Alice and Bob hiking in the woods when they come across an angry bear. Alice takes off running, while Bob pauses to put on some running shoes. Alice stops and says, “What the heck are you doing?”

www.it-ebooks.info

Chapter 7 ■ Processing and Managing Threats 133

c07.indd 09:44:3:AM 01/09/2014 Page 133

Bob looks at her and replies, “I don’t need to outrun the bear, I just need to outrun you.”

OK, it’s not a very good joke. But it is a good metaphor for bad threat model- ing. You shouldn’t assume that there’s exactly one bear out there in the woods. There are a lot of people out there actively researching new vulnerabilities, and they publish information about not only the vulnerabilities they fi nd, but also about their tools and techniques. Thus, more vulnerabilities are being found more effi ciently than in past years. Therefore, not only are there multiple bears, but they have conferences in which they discuss techniques for eating both Alice and Bob for lunch.

Worse, many of today’s attacks are largely automated, and can be scaled up nearly infi nitely. It’s as if the bears have machine guns. Lastly, it will get far worse as the rise of social networking empowers automated social engineering attacks (just consider the possibilities when attackers start applying modern behavior-based advertising to the malware distribution business).

If your iteration ends with “we just have to run faster than the next target,” you may well be ending your analysis early.

Tracking with Tables and Lists

Threat modeling can lead you to generate a lot of information, and good tracking mechanisms can make a big difference. Discovering what works best for you may require a bit of experimentation. This section lays out some sample lists and sample entries in such lists. These are all intended as advice, not straight- jackets. If you regularly fi nd something that you’re writing on the side, give yourself a way to track it.

Tracking Threats

The fi rst type of table to consider is one that tracks threats. There are (at least) three major ways to organize such a table, including by diagram element (see Table 7-3), by threat type (see Table 7-4), or by order of discovery (see Table 7-5). Each of these tables uses these methods to examine threats against the super- simple diagram from Chapter 1, “Dive In and Threat Model!” reprised here in Figure 7-1.

If you organize by diagram element, the column headers are Diagram Element, Threat Type, Threat, and Bug ID/Title. You can check your work by validating that the expected threats are all present in the table. For example, if you’re using STRIDE-per-element, then you should have at least one tampering, information disclosure, and denial-of-service threat for each data fl ow. An example table for use in iterating over diagram elements is shown in Table 7-3.

www.it-ebooks.info

134 Part III ■ Managing and Addressing Threats

c07.indd 09:44:3:AM 01/09/2014 Page 134

Web browser Web server 1 2 3 4 5 6 7

Corporate data center Web storage (offsite)

Business Logic Database

Figure 7-1: The diagram considered in threat tables

Table 7-3: A Table of Threats, Indexed by Diagram Element (excerpt)

DIAGRAM ELEMENT

THREAT TYPE THREAT BUG ID AND TITLE

Data fl ow (4) web server to Biz Logic

Tampering Add orders without payment checks.

4553 “need integrity controls on channel”

Information disclosure

Payment instruments in the clear

4554 “need crypto” #PCI #p0

Denial of service

Can we just accept all these inside the data center?

4555 “Check with Alice in IT if these are acceptable”.

To check the completeness of Table 7-3, confi rm that each element has at least one threat. If you’re using STRIDE-per-element, each process should have six threats, each data fl ow three, each external entity two, and each data store three or four if the data store is a log.

You can also organize a table by threats, and use threats as what’s being iterated “across.” If you do that, you end up with a table like the one shown in Table 7-4.

Table 7-4: A Table of Threats, Organized by Threat (excerpt)

THREAT DIAGRAM ELEMENT THREAT BUG ID AND TITLE

Tampering Web browser (1) Attacker modifi es our JavaScript order checker.

4556 “Add order-checking logic to the server”.

Data fl ow (2) from browser to server

Failure to use HTTPS* 4557 “Build unit tests to ensure that there are no HTTP listeners for endpoints to these data fl ows.”

www.it-ebooks.info

Chapter 7 ■ Processing and Managing Threats 135

c07.indd 09:44:3:AM 01/09/2014 Page 135

THREAT DIAGRAM ELEMENT THREAT BUG ID AND TITLE

Web server Someone who tam- pers with the web server can attack our customers.

4558 “Ensure all changes to server code are pulled from source control so changes are accountable”.

Web server Web server can add items to orders.

4559 “Investigate moving con- trols to Biz Logic, which is less accessible to attackers”.

* The entry for “failure to use HTTPS” is an example that illustrates how knowledge of the scenario and mitigating techniques can lead you to jump to a fix, rather than perhaps tediously working through the threats. Be careful that when you (literally) jump to a conclusion like this, you’re not missing other threats.

N O T E You may have noticed that Table 7-4 has two entries for web server—this is totally fi ne. You shouldn’t let having something in a space prevent you from fi nding

more threats against that thing, or recording additional threats that you discover.

The last way to formulate a table is by order of discovery, as shown in Table 7-5. This is the easiest to fi ll out, as the next threat is simply added to the next line. It is, however, the hardest to validate, because the threats will be “jumbled together.”

Table 7-5: Threats by Order of Discovery (excerpt)

THREAT DIAGRAM ELEMENT THREAT BUG ID

Tampering Web browser (1) Attacker modifi es the JavaScript order checker.

4556 “Add order-checking logic to the server”.

With three variations, the obvious question is which one should you use? If you’re new to threat modeling and need structure, then either of the fi rst two forms help you organize your approach. The third is more natural, but requires checking at the end; so as you become more comfortable threat modeling, jumping around will become natural, and therefore the third type of table will become more useful.

Making Assumptions

The key reason to track assumptions as you discover them is so that you can follow up and ensure that you’re not assuming your way into a problem. To do that, you should track the following:

■ The assumption

■ The impact if it’s wrong

www.it-ebooks.info

136 Part III ■ Managing and Addressing Threats

c07.indd 09:44:3:AM 01/09/2014 Page 136

■ Who can tell you if it’s wrong

■ Who’s going to follow-up

■ When they need to do so

■ A bug for tracking

Table 7-6 shows an example entry for such a table of assumptions.

Table 7-6: A Table for Recording Assumptions

ASSUMPTION

IMPACT IF WRONG

IF NOT OBVIOUS

WHO TO TALK TO

IF KNOWN

WHO’S FOLLOWING UP

FOLLOWUP BY DATE BUG #

It’s OK to ignore denial of service within the data center.

Unhandled vulnerabilities

Alice Bob April 15 4555

External Security Notes

Many of the documented Microsoft threat-modeling approaches have a section for “external security notes.” That name frames these notes with respect to the threat model. That is, they’re notes for those outside the threat discovery process in some way, and they’ll probably emerge or crystalize as you look for threats. Therefore, like tracking threats and assumptions, you want to track external security notes. You can be clearer by framing the notes in terms of two sets of audiences: your customers and those calling your APIs. One of the most illus- trative forms of these notes appears in the IETF “RFC Security Considerations” section, so you’ll get a brief tour of those here.

Notes for Customers

Security notes that are designed for your customers or the people using your system are generally of the form “we can’t fi x problem X.” Not being able to fi x problem X may be acceptable, and it’s more likely to be acceptable if it’s not a surprise—for example, “This product is not designed to defend against the system administrator.” This sort of note is better framed as “non-requirements,” and they are discussed at length in Chapter 12, “Requirements Cookbook.”

www.it-ebooks.info

Chapter 7 ■ Processing and Managing Threats 137

c07.indd 09:44:3:AM 01/09/2014 Page 137

Notes for API Callers

The right design for an API involves many trade-offs, including utility, usability, and security. Threat modeling that leads to security notes for your callers can serve two functions. First, those notes can help you understand the security implications of your design decisions before you fi nalize those decisions. Second, they can help your customers understand those implications.

These notes address the question “What does someone calling your API need to do to use it in a secure way?” The notes help those callers know what threats you address (what security checks you perform), and thus you can tell them about a subset of the checks they’ll need to perform. (If your security depends on not telling them about threats then you have a problem; see the section on Kerckhoffs’s Principles in Chapter 16, “Threats to Cryptosystems.”) Notes to API callers are generally one of the following types:

■ Our threat model is [description]—That is, the things you worry about are… This should hopefully be obvious to readers of this book.

■ Our code will only accept input that looks like [some description]—What you’ll accept is simply that—a description of what validation you’ll perform, and thus what inputs you would reject. This is, at a surface level, at odds with the Internet robustness principle of “be conservative in what you send, and liberal in what you accept”; but being liberal does not require foolishness. Ideally, this description is also matched by a set of unit tests.

■ Common mistakes in using our code include [description]—This is a set of things you know callers should or should not do, and are two sides of the same coin:

■ We do not validate this property that callers might expect us to vali- date—In other words, what should callers check for themselves in their context, especially when they might expect you to have done something?

■ Common mistakes that our code doesn’t or can’t address—In other words, if you regularly see bug reports (or security issues) because your callers are not doing something, consider treating that as a design fl aw and fi xing it.

For an example of callers having trouble with an API, consider the strcpy function. According to the manual pages included with various fl avors of unix, “strcpy does not validate that s2 will fi t buffer s1,” or “strcpy requires that s1 be of at least length (s2) + 1.” These examples are carefully chosen, because as the fi ne manual continues, “Avoid using strcat.” (You should now use SafeStr* on Windows, strL* on unix.) The manual says this because, although notes

www.it-ebooks.info

138 Part III ■ Managing and Addressing Threats

c07.indd 09:44:3:AM 01/09/2014 Page 138

about safer use were eventually added, the function was simply too hard to use correctly, and no amount of notes to callers was going to overcome that. If your notes to those calling your API boil down to “it is impossible to use this API without jumping through error-prone hoops,” then your API is going to need to change (or deliver some outstanding business value).

Sometimes, however, it’s appropriate to use these sorts of notes to API callers— for example, “This API validates that the SignedDataBlob you pass in is signed by a valid root CA. You will still need to ensure that the OrganizationName fi eld matches the name in the URL, as you do not pass us the URL.” That’s a reasonable note, because the blob might not be associated with a URL. It might also be reasonable to have a ValidateSignatureURL() API call.

RFC Security Considerations

IETF RFCs are a form of external security notes, so it’s worth looking at them as an evolved example of what such notes might contain (Rescorla, 2003). If you need a more structured form of note, this framework is a good place to start. The security considerations of a modern RFC include discussion of the following:

■ What is in scope

■ What is out of scope—and why

■ Foreseeable threats that the protocol is susceptible to

■ Residual risk to users or operators

■ Threats the protocol protects against

■ Assumptions which underlie security

Scope is reasonably obvious, and the RFC contains interesting discussion about not arbitrarily defi ning either foreseeable threats or residual risk as out of scope. The point about residual risk is similar to non-requirements, as cov- ered in Chapter 12. It discusses those things that the protocol designers can’t address at their level.

Scenario-Specifi c Elements of Threat Modeling

There are a few scenarios where the same issues with threat modeling show up again and again. These scenarios include issues with customer/vendor bound- aries, threat modeling new technologies, and threat modeling an API (which is broader than just writing up external security notes). The customer/vendor trust boundary is dropped with unfortunate regularity, and how to approach

www.it-ebooks.info

Chapter 7 ■ Processing and Managing Threats 139

c07.indd 09:44:3:AM 01/09/2014 Page 139

an API or new technology often appears intimidating. The following sections address each scenario separately.

Customer/Vendor Trust Boundary

It is easy to assume that because someone is running your code, they trust you, and/or you can trust them. This can lead to things like Acme engineers saying, “Acme.com isn’t really an external entity... ” While this may be true, it may also be wrong. Your customers may have carefully audited the code they received from you. They may believe that your organization is generally trustworthy without wanting to expose their secrets to you. You holding those secrets is a different security posture. For example, if you hold backups of their cryp- tographic keys, they may be subject to an information disclosure threat via a subpoena or other legal demand that you can’t reveal to them. Good security design involves minimizing risk by appropriate design and enforcement of the customer/vendor trust boundary.

This applies to traditional programs such as installed software packages, and it also applies to the web. Believing that a web browser is faithfully executing the code you sent it is optimistic. The other end of an HTTPS connection might not even be a browser. If it is a browser, an attacker may have modifi ed your JavaScript, or be altering the data sent back to you via a proxy. It is important to pay attention to the trust boundary once your code has left your trust context.

New Technologies

From mobile to cloud to industrial control systems to the emergent “Internet of Things,” technologists are constantly creating new and exciting technologies. Sometimes these technologies genuinely involve new threat categories. More often, the same threats manifest themselves. Models of threats that are intended to elicit or organize thinking about skilled threat modelers (such as STRIDE in its mnemonic form) can help in threat modeling these new technologies. Such models of threats enable skilled practitioners to fi nd many of the threats that can occur even as the new technologies are being imagined.

As your threat elicitation technique moves from the abstract to the detailed, changes in the details of both the technologies and the threats may inhibit your ability to apply the technique.

From a threat-modeling perspective, the most important thing that design- ers of new technologies can do is clearly defi ne and communicate the trust relationships by drawing their trust boundaries. What’s essential is not just identifi cation, but also communication. For example, in the early web, the trust model was roughly as shown in Figure 7-2.

www.it-ebooks.info

140 Part III ■ Managing and Addressing Threats

c07.indd 09:44:3:AM 01/09/2014 Page 140

Web server

Browser context Server context

Origin 1

Origin 2

Web browser

Figure 7-2: The early web threat model

In that model, servers and clients both ran code, and what passed between them via HTTP was purely data in the form of HTML and images. (As a model, this is simplifi ed; early web browsers supported more than HTTP, including gopher and FTP.) However, the boundaries were clearly drawable. As the web evolved and web developers pushed the boundary of what was possible with the browser’s built-in functionality, a variety of ways for the server to run code on the client were added, including JavaScript, Java, ActiveX, and Flash. This was an active transformation of the security model, which now looks more like Figure 7-3.

Flash

Java

Active content

Web server

Browser context Server context

Origin 1

Origin 2

Web browser

Figure 7-3: The evolved web threat model

In this model, content from the server has dramatically more access to the browser, leading to two new categories of threats. One is intra-page threats, whereby code from different servers can attack other information in the browser. The other is escape threats, whereby code from the server can fi nd a way to infl uence what’s happening on the client computer. In and of themselves, these changes are neither good nor bad. The new technologies created a dramatic transforma- tion of what’s possible on the web for both web developers and web attackers. The transformation of the web would probably have been accomplished with more security if boundaries had been clearly identifi ed. Thus, those using new technology will get substantial security benefi ts from its designers by defi ning, communicating, and maintaining clear trust boundaries.

www.it-ebooks.info

Chapter 7 ■ Processing and Managing Threats 141

c07.indd 09:44:3:AM 01/09/2014 Page 141

Threat Modeling an API

API threat models are generally very similar. Each API has a low trust side, regardless of whether it is called from a program running on the local machine or called by anonymous parties on the far side of the Internet. On a local machine, the low trust side is more often clear: The program running as a normal user is running at a low trust level compared to the kernel. (It may also be running at the same trust level as other code running with the same user ID, but with the introduction of things like AppContainer on Windows and Mac OS sandbox, two programs running with the same UID may not be fully equivalent. Each should treat the other as being untrusted.) In situations where there’s a clear “lower trust” side, that unprivileged code has little to do, except ensure that data is validated for the context in which that data will be used. If it really is at a lower trust level, then it will have a hard time defending itself from a malicious kernel, and should generally not try. This applies only to relationships like that of a program to a kernel, where there’s a defi ned hierarchy of privilege. Across the Internet, each side must treat the other as untrusted. The “high trust” side of the API needs to do the following seven things for security:

■ Perform all security checks inside the trust boundary. A system has a trust boundary because the other side is untrusted or less trusted. To allow the less trusted side to perform security checks for you is missing the point of having a boundary. It is often useful to test input before sending it (for example, a user might fi ll out a long web form, miss something, and then get back an error message). However, that’s a usability feature, not a security feature. You must test inside the trust boundary. Additionally, for networked APIs/restful APIs/protocol endpoints, it is important to consider authentication, authorization, and revocation.

■ When reading, ensure that all data is copied in before validation for purpose (see next bullet). The low, or untrusted side, can’t be trusted to validate data. Nor can it be trusted to not change data under its control after you’ve checked it. There is an entire genus of security fl aws called TOCTOU (“time of check, time of use”) in which this pattern is violated. The data that you take from the low side needs to be copied into secured memory, validated for some purpose, and then used without further reference to the data on the low side.

■ Know what purpose the data will be put to, and validate that it matches what the rest of your system expects from it. Knowing what the data will be used for enables you to check it for that purpose—for example, ensure an IPv4 address is four octets, or that an e-mail address matches some regular expression (an e-mail regular expression is an easily grasped example, but sending e-mail to the address is better [Celis, 2006]). If the

www.it-ebooks.info

142 Part III ■ Managing and Addressing Threats

c07.indd 09:44:3:AM 01/09/2014 Page 142

API is a pass-through (for example, listening on a socket), then you may be restricted to validating length, length to a C-style string, or perhaps nothing at all. In that case, your documentation should be very clear that you are doing minimal or no validation, and callers should be cautious.

■ Ensure that your messages enable troubleshooting without giving away secrets. Trusted code will often know things that untrusted code should not. You need to balance the capability to debug problems with return- ing too much data. For example, if you have a database connection, you might want to return an error like “can’t connect to server instance with username dba password dfug90845b4j,” but anyone who connected then now knows the DBA’s password. Oops! Messages of the form “An error of type X occurred. This is instance Y in the error log Z” are helpful enough to a systems administrator, who can search for Y in Z while only disclosing the existence of the logs to the attacker. Even better are errors that include information about who to contact. Messages that merely say “Contact your system administrator” are deeply frustrating.

■ Document what security checks you perform, and the checks you expect callers to perform for themselves. Very few APIs take unconstrained input. The HTTP interface is a web interface: It expects a verb (GET, POST, HEAD) and data. For a GET or HEAD, the data is a URL, an HTTP version, and a set of HTTP headers. Old-fashioned CGI programs knew that the web server would pass them a set of headers as environment variables and then a set of name-value pairs. The environment variables were not always unique, leading to a number of bugs. What a CGI could rely on could be documented, but the diverse set of attacks and assumptions that people could read into it was not documentable.

■ Ensure that any cryptographic function runs in a constant time. All your crypto functions should run in constant time from the perspective of the low trust side. It should be obvious that cryptographic keys (except the public portion of asymmetric systems) are a critical subset of the things you should not expose to low. Crypto keys are usually both a stepping stone asset and a thing you want to protect asset. See also Chapter 16 on threats to cryptosystems.

■ Handle the unique security demands of your API. While the preceding issues show up with great consistency, you’re hopefully building a new API to deliver new value, and that API may also bring new risks that you should consider. Sometimes it’s useful to use the “rogue insider” model to help ask “what could we do wrong with this?”

In addition to the preceding checklist, it may be helpful to look to similar or competitive APIs and see what security changes they’ve executed, although the security changes may not be documented as such.

www.it-ebooks.info

Chapter 7 ■ Processing and Managing Threats 143

c07.indd 09:44:3:AM 01/09/2014 Page 143

Summary

There are a set of tools and techniques that you can use to help threat modeling fi t with other development, architecture, or technology deployments. Threat modeling tasks that use these tools happen at the start of a project, as you’re working through features, and as you’re close to delivery.

Threat modeling should start with the creation of a software diagram (or updating the diagram from the previous release). It should end with a set of security bugs being fi led, so that the normal development process picks up and manages those bugs.

When you’re creating the diagram, start with the broadest description you can, and add details as appropriate. Work top down, and as you do, at each level of the diagram(s), work across something: trust boundaries, software elements or your threat discovery technique.

As you look to create mitigations, be aware that attackers may try to bypass those mitigations. You want to mitigate the most accessible (aka “fi rst order”) threats fi rst, and then mitigate attacks against your mitigations. You have to consider your threats and mitigations not as a static environment, but as a game where the attacker can move pieces, and possibly cheat.

As you go through these analyses, you’ll want to track discoveries, includ- ing threats, assumptions, and things your customers need to know. Customers here include your customers, who need to understand what your goals and non-goals are, and API callers, who need to understand what security checks you perform, and what checks they need to perform.

There are some scenario-specifi c call outs: It is important to respect the customer/vendor security boundary; new technologies can and should be threat modeled, especially with respect to all the trust boundaries, not just the customer/vendor one; all APIs have very similar threat models, although there may be new and interesting security properties of your new and interesting API.

www.it-ebooks.info

c07.indd 09:44:3:AM 01/09/2014 Page 144

www.it-ebooks.info

145

c08.indd 12:26:46:PM 01/13/2014 Page 145

So far you’ve learned to model your software using diagrams and learned to fi nd threats using STRIDE, attack trees, and attack libraries. The next step in the threat modeling process is to address every threat you’ve found.

When it works, the fastest and easiest way to address threats is through technology-level implementations of defensive patterns or features. This chapter covers the standard tactics and technologies that you will use to mitigate threats. These are often operating system or program features that you can confi gure, activate, apply or otherwise rapidly engage to defend against one or more threats. Sometimes, they involve additional code that is widely available and designed to quickly plug in. (For example, tunneling connections over SSH to add security is widely supported, and some unix packages even have options to make that easier.)

Because you likely found your threats via STRIDE, the bulk of this chapter is organized according to STRIDE. The main part of the chapter addresses STRIDE and privacy threats, because most pattern collections already include information about how to address the threats.

Tactics and Technologies for Mitigating Threats

The mitigation tactics and technologies in this chapter are organized by STRIDE because that’s most likely how you found them. This section is therefore orga- nized by ways to mitigate each of the STRIDE threats, each of which includes

C H A P T E R

8

Defensive Tactics and Technologies

www.it-ebooks.info

146 Part III ■ Managing and Addressing Threats

c08.indd 12:26:46:PM 01/13/2014 Page 146

a brief recap of the threat, tactics that can be brought to bear against it, and the techniques for accomplishing that by people with various skills and respon- sibilities. For example, if you’re a developer who wants to add cryptographic authentication to address spoofi ng, the techniques you use are different from those used by a systems administrator. Each subsection ends with a list of specifi c technologies.

Authentication: Mitigating Spoofi ng

Spoofi ng threats against code come in a number of forms: faking the program on disk, squatting a port (IP, RPC, etc.), splicing a port, spoofi ng a remote machine, or faking the program in memory (related problems with libraries and depen- dencies are covered under tampering); but in general, only programs running at the same or a lower level of trust are spoofable, and you should endeavor to trust only code running at a higher level of trust, such as in the OS.

There is also spoofi ng of people, of course, a big, complex subject covered in Chapter 14, “Accounts and Identity.” Mitigating spoofi ng threats often requires unusually tight integration between layers of systems. For example, a mainte- nance engineer from Acme, Inc. might want remote (or even local) access to your database. Is it enough to know that the person is an employee of Acme? Is it enough to know that he or she can initiate a connection from Acme’s domain? You might reasonably want to create an account on your database to allow Joe Engineer to log in to it, but how do you bind that to Acme’s employee database? When Joe leaves Acme and gets a job at Evil Geniuses for a Better Tomorrow, what causes his access to Acme’s database to go away?

N O T E Authentication and authorization are related concepts, and sometimes confused. Knowing that someone really is Adam Shostack should not authorize a

bank to take money from my account (there are several people of that name in the

U.S.). Addressing authorization is covered in the Authorization: Mitigating Elevation

of Privilege

From here, let’s dig into the specifi c ways in which you can ensure authen- tication is done well.

Tactics for Authentication

You can authenticate a remote machine either with or without cryptographic trust mechanisms. Without crypto involves verifying via IP or “classic” DNS entries. All the noncryptographic methods are unreliable. Before they existed, there were attempts to make hostnames reliable, such as the double-reverse DNS lookup. At the time, this was sometimes the best tactic for authentication.

www.it-ebooks.info

Chapter 8 ■ Defensive Tactics and Technologies 147

c08.indd 12:26:46:PM 01/13/2014 Page 147

Today, you can do better, and there’s rarely an excuse for doing worse. (SNMP may be an excuse, and very small devices may be another). As mentioned earlier, authenticating a person is a complex subject, covered in Chapter 14. Authenticating on-system entities is somewhat operating system dependent.

Whatever the underlying technical mechanisms are, at some point crypto- graphic keys are being managed to ensure that there’s a correspondence between technical names and names people use. That validation cannot be delegated entirely to machines. You can choose to delegate it to one of the many compa- nies that assert they validate these things. These companies often do business as “PKI” or “public key infrastructure” companies, and are often referred to as “certifi cation authorities” or “CAs”. You should be careful about relying on that delegation for any transaction valued at more than what the company will accept for liability. (In most cases, certifi cate authorities limit their liability to nothing). Why you should assign it a higher value is a question their market- ing departments hope will not be asked, but the answer roughly boils down to convenience, limited alternatives, and accepted business practice.

Developer Ways to Address Spoofi ng

Within an operating system, you should aim to use full and canonical path names for libraries, pipes, and so on to help mitigate spoofi ng. If you are relying on something being protected by the operating system, ensure that the permissions do what you expect. (In particular, unix fi les in /tmp are generally unreliable, and Windows historically has had similarly shared directories.) For networked systems in a single trust domain, using operating system mechanisms such as Active Directory or LDAP makes sense. If the system spans multiple trust domains, you might use persistence or a PKI. If the domains change only rarely, it may be appropriate to manually cross-validate keys, or to use a contract to specify who owns what risks.

You can also use cryptographic ways to address spoofi ng, and these are covered in Chapter 16, “Threats to Cryptosystems.” Essentially, you tie a key to a person, and then work to authenticate that the key is correctly associated with the person who’s connecting or authenticating.

Operational Ways to Address Spoofi ng

Once a system is built, a systems administrator has limited options for improv- ing spoofi ng defenses. To the extent that the system is internal, pressure can be brought to bear on system developers to improve authentication. It may also be possible to use DNSSEC, SSH, or SSL tunneling to add or improve authentication. Some network providers will fi lter outbound traffi c to make spoofi ng harder. That’s helpful, but you cannot rely on it.

www.it-ebooks.info

148 Part III ■ Managing and Addressing Threats

c08.indd 12:26:46:PM 01/13/2014 Page 148

Authentication Technologies

Technologies for authenticating computers (or computer accounts) include the following:

■ IPSec

■ DNSSEC

■ SSH host keys

■ Kerberos authentication

■ HTTP Digest or Basic authentication

■ “Windows authentication” (NTLM)

■ PKI systems, such as SSL or TLS with certifi cates

Technologies for authenticating bits (fi les, messages, etc.) include the following:

■ Digital signatures

■ Hashes

Methods for authenticating people can involve any of the following:

■ Something you know, such as a password

■ Something you have, such as an access card

■ Something you are, such as a biometric, including photographs

■ Someone you know who can authenticate you

Technologies for maintaining authentication across connections include the following:

■ Cookies

Maintaining authentication across connections is a common issue as you integrate systems. The cookie pattern has fl aws, but generally, it has fewer fl aws than re-authenticating with passwords.

Integrity: Mitigating Tampering

Tampering threats come in several fl avors, including tampering with bits on disk, bits on a network, and bits in memory. Of course, no one is limited to tampering with a single bit at a time.

www.it-ebooks.info

Chapter 8 ■ Defensive Tactics and Technologies 149

c08.indd 12:26:46:PM 01/13/2014 Page 149

Tactics for Integrity

There are three main ways to address tampering threats: relying on system defenses such as permissions, use of cryptographic mechanisms, and use of logging technology and audit activities as a deterrent.

Permission mechanisms can protect things that are within their scope of control, such as fi les on disk, data in a database, or paths within a web server. Examples of such permissions include ACLs on Windows, unix fi le permissions, or .htaccess fi les on a web server.

There are two main cryptographic primitives for integrity: hashes and signatures. A hash takes an input of some arbitrary length, and produces a fi xed- length digest or hash of the input. Ideally, any change to the input completely transforms the output. If you store a protected hash of a digital object, you can later detect tampering. Actually, anyone with that hash can detect tampering, so, for example, many software projects list a hash of the software on their website. Anyone who gets the bits from any source can rely on them being the bits described on the project website, to a level of security based on the security of the hash and the operation of the web site. A signature is a cryptographic operation with a private key and a hash that does much the same thing. It has the advantage that once someone has obtained the right public key, they can validate a lot of hashes. Hashes can also be used in binary trees of various forms, where large sets of hashes are collected together and signed. This can enable, for example, inserting data into a tree and noting the time in a way that’s hard to alter. There are also systems for using hashes and signatures to detect changes to a fi le system. The fi rst was co-invented by Gene Kim, and later commercial- ized by Tripwire, Inc. (Kim, 1994).

Logging technology is a weak third in this list. If you log how fi les change, you may be able to recover from integrity failures.

Implementing Integrity

If you’re implementing a permission system, you should ensure that there’s a single permissions kernel, also called a reference monitor. That reference monitor should be the one place that checks all permissions for everything. This has two main advantages. First, you have a single monitor, so there are no bugs, synchronization failures, or other issues based on which code path called. Second, you only have to fi x bugs in one place.

Creating a good reference monitor is a fairly intricate bit of work. It’s hard to get right, and easy to get wrong. For example, it’s easy to run checks on references

www.it-ebooks.info

150 Part III ■ Managing and Addressing Threats

c08.indd 12:26:46:PM 01/13/2014 Page 150

(such as symlinks) that can change when the code fi nally opens the fi le. If you need to implement a reference monitor, perform a literature review fi rst.

If you’re implementing a cryptographic defense, see Chapter 16. If you’re implementing an auditing system, you need to ensure it is suffi ciently perfor- mant that people will leave it on, that security successes and failures are both logged, and that there’s a usable way to access the logs. You also need to ensure that the data is protected from attackers. Ideally, this involves moving it off the generating system to an isolated logging system.

Operational Assurance of Integrity

The most important element of assuring integrity is about process, not technology. Mechanisms for ensuring integrity only work to the extent that integrity failures generate operational exceptions or interruptions that are addressed by a person. All the cryptographic signatures in the world only help if someone investigates the failure, or if the user cannot or does not override the message about a failure. You can devote all your disk access operations to running checksums, but if no one investigates the alarms, they won’t do any good. Some systems use “whitelists” of applications so only code on the whitelist runs. That reduces risk, but carries an operational cost.

It may be possible to use SSH or SSL tunneling or IPSec to address network tampering issues. Systems like Tripwire, OSSEC, or L5 can help with system integrity.

Integrity Technologies

Technologies for protecting fi les include:

■ ACLs or permissions

■ Digital signatures

■ Hashes

■ Windows Mandatory Integrity Control (MIC) feature

■ Unix immutable bits

Technologies for protecting network traffi c:

■ SSL

■ SSH

■ IPSec

■ Digital signatures

Non-Repudiation: Mitigating Repudiation

Repudiation is a somewhat different threat because it bridges the business realm, in which there are four elements to addressing it: preventing fraudulent

www.it-ebooks.info

Chapter 8 ■ Defensive Tactics and Technologies 151

c08.indd 12:26:46:PM 01/13/2014 Page 151

transactions, taking note of contested issues, investigating them, and respond- ing to them. In an age when anyone can instantly be a publisher, assuming that you can ignore the possibility of a customer (or noncustomer) complaint or contested charge is foolish. Ensuring you can accept customer complaints and investigate them is outside the scope of this book, but the output from such a system provides a key validation that you have the right logs.

Note that repudiation is sometimes a feature. As Professor Ian Goldberg pointed out when introducing his Off-the-Record messaging protocol, signed conversations can be embarrassing, incriminating, or otherwise undesirable (Goldberg, 2008). Two features of the Off-the-Record (OTR) messaging system are that it’s secure (encrypted and authenticated) and deniable. This duality of feature or threat also comes up in the LINDDUN approach to privacy threat modeling.

Tactics for Non-Repudiation

The technical elements of addressing repudiation are fraud prevention, logs, and cryptography. Fraud prevention is sometimes considered outside the scope of repudiation. It’s included here because managing repudiation is easier if you have fewer contested transactions. Fraud prevention can be divided into fraud by internal actors (embezzlement and the like) and external fraud. Internal fraud prevention is a complex matter; for a full treatment see The Corporate Fraud Handbook (Wells, 2011). You should have good account management practices, including ensuring that your tools work well enough that people are not tempted or forced to share passwords as part of getting their jobs done. Be sure you log and audit the data in those logs.

Logs are the traditional technical core of addressing repudiation issues. What is logged depends on the transaction, but generally includes signatures or an IP address and all related information. There are also cryptographic ways to address repudiation, which are currently mostly used between larger businesses.

Tactics for Preventing Fraud by External Parties

External fraud prevention can be seen as a matter of payment fraud prevention, and ensuring that your customers remain in control of their account. In both cases, details about the state of the art changes quickly, so talk to your peers. Even the most tight-lipped companies have been willing to have very frank discussions with peers under NDA.

In essence, stability is good. For example, someone who has been buying two romance novels a month from you for a decade and is still living at the same address is likely the person who just ordered another one. If that person suddenly moves to the other side of the world, and orders technical books in Slovakian with a new credit card with a billing address in the Philippines, you

www.it-ebooks.info

152 Part III ■ Managing and Addressing Threats

c08.indd 12:26:46:PM 01/13/2014 Page 152

might have a problem. (Then again, they might have fi nally found true love, and you don’t want to upset your loyal customers.)

Tools for Preventing Fraud by External Parties

In their annual report on online fraud, CyberSource includes a survey of popular fraud detection tools and their perceived effectiveness (CyberSource, 2013). Their 2013 survey includes a set of automated tools:

■ Validation services

■ Proprietary data/customer history

■ Multi-merchant data

■ Purchase device tracing

Validation services include tracking verifi cation numbers (aka CVN/CVV), address verifi cation services, postal address verifi cation, Verifi ed by Visa/ MasterCard SecureCode, telephone number verifi cation/reverse lookups, public records services, credit checks, and “out-of-wallet/in-wallet” verifi cation services.

Proprietary data and customer history includes customer order history, in house “negative lists” of problematic customers, “positive lists” of VIP or reliable customers, order velocity monitoring, company-specifi c fraud models (these are usually built with manual, statistical, or machine learning analyses of past fraudulent orders), and customer website behavioral analysis.

Multi-merchant data focuses on shared negative lists or multi-merchant purchase velocity analyzed by the merchant. (This analysis is nominally also performed by the card processors and clearing houses, so the additional value may be transient.)

Finally, purchase device tracking includes device “fi ngerprinting” and IP address geolocation. The CyberSource report also discusses the importance of tools to help manual review, and how a varied list is both very helpful and time consuming. Because manual review is one of the most expensive components of an anti-fraud approach to repudiation threats, it may be worth investing in tools to gather all the data into one (or at least fewer) places to improve analyst productivity.

Implementing Non-Repudiation

The two key tools for non-repudiation are logging and digital signatures. Digital signatures are probably most useful for business-to-business systems.

Log as much as you can keep for as long as you need to keep it. As the price of storage continues to fall, this advice becomes easier and easier to follow. For example, with a web transaction, you might log IP address, current geoloca- tion of that address, and browser details. You might also consider services that

www.it-ebooks.info

Chapter 8 ■ Defensive Tactics and Technologies 153

c08.indd 12:26:46:PM 01/13/2014 Page 153

either provide information on fraud or allow you to request decision advice. To the extent that these companies specialize, and may have broader visibility into fraud, this may be a good area of security to outsource. Some of the information you log or transfer may interact with your privacy policies, and it’s important to check.

There are also cryptographic digital signatures. Digital signature should be distinguished from electronic signature, which is a term of art under U.S. law referring to a variety of mechanisms with which to produce a signature, some as minimalistic as “press 1 to agree to these terms and conditions.” In contrast, a digital signature is a mathematical transformation that demonstrates irrefut- ably that someone in possession of a mathematical key took an action to cause a signature to be made. The strength of “irrefutable” here depends on the strength of the math, and the tricky bits are possession of the key and what human intent (if any) may have lain behind the signature.

Operational Assurance of Non-Repudiation

When a customer or partner attempts to repudiate a transaction, someone needs to investigate it. If repudiation attempts are frequent, you may need dedicated people, and those people might require specialized tools.

Non-Repudiation Technologies

Technologies you can use to address repudiation include:

■ Logging

■ Log analysis tools

■ Secured log storage

■ Digital signatures

■ Secure time stamps

■ Trusted third parties

■ Hash trees

■ As mentioned in “tools for preventing fraud” above

Confi dentiality: Mitigating Information Disclosure

Information disclosure can happen with information at rest (in storage) or in motion (over a network). The information disclosed can range from the con- tent of communication to the existence of an entity with which someone is communicating.

www.it-ebooks.info

154 Part III ■ Managing and Addressing Threats

c08.indd 12:26:46:PM 01/13/2014 Page 154

Tactics for Confi dentiality

Much like with integrity, there are two main ways to prevent information dis- closure: Within the confi nes of a system, you can use ACLs, and outside of it you must use cryptography.

If what must be protected is the content of the communication, then traditional cryptography will be suffi cient. If you need to hide who is communicating with whom and how often, you’ll need a system that protects that data, such as a cryptographic mix or onion network. If you must hide the fact that communica- tion is taking place at all, steganography will be required.

Implementing Confi dentiality

If your system can act as a reference monitor and control all access to the data, you can use a permissions system. Otherwise, you’ll need to encrypt either the data or its “container.” The data might be a fi le on disk, a record in a database, or an e-mail message as it transits over the network. The container might be a fi le system, database, or network channel, such as all e-mail between two systems, or all packets between a web client and a web server.

In each cryptographic case, you have to consider who needs access to the keys for encrypting and the decrypting data. For fi le encryption, that might be as simple as asking the operating system to securely store the key for the user so that the user can get to it later. Also, note that encrypted data is not integrity controlled. The details can be complex and tricky, but consider a database of salaries, where the cells are encrypted. You don’t need to know the CEO’s sal- ary to know that replacing your salary with it is likely a good thing (for you); and if there’s no integrity control, replacing the encrypted value of your salary with the CEO’s salary will do just fi ne.

An important subset of information disclosure cases related to the storage of passwords or backup authentication mechanisms is considered in depth in Chapter 14.

Operational Assurance of Confi dentiality

It may be possible to add ACLs to an already developed system, or to use chroot or similar sandboxes to restrict what it can access. On Windows, the addition of a SID to a program and an inherited deny ACL for that SID may help (or it may break things). It is usually possible to add a disk or fi le encryption layer to protect information at rest from disclosure. Disk crypto will work “by default” with all the usual caveats about how keys are managed. It works for adversarial custody of the machine, but not if the password is written down or otherwise stored with the machine. With regard to a network, it may be possible to use SSH or SSL tunneling or IPSec to address network tampering issues.

www.it-ebooks.info

Chapter 8 ■ Defensive Tactics and Technologies 155

c08.indd 12:26:46:PM 01/13/2014 Page 155

Confi dentiality Technologies

Technologies for confi dentiality include:

■ Protecting fi les:

■ ACLs/permissions

■ Encryption

■ Appropriate key management

■ Protecting network data:

■ Encryption

■ Appropriate key management

■ Protecting communication headers or the fact of communication:

■ Mix networks

■ Onion routing

■ Steganography

N O T E In the preceding lists, “appropriate key management” is not quite a technology, but is so important that it’s included.

Availability: Mitigating Denial of Service

Denial-of-service attacks work by exhausting some resource. Traditionally, those resources are CPU, memory (both RAM and hard drive space can be exhausted), and bandwidth. Denial-of-service attacks can also exhaust human availability. Consider trying to call the reservations line of a very exclusive restaurant—the French Laundry in Napa Valley books all its tables within 5 minutes of the phone being open every day (for a day 30 days in the future). The resource under contention is the phone lines, and in particular the people answering them.

Tactics for Availability

There are two forms of denial of service attacks: brute force and clever. Using the restaurant example, brute force involves bringing 100 people to a restaurant that can seat only 25. Clever attacks bring 20 people, each of whom makes an ever-escalating list of requests and changes, and runs the staff ragged. In the online world, brute force attacks on networks are somewhat common under the name DDoS (Distributed Denial of Service). They can also be carried out against CPU (for example, while(1) fork()) or disk. It’s simple to construct a small zip

www.it-ebooks.info

156 Part III ■ Managing and Addressing Threats

c08.indd 12:26:46:PM 01/13/2014 Page 156

fi le that will expand to whatever limit might be in place: the maximum size of a fi le or space on the fi le system. Recall that a zip fi le is structured to describe the contents of the real fi le as simply as possible, such as 65,535 0s. That three-byte description will expand to 64K, for a magnifi cation effect of over 21,000—which is awfully cool if you’re an attacker.

Clever denial-of-service attacks involve a small amount of work by an attacker that causes you to do a lot of work. For example, connecting to an SSL v2 server, the client sends a client master key challenge, which is a random key encrypted such that the server does (relatively) expensive public key operations to decrypt it. The client does very little work compared to the server. This can be partially addressed in a variety of ways, most notably the Photuris key management protocol. The core of such protocols is proof that the client has done more work than the server, and the body of approaches is called proof of work. However, in a world of abundant bots and volunteers to run DDoS software for political causes, Ben Laurie and Richard Clayton have shown reasonably conclusively that “Proof-of-Work Proves Not to Work” (in a paper of that name [Laurie]).

A second important strategy for defending against denial-of-service attacks is to ensure your attacker can receive data from you. For example, defenses against SYN fl ooding attacks now take this form. In a SYN fl ood attack, a host receives a lot of connection attempts (TCP SYNchronize) and it needs to keep track of each one to set up new connections. By sending a slew of those, operat- ing systems in the 1990s could be run out of memory in the fi xed-size buffers allocated to track SYNs, and no new connections could be established. Modern TCP stacks calculate certain parts of their response to a SYN packet using some cryptography. They maintain no state for incoming packets, and use the cryptographic tools to validate that new connections are real (Rescorla, 2003).

Implementing Availability

If you’re implementing a system, consider what resources an attacker might consume, and look for ways to limit those resources on a per-user basis. Understand that there are limits to what you can achieve when dealing with systems on the other side of a trust boundary, and some of the response needs to be operational. Ensure that the operators have such mechanisms.

Operational Assurance of Availability

Addressing brute force denial-of-service attacks is simple: Acquire more resources such that they don’t run out, or apply limits so that one bad apple can’t spoil things for others. For example, multi-user operating systems implement quota systems, and business ISPs may be able to fi lter traffi c coming from certain sources.

www.it-ebooks.info

Chapter 8 ■ Defensive Tactics and Technologies 157

c08.indd 12:26:46:PM 01/13/2014 Page 157

Addressing clever attacks is generally in the realm of implementation, not operations.

Availability Technologies

Technologies for protecting fi les include:

■ ACLs

■ Filters

■ Quotas (rate limiting, thresholding, throttling)

■ High-availability design

■ Extra bandwidth (rate limiting, throttling)

■ Cloud services

Authorization: Mitigating Elevation of Privilege

Elevation of privilege threats are one category of unauthorized use, and the only one addressed in this section. The overall question of designing authorization systems fi lls other books.

Tactics for Authorization

As discussed in the section “Implementing Integrity,” having a reference moni- tor that can control access between objects is a precursor to avoiding several forms of a problem, including elevation of privilege. Limiting the attack surface makes the problem more tractable. For example, limiting the number of setuid programs limits the opportunity for a local user to become root. (Technically, programs can be setuid to something other than root, but generally those other accounts are also privileged.) Each program should do a small number of things, and carefully manage their input, including user input, environment, and so on. Each should be sandboxed to the extent that the system supports it. Ensure that you have layers of defense, such that an anonymous Internet user can’t elevate to administrator with a single bug. You can do this by having the code that listens on the network run as a limited user. An attacker who exploits a bug will not have complete run of the system. (If they’re a normal user, they may well have easy access to many elevation paths, so lock down the account.)

The permission system needs to be comprehensible, both to administrators trying to check things and to people trying to set things. A permission system that’s hard to use often results in people incorrectly setting permissions, (tech- nically) enabling actions that policy and intent mean to forbid.

www.it-ebooks.info

158 Part III ■ Managing and Addressing Threats

c08.indd 12:26:46:PM 01/13/2014 Page 158

Implementing Authorization

Having limited the attack surface, you’ll need to very carefully manage the input you accept at each point on the attack surface. Ensure that you know what you want to accept and how you’re going to use that input. Reject anything that doesn’t match, rather than trying to make a complete list of bad characters. Also, if you get a non-match, reject it, rather than try to clean it up.

Operational Assurance of Authorization

Operational details, such as “we need to expose this to the Internet” can often lead to those deploying technology wanting to improve their defensive stance. This usually involves adding what can be referred to as defense in depth or layered defense. There are several ways to do this.

First, run as a normal or limited user, not as administrator/root. While techni- cally that’s not a mitigation against an elevation-of-privilege threat, but a har- binger of such, it’s inline with the “principle of least privilege.” Each program should run as its own limited user. When unix made “nobody” the default account for services, the nobody account ended up with tremendous levels of authorization. Second, apply all the sandboxing you can.

Authorization Technologies

Technologies for improving authorization include:

■ ACLs

■ Group or role membership

■ Role based access control

■ Claims-based access control

■ Windows privileges (runas)

■ Unix sudo

■ Chroot, AppArmor or other unix sandboxes

■ The “MOICE” Windows sandbox pattern

■ Input validation for a defi ned purpose

N O T E MOICE is the “Microsoft Offi ce Isolated Conversion Environment.” The name comes from the problem that led to the pattern being invented, but the approach can

now be considered a pattern for sandboxing on Windows. For more on MOICE, see

(LeBlanc, 2007).

www.it-ebooks.info

Chapter 8 ■ Defensive Tactics and Technologies 159

c08.indd 12:26:46:PM 01/13/2014 Page 159

N O T E Many Windows privileges are functionally equivalent to administrator, and may not be as helpful as you desire. See (Margosis, 2006) for more details.

Tactic and Technology Traps

There are two places where it’s easy to get pulled into wasting time when working through these technologies and tactics. The fi rst distraction is risk management. The tactics and technologies in this chapter aren’t the only ways to address threats, but they are the best place to start. When you can use them, they will be easier to implement and work better than more complex or nuanced risk management approaches. For example, if you can address a threat by chang- ing a network endpoint to a local endpoint, there’s no point to engaging in the more time consuming risk management approaches covered in the next chapter. The second distraction is trying to categorize threats. If you found a threat via brainstorming or just the free fl ow of ideas, don’t let the organization of this chapter fool you into thinking you should try to categorize that threat. Instead, focus on fi nding the best way to address it. (Teams can spend longer in debate around categorization than it would take to implement the fi x they identifi ed—changing permissions on a fi le.)

Addressing Threats with Patterns

In his book, A Pattern Language, architect Christopher Alexander and his col- leagues introduced the concept of architectural patterns (Alexander, 1977). A pattern is a way of expressing how experts capture ways of solving recurring problems. Patterns have since been adapted to software. There are well-understood development patterns, such as the three-tier enterprise app.

Security patterns seem like a natural way to group and communicate about tactics and technologies to address security problems into something larger. You can create and distribute patterns in a variety of ways, and this section discusses some of them. However, in practice, these patterns have not been popular. The reasons for this are not clear, and those investing in using patterns to address security problems would likely benefi t from studying the factors that have limited their popularity.

Some of those factors might include engineers not knowing when to reach for such a text, or the presentation of security patterns as a distinct subset, apart from other patterns. At least one web patterns book (Van Duyne, 2007) includes a chapter on security patterns. Embedding security patterns where non-specialists are likely to fi nd them seems like a good pattern.

www.it-ebooks.info

160 Part III ■ Managing and Addressing Threats

c08.indd 12:26:46:PM 01/13/2014 Page 160

Standard Deployments

In many larger organizations, an operations group will have a standard way to deploy systems, or possibly several standard ways, depending on the data’s sensitivity. In these cases, the operations group can document what sorts of threats their standard deployment mitigates, and provide that document as part of their “on-boarding” process. For example, a standard data center at an organization might include defenses against DDoS, or state that “network information disclosure is an accepted risk for risk categories 1–3.”

Addressing CAPEC Threats

CAPEC (MITRE’s Common Attack Pattern Enumeration and Classifi cation) is primarily a collection of attack patterns, but most CAPEC threat patterns include defenses. This chapter has primarily organized threats according to STRIDE. If you are using CAPEC, each CAPEC pattern includes advice about how to address it in its “Solutions and Mitigations” section. The CAPEC website is the authoritative source for such data.

Mitigating Privacy Threats

There are essentially three ways to address privacy threats: Avoid collecting information (minimization), use crypto in various clever ways, and control how data is used (compliance and policy). Cryptography is a technology, while minimization and compliance are more tactics you can apply. Each requires effort to integrate into your design or implementation.

Minimization

Perhaps obviously, it is impossible to use information you don’t have in a way that impacts someone’s privacy. Therefore, minimizing your collection and reten- tion of information reduces risk. Minimizing what you collect is by far more reliable than attempting to use policy controls on the data. Of course, it also eliminates any utility that you can get from that data. As such, minimization is generally a business call regarding risk and reward. Over the past decade, with breach disclosure laws, the balance of factors related to decisions about the collection and retention of information have changed dramatically. Some legal scholars have gone so far as to compare personal data to toxic waste. Holly

www.it-ebooks.info

Chapter 8 ■ Defensive Tactics and Technologies 161

c08.indd 12:26:46:PM 01/13/2014 Page 161

Towle, an attorney specializing in electronic commerce, offers 10 principles for handling toxic waste, or personally identifying information (PII). Each of these is addressed in depth in her article (Towle, 2009):

■ Do not touch it unless you have to.

■ If you have to touch it, learn how or whether to do so—mistakes can be fatal or at least seriously damaging.

■ Do not use normal methods to transport (transfer) it.

■ Attempt to crack the whip over contractor handling it.

■ Do not store some of it at all.

■ Store what you need but in a manner avoiding spills, and limit access.

■ Be alert for suspicious odors and other red fl ags.

■ Report spills to the relevant people and agencies.

■ Dispose of it only by special means.

■ Get ready to be sued or incur often unreasonable expenses no matter how much care you take.

Minimization is a conceptually simple way to address privacy. In practice, however, it can become complex and contentious. The value of collecting data is easy to see, and it’s hard to know what you’ll be unable to do if you don’t collect it.

Cryptography

There are a variety of ways to use cryptographic techniques to address privacy concerns. The applicability of each is dependent on the threat model, in the sense of who you’re worried about. Each of these techniques is the subject of a great deal of research, so rather than try to provide a full description of each technique and risk leaving out key details, the following sections explain where each is useful as a response to a threat.

Hashing or Encrypting Data

If your privacy concern is someone accidentally viewing data, or running simple database queries, it may help to encrypt the data. If you want a record that can only be accessed once someone has a specifi c string (such as an e-mail address or SSN), you can use a cryptographic hash of that data. For example, if you store hash([email protected]), then only someone who knows that e-mail address can look it up.

www.it-ebooks.info

162 Part III ■ Managing and Addressing Threats

c08.indd 12:26:46:PM 01/13/2014 Page 162

W A R N I N G Simple hashing doesn’t protect your data if the attacker is willing to build a “dictionary” and hash each term in the dictionary. For SSNs, that’s only a billion

hashes to run, which is cheap on modern hardware. Hashing is therefore not the right

defense if your data is low entropy, either because the data is short strings, or because

it’s highly structured. In those cases, you’ll likely want to encrypt, and use a unique

key and initialization vector per plaintext.

Split-Key Systems

Splitting keys is useful when you’re concerned about the threat of someone decrypting the data without authorization. It’s possible to encrypt data with multiple keys, such that all or some fraction of the keys is needed to decrypt it. For example, if you store m = ek1(ek2(plaintext)), then to decrypt m, you need the party that holds k1 to decrypt the m, then send that to whomever holds k2.

If you’re worried about availability threats, there are split-key cryptographic systems for which the keys are mathematically related, and you only need k-of-n keys to get the plaintext out of the system. Such systems encrypt the data with n keys. Those n keys are mathematically related in ways which allow any k of the n keys to decrypt the data. These are useful, for example, for backing up the master key to a system where that master key may be needed a decade later. Such a system is used to backup the root keys for DNSSEC.

Private Information Retrieval

If the threat is a database owner watching a client’s queries and learning from them, then a set of techniques called private information retrieval may be useful. Private information retrieval techniques are generally fairly bandwidth inten- sive, as they often retrieve far more information than is desired to get at the data without revealing anything to the database owner.

Diff erential Privacy

When the threat is a database client running multiple queries to violate the database owner’s privacy policies, differential privacy provides the database owner with a way to fi rst measure how much information has been given out, and then stop answering queries that provide additional information. This does not mean that the database needs to stop answering queries. Many queries will not change, or differentiate, the amount of information that can be inferred, even after the database has reached a specifi ed privacy limit.

www.it-ebooks.info

Chapter 8 ■ Defensive Tactics and Technologies 163

c08.indd 12:26:46:PM 01/13/2014 Page 163

N O T E Diff erential privacy off ers very strong protection for a very specifi c defi nition of privacy.

Mixes and Mix-Like Systems

A mix is a system for preventing traffi c analysis and providing untracability to message senders or recipients. That is, an observer should not be able to trace a message back to a person after it has been through a mix. Mixes work by maintaining a pool of messages, and now and then sending messages out. To avoid trusting a single mix, there may be a network of mixes operated by different parties.

There are two major modes in which mixes operate: interactive-time and batch. Interactive-time mixes can be used for scenarios like web browsing, but are less secure against traffi c analysis. There are also interactive systems that do not mix traffi c but aim to conceal its source and destination. Such interactive systems include Tor.

Blinding

Blinding helps defend against surveillance threats that use cryptographic keys as identifi ers. For example, if Alice is worried that a certifi cate authority might track her vote, then she might want a voting registration system that can do deep checking to ensure that she is authorized to vote, providing her with an anonymous voting chit that can be used to prove her right to vote. Online, this can be done through the use of blinding.

Blinding is a cool math trick that can solve real problems. The math may look a little intimidating, but you can understand it with high school algebra. Think of signing as doing exponentiation modulo p. (Modulo is a remainder—1 mod 12 is 1, 14 mod 12 is 2, where 14 mod 10 is 4. The modulo math is needed for certain security properties.) Therefore if s is the signature, a is Alice’s key, and c is the CA’s key, then a signature is s = ac mod p. Normally, the CA calculates the signature, and sends s back to Alice. Now the CA knows s, and can use that knowledge. So how can the CA calculate s without knowing it? Because multiplication is commutative, the CA can calculate something related to s. Blinding works by Alice multiplying her key by some blinding factor (b) before sending the product of that multiplication (ab) to the CA. The CA then calculates s = (ab)c mod p, and sends s to Alice. Alice then divides s by b, and s/b = ac. So Alice now knows s/b, which appears for all the world like a signature on a, but the CA doesn’t know that a is associated with Alice. The math shown here is

www.it-ebooks.info

164 Part III ■ Managing and Addressing Threats

c08.indd 12:26:46:PM 01/13/2014 Page 164

a subset of what’s needed to do this securely, with the goal of giving you an idea of how it works. Proper blinding requires that you deal with a plethora of mathematical threats, which are covered in a book such as (Ferguson, 2012).

Compliance and Policy

To the extent that a business decision has been made to gather and store sensi- tive information about people, the organization needs to put controls around it. Those controls can either be policy or technical, and they can meet either your business need or regulatory needs, or both. These approaches are not as crisp as the tools and tactics you can apply to security problems, and to the extent that you can apply minimization or cryptography to privacy problems, it will be easier and more effective.

Policies

The fi rst class of controls are organizational policies that specify who can do what with the information. From the technologist’s perspective, these can be frustratingly vague statements, such as “only authorized people will be allowed access.” However, they are an important fi rst step in setting requirements. From a statement like that you can derive a technical approach, such as “only a security group can access the data.” Then you’ll need to ensure that that policy is enforced across a variety of information systems, and then you’re at the level of tactics and technologies.

Regulatory Requirements

Personal data is subject to a long and complex list of privacy rules that differ from jurisdiction to jurisdiction. As with everything else covered in this book, this section on mitigating privacy threats is not intended to replace proper legal advice.

There’s one other thing to be said about mitigating privacy threats to your organization. In many cases, organizations are required by law to collect and protect a set of information that they must treat as toxic waste, at great expense. It makes a great deal of privacy sense for organizations and their industry groups to argue against requirements to gather such data. That includes rolling back existing mandates and holding fi rm against new mandates to collect data that you’d prefer not to hold.

Summary

The best way to address threats is to use standard, well-tested features or prod- ucts that add security against the threats you’ve identifi ed. These tactics and

www.it-ebooks.info

Chapter 8 ■ Defensive Tactics and Technologies 165

c08.indd 12:26:46:PM 01/13/2014 Page 165

technologies are available to address each of the STRIDE threats. There are tactics and technologies available to both developers and operations.

Authentication technologies mitigate spoofi ng threats. You can authenticate computers, bits, or people. Integrity technologies mitigate tampering threats. Generally, you want integrity protection for fi les and network connections. Non-repudiation technologies mitigate repudiation, which can include fraud and other repudiations. Anti-fraud technologies include validation services, or use of customer history that’s either local or shared by others. There are also a variety of cryptographic and operational measures you can take to increase assurance around your logs. Information disclosure threats are addressed by confi dentiality technologies. Those can be most easily applied to fi les or net- work connections; however, it can also be important to protect container data, such as fi lenames or the fact of communication. Preventing denial of service involves ensuring that code doesn’t have arbitrary limits that prevent it from taking advantage of all the available resources. Preventing elevation of privilege generally works by fi rst ensuring that the code is constrained by mechanisms such as ACLs, and then by more complex sandboxes.

Patterns are collections of tactics and technologies. They seem like a natural approach. For reasons which are unclear, they haven’t really taken off, and those who are considering using them would be advised to understand why.

Mitigating privacy threats is best done by minimizing what you collect, and then applying cryptography; however, there are limits to the tactics and technolo- gies available, and sometimes you must fall back to compliance tools or policy.

The issue of standard tactics and technologies not being applicable every- where is not limited to privacy. In the next chapter, you’ll learn about making structured tradeoffs between ways to address threats.

www.it-ebooks.info

www.it-ebooks.info

167

c09.indd 09:44:28:AM 01/09/2014 Page 167

After you create a list of threats, you should consider whether standard approaches will work. It is often faster to do so than to assess the risk trade-offs and the variety of ways you might deal with the problem. Of course, it’s helpful to understand that there are ways to manage risks other than the tactics and technologies you learned about in Chapter 8, “Defensive Tactics and Technologies,” and those more complex approaches are the subject of this chapter.

For each threat in your list, you need to make one or more decisions. The fi rst decision is your strategy: Should you accept the risk, address it, avoid it, or transfer it? If you’re going to address it, you must next decide when, and then how? There are a variety of ways to think about when to address the threat. Table 9-1 provides an example to make these choices appear more concrete and to help separate them:

Table 9-1: Sample Risk Approach Tracking Table

ITEM # THREAT WHY NOT USE STANDARD MITIGATION? STRATEGY APPROACH

1 Physical tampering

We don’t own the hardware. Accept Document on website

This may seem like a lot of things to do for every threat, but the fi rst approach to fi xing most issues is to try to apply standard mitigations, and only look for an alternative when that fails.

C H A P T E R

9

Trade-Off s When Addressing Threats

www.it-ebooks.info

168 Part III ■ Managing and Addressing Threats

c09.indd 09:44:28:AM 01/09/2014 Page 168

This chapter fi rst teaches you about risk management, in the sense of avoid- ing, addressing, accepting, or transferring risks. You’ll learn how to apply risk management to software design. From there, you’ll learn about a variety of threat-specifi c prioritization approaches, ranging from the simple to the complex. The chapter also covers risk acceptance, and closes with a brief discussion of arms races in mitigation strategies.

Classic Strategies for Risk Management

As you consider each threat, you should make three decisions. Logically, these can be ordered as follows:

1. What’s the level of risk?

2. What do you want to do to address that risk?

3. How are you going to achieve that?

Strategizing in this manner can be expensive. Therefore, when there is an easy way to address a problem, you should skip strategizing and just address it. Not only is it easier, it avoids the possibility that your risk management approach will lead you astray.

When you do fi nd it necessary to strategize, a few classic strategies exist for addressing risks: You can avoid them, address them, accept them, or transfer them. You can also ignore risks, but that option is not ignored in this section.

Avoiding Risks

Avoiding risks is a great approach to the extent that you can do so. A good risk assessment can help you determine whether a risk is greater than the potential reward. If it is, then the risk may be worth avoiding. As the saying goes, “a ship in the harbor is safe, but that is not what ships are for.” So how do you avoid a risk? You don’t build the feature, or you change the design suffi ciently that the risk disappears.

Addressing Risks

Addressing risks is also a perfectly valid approach. The main ways you do so are via design changes (such as adding cryptography) or operational processes. Design and operational changes were covered in Chapter 8.

www.it-ebooks.info

Chapter 9 ■ Trade-Offs When Addressing Threats 169

c09.indd 09:44:28:AM 01/09/2014 Page 169

Accepting Risks

You can choose to accept a risk by deciding to accept all the costs of things going wrong. This is easier when you’re operating a service than when you’re building a product. You can’t accept risk on behalf of your users or customers—if there’s risk that affects them, that risk is transferred.

Sometimes a threat is real but the probability is very low, or the impact is minor. In these situations, it’s probably reasonable to accept the risk. Before doing so, it may be helpful to reassess both the probability and the impact, asking whether there are any factors that would change either (e.g., “do not use Panexa while pregnant or when you might become pregnant”). It can also be illuminating to ask whether you would accept the risk for yourself, if you couldn’t hand it to customers or others. Note that the word “illuminating” is used, rather than “useful.”

Risk acceptance differs from “ignore it” or “wait and see” (see the sections on each later in this chapter) insofar as it entails accepting that certain things are real risks. For example, Microsoft treats threats that involve unconstrained access to hardware as non-threats (Culp, 2013), and is explicit about the risk acceptance and the rationale behind that decision.

Transferring Risks

Risks that fall on your customers or end users are transferred risks. Many products bundle some level of risk with them, and use some combination of terms of service, licensing agreements, or user interface to transfer the risk. You should clearly disclose such risks.

Ignoring Risks

A traditional approach to risk in information security is to ignore it. Approaches to measuring risk have been hampered by an orientation towards secrecy and obscurity. Historically, both the occurrence of each breach and impact infor- mation has been generally kept secret. As a result, calculating frequency and making predictions have been hard, and a de facto strategy of ignoring risks emerged. From an executive standpoint, this strategy was highly effective, if frustrating for security staff.

This approach is becoming less effective as a combination of contracts, law- suits, and laws increase the risk of ignoring risks. In particular, a variety of new American laws make ignoring information security risks more risky. They include breach disclosure laws, general information security laws, sectorial information security laws, and Federal law for public companies. Breach disclosure laws

www.it-ebooks.info

170 Part III ■ Managing and Addressing Threats

c09.indd 09:44:28:AM 01/09/2014 Page 170

usually do not (directly) regulate information security, but require disclosure when it fails. Some states now have general security laws that apply to any stor- age of certain types of personal information. Outside of the U.S., there may also be requirements to disclose security breaches, especially those that involve a loss of control of personal information, or in certain sectors (such as telecom- munications in Europe). The overall regulatory situation around security risks is also rapidly evolving, and disclosures are helping security professionals learn from each other’s mistakes, and focus attention on important issues.

Finally, if you are threat modeling and create a list of security problems that you decide not to address, please send a copy of the list to the author, care of the publisher. There will be quarterly auctions to sell them to plaintiff’s attorneys (or other interested parties). Even if you don’t send them to me, they may be revealed by whistleblowers, accidentally shared or disclosed, or discovered as part of a legal action. All in all, it seems that “ignore it” is a riskier proposition than it has been before.

Selecting Mitigations for Risk Management

There are many ways to select mitigations. The following sections describe how to integrate risk management into your decisions about how to mitigate threats. The ways to mitigate cover a spectrum, with risk increasing as you move from changing the design through standard tactics and technologies to designing your own.

Changing the Design

The fi rst way to address risks in a design is to eliminate the features to elimi- nate the risks. For example, if you have payroll software that is accessible to the entire Internet without authentication, then changing the design so that it’s only available on your corporate intranet will reduce the risk. This aligns with the risk-avoidance approach. Unfortunately, carried to its logical conclusion, this leaves you with software that doesn’t do anything at all. While that has a Zen simplicity to it, Zen simplicity is usually not a requirement; features are.

This brings you to the second way to address risks: Change the design by adding features that reduce risks. For example, you can redesign the software to add authentication and authorization features.

There are two ways to think about changing designs: iterative and comparative. Iterative means altering a small number of components, with each change intended to reduce the number of components or trust boundaries or

www.it-ebooks.info

Chapter 9 ■ Trade-Offs When Addressing Threats 171

c09.indd 09:44:28:AM 01/09/2014 Page 171

otherwise eliminate some threats. The comparative method means coming up with two or more designs, and then comparing them. This would appear to be more expensive, and in the short term it is. However, it is common for iterative design to involve many iterations, so often it is more cost-effective (especially in the early days) to consider several designs and choose between them.

For an example of how one might change a design, consider Figure 9-1, which depicts a threat model for sending single-use login tokens to phones. (These tokens are often called one time tokens, abbreviated OTT.) There are many advan- tages to this type of authentication, including the capability to deploy to all sorts of phones, including voice-only phones, “dumb” mobile phones, and smartphones. The downside is a plethora of places where that token is subject to information disclosure.

Those places include the varied systems responsible for mobile phone roaming and the “femtocell” base stations that telephone companies distribute. They also include things such as Google Voice or iMessage, which put text messages onto the Internet in various ways. (These products are mentioned only as examples, rather than a comment on their security.)

Login System Telco interface Number routing

1. Authentication Challenge

Expected OTT Organization

2. Phone # 2a Phone #

2b Telco

2c Telco

Google Voice

4. OTT

5. OTT

Roaming Routing

Femtocell Routing

3. phone #, OTT

Customer Effective Telco

iMessage

Mobile Phone

(or) (Non-exclusive)

Figure 9-1: An OTT threat model

www.it-ebooks.info

172 Part III ■ Managing and Addressing Threats

c09.indd 09:44:28:AM 01/09/2014 Page 172

There’s a variety of ways to change the design and address the information disclosure threats that apply to Figure 9-1. Simplifi ed versions of ways to address these threats follow. In each, m1 is a message from the server to the client, while m2 is a response.

■ Send a nonce, encrypted to a key held on a smartphone, and then send the decrypted nonce to the authentication server. The server checks that the noncen is the one that it encrypted for the phone, and if it is, approves the transaction. (m1 = ephone(noncen), m2 = noncen).

■ Send a nonce to the smartphone, and then send a signed version of the nonce to the server. The server validates that the signature on the noncephone is from the expected phone, and that it is a good signature on the expected nonce. If both checks pass, then the server approves the transaction. (m1 = noncephone, m2 = signphone(noncephone)).

■ Send a nonce to the smartphone. The smartphone hashes the nonce with a secret value it shares with the server, and then sends that hash back. m1 = noncephone , m2 = hashphone(noncephone)).

It’s important to manage the keys appropriately for each of these methods and to understand these are simplifi ed examples; they do not include time stamps, message addressing, and other elements to make the system fully secure. Including those in this discussion makes it hard to see the ways in which cryptographic building blocks could be applied.

The key in all design changes is to understand the differences introduced by the changes, and how those changes interact with the software requirements as a whole.

N O T E This model and some ways to address the threats, are worked through in more detail in Appendix E, “Case Studies.”

For another example of comparative threat modeling, consider the two sys- tems shown in Figures 9-2 and 9-3. Figure 9-2 depicts an e-mail system, and Figure 9-3 is a version of 9-2 with a “lawful intercept” module added. (“Lawful intercept” is an Orwellian phrase for “thing which allows people to bypass the security features of your system.” Setting aside any arguments of “should we as a society have such a mechanism?” it’s possible to assess the technical security implications of adding such mechanisms.)

It should be obvious that Figure 9-2 is more secure than Figure 9-3. Using software-centric modeling, Figure 9-3 adds two data fl ows and a process; thus, by STRIDE-per-element, it has an additional 12 threats (tampering, informa- tion disclosure, DoS with each fl ow, for 6; and the six S,T,R, I, D, and E threats against the process for a total of 12). Additionally, Figure 9-3 has two apparent groupings of elevation-of-privilege threats: those posed by outsiders and those

www.it-ebooks.info

Chapter 9 ■ Trade-Offs When Addressing Threats 173

c09.indd 09:44:28:AM 01/09/2014 Page 173

posed by software-allowed, but human-policy-violating, use. Thus, if Figure 9-2 has a list of threats (1…n), then Figure 9-3 has a list of threats (1…n+14).

Acme E-mail InternetClient

Figure 9-2: An e-mail system

Acme E-mail InternetClient

Acme Lawful Access Module Government

Figure 9-3: The same e-mail system with a lawful access module

If instead of software-centric modeling you use attacker-centered modeling on the systems shown in Figures 9-2 and 9-3, you fi nd two sets of threats: First, each law enforcement agency that is authorized to connect adds its employees and IT systems as possible threats, and possible threat vectors. Second, attackers are likely to attack these features of the system to abuse them. The 2010 “Aurora” attacks on Google and others allegedly did exactly this (McMillan, 2010, and Adida, 2013). Thus, by comparing them you can see that the addition of these features creates additional risk. You might also wonder where those risks fall, but that’s outside the scope of this example.

More subtly, the addition of the code in Figure 9-3 is an obvious source of security vulnerabilities. As such, it may draw attention and possibly effort away from the rest of the system. Thus, the components that comprise Figure 9-2 are likely to be less secure, even ignoring the threats to the additional components. In the same vein, the requests and implementations for such back- doors may be confi dential or classifi ed. If that’s the case, the features may not go through normal tracking for implementation, testing, or review, again reducing

www.it-ebooks.info

174 Part III ■ Managing and Addressing Threats

c09.indd 09:44:28:AM 01/09/2014 Page 174

the odds that they are secure. Of course, because such a system is designed to bypass other security controls, any weaknesses are likely to have outsized impact.

Applying Standard Mitigation Technologies

To the extent that standard approaches will effectively solve a problem, they should be used. Devising new approaches to defense is very simple and lots of fun. Unfortunately, devising effective new approaches, building, and testing them can be very time consuming.

N O T E To be fair, testing newly devised mitigation approaches can actually be super-quick sometimes, as the new defense will fall with just a few minutes of expert

scrutiny. PGP creator Phil Zimmerman tells the story of bringing the cipher he created

for the fi rst version of his popular encryption software to a large gathering of cryptog-

raphers, where he saw months of his painstaking work demolished over lunch. After

that, PGP switched to using standard ciphers.

There are a number of ways in which you can use standard mitigations, includ- ing platform-provided ones, developer-implemented ones, or operational ones.

Platform-Provided Mitigations

Software developers have a number of ways to code mitigations to common threats. Each has pros and cons. That’s not to say that they’re all equal, or that any one has advantages in every situation. As a general rule, using the defenses in whatever platform you’re building on is a good idea for several reasons: First, they often run at a higher trust level than defenses you can build. Second, they’re generally well designed and subjected to a high level of scrutiny before you get to them. Related to that, they’re often more intensively tested than what you can justify. Finally, they’re usually either free or included in the price of the platform.

Developer-Implemented Mitigations

There is an entire set of mitigations that are not platform provided, and must be implemented by developers. Almost all of these can be seen as feature develop- ment work: Implement a cryptographic scheme to address a spoofi ng threat; implement a better logging system to address a repudiation threat, and so on.

An interesting and growing set of software is built and operated by the same organization. This creates an additional class of defensive opportunities that

www.it-ebooks.info

Chapter 9 ■ Trade-Offs When Addressing Threats 175

c09.indd 09:44:28:AM 01/09/2014 Page 175

you can design. This class focuses on attack detection, whereby attempts to execute injection or overfl ow attacks can be found and repaired. With the right investment in detection and response capabilities, this allows some deferment of security costs to fi nd and fi x those bugs. Doing so requires developing the features to detect such attacks. It also increases the importance of information disclosure attacks, which disclose your source (or binary) to an attacker, pos- sibly enabling them to develop a reliable exploit without triggering your attack detection.

Operational Mitigations

Systems administrators should consider a number of trade-offs in terms of policy, procedural, and software defenses. To the extent that software defenses are available, they will scale better and be more reliable than process-oriented controls. Operationally, process-oriented controls provide defense against a broader swath of issues. In particular, effective change control helps manage a wide variety of issues. A fi rewall to control issues at the network or edge is easier to manage and maintain. However, at the machine level, there is less doubt about what a packet means (Ptacek, 1998). Highly targeted defensive programs are often technically superior to more general ones, but they are less well-integrated into operations, which may mean that they are less effective overall. Many classes of mitigations that have become commercial standards are “arms race” technology.

For example, a signature-based anti-virus program is only as good as its last update. In contrast, a fi rewall will block packets according to its rules. Some fi rewalls were implemented as packet fi lters, and can be fooled, but others implemented connection proxying (meaning the connection terminates at the fi rewall, and code on the fi rewall makes an additional connection to the target system). Those fi rewalls don’t engage in a meaningful arms race, although some of them include signature-driven intrusion detection or prevention code, and that code requires regular updating. (Arms races are so much “fun” that they get their own section, towards the end of this chapter.)

It is valuable for developers to understand the system administrator’s perspec- tive on defenses, and vice versa. This is especially true of standard mitigations, where you can rely on a body of knowledge, analogies, and other facets of the mitigation being a standard approach to make it easier to operate. Developing a defensive system that no one can operate is about as bad as developing one that doesn’t work. The ultimate goal of deploying operational technology that meets business needs requires developers and system administrators to understand the limits of available defenses. The example defenses shown in Table 9-2 are for the Acme SQL Database introduced previously in the book.

www.it-ebooks.info

176 Part III ■ Managing and Addressing Threats

c09.indd 09:44:28:AM 01/09/2014 Page 176

Table 9-2: Defenses by Who Can Implement Them

THREAT INSTANCES THAT DEVELOPERS MUST IMPLEMENT

INSTANCES THAT IT DEPARTMENTS MUST IMPLEMENT

Spoofi ng Web/SQL/other client brute-forcing logins

Authentication for DBA (human), DB users

Authentication infrastruc- ture for: web client, SQL cli- ent DBA (human), DB users

Tampering Integrity protection for Data, Management, Logs

Integrity protection for front end(s), Database admin

Repudiation Logs (log analysis must be protected)

Certain actions from web and SQL clients will need careful logging.

Certain actions from DBAs will need careful logging.

Logs (log analysis must be protected)

If DBAs are not fully trusted, a system in another privi- lege domain to log all com- mands might be required.

Information Disclosure

Data, management, and logs must be protected.

Front ends must implement access control.

Only the front ends should be able to access the data.

ACLs and security groups must be managed.

Backups must be protected.

Denial of Service Front ends must be designed to mini- mize DoS risks

The system must be deployed with suffi cient resources.

Elevation of Privilege

Trusting client

DB should support prepared state- ments to make injection harder.

There should be no default way to run commands on the server, and calls like exec()or system() must be permissioned and confi gurable if they exist.

Avoiding improperly trust- ing clients which were writ- ten locally.

Confi gure DB appropriately.

Designing a Custom Mitigation

As explained earlier, custom approaches are risky and expensive to verify, but sometimes you have no choice. Aspects of your design or implementation

www.it-ebooks.info

Chapter 9 ■ Trade-Offs When Addressing Threats 177

c09.indd 09:44:28:AM 01/09/2014 Page 177

could prevent all the standard approaches. For example, if you’re implement- ing a low-cost device with an eight-bit processor, it’s probably not going to be able to use 2,048-bit RSA keys at acceptable speeds. At that point, you need to consider what you can do.

Because the defense will be custom, there are fewer specifi cs to talk about. However, some general guidelines apply. Ensure that you have a clearly written defi nition of the goal of the custom system, along with the constraints you’re operating under. Consider what will happen when it breaks, and in particular what you’ll do to update it.

Custom mitigations differ from non-security code in a very important way: It’s hard to see that they’re ineffective. The mistakes made by inexperienced database designers are easy to see; problems like performance will crop up rela- tively quickly and obviously. With security mitigations, it’s easier to be fooled.

When you’re designing a custom approach to mitigation, it’s worthwhile to take a couple of unusual steps early in the process. The fi rst is to share your motivation and design. You may well get useful feedback on it. You’ll get more useful feedback if you make the design public (rather than requiring an NDA), and/or if you pay for feedback, perhaps by hiring experts to break it or by offer- ing a prize to anyone who can break it.

W A R N I N G Be careful to explain what you did separately from why you think those defenses make it hard to break; the interleaving can inhibit the free fl ow of ideas

in the same way that criticism can shut down a brainstorming session.

Fuzzing Is Not a Mitigation

Oftentimes, to address a variety of threats, people will say “we’ll fuzz that!” Fuzzing is the technique of generating random input for a program, and it is stunningly effective at fi nding bugs in code that’s never been fuzzed. This is especially true of parsers. However, fuzzing is not a way of mitigating threats; it’s a way of testing mitigations. Fuzzing will not make your code secure, it will help you fi nd bugs, and as those bugs are fi xed, the average time to fi nd the next bug using random input goes up. However, the time for a clever human to fi nd that next bug does not change. Therefore, over time, fuzzing becomes less effective. When you are tempted to fuzz, you should ensure it’s in your test plan, but at design time you need to take other actions.

You can do several things to make parsers more secure at design/code time. The fi rst is to design your fi le format or network protocol for safe parsing (Sassaman, 2013). If the format is not Turing complete, parsing it is easier. If it doesn’t contain macros, loops, multiple ways to encode things, or the capability to encapsulate layers of encoding, your parser can be a lot simpler, and thus safer. The next thing you can do to make parsers safer is to use a safer language than the C

www.it-ebooks.info

178 Part III ■ Managing and Addressing Threats

c09.indd 09:44:28:AM 01/09/2014 Page 178

family. C’s lack of type safety and primitive, unsafe string handling functions make safe coding of parsers hard. If the thing being parsed is already defi ned and you can’t redefi ne it (say, HTML), it may help to create a state machine for your parser. You may, quite appropriately, be laughing at the idea of creating a state machine for HTML. Well, if you can’t describe it, good luck with making a safe parser for it. Finally, if there’s a canonicalization step, canonicalize early, and run the input through it until the output matches the input.

Threat-Specifi c Prioritization Approaches

This section is all about risks you’re going to address (rather than avoid, accept, or transfer). For the problems you decide to address, you have choices to make about how to approach those risks. Those choices include simply waiting and seeing whether the risks materialize or fi xing the easy stuff fi rst, using threat- ranking techniques of various sorts, or using some approach to estimate the cost of the problem.

Simple Approaches

The very simplest approaches to threat prioritization don’t involve any math or calculation. They are “wait and see” and “do the easy fi xes fi rst.”

Wait and See

The wait and see approach to security issues sometimes works pretty well, and often fails catastrophically. It can differ from “ignore it” when the system in question is either an internal network or a service offering that can be monitored for problems. Wait and see is a worse technique for, say, a gas tank, than it is for a website. It can also be an example of what 451 Group analyst Wendy Nather calls “the cheeseburger case”: “Doc, I’m gonna keep eating cheeseburgers until I have a heart attack. Then we’ll deal with it” (Nather, 2013). The cheeseburger case is less about accepting risks, but more about ignoring those risks—and doing nothing to mitigate them until a catastrophe forces you to pay attention. Most businesses have long used monitoring as a part of their risk management strategy. For example, if a bank notices that one of its employees suddenly has a fl ashy car, someone is likely to question where the money came from. Monitoring, the “see” part of “wait and see,” needs to be planned effectively. There are four main types of monitoring: change detection, signature attack detection, anomaly attack detection, and impact detection.

www.it-ebooks.info

Chapter 9 ■ Trade-Offs When Addressing Threats 179

c09.indd 09:44:28:AM 01/09/2014 Page 179

N O T E Related to the cheeseburger approach in risk management is the ostrich approach of sticking ones head in the sand. Of course, your colleagues don’t really

stick their heads in the sand, they say “no one would ever do that!” The best response

to this is to say “if I can give you an example where someone did that, will you agree

to fi x it?” (This is another place where the repertoire that people develop en route to

becoming an expert can come in very handy.)

Change Detection

Change detection focuses on the operational discipline of ensuring that change is managed. To the extent that changes are managed, anything outside the change management process must be treated as a problem. (Kim, 2006)

Signature Attack Detection

Signature attack detection is based on the idea that an interesting subset of attacks has certain defi nable signatures, either sequences of bytes or messages that will appear only in an attack. With the rise of production-quality exploit software, this signature-oriented approach appeared promising. However, as the products to detect signatures proliferated, the attack tools added polymorphism, and simple signature detection is less effective than its proponents had hoped.

Anomaly Attack Detection

Anomaly attack detection is based on the idea that there’s a normal and unchang- ing set of network traffi c. This is a pipe dream. As business changes, the normal traffi c changes; and when it does, the anomaly detector needs to be retrained about what’s normal. As business accelerates and change accelerates, it becomes increasingly diffi cult to continue training the system so that it knows what’s normal. There are also normal abnormalities (for example, every Friday evening, there’s an audit run, and every close of quarter there’s another one). It may be possible to use people to investigate every anomaly, although the cost of doing so rises very quickly.

Impact Detection

The fi nal major form of detection is impact detection. If suddenly the shipped product count for the quarter is out of whack with your accounts receivable, there’s a problem. (A friend once did a penetration test in which he ordered minus three copies of a book. The system credited his credit card the cost of the three books, minus shipping, and a week later he got the three books in the mail. When the merchant was told what happened, it turned out that the shipping clerk had seen the negative three copies, fi gured it was a bug, and sent the three copies.)

www.it-ebooks.info

180 Part III ■ Managing and Addressing Threats

c09.indd 09:44:28:AM 01/09/2014 Page 180

Generally, mature operations use some combination of all the techniques— change detection, signature attack detection, anomaly attack detection, and impact detection. The precise combination that will work for a given organization’s threat profi le, risk acceptance, culture, regulatory environment, and so on, are specifi c to that organization. Wait and see is less valid in (at least) three cases. First, when there’s a risk of injury or death. Duh. Enough said. Second, when you’re shipping a product to your customer, especially physical products. Regardless of whether the product is a baby’s crib, a car, or a piece of software, wait and see what goes wrong cannot be the primary risk management approach. Third, when you don’t have a response planned. For example, if you operate banking software, you probably have an adjustable dollar level at which you manually check transactions for fraud. You might adjust it based on current capacity in the fraud management department or on the cost effectiveness of the activity.

Easy Fixes First

Some organizations just starting to threat model may begin by fi xing those things that are easy to fi x before going on to harder fi xes. For many experienced security practitioners, this seems like an odd choice, but it’s worth discussing the pros and cons of the approach. On the pro side, fi xing security issues is a good thing, and demonstrating that threat modeling is producing actionable bugs may help ensure that threat modeling continues. The downside is that the issues you’re fi xing may be the wrong things, or things that don’t seem relevant to other parts of an organization, and thus appear to be a waste of time.

If you need to start from an easy fi x fi rst approach, ensure that you do so in consultation with the people who are performing the fi xes, and ensure they don’t perceive it as busywork. In addition, plan to move from easy fi xes to a more mature approach, as described in the remainder of this section.

Threat-Ranking with a Bug Bar

There are a few techniques that enable you to rank your threats with a little more precision than “easy” and “everything else.” These ranking techniques are designed to provide you with a consistent approach to addressing threats.

DREAD

One of the fi rst of these was DREAD. This awesome acronym stands for discoverability, reproducibility, exploitability, aff ected users, and damage. Unfortunately, DREAD is fairly subjective and leads to odd results in many circumstances. Therefore, as of 2010, DREAD is no longer recommended for use by the Microsoft SDL team.

www.it-ebooks.info

Chapter 9 ■ Trade-Offs When Addressing Threats 181

c09.indd 09:44:28:AM 01/09/2014 Page 181

The most effective simple prioritization technique is a bug bar. In a bug bar, bugs are given a severity based on a shared understanding of their impact. That shared understanding comes from a “bug bar” table that lists the criteria used to classify bugs, and might include examples. Over the product lifecycle, the bar defi ning what bugs must be fi xed is adjusted. Thus, six months before shipping, a medium-impact bug would be fi xed, whereas a bug discovered on the day of shipping would be fi xed in a hotfi x. The bar will likely be refi ned, and what must be fi xed may change as your security processes mature. Microsoft makes fairly heavy use of the bug bar concept.

A version of the complete Microsoft SDL bug bar is available online. The bug bar is made available under a Creative Commons license that allows you to use it within your organization (Microsoft, 2012).

Cost Estimation Approaches

Sometimes there are business reasons to use a threat prioritization approach that produces cost estimates. This section covers two such approaches: probability/impact assessments and FAIR.

Probability/Impact Assessments

Assessing probability and impact is an obvious approach, but effective imple- mentations are rare. There are a few sticking points, including that, unlike hurricanes or tornados, information security events are often caused by malice. However, insurance companies still write theft insurance, and don’t go out of business too often. Over time, the industry will likely learn more about both probabilities and impacts from incidents disclosure because of breach disclosure laws, and probability/impact assessments will become a more useful part of threat modeling. If you’re going to use a probability/impact assessment of any form, you’ll need to fi gure out the cost of mitigation, and few approaches help you do that (Gordon, 2006).

Probability assessments in information security are notoriously hard to get right. Well-engineered systems can often be broken with very inexpensive equipment. Kryptonite bike locks were found vulnerable to a Bic pen (Kahney, 2004). Facial recognition systems have been found to recognize photographs of an authorized person (Nguyen, 2009). Fingerprint readers have been beaten by gummy candy and laser printers (Matsumoto, 2002). Expensive equipment may be easier to get than you anticipate, for example, graduate students often have access to million-dollar lab equipment. At the other end of the spectrum, most people would consider an attack that allows someone to steal a few dollars as not worth a lot of time. However, a billion people worldwide live on less than one dollar a day, and their lives would be improved by stealing just a few dollars.

www.it-ebooks.info

182 Part III ■ Managing and Addressing Threats

c09.indd 09:44:28:AM 01/09/2014 Page 182

(This isn’t to say that everyone living at that level of poverty would become a criminal given the opportunity, only that typical Western assessments of cost/ benefi t trade-offs are challenged by global networking.)

FAIR

FAIR is the acronym for Factor Analysis of Information Risk, developed by Jack Jones while he was Chief Security Offi cer for a large bank. FAIR focuses on defi ning business risk associated with technology systems. FAIR defi nes a threat as “anything (e.g., object, substance, human) that is capable of acting against an asset in a manner that can result in harm.”

The primary use of FAIR is for systems that a business is deploying, regard- less of the source of the components (off the shelf or local). It defi nes risk as a function of loss event frequency and probable loss magnitude. Each of these is decomposed further, as shown in Figure 9-4.

FAIR has 10 defi ned steps in four stages:

Stage 1: Identify scenario components

1. Identify the asset at risk.

2. Identify the threat community under consideration.

Stage 2: Evaluate loss event frequency

3. Estimate the probable threat event frequency.

4. Estimate the threat capability.

5. Estimate control strength.

6. Derive vulnerability.

7. Derive loss event frequency.

Stage 3: Estimate probable loss magnitude

8. Estimate worst-case loss.

9. Estimate probable loss.

Stage 4: Derive and articulate risk

10. Derive and articulate risk.

The document “An Introduction to FAIR” (Jones, 2006) presents FAIR as an approach to risk management in business. The FAIR white paper starts out at what you might see as a very philosophical level. If you fi nd that frustrating, consider jumping to page 64 of that introductory paper, where FAIR is presented in a more concrete and compact fashion.

www.it-ebooks.info

Contact Action

Vulnerability Primary LossFactors Secondary Loss

Factors Threat Event Frequency

Control Strength

Threat Capability

Asset Loss Factors

Threat Loss Factors

Organizational Loss Factors

External Loss Factors

Loss Event Frequency

Probable Loss Magnitude

Risk

Figure 9-4: FAIR’s risk decomposition

www.it-ebooks.info

184 Part III ■ Managing and Addressing Threats

c09.indd 09:44:28:AM 01/09/2014 Page 184

There are two issues to discuss regarding FAIR. The fi rst is the way in which it assigns numbers to various elements of risk. The white paper acknowledges these in a remarkably frank discussion in the conclusions. It’s worth reiterating that without incident data, FAIR is a repeatable way to get to the same results, but the results are hard to compare to results for other systems. The second issue is FAIR’s opening steps, and the asset- and attacker-centricity of the system. An asset is defi ned as “any data, device, or other component of the environment that supports information-related activities, and which can be affected in a man- ner that results in loss.” This broad defi nition implies a remarkable amount of work, as FAIR analysis is run on each asset. This is probably ameliorated by an intuitive ranking approach. FAIR at least provides an “example” set of threat communities, but it does not address the effort needed to analyze their behav- iors, nor the risks in getting that analysis wrong. FAIR is probably the best of the approaches for quantifying risk. My lovely editor would like to know if that means it’s worth using, and so would I. More seriously, if your organization relies on quantifi ed risk assessments, FAIR has many things to recommend it, but if a simpler approach, such as a bug bar will work, that’s probably a better return on investment.

Mitigation via Risk Acceptance

As discussed in the section “Classic Strategies for Risk Management,” it is per- fectly reasonable to address risk via risk acceptance, and there are two ways that is commonly done: either via business risk acceptance or via user risk acceptance. The following sections describe both of these.

Mitigation via Business Acceptance

If an organization is building software for its own use, it is free to make whatever risk acceptance decisions it chooses. For example, if your inventory site exposes your product inventory to the world, that’s a choice you can reasonably make, by applying whatever risk approach works for you.

In a number of cases, a business may choose to take into account other per- spectives beyond its own in risk acceptance decisions. Those cases include privacy and “fi tness for purpose,” a term borrowed from the lawyers to mean something that’s good enough to do the job it’s intended to serve.

If the software involves personal, private information, then the risk accep- tance must take into account the myriad laws that apply to such things. Many of those laws require something like “appropriate security.” This constrains the decisions that the business can make regarding risk. Even if those laws do not apply, losing all your customer data may enable competitors to use that data to

www.it-ebooks.info

Chapter 9 ■ Trade-Offs When Addressing Threats 185

c09.indd 09:44:28:AM 01/09/2014 Page 185

market to your customers, or it may upset your customers. Lastly, it may have an impact on your reputation.

Fitness for purpose is the other element that may infl uence a business’s will- ingness to accept risk. If your system is being sold with the expectation that it can connect to the Internet, then there may be a reasonable expectation that it’s suffi ciently secure for that purpose. If it’s being sold for medical use, “critical infrastructure,” or similar areas, there may be substantial additional expecta- tions. In these types of cases, the business cannot silently accept the risk; at the very least the risk acceptance decisions must be communicated to customers. (I’m not a lawyer, but if you decide not to communicate such risks, I suspect you’ll get to know some very unfriendly lawyers.)

Mitigation via User Acceptance

There are times when a software developer or systems administrator can’t make a decision for the user. For example, there may be a business reason to visit Playboy.com. (For instance, consider an investment bank. No reason to allow adult content, right? Except when Playboy’s CEO visits to pitch her stock to the bank’s analysts, she’ll want to demo their new digital media line of business.) There may be a reason to have viruses on a system. Or maybe there’s something that is “obviously” a security error, but the user wants to accept the risk involved.

If you need to warn someone about a potential risk, ensure that such warnings are NEAT: necessary, explanatory, actionable, and tested. These four relatively simple steps work well for designing warnings (or improving existing ones), and NEAT is covered in more detail in Chapter 15, “Human Factors and Usability.”

When authenticating, ensure that the authentication is two-way. If any details have changed since the last authentication, inform the user about what’s differ- ent. This requires persisting some information, explaining it to the user, and walking them through evaluating it.

Arms Races in Mitigation Strategies

An arms race describes the predictable set of steps that both attackers and defend- ers engage in that leave both sides approximately where they were at the start of the arms race, only poorer. A classic example is signature-driven anti-virus software. Such software is only as good as its latest update. There will almost always be viruses that the signature authors have not yet discovered, or for which the signatures have not been tested, shipped, or applied.

It should perhaps go without saying that such arms races are to be avoided, but because they are frequent it’s worth a few words on why arms races happen, and what you can do should you fi nd it hard to avoid one.

www.it-ebooks.info

186 Part III ■ Managing and Addressing Threats

c09.indd 09:44:28:AM 01/09/2014 Page 186

Arms races happen because some factor makes a perfect defense hard. For example, it turns out that bolting on security defenses that block only what viruses do is nearly impossible. Commercial operating systems support a wide variety of behavior, and legitimate programs make use of those defi ned behaviors, sometimes in ways that are hard to distinguish from malicious use. Therefore, heuristic modules in anti-virus programs have a high rate of false positives. Similarly, whitelisting of programs in advance is an excellent defensive technique, but one that turns out to be very inhibitory to the normal use of computers.

When you fi nd yourself in an arms race, you are playing an economic game, and your goal should be to both minimize your cost while maximizing your profi t, while simultaneously maximizing your opponent’s cost and minimizing their profi t. Your costs go up when you must scramble, and your profi ts will be maximized the longer your opponent spends time reacting to your latest moves. This leads to two related strategies: Aim for the last-mover advantage and have a bag of tricks.

Last-mover advantage is a term credited to cryptographer extraordinaire Paul Kocher (Kocher, 2006). It refers to the idea that the last side to take action has an advantage. Further, your moves should be designed, in part, to fl ummox the other side, and make it tricky for them to respond. Therefore, here, the use of obfuscation or anti-debugging techniques can pay dividends. Having a system that’s designed to roll forward to a new confi guration can also help, and here’s where a bag of tricks comes in. A bag of tricks is a set of moves in the arms race that are already coded and tested. When you detect that your opponent has taken a new move, you deploy something from your bag of tricks. For example, if you have a DRM scheme to prevent people from using your music fi les as they choose, you might have a set of additional restrictions coded up and ready to ship as attackers break your current scheme. This enables you to maximize how long you have the last-mover advantage.

Summary

There are many strategies you can apply to risk management. The classic risk management strategies of avoid, address, accept, or transfer are applicable to how you address threats.

More specifi c to threats and security are the approaches of changing the design, applying standard mitigations, and designing custom mitigations. It’s expensive and time consuming to test your custom mitigation, and easy to get the design wrong, so custom mitigations should be a fallback. The department of “easy to get wrong” also includes fuzzing, which is more appropriately seen as a test technique. (It’s a great technique, but you can’t fuzz your way to a secure design or implementation.)

www.it-ebooks.info

Chapter 9 ■ Trade-Offs When Addressing Threats 187

c09.indd 09:44:28:AM 01/09/2014 Page 187

Sometimes you need to prioritize your mitigation approaches, and there’s a variety of ways to do so, ranging from simple ones like wait and see (and the associated actions to ensure you do see) to bug bars and quantifi ed risk man- agement approaches such as FAIR.

Now and then you need to accept risks, or ask others to do so. When you need to accept risk, you should ensure that you’re doing so with some structure and not using “accepted risk” as a synonym for ignoring it. When you need to ask others to accept risk, you should do so clearly, and the NEAT approach can help you do so.

Sometimes arms races are hard to avoid. If you do fi nd yourself there, there are some strategies to drive your opponent’s costs up while keeping yours low. You want to aim for a last-mover advantage that forces your opponent to scramble while you relax. That’s what good risk management can get you.

www.it-ebooks.info

www.it-ebooks.info

189

c10.indd 09:44:43:AM 01/09/2014 Page 189

You’ve been hard at work to address your threats, fi rst by simply fi xing them, and then by assessing risks around them. But are your efforts working? It is important that you test the fi xes, and have confi dence that anything previously identifi ed has been addressed.

Good testers have a lot in common with good threat modelers: Both focus on how stuff is going to break, and work on preventing it. Working closely with your testers can have surprisingly positive payoff for threat modeling proponents, a synergy explored in more detail in Chapter 17, “Bringing Threat Modeling to Your Organization.”

A brief note on terminology: In this chapter, the term testing is used to refer to a key functional task that “quality assurance” performs: the creation and management of tests. This chapter focuses only on the subset of testing that intersects with threat modeling. As shown in Figure 10-1, threat-model-driven testing can overlap heavily with security testing, but the degree of overlap will vary across organizations. Some organizations have reliability testing specialists. They need to understand the issues you fi nd when looking for denial-of-service threats. Others might manage repudiation as part of customer readiness. Your security testers might also use fuzzing, look for SQL injection, or create and manage tests that are not driven by threat modeling.

C H A P T E R

10

Validating That Threats Are Addressed

www.it-ebooks.info

190 Part III ■ Managing and Addressing Threats

c10.indd 09:44:43:AM 01/09/2014 Page 190

All Testing

Security Testing

Threat-Model- Driven Testing

Figure 10-1: Different types of testing

This chapter will teach you about testing threat mitigations, how to exam- ine software you acquire from elsewhere to check that threats are addressed to your satisfaction, explain how to perform quality assurance on your threat modeling activity, and discuss some of the mechanical and process elements of threat modeling validation. It closes with a set of tables designed to help you track your threat modeling testing activity.

Testing Threat Mitigations

You’ll need both process and skills to test the mitigations you’re developing. This section explains both, and also discusses penetration testing, which is often mistaken for a complete approach to testing security.

Test Process Integration

Anything you do to address a threat is subject to being tested. However you manage test cases, you should include testing the threats you’ve found and chosen to address. If you’re an agile team that uses test-driven development, develop at least two tests per threat: one that exploits the easy (no mitigation) case, and at least one that attempts to bypass the mitigation. It can be easy, fun, and even helpful for your testers to go nuts with this. It may be worth the effort to ensure they start with the highest-risk threats, or the ones that developers don’t want to fi x. If you use bugs to track test development, you might want to fi le two test-creation bugs per threat. One will track the threat, and the other the test code for the threat. Then again, you might fi nd this overkill. Giving threat model bugs unique tags can help you when you search for threat model test bugs. (For example, you can use “threatmodel” to fi nd all bugs that come from threat enumeration, and “tmtest” for the test bugs.)

www.it-ebooks.info

Chapter 10 ■ Validating That Threats Are Addressed 191

c10.indd 09:44:43:AM 01/09/2014 Page 191

If you have some form of test planning, then you should ensure that the list of threats feeds into the test planning. A good list of threats delivered to a tester can produce an avalanche of new tests.

How to Test a Mitigation

If you think of threats as bugs, then testing threats is much like doing heavy testing on those bugs. You’ll want to set up a way to reproduce the conditions that cause the bug to trigger, and create a test that demonstrates whether a fi x is working. That test can be manual or automated. Sometimes the level of effort needed to code an automated test can be high, and you’ll be tempted to test manually. At the same time, these are bugs you’d really prefer do not regress, which is an argument for automation.

Of course, unlike normal bugs, attackers will vary conditions to help them trigger your bugs, so good testing will include variations of the attack. The exact variation you’ll want depends heavily on the threat. You should vary whatever the mitigation code depends on. For a simple example, consider an XSS bug in a website. Such bugs are often demonstrated by use of the JavaScript alert function. Therefore, perhaps someone has coded a test that looks for the string “alert.” Try replacing that with “%61%6C%65%72%74%0A.” Robert “RSnake” Hansen has a site dedicated to creating such variants—his “XSS (Cross Site Scripting) Cheat Sheet Calculator" (OWASP, 2013b). You can use such a site to develop the skills to create your own variations.

As you develop tests for a threat, you can also consider how the mitigations you’re developing can be attacked. This is very similar to attack variations, but goes back to the idea of second-and third-order threats discussed in Chapter 7, “Processing and Managing Threats.”

The skills required to perform testing of threat mitigations vary according to the type of threat and mitigation. There are test engineers at large companies who are experts at shell coding, that is, the specialized attack code used to take over control fl ow. There are others who are expert cryptographers, spend- ing their days looking for mathematical attacks on the cryptography their employers develop.

Penetration Testing

Some organizations choose to use penetration testing to validate their threat models and/or add a level of confi dence in their software. Penetration testing (aka pen testing) can supplement threat modeling. But there’s a saying that “you can’t test quality in.” That means all the testing you might possibly do will never make a product great. It will just help you fi x the defects you happen to fi nd. To make a quality product, you need to start with good design, good raw materi- als and good production processes, and then check that your output matches

www.it-ebooks.info

192 Part III ■ Managing and Addressing Threats

c10.indd 09:44:43:AM 01/09/2014 Page 192

your expectation. In the same way, you can’t test your product to secure. So, pen testing can’t replace threat modeling.

Pen testing can be either black box or glass box. Black box pen testing pro- vides only the software to the testers, who will then explicitly test assumptions you have about how much effort it takes to gain an understanding of the soft- ware. This is an expensive undertaking, especially compared to glass box pen testing, whereby testers are given access to code, designs, and threat models, and they are able to use those to better understand the goals of the software and the intentions of the developers.

If you choose to use pen testing as an adjunct to threat modeling, the most important thing is to ensure that you’re aligned with the penetration testers regarding what’s in scope. If they come back with a list of SQL injections and cross-site scripting attacks, will you be happy? (Either a yes or a no is fi ne as long as you’re clear with the pen testers. An “I don’t know” is likely to leave you unhappy.)

Checking Code You Acquire

Most of this book is focused on what you’re building. But the reality is that much of the code in today’s products comes from elsewhere, in either binary or source form. And not all of it comes with well-formed security operations guidance and non-requirements. Less of it comes with explicit threat models telling you what the creators worried about. This section shows you how to threat model code you’ve acquired, so you can validate if threats are addressed. The steps here are aligned with the four-stage framework expressed in the Introduction and in Chapter 1, “Dive In and Threat Model!”. You’ll use a variety of tools to learn about the software you’re looking at, create a model of the software from what you’ve learned, and then analyze that model. From there, standard opera- tional approaches to addressing threats can come into play. You can apply these techniques to code you acquire from outside, or if you’re an operations team, to code which is handed to you for deployment. This section is inspired by work by consultant Ollie Whitehouse (Whitehouse, 2013).

N O T E The approaches here assume you do not have the time or the skills to de- compile or reverse engineer the binaries. Of course, if you do, you have more options

and the capability to dig deeper. If you have source, you can of course compile it, and

so the binary approaches will work, and may be easier (for example, fi nd listening

ports or account information by running the installer). Of course, that dynamic analy-

sis has limitations; if on the third Wednesday of the month, the software opens a back-

door, bad luck if you start the next day and trust runtime analysis. This section also

assumes you’re looking at software which you have reason to believe you can trust.

(For example, unexpected binaries on your USB drives may well be malware.)

www.it-ebooks.info

Chapter 10 ■ Validating That Threats Are Addressed 193

c10.indd 09:44:43:AM 01/09/2014 Page 193

Constructing a Software Model

Even if the product comes with an architecture diagram, it may not suffi ce for threat modeling. For example, it probably doesn’t show trust boundaries. So you’ll need to construct a model that’s useful for threat analysis. To do that you need to dig into a set of questions, and then synthesize. In theory, you’ll want to move from the outside in, enumerating architectural aspects of the software, including the following:

■ Accounts and processes:

■ New accounts created

■ Running processes

■ Start-up processes

■ Invoked or spawned processes

■ Listening ports:

■ Sockets

■ RPC

■ Web

■ Local inter-process communication

■ Administrative interfaces:

■ Documented

■ Account recovery

■ Service personnel accounts

■ Web and database implementation

■ Technical dependencies and platform information:

■ Changes to OS (for appliances/virtual machines)

■ Firewall rule changes

■ Permission changes

■ Auto-updater status

■ Unpatched vulnerabilities

If the software is delivered in binary form, you can use unix tools like ps, netstat, fi nd (with the –newer option), or their equivalents on your platform to fi nd listeners and processes. Similarly, the operating system can show you new accounts and start-up processes. Tools like Sysinternals’ Autoruns on Windows can walk through the many possible ways to start a program on boot. Administrative interfaces are probably documented, although it’s unfortunately

www.it-ebooks.info

194 Part III ■ Managing and Addressing Threats

c10.indd 09:44:43:AM 01/09/2014 Page 194

common to have backdoors for service or recovery, so you can look into how you’d get in “if you forget your password.”

With this information, you can start to construct a software model. The com- ponents are the trust boundaries, and the processes are the processes within each. The listening ports get drawn as one end of data fl ows. If present, you’ll need to dig deeper into web and databases, in technology-appropriate ways, to better understand what’s happening in those complex subsystems. The admin- istrative interfaces can add additional trust boundaries and possibly data fl ows. Technical dependencies (such as a web server or framework) may also introduce new processes or data fl ows, and the approach here can be applied recursively to those dependencies. There’s one other aspect to check for dependencies: Are the dependencies up to date with security fi xes? Searching a database like the US National Vulnerability Database for the dependency can give you an idea of what’s been discovered in the version you have, or you can use free tools such as nmap or Metasploit to test it.

All of the preceding can be performed by any systems administrator who’s willing to learn a bit about new tools. If you also have source code, it may be possible to, and even worth, digging in. You may learn about additional accounts where Windows impersonates, or unix sets uid. System calls such as fork and exec will show you where processes are spawned. Open, read, and especially socket and listen calls can reveal trust boundaries. (There are a near-infi nite number of higher-level variants which implicitly create a socket and listen.) Finding extra login interfaces is easier with the code, but will likely require more than just grepping.

Using the Software Model

When you have a model of the software you acquired, you can apply the tech- niques in Part II of this book to bear to fi nd threats. You can look to see if they are appropriately mitigated. If you’re looking for threats during a software evaluation or acquisition phase, you have a number of choices about the threats you fi nd that are not mitigated to your satisfaction. Those choices include:

■ Bring them to the attention of the software producer.

■ Look for an alternate package.

■ Mitigate them yourself.

If you are bringing them to the software producer, be prepared for a dis- cussion of the correct requirements, the tradeoffs involved in a fi x, or other disagreements about the threats you’ve found. (For example, many proposed “improvements” to Windows permissions fail to consider what may break if those changes are made.)

www.it-ebooks.info

Chapter 10 ■ Validating That Threats Are Addressed 195

c10.indd 09:44:43:AM 01/09/2014 Page 195

Some backwards looking vendors may even threaten you, asserting that you’ve violated the license, although fortunately this is becoming less common.

If the producer is open source, don’t be surprised if the answer is “we’re accepting patches.”

If you’re looking at threats after the software has been acquired, selected or deployed, your ability to select an alternative is dramatically lower. (Once again, threat modeling early gives you more fl exibility.) Bringing them to the vendor or mitigating them yourself are your main choices.

Mitigating issues yourself means applying the “operational” mitigations found in Chapter 8, “Defensive Tactics and Technologies.” The most common forms of this are “slap a fi rewall in front of it,” and tunnel the network traffi c over SSL or SSH.

You can make all this work a lot easier by looking for operational security guides and threat model documents as you select components. Components that come with such documentation are likely to require a lot less work on your part.

QA’ing Threat Modeling

While it is common for security enthusiasts to claim that you’re never done threat modeling, at some point you need to ship, deploy, or otherwise deliver the software, and you may or may not be planning to do another version. As you get close to fi nishing (say, feature complete), you’ll want to be able to close out the threat modeling work. If you use checklists or some other means of track- ing what needs to be done to pass through the gate and call it feature complete, you should add threat model verifi cation to the checklist. Verifying the threat model consists of ensuring the following:

■ The model actually matches reality closely enough.

■ Everything in the threat list is addressed and the threat model portion of the test plan is complete.

■ Threat model bugs are closed.

Model/Reality Conformance

As you fi nish addressing threats, you need to ensure that the threats you’re addressing were found using a model that relates to what you built. In other words, if you did three major architecture revamps between the last time you did an architectural diagram of the system and now, the list of threats you found might not be relevant to what you’ve built. Therefore, you need to check whether your model is close enough to reality to be useful. Ideally, you do this along

www.it-ebooks.info

196 Part III ■ Managing and Addressing Threats

c10.indd 09:44:43:AM 01/09/2014 Page 196

the way, by including threat modeling in your list of activities to do during an architectural redesign or major refactoring work. This also applies to deploy- ing complex systems. The complexities of deployment often lead to on-the-fl y changes. If those are substantial (say, turning off a fi rewall), then the threat model probably needs a revamp.

If your threat model leads to substantial redesign or architecture changes to address threats, “closing that loop” is tremendously important. If that’s the goal, make sure you re-check the software model against the code around the time the code is getting checked in, or when the new systems are being deployed. Ideally, bring the people who made the changes into a room, and have them explain the changes that were made and the new security features.

N O T E A meeting like this can sometimes devolve into security fi nding new threats against the new system. That may be acceptable, or it may seem like moving the goal-

posts. Upfront agreement on the meeting goals will reduce contention.

Task and Process Completion

Have you gone through each threat, and decided on a strategy and a tactic for addressing it? That might be to accept the risk and monitor, or it might be to use an operating system feature to manage it. You need to track these and ensure that you do an appropriate amount of work to avoid any falling through the cracks. (You might say that anything of severity less than some bar isn’t worth the effort to track.) Good threat modeling tools make it easy to fi le a bug per threat. If you’re using something without that feature, you need to fi le bugs manually.

There are two reasons why threats should lead to bugs that are like your other bugs, rather than a separate document. First, anyone shipping software has a way to manage bugs; and if you want your security issues managed, making them bugs puts them into the same machinery your organization is already using to assure quality. Second, bugs act as an exit point from threat modeling, allowing the normal machinery to take over, and enabling you to say, “We’re done threat modeling that.”

Bug Checking

As you get closer to shipping, review the mitigation test bugs, ensuring that you’ve closed each one. This is where a tag such as tmtest is really helpful. Bugs that haven’t been closed should be triaged like other bugs (with appropriate attention to the gravity of the bug), and fi xed or, if appropriate, moved to the next revision.

www.it-ebooks.info

Chapter 10 ■ Validating That Threats Are Addressed 197

c10.indd 09:44:43:AM 01/09/2014 Page 197

Process Aspects of Addressing Threats

There are a few fi nal aspects of how testing and threat modeling complement each other, and a few red fl ags that testers can use to quickly fi nd issues, includ- ing a belief that data is “validated” where people are making assumptions.

Threat Modeling Empowers Testing; Testing Empowers Threat Modeling

Writing code is hard; thinking through all the things needed to make the code work leaves little space in the brain for thinking about anything else, including what might go wrong. This level of focus and concentration on the problem at hand is part of why engineers can get so upset at interruptions: The problem at hand is complex enough to require full attention, and there’s no room for minor issues like eating properly. There’s “no room” in two senses: rational and emotional. In the rational sense, developers usually want to focus on developing great code. Thinking about all the ways to make the code better, faster, more capable, more elegant, and so on, requires less of a mental shift than thinking about what will go wrong. Many great developers have learned that they need to think about writing testable code and collaborating with testers, but that doesn’t mean they’re great at fi nding what will go wrong. The emotional sense of lacking room to think about what might go wrong means that after a lot of effort to make the code work, thinking about making it break is challenging.

Threat modeling has a natural ally in testers, especially when testing is seen as an “inferior” function to developing. If testing “owns” threat modeling, then test has a reason to be in architecture meetings. They need to be there to talk about how to threat model, and to start conducting tests early. If threat model- ing is the reason testers are brought in early, then testers have a reason to drive effective threat modeling processes.

Validation/Transformation

It’s common to see threat models that assert, incorrectly, that this input or that has been “validated.” This claim should be a red fl ag for testers, because it is never completely true. The data may well have been validated for some purpose or purposes. For example, is the following data valid? http://www.example .org/this/url/will/pwn/you.

It’s a valid URL, in accordance with RFC 1738. It uses a domain that is nomi- nally reserved for example use. Were it a real URL, a path this/url/will/ pwn/you is probably unsafe to visit. It’s easy to construct similar examples.

www.it-ebooks.info

198 Part III ■ Managing and Addressing Threats

c10.indd 09:44:43:AM 01/09/2014 Page 198

For example, the e-mail address [email protected] is valid (see RFC 822), but just try convincing many web forms of that. Many fi lter out the plus sign to make SQL injection attacks harder to carry out, along with the single quote character ('), which annoys a great many Irish people with names like O’Malley or O’Leary. Therefore, data can only be validated for a particular purpose or, better, to comply with certain rules.

The other approach to data is to fi lter and transform it. For example, if you have a system that is taking input that will be displayed on a website, you can do so more safely by ensuring it is ASCII, eliminating everything but a known good set and transforming bracketed strings into approved HTML strings. Your known good set could be A–Z, 0–9 and some set of punctuation. The advantage to a fi lter and transform approach is safety by design. By fi ltering out everything but known good, and then transforming them into something that includes “dangerous” input, attack code will need to pass through multiple bugs to succeed.

Document Assumptions as You Go

As you threat model, you’ll fi nd yourself saying, “I assume that …” You should write those things down, and testers should test the assumptions. How to do that will vary according to the assumption. Generally, you can test the assump- tions by asking, “Could it ever not be true?” and “What can I break if this is false, incomplete, or an overgeneralization?”

This differs in a subtle but important way from the common prescription to “document all assumptions.” That advice leads people to try to document all assumptions as they start threat modeling, but what assumptions? When do you stop? Do you assume that no one will fi nd a new solution to the factoring problem that underlies many public key cryptography schemes? It’s usually a reasonable assumption, but documenting that in advance often feels like an exercise in pedantry. In contrast, documenting as you go is easier, constrained, and helps those who review the threat models.

Tables and Lists

Now that you know about the defensive tactics and technologies and the various strategies you can apply to manage risks, it’s time to learn about the blocking and tackling elements. If you think back to Chapter 7, many of the tables there have a bug ID as their last column. The tables here, therefore, start with a bug ID. A simple table might look something like Table 10-1.

www.it-ebooks.info

Chapter 10 ■ Validating That Threats Are Addressed 199

c10.indd 09:44:43:AM 01/09/2014 Page 199

Table 10-1: Tracking Bugs for Fixing

BUG ID THREAT

RISK MANAGEMENT TECHNIQUE

PRIORITIZATION APPROACH TACTIC TESTER DONE?

4556 Orders not checked at server

Design change Fix now to address, avoid dependencies.

Code changes

Alice

4558 Ensure all changes to server code are pulled from source control so changes are accountable.

Operational change

Wait and see. Deploy ment tooling change

Bob No

4559 Investigate moving con- trols to busi- ness logic, which is less accessible to attackers.

Design change Wait and see. Include in user stories for next refactoring.

Bob no

However, you’ll notice that these threats are all being addressed, or have a strategy for addressing them. As you’ll recall, this isn’t always possible, so you might have additional tables for accepted risks (see Table 10-3) and transferred risks (see Table 10-4). That leaves out avoided risks, which are rarely worth track- ing as bugs. If you maintain architecture documentation, you might include the insecure designs that were avoided. You can use a table like Table 10-2 to track how you’re handling various risks, which can help you get an overview of where you stand overall. (With the right fi elds, you can also extract such a table from your bug tracking system.) In this list and the lists that follow, italics are used for fi elds that you might choose to include if you’re a more process-intensive shop.

Table 10-2 shows the following:

■ Bug ID

■ Threat

■ Risk management approach

■ Risk management technique or building blocks

■ Is it done?

■ Test IDs if you have test code (optional).

www.it-ebooks.info

200 Part III ■ Managing and Addressing Threats

c10.indd 09:44:43:AM 01/09/2014 Page 200

Table 10-2: Overall Threat Modeling Bug Tracking (Example)

BUG ID THREAT

RISK MANAGEMENT APPROACH

TECHNIQUE/ BUILDING BLOCKS DONE?

1234 Criminals send- ing money orders

Avoid Don’t accept checks/ACH/ money orders.

Yes!

4556 Orders not checked at server

Address Fix now. Check with Alice.

1235 People will mistype URL, visit phishing site.

Accept Document in security advice page, advising bookmark, since we can’t fi x the browsers.

Yes

1236 People will order things they don’t want.

Transfer Terms of service state that all orders are fi nal.

Yes

In tracking accepted risks, the key deliverable is an understanding of the risks you’re accepting, so that executives can have a single view. You might choose to sort such a table by bug ID, cost, or the business owner who has accepted the risk. Of course, as you’ve learned, putting a dollar value on threats can be challenging. If you do put a cost on threats, then unlike most of the other tables, the accepted risks table can actually be summarized or totaled.

As shown in Table 10-3, you should track at least the following:

■ Bug ID

■ Threat

■ Cost or impact estimate

■ Who made the estimate and how (optional)

■ Why the risk is accepted

■ Who accepted it

■ The sign off procedure that was followed (optional)

www.it-ebooks.info

Chapter 10 ■ Validating That Threats Are Addressed 201

c10.indd 09:44:43:AM 01/09/2014 Page 201

Table 10-3: Accepted Risks (Example)

BUG ID THREAT COST/IMPACT ESTIMATE

REASON TO ACCEPT

BUSINESS SIGN OFF FROM

1237 A janitor will plug a keylogger into the CEO’s desktop.

High The CEO still has a desktop.

IT director

1238

… … … …

Total (above) $1,000,000 Charlene, Dave

The “transferred to” column may or may not be needed. You may fi nd that all risks are transferred to the customer, but you might fi nd that some risks are transferred to other parties. For example, if your product has a fi le-sharing feature, risk may be transferred to copyright owners; or if your product has a messaging feature, you may create a new channel for spam. Of course, in prac- tice, those risks may fall on your customers.

If you want to track to whom it’s transferred, you can do so as shown in Table 10-4, which includes the following:

■ Bug ID

■ Threat

■ Why you can’t fi x it

■ To whom it’s transferred (optional)

■ The way it’s being transferred

■ Whether the transfer mechanism is completed

Table 10-4: Transferred Risks (Example)

BUG ID THREAT REASON WE CAN’T FIX FORM OF TRANSFER DONE?

1238 Insiders Need an admin role

Non-requirements, pre- sented in security opera- tions guide

Yes

1239 Buff er overfl ow in our custom document format

After careful redesign and fuzzing, resid- ual risks exist.

Warning on opening document

No— requires user testing.

www.it-ebooks.info

202 Part III ■ Managing and Addressing Threats

c10.indd 09:44:43:AM 01/09/2014 Page 202

Summary

Like everything else in the development of software or the deployment of systems, threat models can be subjected to a quality assurance process. It’s important to “close the loop” and ensure that threats have been appropriately handled. Those tasks may fall outside what’s normally thought of as threat modeling. How you handle them will vary according to your organization.

Thus, there’s a need to integrate threat model testing into your test process, and to test that each threat is addressed. You’ll also want to look for variants and second- or third-order attacks as you test for threats.

When you perform quality assurance, you’ll want to confi rm that the soft- ware model conforms to what you ended up building, deploying or acquiring. Modeling the architecture of software you acquire involves a different process than software you create in-house. In addition, you should verify that each task and process associated with threat modeling was completed, and check the threat model bugs to ensure that each was handled appropriately.

There’s an interesting overlap between “the security mindset” and the way many testers approach their work. This may lead to an interesting career oppor- tunity for testers. Also, at larger organizations, this offers an opportunity for security and testers to create a virtuous circle of mutual reinforcement.

The term “validated” is, by itself, a red fl ag for security analysis and testing. Without a clear statement of what the validation is for, it is impossible to test whether it’s correct, and whether the assumption (or group of assumptions) is accurate or shared by everyone touching the supposedly validated data. “The exact purpose data is being validated” is often left as an assumption, rather than being made explicit. There are many others, and all of them should be validated as you threat model, and tested at an appropriate time.

www.it-ebooks.info

203

c11.indd 01:56:23:PM 01/08/2014 Page 203

This chapter covers tools to help you threat model. Tooling can help threat modeling in a number of ways. It can help you create better models, or create models more fl uidly. Tools can help you remember to engage in various steps, or provide assistance performing those steps. Tools can help create a more legible or even beautiful threat model document. Tools can help you check your threat model for completeness. Finally, tools can help you create actionable output from a threat model.

Tools can also act as a constraint. You may fi nd yourself stymied by usability issues, such as fi elds you’re unsure how to fi ll out. Or you might fi nd that a tool cramps your style. Some trade-offs are unavoidable as tools are created, so the chapter starts with general tools that are useful in threat modeling, and then progresses to more specialized tools.

A few disclosures: I do not have personal experience with each tool described here, and some of the tools I created myself. (Those are treated at greater length, because there’s less risk of me insulting the authors.)

This chapter starts by describing some generally useful tools and how to apply them to threat modeling. You’ll then learn about the open-source tools that are available, followed by commercial tools. The chapter closes with a few words about tools that don’t yet exist.

C H A P T E R

11

Threat Modeling Tools

www.it-ebooks.info

204 Part III ■ Managing and Addressing Threats

c11.indd 01:56:23:PM 01/08/2014 Page 204

Generally Useful Tools

This section discusses tools that are not specialized for threat modeling but can be tremendously useful. It covers a few of the more useful tools to encourage you to think about the tools you already use and with which you are familiar.

Whiteboards

I can hardly imagine threat modeling without a whiteboard. No technology I’ve used has the immediacy, fl exibility, and visibility to a group than a whiteboard when iteratively drawing system architecture. Whiteboards also have the advan- tage of transience—drawing on paper just isn’t the same. On a whiteboard, no one tries to correct details such as a line not being connected properly, so the discussion can be focused on how the system actually works.

For distributed teams, a webcam focused on a whiteboard may work, or you may have “virtual whiteboarding” technologies that work for you.

Offi ce Suites

Microsoft Offi ce contains a number of tools that are very useful in threat modeling. Word is a great tool for recording threats in free-form. What to record is dependent on the approach you’ve chosen. Excel can be used for issue tracking and status. Visio is great for turning whiteboards into more precise documents. Of course, Offi ce is one of several suites with word processing, spreadsheet, and drawing functionality. The only caveats would be the limitations of the tools. The document tool should be more than text—a feature such as embedded images is extremely useful. Similarly, use a vector drawing tool that enables you to move symbols as symbols. Automatic connector management is also super-useful, and of course this feature is not unique to Visio.

To state the obvious, Microsoft Word, Excel, and Visio are commercially licensed tools.

Bug-Tracking Systems

Whatever bug-tracking system you use should also be used to track threats. A good bug from threat modeling can take many forms. The form you use will infl uence how you title and discuss bugs, and there is no universally right way to approach it. (The right way is the way that works best for you and your organization.) The title could express any of the following:

■ The threat itself: Here the bug title is of a form such as “an attacker can threaten the component” or “the component is vulnerable to threat.”

www.it-ebooks.info

Chapter 11 ■ Threat Modeling Tools 205

c11.indd 01:56:23:PM 01/08/2014 Page 205

For example, “the front end is vulnerable to spoofi ng because we use reusable passwords.”

■ The mitigation: Here, the bug title is of a form such as “the component needs mitigation.” For example, “the front end needs to run only over SSH.” In the text of the bug, you should also explain the threat.

■ The need to test a mitigation: This is what you can title a bug if someone says, “Oh, the front end isn’t vulnerable to that.” Rather than absorb time in the meeting to discuss or check the threat, fi le a bug, “Test front end vulnerability to threat” and ensure that there are good tests for the bug.

■ The need to validate an assumption: These bugs are fi led to ensure that someone follows up on an assumption you discover while threat modeling, and on which you depend for a security property. The bug should have a title such as “security depends on assumption A” or “security property X of component Y depends on assumption Z.” For example, “Security depends on the assumption that no one would ever fi nd the key in the fake rock that looks exactly like the rocks at our last house.”

■ Other tracking items: You should treat the preceding items as sugges- tions, not a form into which all bugs need to fi t. If you fi nd something worth tracking, fi le a bug.

When tracking security bugs from threat modeling, there are a few fi elds that can make running queries and analysis more reliable. These include whether the bug is a security bug, whether it’s “stop ship,” and how the bug was found (for example, threat modeling, fuzzing, code review, customer report). You can also check the tables in Chapter 7, “Processing and Managing Threats,” and Chapter 9, “Trade-Offs When Addressing Threats,” for useful fi elds. For example, you might want a risk management approach fi eld whose values could be avoid, address, accept, or transfer.

The right fi elds to use will depend in large part on the queries you want to run, which of course depend on the questions you want to ask. Some questions you might want to ask include the following:

■ Do we have any open security bugs?

■ Do we have any open threat modeling bugs?

■ Do we have any high-severity threat modeling bugs left to fi x?

■ How much risk are we transferring to end-users in the security operations guide or via warning dialogs?

■ What department head has signed off on the largest business risk? Which department head has signed off on the most risks?

www.it-ebooks.info

206 Part III ■ Managing and Addressing Threats

c11.indd 01:56:23:PM 01/08/2014 Page 206

Open-Source Tools

A variety of open-source tools for threat modeling are available. The open source tools illustrate some of the challenges in creating a high-quality threat modeling tool.

TRIKE

There are two tools named TRIKE. The fi rst was a standalone desktop tool, written in Smalltalk. That tool is no longer being maintained, and TRIKE is now implemented in a spreadsheet. According to documentation, it works best in Excel 2011 for the Macintosh (Trike, 2013). TRIKE is sometimes referred to as “OctoTrike.”

TRIKE does not fi t cleanly into the four-stage framework defi ned in this book. The TRIKE spreadsheet contains 19 pages, which are grouped as follows: one overview, seven main threat pages (actors, data model, intended actions, con- nections, protocols, threats, and security objectives), four record-keeping pages (use case index, use case details, document index, and development team) and seven reference sheets (actor types, data types, action, network layers, meaning- ful threats, intended response, and guide words). As of this writing, the help spreadsheet appears to be a reference document, not an introduction of the system.

SeaMonster

SeaMonster is an Eclipse-based attack tree and misuse case tool that was devel- oped by students at the Norwegian University of Science and Technology. It appears to be abandoned since 2010 (SeaMonster, 2013). The code is still available.

Elevation of Privilege

Elevation of Privilege (the game) is designed to be the easy way to get started threat modeling. It works by inviting individuals to participate in a game. The game consists of 74 physical playing cards in six suits, named for the STRIDE threats, with most suits having cards 2 through Ace. Two suits have fewer cards in order to avoid redundant threats, and it was challenging to fi nd broadly applicable threat instances that were easily explained on a card. Each card has a specifi c instance of a STRIDE threat. For example, the 6 of Tampering reads “An attacker can write to a data store your code relies on.” Another example card is shown in Figure 11-1.

www.it-ebooks.info

Chapter 11 ■ Threat Modeling Tools 207

c11.indd 01:56:23:PM 01/08/2014 Page 207

Figure 11-1: An Elevation of Privilege card

The Elevation of Privilege fi les can be downloaded from http://www.microsoft.com/ sdl/adopt/eop.aspx. Before starting Elevation of Privilege, participants or the game organizer create a diagram of a system being modeled. People then come together for a game. The organizer explains the rules, and may ask people to “put their skepticism on hold.” The game starts by dealing out the deck, and is then structured into turns. The fi rst card played is always the 3 of Tampering. Play proceeds around the table in hands.

Each hand starts with a player selecting a suit to lead and playing in that suit. Each player plays by selecting a card and connecting it to the diagram. The player must play in the suit that was led if they have a card in that suit. If they don’t, they may play any card. When play has gone once around the table, the hand ends. The player who played the highest card wins the hand. The highest card is either in the suit that was led, or, if a card in the Elevation of Privilege suit card was played, the highest card played from the EoP played wins the hand. (All Elevation of Privilege threat cards are higher ranked than the suit that was led,

www.it-ebooks.info

208 Part III ■ Managing and Addressing Threats

c11.indd 01:56:23:PM 01/08/2014 Page 208

and only Elevation of Privilege cards can win when someone leads in another suit.) Players get a point for connecting the threat on their card to the diagram with a “buggable threat,” and a point for winning the hand by playing the highest card either in the suit that was led or in EoP. Any EoP card trumps the suit that was led. To encourage creativity, each ace card says “You’ve invented a new threat,” and the threats are enumerated on cards included in the pack. The game ends either when time allocated has elapsed or when all the cards have been played. The winner is the player with the most points.

A buggable threat is one a team identifi es and is willing to fi le a bug for. It’s a simple and implicit element of most software development. Some teams may fi nd it more helpful to ask “would we add that to the backlog?” However you want to approach it (in the context of the game), you want an understandable and shared bar to test threats, so that you focus on fi nding the good ones.

Games are less threatening than “serious” work, and they provide structure and hints to the beginner, enabling new players to fi nd a threat based on the cards in their hands. The game is also intended to help players fi nd a fl ow state, a concept that is covered in depth in Chapter 19, “Architecting for Success.” EoP is covered in greater depth in my paper Elevation of Privilege: Drawing Developers into Threat Modeling [Shostack, 2012].

Microsoft makes the fi les (source and PDF) available under a Creative Commons BY-3.0 license, allowing you to take it, modify it, make derivative works, and even sell them.

Commercial Tools

Here are a few commercially licensed threat modeling tools. I mention a few commercial tools as examples, but caveat emptor.

ThreatModeler

ThreatModeler from MyAppSecurity.com is a defense-oriented tool based on data elements, roles, and components. It uses a set of attack libraries, including the MITRE CAPEC (see Chapter 4, “Attack Trees”), the WASC threat classifi ca- tion, and others. The tool generates attack trees with the component as the root, requirements that can be violated as a fi rst level of subnode, and then threats and attacks as the next layers. According to the documentation, ThreatModeler is intended to be used by architects, developers, security professionals, QA professionals, or senior executives. ThreatModeler requires Windows.

Corporate Threat Modeller

Corporate Threat Modeller from SensePost is a tool built to support a method- ology designed after an analysis of the strengths and weaknesses of a number

www.it-ebooks.info

Chapter 11 ■ Threat Modeling Tools 209

c11.indd 01:56:23:PM 01/08/2014 Page 209

of threat modeling approaches. Those approaches included threat trees and OCTAVE, a US-CERT-originated system for threat modeling a business (White, 2010). The analysis also looked at Microsoft’s SDL Threat Modeling Tool v3, and the Microsoft “IT Infrastructure Threat Modeling Guide,” (McRee, 2009) which shows how to use STRIDE-per-element to threat model IT infrastructure.

The Corporate Threat Modeler was explicitly designed for consultants. Insofar as it was developed with an explicitly stated target user (not “everyone”), it is one of the most interesting tools on the market. The approach starts with an architectural overview, and then applies a somewhat complex risk equation. The tool is free to download.

SecurITree

SecurITree is threat risk software from Amenaza Technologies, which launched in 2007 to positive reviews (SC Magazine, 2007). The product seems like a well thought through tool for constructing, managing and interpreting threat trees. It contains not only the ability to manage trees, but a set of ways to fi lter those trees. Each node in the tree has behavioral/capability indicators: a cost to execute, noticability, and a technical ability. It also has impact/attacker benefi t indicators of attacker gain and victim loss, along with stored notes for a node or subtree. You can fi lter the tree based on a given attacker ability. SecurITree comes with a library of threat trees, which is likely to help its customers get to the interesting part of the threat modeling work faster. SecurITree also includes some excellent screencast-delivered training (Ingoldsby, 2009). SecurITree runs on Windows, Mac, and Linux.

Little-JIL

If you’re making use of threat trees at a research institution, the Little-JIL soft- ware may be helpful. “Little-JIL is a graphical language for defi ning processes that coordinate the activities of autonomous agents and their use of resources during the performance of a task.” It has been used for creating an elections process model and a set of fault trees for that model (Simidchieva, 2010). The full fault trees are available as a graphML model. The software used to create and process the models may be freely used at research institutions (Laser, undated).

Microsoft’s SDL Threat Modeling Tool

Microsoft has shipped at least four families of threat modeling tools. They are the Elevation of Privilege card game, the SDL Threat Modeling Tool v3, the Threat Analysis and Modeling Tool, and the Threat Modeling Tool v1 and 2. I was the project lead for Elevation of Privilege and the SDL Threat Modeling Tool v3 and 3.1. The currently available SDL Threat Modeling Tool is (or has been) available free from Microsoft.

www.it-ebooks.info

210 Part III ■ Managing and Addressing Threats

c11.indd 01:56:23:PM 01/08/2014 Page 210

The SDL Threat Modeling Tool v3 was designed in reaction to the complexities and usability issues encountered when engineers who were not threat model- ing experts tried to use the older tools. It was the fi rst tool designed around the four-stage framework. The tool has four major screens, designed around the tasks that naturally fi t together: Draw Diagrams, Analyze Model, Describe Environment, and Generate Reports. The Draw Diagrams screen, shown in Figure 11-2, includes both the capability to draw diagrams with a constrained Visio stencil set and a diagram validation section with heuristics. The Analyze Model screen, shown in Figure 11-3, is automatically fi lled out with threats according to STRIDE-per-element.

Each threat instance contains a set of guiding questions to help engineers think through the threat, and an area to record the threat, mitigation, to track whether work on the threat is complete, and to fi le a bug. The Describe Environment screen is something of a catch-all to track assumptions, external notes and the context of the threat model. The Reports screen includes an all-up report, an open issues report, a list of bugs, and a diagrams-only report intended to facilitate printing. The tool also contains a manual, a sample threat model (for the tool itself), and a getting started guide, all accessible via the Help menu.

As shown in Figure 11-2, the main Draw Diagrams screen contains the following, clockwise from upper left (excluding the menu):

■ The diagrams control, enabling you to create sub-diagrams

■ The Visio shapes you can use as diagram elements

■ The “default” diagram (discussed in the next paragraph)

■ The numbered Screens control

■ Diagram validation (feedback)

■ A help pane

The default diagram is present because human factor testing has shown that less experienced threat modelers are sometimes stymied by a blank screen. Providing them with a starting diagram serves two purposes. One, it dem- onstrates what is expected in that space. Two, rather than needing to create a diagram, a novice can modify what’s there, which is an easier task.

One other feature worth mentioning from Figure 11-2 is the help fi eld. Generally, help is a menu option that software engineers ignore, because they believe they’re too smart to need to read what they expect will be a badly written help fi le. Therefore, the tool has basic help onscreen.

The Analyze Model screen shown in Figure 11-3 has two panes. The left pane is a list of diagram elements and the threats associated with them, presented as a tree with a single level of branches. The right pane is titled with the element name (“Results”) and a description of the element. Under that is a reminder of the STRIDE-per-element threats to which it is subject, and an option to not

www.it-ebooks.info

Chapter 11 ■ Threat Modeling Tools 211

c11.indd 01:56:23:PM 01/08/2014 Page 211

generate threat placeholders, with a reason box. A large portion of the screen is devoted to onscreen guidance: “Some questions to ask about this threat type.” The guidance is specifi c to threat and element category (process, data store, data fl ow, or external entity).

Figure 11-2: The SDL Threat Modeling Tool “Draw Diagrams” screen

There’s also a command link to “Certify that there are no threats of this type.” The word certify was carefully chosen to convey gravity. The last element on the screen is where the threat modeler describes the threat impact, and how it will be mitigated. Most of that is done in two large text entry boxes, but there is also a Finished check box, a “fi le bug” command link, and a completion bar. The completion bar (shown empty, under the word completion) fi lls out in four segments to encourage text entry in the Impact and Solution fi elds, as well as checking “fi nished” and fi ling a bug. There is also an Add Threat command link in case someone discovers an additional tampering threat against the results data fl ow.

The bug fi ling is intentionally abstracted into an API, and the tool ships with sample code to connect to a variety of bug tracking systems, or allow you to connect to whatever you use. When developing the tool, we intentionally spent time to create an API and to ship it under the Microsoft Public License (an Open

www.it-ebooks.info

212 Part III ■ Managing and Addressing Threats

c11.indd 01:56:23:PM 01/08/2014 Page 212

Source Initiative Approved License), because bugs are a critical part of ensuring that the threat model leads to something actionable.

Figure 11-3: The SDL TM Tool Analyze Model screen

N O T E Within the Draw Diagrams screen, the term “validation” causes consterna- tion when diagrams don’t conform. For example, one heuristic is to show where

all data comes from. That can be addressed by including three extra elements: an

installer external entity, a data fl ow, and a trust boundary. Doing so is not obvious,

and requires roughly 10 steps. In a future version, it would be great to see diagram

feedback that includes advice on how to address each. Also, boxed trust boundar-

ies would be one of several improvements that could be made to the shapes in the

default set. Others include rounded rectangles for processes, less curvy lines, and

better positioning of text.

The SDL TM Tool v3.1 series is a no-cost download from Microsoft (www.microsoft.com/security/sdl/adopt/threatmodeling.aspx) and it requires Visio 2007 or 2010 to use. The tool is compatible with the Visio 2010

www.it-ebooks.info

Chapter 11 ■ Threat Modeling Tools 213

c11.indd 01:56:23:PM 01/08/2014 Page 213

evaluation version. Newer versions of the tool may become available (after this book goes to press) with different dependencies.

Tools That Don’t Exist Yet

There are two categories of features that people often ask for that are worth a brief discussion: automated model creation and automated threat identifi ca- tion. A great many people want tools that can take a piece of software that’s already been written and extract a data fl ow or other architectural diagram. This is an attractive goal, and one that might be feasible for programs written in strongly typed languages. Marwan Abi-Antoun has done some work show- ing how to extract data fl ow diagrams for Java (Abi-Antoun, 2009). (The system with open code and published DFDs he found to use for testing was only 3,000 lines of code.) If technology to do this is further developed, it will present great value to threat modeling, but also create a temptation to not perform any threat modeling or analysis until late in a project. Threat modeling after the code has been completed limits options for addressing issues. This is discussed further in Chapter 7, “Processing and Managing Threats” and Chapter 17, “Bringing Threat Modeling to Your Organization.”

Similarly, tools that can take a diagram or other model and produce lists of threats would be lovely. A Spanish graduate student, Guifré Ruiz, and colleagues have created a fi rst version of such a tool (Ruiz, 2012). However, these tools carry a risk that security analysis will focus only on known threats from an attack library. Such tools cannot (currently) analogize from closely related threats the way an experienced person can. Threat analysis that could reliably extend that knowledge to prevent new systems from making mistakes others have made would be a useful step forward. As more such tools are developed, it will be important to consider the balance between human and automated security design analysis. After all, to the extent that you need software engineers to create new functionality, that new functionality and the new combinations that result may expose new threats. It’s not impossible to imagine a tool that would fi nd threats against code not yet written, but it’s hard to imagine one that would do so as comprehensively as an expert threat modeler.

Summary

A wide variety of tools are available for threat modeling. General-purpose tools such as whiteboards and bug-tracking systems can be very helpful, and tools such as word processors, spreadsheets, and diagramming tools can be used to

www.it-ebooks.info

214 Part III ■ Managing and Addressing Threats

c11.indd 01:56:23:PM 01/08/2014 Page 214

help you threat model. Also available are a variety of specialized threat modeling tools. Microsoft has shipped several of these free, including Elevation of Privilege and the SDL Threat Modeling Tool, and you can fi nd other commercial and open-source tools that may aid your efforts. There is also demand for tools that can automate model creation or threat identifi cation, although such tools may come at a high price if they appear to fi nd threats while missing new threats or are used too late in the development process.

www.it-ebooks.info

c12.indd 07:52:27:AM 01/15/2014 Page 215

Par t

IV Threat Modeling in Technologies

and Tricky Areas

Part IV is where this book moves away from threat modeling as a generic approach, and focuses on threat modeling of specifi c technologies and tricky areas. In other words, this part moves from a focus on technique to a focus on the repertoire you’ll need to address these tricky areas.

All of these technologies and areas (except requirements) share three proper- ties that make it worth discussing them in depth:

■ Systems will have similar threats.

■ Those threats and the approaches to mitigating them have been extensively worked through, so there’s no need to start from scratch.

■ Naïve mitigations fall victim to worked-through attacks. Therefore, you can abstract what’s been done in these areas into models, and you can learn the current practical state of the art in handling each.

The following chapters are included in this part:

■ Chapter 12: Requirements Cookbook lays out a set of security require- ments so that you don’t have to start your requirements from a blank slate, but can borrow and adapt. Much like the other chapters in this part, requirements are a tricky area where specifi c advice can help you.

■ Chapter 13: Web and Cloud Threats are the most like other threat model- ing, but with a few recurring threats to consider. (That is, while an awful

www.it-ebooks.info

216 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c12.indd 07:52:27:AM 01/15/2014 Page 216

lot of words have been written on the security properties of web and cloud, they’re actually only a little more complex to threat model than other operational environments.)

■ Chapter 14: Accounts and Identity are far more nuanced than web or cloud, and begin to intrude on the human world, where poor choices in design can cause people to avoid your service or work around your security measures.

■ Chapter 15: Human Factors and Usability issues are at the overlap of the human and technological worlds, and both the modeling of threats and how to address them are less developed. That makes them no less critical, only more of a challenge and opportunity for innovation.

■ Chapter 16: Threats to Cryptosystems is a chapter with more modest goals, not because cryptography is any harder than the other subjects, but because it’s easier to get wrong in ways that look fi ne under casual inspection. As such, this chapter aims to familiarize you with the world of cryptography and the threat terminology that is unique to the fi eld, relating it to the rest of the threats in the book.

www.it-ebooks.info

217

c12.indd 07:52:27:AM 01/15/2014 Page 217

Important threats violate important security requirements. Ideally, those require- ments are explicit, crisp, agreed-on within the development organization, and understood by customers and the people impacted by the system. Unfortunately, this is rarely the case. In part, that’s because requirements are very diffi cult to do well. That makes requirements a tedious way to start a project, and as the agile folks will tell you, YAGNI (“you ain’t gonna need it”)—so we should skip straight to user stories, right? Maybe, but maybe not.

As you discover threats, you’ll be forced to decide whether the threat mat- ters. Some of that decision will be based on a risk calculation, and some will be based on a requirements calculation. If your system is not designed to maintain security in the face of hostile administrators, then all effort spent on mitigating hostile administrators will be wasted.

That said, this chapter starts with an explanation of the cookbook approach and a discussion of the interplay of requirements, threats, and mitigations. You’ll then learn about ways to think about business requirements, and look at how to use common security frames to help with your requirements. (A frame here is a way of structuring how you look at a problem, while a framework is a breakdown which includes specifi c process steps.) The frames are “prevent/ detect/respond” and “people, process, technology,” with requirements in devel- opment contrasted with requirements in acquisition. Next you’ll learn how to use compliance frameworks and privacy to drive your requirements. You can use these sections to decide what sort of requirements you might need.

C H A P T E R

12

Requirements Cookbook

www.it-ebooks.info

218 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c12.indd 07:52:27:AM 01/15/2014 Page 218

Each of these sections may be more useful to product management than to developers. The chapter then delves deep into the more technology-centered STRIDE requirements. The STRIDE requirements are the most deeply technical and “actionable” of the requirements. You should not succumb to the temptation to make those the only requirements you consider. The chapter closes with a discussion of non-requirements.

Why a “Cookbook”?

A great many systems are available for requirements elicitation. Most of them (at best) skim over security. This chapter does not intend to replace your requirements approach, but to supplement it with a set of straw-man requirements, designed to be easily adapted to the specifi c needs of your system. The intent is, much like a cookbook, to give you ideas that you can easily turn into real “food.” Also like a cookbook, you can’t simply take what’s here (the raw ingredients) and serve it to your dinner guests; but you can use what’s here to prepare a scrumptious set of requirements. As you consider what you want to build, you can refer to this section for requirements that crystalize what you’re thinking about.

Of course, you’ll need more detail than the sample requirements can provide. For example, “Anonymous people can create/read/update/delete item” is a good starting point, but what sort of items can they create, read, update, or delete? Wikipedia has a nuanced set of answers for create, update, and delete, and another set of nuanced answers for what can be read. There’s a very different set of answers on Google.com or Whitehouse.gov. The requirements examples shown cannot include the local or specifi c knowledge needed to make them concrete.

Most of these starting points are grouped with several related starting points. To take some examples from the section on STRIDE authentication require- ments, after the requirement “Anonymous people can create/read/update/ delete items,” the next requirement is “All authenticated users can create/read /update/delete item,” followed by “An enumerated subset of users can create/ read/update/delete items.” In this case, you might well need all three require- ments expanded into your project’s authentication requirements. In contrast, the section on authentication strength includes “We will control the authentica- tion database” and “We will allow an outside organization (such as Facebook) to control our authentication database.” These are both reasonable choices that organizations make, but you can’t make both of them. The next require- ment splits the difference with “We will allow an outside organization (such as Facebook) to control part of our authentication database, such as ‘signed in users,’ but not administrators.”

www.it-ebooks.info

Chapter 12 ■ Requirements Cookbook 219

c12.indd 07:52:27:AM 01/15/2014 Page 219

The Interplay of Requirements, Threats, and Mitigations

The same sort of task interplay as discussed in the previous section takes place on a broader scale between threats, requirements, and mitigations. As threats are discovered, some of them will violate explicit requirements. Others will violate implicit requirements, offering an opportunity to improve the requirements list. Other threats may lead to discussion of whether they violate a requirement or not, again leading to possible clarifi cation. Thus, fi nding threats helps you iden- tify requirements. Discussion of threats may also lead to a discussion about the diffi culty of addressing a given threat. Such discussion can also feed back into requirements when a threat can’t be mitigated. Mitigations drive requirements far more often than requirements drive mitigations. (In fact, I can’t think of a case where a requirement drives a mitigation without a threat, except perhaps it’s a fi ne defi nition of compliance, and that may be why so many security profes- sionals resent compliance work.) This interplay is shown visually in Figure 12–1.

Requirements

Threats Mitigations

Threats help identify requirements

Real threats violate requirements Compliance

Impossible to mitigate implies non-requirement

Threats, mitigations interplay

Figure 12-1: The interplay of threats, requirements, and mitigations

You might have noticed a reductio ad absurdum attack on this perspective. That is, taken to an extreme, it quickly becomes ridiculous. For example, if a product were to declare that the network is trusted, and all network threats are irrelevant, that would be a pretty silly decision to make, and hard to justify in front of customers if the product handles any sensitive data. However, the extreme position isn’t what you should take. If you are new to threat modeling, be extremely cautious about removing requirements because you’re unsure how to mitigate the threats. To check your assumption that mitigation is impossible, you should plan to spend days to weeks investigating what others have done in similar circumstances. As you develop more experience in security, that investigation will go faster, as your toolbox will be more varied.

www.it-ebooks.info

220 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c12.indd 07:52:27:AM 01/15/2014 Page 220

Business Requirements

In this section you learn about the business requirements for security. These are the requirements that will make the most sense to business people. Some organizations may call these “goals,” or “mission,” or “vision.”

Outshining the Competition

Your organization may be in a situation in which security properties or features are either a customer requirement or a competitive differentiator. In those scenarios, you can start with a requirement or requirements selected from the following list:

1. The product is no less secure than the typical competitor.

2. The product is no less secure as measured by X than the typical competitor.

3. The product is no less secure as measured by X than market leader Y.

4. The product will ship fewer security updates than the competition. (This can incentivize hiding vulnerabilities, which you shouldn’t do.)

5. The product will have fewer exploitable vulnerabilities than the typical competitor.

6. The product will have fewer exploitable vulnerabilities than the market leader Y.

7. The product will be viewed as more secure than the typical competitor.

8. The product line will be viewed as more secure than the typical competitor.

9. We will be able to use security as a competitive advantage.

10. We will be able to use this security feature/property as a competitive advantage.

Industry Requirements

If your product is sold for a particular industry or use, there may be industry- specifi c requirements which apply. For example, if you’re building payment processing software, you’ll need to comply with industry rules. If you sell medi- cal devices, you’ll want to ensure that your devices have substantial defense in depth, as your ability to get them recertifi ed quickly may be controlled by law. If you’re building tools for emergency responders, they may come under particular attack. You should consider how your particular circumstances will drive your requirements.

www.it-ebooks.info

Chapter 12 ■ Requirements Cookbook 221

c12.indd 07:52:27:AM 01/15/2014 Page 221

Scenario-Driven Requirements

The use of stories, scenarios, or use cases can be a strong general approach to requirements elicitation, so it is natural to hope that security requirements might also be derived from them. That is a reasonable hope, but I am not aware of any structured approach for doing so for security requirements.

The primary challenge is that security is rarely a goal of a feature. When explaining what Alice will do as she goes about her day, few product managers say “and then we’ll force her to log in again, since her session timed out over lunch” or “and then Mallory will try to break into the product by. . .” Even for security features, such as access control, it’s rare to have a product manager defi ne a scenario like “Alice adds a group to allow write access to this fi le, and then tests to see whether the explicit deny rule for Bob still holds.”

Prevent/Detect/Respond as a Frame for Requirements

Prevent/detect/respond is a common way for organizations to think about operational security. I’m aware of a large bank that, having adopted this way of seeing the world, focuses most of its energy on response, assuming that its systems will be compromised, and focusing on reducing mean time to repair. Prevent/detect/respond can be used as a frame for thinking about requirements in development, to ensure that your technology addresses each area. It can also be used in technology acquisition or operations.

Prevention

STRIDE threat modeling is very focused on ensuring that you prevent threats. One element not well covered by STRIDE is vulnerabilities, and their manage- ment. Another is operational security approaches to prevention.

Operational Security Preventive Requirements

Operational isolation requirements focus on reducing attack surface by isolat- ing systems from each other.

1. All production systems will be isolated from the public Internet by fi rewalls.

2. All production systems will be isolated from internal development and operational systems (such as HR).

3. All traversals of isolation boundaries will be authenticated by something beyond IP and port numbers.

www.it-ebooks.info

222 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c12.indd 07:52:27:AM 01/15/2014 Page 222

4. All production systems will be deployed on VMs that are distinct from other production systems (that is, one application, one VM).

5. All cloud systems will be isolated from competitors. (This is easy to write, and hard to achieve.)

Least privilege requirements are focused on making it more diffi cult for an attacker to take advantage of an intrusion. Least privilege is easy to say and practically challenging to achieve. In the list that follows, “with privileges” means emphatically “as root/administrator,” but also those points in between a “normal” user and the most privileged.

1. No production application will run with privileges.

2. No production application will run with privileges without undergoing a threat model analysis and penetration test.

3. Any production application we create that requires privileges will have those privileges isolated into a small component.

4. Application acquisition will include a discussion of threat models and operational security guidance.

Account management for prevention includes account lifecycle (as discussed in Chapter 14, “Accounts and Identity”), when it’s made operational. Managing accounts across a small organization was challenging even before the days of cloud services, which make it even harder.

1. Account creation will be managed and tracked, and all service accounts will have a responsible person.

2. Accounts for people will be terminated when a person leaves.

3. Accounts will be periodically audited or reviewed to ensure that there is someone responsible for each.

Vulnerability Management

Vulnerability is a term of art that refers those accidental fl aws which can be exploited by an attacker. This is in contrast to features that can be abused. In other words, a vulnerability is something that everyone agrees ought to be fi xed. The exploitation of vulnerabilities can often be automated, so in order to prevent exploitation it’s important to consider the vulnerability lifecycle from discovery through coding a fi x, testing that fi x, and delivering it. If you develop software, some of the vulnerabilities discovered will be in your code, while others will be in code on which you depend. You’ll need to be able to handle both sorts. In each case, you’ll need to take the fi x, test it, and deliver it onwards to people in operations. Sometimes those operations people will be in the same organization,

www.it-ebooks.info

Chapter 12 ■ Requirements Cookbook 223

c12.indd 07:52:27:AM 01/15/2014 Page 223

but oftentimes they’ll be external. If you’re in operations, you’ll need to be able to discover vulnerability reports from your suppliers, and manage them. You may also get vulnerability reports about the systems you operate, and you’ll need to manage those as well.

Once you have a way to learn about the updates, you’ll need a way to fl ow those updates through to your software and/or deployed systems. If you develop complex software, good unit tests of the functionality you use in outside com- ponents will help you rapidly test security updates and identify issues that might come with them. Maybe that’s part of a continuous deployment strategy for your site, or perhaps it’s something like running a security response process the way Microsoft does (Rains, 2013) so your customers can learn about, receive, and deploy your updates in their environment.

Vulnerability Reports about Your Products or Systems

Properly managing inbound vulnerability reports is an important task, and you should consider how you’ll encourage people to report vulnerabilities to you. There is an alternative approach, which is “we will sue security researchers to inhibit discovery and reporting of security fl aws.” This backfi res. For example, after Cisco sued researcher Mike Lynn for exposing fl aws in its products, Cisco CSO John Stewart said “we did some silly things” and “we created a fi restorm” (McMillan, 2008).

Product vulnerability management requirements might include:

1. We will have a public policy encouraging the reporting of security fl aws.

2. We will have a public policy encouraging the reporting of security fl aws and make it easy to do so.

3. We will have a public policy encouraging the reporting of security fl aws by paying for them.

4. We will have a public policy encouraging the reporting of security fl aws in our online services.

5. We will have a public policy encouraging the reporting of security fl aws in our online services, and explain what testing is acceptable or unacceptable.

6. We will have automatic update functionality built into our product.

7. We will have automatic update functionality built into our product, and it will be on by default.

8. We will support only the latest version of this product, and security fi xes will require an update.

9. We will support only the latest version of this product, and security fi xes will require an update and those updates may include feature changes.

www.it-ebooks.info

224 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c12.indd 07:52:27:AM 01/15/2014 Page 224

10. We will have a public policy on support lifetime, and produce security fi xes for all currently supported products.

11. We will have a public channel for vulnerability announcements that is designed to satisfy those looking to sign up to lots of channels to track all the software they operate. As such, it will be nothing but vulnerability announcements.

Managing Vulnerabilities in External Code

It is important to track the components on which you depend and their security updates. This applies to both development and operations, and to both com- mercial and open-source components. Your acquisition process should include understanding how an organization notifi es customers of security updates. If there’s no mail list, RSS feed, or other mechanism established for security update notifi cations, that’s probably a problem. Some possible requirements include:

1. We will only discover security issues when they’re important enough for the media to talk about.

2. Each group will maintain its own list of dependencies.

3. We will maintain a single list of dependencies to track for software updates.

4. We will ensure that we track dependencies, and have a person assigned to reading the updates and generating action as appropriate.

5. Our dependency-tracking SLA will be no more than four hours from announcement to bug fi led, 24 × 365.

a. The response will be a risk assessment and possibly an action plan.

b. The response will be to test and roll all patches of severity X without bothering with risk assessment.

c. The response will be to deploy all patches and believe in our rollback practices.

6. We will maintain a testbed to roll out new patches before putting them into production.

7. We will use virtual machines taken from production to test new patches before rolling them into production.

8. We will have patch management software that can deploy to all operational services.

Operationally managing vulnerabilities is more broad than managing patches. Sometimes a vendor will release an advisory about a problem before there’s a patch, and you’ll have to decide if and how you want to manage such reports. An advisory may involve work in addition to or in place of patching.

www.it-ebooks.info

Chapter 12 ■ Requirements Cookbook 225

c12.indd 07:52:27:AM 01/15/2014 Page 225

Detection

Detecting security problems is a challenge in the chaos of modern operations. Even in regulated industries, most intrusions are detected by a third party. The right way to perform security logging and analysis seems to be elusive. Nevertheless, if you want to use prevent/detect/respond as a frame for thinking about requirements, useful requirements can be found in this section. There are two major goals to think about: incident detection and incident analysis. From a requirements perspective, incident detection involves logging and ensuring that the logs are analyzed. There are four main types of monitoring: change detection, signature attack detection, anomaly attack detection, and impact detection. These are discussed further in Chapter 9 “Tradeoffs When Addressing Threats” under “Wait and See.” Incident analysis involves recording enough state transitions that an analyst can reconstruct what happened.

Operational requirements:

1. We will detect attacks of type X within time Y.

2. We will detect 75 percent of attacks of type X within time Y, and 50 percent of the remainder within Y.

Product requirements:

1. Our product will use the word “security” in all log messages that we expect are security related.

2. Our product will log in a way to help detect attacks.

3. Our product will log login attempts in a way to help detect attacks.

4. Our product will track repeated attempts to perform any action, and fl ag such in the logs.

There is a fuzzy line between detection and response requirements. Rigidly categorizing them probably isn’t useful.

Response

Incident response teams often use an approach that mirrors the one sug- gested by Ripley in the movie Aliens, “I say we take off and nuke the entire site from orbit. It’s the only way to be sure.” Planning for that in product threat modeling involves a good separation between your product and its confi guration and data. That separation enables the response team to nuke the product install (which may be compromised) while preserving the con- fi guration and data (which may also be compromised, but can perhaps be cleaned up). In the same vein, publishing a set of signatures or hashes for the code you ship will help a response team check the integrity of your product after a compromise.

www.it-ebooks.info

226 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c12.indd 07:52:27:AM 01/15/2014 Page 226

In operational threat modeling for response, there is a much longer set of requirements from which you can build:

1. We will have an incident response plan.

2. We will have an incident response plan in a binder on a shelf somewhere.

3. We will have an incident response plan and run annual/quarterly/monthly drills to ensure we know how to operate.

4. Our intrusion-detection SLA will be no more than four hours from detec- tion to incident response execution, 24 × 365.

5. Our intrusion-detection SLA will be no more than eight business hours from detection to incident response execution during normal business hours.

6. Our incident response plan will [not] be designed to preserve court-quality evidence.

7. The senior administrators will be trained in our incident response plan.

8. All administrators will be trained in our incident response plan.

9. All administrators will have a wallet card with fi rst response steps and contact information.

10. All incidents will have a lessons learned document produced.

11. All incidents will have a lessons learned document produced, appropriate to the scale of the incident.

12. Lessons learned documents will be shared with the appropriate people.

13. Lessons learned documents will be shared with all employees.

14. Lessons learned documents will be shared with all employees and partners.

15. Lessons learned documents will be published when we are required to report a breach so others can learn from our mistakes.

16. Lessons learned documents will be published so others can learn from our mistakes.

The act of publishing lessons learned documents may seem unusual, but it is increasingly common practice, and the transparency has been benefi cial to business. For example, after a major outage at Amazon, they published a root cause analysis, and Netfl ix announced that they had used the information to improve their own service (Netfl ix, 2011).

C R O S S  R E F E R E N C E See also the earlier section “Vulnerability Management.”

www.it-ebooks.info

Chapter 12 ■ Requirements Cookbook 227

c12.indd 07:52:27:AM 01/15/2014 Page 227

People/Process/Technology as a Frame for Requirements

It’s also common for people to use people/process/technology as a frame for thinking about security, so it may help you to use it as a way to fi nd requirements.

People

There are two major categories of security requirements for people: trustworthi- ness and skills. Trustworthiness is a matter of how much authority and discretion the people in the system are expected to have. Organizations do various levels of background checking at hiring time or as an ongoing matter. Such checks can be expensive and intrusive, and may be constrained or required by local law. If your product is intended for use by highly trusted people, you should be explicit about that. Requirements for trustworthiness can be somewhat managed by audit and retributive measures, which impose development requirements regarding logging and operational requirements around the audit process. Skill requirements should also be clearly documented. It is sometimes helpful (and usually tricky) to have a certifi cation of some sort that attempts to assess the skills of an individual.

Security requirements focused on people might include:

1. Employees with cash management responsibility will undergo background checks for fi nancial crimes.

2. Employees with cash management responsibility will undergo credit checks.

3. Employees with cash management responsibility will undergo regular credit checks.

4. Employees dealing with children will be required to certify that they do not have a criminal conviction.

5. Employees dealing with children will be required to certify that they do not have a criminal conviction in the last seven years.

6. Employees dealing with children will undergo a criminal background check.

7. Prospective employees will have an opportunity to contest information that is returned, as we are aware that background checks are often inaccurate.

www.it-ebooks.info

228 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c12.indd 07:52:27:AM 01/15/2014 Page 228

If you are not familiar with the issues of accuracy in background checks, there are fascinating reports available, including ones from the National Consumer Law Center (Yu, 2012) or the National Employment Law Project (Neighly, 2013).

Process

Understanding how the product’s technology is to be operated can also drive security requirements. Drafting a security operations guide can be a good way to elicit product security requirements.

Technology

It is very tempting to say that this entire book is about the technology, so “this section intentionally left blank.” But that’s not really the case. This book shows that models of your technology are the best place to start threat modeling. But all technology interacts with other technology, so what are the limits or scope of how you approach this? The best answer when you’re getting started is to focus on those technologies where you can most directly address threats. As your skills grow, looking along your supply chain (and possibly the supply chains you are part of) may be a good way to expand your horizons.

Development Requirements vs. Acquisition Requirements

When you’re developing technology from scratch, the set of security require- ments you might choose to address is much larger than when you’re acquir- ing technology from someone else. This issue is exacerbated by technological ecosystems. For example, the Burroughs 5500 computers of the 1960s had a memory architecture that was resistant by design to stack-smashing sorts of attacks (Hoffman, 2008; Shostack, 2008). However, it is challenging to procure a system with such features today.

In reality, few systems are really developed “from scratch.” The security requirements of every project are constrained by its inputs. It is helpful to every- one along the chain to document what security requirements you do or do not support. For example, if you are developing on Windows, defending against the administrator account is not supported (Culp, 2013). Similarly, before Windows 8, you cannot defend against apps running as the same user. Windows 8 adds some capabilities in this area (Hazen, 2012).

www.it-ebooks.info

Chapter 12 ■ Requirements Cookbook 229

c12.indd 07:52:27:AM 01/15/2014 Page 229

Compliance-Driven Requirements

An ever-expanding set of security requirements is driven by compliance pro- grams. Those compliance regimes may be imposed on you or your customers, and they make an excellent source of security requirements. More rarely, they’re even a source of excellent security requirements. (It’s challenging to write broad requirements that are specifi c enough to engineer from.) This section covers three such requirement sets: the Cloud Security Alliance (CSA) control matrix and domains, the United States NIST Publication 200, and PCI-DSS (the Payment Card Industry Data Security Standard). To the extent that you’re building tech- nology for a single organization, you may have security requirements directly imposed. If you’re building technology to sell, or give away, then your customers may have requirements like these. If you expect to encounter a set of compli- ance requirements from your customers, it will probably be helpful to create or use a single meta-framework that covers all the issues in the frameworks you need to comply with, rather than attempt working with each one. There are commercially available “unifi ed compliance frameworks,” and depending on the number of compliance requirements you face, it may be worth buying one for speed, coverage, or expertise. The CSA framework can be a useful starting point if you need something free. The requirements that follow are presented as places from which to derive more detailed requirements. They should not be used to replace an appropriately detailed understanding of your requirements for compliance purposes.

Cloud Security Alliance

The CSA is a non-profi t organization dedicated to cloud security. They have produced two documents that are helpful for security requirements: the controls domains and the controls matrix.

The fi rst document maps controls into a set of domains, including governance and enterprise risk management, legal issues, compliance and audit, informa- tion management and data security, traditional security, business continuity and disaster recovery, data center operations, incident response, application security, encryption and key management, identity and access management, virtualization, and security as a service. This is a fi ne list of areas to consider, and the CSA has a lot more documentation for you to dig into. (They also consider architecture, and portability and interoperability, but those seem somewhat less relevant for security.)

The CSA also has a Cloud Control Matrix of 98 control areas, with mappings that show their applicability to architectural areas (physical, network, compute,

www.it-ebooks.info

230 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c12.indd 07:52:27:AM 01/15/2014 Page 230

storage, application, and data), to corporate governance, to cloud service delivery models (SaaS/PaaS/IaaS), and to service providers versus tenants. Each control area is also mapped to COBIT, HIPAA/HITECH, ISO/IEC 27001–2005, NIST SP800–53, FedRAMP, PCI DSS, BITS Shared Assessments SIG v6 & AUP v5, GAAP, Jericho Forum, and the NERC CIP.

The requirements documented in the Cloud Controls Matrix are designed to be used as a basis for cloud operational security. Some of the requirements are most relevant to cloud services, but it’s a fi ne resource to start with, with the advantages of being both freely available and already mapped to a large set of other sets of controls.

NIST Publication 200

Requirements in this publication are a mixed set, ranging from planning and risk assessment to physical environment protection and technical requirements, such as authorization and system integrity. Federal agencies are required to “develop and promulgate formal, documented policies and procedures . . . and ensure their effective implementation” (NIST, 2006). US Government agencies must also meet the controls laid out in NIST Special Publication 800–53. Items marked with a star align to one or more STRIDE threats, so you might cross- check the section “The STRIDE Requirements” later in this chapter.

For each item in this list, consider if there’s a need to address the issue in your product requirements:

■ Access control, including authorization*

■ Awareness and training

■ Audit and accountability including traceability of actions back to accounts*

■ Certifi cation, accreditation, and security assessments, including assess- ments of whether information systems are suitably protected

■ Confi guration management, which imposes confi guration and manage- ment requirements

■ Contingency planning

■ Identifi cation and authentication*

■ Incident response

■ Maintenance

■ Motherhood and apple pie (just kidding)

■ Media protection

■ Physical and environment protection

■ Planning, including plans for the organizational information systems and controls

www.it-ebooks.info

Chapter 12 ■ Requirements Cookbook 231

c12.indd 07:52:27:AM 01/15/2014 Page 231

■ Personnel security, including criteria for who’s hired, managing termina- tion, and penalties for compliance failures

■ Risk assessment of threats (including risks to mission, functions, image, or reputation) to the organization or individuals

■ Systems and service acquisition, including allocating resources and deploy- ing system development life cycles that address security

■ System and communication protection*

■ System and communication integrity*

PCI-DSS

The Payment Card Industry Data Security Standard (PCI-DSS) is a set of stan- dards for those processing payment card data (PCI, 2010). These standards are generally incorporated into contracts associated with credit card processing. Note the wide variance in specifi city—from “have a security policy” to “do not use default passwords”—in the following requirements:

1. Install and maintain a fi rewall confi guration to protect cardholder data.

2. Do not use vendor-supplied defaults for system passwords and other security parameters.

3. Protect stored cardholder data.

4. Encrypt transmission of cardholder data across open, public networks.

5. Use and regularly update anti-virus software.

6. Develop and maintain secure systems and applications.

7. Restrict access to cardholder data by business need-to-know.

8. Assign a unique ID to each person with computer access.

9. Restrict physical access to cardholder data.

10. Track and monitor all access to network resources and cardholder data.

11. Regularly test security systems and processes.

12. Maintain a policy that addresses information security.

Privacy Requirements

You’ll fi nd there are a few main motivations for privacy, including legal com- pliance and a desire to make only those promises that can be kept—to avoid customer anger. This section covers a selection of important privacy require- ments frameworks. They are important because they underlie many laws or are infl uential with regulators (Fair Information Practices, Privacy by Design), can

www.it-ebooks.info

232 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c12.indd 07:52:27:AM 01/15/2014 Page 232

help you avoid privacy blow-ups (Seven Laws of Identity), are pragmatic, and are designed to be accessible to non-specialist software developers (Microsoft’s Privacy Standards for Developers).

Fair Information Practices

Fair Information Practices (FIP) is a concept that goes back to a 1973 report for the United States Department of Health, Education, and Welfare, which put forth fi ve core fair information Practices. Along the way, practices were promoted to principles, and if you’re paying close attention, you’ll notice FIP expanded to either, or sometimes even both (Gellman, 2013). The difference is minor, but I’ll hew to the term used in the source discussed. The original enumeration was as follows:

1. Notice/Awareness

2. Choice/Consent

3. Access

4. Security

5. Enforcement/Redress

These form the basis for the 1980 OECD “Guidelines on the Protection of Privacy and Transborder Flows of Personal Data.” The OECD put forth eight Principles. The European Union’s Data Protection Directive and Canada’s Personal Information Protection and Electronic Documents Act are also based on FIPs, and each has divided them into slightly different lists. These high-level prin- ciples may be a useful checklist when evaluating security and privacy issues at design time. Which list you should use will depend on a number of factors, including the location of the organization and its customer base. Be aware that many software engineers fi nd these lists are too generic to be of much use when designing a system.

Privacy By Design

Privacy by Design is a set of principles created by the Ontario Privacy Commissioner with the goal of helping organizations embed privacy into product design. It outlines seven main principles, quoted below:

1. Proactive

2. By Default

3. Embedded

4. Positive Sum

5. Life-cycle Protection

www.it-ebooks.info

Chapter 12 ■ Requirements Cookbook 233

c12.indd 07:52:27:AM 01/15/2014 Page 233

6. Visibility/Transparency

7. Respect for Users

Privacy by Design has been criticized as “vague” and leaving “many open questions about their application when engineering systems” (Gürses, 2011).

The Seven Laws of Identity

Kim Cameron of Microsoft has put forward a set of seven principles he calls the Laws of Identity for digital identity systems (Cameron, 2005). They are not overall privacy requirements, but a great deal of privacy relates to how a system treats “identity,” a concept further discussed in Chapter 14 “Accounts and Identity.” They may be an interesting complement to contextual integrity (discussed in Chapter 6 “Privacy Tools”). These are extracted from a document that describes and contextualizes each law:

1. User Control and Consent: Technical identity systems must only reveal information identifying a user with the user’s consent.

2. Minimal Disclosure for a Constrained Use: The solution that discloses the least amount of identifying information and best limits its use is the most stable long-term solution.

3. Justifi able Parties: Digital identity systems must be designed so the dis- closure of identifying information is limited to parties having a necessary and justifi able place in a given identity relationship.

4. Directed Identity: A universal identity system must support both omni- directional identifi ers for use by public entities and uni-directional identi- fi ers for use by private entities, thus facilitating discovery while preventing unnecessary release of correlation handles.

5. Pluralism of Operators and Technologies: A universal identity system must channel and enable the inter-working of multiple identity technolo- gies run by multiple identity providers.

6. Human Integration: The universal identity metasystem must defi ne the human user to be a component of the distributed system integrated through unambiguous human-machine communication mechanisms offering protection against identity attacks.

7. Consistent Experience Across Contexts: The unifying identity metasystem must guarantee its users a simple, consistent experience while enabling separation of contexts through multiple operators and technologies.

These laws can be reasonably easily inverted into threats, such as “does the system get appropriate consent before revealing information?” or “does the sys- tem disclose identifying information not required for a transaction?” However,

www.it-ebooks.info

234 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c12.indd 07:52:27:AM 01/15/2014 Page 234

this runs the risk of losing the richness of the Laws of Identity, so it’s left as an exercise for you.

Microsoft Privacy Standards for Development

The Microsoft Privacy Standards for Developers (MPSD) is a prescriptive docu- ment that, for a set of scenarios, gives advice about how to address them. However, the standards are not focused on the discovery of privacy issues. They focus instead on a set of scenarios (notice, choice, onward transfer, access, security, and data integrity) and requirements for those scenarios (Friedberg, 2008).

The difference between these privacy standards and the more principle-oriented approaches is a result of the customer-focus. The MPSD are explicitly based on the FIPs, but are designed to provide practical advice for developers. Making it easy for (a defi ned) someone to use the document increases the effectiveness of an approach, and the audience-focus of the MPSD could well be emulated by other documents to help improve privacy.

Which of these privacy requirements frameworks will best inform your technology depends on what you’re building, and for whom. FIPs or Privacy by Design may spark valuable discussion of your designs or goals. If your system is focused on people, the “Seven Laws” can help. If you lack privacy expertise, the MPSD will help (but it is not intended to replace professional advice).

The STRIDE Requirements

You may recall that STRIDE is the opposite of properties that you want in a sys- tem, so properly this section ought to be called “The AINCAA Requirements,” but that’s just not very catchy. The relationship between STRIDE and the desired properties is shown in Table 12–1.

Table 12–1: STRIDE and AINCAA

THREAT DESIRABLE PROPERTY

Spoofi ng Authentication

Tampering Integrity

Repudiation Non-Repudiation

Information Disclosure Confi dentiality

Denial of Service Availability

Elevation of Privilege Authorization

The following subsections are organized according to the desirable property shown in Table 12–1.

www.it-ebooks.info

Chapter 12 ■ Requirements Cookbook 235

c12.indd 07:52:27:AM 01/15/2014 Page 235

Authentication

Authentication is the processes or activity which increases your confi dence that something is genuine. For example, entering a password increases the system’s confi dence that the person behind the keyboard really is authorized to use the system.

Is Authentication Required for Various Activities?

Different systems require different levels of authentication. As discussed in the chapter introduction, many websites offer read content to anonymous people (that is, no authentication is required). Some, like Wikipedia, offer write access as well. Some requirements you can build on include:

1. Anonymous people can create/read/update/delete items.

2. All authenticated users can create/read/update/delete items.

3. An enumerated subset of users can create/read/update/delete items.

How Strong an Authentication Is Required?

As stated above, different systems require different levels of authentication, and different forms of control for the authentication system. Sample requirements include:

1. Single-factor authentication is suffi cient for activity X.

2. Single-factor authentication plus a risk management check is suffi cient for activity X.

3. Two-factor authentication is suffi cient for activity X.

4. We will control the authentication database.

5. The IT/marketing/sales department will control the authentication database.

6. We will allow an outside organization (such as Facebook) to control our authentication database.

7. We will allow an outside organization (such as Facebook) to control part of our authentication database, such as “signed-in users” but not administrators.

8. Authentication should only be possible for people in IP range X.

9. Authentication should only be possible for people in physical location X (such as in the building or in the United States).

Note that systems to physically locate people are either weak, buggy (that is, high false positive, false negative rates) or exceptionally expensive (such as a dedicated air-gap network).

www.it-ebooks.info

236 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c12.indd 07:52:27:AM 01/15/2014 Page 236

Account Life Cycle

The ways in which accounts are managed will vary based on your requirements. Sample requirements include:

1. Anyone can create an account.

2. Anyone with an e-mail address can create an account.

3. Anyone with a validated e-mail address can create an account.

4. Anyone with a credit card number can create an account.

5. Anyone with a valid, authorizable credit card can create an account.

6. Anyone with a credit card who can tell us how much we charged can create an account.

7. Anyone with a bank account who can tell us how much we charged can create an account.

8. Only an authorized administrator can create normal accounts.

9. Only two authorized administrators can create a new administrator account,

a. and all other admins are notifi ed,

b. and an auditing team is notifi ed. (This is perhaps a non-repudiation requirement, but you can also think of it as part of the account life cycle.)

10. Anyone can close their account at any time, and the data is deleted as soon as possible.

11. Anyone can close their account at any time, and the data is deleted after a cooling off period.

12. Anyone can close their account at any time, and the data is kept for busi- ness purposes.

13. Anyone can close their account at any time, and the data is kept for N time to satisfy regulatory requirements.

14. Administrator participation is required to close an account.

15. When administrator participation is required to close an account, the account must be globally inaccessible within N minutes.

Integrity

Sample integrity requirements include:

1. Data will be protected from arbitrary tampering.

2. This data will only be subject to modifi cation by an enumerated set of authorized users.

www.it-ebooks.info

Chapter 12 ■ Requirements Cookbook 237

c12.indd 07:52:27:AM 01/15/2014 Page 237

3. These fi les/records will only be subject to modifi cation by enumerated authorized users.

4. These fi les/records will only be subject to modifi cation by enumerated authorized users, and edit actions will be logged.

5. These fi les/records will only be subject to modifi cation by enumerated authorized users, and edit content will be logged.

6. These fi les/records will be unwritable when the system is in operation.

7. These fi les/records will be append-only when the system is in operation.

8. Modifi cations to these fi les/records will be cryptographically detectable.

9. Modifi cations to these fi les/records will be cryptographically detectable with commonly available tools.

10. Data sent over this inter-process channel will be protected from tampering by the operating system.

11. Data sent over this channel will be protected from tampering by a cryp- tographic integrity mechanism.

12. Messages will be protected from tampering by a cryptographic integrity mechanism.

You’ll notice that there are integrity requirements on channels and messages. The channel is what the message travels down. Protection in the channel doesn’t protect the data once it has left the channel.

For example, consider e-mail. Imagine a well-encrypted, tamper-resistant, replay-protected channel between two mail servers. The operators of those servers have confi dence that the messages are well protected in that channel; but a person on one end or the other could alter a message, and forward it. If the messages themselves have tamper resistance, then that can be detected. Such tamper resistance could be imparted, for example, by a cryptographic signature scheme.

Non-Repudiation

Recall that non-repudiation can cover both business and technical requirements. Sample requirements include:

1. The system shall maintain logs.

2. The system shall protect its logs.

3. The system shall protect its logs from administrators.

4. The logs will survive compromise of a host; for example, by writing logs to a remote system.

www.it-ebooks.info

238 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c12.indd 07:52:27:AM 01/15/2014 Page 238

5. The logs will allow account compromise to be differentiated from a behav- ior change.

6. The logs will resist counterparty fraud; for example, by using cryptographic signatures.

7. The logs will resist insider tampering; for example, with hash chains or write-once media.

Confi dentiality

Sample confi dentiality requirements include:

1. Data in fi le/database will be available only to these authorized users.

2. Data in fi le/database will be available only to these authorized users, even if computers/disks/tapes are stolen.

3. Data in fi le/database will be available only to these authorized users, even if computers are stolen while turned on.

4. The name/existence of this datastore will only be exposed to these autho- rized users.

5. The content of communication between Alice and Bob will only be exposed to these authorized users.

6. The topic of communication between Alice and Bob will only be exposed to these authorized users.

7. The existence of communication between Alice and Bob will only be exposed to these authorized users.

Availability

Sample availability requirements include:

1. The system shall be available 99 percent of the time.

2. The system shall be available 100 percent of the time.

3. The system shall be available 100 percent of the time, and we will pay our customers if it’s not.

4. The system shall be available for N percent of the time, including planned maintenance.

5. Only authenticated users will be able to cause the system to spend 10× more CPU than they have spent.

6. The system will be able to resist a simple DoS such as synfl ooding by a 50,000-host botnet.

www.it-ebooks.info

Chapter 12 ■ Requirements Cookbook 239

c12.indd 07:52:27:AM 01/15/2014 Page 239

7. The system will be able to resist a simple DoS such as HTTPS connection initiation by a 50,000-host botnet.

8. The system will be able to resist a customized DoS by a 50,000-host botnet.

The selection of 50,000 is only somewhat arbitrary. It is large enough to be a substantial threat, while small enough that there are likely quite a few of them out there.

Authorization

Sample authorization requirements include:

1. The system shall have a central authorization engine.

2. The system shall have a central authorization engine with a confi gurable policy.

3. Authorization controls shall be stored with the item being controlled, such as with an ACL.

4. Authorization controls shall be stored in a central location.

5. The system shall limit who can read data at a higher security level.

6. The system shall limit who can write data to a higher integrity level.

7. The authorization engine should work with accounts or account groups.

8. The authorization engine should work with roles (properties of accounts).

There is a tension between storing authorization controls centrally versus with the objects being protected. The former is easier to manage but harder for normal people to understand (often to the point of being unsure where to go). The system controlling who can read from a higher integrity level is analogous to military data classifi cation schemes. Someone with a secret clearance can’t read a top-secret document. (This was fi rst formalized as the Bell-LaPadula model [Bell, 1973]). Controlling who can write to a higher integrity level is also very useful, and is described by the Biba model (Biba, 1977). The Biba model is a description of how an operating system protects itself from programs run- ning as a normal user.

Cloud and DevOps Authorization and Audit

Much of cloud operations eventually boils down to a set of authorization ques- tions. Who is authorized to make which changes, and who made which changes. The lists might not line up, because policy and implementation don’t always perfectly line up. As organizations move, intentionally or not, toward DevOps models, these questions become trickier. The requirements also change. Rather

www.it-ebooks.info

240 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c12.indd 07:52:27:AM 01/15/2014 Page 240

than a formal handoff from development to test, the code is promoted to pre- production, and then to production. Sample DevOps requirements include:

1. Any developer is authorized to push code to pre-production.

2. Any developer is authorized to push code to production.

3. Any production change requires only test pass complete.

4. These production changes only are possible with test pass completion.

5. Every production change requires human signoff.

6. Every change must be designed and tested to roll back.

7. Every production change can be tracked to a person.

8. Every production change can be tracked to a test run.

Non-Requirements

Just as enumerating requirements is important, so is being explicit about what the system won’t do. Some attacks are too expensive to deal with (for example, you might accept that the KGB could subvert your employees). Others may require capabilities that the operating system, chip-maker, and so on, do not currently supply. Whatever the reason, the things that your system doesn’t do should be explicit, so your management, operations, or customers are not sur- prised. The following sections consider three ways to express and communicate non-requirements: operational guides, warnings and prompts, and Microsoft’s “10 Immutable Laws of Security.”

Operational Non-Requirements

Sometimes there will be things that can’t be secured in the code but must be addressed in operations. The simplest example is reading the logs to detect attacks. Other goals, such as defending against malware or a malicious admin, can be attractive distractions. They generally fall outside of what you should worry about. You should document these as either requirements or non-requirements and engineer appropriately.

Have an Operational Guide

Documenting these in an operational guide serves two purposes: transparency and requirements elicitation. Transparency is useful because it helps your cus- tomers set their expectations appropriately, and avoid unpleasant surprises. The second purpose, requirements elicitation, means that as you document what an operator needs to do, you may well decide that the requirements are unrealistic,

www.it-ebooks.info

Chapter 12 ■ Requirements Cookbook 241

c12.indd 07:52:27:AM 01/15/2014 Page 241

and decide to either add features to make something more feasible, or remove features that can’t be used securely.

Defend against Malware in the Right Way

Trying to defend a system against malware that is already on the system is a complex and probably futile effort for most products. The exceptions are oper- ating systems creators and security software creators. If you’re not in either group, and if the malware is running inside the same trust boundaries as your code, there’s little you can do.

Most software systems should focus on preventing the elevation of privi- lege threats that allow malicious software to run. You should assume that the system is working on behalf of the people authorized to run code. (Obviously, this doesn’t apply if you’re creating an operating system or anti-virus product.)

Decide if Defending against Admin Is a Requirement

Most systems should not try to defend against the malicious admin. (The same principle as malware applies, but worse.) If the admin of a system is malicious, then they can do a wide variety of things. Some cryptographic systems may allow your data to transit these systems safely, but if you decrypt data on a system controlled by a malicious admin, they can capture your password or the plaintext of your documents. (Worse, they can get tricky. For example, telling you that a good cryptographic key didn’t work, tricking you into entering other passwords you commonly use.)

If you’re creating a high-assurance system of some form, you might have a requirement to always apply a two-person rule to defend against administra- tors. If that’s the case, then you have a fi ne challenge ahead.

Warnings and Prompts

Some things that can’t be secured in the code are suffi ciently dangerous that there should be a warning before allowing the system to proceed. These things can be considered non-requirements, or they can be used to drive requirements that improve the architecture of the system. See Chapter 15 “Human Factors and Usability” for more information.

Microsoft’s “10 Immutable Laws”

Microsoft’s “10 Immutable Laws of Security” are an example of how to explain what your system doesn’t do. The second paragraph of that document opens with “Don’t hold your breath waiting for an update that will protect you from

www.it-ebooks.info

242 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c12.indd 07:52:27:AM 01/15/2014 Page 242

the issues we’ll discuss below. It isn’t possible for Microsoft or any software vendor to ‘fi x’ them, because they result from the way computers work” (Culp, 2013). The fi rst several “laws” are as follows:

Law #1: If a bad guy can persuade you to run his program on your com- puter, it’s not solely your computer anymore.

Law #2: If a bad guy can alter the operating system on your computer, it’s not your computer anymore.

Law #3: If a bad guy has unrestricted physical access to your computer, it’s not your computer anymore.

Law #4: If you allow a bad guy to run active content in your website, it’s not your website anymore.

Law #5: Weak passwords trump strong security.

Law #6: A computer is only as secure as the administrator is trustworthy.

You might note that laws 1, 2, and 4 sure do seem like slight variations on one another. That’s to draw out the implications.

Summary

In this chapter, you’ve learned that good security requirements act as a comple- ment to threat modeling, enabling you to make better decisions about threats you discover with the techniques in Part II of this book.

You’ve been given a set of base requirements that are designed to help you do the following:

■ Understand the space of security requirements better.

■ Quickly crystalize more precise requirements.

This chapter should serve as a practical, go-to resource when you’re work- ing through requirements at a business or technology level. The business level includes requirements driven by competitive pressure and industry, and require- ments to handle vulnerabilities that your code might contain.

You’ve also seen how to use people/process/technology and prevent/detect/ respond to inform your requirements process. Compliance frameworks, including those from the Cloud Security Alliance, the U.S. government, and the payment card industry can be used as a base for your requirements.

You’ve learned about a variety of sources for privacy requirements. Lastly, you’ve been reminded how the STRIDE threats violate properties, and how those properties can be developed into requirements.

www.it-ebooks.info

243

c13.indd 02:56:22:PM 01/08/2014 Page 243

In many ways, threat modeling for the web and cloud are very much like threat modeling for anything else, but these unique environments have some recur- ring threats, which are covered in this chapter.

This chapter is organized into web threats, cloud threats, cloud provider threats, and mobile threats. Web threats are broken into website threats, web browser, and plugin threats. Many of the cloud threats are expressed with respect to infrastructure as a service (IaaS) and platform as a service (PaaS). It closes with a section on mobile threats.

Web Threats

The web is composed of a simple and powerful set of protocols and languages. It has become a cliché to say that it has changed everything. It’s easy to forget that the web is software like other software. Although you might assume that you need to threat model it in some new ways, the truth is that it’s like most other software, so techniques such as STRIDE and attack trees work well for web technologies.

C H A P T E R

13

Web and Cloud Threats

www.it-ebooks.info

244 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c13.indd 02:56:22:PM 01/08/2014 Page 244

Website Threats

Public websites receive large amounts of scrutiny, and suffer from all that the world can throw at them. The classic STRIDE threats all apply, as do a slew of web-specifi c attacks that happen when you forget that there’s a trust bound- ary between them and the apparently nice doggy that is wagging its tail and slobbering in remarkably cleverly formed SQL in your forms and JavaScript in your URLs in order to cause harm.

Usually, threats such as SQL injections and XSS are handled later in the soft- ware engineering process. You’ll be developing using patterns, libraries, and frameworks that make each threat less likely, using appropriate testing tools to catch problems during testing, and watching the logs after deployment. So you’re done, right? Unfortunately, no. You should be threat modeling to fi nd the unique threats your site will be vulnerable to, such as your ad provider, your analytics code, and that authentication database you’re using from some crazy start-up in San Francisco. A standard data fl ow diagram (DFD) showing where the data comes from for your server is essential, and a client-side DFD is also a pretty good idea, if only to ensure that you have a good test suite. A client-side DFD can also be a good way to create a list that helps you track when your dependencies issue security updates.

Web Browser and Plugin Threats

The web browser has become people’s primary portal onto the Internet, and occasionally their last line of automated defense against attacks. Anyone consid- ering building a web browser needs at least one expert with a deep knowledge of the history of browser security issues.

Browser companies could substantially help matters by being super-diligent about their security goals, and how those goals manifest between tabs, websites, and the OS hosting the browser. Clarity from browser creators on how to create a plugin that does not violate security would also be most welcome.

Web plugins or add-ons that extend browser security are becoming less and less common, in part because the two best-known and widely deployed plugins, Java and Flash, have both suffered serious and ongoing security problems. Another reason plug-ins are becoming less common is Apple’s willingness to exclude Flash from the iPhone.

Any of the building blocks for fi nding threats can be applied to a web browser. For example, the STRIDE threats all apply to a web browser. There’s spoofi ng of web pages for phishing or other goals, cross-tab tampering and tampering with fi les included from other servers. There’s information disclosure about

www.it-ebooks.info

Chapter 13 ■ Web and Cloud Threats 245

c13.indd 02:56:22:PM 01/08/2014 Page 245

browser history vis CSS sniffi ng, and similar examples exist for each STRIDE threat. There are also very specifi c attack libraries available for web browsers and website designers.

If you’re going to create a browser plugin, there are two unique elements to consider: the browser’s security model and the browser’s privacy model. You should also realize that auto-update is important and must be done securely.

Browser Security Model

You must deeply understand and respect the browser’s security model, and not accidentally break it. This security model includes elements such as the same- origin policy, the boundaries between pages, and what can and can’t open a new window, resize a window, and so on. You must also remember to treat the other sides of connections as malicious with respect to the browser. That is, from the browser’s perspective, your component that sits on a web server could be under the control of an attacker, who can then send malicious content to your plugin and compromise additional systems. (Similarly, your plugin may be modifi ed or run through a proxy that rewrites data to attack the server compo- nents.) However, there are times when breaking the browser security model is intentional and appropriate. For example, some security testing plugins do so.

Several experts have told me in all seriousness that browser security models are now so complex that I should not even write a section about this. I’m tempted to say that browser plugins, like crypto, is a domain for which you need expertise and penetration testing. It would be wonderful if browser manufacturers could fi x this, and offer easier to understand plugin models so that plugins could be developed without putting the people who install them at risk. For a history of the three fl aws in one popular plugin, see Mark Pilgrim’s article “Avoid Common Pitfalls in Greasemonkey” (O’Reilly Network, 2005). (As an aside, Pilgrim’s blog post is a good example of a “Note to API Callers,” as discussed in Chapter 7, “Processing and Managing Threats.”) For a book-length treatment of the full complexity of modern browsers, see Michal Zalewski’s The Tangled Web (No Starch Press, 2011).

Browser Privacy Model

Similar to respecting the browser’s security model, your plugin should respect the browser’s privacy model. You should not allow tracking or surveillance in any way that the browser does not, and you should ensure that your controls are at least as accessible as those offered by the browser.

www.it-ebooks.info

246 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c13.indd 02:56:22:PM 01/08/2014 Page 246

Auto-Update

It is very likely that you will have numerous security issues with your plugin, and you should therefore ensure that it’s easy to report bugs to you, and that your updater works well with the browser’s auto-update mechanism, and that you have a security update process that can deliver security updates without any tradeoffs such as new or changed user interfaces, new licenses, or similar impediments to upgrading.

Cloud Tenant Threats

You can use an attacker grouping approach to break out cloud threats. There are two main classes of new attackers (threats) when you move an IT system to the cloud: those from insiders at the cloud operator, and those from your fellow tenants of the cloud system. There are generally some new instances of availability threats, based on the increased complexity of connectivity. There are also two sets of legal threats: those that add complexity and/or effort or reduce the assurance of compliance, and the different legal standards around data given to third parties. Lastly, there’s a hybrid of those legal threats, which are threats to forensic response. In this section, you’ll see the term cloud provider used to refer to an organization that offers any combination of infrastructure, platform, or software as a service. Their customer is you, and like their other customers, you are a tenant of their service. Attackers might be tenants, or those who have broken in.

Insider Threats

When you move your data or operations to someone else’s cloud, you add a trust boundary. That boundary has the employees and contractors of the cloud operator inside of it, with your data. As administrators, they have unavoidable technical access to the data you provide. They may intentionally attack you, fall victim to an attack themselves, accidentally misconfi gure software, or fail to perform maintenance, such as wiping disks between re-allocations.

There are two ways to mitigate this threat: contractually and cryptographically. Contractual approaches dominate today because they’re easier; and for most of the risks, it turns out that a contract is suffi cient. However, contracts may not be subject to negotiation unless you’re spending lots of money. Unfortunately, companies often use contracts to protect personal information, where much of the risk is external to the companies signing the contracts.

The cryptographic approach is to encrypt the data (and possibly obfuscate the code) before sending it. This is easier with a cloud storage system than with software as a service. Well-encrypted, integrity-protected, and authenticated

www.it-ebooks.info

Chapter 13 ■ Web and Cloud Threats 247

c13.indd 02:56:22:PM 01/08/2014 Page 247

data can be stored anywhere with a very low reduction in safety. (The encrypted state of the data will infl uence what processing can be done on the encrypted data. There is interesting cryptographic research on processing encrypted data, which as of this writing usually isn’t part of the standard crypto libraries that you should use.) Of course, the keys need to be stored securely, or you’re trading confi dentiality and integrity for availability.

Co-Tenant Threats

These threats are to tenants of “infrastructure as a service” (IaaS) and to a lesser degree “platform as a service” (PaaS), whereby the provider/tenant trust bound- ary allows tenants to execute arbitrary code inside the data center. In IaaS, the code execution privileges are effectively unlimited, although they may formally be limited to what a non-administrative account can do. In PaaS, the code that is supposed to execute is far more limited. Historically, few platforms were constructed with a threat model that the most important attacker is already on the system. It is far more common, and probably even appropriate, for platform designers to worry about the trust boundary between the system and those who cannot execute any code. Unless the platform software has been carefully constructed to resist elevation of privilege attacks and to prioritize fi nding and fi xing those, there are likely vulnerabilities that allow an attacker to violate the rules. That leads to second-order threats to the cloud tenants.

There is also a set of threats from other tenants, ranging from the trivial to the movie plot. At the trivial end, you may be behind the same single fi rewall as your competitor, or someone without an IT department. You might also be on the same domain as they are, such as cloudapp.net or s3.amazonaws.com. Some defenses, such as fi rewalls, need to be managed as part of the cloud deployment. Your systems may also come under attack as stepping-stones to other tenants of the cloud provider. Beyond that, another tenant might try to bust out of their virtual machine and take over a host, giving them access to your machine as well (if you’re sharing machines). An attacker might also be able to access the network or storage (either local cache or storage specifi c to the cloud). Another tenant might be taken over to run a DoS attack against you, or an attacker might sign up to do so.

Threats to Compliance

There are three typical issues here. First, for many compliance regimes—but most notably PCI and HIPAA—the entire stack, including physical security, needs to be compliant. Therefore, the only way to have a PCI assessed app is for your cloud provider to be assessed PCI compliant. Second, there can be issues with auditing and logging. Not all cloud operators will provide logs of access to their APIs or web consoles. This can lead to technical issues, such as it may be

www.it-ebooks.info

248 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c13.indd 02:56:22:PM 01/08/2014 Page 248

impossible to see who added or changed accounts. That technical issue can lead to a compliance issue. The fi nal issue is when you get into the PaaS and SaaS market, you often lose the ability to leverage cryptography at the fi lesystem or database level, as well as any concept of end-to-end encryption.

Legal Threats

The primary new legal threat when you move data to the cloud relates to laws that (at least in the U.S.) substantially reduce your ability to know about or challenge legal requests for your information, such as subpoenas or warrants. Data you store on your own systems is often more legally protected than data you store on someone else’s systems. The legal demands served on your cloud provider may also contain provisions forbidding them from telling you about those demands, or you may be informed after the data has already been provided.

Of course, there’s also the need to negotiate agreements related to privacy, security, and reliability. Contractual provisions for both privacy and security need to cover your business needs and your compliance needs. Privacy is cov- ered by a hodge-podge of U.S. regulations. In the European Union, and those places with a safe harbor agreement for data from European countries, there’s a set of requirements, most importantly for the organization holding the data to name a data custodian. That custodian has certain responsibilities, and you’ll need to address those if you’re moving data collected under EU or other similar privacy regimes.

It is, of course, important to consider these issues with your attorney; this section is intended only to outline some of the points you should discuss with them. Your attorney may have additional concerns.

Threats to Forensic Response

After an intrusion, your VM may be shut down by the cloud provider. You may have instantiated a system large enough that performing complete snapshot or a memory dump is time consuming; but most important, you may not have a defensible chain of custody.

Miscellaneous Threats

Some cloud providers offer easy to use virtual machine images, uploaded by kind strangers for your convenience, and out of the goodness of their hearts.

www.it-ebooks.info

Chapter 13 ■ Web and Cloud Threats 249

c13.indd 02:56:22:PM 01/08/2014 Page 249

Sometimes, the people who uploaded them really did so out of the goodness of their hearts. Other times, the images are not so safe, and trusting them for anything serious is probably foolish.

Cloud Provider Threats

There is also a set of threats to the cloud provider that cloud providers must consider, and cloud customers should consider. These are all threats from or caused by the folks to whom you give access to your systems. These are split into threats caused by malicious behavior by the tenant that targets the pro- vider, such as attempts by the tenant to hack the provider; and threats caused by tenant behavior, such as blacklisting.

Threats Directly from Tenants

The largest threat is that a tenant will fi nd a way to break out of whatever sand- box you put them in, and be able to take actions you don’t want them taking. Those actions might be on the order of running code on the raw hardware and tampering with either your billing or your other customers, or it might be con- necting to networks that should be fi rewalled off. In particular, US-CERT has warned about threats to the IPMI (Intelligent Platform Management Interface) (US-CERT, 2013). These threats are sometimes managed by putting IPMI on an isolated network. If a tenant breaks out, they may be able to access such a management network. The sandbox escape threats are more likely when clients are held back by fewer security boundaries. Thus, a client in a Software as a Service (SaaS) environment has more barriers than one trying to escape from an Infrastructure as a Service offering.

There’s also a fraud (repudiation) threat, which is that a new tenant might sign up with someone else’s personal data and/or credit card, preparatory to committing fraud, running a botnet, or DDoSing a game server. These threats can be partially addressed by charging the card immediately for the fi rst por- tion of service, or charging a sign-up fee, although that may be inhibitory to new business. If there are throttles for e-mail sent or similar things, it may be useful to tie them to length of tenancy.

Unfortunately, many of the ways to address these threats are at odds with the low-friction, get-started-quickly value proposition that cloud providers want to offer. There may be interesting ways to address these trade-offs that have yet to be invented, but for now appropriate monitoring for anomalies is important.

www.it-ebooks.info

250 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c13.indd 02:56:22:PM 01/08/2014 Page 250

Such monitoring has to build on the unique elements of the business, so custom code will be needed.

Threats Caused by Tenant Behavior

There’s a set of threats that tenants can cause, including spamming and piracy. These problems are magnifi ed in those cases where accounts are free to anyone, and essentially anonymous. These threats are less clear-cut than issues such as spoofi ng, tampering, or information disclosure. If your requirements are clear, there’s no question about whether the threat violates them. In contrast, perhaps the e-mails are all going to an authorized list? Perhaps the person who uploaded that song is the artist, or is authorized to do so? Or perhaps the use of an image is permissible under the law? This lack of clarity makes these threats harder to manage than the classical security violations.

The United States has a somewhat clear set of rules regarding notice and take down. These give cloud providers a legal defense against copyright claims until they receive a formalized notice of infringement, after which they must take certain actions, generally taking down the content. Following those processes is prudent, as jumping to judgment and action on a notice may threaten the protections you otherwise have.

These threats lead to an indirect threat to the provider and other tenants, which is backlash from the attacks. That backlash can include visits or calls from law enforcement, reports from other parties that require investigation or response, and blacklisting. Once a tenant has sent spam, been part of a botnet, and so on, there’s a risk that the IP, subnet, or ASN may be blacklisted. The behaviors that lead to blacklisting may be violations of the terms of service. Responding to the behavior and getting IP addresses de-listed is likely a manual and time- consuming task.

Mobile Threats

Threats to mobile devices are generally similar to threats to other computers. For example, someone who can run code on the device may read your fi les, use your authentication data, etc. There are a few additional threats for mobile devices, including increased likelihood of device loss, diffi culty managing the devices, and business models confl icting with security updates.

The threat of device loss is clearly higher for mobile devices than for serv- ers. The two main ways to address device loss are device wipe and data wipe. Device wipe can cause confl ict, as many devices are owned by your employees

www.it-ebooks.info

Chapter 13 ■ Web and Cloud Threats 251

c13.indd 02:56:22:PM 01/08/2014 Page 251

or contractors, so in wiping their device, you may be deleting data that is not yours to delete. Data wipe can be accomplished by sending encrypted data to the device, and only sending the key needed to access the data when the data is accessed.

Many mobile devices are locked to specifi c software providers, and constrain which software can be loaded onto the device. This may threaten the ability of a compliance team to load them up with compliance-ware. When a mobile device is locked, then you defi ne the person holding the device as a threat. The device and its physical shell become a trust boundary that you must carefully consider, as all communication is subject to tampering, denial of service, and information disclosure.

Lastly, many wireless carriers threaten the ability of the software makers to deliver patches to devices in the fi eld, claiming that arbitrary patching threatens them with bricked devices and support costs. This means that fi elded mobile devices can be vulnerable to security problems that are fi xed in newer versions of software.

Summary

The web is comprised of software which can be threat modeled like any other software. There are a variety of recurring threat classes that are best managed by using safer languages and test frameworks that focus on those classes. These threats are things like XSS and SQL injection. There are also recurring patterns within the web and cloud that you should consider when threat modeling.

The browser and its plugins have threat models that should be documented by the browser maker. Those threat models should cover both security and privacy, and you need to understand them to program the browser.

Cloud tenants come under threat from insiders at the cloud provider and from co-tenants. Limits on code execution make attacking your co-tenants harder, so as you move up the stack from IaaS to PaaS to SaaS, co-tenant attacks become less likely. There are also a variety of threats to compliance and legal threats that cloud tenants must account for. In addition, forensic response may be threatened if you are a cloud tenant.

Cloud providers have to worry about threats of their tenants attacking them, and the side effects that can occur when their tenants attack others. In particu- lar, if tenants can access management networks that are normally isolated, the impact on security can be exceptionally large.

Mobile computers, including laptops, tablets, and phones are much like other computers, except the frequency of device loss is far higher. Some mobile devices

www.it-ebooks.info

252 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c13.indd 02:56:22:PM 01/08/2014 Page 252

are locked in ways that restrict the loading of software, making it hard to add compliance-ware, and possibly changing the threat model to include the person who uses the device. Such locking may also threaten patching, leaving devices vulnerable to known issues, and effectively removing the “need to fi nd a vul- nerability” barrier to attack.

www.it-ebooks.info

253

c14.indd 07:52:38:AM 01/15/2014 Page 253

If you don’t get account management right, you open the door to a slew of spoofi ng threats. Your ability to rely on the person behind a keyboard being the person you’ve authorized falls away. This chapter discusses models of how computers identify and account for their users, and the interaction of those accounts with a variety of security and privacy concerns. Much of the chapter focuses on threat modeling, but some of it delves into thinking about elements of security and the building blocks that are used. The repertoire in this chapter is specialized, but frequently needed, which makes it worth working through these issues in detail.

As the world becomes more digital, we interact not with a person in front of us but with their digital avatars and their data shadows. These avatars and shadows are models of the person. Remember: All models are wrong, and some models are useful. When the model is a model of a person, he or she may take offense at how they have been represented. The offense may be fair or misplaced, but effective threat modeling when people are involved requires an understanding of the ways in which models are wrong, and the particular ways in which wrong models can impede your security, your business, as well as the well-being, dignity, and happiness of your current or prospective customers, citizens, or visitors.

Despite the term identity being in vogue, I do not use it as a synonym for account. The English word identity has a great many meanings. At its core is the idea of oneness, a consistency of nature—that this person is the same one

C H A P T E R

14

Accounts and Identity

www.it-ebooks.info

254 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c14.indd 07:52:38:AM 01/15/2014 Page 254

you spoke with yesterday. Identity refers to self, personhood, character, and the presentation of self in life. To speak of “identity issuance” reveals either a lack of grounding in sociology and psychology, or a worrisome authoritarian streak, in which the identity of a human is defi ned and controlled by an external authority.

It is often tempting to model accounts as permanently good or bad, as a description of the person behind it. This is risky. Accounts can change from good to bad when an attacker compromises one, or when a bad creator decides to abuse them, which may be immediate or after a long period of “reputation establishment.”

This chapter starts with the life cycle of accounts, including how they are created, maintained, and removed. From there, you’ll learn about authentication, including login, login failures, and threats against authenticators, especially the most common authentication technology, passwords, as well as the various threats to passwords. That’s followed by account recovery techniques, threat models for those, and a discussion of the trade-offs associated with them. The chapter closes with a discussion of naming and name-like systems of identifi ers such as social security numbers.

Account Life Cycles

Over the lifetime of most systems, accounts will be created, maintained, and removed. Creation may take many forms, including the authorization of a feder- ated account. Maintenance can include updating passwords or other informa- tion used for security. People are often not aware of or motivated to perform such maintenance, which has resulted in the creation of technical tools such as password expiry, which tries to either prod them or force their hand. People’s awareness varies and is usually lower with lower-value accounts (when was the last time you looked at your e-mail from Friendster, or logged into your home wireless router?) Lastly, accounts will eventually need to be removed for a variety of reasons, and doing so exposes threats such as identifi er re-use.

Account Creation

The life cycle of an account starts when the account is opened. Ideally, that hap- pens with proper authorization, which means very different things in different contexts. You can roughly model this based on how deep the relationship is between the person and an organization.

For Close Relationship Accounts

Some accounts require and validate a good deal of information about a person as the account is created. These accounts are typically of very high value to the

www.it-ebooks.info

Chapter 14 ■ Accounts and Identity 255

c14.indd 07:52:38:AM 01/15/2014 Page 255

person, and incur high risk to the organization. Examples include a bank account or a work account with access to corporate e-mail and documents. Because of the sensitive nature of these accounts, the account creation process is typically more cumbersome, possibly including verifi cation steps and approvals. Such accounts are likely to be strictly limited in terms of how many a person may have at one time.

When people understand that the security requirements for these accounts may exceed those of other types of accounts, they might use better (or at least different) passwords, and the threat of lying about key data is somewhat reduced.

For Free Accounts

There are many online accounts that require little to nothing when signing up: an e-mail address, perhaps. For these types of accounts, people will often pro- vide personal information that is funny, aspirational, or completely inaccurate. They do this to present an identity or persona, or for political or privacy reasons. (I discovered recently that several of my online accounts say I am in Egypt, a country I haven’t visited in 20 years. I likely set it during the Arab Spring, and had no reason to change it since then.) If you rely on this information, these ten- dencies to enter funny or aspirational information are a threat to you, while your attempts to “validate” or constrain it can seem like a threat to your customers.

Free accounts are more likely to be used by more than one person—for example, a family or team calendar. When someone leaves the team, the account manage- ment implications are less than when someone is no longer authorized to access a bank account. In the worst case scenario, the free account can be replaced trivially, although the contents might not be. Free accounts are often used for a variety of mischief, including but not limited to spam, or fi le-sharing for music, books, or movies. Each of these threatens to annoy you, your customers, or the Internet at large in various ways and to various degrees.

At the Factory

Systems that ship with a single default account created at the factory should require a password change at setup; otherwise, the password will be available to anyone via a search engine. You might also be able to ship with a unique password printed onto a device or a sticker.

Federated Account Creation

Accounts are often created based on an account elsewhere, such as a Facebook or Twitter login, or a corporate (active directory) account. Federation systems such as OAUTH or Active Directory Federation Services (ADFS) allow this to happen

www.it-ebooks.info

256 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c14.indd 07:52:38:AM 01/15/2014 Page 256

somewhat seamlessly. While federation reduces the burden on individuals who want to create new accounts, it exposes risk in that a breached federated system may be a stepping-stone to your system (depending on where and how authen- tication tokens are stored). It may pose privacy threats by requiring the linkage of accounts. It may also increase the impact of threats by making the federated account a more valuable target. Lastly, users are often forgetful about where they’ve federated, and leave federation in place even after they no longer use it.

Creating Accounts That Don’t Correspond to a Person

We often want to think “one account for one person.” Security experts advise against shared accounts for good reasons of (ahem) accountability. There are many reasons why that advice is violated. Some accounts are set up for more than one person—for example, a married couple may have a joint bank account. Often times, they will share a single login/password combination, even if you’ve made it easy to set up several (computer) accounts to connect to a single bank account. Similarly, many people might share one work account. It is important to think about what happens when one or more participants in such a shared account are no longer authorized to use it. For the married couple with a joint bank account, what’s the right system? That both spouses have a (system) account that is authorized to access the (bank) account? Similarly, a traditional landline phone is an account for a family. If you call 867–5309, you might get someone in Jenny’s family.

Andrew Adams and Shirley Williams have been exploring these issues and have a short, readable paper “What’s Yours Is Mine, and What’s Mine Is My Own” (Adams, 2012). Taking a cue from the world of law, they suggest consider- ing several types of joint accounts: several, shared, subordinate, and nominee. Several accounts are those that refl ect the intersection of individuals where each has complete authority over the account. Shared accounts are those for which all members of a group can see information, but some subset of users can control what others can change. For example, members of an LLC could all see the fi nancials, but only the treasurer can issue payments. Subordinate accounts might be created by a parent or guardian, and allow one or more supervisory accounts to see some, but perhaps not all, of the child’s activity. (For example, parents might be able to see correspondent e-mail addresses, but not contents.) Finally, nominee accounts might allow access to the account after some circumstance, such as death. (Nominee accounts in this sense relate closely to Schechter, Egelman, and Reeder’s trustees, covered later in the section “Active Social Authentication.”) In any or all of these sorts of systems, there might be one login account that is used by everyone who has access to the joint account, or one login per person—it depends primarily on your development decisions and the usability of a joint account system.

www.it-ebooks.info

Chapter 14 ■ Accounts and Identity 257

c14.indd 07:52:38:AM 01/15/2014 Page 257

It’s also important to avoid believing that there is a one-to-one correspon- dence between the existence of an account and a human being who controls it. This is easily forgotten, although identity masking is common enough that it has many names, such as sockpuppets, astroturfi ng, Sybyls, and tentacles. Each of these is a way to refer to a set of fake people under the control of a group or person hoping to use social proof for the validity of their position. (Social proof is the idea that if you see a crowd doing something, you’re more likely to believe it’s OK.) Credit for this point belongs to Frank Stajano and Paul Wilson, as described in their paper “Understanding scam victims: seven principles for systems security” (Stajano, 2011).

Account Maintenance

Over time, it turns out that nearly everything about the person who uses an account can change, and many of those changes should be refl ected with the account. For example, almost everyone will change data such as their phone number or address; but other things about them can change as well—name, gender, birthday, biometrics, social security number. (Birthdays can change because of inaccurate entry, inaccurate storage, or even discovery of a discrep- ancy between documents and belief.) Even biometrics can change. People can lose the body part being measured, such as a fi nger or an eye. Less dramatically, cuts to fi ngers can alter a fi ngerprint pattern, and older people have harder to read fi ngerprints.

If you store such data, you need mechanisms for changing it. It’s important to align the change of these records with the security implications of a change. If you use the phone or physical mail to authenticate, ensure that you do every- thing appropriate to authenticate a phone number change. For a customer change of address, many wise banks send letters to both the new address and the old address, and turn their risk algorithms way up for a month or so after such a change. When users are not aware that data such as phone number will be used as an authentication channel, they are far less likely to keep it up to date. Even when they are aware that such data may be used for that purpose, they are unlikely to remember to update every system with which their phone number is associated.

Notifying the Real Person

Many services will let their customers know about unusual security events. For example, Microsoft will send text messages or e-mail to notify that additional e-mail addresses have been added to an account, and LiveJournal will send an e-mail if a login happens from a browser without a cookie. Such notifi cations are helpful, insofar as your customers are probably motivated to protect their

www.it-ebooks.info

258 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c14.indd 07:52:38:AM 01/15/2014 Page 258

accounts; and risky, as your customers may not understand the messages. They may become more concerned than is appropriate, driving customer support costs, dissatisfaction, or brand impact. Additionally, there’s a risk that attackers will fake your security messages for phishing-like attacks. The right call is a matter of good threat modeling, including good models of scenarios.

You can mitigate the risks by using scamicry-resistant advice (see Chapter 15 “Human Factors and Usability”), ensuring that you have low false positives, and using time as your ally. Time is an ally because you can delay authorizing important changes such as payments or backup authentication options until you have more information. That might be someone logging in from a frequently seen IP address or computer, demonstrating access to e-mail, or otherwise adding more information to your decisions. For more on delays, see “Avoiding Urgency” in Chapter 15. See also the “Account Recovery” section later in this chapter.

Account Termination

Customers may stop paying, leave, or pass away. Workers may quit or be fi red. Therefore, it is important to have ways to terminate accounts fully and prop- erly. Changing their passwords may not be enough. They might have e-mail forwarding or scheduled processes that run; or they may be able to use account recovery tools to get back in. This is actually the space in which many “identity management systems” play: helping enterprises manage the relationship between a person and the dozens to hundreds of accounts on disparate systems that they might possess. (Now get off my lawn before I hit you with my typewriter.)

When you terminate an account, you need to answer at least two important questions. First, what do you do with the objects that the account owns, including fi les, e-mail messages, websites, database procedures, crypto keys, and so on? Second, is the account name reserved or recycled? If you recycle account names, then confusion can result. If you don’t, you risk exhausting the namespace.

Account Life-Cycle Checklist

This checklist is designed to be read aloud at a meeting. You’re in a bad state if for any of these questions:

■ You can’t answer yes.

■ You can’t articulate and accept the implications of a no.

■ You don’t know.

1. Do we have a list of how accounts will be created?

2. Do all accounts represent a single person?

www.it-ebooks.info

Chapter 14 ■ Accounts and Identity 259

c14.indd 07:52:38:AM 01/15/2014 Page 259

3. Can we update each element of an account?

4. Does each update notify the person behind the account?

5. Do we have a way to terminate accounts?

6. Do we know what happens to all the data associated with an account when it’s terminated?

Authentication

Authentication is the process of checking that someone is who they claim to be. Contrast that with authorization, which is checking what they’re allowed to do. For example, the person in front of you might really be Al Cohol, but that doesn’t entitle them to a free drink.

Figure 14-1 shows a simple model of the authentication process. First, a person enrolls in some way, which varies from the simple (entering a username and password on a website) to the complex (going through an extensive background check and signing paperwork to trigger a create account process). Later, someone shows up and attempts to authenticate as the person who enrolled.

Enrollment

Authentication Requestor

Account name

Response

Authenticator(s)

Please authenticate

Account, validation data

Figure 14-1: A simple model of authentication

Many of the threats covered in this chapter can be discovered by applying techniques such as STRIDE or attack trees to Figure 14-1. However, because the threats have been discovered, and the mitigations have been worked through, this chapter summarizes much of what’s known so that you don’t have to re- invent it.

The traditional three ways to authenticate people have required either “some- thing you know, something you have, something you are,” or some combination

www.it-ebooks.info

260 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c14.indd 07:52:38:AM 01/15/2014 Page 260

of those. Passwords are an example of something you know. Something you have might include proximity cards or cryptographic devices such as RSA tokens. Something you are is measured with tools such as fi ngerprint or retina scanners.

These methods of authenticating people are often termed authentication factors, and multi-factor authentication refers to the use of several factors at one time. It is important that factors be independent and different. If a system consists of something you have, an ATM card, and something you know, a PIN, and you write your PIN on the card, the security of the system is substantially reduced. Similarly, if the something you have is a phone infected with malware because it’s synced to a desktop computer, then the ability of the phone to add security as a second factor is greatly reduced. Using more than one of the same sort of factor is sadly frequently confused with multi-factor authentication.

These three factors have been sarcastically reframed as “something you’ve forgotten, something you’ve lost, and something you were.” (This is often attrib- uted to Simson Garfi nkel, although he does not take credit for it [Garfi nkel, 2012]). This reframing stings because each is a real problem. Two additional factors are also sometimes considered: who you know, also called social authen- tication, and discussed later in the section “Account Recovery,” and how you get the message, sometimes called multi-channel authentication. Multi-channel is heavily threatened by the integration of communication technologies, and good modeling of the channels often reveals how they overlap. For example, if you think that a phone is a good additional channel, walking through how phones are used might uncover possible vulnerability vectors such as syncing (and associated infection risk) and how products like Google Voice are putting text messages in e-mail.

In this section, you’ll learn about login and especially handling login fail- ures, followed by threats to what you have, are, and know. Threats to what you know spans passwords and continues into the knowledge-based authentication approach to account recovery.

Login

We’re all familiar with the login process: Someone presents an identifi er and authenticator, and asks to be authenticated in some way. As shown in Figure 14-2, there are many spoofi ng threats in this simple process, including that the client or server may make false claims of identity, and the local computer may be presenting a false UI. (It’s that last threat which pressing Ctrl+Alt+Delete [CAD] addresses: CAD is a secure attention sequence, one that the operating system will always respond to, rather than passing to an application.)

www.it-ebooks.info

Chapter 14 ■ Accounts and Identity 261

c14.indd 07:52:38:AM 01/15/2014 Page 261

Spoof Client

Obtain credentials

Transit

Change management

Storage

At server

At KDC

Authentication UI

Insufficient authentication

Local login Null creds

Guest/anon creds

Predictable creds

Factory default creds

Downgrade authentication

Privileged access

Remote spoof

At 3rd party

At client

Federation issues

Backup authentication

Knowledge based authentication

(KBA)

Chained authentication

Information disclosure (e-mail)

No authentication

Other authentication

attack

Figure 14-2: Spoofing an external entity threat tree

Let’s fi rst consider spoofi ng threats at the server, whether you describe the threat as threats of the server being spoofed or of the server spoofi ng; it’s six of one, half a dozen of the other. The key is that the client is, for whatever reason, confused about the identity of the server it’s talking to. As discussed previously, the key to mitigating these threats is mutual authentication, and in particular cryptographic authentication. If you’re implementing a new login

www.it-ebooks.info

262 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c14.indd 07:52:38:AM 01/15/2014 Page 262

system, performing a literature review of the many previous systems and their failure modes is a very worthwhile endeavor. Spoofi ng of the client occurs in several forms, and an attack tree is a useful way to keep track of them.

The spoofi ng of an external entity threat tree is discussed in detail in Appendix B, “Threat Trees.”

Login Failures

A login system is designed to keep the wrong people out. In a very real way, it’s an interface that’s intended to stymie people and present error messages, except when the right person jumps through a set of carefully designed hoops. Therefore, you have to design for failures by both those you want to see fail and those you want to see succeed. These are a fi rst tension. Keep that tension in mind as you make choices. Do you tell the person what went wrong, and do you lock the account in some way? The second tension is that people are frustrated with security measures, and angry if their accounts are compromised. They’ll be angry if their low-value accounts are compromised and used for spam or whatnot, and they’ll be more deeply hurt if their bank account is drained or there are other real-world impacts.

No Such Account

“Incorrect username or password” is a common error message, based on the reasonable thinking that attackers who cannot discern if an account exists must waste energy attacking it, possibly also tripping alarms. There may well have been a time when this was commonly true, and there may yet be systems where account existence is hard to check. Today, with account recovery systems, it is harder to justify the usability loss associated with the “username or password” message. Additionally, with many systems using an e-mail address as a login mechanism, even if your system diligently hides all information disclosure threats around the existence of an account, someone else may well give it away (Roberts, 2012). It is time to accept that account existence is something an attacker can easily learn, and gain the usability benefi ts of telling real people that they’ve misspelled their account identifi er.

Insuffi cient Authentication

As the many inadequacies of passwords become increasingly apparent, services are choosing to look at more data available at login time to make an authentication decision. With a classical web browser, this will frequently include checking the IP address, a geolocation based on that IP, browser version information, cookies,

www.it-ebooks.info

Chapter 14 ■ Accounts and Identity 263

c14.indd 07:52:38:AM 01/15/2014 Page 263

and so on. If you are going to create such a system, note that there’s a threat. The specifi c version is cookie managers. You should plan for 20 percent–50 percent of your real customers to regularly delete their cookies (Nguyen, 2011; Young, 2011). Tell people if cookie deletion will affect the need for account recovery.

This can be generalized to a mismatch between expectations and your sys- tem. If other events will trigger account recovery, you should strongly consider setting people’s expectations. Be very clear about what information you expect would be diffi cult for an attacker to determine about your algorithms. That’s not to say you should reveal the information. Joseph Bonneau makes the claim that obscurity is an essential part of doing authentication well (Bonneau, 2012a). His arguments are solid, but they resolve the tension between limiting attacker information and usability in a way that leads to frustration for the real account owners. Of course, the frustration of your having a bank account drained is also very real. For more on obscurity, see the section “Secret Systems: Kerckhoffs and His Principles” in Chapter 16 “Threats to Cryptosystems.”

Account Lockout

When a login attempt fails, you can choose to lock the account for some length of time—ranging from a few seconds to forever. The forever end of the spectrum requires some form of reset management, an exercise left to the reader. If you select shorter lengths of time, you can use fi xed or increasing delays (also called backoff). You can apply delays to accounts or endpoints, such as an IP address or an address range, or both.

If the number of failures is represented as f, something like (f−3) × 10 or (f−5)2 seconds offers a reasonable mix of increasing security with each failure while not annoying people with unreasonable delays. (Of course, (f−5)2 is sort of pathological if you don’t handle the small integers well.) The system designer can either expose the backoff or hide it. Hiding it (by telling the person that their attempt to log in failed) may result in people incorrectly believing that they’ve lost their password, and thus driving the need for backup authentica- tion that works faster.

Requirements for scenarios of keyboard login attempts versus network login attempts may be different. It might be reasonable to allow more logins via a physical keyboard. Of course, a physical keyboard is sometimes a slippery concept, easily subject to spoofi ng over USB or Bluetooth.

Threats to “What You Have”

Using the authentication categories of what you have, what you are, and what you know, “what you have” includes things such as identity cards or cryptographic hardware tokens. The main threats to these are theft, loss, and destruction.

www.it-ebooks.info

264 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c14.indd 07:52:38:AM 01/15/2014 Page 264

The threat of theft is in many ways the most worrisome of the threats to “what you have.” Some of these authenticators, such as proximity cards to unlock doors do not require anything else. (That is, you don’t login to most doors with your handprint.) In most organizations, there is not a strong norm ensuring that the card is worn such that the face is easily visible, and an attacker who steals a card can simply put it in their wallet. Even with a “visible face” norm, matching photo to face tends to be challenging (see the next section for more discussion about this).

Other authenticator devices are often carried in bags, so someone who steals a laptop bag will obtain both the laptop and the authentication token. Some of these tokens only have a display, while others also have an input function, so using the token is a matter of what you have and what you know. The display- only tokens are cheaper, easier to use, and less secure. The appropriate trade-off is likely a matter of “chess playing.” If you use the display-only tokens, is that still what an attacker would go after, or does it make it hard enough that the attacker will attack somewhere else? As of 2012 or so, a number of companies are making authenticators that are the same size and thickness as credit cards and which have both input buttons and e-ink displays. These are more likely to be carried in a wallet than the older “credit card–size” tokens, which were very thick compared to a credit card.

The threats of loss and destruction are relatively similar. In each case, the authorized person becomes unable to authenticate. In the case of destruction or damage, there’s more certainty that the authentication token wasn’t stolen.

Threats to “What You Are”

Measuring “what you are” is a tremendously attractive category of authentica- tion. There’s an intuitive desire to have ways to authenticate people as people, rather than authenticate something they can loan, lose, or forget. Unfortunately, it turns out that the desire and the technical reality are different. All biometric systems involve some sort of sensors, which take measurements of a physi- cal feature. These sensors and their properties are an important part of the threat model. The data that is stored must be stored in a way which the sensor can reliably generate. The form into which sensor data is converted is called a template. All of this is shown in Figure 14-3, which looks remarkably similar to 14-1. Clever readers may notice that neither fi gure contains trust boundaries. Different systems place the trust boundaries in different places. For example, if you log in to your bank over the Internet using a fi ngerprint reader, there’s a different trust boundary than if that reader is located in their branch offi ce. Many of the threats against biometrics can be quickly derived from a model like the one shown in Figure 14-3.

www.it-ebooks.info

Chapter 14 ■ Accounts and Identity 265

c14.indd 07:52:38:AM 01/15/2014 Page 265

Authenticator(s) Sensor

Sensor

Response

Authenticator(s)

Please authenticate

Measures

Enrollment

Authentication Requestor

Account, template

Figure 14-3: A model of biometric authentication

Some of those threats include the following:

■ Attacking templates, either tampering with them to allow impersonation, or analyzing disclosed templates to create input that will fool the system

■ Acquiring an image of the body part being measured, and using it to create input that will fool the system. This is spoofi ng against the sensor.

■ Adjusting the body so that the measurement changes. This is often a denial-of-service attack against the system, possibly executed for innocent reasons.

Each of these is discussed in more depth over the next several paragraphs. Attacking templates may be easier than is comfortable for advocates of bio-

metrics, many of who have claimed that the stored template data could not be used to reconstruct the input that will pass the biometric. This has proven to be false for fi ngerprints (Nagar, 2012), faces (Adler, 2004), and irises (Ross, 2005), and will likely prove false for other templates. Resisting such attacks is probably in tension with managing the false positive and false negative rates associated with the system. Worse, the people who are being authenticated are usually not given a choice about the system. Therefore, even if reconstruction- resistant templates were designed, the distribution of incentives likely argues against their use.

Acquiring an image of the body part being measured can often come from an “in the wild” image. For facial biometrics, this may be as easy as search- ing (ahem) Facebook or LinkedIn. Fingerprints can come from a fi ngerprint left on a glass, a car door, or elsewhere. A search for “fi ngerprint” on Flickr shows thousands of images of people’s fi ngerprints. One person’s art leads to

www.it-ebooks.info

266 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c14.indd 07:52:38:AM 01/15/2014 Page 266

another’s artful attack. Of course, there are less artful attacks, such as chopping off the body part to be measured. This has actually happened to the owner of a Mercedes-Benz with a fi ngerprint reader—a Mr. Kumaran in Malaysia (see “Malaysia car thieves steal fi nger.” [Kent, 2005]).

Once you have a body part or an image thereof, you need to present it to the sensor in a convincing way. To reduce the incentive for taking body parts, there is sometimes “liveness” testing by the sensors. For example, fi ngerprint sensors might look for warmth or evidence of a pulse, while eye scanners might look for natural movements of the eye. Such testing turns out to be harder than many people expect, with facial recognition bypassed by waving a photo around; and, famously, fi ngerprints rendered with Gummy Bears (Matsumoto, 2002), recently repeated to some fanfare when a well known phone manufacturer experienced the same problem, possibly having failed to perform a literature review (Reiger, 2013).

Some systems measure activity, such as typing or walking, so the “image” captured needs to be more complex, and replaying it may similarly be harder, depending on where the sensor and data fl ows are relative to the trust bound- aries. Attacks that are easy in a lab may be hard to perform in front of an alert, motivated, and attentive guard. However, in 2009, it was widely reported that Japan deported a South Korean woman who “placed special tapes on her fi ngers” to pass through Japan’s fi ngerprint checks at immigration (Sydney Morning Herald, 2009).

For honest people, changing their fi ngerprints (or other biometrics) seems very diffi cult. However, there are a variety of factors and scenarios that make fi ngerprints harder to take or measure, including age; various professions such as surgeons, whose fi ngerprints are abraded by frequent scrubbing; hobbies such as woodworking, where heavy use of sandpaper can abrade the ridges; and cancer treatment drugs (Lyn, 2009).

There are also a few threats that are harder to see in the simple model. The fi rst is “insult rates,” the second is suitability and externalities. The biometrics industry has long had a problem with what they call the “insult rate.” People are deeply offended when the machine doesn’t “recognize” them, and if the people being measured are members of the general public, staff need to be trained to handle the problem gracefully. The second insult issue is that many people associate fi ngerprinting with being treated like a criminal. Is it suitable to treat your customers in a way that may be so perceived? Organizations should carefully consider the issue of offense, and well-meaning people should also consider the issue of desensitization caused by overuse of biometrics. This is an externality, where the act of an organization has a cost on others.

Many of the desirable properties of authenticators are not available in bio- metrics. Biometric sensors are less precise than a keyboard, which makes it harder to derive cryptographic keys. Another property that you’d like for your

www.it-ebooks.info

Chapter 14 ■ Accounts and Identity 267

c14.indd 07:52:38:AM 01/15/2014 Page 267

authentication secrets is that they are shared between one person and one orga- nization, and you want to change those secrets now and then. But most people only have 10 fi ngers. (A few have more, and more have less.) Therefore, the threat of information disclosure by anyone who has a copy of your fi ngerprint has an impact on everyone who relies on your fi ngerprint to authenticate you. Of course, that applies to all biometrics, not just fi ngerprints. Overuse of biomet- rics makes these threats more serious for everyone who needs to rely on them.

One last comment on biometrics before moving on: Photographs are also a biometric, one that people can use very well, if the photograph is shown to motivated people in a high quality and reasonable size. People do an excellent job of recognizing family and friends. However, using photographs to match strangers to their ID photos turns out to be far more diffi cult, at least when the photograph is the size that appears on a credit card. (See Ross Anderson’s Security Engineering, Second Edition [Wiley, 2008] for a multi-page discussion of the issue. To summarize, participants in an experiment rarely detected that a photo ID did not match the person using it. However, it appears that the use of such photos may act as a deterrent to casual crime.)

Threats to “What You Know”

Passwords are the worst authentication technology imaginable, except for all those others that have been tried from time to time. We all have too many of them, decent ones are hard to remember, and they’re static and exposed to a potentially untrustworthy other party. They are also memorizable, require no special software, can be easily transmitted over the phone, and are more crisp than a biometric, which allows us to algorithmically derive keys from them. For a detailed analysis of these trade-offs, see Joseph Bonneau’s “The quest to replace passwords” (Bonneau, 2012c). These tradeoffs mean that passwords are unlikely to go away, and most systems that have accounts are going to need to store passwords, or something like them, in some way.

In the aforementioned paper, Bonneau and colleagues lay out a framework for evaluating the usability, deployability, and security of authentication systems. Their security characteristics are outlined in the form of a security proper- ties checklist, which will be useful to anyone considering the design of a new authentication scheme. They are listed here without discussion because anyone considering such a task should read their paper:

1. Resilient to physical observation

2. Resilient to targeted impersonation (such as attacks against knowledge- based backup authentication)

3. Resilient to throttled guessing (this is my online attacks category)

4. Resilient to unthrottled guessing (this is offl ine attacks)

www.it-ebooks.info

268 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c14.indd 07:52:38:AM 01/15/2014 Page 268

5. Resilient to internal observation (such as keylogging malware)

6. Resilient to leaks from other verifi ers

7. Resilient to phishing

8. Resilient to theft

9. No trusted third party

10. Requiring explicit consent

11. Unlinkable

Threats to Passwords

Threats to password security can be categorized in several different ways, and which is most useful depends on your scenario. I’ll offer up a simple enumeration:

■ Unintentional disclosure, including passwords on sticky notes, wikis, or SharePoint sites, and phishing attacks

■ Online attacks against the login system

■ Offl ine attacks against the stored passwords

Online attacks are attempts to guess the password through defenses such as rate limiting. Offl ine attacks bypass those defenses and test passwords as quickly as possible, often many orders of magnitude faster. Password leaks are a common problem these days, and they’re a problem because they enable offl ine attacks, ranging from simple lookups to rainbow tables to more complex crack- ing. Good password storage approaches can help guard against offl ine attacks. Passwords stored on web servers is a common issue in larger operational envi- ronments. Phishing attacks come in a spectrum from untargetted to carefully crafted after hours of research on sites such as Facebook or LinkedIn. There's a model of online social engineering attacks is in Chapter 15.

Password Storage

This section walks you through information disclosure threats to stored pass- words, building up layers of threats and cryptographic mitigations to bring you to an understanding of the modern approaches to password storage.

If you don’t cryptographically protect your list of usernames and passwords, then all that’s required to exploit it after information disclosure is to look up the account the attacker wishes to spoof, and then use the associated login. The naive defense against this is to store a one-way hash of the password. (A one-way hash is a cryptographic function that takes an arbitrary input and produces a fi xed-length output.) The way to attack such a list is to take a list of words and for each word in the list, hash the word, and search for that hash in the list of

www.it-ebooks.info

Chapter 14 ■ Accounts and Identity 269

c14.indd 07:52:38:AM 01/15/2014 Page 269

hashed passwords. The list of words is called a dictionary, and the attack is called a dictionary attack. Each hashed dictionary word can be compared to each stored password. (If it matches, then you know what word you hashed, and that’s the password.)

The defense against the speedup from comparing to each stored password is to add a salt to the password before hashing it. A salt is a random string intended to be different with different account passwords. Using a salt slows down an attacker from comparing each hash to all stored passwords to compar- ing each hash to that one password with the same salt (or those few passwords with the same salts). Salts should be random so that [email protected] has a different salt at different systems, and thus her stored, hashed password will be different even if she’s using the same password at different systems. Salts are normally stored as plaintext with the hashed passwords.

Attackers might try to use modifi ed versions of dictionary words. Software has been available for a long time that will take a list and permute each word in it, by changing o to 0, l (the letter, el) to 1, adding punctuation, and so on. Dictionaries are available for a wide variety of common languages, ranging from Afrikaans to Yiddish. There are also password-cracking lists of proper names, sports teams, and even Klingon. It’s possible to store the hashed wordlist for a given hash, engaging in a “time-memory trade-off.” These lists are called rainbow tables, and many are downloadable from the Internet. (Technically, rainbow tables store a special form of chains of hashes, which matter more if you are going to design your own storage mechanism; but given that smart people have spent time on this, you should use a standard mitigation approach.)

Unless you’re implementing an operating system, it’s almost certain that you’ll want to store passwords that have been hashed and salted. The best pattern for this is roughly to take a password and salt, and shove them through a hash function thousands of times. The goal is to ensure that there’s time after the detection of an accidental information disclosure for your customers to change their passwords. There are three common libraries for this: bcrypt, scrypt, and PBKDF2. All are likely to be better than anything you’d whip up yourself, and they are freely available in many languages.

■ bcrypt: This is what I recommend. There is more freely available code, which is better documented and has more examples (Muffett, 2012). The bcrypt library also has an adaptive feature, which enables password stor- age to be strengthened over time without access to the cleartext (Provos, 1999). Do note that the name bcrypt is overloaded; at least one Linux package named bcrypt performs Blowfi sh fi le cryptography.

■ scrypt: This may offer more security if you use it correctly. However, while documentation explains how to use the function for fi le encryption, it does not clearly address its use as a password storage tool (as opposed to a fi le encryption tool).

www.it-ebooks.info

270 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c14.indd 07:52:38:AM 01/15/2014 Page 270

■ PBKDF2 (Password-Based Key Derivation Function 2): This offers less defense, does not offer adaptivity, and there’s only one reason to use it when bcrypt is available for your platform: because it’s a NIST Standard (Percival, 2012; Openwall, 2013).

Even after you hash the passwords, you want to ensure that the hashes are hard to get. Even with salting, modern password cracking with dictionaries of common passwords is shockingly fast, on the order of eight billion MD5 passwords tested per second (Hashcat, 2013). At that speed, testing the million most common passwords with 100 variants of each takes less than a second.

If you are implementing an operating system, then the storage of passwords (or password equivalents) is a more complex issue. There are good usability reasons to keep authenticators in memory. This allows for the authenticator to be used without bothering the user. For example, Windows domain members can authenticate to a fi le share or Exchange without retyping their password, and unix users will often use ssh-agent, or the Mac OS keychain to help them authenticate. These patterns also lead to issues where an attacker can imperson- ate the account if they’re inside the appropriate trust boundary.

There is no great solution short of keeping them out, which is, empirically, challenging. Relying on hardware such as smartcards or TPMs to ensure that the secrets don’t leave the machine is helpful, as it provides a temporal bound, but an attacker who can execute code as the user can still authenticate as the user. The alternative of requiring action per authentication is highly inconvenient and would probably not solve the problem, because frequent authentication requests would desensitize people to requests for authentication data.

Using static strings for authentication means that the system to which a person is authenticating can spoof that person. There are a variety of clever cryptographic ways to avoid this, all of which have costs at deployment time, or other issues that make them harder to deploy than passwords (as noted earlier in “Authentication”).

Password Expiration

In an attempt to respond to the threat of information disclosure involving passwords, many systems will expire them. By forcing regular changes, an attacker who comes into possession of a password has a limited window of time in which to use it. When systems force password changes, there are a few predictable ways in which people respond. Those include changing their pass- word and then changing it back, transforming it to generate a new password (password1 becomes password2, or Decsecret becomes Jansecret), or picking a completely new password. (Hey, it could happen, and the person might not even write down the new password.)

This leads to an argument for controlling those human behaviors by storing password history, and sometimes also limiting the rate of password change.

www.it-ebooks.info

Chapter 14 ■ Accounts and Identity 271

c14.indd 07:52:38:AM 01/15/2014 Page 271

Systems that store password history create an additional risk, which is that the stored passwords might be obtained, and an attacker who has access to a series of passwords can use either human or algorithmic pattern recognition to make very accurate guesses about what a person’s next password is likely to be (Thorsheim, 2009; Zhang 2010). If you must store history, it is probably sensible to use an in-place adaptive algorithm such as bcrypt to store the historical pass- words with a level of protection that would lead to unacceptable performance in interactive login.

I am not aware of any evidence that password expiration systems have any impact on the rate at which compromises happen. There is excellent evidence that normal people do not change their passwords unless forced, and as such there is a cost to password expiry that is hard to justify. The main reason to pay the cost of such a system is because a compliance program insists on it.

Authentication Checklist

This checklist is designed to be read aloud at a meeting. You’re in a bad state if for any of these questions:

■ You can’t answer yes.

■ You can’t articulate and accept the implications of a no.

■ You don’t know.

1. Do we have an explicit list of what data is used in authentication?

2. Is it easy for us to add factors to that list?

3. Do we have easy to understand error messages?

4. If we’re hiding authentication factors, what is our estimate of how long it would take for an attacker to fi nd out about each?

5. Have we reviewed at least the information in the book for each authenti- cation factor?

a. Have we looked up the relevant references?

b. Have we looked for additional threats based on those references?

6. Are we storing passwords using a cryptographically strong approach?

Account Recovery

We’ve all seen the “Forgot your password?” feature that so many sites offer. There are many varieties of this, including the following:

■ E-mail authentication

■ Social authentication

www.it-ebooks.info

272 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c14.indd 07:52:38:AM 01/15/2014 Page 272

■ Knowledge-based authentication

■ Secret question/secret answer (or)

■ Data from public records, such as your address on a given date

There are many varieties of these systems because, much like passwords, they are highly imperfect. This section provides an overview of the trade-offs involved. It begins with a discussion of time in account recovery, then explains account recovery via e-mail and social authentication, both of which are simple and relatively secure when compared to the more familiar knowledge-based (“secret question”) systems covered at the end of this section.

All of these systems are abused by attackers. Famous examples include U.S. vice presidential candidate Sarah Palin’s e-mail being taken over (the questions were birth date, zip code, and “Where did you meet your husband?” [Campanile, 2008]).

These systems should focus on recovering the account. The focus on “forgot- ten passwords” often leads system designers into the trap of giving people their previous password. This leads to the plaintext storage of passwords, which is usually not needed. Customers don’t really care about their password or need it back. They care about the account (or what it enables) and need access to the account. Therefore, you should focus on restoring account access. The only substantial exception to this is encrypted data, for which it may be the case that an encryption key is derived from the password. You should investigate other ways to back up those encryption keys, such that you don’t need to store passwords in plaintext.

When an account is recovered, it may be possible to throw away informa- tion that an attacker could exploit, such as payment information or mailing address. This information is usually easy to re-enter and has value to an attacker. (Addresses are useful for chained authentication attacks and stalking.) If you inform people why the data is gone, they are likely to be understanding. What data you choose to destroy as part of the account recovery process will depend on your business.

Time and Account Recovery

Time is an important and underrated ally in account recovery systems. The odds that users will need to use their account recovery option within fi ve minutes (or even fi ve days) of their last successful login are low. You might want the account recovery options for an account to be disabled until some specifi ed amount of time has elapsed since the last successful login. (This point was made by my colleague and usable security expert Rob Reeder.) It is tempting to suggest offering fake account recovery options until then, but that might frustrate your real users.

www.it-ebooks.info

Chapter 14 ■ Accounts and Identity 273

c14.indd 07:52:38:AM 01/15/2014 Page 273

The person driving an account recovery process might be the authorized account holder or an attacker. Your process should balance ease of use with challenge, and speed with an opportunity for the real account holder to inter- vene. For example, if you send password recovery e-mails, you should notify the person via every channel available to you. These notifi cations should contain instructions for the case that the receiver didn’t initiate the account recovery, and possibly an explanation of why you’re “spamming” them. These notifi cations and delays are more important when other risk factors are visible (for example, the account logs in from one IP address 90 percent of the time, but today logged in from the other side of the world).

Time also plays into account recovery in terms of what happens after the instant of getting into the account. Some system designers think they’re done when a new password has been set. If the password was changed by an attacker, how does your customer recover his or her account? Perhaps after account recovery (but not normal password changes) the old password could work for some time period? However, then an attacker with the password is not locked out. There is no single obvious answer, and the right answer will differ for accounts associated with close relationships (work, banks, etc.) versus casual accounts.

Time is also a threat to account recovery systems. Over time, information that people have given you will decay. Their credit cards will become invalid, and their e-mail addresses, billing addresses, and phone numbers will likely change. If you rely on such information, consider allowing it to decay over time, being revalidated or removed from the system’s recovery options.

E-mail for Account Recovery

If you have an e-mail address for your customer, you can send mail to it. That mail can contain a password or a token of some form. If you e-mail the customer a password, you should e-mail a new, randomly generated password. In fact, that should be all you can do, because you should take the preceding advice and not be able to e-mail them their old password. However, e-mailing them a password exposes the password to information disclosure threats on a variety of network connections and in storage at the other end. Some people will argue that it’s better to not expose the password, but instead use a one-use token that allows the person to reset their password. The token should be a large random number, say 128–1,024 bits. You can send this either as a string they copy and paste into a browser form or as a URL. Hopefully it is obvious that the code which actually resets the password must confi rm that the account actually requested a password reset, that the e-mail didn’t bounce, and that the token is the one that was sent.

Either approach is vulnerable to information disclosure attacks. These attacks rely on either network sniffi ng or access to the backup e-mail address. You can

www.it-ebooks.info

274 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c14.indd 07:52:38:AM 01/15/2014 Page 274

partially mitigate these threats by crafting a message that does not contain all the information needed to recover the account. For example, if the system sends a URL with a recovery token, then it should not contain the account name. Thus, someone who intercepts or obtains the e-mail would need additional informa- tion if the account name is not the e-mail address. In systems where the e-mail address is the account name, that obviously doesn’t work. You can add a layer of mitigation by checking cookies and possibly also by confi rming that the browser they’re using is sending the same headers as it was when someone requested a password reset. If you do that, there is a cookie deletion threat, which can be managed by telling people that they need to keep the page where they’ve requested a password reset.

Knowledge-Based Authentication

A variety of systems use various non-password information, which, it is hoped, only the real person behind the account knows. Of course, such information isn’t really perfect for authentication. It must be known by the relying party so that it can be confi rmed. Such information has the weakness that it may be known to other relying parties, or it may be discoverable by an attacker. The spectrum of such information runs from information that’s widely available to the public through information that might be known only to the relying party. (Known only to the relying parties is a property that many hope is true of passwords.) This section considers three forms of knowledge-based authentication:

■ Secret questions/secret answers

■ Data from public records, such as your address on a given date

■ Things only a few groups other than the organization and customer know, such as the dollar amount of a given transaction. “What is your password?” is one logical end of this knowledge-based authentication spectrum.

As mentioned earlier, ideally, the answers to these questions are known only to the real person behind the account, are exposed only to the proper account system, and people know they should keep it secret. Each of these systems can be threat modeled in the same ways.

Security

There are several classes of attack on the security of questions. The biggest ones are guessing and observation. The diffi culty of each can be quantifi ed, (possibly with respect to a specifi c category of attacker). Guessing diffi culty is based on the number of possible answers. For example, “what color are your

www.it-ebooks.info

Chapter 14 ■ Accounts and Identity 275

c14.indd 07:52:38:AM 01/15/2014 Page 275

eyes?” has very few possible answers. Guessing diffi culty is also based on prob- ability. Brown eyes outnumber blue, which outnumber green. There are also attacks where the answer to the question can be found (observed) elsewhere. For example, information about addresses is generally public. This approach to measurement has been formalized by Bonneau, Just, and Matthews (Bonneau, 2010). They measured the guessing diffi culty for people’s names (parent names, grandparent names, teacher names, best friends) and place names (where you went to school, last vacation). They also comment on the relatively small num- ber of occupations that our grandparents engaged in, and the small number of movies that are common favorites.

Observation diffi culty refers to how hard it would be to fi nd the answer from a given perspective. For example, mothers’ maiden names are often accessible because of genealogy websites. Some questions are also subject to attack by memes such as “Your Porn Star Name” or “20 Facts About Me.” The current fi rst search result for “porn star name” suggests that you should form it from “your fi rst pet’s name and the street you grew up on.” Relying on such information being secret is a bad choice. It is a bad choice because people don’t expect that the information in question will need to be kept private. No one would fi nd it entertaining to be asked for a password in order to generate a porn star name; they’d fi nd it worrisome. Observation diffi culty can only be measured in rela- tion to some set of possible observers. Therefore, attacker-centered modeling makes sense for measuring observation diffi culty.

Usability

Knowledge-based authentication systems also suffer many usability problems, including the following:

■ Applicability: Does the item apply to the whole population? Questions about the color of your fi rst car only apply to those who have owned cars; questions about the name of your fi rst pet only apply to pet owners.

■ Memorability: Will people remember their answer? Studies have shown that 20 percent or so will forget their answers within three months (Schechter, 2009a).

■ Repeatability: Can someone correctly re-enter their answer (“Main St” vs. “Main Street”), and is the answer still the same as it was 10 years ago? (Is Avril Lavigne still your favorite singer?) Repeatability is also called stability in some analyses.

■ Facts versus preferences: There is some evidence that preferences are more stable over time. However, entropy, and thus resistance to guessing attacks, is lower, and some preferences may be revealed to observation.

www.it-ebooks.info

276 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c14.indd 07:52:38:AM 01/15/2014 Page 276

■ Privacy: Some questions can seem intrusive, creepy, or too personal to users. Also, under the privacy laws of many countries, information can only be used for the purpose for which it was collected. Using such infor- mation for authentication may run afoul of such laws.

■ Internationalization: The meaning of some questions does not carry across cultures. Facebook has reported that people in Indonesia don’t name their pets, and so the usual answer to that question is “cat” (Anderson, 2013b).

Aligning security with the mental models of your customers is great if you can manage it, but it may be suffi cient to avoid surprising your customer. To illustrate the difference, a website decided that cookies were part of the login process, and if your cookies were deleted, you would need not only your pass- word, but also your secret question. Because my secret answer was something like “asddsfdaf,” this presented a problem. The design surprised me, and pre- vented me from logging back in, which is the state my account is probably in to this day.

Additionally, systems can be open question or open answer. Open-question systems are of the form “please enter your question.” It turns out that such systems result in questions with few answers (e.g., “What color are my eyes?”; “What bank is this?”).

If You Must Use a Knowledge-Based Authentication System

Look for information in your system that can be used, such as “What was the dollar amount of your last transaction?” The information should be available to few parties;, thus, the last LinkedIn connection you added is poor. If your organization has features to enable social sharing, nothing that is shared should be usable to authenticate. It should also be hard for an attacker to infl uence; thus, “Who was the last person who e-mailed you?” is bad. Finally, it should be hard for an attacker who has broken in to extract all the information they would need to retake the account.

One way to reduce the chance that the attacker can extract all the information from the account is to augment the last dollar amount question with a fi nancial account number question. If you use fi nancial account numbers, be sure to require more than fi ve digits. (Four digits is the U.S. limit to how many may be displayed; using fi ve means an attacker only needs to guess, on average, fi ve times to get in. Consider asking for six or eight.) When someone gets the answers right, consider sending a message to all the backup contact information you have, offering the real account owner a chance to challenge the authentication. That said, however, consider using social authentication, covered in the section of the same name, in place of knowledge-based authentication.

www.it-ebooks.info

Chapter 14 ■ Accounts and Identity 277

c14.indd 07:52:38:AM 01/15/2014 Page 277

Chained Authentication Failures

As many organizations develop backup authentication schemes, avoiding chained and interlock authentication failures is an important issue. A chained failure is where an attacker who takes over an account at one site can see all the infor- mation they’ll need to authenticate to another site. An interlocking failure is where that’s bi-directional. For example, if you set your Gmail account recovery to Yahoo mail, and your Yahoo mail to recover with Gmail, then your recovery options are interlocked. Unfortunately, it’s normally an issue that end users must manage for themselves.

N O T E This issue came to widespread attention after an August 2012 article in Wired, “How Apple and Amazon Security Flaws Led to My Epic Hacking” (Honan, 2012).

It’s worth recounting the details and then analyzing them. The names of the compa-

nies are taken from the Wired story. Note that these are large, mature companies with

employees focused on security. The story is presented solely to help others learn. The

steps that led to the “epic hacking” are as follows:

1. Attacker calls Amazon and partially authenticates with victim’s name, e-mail address, and billing address. Attacker adds a credit card number to the account.

2. Attacker calls Amazon again, authenticating with victim’s name, e-mail address, billing address, and the credit card number added in step 1. Attacker adds a new e-mail address to the account.

3. Attacker visits Amazon.com, and sends password reset e-mail to e-mail address from step 2.

4. Attacker logs in to Amazon with the new password. Attacker gathers last four digits of real credit card numbers.

5. Attacker calls Apple with e-mail address, billing address, and last four digits of the credit card. Apple issues a temporary credential for an iCloud account.

6. Attacker logs into iCloud and changes passwords.

7. Attacker visits Gmail and sends account reset e-mail to compromised Apple account.

8. Attacker logs into Gmail.

So what went wrong here?

■ All the attacks used backup authentication methods.

■ Several places used data that’s mostly public to authenticate.

■ Amazon allowed information that can be used to authenticate to be added with less authentication.

■ The Wired writer took the advised approach and interlinked all of his accounts.

www.it-ebooks.info

278 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c14.indd 07:52:38:AM 01/15/2014 Page 278

Returning to the issue of chained authentication failures, it is hard to prevent people from using their preferred e-mail provider. It is perhaps impossible for a company to determine where it stands in a daisy chain of authentications. It might be possible to detect authentication loops, manage the privacy concerns such a process could raise, and handle the loop in a clever way.

Social Authentication

As mentioned earlier, the traditional methods of authentication are what you have, what you know, and what you are. In a 2006 paper, John Brainard and colleagues suggested a fourth: who you know, presenting it as a way to handle primary authentication (Brainard, 2006).

Social authentication is authentication based on social contact of various forms. It is already in use at a great many businesses that send a replacement password to your manager if you’re locked out. This leverages the expectation that managers ought to be able to get in touch with their employees and authen- ticate them. Social authentication may also help with making your system more viral. When someone is selected as a trustee, you might want to round-trip an e-mail message to them to ensure that the details are valid, and you may want to recheck that address from time to time. Such messages can have the added value of marketing your service, as you might need to explain what the service is. Try hard to not eliminate the security value by spamming.

Passive Social Authentication

In early 2012, Facebook deployed a backup authentication mechanism that asks users to identify a set of people connected to that user (Fisher, 2012; Rice, 2011). This is passive authentication in the sense that your Facebook contacts are not actively involved in the authentication process. Such systems are easier to deploy if you have a rich social graph of some type.

However, the data needed to bypass such a system is sometimes available to (and perhaps via) Facebook, LinkedIn, Google, and a growing number of other companies. Other problems include photos that don’t include an identifi able face and photos that contain a name badge (for example, from a conference). Facebook also explored use of their social graph to make collaboration more diffi cult, selecting people who are less likely to be known to a single attacker, or friends of the attacker. Of course, because the attacker is unknown, they can only do this by looking for distinct subgraphs (Rice, 2011).

Active Social Authentication

Active social authentication uses a real-world relationship to recover access to an account. Such systems are most obvious when a manager gets a new password

www.it-ebooks.info

Chapter 14 ■ Accounts and Identity 279

c14.indd 07:52:38:AM 01/15/2014 Page 279

and provides it to an employee, but it’s possible to build deeper systems that use a variety of account “trustees.” One such system was developed experimentally by Stuart Schechter, Serge Egelman, and Rob Reeder (Schechter, 2009b). It is reviewed here in depth because it’s superior in many ways to knowledge-based systems, and because many of the details show interesting design points.

They use a system of trustees who reauthorize access to an account. Their test used a system of four trustees selected by the account holder, and required concurrence of three trustees. When an account goes into recovery, each trustee is sent an e-mail with a subject line of “**FOR YOU ONLY**.” The body of the message starts with “Do not forward any part of this e-mail to anyone,” and con- tinues with an explanation of what to do. The e-mail then goes on to encourage the recipient to visit a given URL. When the recipient does so, he or she is asked to explain why an account recovery code is being requested. The reasons are listed from riskiest to least risky so that someone who selects the fi rst option(s) will see an additional level of warning before being allowed to get a recovery code. The reasons given are as follows:

■ Someone helping William Shakespeare (or who claimed to be helping) asked for the code

■ An e-mail, IM, SMS, or other text message that appears to be from William Shakespeare asked for the code

■ William Shakespeare left me a voicemail asking for the code, and I will call him back to provide it

■ I am speaking to William Shakespeare by phone right now and he has asked for the code

■ William Shakespeare is here with me in person right now and he has asked me for the code

■ None of the above reasons applies. I will provide my own:

After selecting one of these, and possibly seeing additional warnings about signs of fraud, the trustee is asked to pledge to the veracity of the answer and their understanding of the consequences of being duped. They are required to type their name and then press a button that says “I promise the above pledge is true.” The trustee is then given a six-character code, and instructed to provide it either in person or in a phone call. The system will then e-mail all the other trustees, encouraging them to call or visit the account holder. This process has several useful properties, including increased security and alerting the account holder if they didn’t initiate the account recovery.

There are a number of issues with the system as presented. One is time. This social approach takes longer to complete than knowledge-based approaches. Another is that in the initial reliability experiment, many participants did not succeed at recovering their account. Of 43 participants in the weeklong

www.it-ebooks.info

280 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c14.indd 07:52:38:AM 01/15/2014 Page 280

reliability experiment, 17 abandoned the experiment. Of the 26 who actively participated, 65 percent were able to recover their account. Two of the failures didn’t remember their trustees or use the “look them up” feature, and another two were too busy to participate in the study. Looking at the 22 who actively participated, 77 percent of them succeeded at account recovery, which seems poor, but it’s comparable to knowledge-based systems. It seems likely that more people would succeed if they really were trying to recover their account, and had nominated appropriate trustees.

Despite these concerns, social authentication is a promising area for account recovery, and I am optimistic that new systems will be developed and deployed, replacing knowledge-based account recovery for casual accounts.

Attacker-Driven Analysis of Account Recovery

Knowing who is attacking is a useful lens into how account recovery systems work, because there are sets of attackers who obviously have more access to data. Spouses or ex-spouses, family members, and others will often know (or be able to tease out information about) your fi rst car, the street you grew up on, and so on. Attackers include, but are not limited to, the following:

■ Spouses

■ Friends

■ Social network “friends” and contacts

■ Attackers with current access to your account

■ Attackers with access to an account on another system

■ Attackers with access to a data broker’s data

In “It’s No Secret: Measuring the Security and Reliability of Authentication via ‘Secret’ Questions” (Schechter, 2009a), the authors categorize 25 percent of question/answer pairs in use at large service providers as vulnerable to guess- ing by family, friends, or coworkers. Spouses generally have unfettered access to records, fi nancial instruments such as credit cards, and the like. Friends will often know about biographical information. Social network contacts can use attacks like the “Your Porn Star Name” game to get access to elements of personal history that social networks such as Facebook don’t make public. The social networks themselves change what information they show as public, meaning your analysis of knowledge-based account recovery must be regularly revisited.

Another important set of attackers is online criminals, who have found ways to access information stored by data brokers. This information is often marketed as “out of wallet” authentication. That criminals have access to such data empha- sizes the need to use data known only to you and your customers (Krebs, 2013).

www.it-ebooks.info

Chapter 14 ■ Accounts and Identity 281

c14.indd 07:52:38:AM 01/15/2014 Page 281

Multi-Channel Authentication

Many systems claim to use additional channels for authentication. Some of them even do, but modeling what counts as a channel in a converged world is challenging. In particular, when a smartphone is synced to a computer, the act of syncing often involves giving one computer full authority to read or write to anything in the other. This means that smartphones are a risky source of additional channels. The computer is a threat to the smartphone, and vice versa. Physical mail is a fi ne example of an additional channel, with the obvious issue that it’s quite slow.

Account Recovery Checklist

This checklist is designed to be read aloud at a meeting. You’re in a bad state if for any of these questions:

■ You can’t answer yes.

■ You can’t articulate and accept the implications of a no.

■ You don’t know.

1. Do we have explicit reasons that we need an online account recovery feature?

2. Have we investigated active social authentication as an approach?

3. If you’re using active social authentication, then have we tried to address the attacks by friends problem?

4. Have we tested the usability and effi cacy of our approach for both the authorized customer and one or more sorts of attacker?

If you’ve decided to use a knowledge-based authentication system, you’ll probably jump from #2 to the following list:

1. Will our system use only information that only we and our customer should know?

2. Will our system resist attacks like “Your Porn Star Name” game?

3. Will our system resist attack by spouses?

4. Will our system resist attackers who have broken in and scraped the account?

5. Will our system resist attackers who have access to information bought and sold by data brokers?

www.it-ebooks.info

282 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c14.indd 07:52:38:AM 01/15/2014 Page 282

6. Have we considered the various usability aspects of knowledge-based authentication?

7. Do we use time and account-holder notifi cation as allies?

Names, IDs, and SSNs

A great many technological systems seem to end up using human names and identifi ers. The designers of these systems often make assumptions that are not true, or are perhaps even dangerous. A good model of names, identifi ers, and related topics helps you threat model effectively, limiting what you expect to gain from using these identifi ers. Many reliability and usability threats are associ- ated with these topics. In this section, you’ll learn about names, identity cards and documents, social security numbers and their misuse, and identity theft.

Names

“What’s in a name?” asked Shakespeare, for a very good reason. A name is an identifi er, and between diminutives, nicknames, and married names, many people use more than one name in the course of their life. Useful names are also relative. You and I mean something different by “mom”‘ or “my wife.” I often mean different people when I say “Mike.” (At work, my three-person team has a Mike, and my boss’s boss is named Mike; and that’s just at work.)

“Real Names”

It is tempting to set up a new system that requires people to use their “real names.” It is widely believed that a real-name system is easier to validate, and that people will comment more usefully under their real names. Both of these claims are demonstrably false. South Korea rolled out and then scrapped a regu- lation requiring real names, having found that it did not prevent people “from posting abusive messages or spreading false rumors” (Chosunilbo, 2011). Internet comment management company Disqus has analyzed its data and discovered that 61 percent of its commenters were obviously using pseudonyms, and only 4 percent used what appeared to be a real name. They also found that both real name and pseudonym comments were marked as spam at approximately the same rates (9 percent and 11 percent respectively).

However, the cost of real name requirements can be high. Google’s G+ system required real names on rollout. They did so with a set of “name police,” who rejected real names and violated their own policies. These “nymwars” as they came to be known had a dramatic effect on the adoption of Google+, and I’ve argued elsewhere that it was singularly responsible for G+ not killing Facebook

www.it-ebooks.info

Chapter 14 ■ Accounts and Identity 283

c14.indd 07:52:38:AM 01/15/2014 Page 283

(Shostack, 2012b). Germany has a law forbidding such policies (Essers, 2012). Both demands for real names and attempts to impose rules on those names continues to happen and cause outrage. For example, see “Please Enter a Valid Last Name” (Neilsen Hayden, 2012).

Patrick McKenzie published a list titled “Falsehoods Programmers Believe About Names.” The list is worth reading in full, but here are the fi rst nine items (McKenzie, 2010):

1. People have exactly one canonical full name.

2. People have exactly one full name which they go by.

3. People have, at this point in time, exactly one canonical full name.

4. People have, at this point in time, one full name which they go by.

5. People have exactly N names, for any value of N.

6. People’s names fi t within a certain defi ned amount of space.

7. People’s names do not change.

8. Peoples names change, but only at a certain enumerated set of events.

9. Peoples names are written in ASCII, or some other single character set.

The list continues through another 30 false assumptions. Two items can be added to the list, based on reading the comments. First, you can exclude certain characters from people’s names without offending them. Second, the name people give you is one they use outside your system. Many programmers seem offended at being asked to deal with this. A great many comments are of the form “people should just deal with it and offer a Romanized version of their name.” Other commenters point out that lying about your name is in some instances a felony, and how changes in names or transliteration can lead to real problems. Many Americans were introduced to this by the TSA (airport security) when they checked that identity documents matched names on tickets. Discrepancies such as “Mike/Michael” lead to trouble. At best, a system that refuses people’s preferred names threatens to appear arrogant and offensive to those who are mis-addressed.

So what should you do? Don’t use human names as account names. Treat human names as a single fi eld, and use what people enter. If you’d like to be informal, ask your customers for both a full name and how the system should address them.

Account Names

Unlike human names, it is possible to make account names unique by having some authority that approves names before they can be used. (I am aware of some countries that do this at birth, but none that require a renaming when

www.it-ebooks.info

284 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c14.indd 07:52:38:AM 01/15/2014 Page 284

someone moves there.) Names can be unique or currently unique. Someone had my microsoft.com e-mail before me. The last I heard, his kids were knocking it out of the little league park, and his wife still loved him (muscle memory on typing his e-mail address, or so she claimed). Therefore, uniqueness is helpful, but it leads to all the easily remembered e-mail addresses being taken.

If your account system relies on someone else maintaining uniqueness for- ever, while in fact they maintain uniqueness at a given time, you can run into trouble. For example, if that other Adam and I both sign up for LinkedIn with our @microsoft.com addresses, what should LinkedIn do? They can probably handle that by now, and you’ll need to do so as well. (A draft of this section threatened to magnify this problem by including my e-mail address.)

“Meaningful ID”

Carl Ellison has coined the term meaningful ID and defi ned it as follows: “A meaningful ID for use by some human being is an identifi er that calls to that human user’s mind the correct identifi ed entity” (Ellison, 2007). This is a good model for an important use case for names, which is to call to mind a specifi c person.

Ellison provides some requirements for meaningful IDs:

1. Calling a correct entity to the human’s mind implies that the human being has a body of memories about the correct entity. If there are no such memories, then no ID can be a meaningful ID.

2. What works to call those memories to one observer’s mind may not work for another observer. Therefore, a meaningful ID is in general a function of both the identifi ed entity and the person looking at it.

3. The meaningful ID needs to attract attention or be presented without distracting clutter, so that it has the opportunity to be perceived.

4. When a security decision is to be made, the meaningful ID(s) must be derived in a secure manner, and competing IDs that an attacker could introduce at-will should not be displayed.

Ellison suggests that the best identifi er to present to a person is probably a nickname selected by the person who must rely on it, or a picture taken by that relying party. Implicitly, a system must perform the translation, and be designed to mitigate spoofi ng threats around the nickname shown.

Zooko’s Triangle

You may have noticed that the requirements for a meaningful ID are in tension with the requirements for account names. Meaningful IDs are selected by people, to be evocative for themselves, while account names are mediated by some

www.it-ebooks.info

Chapter 14 ■ Accounts and Identity 285

c14.indd 07:52:38:AM 01/15/2014 Page 285

authority. It turns out that there are many such tensions associated with names. One very useful model of these tensions is “Zooko’s Triangle,” named after its creator, Zooko Wilcox-O’Hearn, and shown in Figure 14-4. The triangle shows the properties of decentralized, secure, and human-meaningful (Zooko, 2006).

Human-Meaningful

Secure Decentralized

Figure 14-4: Zooko’s Triangle

The idea is that every system design focuses on one or two of these proper- ties, which requires a trade-off with the others. The choice can be implicit or explicit, and the Triangle is a useful model for making it explicit.

Identity Documents

Deferring the issue of identifying your users to a government agency is attrac- tive. Having someone look at or make a copy of a government-issued identity document may constitute suffi cient due diligence, especially if you are in a regulated industry. Looking at the documents and storing copies exposes you to very different threats. If you store them and lose control of the storage, you’ll need to notify those people if you lose control of the documents (under a wide variety of state breach disclosure laws). This is an information disclosure threat against a data store, and the risk elimination mitigation is to not have that data store. It may be suffi cient to ensure that you check the documents. If you are not required by law to expose yourself to such liabilities, then having copies of identity documents seems foolishly risky.

Another issue with identity documents is that many people lack them for a wide variety of reasons. Many American citizens lack identity documents. This issue gained attention in the run-up to the 2012 election, as many states passed voter ID laws. Setting aside the emotional political questions, these laws drew attention to the fact that roughly 11 percent of voting-age Americans do not have current and valid government-issued photo ID (Brenner, 2012). It is important to consider how you’ll handle those customers without valid ID, or whose ID is not of the form you expect. For example, 20 million Americans don’t have a driver’s license (Swire, 2008). Your response to those without the ID you want might be to threaten to or actually refuse them service, but check with your sales and marketing departments before cutting off so many of your potential customers.

www.it-ebooks.info

286 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c14.indd 07:52:38:AM 01/15/2014 Page 286

Many times, it seems attractive to address fraud issues by asking people to show copies of such documents. If the document is shown over the Internet, any fool with a printer can make a spoof that’s good enough for their webcam. Similarly, any college student or illegal immigrant can acquire a document that’s good enough to fool the majority of people paid to check the documents. As the value of such documents increases, so does the motivation to either forge them or corrupt the offi cial issuance process. This is an economic threat to both your system and the integrity of the system as a whole. Regardless, knowing what an ID card says does not mean that the person with a given ID card is the same person who opened the account. This is a spoofi ng threat. (After all, given the breaches that happen, all the data on a given ID card may be avail- able to a fraudster.)

In short, looking at ID cards may be a helpful step, but it would refl ect poor threat modeling to assume that it will solve all your authentication problems.

Social Security Numbers and Other National Identity Numbers

This section reviews risks associated with the United States social security number (SSN). Over the last fi ve to ten years, the use of SSNs has declined substantially as a result of new laws. As of this writing, it is illegal to deny someone goods or services because they will not give you their SSN in Alaska, Kansas, Maine, New Mexico, and Rhode Island (Hillebrand, 2008). A large number of state laws also restrict their use, too numerous to list here (Bovbjerg, 2005). The fi rst problem is that many developers are unaware of these laws, and their lack of awareness may put their employers at risk. The second problem with SSNs is that some organizations use them as identifi ers, while others use them for authenticators. (Recall that an identifi er is a label for a person or other entity. An authenticator is how you prove that claim.) A few remarkable organizations manage to use them for both, but that’s not a model you want to emulate.

SSNs Are Poor Identifi ers

Social security numbers make poor identifi ers even if all your customers are American citizens who happen to be willing to give you their SSN. Not every American has an SSN, and not everyone legally residing in the United States is a citizen. For example, many teachers participate in the Teacher Retirement System, which does not use SSNs. Many legal residents are not able to get an SSN. Second, SSNs lack a check digit, so it’s easy to accidentally transpose or mistype digits. When you do, you have a roughly one-in-three chance of get- ting someone else’s SSN. (Roughly 300–400 million SSNs have been issued, and the number space is nine digits.) Third, except for those who work at the Social Security Administration, the identifi ers are outside your control.

www.it-ebooks.info

Chapter 14 ■ Accounts and Identity 287

c14.indd 07:52:38:AM 01/15/2014 Page 287

SSNs Are Poor Database Keys

Even if you collect SSNs, you should ensure that your account identifi er and database keys are ones that you control. Otherwise, you’ll need to deal with a way to change your account identifi er. As the Social Security Administration notes, “Applying for a new number is a big decision. It may impact your abil- ity to interact with federal and state agencies, employers, and others. This is because your fi nancial, medical, employment and other records will be under your former Social Security number . . .” (Social Security Administration, 2011). It is unclear whether the SSA is using SSNs as account identifi ers, or if they are warning that many others do so. If they are using the SSA as an account identifi er, that might be reasonable, but it would also seem reasonable that they have a way to manage an update. If they have chosen a mitigation strategy of risk transfer to citizens, then reasonable people might be skeptical that that’s a decision a government should be making.

SSNs Are Poor Authenticators

SSNs are known to many parties. As stated earlier in the discussion of knowledge- based authentication, answers to authentication questions should be known to only a few parties. Unfortunately, between schools, banks, insurance companies, utility companies, employers, tax preparation, gyms, and other entities that have made SSNs a condition of participation, the SSNs of most Americans are available fairly widely, and are often leaked in data breaches.

Even if they were not, Alessandro Acquisti and Ralph Gross have shown that they can predict many SSNs based on date and location of birth. They could “identify in a single attempt the fi rst fi ve digits for 44 percent of deceased individuals who were born after 1988 and for 7 percent of those born between 1973 and 1988.” They were also able to “identify all nine digits for 8.5 percent of those individuals born after 1988 in less than 1,000 attempts” (Power, 2009). In further work with Fred Stutzman, they were able to add facial recognition, using a dataset of faces on Facebook and an online dating site to fi nd names and birth dates. They were able to guess a substantial number of SSNs or par- tial SSNs from three webcam pictures (Acquisti, 2011). As they point out, this technology will get better and better as facial recognition technology improves, and as labeled face data becomes more widely available.

Finally, social security numbers are routinely published by the Social Security Administration after people die, in the Social Security Death Master File (SSDMF) [Social Security Administration, 2012]. Some organizations check the SSDMF to prevent new account fraud. However, when someone dies, their account does not necessarily go away. For example, bank accounts might be maintained by an estate during probate. A phone might be kept active for several months to

www.it-ebooks.info

288 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c14.indd 07:52:38:AM 01/15/2014 Page 288

enable people to get in touch. If you are using SSNs or some portion of the SSN as an authenticator, it will have been made public, making your authenticator far less useful.

Why Not Publish the SSN List?

One innovative response to the issue of “are SSNs well known?” is to simply publish the list, thus clarifying the true state of the system (Lindstrom, 2006). This is attractive in a number of ways. However, the law states “Social secu- rity account numbers and related records that are obtained or maintained by authorized persons pursuant to any provision of law enacted on or after October 1, 1990, shall be confi dential, and no authorized person shall disclose any such social security account number or related record” (42 USC 405(c)(2) (C)(viii)). Therefore, any source with an authoritative list is legally prohibited from publishing it. Rather than change the law to allow publication of the list, a sane new law should prohibit the use of SSNs as identifi ers or authenticators, and restrict their use in both the public and private sectors. (Given the lack of a check digit, associating two records based on an SSN should probably be treated as negligent.)

Other National Identity Schemes

The preceding section is very U.S.-centric, and may therefore seem irrelevant to those in other countries, but it is not. The same sorts of issues crop up with any national identifi er scheme, although the semi-private nature of the SSN seems to have exacerbated the confl ation of its use as both identifi er and authenticator.

Wherever you are considering using a government-issued identifi er, (or other identifi er provided by another organization) you should ask the following questions:

■ Does everyone have one?

■ Does it have a check digit?

■ Will you ever serve tourists, students, refugees, or other foreigners?

■ Can an individual get a new ID, and if so does it have the same fi eld values?

■ How will the system handle identity fraud in the future? (Even if the citizen can’t change their number now, they may be able to in the future. You should store it as a fi eld, not as your identifi er or database key.)

www.it-ebooks.info

Chapter 14 ■ Accounts and Identity 289

c14.indd 07:52:38:AM 01/15/2014 Page 289

If you want to use it as an authenticator, perhaps as part of an electronic ID scheme, ask how many people have access to the authentication data associated with the card.

Identity Theft

There is a common worry in the United States, referred to as identity theft. Elsewhere, I’ve written that the term fraud by impersonation is a more accurate description of the crime, and that it’s a crime facilitated by inappropriate alloca- tion of costs and benefi ts (Shostack, 2003). In particular, it’s easy for organizations that trade in information to be paid even when the information turns out to be highly inaccurate. The cost to them is that they may need to update it. The threat to a data subject may be denial of credit, denial of a job or a volunteer opportunity, or even an inappropriate arrest. Other cost distributions would likely result in more socially desirable outcomes.

However, the term identity theft may be appropriate when you consider its emotional impact on a subset of victims where records with information about them have become inextricably linked to those of the perpetrator, and whose ability to engage in normal activities is inhibited by the emotional burden of further false accusations, and the fear of those accusations. Those people really have had their “good name” stolen (Identity Theft Resource Center, 2009). There may be a good case that their good name isn’t stolen by the impersonator, but by those who intermix records and re-distribute the misinformation.

How does this play into threat modeling? Linkage is a threat to your data integrity and to your customer’s satisfaction. System designers should take care when linking information from disparate sources. If your system has features or processes that allow people to access and correct information you store, you should ensure that those corrections are not accidentally overridden by the same data from a source whose data has been corrected. (That is, if Alice tells you that Bob is a deadbeat, and Bob shows you his cancelled checks paying his debt to Alice, you’ll correct your record. Next month, Alice may well include Bob in her list of deadbeats again. Do you want Bob to be a happy customer? If so, you should be careful to not require that he correct your data repeatedly.) Tracking where your data is from can help with these issues. If you sell data about people, you can provide information about where you got the data to help your customers deliver better service. The most important distinction is between your direct observation and information you receive from elsewhere. You can either name the source, or, if the source is commercially sensitive, give them an alias.

www.it-ebooks.info

290 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c14.indd 07:52:38:AM 01/15/2014 Page 290

Names, IDs, and SSNs Checklist

This checklist is designed to be read aloud at a meeting. You’re in a bad state if for any of these questions:

■ You can’t answer yes.

■ You can’t articulate and accept the implications of a no.

■ You don’t know.

1. Do we know who controls our account namespace?

2. Do we have a way for our customers to assign a meaningful ID to things they need to securely identify?

3. Do we know what trade-offs we’ve made between global, meaningful, and secure?

4. Are we relying on identity documents? If so, are we storing copies of those documents, and who has signed off on that risk?

5. Are we using SSNs or similar identifi ers at all?

Summary

In a good model of accounts, they exist at the intersection of the human and the machine, and threats are abundant at that boundary. Those threats include authentication threats and confusion about names and identifi ers.

Accounts are a focal point for threats, which occur throughout the account life cycle, including account creation, maintenance, and termination. Risks of information disclosure to bad actors cause many systems to treat the people behind the accounts as attackers, rather than allies. Many of the authentication factors that are hidden are easily learned by a motivated attacker, while your real customers are left frustrated and in the dark. Good modeling that exposes how easy or hard it is to learn the obscured elements of authentication can help you make good choices, especially about notifying the person behind the account of what’s happening.

Authentication is hard. Creating a workable authentication system that sat- isfi es both an organization and its customers is hard. The framework of what you know, what you have, and what you are gives you a way to think about authentication, with “what channel are we using?” and “who you know” being interesting additions to the framework. However, each of these elements has threats against it, which have been worked through in great depth, and there are established patterns and limits for each.

If authentication is hard, backup authentication is even harder. A variety of knowledge-based authentication techniques can be used to recover access to

www.it-ebooks.info

Chapter 14 ■ Accounts and Identity 291

c14.indd 07:52:38:AM 01/15/2014 Page 291

an account. When you use one, you should focus on account recovery, not pass- word recovery, because you should not store passwords in plaintext. There are many problems with knowledge-based authentication, including security and usability. It may be preferable to use a social authentication technique; although they are new and less familiar, they are likely more secure.

Unlike authentication, names are relatively easy, once you accept that they lack certain properties, and you understand the risks that they carry. Avoid the “real name” trap, and consider where on Zooko’s Triangle your system should be: Are names memorable, globally unique, or secure? (Pick any two.) It is tempting to defer the problem to identity documents, which can help a carefully designed system but carry risks of their own. Designers often want to use social security numbers, which are poor identifi ers, poor database keys, and poor authenticators. Despite those fl aws, they are often used, bringing privacy and security threats along with them. One of those threats is identity theft, and there are patterns that can help you avoid contributing to the problem.

www.it-ebooks.info

www.it-ebooks.info

293

c15.indd 02:16:32:PM 01/16/2014 Page 293

Usable security matters because people are an important element in the security of any system. If you don’t consider how people will use your system, the odds are against them using it well. The security usability community is learning to model people, the sorts of decisions you need them to make, and the sorts of scenarios in which they act. The toolbox for addressing these issues is small but growing, and the tools are not yet as prescriptive as you might like. As such, this chapter gives you (in particular the security expert) deeper background than some other chapters, and the best advice available.

Because humans are different from other elements of security, and because the models, threats, and means of addressing them are different, this chapter covers challenges of modeling human factors in terms of security, and the techniques available to address them.

The chapter starts with models of people, as they motivate everything in this chapter. It continues with models of software which are useful for human factors work. These two models interact with threat elicitation techniques which are covered next, and are similar to those you learned about in Part II of this book. Threats in this chapter include those in which an attacker tries to convince a person to take action, and threats in which the computer needs to defer to a person for a decision. After threat elicitation, you’ll learn tools and techniques for addressing human factors issues, as well as some user interface principles for addressing those threats, and advice for designing confi guration, authentication,

C H A P T E R

15

Human Factors and Usability

www.it-ebooks.info

294 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 294

and warnings. These are the human-factors equivalents of the defensive tactics and technologies covered Chapter 8 “Defensive Tactics and Technologies.” From there, you’ll learn about testing for human factors issues, mirroring what you learned in Chapter 10 “Validating That Threats are Addressed.” The chapter closes with my perspective on usability and ceremonies.

Threat modeling for software or systems generally involves keeping two models in mind: a model of the software, and a model of the threats. Considering human factors adds a third model to the mix: a model of people. Ironically, that additional cognitive load can present a real usability challenge for those threat modeling human factors.

A FEW BRIEF NOTES ON TERMINOLOGY

■ Usability refers to the subset of human factors work designed to help people accomplish their tasks. Usability work includes the creation of user interfaces. User interfaces, along with their discoverability, suitability for purpose, and the success or failure of the people using those interfaces, make up one or more user experiences.

■ Human factors covers how technology needs to help people under attack, how to craft mental models, how to test the designs you’re building.

■ Ceremony refers to the idea of a protocol, extended to include its “human nodes.” For now, consider a ceremony to be similar to a user experience, but from the perspective of a protocol analyst. Ceremonies are explained in depth later in the chapter.

■ People are at the center of this chapter. You’ll see them referred to as users in user testing, user interface, and similar terms of art, and sometimes as humans, to align with sources.

Models of People

This chapter opens with models of people because they are at the center of the work you will sometimes need to do when addressing security where people are in the loop. This is a new type of model which parallels models of software and threats.

We all know some people, know how they behave, what they want. Aren’t these informal models enough? Unfortunately, the answer seems to be no. (Otherwise, security wouldn’t have a usability problem.) There are two reasons to create more structured models. First, people who work in software usually construct informal models of people that appear good enough for day-to-day use (but are often full of contempt for normal folks). They aren’t robust enough

www.it-ebooks.info

Chapter 15 ■ Human Factors and Usability 295

c15.indd 02:16:32:PM 01/16/2014 Page 295

to lead us to good decisions. Second, and more important, our implicit models of people are rarely focused on how people make security decisions.

As a profession, we need better models showing how people arrive at a security task, their mental models of the security tasks they’re being asked to do, or the security-related skills or knowledge we can expect various types of people to have. It might be possible to build all that into a single model, or we might have several models which are designed to work together. Additionally, we’d like a set of models that help those who are not experts in usable security.

Even if we had those models, attaining a consistent and repeatable auto- mated analysis of the human beings within a ceremony is unlikely. (There’s an argument that it is probably equivalent to developing artifi cial intelligence: If a computer could perfectly predict how a human being will respond to a set of stimuli, then it could exhibit the same responses. If it could exhibit the same responses, then it could pass the Turing test.) However, recall that all models are wrong, and some models are useful. You don’t need a perfect model to help you predict design issues. All of the models in this chapter are on the less structured end of the spectrum and are most useful in a structured brainstorm or expert consideration of a ceremony.

Applying Behaviorist Models of People

Does the name Pavlov ring a bell? If so, one explanation is that your repeated exposure to a stimulus has conditioned a response. The stimulus is stories about Pavlov’s famous dog experiments. The behaviorists believe that all observable behaviors are learned responses to stimuli. The behaviorist model has obvious and well-trod limitations, but it would not have had such a good run if it didn’t at least have some explanatory or predictive power. Some of the ways in which behavioral models of people can apply to ceremonies are explored in this section.

Conditioning and Habituation

People learn from their environment. If their environment presents them with frequently repeated stimuli, they’ll learn ways to respond to those stimuli. For example, if you put a username/password prompt in front of people, they’re likely to fi ll it out. (That’s a conditioned response, as pointed out by Chris Karlof and colleagues [Karlof, 2009].) It is hard for people to evoke deep, careful thought each time they encounter the stimulus, because such effort would usually be wasted. Closely related to this idea of a conditioned response is a habituation response such as automatically clicking a button in a dialog you have repeatedly seen (e.g., “Some fi les might be dangerous”).

It’s hard to argue that such behavior is even wrong. Aesop’s tale about the boy who cried wolf ends with the boy not getting help when a real wolf appears.

www.it-ebooks.info

296 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 296

People learn to ignore repeated, false warnings, for good reasons. If thinking carefully about the dialog mentioned in the example takes fi ve seconds and a person sees it 30 times per day, that’s three minutes per day, or 1,000 min- utes per year spent thinking about that one dialog. Suppose someone were successfully attacked once per year by a malicious fi le that exploits a bug not yet patched on the computer, and their anti-virus wouldn’t catch it. Will the cost of cleanup exceed 16 hours? If not, then the person is rationally ignoring the advice to spend those fi ve seconds (Herley, 2009).

Conditioning and habituation can be addressed by reducing the frequency of the stimulus. For example, the SmartScreen feature in Windows and Internet Explorer checks whether a fi le is from a leading publisher or very frequently downloaded and simply asks the person what to do with the fi le, rather than issuing a warning. Conditioning can also be addressed by ensuring that the ceremony conditions people to take steps for their security (see “Conditioned- Safe Ceremonies” later in this chapter for more information).

Wicked Environments

Educators have a concept of “kind” learning environments. A kind learning environment includes appropriate challenges and immediate feedback. These environments can be contrasted with wicked environments, which make learning hard. Jay Jacobs has brought the concept of wicked environments to information security. Quoting his description:

[Feedback is] the prime discriminator between a kind and wicked environment and consequently the quality of our intuition. A kind environment will offer unambigu- ous, timely, and accurate feedback. For example, most sports are a kind environ- ment. When a tennis ball is struck, the feedback on performance is immediate and unambiguous. If the ball hits the net, it was aimed too low, etc. When the golf ball hooks off into the woods, the performance feedback to the golfer is obvious and imme- diate. However, if we focus on the feedback within information security decisions, we see feedback that is not timely, extremely ambiguous, and often misperceived or inaccurate. Years may pass between an information security decision and any evidence that the decision was poor. When information security does fail, proper attribution to the decision(s) is unlikely, and the correct lessons may not be learned, if lessons are learned at all. Because of this untimely, ambiguous, and inaccurate feedback, decision makers do not have the opportunity to learn from the environ- ment in which the risk-based decisions are being made. It is safe to say that these decisions are being made in a wicked environment.

—Jay Jacobs, “A Call to Arms” (ISSA Journal, 2011)

www.it-ebooks.info

Chapter 15 ■ Human Factors and Usability 297

c15.indd 02:16:32:PM 01/16/2014 Page 297

As a simple heuristic, you can ask, is this design wicked or kind? What can you do to provide better feedback or make the feedback more timely or actionable?

Cognitive Science Models of People

Cognitive science takes an empirical approach to behavior, and attempts to build models from observation of how people really behave. In this section, you’ll learn about a variety of models of people that are in the cognitive science mold. They include the following:

■ Carl Ellison’s “ceremonies” model of people

■ A model based on the work of behavioral economist Daniel Kahneman

■ A model derived from the work of safety expert James Reason

■ A framework explicitly for reasoning about humans in computer security by CMU professor Laurie Cranor

■ A model derived from work on humans in security by UCL professor Angela Sasse

These models overlap in various ways. Each is, after all, a model of people. In addition, each of these can inform how you threat model the human aspects of a system you’re building.

Ellison’s Ceremonies Model

A ceremony is a protocol extended to include the people at each end. Security architect and consultant Carl Ellison points out that we, as a community, can learn from post-mortems of real ceremony errors, or perform tests on candidate ceremo- nies. He describes a model of “installing” programs on “human nodes” by means of a manual, training, or a contract that mandates certain user behavior. Models that require people to make decisions using the identity of the source require that the system effectively communicates about identity to the human node, using a meaningful ID—a concept covered in depth in Chapter 14, “Accounts and Identity.”

Ellison defers the creation of a full model to experimental psychologists or cognitive scientists; and in that mode, the following sections summarize a few models that refl ect some of the more insightful cognitive scientists whose work applies to security.

The Kahneman Model

This model is an attempt to extract some of the wisdom in Thinking Fast and Slow (Straus and Giroux, 2011). That excellent book is by Daniel Kahneman, the

www.it-ebooks.info

298 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 298

Nobel Prize-winning founder of behavioral economics. It is chock-full of ways in which people behave that do not resemble a computer, and anyone preparing to threat model for human factors will probably fi nd it repays a close reading. Following are some of the concepts presented in Kahneman’s book and ideas about how they can be applied to threat modeling:

■ WYSIATI, or What You See Is All There Is: This concept appears so frequently that Kahneman abbreviates it. The concept is important to threat modeling because it underscores the fact that the information currently presented to a person carries great weight in terms of any decisions they’re being asked to make. For example, if a person is told they must contact their bank right now to address a problem, they are perhaps unlikely to remember that their bank never answers the phone after 4:00 P.M., never mind on a Saturday. Similarly, if a person doesn’t see a security indicator (such as a lock icon in a browser), they’re unlikely to notice it’s not there. If an attacker presents cleverly designed advice “for security,” it might crowd out other advice the person has already received.

■ System 1 versus System 2: System 1 refers to the fast part of your brain, which does things like detect danger, add 2+3, drive, or play chess (if you’re a chess master). System 2 refers to the part of your brain that makes rational, considered decisions. System 1 infl uences our decision-making more than most people realize, or are willing to admit, perhaps including clicking away annoying dialogs. If clicking away dialogs is really system 1 activity, a system design that relies on system 2 when a dialog appears requires you to work hard to ensure that system 2 kicks in.

■ Anchoring: Anchoring effects are absolutely fascinating. If you ask people to write down the last two digits of their SSN, people with low values for those digits will subsequently estimate unrelated numbers to be lower, whereas those with high values will estimate higher. Similarly, the sales technique of saying something like “this is a $500 camera, but just today it’s $300” makes you think you’re getting a great deal, even if your budget a moment ago was $200. Perhaps anchoring effects crowd out wisdom when people are presented with scams or are being conned into behaving in ways that they otherwise would not.

■ Satisfi cing: Satisfi ce is a rotten word but it’ll do, absent a better one to describe the reality that people attempt to make decisions that are good enough, because the cost of a great decision is too high, or because “decision- making energy” has been exhausted. For example, most people’s savings

www.it-ebooks.info

Chapter 15 ■ Human Factors and Usability 299

c15.indd 02:16:32:PM 01/16/2014 Page 299

and investment choices are based on a subset of all possible investments, because evaluating options is time-consuming. In security, perhaps people allow system 1 to assess something, make a call regarding whether it’s suffi cient, and move on.

Reason’s Many Models

Professor James Reason studies the ways in which accidents happen in large systems. His work has lead to the creation of a plethora of models of human error. It could be highly productive to take those models and create prescriptive advice for threat modeling.

For example, his model of “strong habit intrusions” describes how the rules, or habits, that help us get through the day can be triggered inappropriately by “environment[s] that contain elements similar or identical to those in highly familiar circumstances. (For example, ‘As I approached the turnstile on my way out of the library, I pulled out my wallet as if to pay—although I knew no money was required.’)” These strong habit intrusions might be usable as a threat elicitation heuristic.

Some of the other models he has presented include the following:

■ A Generic Error-Modeling System, or GEMS (covering errors, lapses, and slips)

■ An intention-centered model

■ A model driven by the ways errors are detected

■ An action model that includes omissions, intrusions, repetitions, wrong objects, mis-orderings, mis-timings, and blends

■ A model based on the context of the errors

All but the fi rst model are covered in depth in The Human Contribution: Unsafe Acts, Accidents and Heroic Recoveries (Ashgate Publishing, 2008).

The Cranor Model

In contrast to the creators of the previous models, CMU Professor Lorrie Faith Cranor focuses specifi cally on security and usability. She has created “A Framework for Reasoning About the Human in the Loop” (Cranor, 2008). Like Ellison’s ceremonies paper, her paper is easy and worthwhile reading; this sec- tion simply summarizes her framework, which is shown in Figure 15-1.

www.it-ebooks.info

300 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 300

Human Receiver

Personal Variables

Intentions

Co m

m un

ic at

io n

De liv

er y

Co m

m un

ic at

io n

Pr oc

es si

ng Ap

pl ic

at io

n

Capabilities

Communication Impediments

Environmental Stimuli

Interference

Demographics and Personal

Characteristics

Knowledge and

Experience

Attitudes and Beliefs

Motivation

Attention Switch

Attention Maintenance

Comprehension

Knowledge Acquisition

Knowledge Retention

Knowledge Transfer

Communication Behavior

Figure 15-1: Cranor’s human-in-the-loop framework

Much like Ellison, Cranor’s model puts a person at the center of a model in which communication of various forms may infl uence behavior. However, “it is not intended as a precise model of human information processing, but rather it is a conceptual framework that can be used much like a checklist to systemically analyze the human role in secure systems.” The components of her model are:

■ Communications: The model considers fi ve types of communication: warnings, notices, status indicators, training, and policy. These are roughly ordered by how active or intrusive the communications are, and they are generally self-explanatory. One noteworthy element of training that Cranor calls out is that effective training (by defi nition) must lead people to recognize a situation where their training should be applied. There is an interesting relationship to what Reason calls rule misapplication. Rule misapplication may be more common when training is not regularly re-enforced or when the situations in which the training should be trig- gered are infrequent.

■ Communications impediments: These include any interference that prevents the communication from being received, and environmental stimuli that may distract the person from a communication that they have received, leading them to perform a different action. It is important to look for both behavior under attack and under non-attack, and ensure that communications reliably occur at the right time and only the right time.

■ The human receiver: This has six different components, and the “relation- ships between the various components are intentionally vague.” However, the left-hand column relates to the person, while the right-hand column roughly refl ects the chain of events for handling the message.

www.it-ebooks.info

Chapter 15 ■ Human Factors and Usability 301

c15.indd 02:16:32:PM 01/16/2014 Page 301

■ Personal Variables: Includes the person’s background, education, demographics, knowledge, and experience, each of which may play into how a person reacts to a message

■ Intentions: Includes attitudes, beliefs, and motivations

■ Capabilities: Even a person who gets a message, wants to act on it, and knows how to act on it may not be able to. For example, he or she might not have a smartcard or a smartcard reader.

■ Communication Delivery: This is related to the person’s attention being switched to the communication and maintained there.

■ Communication Processing: Determine whether the person under- stands the message and has the knowledge to act on it.

■ Application: Does the person understand, from formal or informal training, how to respond to the situation, and can they transfer that knowledge to the specifi c facts at hand?

Cranor presents a model that describes how to use her framework, shown in Figure 15-2. It consists of task identifi cation, task automation, failure iden- tifi cation in two ways (her framework and user studies), and mitigating those failures, again using user studies to ensure that the mitigations are functional. Cranor also usefully presents a set of questions to ask and factors to consider for each element of the model.

Task Identification

Task Automation

Failure Identification

Failure Mitigation

Human-in-the-loop Framework

User Studies

User Studies

Figure 15-2: Cranor’s human threat identification and mitigation process

Sasse’s Model

UCL Professor Angela Sasse also has a model of failures situated in organi- zational systems, including management and policies, preconditions, produc- tive activities, and defenses that surround the decision-makers (meaning any person, rather than only executives). It is fundamentally a compassionate model of people, based on the reality that most people generally want to be secure, and that their deviations from security are understandable if you take the time to “walk a mile in their shoes.”

www.it-ebooks.info

302 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 302

The most important threat modeling take-away from Sasse is her approach to those who make decisions that confl ict with the suggestions of security experts. Oftentimes, security experts bemoan that “in a contest between being secure and dancing pigs, dancing pigs will win every time” or “stupid users will click anything you put in front of them.” In contrast, she seeks to understand those decisions and the logic that underlies them. Such an understanding is essential to useful models of people.

Heuristic Models of People

The models of people described in this section are less associated with a single researcher, or otherwise don’t quite fi t into the way behaviorist or cognitive science models of people are presented, but they have been repeatedly and effectively used in studying how people’s observed behavior differs from the expectations embedded in systems.

Goal Orientation

People want to get to the task at hand. If a security process stands between them and their goal, then they are likely to view that process as a hindrance. That means they may read a warning dialog as if it says “Do you want to get the job done?” and click OK. Because of this usability, practitioners will often state that, “security is a secondary task.”

There are two main ways to address issues found in goal orientation. The fi rst is to make the obvious path secure. If that’s possible, it’s by far the best course. The second is to think about when the security information is provided in the ceremony. There are two options: Go early or go late. Going early means providing the information needed as the person is thinking about what to do, rather than as they’re committed to a path. For example, an operating system could display an application downloaded from the Internet with a spiky icon overlay, rather than (or in addition to) displaying a warning after someone has double-clicked it. Going late means delaying the decision as long as possible, so that the person has a chance to back out. This approach is used in the gold bar pattern, whereby the secure option is taken by default, and insecure options are made available in a gold bar along the top of the window. This is suffi cient to review the document, but not to edit or print it. A nontargeted attack is unlikely to convince anyone to exit the sandbox and expose themselves to more risk. See “User Interface Tools and Techniques” later in this chapter for more information about the gold bar pattern.

www.it-ebooks.info

Chapter 15 ■ Human Factors and Usability 303

c15.indd 02:16:32:PM 01/16/2014 Page 303

Confi rmation Bias

Confi rmation bias is the tendency people have to look for information that will confi rm a pre-existing mental model. For example, believers in astrology will remember the one time their horoscope just nailed what was about to happen, and discount the other 364 days when it was vague or just plain wrong. Similarly, if someone believes they urgently need to contact their bank, they may be less likely to notice slight oddities in the bank’s login page.

Scientists and engineers have learned that looking for ways to disprove an idea is much more powerful that looking for evidence which confi rms it. Unfortunately, looking for counter-evidence seems to be at odds with how people tend to work. Recalling the discussion of system 1 and system 2 in the section on Kahneman’s work, there may be elements of system 1 versus system 2 at play here. System 1 may see a data point and collect it (feeding confi rma- tion bias) or discard it as anomalous, preventing you from seeing the problem.

Addressing confi rmation bias is tricky in general, and may be trickier in security circumstances. It might be possible to condition or train people to look for evidence of evil, and use that to undercut confi rmation bias.

IS THIS SECTION FULL OF CONFIRMATION BIAS?

It is easy to fi nd examples of these hueristics once you watch for them, which raises a worry that they are easy, rather than good explanations. Worse, in putting them here, I may be subject to confi rmation bias. Perhaps in reality, some other factor is at work, and using one of these will be misleading. This risk is associated with many of the heu- ristics in this section, and a poor understanding of the cause may lead to selecting a poor solution. This should not be taken as an argument against using them, but a cau- tion and a reminder of the importance of doing usability testing.

Compliance Budget

The term compliance budget appears in the work of Angela Sasse (introduced above), who has performed anthropological studies of British offi ce workers. Her team noticed that after repeated exposure to the same security policies or tasks they were supposed to perform, the workers would respond differently (Beautement, 2009). During their interviews about security compliance, she noted that the workers were effectively allocating a “budget” to perform secu- rity tasks. They may or may not understand the tasks, but they would spend time and energy on them until their budget was exhausted, and then move on to other work tasks. When security requests were considered not as platonic

www.it-ebooks.info

304 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 304

tasks to be accomplished, but as real tasks situated against other tasks, worker behavior was fairly consistent.

Therefore, as you design systems, track how many security demands are being made in each of your scenarios. There is no “right” amount, but fewer is better.

Optimistic Assumptions

Many protocols impose optimistic assumptions about the capabilities of their human nodes. For example, people are sometimes expected to know where a browser’s lock icon should be. (Top or bottom, left or right? In the main display area, obviously.) The lock cleverly disappears when it’s not needed, which we optimistically assume doesn’t make this question harder.

So how do you use this in threat modeling? As you go through the process, you should be keeping a list of assumptions you fi nd yourself making (see Chapter 7, “Processing and Managing Threats”). For each optimistic assumption, look for a weaker assumption, or a way to buttress the assumption.

Models of Visual Perception

In work released as this book was being completed, Devdatta Akhawe and colleagues argue that limitations of human perception make UI security diffi cult to achieve. They present a number of attacks, including destabilizing perception of the pointer, attacking peripheral vision, attacking motor adaptations of the brain, mislocalization related to fast-moving objects, and abusing visual cues (Akhawe, 2013). Studying models of visual perception will probably be a fruitful area of research over the next few years.

Models of Software Scenarios

There are (at least) two ways to model a scenario, including a software-centered model and an attack-centered model. In this section, you’ll learn about model- ing both for threat modeling human factors. The software-centered models are somewhat easier, insofar as you know what sorts of circumstances will invoke them. The software models include scenario models of warnings, authentications, and confi gurations and models that use diagrams to represent the software. The attack-centered model is somewhat more broad. That is, if an attacker wants your software to do something, where can they force it?

Modeling the Software

You can model the goals and affordances of the software that a particular feature is intended to offer. (An affordance is whatever element of the user

www.it-ebooks.info

Chapter 15 ■ Human Factors and Usability 305

c15.indd 02:16:32:PM 01/16/2014 Page 305

interface that communicates about the intended use.) One useful model of the interactions you’ll offer includes warnings, authentications, and confi guration (Reeder, 2008b). In that model, warnings are presented in the hope of deter- ring dangerous behavior, and include warnings dialogs, prods, and notifi ca- tions. Authentications include the person authenticating to a computer either locally or via a network, and the remote system authenticating to the person. Confi guration includes all those ways in which the person makes a security decision regarding the confi guration of a computer or system.

These are not always distinct in practice. A single dialog will commonly both warn and ask for confi guration or consent. Considering them separately helps you focus on the unique aspects of each, and ensure that you’re provid- ing the information that a person needs. In addition, the same advice applies to the warning regardless of the further content. So it applies to authentication, confi guration, and consent. The types of information that you’ll need to present so that the warning is clear doesn’t change because of the additional context or decisions.

Warnings

A warning is a message from one party to another that some action carries risk. Generally, such messages are intended as a risk transference from one party to another, and they may have varying degrees of legal or moral effectiveness in actually transferring risk. Good warning design is covered later in this chapter in the section “Explicit Warnings.”

Good warning design clearly identifi es the potential problem, how likely it is, and what can be done to avoid it. Good warnings avoid “the boy who cried wolf” syndrome.

Authentication

People are astoundingly good at recognizing other people. It’s probably an evolved trait. However, it turns out to be very diffi cult to authenticate people to machines or machines to people. Authentication is the process of proving that you are who you’ve said you are in a manner that’s suffi cient for a given context—authenticating that you’re a student at State University is generally less important than authenticating that you’re the missing heir to a massive fortune. Authentications are generally one party authenticating to another, such as a client authenticating itself to a server. They are sometimes bi-directional, and the authentication system in use in each direction may differ. For example, a web browser might connect to a bank and validate a digital certifi cate to start SSL. This is the server being authenticated via PKI to a client. The client will then authenticate via a password to the bank. There are also authentications that involve more parties.

www.it-ebooks.info

306 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 306

The best advice on authentication (in both directions) is to treat it as a ceremony and pay close attention to where information comes from, how it could be spoofed, and how it will be validated in the real world.

Confi guration

There are many ways to group confi guration choices that can be made, including confi gurations made in advance, and confi gurations to allow or deny a specifi c activity. For each, it is helpful to consider the person’s mental model, and the diffi culty of testing their work. Useful techniques might include asking “how others might see the state of the system,” whether “X can do Y,” or having the person “describe the changes just made.” Confi guring a system for security is hard. The person performing the confi guration needs to instruct the computer with a degree of precision that’s required in few other areas of life, and con- structing tests to ensure that it’s been done right is often challenging for both technical and nontechnical people. Confi guration tasks can be broken out into:

■ Confi guration such as altering a setting

■ Consenting to some set of terms

■ Authorization or permission settings

■ Verifi cation of settings or claims (such as the state of a fi rewall or identity of a website)

■ Auditing or other investigation into the state of a system so people can view and act if appropriate

As you think about confi guration, consider using a framework of who, what, why, when, where and how:

■ Who can perform the confi guration? Is it anyone? An administrator? Is there a parental role?

■ What can they confi gure, and to what granularity? Is it on/off? Is it details intrinsic to the feature being confi gured (e.g., the fi rewall can block IP addresses or ports), or extrinsic, such as “this user cannot contact me”? The latter requires each channel of contact to know it must check the ACL.

■ Why would someone want to confi gure the feature, and use those sce- narios to determine what the user interface will be, and what mistakes they might make.

■ When will someone be making confi guration choices? Is it proactive or reactive? Is it just in time?

www.it-ebooks.info

Chapter 15 ■ Human Factors and Usability 307

c15.indd 02:16:32:PM 01/16/2014 Page 307

■ Where will someone make the change? (And how will they fi nd that interface?)

■ How will users implement their intent? “How” is not only how they make a change, but how they can test it—for example, Windows “effec- tive permissions” or the LinkedIn “Show me how others see this profi le.” Consider how users can see the confi guration of the system as a whole.

Good design of a confi guration system is hard. Decisions made early can make changes impossible or extremely expensive. For example, Reeder has shown that people have a hard time with elements of the Windows fi le per- mission model, and that changing to a “specifi city preference” over the deny precedence appears to align better with people’s expectations (Reeder, 2008a). However, changing that ordering would break the way one access control rules have been confi gured on possibly hundreds of millions of systems, and each rule would require manual analysis to understand why it’s set the way it is, what the change would mean, and how to best repair it.

Diagramming for Modeling the Software

There are a variety of diagram types that can be useful for human factors analysis, including swim lanes, data fl ow diagrams, and state machines. Each is a model of the software, system, or protocol. In this section, you’ll learn about how each can be modifi ed for use in looking at the human in the loop.

Swim Lanes

You learned about swim lane diagrams for protocols in Chapter 2, “Strategies for Threat Modeling.” These diagrams can be adapted for modeling people. In ceremony analysis, you add swim lanes representing each participant, as shown in Figure 15-3 (reproduced from Ellison’s paper on ceremonies [Ellison, 2007]). There is an implicit trust boundary between each lane. In this diagram, S is the server, A1 and A2 are attackers, CC is the client, and CA is a certifi cate authority.

This diagram is worth studying, especially when you realize that A1 and A2 are machines controlled by attackers. Note that the attacker A is not shown; what this attacker knows is not relevant to the security of C, the person being tricked. Also note that messages from computers to humans (“S,” short for server) and humans to computers (“click”) are shown as protocol messages. It is very important to ensure the messages between computers and humans are clearly shown, and the contents of each is modeled. Such modeling will help you identify unreasonable assumptions, information not provided, and other communications issues.

www.it-ebooks.info

308 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 308

a

S A1 A2 CC C R

CA

b

TLS a1 to aS

TLS to a1

GET page at aS GET page at a1

login login

login

password

password password

click S

S, a1

KR KR

KR

Figure 15-3: Ellison’s diagram of the HTTPS ceremony

State Machines

State machines are often used to model the state of inanimate objects. The states of machines are simple compared to the states of humans. However, that doesn’t mean that a state machine can’t help you think about the state of people using your software. For example, consider Figure 15-4, which explores possible states of a person trying to visit a website and being blocked by a security warning. The state machine has two exit states: reading a web page and considering alternative options. They are indicated by darker lines around the states.

Quick consideration of this model reveals several issues:

■ If a “bypass this warning” button is easily visible, it may be pressed by an annoyed person.

■ If people feel overwhelmed by security decisions, they’ll likely bypass the warning.

■ The “stars must align” to get someone to the state of considering alterna- tive plans.

www.it-ebooks.info

Chapter 15 ■ Human Factors and Usability 309

c15.indd 02:16:32:PM 01/16/2014 Page 309

Start

Visiting web page

Reading web page

Click on link (or) type URL

Page renders

Security interrupt

Click bypass

Click bypass

Annoyance

Decide it’s not risky/ they won’t

understand/etc.

Overwhelmed

Can’t find bypass

Worrying about security

Recall past experience

Investigate warning

Warning is unclear

Frustration

Considering alternate plans

Warning is clear

Figure 15-4: A human state machine

You can also use state machines in conjunction with ceremonies. In a ceremony, a node can be modeled as state (including secrets), a state machine with its inputs and outputs, service response times, including bandwidth and attention, and a probability of errors of various sorts. (Ellison provides a longer list with fi ner granularity for the same set of attributes [Ellison, 2007].)

Modeling Electronic Social Engineering Attacks

A project at Microsoft needed a comprehensive way to describe electronic social engineering attacks. In this context, electronic means roughly all those social engineering attacks that are not in person or over the phone.

We came up with the channels and attributes shown in Table 15-1, which we have found to be a useful descriptive model. It is intended to be used like a Chinese menu, whereby you choose one from column A, one from column B, and so on.

You’ll probably fi nd that the descriptors here are suffi cient for describing most online social engineering attacks. Unfortunately, some evocative detail is lost (and some verbosity is gained) as you generalize from “a Nigerian prince spam” to “e-mail pretending someone to be someone you don’t know, exploiting

www.it-ebooks.info

310 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 310

greed” or from “phishing” to “a website pretending to be an organization you have a relationship with and for which you need to enter credentials in order to do business.” However, the resultant model descriptions capture the relevant details in a way that can inform either training or the design of technology to help people better handle such attacks.

Table 15-1: Attributes of Electronic Social Engineering Attacks

CHANNEL OF CONTACT

THING SPOOFED

PERSUASION TO INTERACT

HUMAN ACT EXPLOITED

TECHNICAL SPOOFING

E-mail An operating system or product user interface element, such as a Mac OS warn- ing, or a Chrome browser pop-up

Greed/promise of reward

Open document

System dialog or alert

Website A product or service

Intimidation/ fear

Click link Filename (extension hiding)

Social network

A person you know

Maintaining a social relationship

Attach device/ USB stick

File type (other than extension hiding)

IM An organization you have a rela- tionship with

Maintaining a business relationship

Install/run program

Icon

Physical* An organization you don’t have a relationship with

Curiosity Enter credentials

Filename (multi-lingual)†

A person you don’t know

Lust/prurient interest

Establish a relationship‡

An authority

* Physical is at odds with the online nature of most of these, but sometimes there’s an interesting overlap, like a USB drive left in a parking lot.

† Multi-lingual spoofing involves use of languages written left to right and right to left in the same name. There is no way to do so which meets all cultural and clarity requirements.

‡ Establishing a relationship also overlaps; much, but not all electronic social engineering is focused on the instal- lation of malware.

The Technical Spoofi ng column contains details that are sometimes clarify- ing. Those elements are more specifi c versions of the “user interface element” at the top of the Thing Spoofed column.

www.it-ebooks.info

Chapter 15 ■ Human Factors and Usability 311

c15.indd 02:16:32:PM 01/16/2014 Page 311

Threat Elicitation Techniques

Any of the threat discovery techniques covered in Part II of this book can prob- ably be applied to a user experience. Spoofi ng seems particularly relevant. You can bring any of those to bear while brainstorming. The ceremony approach offers a more structured way to fi nd threats. You can also consider the models of humans presented earlier while considering what a “human state machine” might do, or as ways to explain the results of the tests you perform.

Brainstorming

The models of people and scenarios provided so far can help anyone fi nd threats by brainstorming. That’s much more likely to be productive with participation from either security or usability experts. Different goals of these specialists may lead to clashes, however, as usability experts will likely tend toward making it easy to get back to the primary task, and security experts will tend to focus on the risk of doing so. Either structure or moderators may help you.

The Ceremony Approach to Threat Modeling

The ceremony approach to threat modeling, created by Carl Ellison, (and mentioned briefl y earlier) is most like “traditional” threat modeling. It starts from the observation that any network protocol, suffi ciently fully considered, involves both computers and humans on each end. He developed this obser- vation into an approach to analyzing the security of ceremonies. “Ceremony Design and Analysis,” the paper in which Ellison presents his model, is free, easy to understand, and worth reading when considering the human aspects of your threat model. It introduces ceremonies as follows:

[The ceremony is] an extension of the concept of network protocol, with human nodes alongside computer nodes and with communication links that include UI, human-to-human communication and transfers of physical objects that carry data. What is out-of-band to a protocol is in-band to a ceremony, and therefore subject to design and analysis using variants of the same mature techniques used for the design and analysis of protocols. Ceremonies include all protocols, as well as all applications with a user interface, all workfl ow and all provisioning scenarios. A secure ceremony is secure against both normal attacks and social engineering. However, some secure protocols imply ceremonies that cannot be made secure.

—Carl Ellison, “Ceremony Design and Analysis” (IACR Cryptology ePrint Archive 2007)

www.it-ebooks.info

312 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 312

In a “normal” protocol, there are two or more nodes, each with a current state, a state machine (a set of states and rules for transitioning between them), and a set of messages it could send. Ellison extends this to humans, noting that humans can be modeled as having a state, messages they send and receive, and ways to transition into different states. He also notes that they are likely to make errors parsing messages. Ellison calls out three important points about ceremonies:

■ “Nothing is out of band to a ceremony.” If there’s an assumption that something happens out of band, you must either model it or design for it being insecure.

■ Connections (data fl ows) can be human to human, human to computer, computer to computer, physical actions, or even legal actions, such as Alice sells Bob a computer.

■ Human nodes are not equivalent to computer nodes. Failure to take into account the way humans really act will break the security of your system.

Ceremony Analysis Heuristics

There are a set of threats that can be discovered simply and easily with some heuristics, and a set of threats that are more likely to be found with more struc- tured work. Each heuristic has advice for addressing the threat. The fi rst four are extracted from Ellison’s “Ceremony Design and Analysis” (IACR Cryptology ePrint Archive 2007) to contextualize them as heuristics, while the fi nal two are other heuristics that may help you.

Missing Information

The fi rst issue to look for is missing information. Does each node has the information you expect it to act upon? For one trivial example, in the HTTPS scenario, the person is expected to know that the server name (S) does not match the URL (A1). However, the message from the computer contains only a server name, not a URL. Therefore, the analysis is particularly trivial: The designer has asked a person to make a decision but has not provided the information necessary to make it.

Ensure that you’re explicit about decisions people will need to make, what information is needed to make them, and display it early enough that the person won’t anchor on some other element, and close enough to where it is needed that other information will not be distracting.

Distracting Information

If you’d like to ensure that a person acts on information you present, keep in mind that each additional piece of information acts as a distractor, and may

www.it-ebooks.info

Chapter 15 ■ Human Factors and Usability 313

c15.indd 02:16:32:PM 01/16/2014 Page 313

feed into confi rmation bias. In discussing the human phase of verifying that a cryptographically signed e-mail was really signed, Ellison offers the screenshot shown in Figure 15-5, pointing out fi ve elements that a human might look at, including the “From: address,” the picture, the “Signed by” address, the text of the message, and other contextual information that the human might have available. The only element that the (cryptographic) protocol designer wants you to look at is the “Signed by” address, which is the fi fth element displayed, after “From,” “To,” “CC,” and “Subject”—all of which appears underneath various GUI elements, including the window title, taken from the e-mail subject. There is also a picture, and a small icon (reproduced from Ellison, 2007).

Figure 15-5: An e-mail interface

To address the possibility of distraction, information not key to the ceremony either should not be displayed or should be de-emphasized in some way or ways. In particular, you should take care to avoid showing information that is easily substituted or confused for the information that you want the person to use in the ceremony.

Underspecifi ed Elements

Because a ceremony is all inclusive, it requires us to shine light where system architects might otherwise hand-wave. For example, how did a PKI root key on a system become authorized to suppress or send messages to the owner of that computer? (PKI root keys often suppress warning messages, and may activate messages, such as a green URL bar or a lock within the browser.) To address underspecifi ed elements, specify them. It may be a practical requirement to accept risks associated with them, but a concrete statement of the risk might enable you to fi nd a way to address it.

www.it-ebooks.info

314 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 314

Fuzzy Comparison

People are pretty bad at comparing strings. For example, are the strings 82d6937ae 236019bde346f55bcc84d587bc51af2 and 82d6937ae236019bde346f55bcc84d857b c51af2 the same string? Go on, check. It’s important. I’m hopeful that when this book is laid out, the strings don’t show up one above the next (and put in extra words to try to mitigate that threat from layout). So, did you check? I’m willing to bet that even most of you, reading the freaking gnarly specifi cs part of a book on threat modeling, didn’t go through and compare them character by character. That may partly be because the answer doesn’t really matter; but mostly it’s because people satisfi ce—that is, we select answers that seem good enough at the time. The fi rst (and maybe last) characters are the same, so you assume the rest are too. Often, attackers can do something to push the person to a fuzzy match that they will pass.

To address fuzzy comparison, look for ways to represent the bits so that they are easily compared. For example, because people are excellent at recognizing faces, can you use faces to represent data? How about other graphical representations? For example, there’s a set of techniques for turning data into an image called a visual hash, as shown in Figure 15-6 (Levien, 1996; Dalek, 1996), although there is little or no security usability analysis of these, and they have not taken off.

Figure 15-6: The snowflake visual hash

It may be possible to replace a long string of digits with a string of words, tapping into system 1 word recognition and improving comparability. Lastly, if none of those methods will work, try to create groups of four or fi ve digits so that at least people fi nd natural breakpoints when carrying out the comparative task.

www.it-ebooks.info

Chapter 15 ■ Human Factors and Usability 315

c15.indd 02:16:32:PM 01/16/2014 Page 315

Kahneman, Reason, Behaviorism

The entire section “Models of People,” presented earlier, is suffused with perspectives that can be used as heuristics, or as a review list, asking yourself “how could this apply?”

The Stajano-Wilson Model

Frank Stajano and Paul Wilson have worked with the creators of the BBC’s “Real Hustle” TV show to create a model of scams (Stajano, 2011). Their principles are quoted verbatim here, and could be adapted to use for threat elicitation:

1. Distraction Principle: While we are distracted by what grabs our interest, hustlers can do anything to us and we won’t notice.

2. Social Compliance Principle: Society trains people to not question author- ity. Hustlers exploit this “suspension of suspiciousness” to make us do what they want.

3. Herd Principle: Even suspicious marks let their guard down when every- one around them appears to share the same risks. Safety in numbers? Not if they’re all conspiring against us.

4. Dishonesty Principle: Our own inner larceny is what hooks us initially. Thereafter, anything illegal we do will be used against us by the fraudsters.

5. Kindness Principle: People are fundamentally nice and willing to help. Hustlers shamelessly take advantage of it.

6. Need and Greed Principle: Our needs and desires make us vulnerable. Once hustlers know what we want, they can easily manipulate us.

7. Time Principle: When under time pressure to make an important choice, we use a different decision strategy, and hustlers steer us toward one involving less reasoning.

The discussion of the herd principle points out how many variations of the sock puppet/Sybil/tentacle attacks exist, a point for which I’m grateful.

Integrating Usability into the Four-Stage Framework

This chapter advocates looking at usability and ceremony issues as a distinct set of activities from modeling technical threats. It may also be possible to bring usability into the four-stage framework by considering it in two places: mitigation and validation. If the general approach of thinking of mitigations as “change the design, use a standard mitigation; use a custom mitigation, accept the risk” is extended with “transfer risk by asking the person,” then that makes

www.it-ebooks.info

316 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 316

a natural jumping off point to the techniques outlined in this chapter. Similarly, in validation, if a user interface asks the person to make a security or privacy decision, then assessing it with techniques from this chapter will help. This approach, illustrated in Figure 15-7, was developed by Brian Lounsberry, Eric Douglas, Rob Reeder, and myself.

Model

Find Threats

Address Threats

Validate

Design Interface

User Testing

Transfer Risk?

User Interface?

Figure 15-7: Integrating usability into a flow

Tools and Techniques for Addressing Human Factors

Having discussed how to fi nd threats, it’s now time to talk about how you can address them. This section begins with myths about people that we as a com- munity need to set aside, it continues with design patterns for helping you fi nd a good solution, and then describes a few design patterns for a kind learning environment.

In an ideal world, everyone would have time to carefully consider each secu- rity decision that confronts them, research the trade-offs, and make a call that refl ects their risk acceptance given their goals and options. (Also in that ideal world, technology producers would be happy to invest the effort needed to help them.) In the real world, people typically balance their investment of effort against perceived risk, their perception of their own skill, time available, and a host of other factors. Worse, in the case of security, there sometimes is no “right” decision. If you get an e-mail from your boss asking you to print a presenta- tion for an impromptu meeting with the CEO that’s happening right now, you can’t call to make sure it’s really from the boss. You might want to accept the risk that someone has taken over your boss’s e-mail. You want to make it easy for people to quickly reach the decision that they would make if they had all

www.it-ebooks.info

Chapter 15 ■ Human Factors and Usability 317

c15.indd 02:16:32:PM 01/16/2014 Page 317

the time in the world. You want to make it possible for people to dig deeper, without requiring they do so.

Myths That Inhibit Human Factors Work

There are a number of myths that are threats to human factors work in secu- rity. They all center on contempt for normal people. People are not, as is often claimed, the weakest link, or beyond help. The weakest link is almost always a vulnerability in Internet-facing code. Society trusts normal adults to drink alcohol, to operate motor vehicles (ideally not at the same time), and to have children. Is using a computer safely so much harder? If so, whose fault is that? Following are some common myths about human behavior in terms of security:

■ “Given a choice between security and dancing pigs, people will choose dancing pigs every time.” So why make that a choice? More seriously, a better way to state this is, “Given a choice between ignoring a warning that they’ve clicked through a thousand times before without apparent ill effects and without being entertained, people will bypass a warning every time.”

■ “People don’t care about security,” or “People don’t want to think about security.” Either or both may well be true. However, people do care about consequences such as having to clean their system of a virus or dealing with fraudulent charges on their debit cards.

■ “People just don’t listen.” People do listen. They don’t act on security advice because it’s often bizarre, time consuming, and sometimes followed by, “Of course, you’ll still be at risk.” You need to craft advice that works for the people who are listening to you.

■ “My mom couldn’t understand that.” Your mom is very smart. After all, she raised you, and you’re reading this book. QED. More seriously, you should design systems that your customers will understand.

Design Patterns for Good Decisions

The design patterns for mitigating threats that exploit people are similar to those for other categories of threats. This section describes the four high-level patterns: minimize what you ask of people, conditioned-safe ceremonies, avoid urgency, and ensure a path to safety.

Minimize What You Ask of People

If people are not good at being state machines running programs, it seems likely that the less that is asked of them, the fewer insecurities a ceremony

www.it-ebooks.info

318 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 318

will have. People must be involved when they have (or can reasonably be expected to have) information that other nodes need to make appropriate decisions. For example, Windows Vista introduced a dialog that asked people what sort of network they had just connected to. This elicited information (such as home, work, or coffee shop) that the computer can’t reliably determine, and it was used to confi gure the fi rewall appropriately. A person must also be involved when making decisions about meaningful IDs. There are likely other data points or perspectives that a program must ask of a person.

There are two specifi c techniques to minimize what’s asked of people: listing what they need to know, and building consistency:

■ Listing what people need to know: Simply making a list of what people need to know to make a security decision is a powerful technique that Angela Sasse advocates. The technique has several advantages. First, it’s among the simplest techniques in this book. It requires no special training or equipment. Second, the act of writing down a list tends to impose a reality check. When the list gets longer than one unit (whiteboard, page, etc.), even the most optimistic will start to question whether it’s realistic. Finally, making a list enables you to use that list as a checklist, and asking how the customer will learn each item on the list is an easy (and perhaps obvious) next step.

■ Building consistency: People are outstanding at fi nding patterns, some- times even fi nding patterns where none exist. They use this ability to build models of the world, based on observations of consistency. To the extent that your software is either inconsistent with itself or with the expecta- tions of the operating environment, you are asking more of people. Do not do so lightly. Consistency relates to the wicked versus kind environment problem raised by Jacobs. Ensuring that your security user experiences are consistent within a product is a very worthwhile step, as is ensuring that it’s consistent with the interfaces presented by other products. This consistency makes your product a kind learning environment. Of course, this must be balanced with the value of innovation and experimentation. If everything is perfectly consistent, then we can’t learn from differences. However, we can try to avoid random inconsistency.

Conditioned-Safe Ceremonies

Consistency will probably help the people who use your system, but a conditioned-safe approach may be even better. The concept of a conditioned- safe ceremony (CSC) is built on the observation that conditioning may play both ways. In other words, rather than train people to expect random shocks or

www.it-ebooks.info

Chapter 15 ■ Human Factors and Usability 319

c15.indd 02:16:32:PM 01/16/2014 Page 319

interruptions from security, you could train or condition them to act securely. In their paper introducing the idea (Karlof, 2009), Chris Karlof and colleagues at University of California, Berkeley, proposed four rules governing the use of conditioned-safe ceremonies:

1. CSCs should only condition safe rules, those that can safely be applied even in circumstances controlled by an adversary.

2. CSCs should condition at least one immunizing rule—that is, a rule which will cause attacks to fail.

3. CSCs should not condition the formation of rules that require a user decision to apply the rule.

4. CSCs should not assume people will reliably perform actions which are voluntary, or that they have not been trained/conditioned to perform each time.

They also discuss the idea of forcing functions. In this context, a forcing function is “a type of behavior-shaping constraint designed to prevent human error,” for example, a car that can be started only when the brake pedal is pressed. People regularly make the mistake of omitting steps in a process (it may be one of the most common types of mistakes), and they rarely notice when they’ve omitted a step. Forcing functions can help ensure that people complete important steps. The forcing function will only work when it is easier to execute than avoid, where avoidance may include avoiding use of the feature set that includes a forcing function. They use the example of Ctrl+Alt+Delete to bring up the Windows Login screen as a forcing function.

Avoiding Urgency

One of the most consistent elements of current online scams is urgency. Urgency is a great tool for the online attackers. They maintain control of the experience, sending the prospective victim to actions of their choice. For example, in a bank phishing scam, if the person sets the e-mail aside, they may later fi nd a real e-mail from the organization, they may use a bookmark, or they may call the number on the back of their card. The fake website may be taken down, blacklisted, or otherwise become unavailable. Therefore, avoiding urgency helps your customers help themselves.

Avoiding urgency can involve something like saying, “We will take no action unless you visit our site and click OK. You can always reach our real security actions through a bookmark, or by using a search engine to get to our site.”

If you’ve conditioned your customers to expect urgency from you, then urgency is less likely to act as a red fl ag. Make it harder for the attacker by avoiding urgency in your messages to customers.

www.it-ebooks.info

320 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 320

Ensuring an Easy Path to Safety

When people are being scammed, is there a way to get back to a safe state? Is it easy to discover (or remember) and take that path to safety? For example, is there always an easy way to access messages you send from your home page? If there is, then potential phishing victims can take control of the experience. They can go to their bookmarked website and get the message. You might use a pattern like the gold bar (discussed in the section “Explicit Warnings,” later in this chapter) or an interstitial page to notify people about important new messages, so they learn that they can fi nd new messages from your site.

If that’s always available, then attackers will need to convince people that that system is broken for whatever reason. Today, that’s easy, because “go to your bookmark” is lumped in with a barrage of confusing and questionable security advice. (The path to safety pattern was identifi ed by Rob Reeder.)

Design Patterns for a Kind Learning Environment

A kind learning environment contrasts with a wicked one. If you’re an attacker, you want the world to be wicked. You want people to be confused and unsure of how to defend themselves, because it makes it easier to get what you want. The following sections provide advice for how to promote a kind environment.

Avoid “Scamicry”

Scamicry is an action by a legitimate organization that is hard for a typical per- son to distinguish from the action of an attacker—for example, a bank calling a customer and demanding authentication information without an easy way to call back and reach the right person. How can Alice decide whether it’s her bank or a fraudster? Trust caller ID? That’s trivial to spoof. Similarly, the bank can choose how to track clicks on its marketing campaigns. It can use the domain of the marketing company (example.com/bank/?campaign=123;emailid=345) or it can use a tracking URL within its own domain (bank.com/marketing /?campaign=123;emailid=345) Alice might want to look at the domain to make a decision (and many security people advise her to do so); but if the bank rou- tinely sends e-mail messages with links to tracking domains, then Alice can’t look at the URL and decide if it’s really her bank. Scamicry makes the world a more wicked environment, disempowers people, and empowers attackers.

www.it-ebooks.info

Chapter 15 ■ Human Factors and Usability 321

c15.indd 02:16:32:PM 01/16/2014 Page 321

WHAT IS SCAMICRY?

My team coined the term scamicry after I received a voicemail claiming to be from a bank. The caller said they needed a callback that day to a number that wasn’t listed on their website’s Contact Us page. I had written a large check for something, and it trig- gered their fraud department. However, there was no clear reason for the “call in the next four hours” requirement in the voicemail. The bank could presumably have put their fraud team’s number on their website, held the check for a day or so (although this may have been constrained by regulations on check processing). In discussing the experience, we realized that it was a common pattern, and that perhaps naming it would help to identify and then overcome it.

Sometimes it can be remarkably hard to avoid violating common advice, and that should cause us to question the wisdom of that advice. (See the next section for more on good security advice.)

If the steps of a ceremony involve one party violating common security advice, or encouraging another party to do so, then that’s a sub-optimal ceremony design. You can think of scamicry as an organization making the usable security problem more wicked.

In their paper on scams, Frank Stajano and Paul Wilson include an insight closely related to scamicry:

System architects must coherently align incentives and liabilities with overall system goals. If users are expected to perform sanity checks rather than blindly follow orders, then social protocols must allow “challenging the authority”; if, on the contrary, users are expected to obey authority unquestioningly, those with authority must relieve them of liability if they obey a fraudster. The fi ght against phishing and all other forms of social engineering can never be won unless this principle is understood.

—Stajano and Wilson, “Understanding Scam Victims” (Communications of the ACM, 2011)

I would go further, and say that understanding is insuffi cient; intentional design and coordination between a variety of participants in the “phished ecosystem” is needed.

www.it-ebooks.info

322 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 322

Properties of Good Security Advice

There’s a tremendous amount of advice about how people should act online. In the discussion of behaviorist models of people, you learned about people rationally ignoring security advice. Part of the problem is that there’s too much of it, creating a wicked environment. The reasonable question is how should you form good advice? Consider whether your advice meets the following fi ve properties (Shostack, 2011b; Microsoft, 2011):

■ Realistic: Guidance should enable typical people to accomplish their goals without inconveniencing them.

■ Durable: Guidance should remain true and relevant, and not be easy for an attacker to use against your people.

■ Memorable: Guidance should stick with people, and should be easy to recall when necessary.

■ Proven effective: Guidance should be tested and shown to actually help prevent social engineering attacks.

■ Concise and consistent: The amount of guidance you provide should be minimal, stated simply, and be consistent within all the contexts in which you provide it.

User Interface Tools and Techniques

Ideally, you should address threats in a way that doesn’t require a person to do something. This has the advantage of being more reliable than even the most reliable person, and not conditioning people to click through warnings. Unfortunately, it’s not always possible. The methods in this section are a strong parallel to those in Chapter 8. You’ll learn about confi guration, and how to design explicit warnings and patterns that attempt to force a person to pay attention. This section does not cover authentication, as threats to such systems are covered under spoofi ng in Chapter 3, “STRIDE,” in the discussion of threats to usability in this chapter, and in Chapter 14.

Confi guration

Confi guration tasks are generally performed by people because a prompt encour- ages them to check or improve their security. That prompt could be a story in

www.it-ebooks.info

Chapter 15 ■ Human Factors and Usability 323

c15.indd 02:16:32:PM 01/16/2014 Page 323

the news or told by a friend, a policy or reminder e-mail, or an attacker trying to reduce their security (probably as a step to a more interesting goal).

The goals of confi guration are as follows:

■ Enable the person to complete a security- or privacy-related goal.

■ Enable the person to complete the goal effi ciently.

■ Help people understand the effects of the task they’re undertaking.

■ Minimize undesirable side-effects.

■ Do not frustrate the person.

Some metrics you can consider or measure as you’re building confi guration interfaces include the following:

■ Discoverability: What fraction of people can go from getting a prompt to fi nding the appropriate interface to accomplish their task? Obviously, it’s important that a test not tell the person how to do this, nor start them on the confi guration screen.

■ Accuracy: What fraction of people can correctly complete the task they are given?

■ Time to completion: How long does it take to either complete the task or give up? Time to abandonment is likely to be higher, as test subjects will try to please the experimenter or appear diligent or intelligent.

■ Side-effect introduction: What fraction of people do something else by accident while accomplishing the main task?

■ Satisfaction: What fraction of people rate the experience as satisfying, given a scale from “satisfying” to “frustrating”?

The preceding two lists are derived from (Reeder, 2008b).

Explicit Warnings

A useful warning consists of a message that there’s danger, an assessment of the impact associated with the danger, and steps to avoid the danger. A good warning may also contain some sort of attention-capturing device, such as a picture of a person falling off a ladder. That’s a very general defi nition of a useful warning. For more specifi c advice, consider the NEAT/SPRUCE combination or the Gold Bar pattern.

www.it-ebooks.info

324 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 324

NEAT and SPRUCE

NEAT and SPRUCE are mnemonics for creating effective information security warnings. Wallet cards created by Rob Reeder read: “Ask yourself: Is your security or privacy UX (user experience)”:

■ Necessary? Can you change the architecture to eliminate or defer this user decision?

■ Explained? Does your UX present all the information the user needs to make this decision? Have you followed SPRUCE (see below)

■ Actionable? Have you determined a set of steps the user will realistically be able to take to make the decision correctly?

■ Tested? Have you checked that your UX is effective for all scenarios, both benign and malicious? (Our cards refer to the UX being NEAT, which has been changed here to effective to encourage testing the warning.)

SPRUCE is a checklist for what makes an effective warning:

■ Source: State who or what is asking the user to make a decision

■ Process: Give the user actionable steps to follow to make a good decision

■ Risk: Explain what bad result could happen if the user makes the wrong decision

■ Unique knowledge user has: Tell the user what information they bring to the decision

■ Choices: List available options and clearly recommend one

■ Evidence: Highlight information the user should factor in or exclude in making the decision

The cards are available as a free PDF from Microsoft (SDL Team, 2012). So why a wallet card? In a word, usability. Rob Reeder reviewed everything he could fi nd on creating effective warnings, and summarized it in a 24-page document. That document contained 68 elements of advice. In working with real programmers, however, he discovered that was way too long, and created our NEAT guidance: Warnings should be necessary, actionable, explained, and tested. The SPRUCE extension was joint work with myself and Ellen Cram Kowalczyk (Reeder, 2011a).

The “Gold Bar” Pattern

A gold bar pattern combines warning, confi guration, and sometimes other safety features into a non-intrusive dialog bar appearing across the top or bottom of a program’s main window. You’ve probably seen the pattern in Microsoft Offi ce and in browsers, including IE and Firefox.

www.it-ebooks.info

Chapter 15 ■ Human Factors and Usability 325

c15.indd 02:16:32:PM 01/16/2014 Page 325

The interface appears non-modally while the program does as much as it safely can. For example, a Word document from the Internet will render, allow- ing you to read it. A bar appears across the top, informing you that this is a safer, limited version of Word. (Underneath, Word has opened the document in a sandboxed and limited version of the program that has less attack surface and functionality. If you watch carefully when clicking one of these, you’ll notice that Word disappears and reappears. That’s the viewer closing and the full version launching Malhotra, 2009.)

This pattern has a number of important security advantages. First and fore- most, it may eliminate the need for the person to make a decision at all. Second, it delays security decision-making to a point when more information is available. Third, it replaces dialogs that said things like “While fi les from the Internet can be useful, this fi le type can potentially harm your computer,” or “Would you like to get your job done?”

The gold bar pattern has limits. The bars are intended to be subtle, which can result in them being overlooked. Their size limits the amount of text within the bar. They can also fail to align with the mental model of the person expe- riencing them. For example, someone might ask, “Why is e-mail from my boss treated as untrusted, and why do I need to leave the protected mode to print?” (Because your boss’s computer might be infected, and because printing exposes substantial attack surface. However, those are neither obvious nor explained in the interface.) Lastly, because the bars are subtle, they fail to impose a strong interrupt and may therefore fail to invoke system 2, described earlier. System 1 is the faster, intuitive system, whereas system 2 processes complex problems. They were introduced earlier in the section “Cognitive Science Models of People.”

Patterns That Grab Attention

There is a whole set of patterns that try to force people to slow down before doing something they’ll regret. This section reviews some of the more interest- ing ones. It might help to conceptualize these as ways to invoke system 2. The goal of these mechanisms is to get people encountering them to avoid jumping to the answer using system 1 and instead bring in their system 2.

Hiding the Dangerous Choice

Internet Explorer’s SmartScreen Filter doesn’t put the dangerous run button directly in front of people. It has used a variety of strategies for this. For example, one post-download dialog, shown in Figure 15-8, has buttons labeled Delete, Actions, and View Downloads. When shown this warning, IE users choose to delete or not run malware 95 percent of the time. This interface design has been combined with other usable security wins, including showing the dialog (on average) only twice per year, which may make the hiding pattern less intrusive

www.it-ebooks.info

326 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 326

(Haber, 2011). It is hard to disentangle the precise elements that make this so effective, but much of it has to do with the combination of elements, refi ned through extensive testing.

Figure 15-8: An IE SmartScreen Filter warning dialog

Modal Dialogs

Modal dialogs are those that capture focus and cannot be closed until a person resolves the choice in the dialog. Modal dialogs fell out of favor for a variety of accidental reasons. For example, a fl oating modal dialog box was not always the front-most window in an interface, or it could pop up on another monitor or virtual workspace. Modal dialogs also interrupt task fl ow. Lightboxing is the practice of displaying a darkened interface with only the modal dialog “lit up.” This addresses many of the accidental issues with modal dialogs, and it is com- monly used by websites.

Spiky Buttons

Keith Lang has proposed the use of less friendly user interface elements, such as spiky buttons (Lang, 2009). A sample is shown in Figure 15-9. In a blog discussion about it, the question of how quickly people would habituate to it was raised. Others suggested more standard stop imagery would work better (37Signals, 2010). Spiky buttons may be more of a “don’t jump there” message than a “pay attention” message, but without experimentation and a set of people with ongoing exposure to them, it’s hard to say. To the best of my knowledge, it has not been used.

Figure 15-9: A spiky button

Delay to Click

When you attempt to install a Firefox add-in, Firefox imposes a delay of four seconds. As shown in Figure 15-10, the Install button is grayed out, and displays

www.it-ebooks.info

Chapter 15 ■ Human Factors and Usability 327

c15.indd 02:16:32:PM 01/16/2014 Page 327

a countdown from (4). Somewhat surprisingly, this is actually not designed to get you to stop and read the dialog, although it may have that effect. It is intended to block an attack whereby a site shows a CAPTCHA-like interface with the word “only” displayed. The site attempts to install the software when you type the “n,” triggering a security dialog; when you type the “y,” it triggers the “yes” in the dialog, which now has focus (Ruderman, 2004). A related technique, a 500ms delay, is used for other dialogs, such as the geolocation prompt (Zalewski, 2010).

Figure 15-10: Firefox doesn’t care if you read this.

Testing for Human Factors

Like every way you address threats, it is important that you test those mitiga- tions for threats that involve people. If an attacker is trying to convince someone to take action, does your interface help the potential victim make a security choice, while letting them get their normal work done? Unfortunately, people are surprising, so testing with real people is a useful and important step.

There are a few issues that make the testing of any security-relevant user interface challenging. The best summary of those issues is in a document by usable security researcher Stuart Schechter (Schechter, 2013). That document is focused on writing about usable security experiments (rather than designing or performing them). However, it still contains very solid advice that will be applicable. You can also fi nd excellent guidance on usability testing in books such as Rubin’s “Handbook of Usability Testing” (Wiley, 2008), which unfortunately doesn’t touch on the unique issues in testing security scenarios.

www.it-ebooks.info

328 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 328

Benign and Malicious Scenarios

Normal usability testing asks a question such as “Can someone accomplish task X with reasonable effort?” Good ceremony testing requires testing “Can someone accomplish task X and raise the right exception behavior only when under attack?” That requires far more subtle test design. (That is, unless you expect that your attackers will start with “This fi le really is malicious! Please click on it anyway!”) It also requires at least twice as many tests. Lastly, unless you have reason to believe that most people will only encounter the user experi- ence being tested extremely rarely, a good test needs to include some element of getting people familiar with the new user experience before springing an attack on them.

Ecological Validity

Participants in modern user studies tend to be aware that they’re being studied. The release forms and glass walls tend to give it away. Also, people participating in a user study tend to expect that the tasks asked of them are safe. They have good reason to expect that. The trouble is that if you think what’s being asked of you is safe, or at least a designed experience, then you may not worry about things (such as tasks, indicators, or warnings) that would normally worry you. Alternately, people being studied often spend time trying to fi gure out what’s being studied, or what behaviors will please the experimenters. Researchers refer to this as the ecological validity problem, and it’s a thorny one. The best sources of current thinking are SOUPS (the Symposium On Usable Privacy and Security) and the security track of the ACM SIGCHI conference. (SIGCHI is the Special Interest Group on Computer-Human Interaction.)

General Advice

The following list is derived from a conversation with Lorrie Cranor, who accurately points out that a simple checklist will be suffi cient to explain how to perform effective usability testing. Any errors in this are mine, not Dr. Cranor’s:

■ It is hard to predict how people are going to behave without doing a user study, so user studies are important.

■ Find someone, either a colleague or a consultant, who knows how to do user studies and work with that person.

■ The user study expert you work with is unlikely to know much about threat modeling or to have done a usable security study before. You will need to help them understand what they need to know about security and threat modeling.

www.it-ebooks.info

Chapter 15 ■ Human Factors and Usability 329

c15.indd 02:16:32:PM 01/16/2014 Page 329

■ As you work with the usability expert to design a user study, keep in mind that you need to fi nd a way to study how people behave in situa- tions where they are under an active attack. The tricky part is fi guring out how to put your study participants in a situation that simulates these threats without actually putting anyone at risk. (See the preceding section on “Ecological Validity.”)

■ Security user studies frequently use deception in order to simulate a security threat. For example, a study on web browser phishing warn- ings was advertised as being about online shopping. Participants were instructed to make an online purchase and check their e-mail to get the receipt so they could be reimbursed. When they checked their e-mail, they had a message that appeared to be from the vendor where they made a purchase, encouraging them to log in to a website that was actually a phishing website. When they tried to log in to that website, it triggered a warning in their web browser and experimenters could observe how participants responded to such warnings. Use of deception in a study design will raise ethical issues.

■ Some security user studies use role-playing or hypothetical scenarios. For example, a study on text passwords asked participants to imagine that their e-mail account password had been compromised and they were required to create a new password. (Ur, 2012) A study on the usability of PGP asked participants to play the role of a campaign volunteer in a political election and send and receive e-mails, being careful to use PGP to ensure that campaign strategy information didn’t fall into the wrong hands (Whitten, 1999).

Perspective on Usability and Ceremonies

As of 2011, roughly half of the malware running on Windows computers was being installed using techniques that tricked a person into doing something they didn’t intend (Microsoft, 2011). Phishing attacks continue to plague the Internet, although it’s hard to measure how much impact they have. The time when we can design systems without considering the people using them is long past. We need to learn how to do better at modeling both people and systems, and integrating the models, but we have ways that can be made to work. They are less prescriptive than I would like to see, and that should inform your deci- sions about who engages with them, and how you plan for them.

You’ll need people with more breadth than is required for many other security tasks. People with backgrounds in sociology, anthropology, or psychology will

www.it-ebooks.info

330 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 330

have perspectives that are different from those with a computer science or engi- neering background. Usability specialists will also be useful in threat modeling sessions that focus on people, and in discussions of design changes to address usability. In addition, keep in mind that performing ceremony analysis or using other techniques in this chapter will take longer than other threat modeling techniques, and you’ll also need more time for ancillary tasks like training and making sure everyone agrees on the goals and scope of an analysis. There is a risk of getting bogged down in elements which feed into your ceremony, rather than analyzing what’s unique to you.

Ceremonies provide a fascinating framework for bringing humans into threat modeling. They show us ways to integrate people into existing analysis tech- niques, and support the assertion that ignoring people is no longer acceptable. At the same time, they do have limitations, many acknowledged in Ellison’s paper. I’d like to address two:

■ Effective ceremony analysis requires more maturity and experience than other forms of threat modeling, as it involves an understanding of both technology and people. I am hopeful that this will not dissuade anyone from trying it, and that we will learn a great deal about how to do it well over the next few years.

■ The other aspect is the idea that “nothing is out of band to a ceremony.” This is challenging for a prescriptive and bounded threat modeling exer- cise. It summons and celebrates the specter of projects designed to run forever, so when can you effectively say that a ceremony analysis has run its course? One part of the answer is that you know where all data needed for, or used in, the ceremony crosses a trust boundary. You may not need to know where it ultimately came from, but only how much trust you’re placing in the provider. Similarly, in looking for threats, it may be suffi cient for your business purposes to run through the section “Ceremony Analysis Heuristics,” or you might prefer to run usability tests. Similarly, you may get to a point where the next useful mitigation requires 25 percent of the project’s budget, and decide that a particular risk is thus acceptable and document it.

A last quote from Reason sums up issues relating to human factors and usability better than I can.

The managers of complex and hazardous technologies face a very tough question: how do they control human behavior so as to minimize the likelihood of unsafe viola- tions without stifl ing the intelligent wariness necessary to recognize inappropriate

www.it-ebooks.info

Chapter 15 ■ Human Factors and Usability 331

c15.indd 02:16:32:PM 01/16/2014 Page 331

procedures and avoid mis-appliances? The answer must surely lie in how they choose to deploy the variety of systemic controls that are available for shaping the behavior of its human elements.

James Reason, The Human Contribution (Ashgate Publishing, 2008)

It’s a fi ne issue to consider in designing and analyzing technologies of any complexity, and all technology today operates in the hazardous world of the Internet.

Summary

If you don’t try to help the people who will use your system to use it securely, they will probably fail. Modeling for systems that involve people is harder than other threat modeling. You must add models of people to both your model of the software and your model of the threats. Working with three models is harder than working with two, and models of people are less crisp than models of technology.

Models of people from behaviorists and cognitive scientists are useful for understanding how people will behave in a variety of circumstances. The behaviorist models include conditioned responses and kind versus wicked learn- ing environments. Kahneman teaches us that what you see is all there is; that system 1 and 2 interact in surprising but predictable ways; and that anchoring and satisfi cing are common and important biases we all have. Others, like Reason, Cranor, and Sasse, also offer important lessons. Reason has many models of errors; Cranor provides a systematic approach to thinking about security with a human in the loop; and Sasse reminds us that security is but one of many things people want.

There are also heuristics you can use when working to help people, ranging from goal orientation to confi rmation bias or a compliance budget. You also saw that there’s a tendency for system designers to make optimistic assump- tions about people. Models of the software can be extended to take into account human factors issues. You can model scenarios as warnings, authentications, or confi guration; and you can extend a variety of diagrams to include or represent people. Lastly, you can model the techniques of electronic social engineering.

All the techniques for fi nding threats work when you include models of people. Brainstorming about them is, like other brainstorming, a tool that works better when you have security and usability experts in the room. You can also

www.it-ebooks.info

332 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c15.indd 02:16:32:PM 01/16/2014 Page 332

use ceremony analysis and a variety of ceremony-specifi c heuristics for fi nding threats, including missing information, distracting information, underspecifi ed elements, fuzzy comparison, and a set of principles taken from cognitive scientists.

There are a plethora of building blocks for addressing threats. Many of these are design patterns, such as minimizing what you ask of people, conditioned safe ceremonies, avoiding urgency, and ensuring an easy path to success. There are also patterns for kind learning environments, such as avoiding scamicry and creating good security advice. You can also use more specifi c techniques, such as Gold Bars, NEAT and/or SPRUCE, and a set of patterns for grabbing people’s attention.

Lastly, like all solutions to threats, you need to test for human factors. Such testing is challenging and worthwhile. You need to test in both benign and malicious scenarios, ensure that your tests don’t cause people to behave differ- ently (that is, ensure they have ecological validity), and work with people who have experience with testing for human factors.

www.it-ebooks.info

333

c16.indd 07:53:2:AM 01/15/2014 Page 333

Cryptography is the art of communicating securely in the presence of adversaries. Cryptography, or crypto, as nearly everyone calls it (to save two syllables), fi gures into a good number of defenses. You can use it as part of how you address spoofi ng, tampering, repudiation, and information disclosure. It’s important for people working in and around security to understand cryptographic tools; and perhaps more important, to understand the common mistakes made while using them, because failing to understand these mistakes can lead to overconfi dence, and mistakes made by overconfi dent people are a major source of real-world problems.

Cryptographers have been enumerating threats for a very long time. They have done more than most other branches of security to quantify the security of their systems, and have over a long time evolved their thinking about threat models (in the sense of what attacks they worry about). And frankly, crypto can be a lot of fun. If you want to dabble in cryptography, the right place to get started is by breaking cryptosystems. (In a lab, not breaking into other people’s systems in possible violation of ethics or law. Thanks! I knew you’d understand.) There are many targets, and the worst you’ll do is not break anything. In contrast, if you try to get started by building cryptosystems without developing knowledge of how to break them, you’ll inevitably have false confi dence in what you’ve built. It’s easy to build a cryptosystem whose fl aws you can’t see.

This chapter is mostly a reference chapter. It does not replace a good text on cryptography, but encourages you to use the largest building blocks available,

C H A P T E R

16

Threats to Cryptosystems

www.it-ebooks.info

334 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c16.indd 07:53:2:AM 01/15/2014 Page 334

rather than rolling your own. If you need to roll your own, you need to fi rst read all of Cryptography Engineering by Niels Ferguson, Bruce Schneier, and Yoshi Kohno (Wiley, 2012). I don’t want to scare anyone away from crypto, nor do I want you to be overconfi dent.

In this chapter, you’ll fi rst learn about cryptographic primitives. Primitives are the smallest units of cryptography that anyone will ever use. (They’re often called building blocks, but this book uses “building block” to refer to the build- ing blocks in the 4-stage model.) After the cryptographic primitives, you’ll learn about the classic crypto attackers and attacks against cryptosystems. Those are followed by advice on building with crypto, and a set of things worth remember- ing. The chapter closes with a discussion of an over-reliance on secrecy to protect a system in the context of Kerckhoffs, the person who crystalized that discussion.

Cryptographic Primitives

There are only a few basic cryptographic primitives: symmetric and asymmetric encryption, hash functions, and pseudo-random number generators (PRNGs). Understanding them will help you avoid mistakes in their use. This section also covers a set of techniques that are useful for preserving privacy—that is, preventing certain types of information disclosure attacks. Lastly, you’ll look at a few important modern constructions that you should be familiar with.

Basic Primitives

The primitives just mentioned are at the very heart of cryptography. If you don’t understand them, they’re easy to confuse, and confusing them will leave you or your customers insecure. This section uses some standard conventions and terminology: Alice and Bob are typically the people who want to commu- nicate. They do this by sending messages, which are also called plaintext. The cryptosystems they’ve agreed to use often have ciphers and sometimes keys. A cipher has various functions, such as encrypt and decrypt. The following sections begin with the category of system you’re most likely familiar with, symmetric cryptosystems. The others are shown in table 16–1.

Table 16–1: Cryptographic Primitives

PRIMITIVE GOAL INPUT OUTPUT

Symmetric cryptography

Confi dentiality Plaintext (any length)

Ciphertext (same length)

Asymmetric Authentication Plaintext (any length)

Fixed length signature

www.it-ebooks.info

Chapter 16 ■ Threats to Cryptosystems 335

c16.indd 07:53:2:AM 01/15/2014 Page 335

PRIMITIVE GOAL INPUT OUTPUT

Asymmetric Key agreement Counterparty information

A session key

Hash Fingerprint Plaintext (any length)

Fixed length message depen- dent fi ngerprint

PRNG Randomness Various Hard to predict bits

Symmetric Cryptography

Symmetric encryption systems—also known as private key or secret key systems, or even just ciphers—use a key known to both (or all) sides of a conversation, thus the name symmetric. The Caesar cipher is an example of a (very simple) symmetric system. Letters are shifted three letters to the right, with A being represented by D, B by E, and so on. Someone must either know the key, guess the key or break the system to read the message. Ideally, you use a system that is harder to break.

Assume that Alice and Bob have shared a key. The original, unencrypted message Alice wants to send to Bob is the plaintext. She and Bob have previ- ously agreed on a symmetric cipher to use, along with a key. The plaintext and key are inputs to the cipher’s encryption function, which outputs a ciphertext. She sends this ciphertext to Bob, not worrying about who might see it, because if the cryptosystem is strong, that ciphertext is unintelligible to everyone without the key. Those with the key feed the key and the ciphertext to the decryption function, which outputs the plaintext. The symmetry of the system results from the same key being used on both ends. Symmetric systems protect against infor- mation disclosure attacks when the key is secure. Unfortunately, establishing the key in a secure way is tremendously tricky. Years ago, governments would send keys in briefcases handcuffed to couriers, which is expensive and slow.

There are two types of symmetric algorithms, block ciphers and stream ciphers. A block cipher takes a block of input (often 128 bits) and encrypts it with a key, producing an output block of the same size as the input. Many constructions that use a block cipher also use an initialization vector (IV). This is a random number that is input to the cipher along with the key and the fi rst block to ensure that identical plaintexts are not encrypted the same way.

A stream cipher takes a key and produces a key stream. That key stream can be XORed with a plaintext bit stream to produce ciphertext. The same key stream is produced at the receiver, which can then use XOR to get the plaintext. If you ever reuse a key stream, your ciphertexts can be XORed, and the output of that is the XOR of the plaintexts. That’s highly undesirable. The most com- mon stream cipher, RC4, has a host of issues. Be sure to review the advice in

www.it-ebooks.info

336 Part IV ■ Threat Modeling in Technologies and Tricky Areas

c16.indd 07:53:2:AM 01/15/2014 Page 336

Cryptography Engineering before you use RC4. (Let me save you time—RC4 is not mentioned in Cryptography Engineering, because it’s so hard to use securely, but as you review the advice there, you’ll fi nd something that will work safely. Just don’t use RC4.)

The simple rule for symmetric encryption is to use AES in CBC or CTR mode. AES is the U.S. “Advanced Encryption Standard,” and the modes refer to the precise ways bits are managed. (CBC is “cipher block chaining,” while CTR is “counter” mode.) The pros and cons of each are subtle, and debate rages on. However, using one or the other will almost always be the right choice. To be explicit, this means you should avoid ECB mode. ECB stands for “electronic code book,” although you can think of it as “extreme cryptographic blunder” because it’s an extreme cryptographic blunder to use the mode.

Asymmetric Cryptography

In an asymmetric encryption system, each party has a set of mathematically related keys, and through the use of really cool math, manages a variety of useful results, including encryption, key agreement, and digital signatures. Asymmetric encryption systems are also called public key systems, because one of the keys is made public.

N O T E Asymmetric cryptography is also known as public key encryption.

This section presents a simplifi ed version of the math that doesn’t use much more than high school algebra and then discusses how asymmetric systems can be used.

Alice and Bob each pick a number. Alice picks a, Bob picks b. They then agree on a number g. Maybe their number is the closing of the Dow Jones index on the previous day. It can be public; it just needs to be hard for an adversary to infl uence.

Alice calculates A = ga and Bob calculates B = gb. Alice then sends A, and Bob sends B. Then Alice calculates a secret key, s = Ba, and Bob calculates s = Ab. Now, because multiplication is commutative, Ab = Ba (or, really, gab = gba ). If you assume that it’s hard to calculate a from g and ga, then it’s hard for anyone who doesn’t know a or b to fi gure out what s is. Alice and Bob have now agreed on a secret without ever sending that secret back and forth. It’s not that hard to fi gure the nth root of ga, and from there you can approximate, but this is a simplifi ed description. Therefore, to get more security, we’ll add one more element, which is that Alice and Bob do all of this math modulo p. Math modulo p is like clock arithmetic. 12 hours after 7, it’s 7. So 7 modulo 12 is 7, and 19 modulo 12 is also 7. We can replace 12 with any other number, which is called p. When a, b, g, and p are large, and satisfy certain mathematical properties, then fi guring out a or b from ga and gb is hard. Because it’s hard, Alice and Bob could use s as a key

www.it-ebooks.info

Chapter 16 ■ Threats to Cryptosystems 337

c16.indd 07:53:2:AM 01/15/2014 Page 337

in a symmetric encryption system. They want to do this because symmetric encryption is fast, and asymmetric encryption is slow.

This is way cool, and it’s worth taking a few moments to work through some of the math yourself to see it work. However, if you use the preceding system as presented, you will be successfully attacked. The example is designed to help you get a feel for what’s possible and why. It is not intended to replace a full college-level course in mathematical cryptography, which is needed before you can make interesting mistakes in this space.

Once you’ve worked through the math, you can use asymmetric systems for three things beyond online key agreement: encryption, offl ine key exchange, and signatures. The cryptosystem’s encrypt function allows anyone to encrypt a message using a public key so that only the person with the private key can read it. If the message is the key to another, then that key can be used to encrypt or decrypt messages. (The usual reason for this is that symmetric systems are much, much faster than asymmetric systems.) Most important, asymmetric signature systems can be used to sign a message, and anyone with the public key can verify the signature. A quirk of the math in RSA, one of the fi rst public key cryptosystems, leads many people to overgeneralize with the claim that signatures are the inverse of encryption.

The pitfalls related to effective key exchange, signatures, and encryption are substantial and often subtle. Using systems such as Diffi e-Hellman, RSA, or DSA as originally presented is foolish.

Hashes

The third important building block are hashes, also called one-way functions, or message digests. A hash takes an input of some arbitrary length and produces a fi xed-length digest or hash of the input. Ideally, any change to the input com- pletely transforms the output. This means that other building blocks can assume they’ll always see a fi xed-length input. For example, a signature algorithm can assume that it will always see input of the same length, and defer the problem of arbitrary length input to the hash function.

Hashes are also used to store passwords, usually with a salt. A salt is an additional input to a hash function, designed to ensure that identical inputs create different outputs. Therefore, if Alice and Bob are both using the password Secret1, then the simple approach to storing a hash of each of their passwords would reveal that Alice and Bob have the same password (but not what it is). With salting, the threat from sorting is mitigated. There’s other threats, and more information on the storage of passwords in Chapter 14, “Accounts and Identity.”

Hashes can also be used to create fi ngerprints. A fi ngerprint in this context is a hash of something longer, designed to be used by people to authenticate that each person has the same copy of the longer thing (such as a fi le or message).

www.it-ebooks.info